There’s no simple answer to that (it depends on too many factors) but there’s no way that you could reach it quickly in normal use.
A good internal SSD should be good for thousands of writes of the entire drive. A good SSD will allow you to monitor the aging of the drive (so that you can replace it before it becomes unreliable). Any decent SSD will quote you a reliability lifetime figure in TBW (TeraBytes Written).
Unfortunately USB flash drives would not typically do any of those things. Hence any file that exists on a USB flash drive should exist somewhere else as well.
I’ve run an entire computer exclusively off a USB flash drive for a period of some years (yeah, don’t ask) and it lived. So failure is not certain but they are not designed for that kind of load.
Various techniques are used to get the most out of flash storage and it can be that you get what you pay for (internal SSDs at the higher end all the way down to USB flash drives and SD cards at the lower end).
Or a faulty batch?
Or a counterfeit batch?
When copying large files to external media (let’s say 500MB+), I will checksum the original and checksum the copy. If the checksums match then that is a cryptographically strong indication that the copy is good (for now ). (That is not contradicting what you wrote since in order to checksum the copy it is effectively reading the entire file back.)
Except where the drive is too large for FAT32, in which case it will generally come preformatted with an exFAT file system.
Teracopy will verify files and is a free tool for non commercial use Teracopy Pro has an extra few features that almost every home user will not need and is commercial use allowed. It has usually great transfer/copy speeds as well. Windows xcopy has a /v option but as I understand it, it is only a file size check.
Looks like you might be right. I don’t use Windows at all but I found one article that suggested using the command “VERIFY ON” as an alternative to the non-functional xcopy /v.
That article pointed out though that if file contents are cached in RAM then you will need to defeat that if you really want to verify the copy (and that also applies in my case of doing a checksum on the copy, but I was aware of this issue).
Sorry, but I do need to ask. Are you saying that the only disk storage on the computer was a USB flash drive, or was it the device you booted from? I do the same with Linux on a Win computer.
Computer had no internal drive. The only storage was a USB flash drive sticking out the back. The flash drive contained the operating system (Linux), all user files, everything.
This could be roughly comparable with putting a full Linux install on a flash drive in order to boot on a computer that normally runs Windows but the difference is that in my case the computer was running Linux 24x7 whereas in your scenario it may only be the occasional boot of Linux.
This is not comparable with putting a Live Boot of Linux on a flash drive in order to boot on a computer that normally runs Windows because a Live Boot is basically read-only boot media i.e. there aren’t typically any write operations to the flash drive. A flash drive used like this is basically a CD-ROM (for those old enough to remember booting from CD-ROM1).
My point was that even though USB flash drives are not rated for that kind of use (full Linux install, sole disk, 24x7), they are not completely horrible.
Which is why I asked the question regarding the OP whether this was just a faulty batch.
1 Yes, this is a deliberate red rag. I know that CD-ROM is not that old and that someone will come out with how they used to boot their PC from paper tape.
To copy very large files the recommendation is to use /j as this disables buffering and this may be a suitable choice to avoid caching in RAM regardless of file sizes.
Whonix and TAILS are certainly able to be used this way. TAILS can allow persistence from a Live Boot USB drive, it just needs that to be selected and installed as part of the setup. Whonix has a Live mode that has no persistence, if installed within a Host OS it can have persistence.
Yes, TAILS encourages you to use it purely as a Live Boot. Read-only. No writes. So no aging of the medium caused by erase-write cycles.
TAILS allows you to have some persistent storage on the medium (separate from the boot) - in which case the storage should be encrypted.
The point is that the kind of user who needs to use TAILS (dissident resisting an unpleasant government) should be really really careful about leaving traces of any kind. It is by definition evident to the government that the dissident has chosen to use persistent storage, but the contents therein are not evident, and rubberhose decryption may ensue.
So, yes, TAILS can be used in either configuration.
I can only assume that your Linux system, and the apps running, had very little need to go to disk once loaded and running. No need for OS virtual storage paging. Files loaded into RAM buffers and read from there. Files created sit in the same buffers and not needed to be constantly written back to physical disk for persistance.
Fairly normal file system usage. At the very least, Linux computers tend to be busy with software updates, sometimes coming in more than one a week.
Probably not a lot of file caching going on. The computer only had 2 GB RAM total. That was enough to run Linux comfortably, but not leaving masses of RAM free for caching 1GB files.
But at least two different brands exhibiting the same issue, Emtec and iDigital/Lexar. Either extremely unlucky or is a hardware/USB port issue with the PC/Mac. Being at least two different brands with potentially multiple flash drives (iDigital three pack was purchased), possibly points to the later.
Buying another flash drive, trying to get current ones working etc will only result in further issues if it is a USB port issue. I would be investigating whether the faults lie with the USB ports being used on the Mac or PC. Overvoltage problems exist (has a mate who had one on an older PC) which can cause issues with devices connected to them. Flash drives can potentially be ‘fried’ when overvoltage occurs.
For many devices they also rely on the 12 V power supply and this can suffer larger voltage increases that may not affect the usual expected running of the computer. Often the 12 V supply from the power supply is further corrected before the CPU receives it. 5 V components such as USB ports do not receive any further correction and if they had over-voltage there should be more faults than just the USB ports, e.g, keyboards and mice. A power surge would be a different issue, this should lead to a power supply failure though.
I am more inclined to either believe the USB devices are faulty (unlikely as various USBs have been used in the Mac PC) or that one of the PCs (Mac or Win) have an under voltage issue on at least one 5V output. If as they said after being on a Win PC as @Jon01 stated “Using the same USB with a mac and then using it in a pc (to download articles in a library) and then returning to use it with the mac resulted in the folder into which I thought I saved the article not only did not contain the article but when I tried to access that folder on the mac, the entire contents of that folder vanished”. the stick could at first be read on the Mac as having files even though they couldn’t be accessed, points more to a FAT corruption and under-powered USB ports definitely can lead to erroneous FAT and file writes and finally a failure to read the drive.
FAT32 uses two FAT , one being a backup to the other. This usually allows one FAT to fail and the second then allows access to the files, an unsuccessful FAT save could lead to further corruption. The Lexar USB was subjected to a forced ejection and so a further corruption of the file allocation tables most likely occurred… As the issue seems to have appeared to start after being inserted into the Win PC, this could have the under-power and the 12 V and 5 V supply from the power supply needs to be checked. If voltage from the 12 V supply is 11.5 V or lower, replace the power supply. If the 5 V supply is below 4.5 V replace the power supply. These low voltages are the signs the PSU is failing. @Jon01 As it seems the Win PC is a library one, the library might need to be informed that the PC may have a faulty PSU and as @person notes in their post below
A strong reason the USB had to be force ejected from the Mac was that the file manager Finder was trying (unsuccessfully) to read the file contents. It found a “valid” file pointer in the FAT (likely corrupt) and so continued to seek the information from the next file cluster. FAT only contains the first pointer and at the end of each subsequent file cluster is the pointer to the next file cluster until the EOF (end of file) marker is found. Finder would continue to try to seek the next cluster until a time out occurred, this does not always occur. To get past this locked up state, Finder would need to be shut down and then the USB could be ejected as the program trying to use it had ceased the operation. This ejection would not fix the FAT corruption or any file errors on the drive. Even if the FAT was not corrupted and had a valid file pointer, an under-powered USB hub would have also not written the file contents successfully to the drive, so it is a compounding series of faults on the drive. Any attempt to fix the issues by “repairing” the drive would also likely fail as each link in the file chain must be correct, if not then a file cannot be repaired and this results in lost clusters. FAT corruption end result often is that the drive becomes unreadable and cannot be “mounted” as the file system manager cannot read the drive contents.
I stress again that if a user wants to limit the issue of any possible under-powered port then using a powered hub helps mitigate the read and write need for adequate power. At libraries they often have power points for a user to plug in their own storage device power plugs and plugging the power supply of the hub into this will ensure enough power for the USB stick.
Your post and all the others gave me a lot to ponder. I may take Friday off work to attend to your suggestions.
You point on possibly a port is faulty, I do not think is true in my case because the usb (Lexar) does not work in several computers: macbook, Dell PC and HP PC.
I use a basic hub, 4 x USB-A that connects via a usb plug to the macbook. Other usbs work just fine in the hub (verbatim and sandisk). With the pc, I use no hub, but plug the usb into the computer’s port and again it fails to detect or to read.
To be certain of that however you would have to test each time from scratch - because if one USB port has corrupted the file system on the flash drive then that corruption would then cause problems on other USB ports, regardless of whether the later USB ports are working.
I use f3 (fight flash fraud) to test for counterfeit drives, but it would also be a good thrash test of any drive i.e. push it hard to see whether it is faulty. (However it looks as if that would complicated to use on Windows or Mac. It might actually be easier to do a Linux Live Boot in order to install and run f3, provided that you don’t have the same problem when you try to boot Linux from a flash drive. )
Note also that these are destructive tests. So you either test a spare flash drive or you image it (where possible!) before testing.
I’ve enjoyed reading all these comments and two things come to mind:
You get what you pay for and not sure I would be buying technology from TRS that I wanted to use for an extended period. I buy my USBs from Officeworks/JB Hi Fi when I see a well known brand (eg. SanDisk) at a good price to always make sure I have a supply available.
Assume that any storage device will fail in the next millisecond, so if you have data that you value, make sure you have multiple backup copies on multiple devices at multiple locations. And make sure you read the data regularly by making another backup copy and checking it.
Thanks for the feedback. Along the way I mentioned, but maybe you did not see, that not only were TRS USB purchased proven to be duds, but so were those from Harvey Norman (brand name LEXAR).
Yesterday another of the several LEXAR USBs failed. Luckily I backed it up 2 days earlier, so I lost little data.
It looks like there is an issue with your USB port or drivers. To have so many failures is very unusual. We have a range of USB drives, different brands and capacities, and are yet to have one fail (oldest one about 15 years and still working). We use them for document transfers, loading docs/photos when using a printing service and third backup for our accounting files which is a write every two or so days.
Sorry for the late reply. The failures are all LEXAR branded USBs. i use the same mac with the same hub for imation USBs, SanDisk and Emtec with no issues.
I wonder if the drivers in the Lexar USBs may be the problem.
). (That is not contradicting what you wrote since in order to checksum the copy it is effectively reading the entire file back.)
)