2013-08-26 Testing real capacity of an USB stick

I've read about fake oversized thumb drives / USB sticks, which report a much bigger size than they actually are.

To test the real capacity of an USB stick under OS X, we'll create a test file and write it to the USB stick a number of times. Then we'll verify what has been written with a couple of terminal commands.

This is a pretty brute-force test and I'm sure it can be done much smarter. I'll give an idea on speeds, and I assume USB 2.0, and also assume a maximum write speed of 15 MB/s, and maximum read speed of 30 MB/s. Furthermore, we're working with 1 GB equalling 1 thousand million bytes, because that's what they use on the packaging.

First, we format the disk. Start Disk Utility and erase it. This is to remove any files in the Trash.

Notice the capacity in the lower right. In this case, it's 16 GB. We'll need that later.

Now open a terminal window. See under which directory the stick is mounted:

 $ ls -l /Volumes
 total 24
 lrwxr-xr-x  1 root    admin     1 Aug 20 07:12 Macintosh HD -> /
 drwxrwxrwx@ 1 root  staff  8192 Aug 26 12:07 STORAGE

In the case above, the USB stick is called 'storage'. Go there.

 $ cd /Volumes/STORAGE

First create a 100 megabyte test file on the USB stick like so (optimally, this takes 1 minute and 10 seconds):

 $ dd if=/dev/urandom of=test.bin bs=1000000 count=100
 100+0 records in
 100+0 records out
 100000000 bytes transferred in 9.029708 secs (11074555 bytes/sec)

Now copy the test file a number of times, as much as the number of gigs you expect on the USB stick, in this example, 16 gigabytes. To write a gigabyte, we need to write our 100 megabyte testfile ten times. 16 gigabyte times 10 is 160. A gigabyte takes about 1 minute and 10 seconds to write optimally. The OS might be able to completely cache the source test.bin file in memory, which saves on reading time.

 $ for i in $(seq 1 160); do echo "Writing file $i"; cp test.bin test$i.bin; done

Probably, the last file will fail because although the stick might be 16 gigabytes, the filesystem also uses up some space.

Display the checksum of the first file. This can take half a minute.

 $ md5 test.bin
 MD5 (test.bin) = 0d3c4fe338109b09b61fcea3e11b0e4b

Now test the checksums of all files, excluding the correct results. My 16 GB stick took 8 minutes.

 $ md5 *.bin | grep -v 0d3c4fe338109b09b61fcea3e11b0e4b

If you're testing a big SSD drive or something similar, better use xargs (because simply using an asterisk might be too much for the shell):

 $ ls *.bin | xargs md5 | grep -v 0d3c4fe338109b09b61fcea3e11b0e4b

If something went wrong, then you should either see the OS give errors, or some files should give a different MD5 hash. As mentioned above, you can probably ignore the last file, which may not be fully written to disk.