Why is writing SLOW on raw device, and FAST on filesystem (USB key)

ddfilesystemsperformanceusb-drive

I have a USB key (PQI U822V-Speedy 32G) that I am trying to benchmark quick'n'dirty on Linux. I'm testing write bandwith.

dd on raw partition

I created a partition starting at sector 2048, then did a 4 GB sequential write: 

dd if=/dev/zero of=/dev/sdb1 bs=1M count=4096

I get ~22 MB/s.

I also tried several (4) dd running in parallel like the one above but using count=1024 and the seek= option to write to different areas of the drive. Same results.

dd on filesystem

However, when I format the sdb1 partition with either ext4 or NTFS, and copy big files to it (either real or /dev/zero), like this: 

time dd if=/dev/zero of=/media/USBKEY/file.bin bs=1M count=4096 ; time sync

I do achieve > 66 MB/s as advertised by the manufacturer. Of course, I considered the sync duration just after the copy.

Why is there such a big performance difference?

Best Answer

Now that I look again, I realized you said this was a usb key ( flash drive ) not a hard drive. Flash memory can only be erased in large blocks, and individual sectors can not be written without erasing them ( and the whole block they are in ) first. Since software expects to be able to write wherever it wants on the disk at any time, the disk has translation logic in it to transparently handle the erasing. How this is done has a dramatic affect on write performance. Many devices use an algorithm for most of the disk that handles sequential writes very well, but sucks at random writes. The area near the start of the disk is normally used by the FAT in the FAT filesystem they come preformatted with, and this area is randomly written to frequently, so they use a different algorithm in this area that is slower at sequential writes, but not terrible at random writes.

Thus, I am now pretty sure that my initial guess I added as a comment was right. What you are seeing when you write to the filesystem is the performance of the rest of the disk, and when you dd at offset zero, you are writing to the fat area. If you seek the dd destination a few hundred mb in, it should speed up considerably.

Related Question