Linux usb-drive write cache

file-copykernellinuxusb-drive

Is there any way to change the way the Linux kernel handles external storage and the external storage cache? Every time I copy something to it, the copy process seen from userspace seems really fast. I have seen this behaviour many times while copying files, but I just noticed that it also happens when I am copying data with dd directly to the drive. So nothing is mounted here!

Copying a .iso image:

$ sudo dd if=/mnt/mountpoint/Fedora-Live-Workstation-x86_64-23-10.iso of=/dev/sdc
2869248+0 Datensätze ein
2869248+0 Datensätze aus
1469054976 Bytes (1,5 GB) kopiert, 13,8922 s, 106 MB/s
$ sync

The first command takes 13s, but the second sync stage takes 90 secs or so.

Why is so much data (over 1,3 GB) held in cache? Is there any possibility of avoiding this behaviour?

EDIT:

If this is relevant for you. I am using a Ubuntu 15.10 4.2.0-34-generic standard kernel and it is a USB 2.0 usb-drive.

Best Answer

sudo dd bs=10M oflag=direct conv=fsync if=yourfile of=/dev/sdc

oflag=direct bypasses the OS cache, and conv=fsync makes sure everything is written before exitting.

As data is written directly to the disk, you need to write a lot of data at once to avoid wasting time writing it byte per byte, which is the default behavior. You can control this parameter using bs=yoursize.

If your version of dd is recent enough, you can even add status=progress to get a fancy progressbar.

Related Question