Linux – How to Fix Slow rsync Copy Between Disks

linuxperformancersyncubuntu 12.04

I got two drives of the same model which are connected using SATA.

When benchmarking read speeds using hdparm -t /dev/sdX, speeds of about 160MB/s are returned. When benchmarking write-speed using dd if=/dev/zero of=testfile bs=1M count=500 conv=fdatasync, speeds of about 140MB/s are returned.

However, when copying a single 10GB-file using rsync --progress /mnt/hd1/file /mnt/hd2/file, the writespeed is only about 35MB/s.

Why is it so slow? How can I make it faster?

Best Answer

According to many sites I found while researching this issue (for example this one), this is just normal as the bottleneck of rsync usually is CPU-power.

Results with dd and cp were near the speeds I initially had benchmarked. Seems like a 2,2Ghz dual core is just not enough for hdd-speed rsync.

During further research I also found out about this:

Correct, rsync has no option to disable the post-transfer checksum completely. I implemented a patch to rsync 2.6.9 that adds an option --trust-append that limits the post-transfer checksum to the added portion, not the entire file. The patch is attached. That should be good enough, but if you really want to disable the checksum completely, just comment out the remaining sum_update calls in match.c and receiver.c .

Rsync always checksums the whole file which takes alot of time. Using the patch mentioned above I managed to increase rsync speed to about 90MB/s. Still not great, but much better then before. Sadly, the patch has not made it into the rsync-trunk.

Related Question