How to slow down rsync

rsync

I'm trying to copy the contents of a failing USB thumb drive. If I read the data too fast, the drive's controller chip overheats and the drive vanishes from the system. When that happens, I need to unplug the drive, wait a minute or so for it to cool, plug it back in, and re-start the copy.

I've got an old backup of the contents of the drive, so the obvious way to get the rest of the data is to use rsync to bring the backup up to date, but this runs into the whole "read too fast, the drive vanishes, and I need to start over" issue. Is there a way to tell rsync to only read X megabytes of data per minute? Alternatively, is it possible to tell it to suspend operations when the drive vanishes, and resume when it gets plugged back in?

Best Answer

Unlike DopeGhoti's experience, the --bwlimit flag does limit data transfer, with my rsync (v3.1.2).

test:

$ dd if=/dev/urandom bs=1M count=10 of=data
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0.0871822 s, 120 MB/s

$ du -h data
10M     data

$ time rsync -q data fast
0.065 seconds

$ time rsync -q --bwlimit=1M data slow
10.004 seconds

(note: my time output looks different to most time invocations (zsh feature), those times weren't edited by me)

Else, perhaps something along the lines of a double -exec in find. I believe that rsync -R should create & copy the parent folders, but if it doesn't, then cp --parents should.

$ find /failing/usb -exec rsync -R {} /somewhere/safe/ \; -exec sleep 1 \;

Note: also check out ddrescue, it might be right what you're looking for :)

Related Question