Scp does not transfer files correctly

scp

I am using scp to transfer some large files (> 1 GB) from my laptop (Ubuntu 12.04) to my desktop (Ubuntu 14.04). The laptop has a wireless connection to my router and the desktop has a cable connection.

Out of curiosity I decided to verify the transferred files using an md5 digest: I created the digest on the source computer, transferred it to the target computer, and used it with the -c option to verify the transferred files. To my surprise, more than 50% of the times I get an error:

md5sum: WARNING: 1 computed checksum did NOT match

My question: does data corruption occur so often with scp? I would expect that (1) data transfer errors are rather rare, and (2) scp checks its data and possibly resends it in case of error. What can be the cause of these errors? My hypotheses:

  1. Bad WLAN connection that frequently corrupts data during transfer.
  2. Bad memory: md5sum -c fails even thought the data is correct because the RAM is not working properly.
  3. Bad hard disk: md5sum -c fails because the disk contains error.
  4. A combination of the above reasons.

Note that my hard disk and RAM are pretty new: I bought them 5 months ago.

Best Answer

scp transfers files over SSH, which does cryptographic authenticity & integrity checking. That basically rules out the bad WLAN possibility.

Bad memory is reasonably likely. Note that bad memory often starts bad, it's not typically from getting old. Installing and running memtest86/metest86+ will either confirm this or mostly rule it out. (For ruling it out, you want to leave the test running for a while, at least overnight). If it finds an error, you don't need to keep it running, you can stop immediately and proceed to replacing DIMMs.

The disk corrupting it is also possible. Similarly, you could have bad cabling to the disk, or a defective controller, etc.

Other possibilities are filesystem bugs (unlikely if you're using something common like ext4), malware (thankfully fairly uncommon on Linux), but this is most likely a hardware problem.

Related Question