How reliable is (or will be) btrfs send/receive

backupbtrfs

We have at work a 15TB NFS file server using XFS, with a distant clone for backups using rsync, over a moderately slow connection (100Mb ethernet).

We are now thinking about using btrfs instead of XFS and use send/receive to synchronize the data more efficiently.

However fs corruption is probably unavoidable (we already experienced a 4-days fs repair operation with our XFS system), so the question is:

Will a future corruption be transferred and happily applied to the remote machine, if we use send/receive?

Best Answer

I have been using btrfs send for backups, and found it to be workable, though with frequent failures. It's important to detect a failed send and delete any partial/corrupt received volumes. I have not had any corruption problems after a successful send. I've written a "sync" utility for btrfs snapshots that detects failures, and also includes a checksumdir script for verifying correctness: https://github.com/AmesCornish/buttersink If you look at it, I'd love to know whether you find it useful or not.

Related Solutions

Linux – How to use btrfs send/receive for transmitting backup snapshots over a slow and unreliable network connection

I've been using "btrfs send" for backups for a while. It does not handle resuming broken transfers. I've written a utility that does resume transfers, and automatically syncs snapshots in the local btrfs file system to Amazon S3. You're welcome to try it out to see if it helps, and I'd love any feedback! It's both on GitHub (https://github.com/AmesCornish/buttersink) and PyPi (https://pypi.python.org/pypi/buttersink/).

Resumable transfer for btrfs send

As I understand it, your remote storage is exposed as a filesystem. I don't use btrfs myself but I assume the snapshots are equivalent to one large "full backup" file followed by a number of smaller "incremental" files.

On that basis I'd still go with rsync because it's restartable. You can't use its snazzy delta differences algorithm unless there's an rsync server available on the remote host, but you can tell rsync to assume the source file hasn't changed and to continue after a break from the byte offset it had reached:

test -t 2 && progress=--progress
rsync -av $progress --partial --append --sparse /path/to/source.img /path/to/remote/storage/

If you can usefully gzip your source file before transferring it, do so. (Neither --rsyncable nor rsync -z is relevant for what rsync sees as a local to local file transfer.)

Best Answer

Related Solutions

Linux – How to use btrfs send/receive for transmitting backup snapshots over a slow and unreliable network connection

Resumable transfer for btrfs send

Related Question