Linux – How to use btrfs send/receive for transmitting backup snapshots over a slow and unreliable network connection

backupbtrfslinux

I'd like to use btrfs' send/receive feature for transmitting backup snapshots over a rather slow (initial seed of about 50-100GB, upstream bandwith ~1-2MBit/s) and unreliable (daily forced interruption on both ends) connection.

I see following requirements:

  • encrypted transfer (usually achieved by using an SSH tunnel)
  • robustness to interrupted connections

It seems ZFS is able to resume interrupted transfers automatically, similar to how rsync does. Does this also apply to BTRFS? The send/receive wiki page is not useful with respect to interrupted transfers. If btrfs would resume interrupted transfers, all I would have to do is using an SSH tunnel and resume if interrupted.

If not, I'd have to use some buffer in-between that make sure the btrfs-connection survives interrupts, or get both servers close to each other for seeding (which will be a problem with respect to added files that excel the daily transmission capacities and sending snapshots).

What will I have to consider for transmitting the seed and snapshots?

Best Answer

I've been using "btrfs send" for backups for a while. It does not handle resuming broken transfers. I've written a utility that does resume transfers, and automatically syncs snapshots in the local btrfs file system to Amazon S3. You're welcome to try it out to see if it helps, and I'd love any feedback! It's both on GitHub (https://github.com/AmesCornish/buttersink) and PyPi (https://pypi.python.org/pypi/buttersink/).

Related Question