How to clone btrfs filesystem into different medium preserving snapshots’ sharing data

backupbtrfs

I have decided to give btrfs raid capabilities a try. I set up a btrfs with

sudo mkfs.btrfs -m raid10 -d raid10 /dev/sda9 /dev/sdb9 /dev/sdc9 /dev/sdd9

Now I want to clone my existing btrfs partition, (which sits on top of linux-raid). can't use a simple cp -a, because there is over 40 snapshot-based backups (which I wish to preserve), and I would simply overfill all the storage I can spare multiple times.

So far I can see two options:

partclone.btrfs -s /path/to/original/fs -o /dev/sda9 -b

and I suppose I would also need to btrfs balance start /dev/sda9

and

do: incrementaly copy with cp -a as much as fits into the storage, and then use bedup to de-duplicate files, and loop.

What is the preferred (i.e. best-practice) method? I would much prefer the first one; it should take much less time. Or maybe there is some evil "gotcha" lurking in any of this procedures (beside the fact that btrfs is experimental, of course)

The first question is simply out of question; however wonderful tool the partclone.btrfs is, it obviously doesn't support multi-device file systems. 🙁

Best Answer

I asked a similar question 2 years ago.

However in my case, I was only planning to copy a single device onto raid0.

I eventually found a solution. At the time you couldn't convert from raid0 to raid10, but it looks like that since kernel 3.3, you can now. So that solution may work for you in the end.

A problem with that approach is that it copies the fsuid. Which means you can't mount both the FS and its copy on the same machine. At the time, there was no tool to change the fsuid of a FS, but it might have changed now.

The idea is to add a copy-on-write layer on top of the original device so that it can be written to, but any modification is done somewhere else which you can discard later on. That means you need additional storage space (for instance on an external drive).

Then mount that COW'd FS instead of the original, add the devices for the FS copy and remove the COW's device.

For copy-on-write, you can use the device mapper.

For the disposable copy on write area, here I use a loop device.

Let's say you want to clone /dev/sda onto /dev/sd[bcde]:

Create the COW back store:

truncate -s 100G /media/STORE/snap-store
losetup /dev/loop0 /media/STORE/snap-store

Now unmount the origin FS if mounted and modprobe -r btrfs to make sure it's not going to interfere and make it forget its device scan.

Then make the COW'd device:

echo "echo 0 $(blockdev --getsize /dev/sda) snapshot /dev/sda /dev/loop0 N 8 | dmsetup create cowed

Now /dev/mapper/cowed is like /dev/sda except that anything written to it will end up in /dev/loop0 and /dev/sda will be untouched.

Now, you can mount it:

mount /dev/mapper/cowed /mnt

Add the other devices:

btrfs dev add /dev/sd[bcde] /mnt

And remove the old one:

btrfs dev del /dev/mapper/cowed /mnt

When that's over, you may want to shutdown and unplug or make /dev/sda readonly as because it's got the same fsuid as the other ones, btrfs might still mess up with it.

Now, if I understand correctly, assuming you've got recent btrfs-prog, you should be able to do a:

btrfs balance start -d convert=raid10 /mnt

To convert to raid10. In theory, that should make sure that every data chunk is copied on a least 2 disks.

I would strongly recommend that you do tests on a dummy btrfs on loop devices first as all that is from memory and I might have gotten it wrong (see for instance my initial answer before my edit).

Note that since kernel 3.6, btrfs implements send/receive a bit like in zfs. That might be an option for you.

Related Question