Is it possible to have subvolumes with different replication modes in btrfs

btrfsraid

I am setting up a system with two 3TB hard drives as storage backing. As it's a desktop station, I thougth about the following disk setup:

sda
+ /boot with 500MB
+ btrfs for the remaining GBytes
sdb
+ btrfs for all the 3000GB

I then thought about using the single-mode for data on / and create a subvolume for /home (and potentially others) using raid1 as data mode. Is that possible with current (kernel 3.7.X) btrfs? Does anyone have sources for that, I was unable to find anything.

UPDATE: I made a small test setup consisting of two plain files hooked in as loopback devices for playing around. On those files, I created a plain btrfs. Then I did the following:

btrfs subvolume create test
dd if=/dev/urandom of=foo bs=1024 count=1  # use random data to prevent dedup
dd if=/dev/urandom of=test/bar bs=1024 count=1  # use random data to prevent dedup
btrfs balance start -dconvert=raid1 test
btrfs balance start -dconvert=single .
btrfs filesystem df .

That showed the filesystem having all its usage in single mode, instead of partial RAID1 and partial single mode… what did I do wrong?

Best Answer

The btrfs balance command applies to filesystems as shown by the help even thought the "filesystem" parameter is optional:

# btrfs balance start --help
usage: btrfs [filesystem] balance start [options] <path>

Until there is a "btrfs subvolume balance" command then I suspect this feature doesn't exist.

(Above based on Btrfs Btrfs v0.19)

Related Solutions

Why is moving files between btrfs subvolumes an expensive operation

How is that different subvolumes can share data blocks when using COW, but not in the should-be easier operation of moving data?

mv uses the rename syscall to attempt the move. btrfs's kernel rename impl detects the cross subvolumes move and explicitly disallows this (even if under the same mount point):

/* we only allow rename subvolume link between subvolumes */
if (old_ino != BTRFS_FIRST_FREE_OBJECTID && root != dest)
    return -EXDEV;

This probably has to do with subvolume inode accounting and the code paths these operations take. The reflink-copy is actually creating new metadata (but the data itself is CoW) accounted in the new subvolume. In theory they probably could make rename "move" the metadata by doing something similar to what copy --reflink followed by rm source does... simply no one has taken the effort to do it.

Linux BTRFS – convert to single with failed drive

Alright, I figured it out with the help of this Trello link. In case anyone else wants to do this, here's the procedure.

Procedure

From a RAID1 array of two disks, one /dev/sda which is faulty and another /dev/sdc known-good:

Disable auto-mounting of this array in /etc/fstab, reboot. Basically, we want btrfs to forget this array exists, as there's a bug where it'll still try to use one of the drives if it's unplugged.
Now that your array is unmounted, execute:

echo 1 | sudo tee /sys/block/sda/device/delete

replacing sda with the faulty device name. This causes the disk to spin down (you should verify this in dmesg) and become inaccessible to the kernel.

Alternatively: just take the drive out of the computer before booting! I chose not to opt for this method, as the above works fine for me.
Mount your array, with -o degraded mode.
Begin a rebalancing operation with sudo btrfs balance start -f -mconvert=single -dconvert=single /mountpoint. This will reorganise the extents on the known-good drive, converting them to single (non-RAID). This will take almost a day to complete, depending on the speed of your drive and size of your array. (mine had ~700 GiB, and rebalanced at a rate of 1 1GiB chunk per minute) Luckily, this operation can be paused, and will keep the array online while it occurs.
Once this is done, you can issue sudo btrfs device remove missing /mountpoint to remove the 'missing' faulty device.
Begin a second rebalance with sudo btrfs balance start -mconvert=dup /mountpoint to restore metadata redundancy. This takes a few minutes on my system.
You're done! Your array is now single mode, with all redundancy removed.
Take your faulty drive outside, and beat it with a hammer.

Troubleshooting

Help, btrfs tried to write to my faulty disk, errored out, and forced it readonly!
- Did you follow step 1, and reboot before continuing? It's likely that btrfs still thinks the drive you spun down is present. Rebooting will cause btrfs to forget any errors, and will let you continue.