Btrfs – Why Metadata Conversion to RAID1 Fails

btrfslinux

I created a single-device btrfs filesystem. According to the btrfs wiki article on using multiple devices, I should be able to convert that to RAID1 using:

btrfs balance start -dconvert=raid1 -mconvert=raid1 /path

I started that on Linux 3.16, and it failed (kernel panic). Upgraded to Linux 4.0, after mounting the filesystem it continued and finished. But it only did the data, not the metadata or system (according to btrfs fi df). I grabbed the latest btrfs-progs from git (just to make sure it wasn't due to an old version), and did:

Watt:/home/anthony/src/btrfs-progs# ./btrfs balance start -v -mconvert=raid1 /path
Dumping filters: flags 0x6, state 0x0, force is off
  METADATA (flags 0x100): converting, target=16, soft is off
  SYSTEM (flags 0x100): converting, target=16, soft is off
Done, had to relocate 6 out of 1411 chunks

But that didn't actually mirror it, apparently. Right now, I have:

Watt:/home/anthony/src/btrfs-progs# ./btrfs fi usage /path
Overall:
    Device size:                   7.28TiB
    Device allocated:              2.75TiB
    Device unallocated:            4.53TiB
    Device missing:                  0.00B
    Used:                          2.74TiB
    Free (estimated):              2.26TiB      (min: 2.26TiB)
    Data ratio:                       2.00
    Metadata ratio:                   2.00
    Global reserve:              512.00MiB      (used: 0.00B)

Data,RAID1: Size:1.37TiB, Used:1.37TiB
   /dev/mapper/luks-562e4e2f-2894-415a-aaf1-7c94a11c33b9           1.37TiB
   /dev/mapper/luks-ec97c1ad-21d8-41bb-9072-e5a74f68e416           1.37TiB

Metadata,DUP: Size:2.50GiB, Used:1.58GiB
   /dev/mapper/luks-562e4e2f-2894-415a-aaf1-7c94a11c33b9           5.00GiB

System,DUP: Size:32.00MiB, Used:224.00KiB
   /dev/mapper/luks-562e4e2f-2894-415a-aaf1-7c94a11c33b9          64.00MiB

Unallocated:
   /dev/mapper/luks-562e4e2f-2894-415a-aaf1-7c94a11c33b9           3.17TiB
   /dev/mapper/luks-ec97c1ad-21d8-41bb-9072-e5a74f68e416           1.36TiB

I tried the full balance again (with both -dconvert=raid1 and -mconvert=raid1) and that didn't do it, either.

NOTE: The larger disk (56…b9) is the one I added.

How can I get the metadata and system mirrored?

Best Answer

This is a regression in kernel 4.0, causing conversion filters in balance to have no effect; it looks like all conversions are affected (not just single->raid1 or raid1->raid5). See a recent mailing list thread, where there's currently no official fix. If you're up to patching your kernel, there's an easy patch to apply as a temporary fix.

This is a known bug in v4.0. I sent in a patch [1] to revert the commit that caused the regression, but it didn't get any response. You could apply that or just revert 2f0810880f08 ("btrfs: delete chunk allocation attemp when setting block group ro") to fix your problem for now.

[1]: https://patchwork.kernel.org/patch/6238111/

Related Solutions

Why does btrfs allow to create a raid1 with mismatched drives

I found out the answer by asking on the mailing-list.

btrfs doesn't do RAID per-volume, but rather on a per-chunk basis. The filesystem reserves "raw" space in (p.e.) 1GB chunks. Initializing the fs with raid1 means that every time it tries to allocate a chunk, it tries to allocate a copy of this chunk on another device.

This architecture allows mixed-size devices and its future features might include per-file raid-levels.

Currently df shows you the amount of free raw space on the devices, which is the sum of all the device-sizes. Assuming the chunk-size is 1GB, writing just a 5MB file onto a raid1-btrfs will therefore decrease the raw space by 2GB. That is why btrfs includes the btrfs fi df command to give you the actual usage. This will probably be adapted in the future to show more precisely what is going on.

BTRFS convert RAID0 to RAID1

When making a btrfs filesystem across multiple devices, the default is to use RAID0 for data, and RAID1 for metadata. If you want to change this you can pass arguments to the mkfs command to change RAID level of data (-d) and metadata (-m):

mkfs.btrfs -d raid1 -m raid1 /dev/sda /dev/sdb

To confirm what RAID level is being used on an existing filesystem, you can use the btrfs utility:

$ btrfs fi df /mountpoint
Data, RAID1: total=15.00GiB, used=14.65GiB
System, RAID1: total=8.00MiB, used=16.00KiB
Metadata, RAID1: total=1.00GiB, used=15.59MiB

The example output shows RAID1 used for both data and metadata. Running btrfs fi usage /mountpoint will provide more detail.

You can convert an existing filesystem to a different RAID level by passing arguments to the balance command.

btrfs balance start -dconvert=raid1 -mconvert=raid1 /mountpoint

A balance writes all data to the filesystem again, and adding arguments will cause the data to be converted as it is rewritten. This may take a while, you can run btrfs balance status /mountpoint to see the status. Once it finishes, you can confirm that all data was converted as expected with btrfs fi df. Data written during a balance may still use the old format so a second balance may be needed.

Utilities such as the regular df command or btrfs fi show do not take RAID settings into account, as that is done by btrfs itself. These tools only show the total amount allocated by btrfs on the disks, they don't know what format btrfs is using for the data. This also means sharing via SAMBA or other tools will not account for RAID. For more info, see the btrfs FAQ.

Best Answer

Related Solutions

Why does btrfs allow to create a raid1 with mismatched drives

BTRFS convert RAID0 to RAID1

Related Question