So I've got two fresh (4TB) drives destined to serve as main storage for a desktop system. The primary requirement is redundancy and reliability. Secondary considerations are data read speed and maintainability especially in case of drive failure.
The file system of choice will be BTRFS for reasons outside the scope of this question.
What I can not decide on is, whether to create an MD raid 10 array in 'far 2' configuration with non-raided BTRFS on top or to use the raid 1 functionality included in BTRFS for both metadata and file content.
Either option will fulfil the redundancy requirement and the fact that BTRFS is included in the Linux kernel and not flagged as experimental reassures me that it will probably be reliable enough, escpecially in respect to potential data corruption (I am writing this as kernel 4.10.3 has just been released).
I have not been able to find any performance data so far that covers this setup specifically. While the administrative details are different for each setup I dont' think there is much difference in the complexity of maintaining the two – but I am saying this without experience with btrfs so far (but with md).
I currently favour md10 over btrfs raid 1 because it is raid 10 that can run on only two disks and has there a decided performance advantage over md1. However the underlying architecture of btrfs raid 1 is different to md1 as it only guarantees two copies of each dataset, even on setups with more than two disks, rather than a complete mirror of each disk.
The question is now if this freedom of data allocation has been implemented in a way that there are speed improvements in btrfs raid 1 over md1 that make it perform on par with md10? Are there configuration options I should consider?
A second question is, whether the integration of the raid architecture into the file system itself gives any benefits at all?
And then of course I would like to hear if any of my assumptions regarding redundancy, reliability and maintainability are incorrect.
Best Answer
As a general rule, MD will outperform BTRFS for the foreseeable future. BTRFS currently serializes writes to multiple devices, and only reads from one for a given read() call.
As far as data safety though, you're better off using BTRFS for replication than MD or LVM2. Because of the internal block checksumming done by BTRFS, it can tell when you make a read call whether or not the data is good and automatically fall back to the other copy if it isn't, and it will correctly select the good copy of a block on a per-block basis when scrubbing.
Now, all of that said, the best option I've found so far is actually to use BTRFS in raid1 mode on top of two MD or LVM2 RAID0 volumes. While this does not perform quite as well as ext4 or XFS on top of MD RAID10, it still performs far better than the BTRFS raid10 implementation or running a single BTRFS device on top of MD RAID10 on the same devices, and provides the same data safety guarantees. Unfortunately, that does require four devices.