Software RAID 10 + LVM on mixed drives, sector alignment

fdisklvmmdadmpartitionsoftware-raid

I have 4 2.5" 7.2k drives lying about and I'd like to build some RAID arrays for a bit of VM storage on my KVM lab box.

There are 2x 320GB drives and 2x 500GB drives. I'm looking to create a RAID 10 array using 4x 320GB partitions. Then the remaining 180GB on the 500GB drives I'll build into a RAID 5 partition with another spare drive. I realise that performance will take a hit because there'll be contention on 2 of the drives in the RAID 10 array and using non-matched drives is also not-optimal but this is solely for a lab/test environment.

Initially I thought I would just create 4 matching partitions, create the RAID 10 array, LVM on top of that and done. But once I started looking into LVM on top of raid and 4k disks the subject of sector/block alignment came up and now I don't know where to start.

The 320GB drives report Sector size (logical/physical): 512 bytes / 512 bytes
The 500GB drives report Sector size (logical/physical): 512 bytes / 4096 bytes

So from some reading there are the following variables that I need to take into account for this:

  • Partition start/alignment
  • RAID stripe size
  • RAID superblock position (?)
  • LVM PV –dataalignment
  • then alignment of any file systems in the LVs

Here are the resources that I am basing this on: https://askubuntu.com/a/19572 and http://tytso.livejournal.com/2009/02/20

In some brief exchanges on IRC I have also received conflicing advice, that mdadm, the LVM tools and even fdisk are clever enough these days to take this into account and that I shouldn't worry. Is that the case?

Is anyone able to run through what I'd need to do in this specific case to get optimal alignment/sector/stripe size?

Please try and ignore the fact that the mixed drives mean the whole thing isn't optimal to start with, as I said, this is a test/lab environment.

Best Answer

  • Partition start/alignment

Make them start at 1 MiB boundaries, for example using parted and unit mib. That way you won't have an issue with today's 4k sector disks, and not with tomorrow's 8k or 16k disks... and you only waste 1MiB per disk.

You can verify the partition alignment of any given disk using parted /dev/disk unit b print free. It prints units in bytes so you can see whether a start of 1250249670656 is really a multiple of 4096 (4k) or 1048576 (1MiB).

Note that in a msdos partitioning scheme, only primary and logical partitions have to be aligned. The extended partition (container for the logicals) does not matter.

  • RAID stripe size

Usually 64k or 512k, so it's a multiple of 4k already and not an issue.

  • RAID superblock position

Doesn't matter. If the superblock is at the end, the alignment is the partition itself. If the superblock is at the start, mdadm uses a data offset that will be multiple of MiB (up to 128 MiB). Check the data offset with mdadm --examine /dev/sda1.

  • LVM PV --dataalignment

The LVM offset is usually 1 MiB as well, check with pvs -o +pe_start.

  • then alignment of any file systems in the LVs

Once the PV itself is aligned, all LV are automatically aligned as well, because the PE size is (usually) also a multiple of 1 MiB. (4 MiB or larger)

And filesystems have used a 4k blocksize for a very long time. At least with the standard Linux filesystems you can't really do anything wrong at this point.

even fdisk

While there is some development with fdisk lately, I'd go for parted or gdisk.

I'd not worry about mixed drives either. That's the kind of flexibility you get from Linux Software RAID, it's a good thing. I have mixed disk sizes myself and never noticed any issue because of it.

Related Question