LVM Management – How to Extend a Partition with LVM, Physical Volume, and Logical Volume?

• Partition start/alignment

Make them start at 1 MiB boundaries, for example using parted and unit mib. That way you won't have an issue with today's 4k sector disks, and not with tomorrow's 8k or 16k disks... and you only waste 1MiB per disk.

You can verify the partition alignment of any given disk using parted /dev/disk unit b print free. It prints units in bytes so you can see whether a start of 1250249670656 is really a multiple of 4096 (4k) or 1048576 (1MiB).

Note that in a msdos partitioning scheme, only primary and logical partitions have to be aligned. The extended partition (container for the logicals) does not matter.

• RAID stripe size

Usually 64k or 512k, so it's a multiple of 4k already and not an issue.

• RAID superblock position

Doesn't matter. If the superblock is at the end, the alignment is the partition itself. If the superblock is at the start, mdadm uses a data offset that will be multiple of MiB (up to 128 MiB). Check the data offset with mdadm --examine /dev/sda1.

• LVM PV --dataalignment

The LVM offset is usually 1 MiB as well, check with pvs -o +pe_start.

• then alignment of any file systems in the LVs

Once the PV itself is aligned, all LV are automatically aligned as well, because the PE size is (usually) also a multiple of 1 MiB. (4 MiB or larger)

And filesystems have used a 4k blocksize for a very long time. At least with the standard Linux filesystems you can't really do anything wrong at this point.

even fdisk

While there is some development with fdisk lately, I'd go for parted or gdisk.

I'd not worry about mixed drives either. That's the kind of flexibility you get from Linux Software RAID, it's a good thing. I have mixed disk sizes myself and never noticed any issue because of it.

Does the LV become mountable if you do a sudo vgscan and sudo vgchange -ay? If those commands result in errors, you probably have a different problem and should probably add those error messages in your original post.

But if the LV becomes ready for mounting after those commands, read on...

The LVM logical volume pathname (e.g. /dev/mapper/vgNAME-lvNAME) in /etc/fstab alone won't give the system a clue that this particular filesystem cannot be mounted until networking and iSCSI have been activated.

Without that clue, the system will assume that filesystem is on a local disk and will attempt to mount it as early as possible, normally before networking has been activated, which will obviously fail with an iSCSI LUN. So you'll need to supply that clue somehow.

One way would be to add _netdev to the mount options for that filesystem in /etc/fstab. From this Ubuntu help page it appears to be supported on Ubuntu. This might actually also trigger a vgscan or similar detection of new LVM PVs (+ possibly other helpful stuff) just before the attempt to mount any filesystems marked with _netdev.

Another way would be to use the systemd-specific mount option x-systemd.requires=<iSCSI initiator unit name>. That should achieve the same thing, by postponing any attempts to mount that filesystem until the iSCSI initiator has been successfully activated.

When the iSCSI initiator activates, it will automatically make any configured LUNs available, and as they become available, LVM should auto-activate any VGs on them. So, once you get the mount attempt postponed, that should be enough.

The lack of PARTUUID is a clue that the disk/LUN does not have a GPT partition table. Since /dev/sdc is listed as TYPE="LVM2_member" it actually does not have any partition table at all. In theory, it should cause no problems for Linux, but I haven't personally tested an Ubuntu 18.04 system with iSCSI storage, so cannot be absolutely certain.

The problem with disks/LUNs with no partition table is that other operating systems won't recognize the Linux LVM header as a sign that the disk is in use, and will happily overwrite it with minimal prompting. If your iSCSI storage administrator has accidentally presented the storage LUN corresponding to your /dev/sdc to another system, this might have happened.

You should find the LVM configuration backup file in /etc/lvm/backup directory that corresponds to your missing VG, and read it to find the expected UUID of the missing PV. If it matches what blkid reports, ask your storage administrator to double-check his/her recent work for mistakes like described above. If it turns out the PV has been overwritten by some other system, any remaining data on the LUN is likely to be more or less corrupted and it would be best to restore it from backup... once you get a new, guaranteed-unconflicted LUN from your iSCSI admin.

If it turns out the actual UUID of /dev/sdc is different from expected, someone might have accidentally run a pvcreate -f /dev/sdc somehow. If that's the only thing that has been done, that's relatively easy to fix. (NOTE: check man vgcfgrestore chapter REPLACING PHYSICAL VOLUMES for updated instructions - your LVM tools may be newer than mine.) First restore the UUID:

pvcreate --restorefile /etc/lvm/backup/<your VG backup file> --uuid <the old UUID of /dev/sdc from the backup file> /dev/sdc

Then restore the VG configuration:

vgcfgrestore --file /etc/lvm/backup/<your VG backup file> <name of the missing VG>

After this, it should be possible to activate the VG, and if no other damage has been done, mount the filesystem after that.