Linux – root filesystem on NVMe device

bootgrub2linuxnvme

I have an Intel 750 NVMe drive. Works fine. Well.. sort of. I would like to use this device as my root file system.

I have no bios support for NVMe devices.

  1. I have the nvme driver enabled in the kernel and not a module.
    I can mount the nmve device. I copied the root file system to it.

When I try and use this device as a root file system in grub2 I get a unknown device.

This is the grub2 line for the kernel and parameters:

linux   /boot/kernel-4.1.6-gentoo root=/dev/nvme0n1p1 ro quiet

why is this? The driver is in the kernel. Its not a module. It needs nothing from the root file system. Is it using bios to access /dev/nvme0n1p1?

  1. My second attempt was to use an initramfs. I used genkernel to create
    the initramfs.

    This also has a problem with /dev/nvme0n1p1. It boots into linux using the initramfs. But, when it tries to mount the real root file system via the
    linuxrc script it fails on the check of the block device

….
Check for a block device or /dev/nfs
elif [ -b "${REAL_ROOT}" ] || [ "${REAL_ROOT}" = "/dev/nfs" ]
…

REAL_ROOT is set to dev/nvme0n1p1 (I added some debug code)

I hacked the above script to just mount /dev/nvme0n1p1 and that also
fails. Its not there.

The strange thing is… after failing it asks the user to enter the root file system… and …
I enter "/dev/nvme0n1p1" and it works fine. Boots.

Also, it will let you enter a shell.. and /dev/nvme0n1p1 is there… and the
test 

elif [ -b "${REAL_ROOT}" ]

passes in the shell.

So, what is happening in #1. Should this work? Is the BIOS being used
(which would fail because my BIOS does not support NVMe)?

#2 is very strange and looks like a bug to me. That being said… how can
anyone use an NVMe as a root device then. I'm sure google would have
returned something from my search.

Best Answer

Nvme devices, in line with many other MTD devices in Linux kernel, rely on asynchronous initialization. After the controller is discovered by kernel, the block layer will initiate the partition scan, but the kernel will not wait for it (by default). This means that an attempt to mount a root file system on such a device will either fail or will be racy, because by the time kernel attempts to mount the root fs the partition layout may not yet be known.

To make sure kernel actually waits for root device to appear instead of rushing forward to an inevitable panic, it must be passed a rootwait option on the boot loader/embedded command line (rootdelay with explicit timeout is another option, but rootwait is clearly simpler and safer).

The problem only manifests itself with direct booting kernels, because initramfs activation ensures completion of initialization tasks (or may be simply introduces enough delay, not sure about it) for partition scan to finish and all partitions to become registered and available.

Related Question