Linux – chroot before pivot_root causes busy error

linux

# unshare -m
# mount --bind / /mnt
# cd /mnt
# chroot .
# pivot_root . mnt
pivot_root: failed to change root from `.' to `mnt': Device or resource busy

Why does it fail? I was following the instructions from man 2 pivot_mount.

pivot_root() may or may not change the current root and the current
working directory of any processes or threads which use the old root
directory. The caller of pivot_root() must ensure that processes with
root or current working directory at the old root operate correctly in
either case. An easy way to ensure this is to change their root and
current working directory to new_root before invoking pivot_root().

I don't see how this matches the documented EBUSY error.

ERRORS

pivot_root() may return (in errno) any of the errors returned by
stat(2). Additionally, it may return:

EBUSY new_root or put_old are on the current root filesystem, or a
filesystem is already mounted on put_old.

Best Answer

That part of the manpage is misleading. Generally you want a different ordering, as described in man 8 pivot_root.

cd new_root             # chdir(new_root);
pivot_root . put_old    # pivot_root(".", put_old);
exec chroot .           # chroot(".");

This seems to be yet another subtle detail with pivot_root. Although the point of pivot_root is to rearrange the mount namespace, the kernel code seems to say that the root filesystem that it moves is determined by the per-process root, which is what chroot sets.

As a result, we hit the error "new_root or put_old are on the current root filesystem".

This subtle detail of pivot_root is necessary in order for it to work at all on modern Linux. If it was defined to work on the root mount of the mount namespace, it would try to move the special rootfs filesystem which you normally can't see. But this is not allowed, because rootfs must always be the root mount of the namespace.

We can confirm pivot_root works this way, by continuing the example as follows.

# unshare -m
# mount --bind / /mnt
# cd /mnt
# chroot /mnt
# pivot_root . mnt
pivot_root: failed to change root from `.' to `mnt': Device or resource busy

# exit  # leave chroot
# mount --bind . mnt
# cd mnt
# mount --bind /proc proc
# findmnt | grep mnt
└─/mnt                                /dev/mapper/alan_dell_2016-fedora ext4            rw,relatime,seclabel
  └─/mnt                              /dev/mapper/alan_dell_2016-fedora ext4            rw,relatime,seclabel
    └─/mnt/proc                       proc                              proc            rw,nosuid,nodev,noexec,relatime

# chroot /mnt  # re-enter chroot
# cd /mnt
# pivot_root . mnt  # this one works
# exit  # leave chroot
# findmnt | grep mnt
└─/mnt                                /dev/mapper/alan_dell_2016-fedora        ext4            rw,relatime,seclabel
  ├─/mnt/mnt                          /dev/mapper/alan_dell_2016-fedora        ext4            rw,relatime,seclabel
  └─/mnt/proc                         /dev/mapper/alan_dell_2016-fedora[/proc] ext4            rw,relatime,seclabel

The second pivot_root call works. But it didn't have any effect on the root of the mount namespace. Looking from outside the chroot, it swapped /mnt and /mnt/mnt.

Related Question