OverlayFS – Fix OverlayFS Over Read-Only Rootfs Fail

overlayfs

I need to keep one system as much intact as possible. Only soldering of HW stuff is allowed :-). I need to install a test software package and this package must not stay there in the future.

I have a following situation:

mmcblck partition mounted as /, ext4, read only, kernel v4.6.0
usb stick (only one partition), mounted to /tmp/usbstick, ext4
Created directories on usbstick /tmp/usbstick/upperdir, /tmp/usbstick/workdir
Using the following line: mount -t overlay overlay -o lowerdir=/,upperdir=/tmp/usbstick/upperdir,workdir=/tmp/usbstick/workdir /

After that the / is still read only.
The only partial success I had was when I created /tmp/usbstick/merged and gave it as a "merged" directory, instead of / to the module. Then I saw all my rootfs in that directory and it was rw mounted, but I can't use it there. What should I do?

Best Answer

I tried to chroot to the merged directory. The result was just as expected: I had rw rootfs, the only thing I missed was virtual kernel filesystems. So after mounting the overlay I did the following:

TARGETDIR="/tmp/usbstick/merged"
mount -t proc proc $TARGETDIR/proc
mount -t sysfs sysfs $TARGETDIR/sys
mount -t devtmpfs devtmpfs $TARGETDIR/dev
mount -t tmpfs tmpfs $TARGETDIR/dev/shm
mount -t devpts devpts $TARGETDIR/dev/pts

And then linked the mtab:

chroot $TARGETDIR rm /etc/mtab 2> /dev/null 
chroot $TARGETDIR ln -s /proc/mounts /etc/mtab
chroot $TARGETDIR

Related Solutions

Overlayfs with a lot of directories

(I have exactly the same problem.) The only info I could find about it, is this:

From https://lkml.org/lkml/2014/11/10/129 :

Maybe it wasn't clear, but the number of lower layers isn't limited by
FILESYSTEM_MAX_STACK_DEPTH, only by the max size of the mount option buffer in
the kernel (1 page, usually 4096bytes).  So you could have a hundred read-only
layers stacked in a single overlayfs mount.

As for changing the stacking while the overlayfs is mounted: currently this is
not supported, the layers specified at the mount time remain there until the
overlay is unmounted.  Currently there's no possibility to add or remove layers
in a dynamic way, and it is definitely more tricky to implement than the static 
configuration.

-- Miklos Szeredi Mon, 10 Nov 2014

OverlayFS – Seamlessly Edit Files in Lower Directory

I ran into the exact same problem (trying to do the exact same thing: a read-only base system for a PXE boot environment).

The behavior I'm seeing exactly matches that in this Ubuntu bug report. I cannot write changes to existing files, but I can delete them (but they're preserved in the lower filesystem... with the upper layer just recording their absence) and then write them back again (at which point the upper layer has a copy which I can edit).

From what I can dig up, this is happening because OverlayFS and NFS don't play nice together. The thing that we're expecting to happen, when we try to modify a file from the lower filesystem, is called "copy up", and it breaks when OverlayFS tries to use a NFS filesystem because NFS doesn't support xattrs.

Thus far, the only promising avenue I can find is that it's possible to use fuse_xattrs to emulate xattrs on your NFS mount (described here), but it requires that you have two mounts of your NFS share: the "real" one, and the "xattr-enhanced" one that needs the first.

Best Answer

Related Solutions

Overlayfs with a lot of directories

OverlayFS – Seamlessly Edit Files in Lower Directory

Related Question