I would like to be able to merge changes from the upper filesystem of an overlay mount to a lower filesystem.
I am interested both online (i.e. merge while the overlay is mounted) and offline (unmount the overlay and then merge) solutions.
I have found a couple of offline solutions, which I have added as answers.
Does anyone know of any online solutions? It would be good to have a "commit" type command you could run to merge down the layers while the overlay is still mounted.
Something like this has been asked in the following questions with no answer:
- Is there anyway to modify one file and sync to the lower directory in OverlayFs?
- How to modify a lower file through an OverlayFS directory?
- OverlayFS Seamlessly Edit File in Lower Directory
- OverlayFS – How to make changes to upper filesystem persistent without unmounting?
Comments in these posts suggest variously mergerfs and bcache, both of which solve specific use cases but not the generic filesytem-agnostic use case that overlays provide.
My goal is to have a safe filesystem sandbox with snapshots that can be used with any Linux application over any (where practical) underlying filesystem, allowing you to roll back changes or manually commit them when you are ready.
I have a suspicion that modern mainline Linux has all the necessary features to do this built-in, thanks to all the sandboxing/virtualization innovations of the last few years, if only I knew how to use them.
Best Answer
An attempt at an online solution, but its not quite there.
The setup (in e.g.
/tmp
directory, as root):Then you can work in
O
directory, which is an overlay over$LOWER
. When you want to do the snapshot:(Note that nested overlays like this won't work on older kernels).
But then I want some way to atomically change the bind mount at
O
to point too2
instead ofo1
. I don't know how to do this other than:(Not atomic; there is a window where
O
is unmounted).Ideally, running processes could continue to run without knowing that the underlying filesystem of
O
had changed fromo1
too2
. I don't know if this is possible, or whether changing the underlying filesystem ofO
like this will disrupt open applications too much. I need to investigate further.Then, once
O
has been redirected too2
, we can remounto1
read-only as a precaution, then perform an offline merge using for example rdiffdir or overlayfs-tools.Finally, we would want some way to atomically remount
o2
aslowerdir=$HOME,upperdir=u2,workdir=w2
so thato1
,u1
andw1
(all now empty dirs) could be removed. Again, I don't know if this is possible.Otherwise, we can achieve snapshots by just nesting overlays deeper and deeper and leaving the overlay and upper dirs for each mounted without attempting to merge or cleanup. But there is probably a limit to the number of nested overlays that can be mounted. And at some point, we still need to merge the layers downwards if we want to persist changes.