Ubuntu – Backup LVM snapshots and Grub2 problem

backupgrub2lvm

Long story short, I'm using obnam to take backups of my system.

This is what I do:

  1. take LVM snapshot of /, /usr, /boot, /var, /home
  2. mount the snapshots on /bck-snapshots, /bck-snapshots/usr, /bck-snapshots/boot, /bck-snapshots/var, /bck-snapshots/home

  3. run the backup of snapshots:

    obnam backup \
        /bck-snapshots \
        /bck-snapshots/usr  \
        /bck-snapshots/boot \
        /bck-snapshots/var  \
        /bck-snapshots/home \
    
  4. once the backup finishes, umount snapshot filesystems mounted in step 2.

  5. remove the snapshots

Problem

during the backup, a system update was done, with a new kernel installed and all the grub2 update dance. Now when I try to remove the root snapshot (step 5.) I get this:

device-mapper: remove ioctl on failed: Device or resource busy

because the snapshot root device is busy and used by another process:

$ sudo lsof /dev/dm-6 # This is the root snapshot device.
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
COMMAND    PID USER   FD   TYPE DEVICE   SIZE/OFF    NODE NAME
grub-moun 9160 root    3r   BLK  252,6 0t36339712 5295479 /dev/dm-6

Questions

  1. I guess that to lvremove the snapshot I have to stop grub-mount to use the device. How to do that safely?

  2. I think that the backup snapshot has been added to grub as a viable option to show in the grub menu shown at boot time. What if I remove the corresponding snapshot?

  3. Any way to prevent grub from adding the root snapshot found on /bck-snapshots ?

I have to reboot the system in order for the updates to be active, but first I have to know the answer to these questions.

Best Answer

It looks like grub-mount is trying to edit a read only file system and has hung.

It should be safe to send a signal with kill to ask it to give up (send 15, and if needed after a few seconds 2, and if needed after a few seconds 1, if it's still running use -9 and report it to the maintainer) (if the pid number changes start with 15 again )

It's likely to exit non-zero which may effect scripts waiting on it, but the only thing you can do about that is file a bug report and ask the maintainer to have it skip read only file systems.

Note that there are some related bugs they are working on.