Identify files/processes that prevent remounting read-only

filesmountopen filesprocess

My system runs Debian Testing, and / is generally mounted as read-only.

The main problem with this is, every time I install something or update the system, I cannot remount the filesystem (I get the error: mount: / is busy) as read-only again.

Used command:mount -o remount,ro /dev/mapper/sda9_crypt /

I have searched for processes and file-locks (with lsof/fuser), but I have no idea what I have to specifically search for.

I do not want to reset the system after an update, just to be able to get it remounted read-only.
Therefore, I look for the key processes that I need to kill to make this possible.

Best Answer

In the output of lsof /, in addition to files opened for writing (w suffix in the FD column), look for files marked (deleted). Files outside /var, /tmp (and /run, etc.) and home directories usually don't remain open for writing for a long time. But after an upgrade of some software, there may be some executable, library or data file that's still in use (open for reading or executing) by a running process that was started before the upgrade.

When a file is deleted but still open, that actually only removes its directory entry. The file itself is only deleted when it has no directory entry and it isn't open. That last part of deletion can't happen on a filesystem that's mounted read-only, so such a half-deleted file prevents remounting the filesystem as read-only just like a file open for writing does.

Related Solutions

Linux Filesystems – How to Remount a Busy Disk to Read-Only Mode

You normally can't remount a filesystem as read-only if processes have a file on it that's open for writing, or if it contains a file that's deleted but still open. Similarly, you can't unmount a filesystem that has any file open (or similar uses of files such as a process having its current directory there, a running executable, etc.).

You can use umount -l to release the mount point and prevent the opening of further files, but keep the filesystem mounted and keep processes that already have files open running normally.

I can't think of a generic way to force a filesystem to be remounted read-only when it shouldn't be. However, if the filesystem is backed by a block device, you can make the block device read-only, e.g.

echo 1 >/sys/block/dm-4/ro
echo 1 >/sys/block/sda/sda2/ro

echo u > /proc/sysrq-trigger is a rather extreme way to force remounting as read-only, because it affects all filesystems. It's meant as a last-ditch method to leave the filesystem in a clean state just before rebooting.

Remounting a filesystem as read-only does not cause a reboot. Whatever is causing the reboot is not directly related to remounting the partition as read-only. Maybe it's completely unrelated, or maybe this triggers a bug in the application which causes it to spin and make the processor overheat and your processor is defective or overclocked and eventually reboots. You need to track down the cause of the reboot.

Linux – how to reliably determine processes using files after `umount -l` on linux

Because of the way the mount point gets hidden with umount -l, there is no way to find which processes are still using affected files.

The only way to get the list is to use lsof before umount -l to grep the relevant path. Example: lsof | grep "/mountPoint/".

If you want, you can take that output to extract the PIDs and continue monitoring them.

Best Answer

Related Solutions

Linux Filesystems – How to Remount a Busy Disk to Read-Only Mode

Linux – how to reliably determine processes using files after `umount -l` on linux

Related Question