How Can a Log Program Continue to Log to a Deleted File?

file-descriptorsfileslogsopen files

From the Unix Power Tools, 3rd Edition: Instead of Removing a File, Empty It section:

If an active process has the file open (not uncommon for log files),
removing the file and creating a new one will not affect the logging
program; those messages will just keep going to the file that’s no
longer linked
. Emptying the file doesn’t break the association, and so
it clears the file without affecting the logging program.

(emphasis mine)

I don't understand why a program will continue to log to a deleted file. Is it because the file descriptor entry not getting removed from the process table?

Best Answer

When you delete a file you really remove a link to the file (to the inode). If someone already has that file open, they get to keep the file descriptor they have. The file remains on disk, taking up space, and can be written to and read from if you have access to it.

The unlink function is defined with this behaviour by POSIX:

When the file's link count becomes 0 and no process has the file open, the space occupied by the file shall be freed and the file shall no longer be accessible. If one or more processes have the file open when the last link is removed, the link shall be removed before unlink() returns, but the removal of the file contents shall be postponed until all references to the file are closed.

This piece of advice because of that behaviour. The daemon will have the file open, and won't notice that it has been deleted (unless it was monitoring it specifically, which is uncommon). It will keep blithely writing to the existing file descriptor it has: you'll keep taking up (more) space on disk, but you won't be able to see any of the messages it writes, so you're really in the worst of both worlds. If you truncate the file to zero length instead then the space is freed up immediately, and any new messages will be appended at the new end of the file where you can see them.

Eventually, when the daemon terminates or closes the file, the space will be freed up. Nobody new can open the file in the mean time (other than through system-specific reflective interfaces like Linux's /proc/x/fd/...). It's also guaranteed that:

If the link count of the file is 0, when all file descriptors associated with the file are closed, the space occupied by the file shall be freed and the file shall no longer be accessible.

So you don't lose your disk space permanently, but you don't gain anything by deleting the file and you lose access to new messages.

Related Question