Ubuntu – Partition is reported to be full, but I cannot “see” any culprit files

11.04filesystem

So here's the deal:

—df reports 100% usage on the partition hosting my /home directories (and only those)

–Tools that use du (a la baobab or xdiskusage) report that the excessive space consumption is clearly related to my user.

–Tools that use du do not, however, seem to be able to locate any files that are the culprit. For example, xdiskusage shows my profile taking up > 260GB but only about half that amount of data actually in files (the rest is shown as just empty space).

–Running du on CLI just shows what I've described above: my user's ~/ folder is consuming like twice as much disk space than it actually has files inside of it.

And my favorite:

–This appears to be correlated with uptime, as though some process is slowly filling the partition with data until the drive is full. And when I reboot — pooof! tons of free space. Is there an introspection tool that'd let me ascertain, e.g., what process is writing to disk at any given time?

Any suggestions?

Best Answer

I don't know what is causing the behavior you're observing, but I can imagine how it is happening:

Due to the way Unix filesystem works, it is possible for an application to open a file, then unlink it (i.e. delete the file name entry from its parent directory) - the file handle still remains valid though there is no file name associated with it, and the application is still able to write to the file, because the handle is pointing to the inode, not to the filename.

Such a file exists until the application closes it. My understanding is that this is a common way to create a "private" temp file for an application.

To find such files, use

lsof +L

which will show you the number of links to each open file (NLINK column) - the unlinked files have 0 there. Here's how an unlinked file looks:

COMMAND     PID       USER   FD      TYPE             DEVICE SIZE/OFF NLINK       NODE NAME
...
mysqld     1126      mysql   11u      REG                8,1        0     0   33603687 /tmp/ibMlwWQA (deleted)
Related Question