How to determine what process is eating up all available disk space

disk-usage

Suddenly all the available disk space on / has disappeared.

If I make room in the disk (by deleting ~50GB of stuff, for example), after a few minutes I am back to 0 available disk space (according to df).

Clearly, some process is eating up disk space at a rapid rate, but I can't figure out what it is.

One thing is certain, though: whatever it is, it must be creating many small files, because there are no files bigger than 10GB on the disk, and all the ones bigger than 1GB are much older than today.

How can I find what's eating up disk space?


FWIW, only df sees the problem, not du.

For example, below I show several "snapshots" from du and df taken 60s. apart. (I did this after I had made some room in the disk.) Notice how du's output remains steady (at 495G), but df shows a steadily shrinking amount of available space. (I've followed the recommendation given here. IOW, /mnt/root is pointing to /.)

# while true; do du -sh /mnt/root && df -h /mnt/root; sleep 60; done
495G    /mnt/root
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1       880G  824G   12G  99% /mnt/root
495G    /mnt/root
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1       880G  825G   11G  99% /mnt/root
495G    /mnt/root
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1       880G  827G  8.9G  99% /mnt/root
495G    /mnt/root
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1       880G  827G  8.1G 100% /mnt/root
495G    /mnt/root
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1       880G  828G  7.5G 100% /mnt/root

Best Answer

You are dealing with deleted files, that is why du does not register used space, but dfdoes.

Deleted files only disappear after the owner process is stopped; they remain in use while that does not happen.

So to find the culprit process, I recommend you doing:

sudo lsof -nP | grep '(deleted)'
Related Question