Ubuntu – My root partition keeps filling to 100% capacity, but I can’t figure out why!

acpidisk-usage

Please see Edit #3 if you're looking for a solution.

I'm running Xubuntu 16.04 LTS, and I have a ~40 GB root partition which is 100% full according to System Monitor (as root). It's definitely full, since many programs aren't functioning correctly.

However, for some reason, I can't seem to figure out what is using the space! Baobab (as root) only reports a total of 15.5 GB used on my root partition!

EDIT: Also, here's /var – people said that it's big. Baobab only reports 1 GB for /var, and /var/log is empty. I've tried running sudo rm -R /var/log and there was no effect.

So, how do I find out what is using my disk space, and how do I prevent it from filling up my root partition? This is a huge problem, please help! Thank you in advance 🙂

EDIT 2: As posted in the answer section, sudo lsof / | awk '{if(\$7 > 1048576) print \$7/1048576 \"MB\" \" \" \$9 }' | sort -n -u returns 11222.7MB /var/log/kern.log 11222.9MB /var/log/syslog, however, I can't seem to figure out how to delete these files, and additionally, I would like to figure out how I can permanently prevent these files from growing this large. This answer to another question suggested that I look into the logs and see what's filling them up, so ideally I'd like some way to read the contents of these mystery files.

EDIT 3: I have temporarily fixed this issue by mounting /var/log on a separate partition.

However, there is still some kind of bug that's causing this. Please, if you want this bug to be fixed, please bring information (or at least give attention) to the bug reports: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1643719 and https://bugzilla.kernel.org/show_bug.cgi?id=188331 Thank you 🙂

Best Answer

There are two types of file access that use disk space, but don't show up with your tools: deleted (but still open) files, and files being written to.

I have these two aliases defined that I find very useful:

# from http://www.certpal.com/blogs/2010/12/find-open-files-in-linux-using-lsof/
alias bigopenfiles="sudo lsof / | awk '{if(\$7 > 1048576) print \$7/1048576 \"MB\" \" \" \$9 }' | sort -n -u" 

alias deletedfiles="sudo lsof / | egrep 'PID|\(deleted\)'"

Related Solutions

Ubuntu – Root filling up although it should have some space

A first shot: High traffic in an error log. I guess if you would download videos, you would know it. :)

 sudo du -sh /var/log

-s is summary
-h is human readable (k, M, G, T) suffix

For me it is 20M, and beside the log-directory, there is a cache for the installations. Use

 apt-get clean

If this is your problem.

Of course, you should check /home if it is in your root partition.

If you have to search big files iteratively, I can suggest a series of commands, which can be repeated and is pretty fast after the first initial step:

You start at a suspicious directory at top level
Then you search all subdirs and files for their size, and sort numerically:

sudo du -s /var/* | sort -n 
0   /var/crash
4   /var/www
44  /var/games
124 /var/run
2306    /var/tmp
18538   /var/log
251876  /var/cache
1053231 /var/lib

I stripped the example, to keep it short. So we see, that the bigest subdir is /var/lib, and repeat the step for /var/lib. Now the search is much faster, because the computated results are somehow cached:

sudo du -s /var/lib/* | sort -n 
78116   /var/lib/apt-xapian-index
104580  /var/lib/dpkg
680503  /var/lib/postgresql

and so on. Follow the biggest dirs to find the bigger files. You can't use -sh here, because sorting numerically does not work with the k/M/G/T for kilobyte and so on.

Ubuntu – Who is filling the disk

The problem with your cronjob is that .xsession_errors is most likely still opened by some applications or system services which is why it will be hidden from the filesystem table when deleted, but it still is on disk and still errors will be written to it.
So it will fill up the disk, but now you can't see it anymore.

@rinzwind aims for exactly this behaviour when he (correctly) suggests to remove the cronjob and look for the errors. This is the only way to fix this problem properly.

As a workaround you could truncate the .xsession_errors file with a cronjob like this:

17 */2 * * *  truncate -cs 0 path/to/.xsession_errors

But before doing this you REALLY should try to fix the underlying problems that create those error messages in .xsession_errors

Best Answer

Related Solutions

Ubuntu – Root filling up although it should have some space

Ubuntu – Who is filling the disk

Related Question