Debian – How to increase the size of rootfs

debianfilesystemslinux

So I got this error today and decided to investigate:

/etc/init.d/mysql: ERROR: The partition with /var/lib/mysql is too full! ... failed!

I'm not sure how… but I've filled up my rootfs

root@crunchbang:/var/lib/mysql# df -h
Filesystem                                              Size  Used Avail Use% Mounted on
rootfs                                                  9.2G  8.9G     0 100% /
udev                                                     10M     0   10M   0% /dev
tmpfs                                                   397M  648K  396M   1% /run
/dev/disk/by-uuid/84406341-a86f-444a-a89c-0b3f37a6e0a3  9.2G  8.9G     0 100% /
tmpfs                                                   5.0M     0  5.0M   0% /run/lock
tmpfs                                                   2.4G  2.3M  2.4G   1% /run/shm
/dev/sda6                                               277G   19G  244G   8% /home
tmpfs                                                   2.4G  324K  2.4G   1% /tmp

Most of the space seems to be taken up by /var/log

root@crunchbang:/var/lib/mysql# du -hx --max-depth=1 /
0   /run
3.6G    /usr
7.4M    /bin
1.7G    /opt
12M /sbin
4.0K    /mnt
0   /sys
4.0K    /selinux
3.3G    /var
146M    /lib
4.0K    /.pulse
16K /lost+found
8.8M    /etc
0   /proc
4.0K    /srv
12K /media
57M /root
4.0K    /lib64
4.0K    /home
18M /boot
0   /dev
0   /tmp
8.7G    /

Here is my /var/log

root@crunchbang:/var/log# ls -al
total 3108
drwxr-xr-x 17 root        root      4096 Jun  2 12:59 .
drwxr-xr-x 12 root        root      4096 Mar 22 17:42 ..
-rw-r--r--  1 root        root       935 Jun  2 19:14 alternatives.log
-rw-r--r--  1 root        root       310 May 30 22:20 alternatives.log.1
-rw-r--r--  1 root        root       982 Apr 16 01:54 alternatives.log.2.gz
-rw-r--r--  1 root        root      4847 Mar 24 16:28 alternatives.log.3.gz
drwxr-x---  2 root        adm       4096 Jun  2 07:35 apache2
drwxr-xr-x  2 root        root      4096 Jun  1 07:35 apt
-rw-r--r--  1 root        root         0 Apr  1 07:35 aptitude
-rw-r--r--  1 root        root       389 Mar 29 08:02 aptitude.1.gz
-rw-r-----  1 root        adm      15367 Jun  2 19:21 auth.log
-rw-r-----  1 root        adm     120508 Jun  2 07:30 auth.log.1
-rw-r-----  1 root        adm       5329 May 26 07:30 auth.log.2.gz
-rw-r-----  1 root        adm       6070 May 19 07:30 auth.log.3.gz
-rw-r-----  1 root        adm       7739 May 12 07:30 auth.log.4.gz
-rw-r--r--  1 root        root     41044 Jan 19 10:35 bootstrap.log
-rw-rw----  1 root        utmp         0 Jun  1 07:35 btmp
-rw-rw----  1 root        utmp         0 May  1 07:35 btmp.1
drwxr-xr-x  2 root        root      4096 Jun  1 07:35 ConsoleKit
-rw-r-----  1 root        adm      77597 Jun  2 19:26 daemon.log
-rw-r-----  1 root        adm     181164 Jun  2 03:51 daemon.log.1
-rw-r-----  1 root        adm       6464 May 26 00:46 daemon.log.2.gz
-rw-r-----  1 root        adm       4139 May 19 03:21 daemon.log.3.gz
-rw-r-----  1 root        adm      20493 May 12 06:46 daemon.log.4.gz
drwxr-xr-x  2 root        root      4096 Mar 22 17:42 dbconfig-common
-rw-r-----  1 root        adm      56560 Jun  2 12:59 debug
-rw-r-----  1 root        adm     119093 May 31 15:17 debug.1
-rw-r-----  1 root        adm       5038 May 25 13:46 debug.2.gz
-rw-r-----  1 root        adm        482 May 15 07:43 debug.3.gz
-rw-r-----  1 root        adm      14620 May 11 19:26 debug.4.gz
-rw-r-----  1 root        adm      53357 Jun  2 12:59 dmesg
-rw-r-----  1 root        adm      53776 Jun  2 12:55 dmesg.0
-rw-r-----  1 root        adm      14006 May 31 15:16 dmesg.1.gz
-rw-r-----  1 root        adm      13999 May 31 15:15 dmesg.2.gz
-rw-r-----  1 root        adm      14236 May 31 15:14 dmesg.3.gz
-rw-r-----  1 root        adm      14384 May 30 22:06 dmesg.4.gz
-rw-r--r--  1 root        root    110430 Jun  2 19:22 dpkg.log
-rw-r--r--  1 root        root     64780 May 30 22:21 dpkg.log.1
-rw-r--r--  1 root        root      5205 Apr 23 15:51 dpkg.log.2.gz
-rw-r--r--  1 root        root     90165 Mar 29 08:03 dpkg.log.3.gz
drwxr-s---  2 Debian-exim adm       4096 Jun  2 07:35 exim4
-rw-r--r--  1 root        root     32064 May 27 21:48 faillog
-rw-r--r--  1 root        root      2726 May 30 22:20 fontconfig.log
drwxr-xr-x  2 root        root      4096 Mar 20 22:31 fsck
drwxr-xr-x  3 root        root      4096 Mar 20 22:34 installer
-rw-r-----  1 root        adm     175398 Jun  2 19:14 kern.log
-rw-r-----  1 root        adm     356143 Jun  1 22:08 kern.log.1
-rw-r-----  1 root        adm      17875 May 25 13:46 kern.log.2.gz
-rw-r-----  1 root        adm       2101 May 18 21:48 kern.log.3.gz
-rw-r-----  1 root        adm      51111 May 11 19:26 kern.log.4.gz
-rw-rw-r--  1 root        utmp    292584 May 27 21:48 lastlog
-rw-r-----  1 root        adm          0 Mar 20 22:36 lpr.log
-rw-r-----  1 root        adm        285 Jun  2 07:35 mail.err
-rw-r-----  1 root        adm        283 Jun  1 07:35 mail.err.1
-rw-r-----  1 root        adm        179 May 31 07:35 mail.err.2.gz
-rw-r-----  1 root        adm        285 Jun  2 07:35 mail.info
-rw-r-----  1 root        adm        283 Jun  1 07:35 mail.info.1
-rw-r-----  1 root        adm        179 May 31 07:35 mail.info.2.gz
-rw-r-----  1 root        adm        285 Jun  2 07:35 mail.log
-rw-r-----  1 root        adm        283 Jun  1 07:35 mail.log.1
-rw-r-----  1 root        adm        179 May 31 07:35 mail.log.2.gz
-rw-r-----  1 root        adm        285 Jun  2 07:35 mail.warn
-rw-r-----  1 root        adm        283 Jun  1 07:35 mail.warn.1
-rw-r-----  1 root        adm        179 May 31 07:35 mail.warn.2.gz
-rw-r-----  1 root        adm     120680 Jun  2 19:14 messages
-rw-r-----  1 root        adm     238251 Jun  2 07:35 messages.1
-rw-r-----  1 root        adm      13398 May 26 07:35 messages.2.gz
-rw-r-----  1 root        adm       1947 May 19 07:35 messages.3.gz
-rw-r-----  1 root        adm      37332 May 12 07:35 messages.4.gz
drwxr-xr-x  2 mongodb     mongodb   4096 Apr  4 15:38 mongodb
drwxr-s---  2 mysql       adm       4096 Mar 22 17:43 mysql
-rw-r-----  1 mysql       adm          0 Jun  2 19:22 mysql.err
-rw-r-----  1 mysql       adm          0 Jun  2 19:22 mysql.log
-rw-r-----  1 mysql       adm         20 Jun  1 07:35 mysql.log.1.gz
-rw-r-----  1 mysql       adm         20 May 31 07:35 mysql.log.2.gz
-rw-r-----  1 mysql       adm         20 May 30 07:35 mysql.log.3.gz
-rw-r-----  1 mysql       adm         20 May 29 07:35 mysql.log.4.gz
-rw-r-----  1 mysql       adm         20 May 28 07:35 mysql.log.5.gz
-rw-r-----  1 mysql       adm         20 May 27 07:35 mysql.log.6.gz
-rw-r-----  1 mysql       adm         20 May 26 07:35 mysql.log.7.gz
drwxr-xr-x  2 root        root      4096 Mar 20 22:36 news
drwxr-xr-x  2 ntp         ntp       4096 May 12  2012 ntpstats
-rw-r--r--  1 root        root      1865 Jun  2 12:59 pm-powersave.log
-rw-r--r--  1 root        root     11190 May 31 15:17 pm-powersave.log.1
-rw-r--r--  1 root        root       571 Apr 30 20:33 pm-powersave.log.2.gz
-rw-r--r--  1 root        root       484 Mar 29 16:12 pm-powersave.log.3.gz
drwxr-sr-x  2 proxy       adm       4096 Jun  2 07:35 polipo
-rw-r--r--  1 root        root         0 Apr  8 12:25 pycentral.log
drwxr-x---  2 root        adm       4096 Jan  5 14:18 samba
-rw-r--r--  1 root        root       122 Jun  2 12:59 slim.log
-rw-r--r--  1 root        root       183 Jun  2 07:35 slim.log.1
-rw-r--r--  1 root        root        61 May 31 07:35 slim.log.2
-rw-r--r--  1 root        root        61 May 24 18:33 slim.log.3
-rw-r--r--  1 root        root       183 May 12 07:35 slim.log.4
-rw-r-----  1 root        adm     267596 Jun  2 19:26 syslog
-rw-r-----  1 root        adm      38816 Jun  2 07:35 syslog.1
-rw-r-----  1 root        adm      60952 Jun  1 07:35 syslog.2.gz
-rw-r-----  1 root        adm      23526 May 31 07:35 syslog.3.gz
-rw-r-----  1 root        adm       1881 May 30 07:35 syslog.4.gz
-rw-r-----  1 root        adm       1789 May 29 07:35 syslog.5.gz
-rw-r-----  1 root        adm       3166 May 28 07:35 syslog.6.gz
-rw-r-----  1 root        adm       3400 May 27 07:35 syslog.7.gz
drwxr-s---  2 debian-tor  adm       4096 May 31 07:35 tor
drwxr-xr-x  2 root        root      4096 Apr 12 19:19 unattended-upgrades
-rw-r-----  1 root        adm       4773 Jun  2 19:14 user.log
-rw-r-----  1 root        adm       7094 May 31 15:17 user.log.1
-rw-r-----  1 root        adm        107 May 24 18:29 user.log.2.gz
-rw-r-----  1 root        adm        296 May 17 07:35 user.log.3.gz
-rw-r-----  1 root        adm        451 May 11 19:26 user.log.4.gz
-rw-rw-r--  1 root        utmp     20352 Jun  2 18:59 wtmp
-rw-rw-r--  1 root        utmp    122880 May 31 23:56 wtmp.1
-rw-r--r--  1 root        root     44007 Jun  2 12:59 Xorg.0.log
-rw-r--r--  1 root        root     31638 Jun  2 12:55 Xorg.0.log.old

So essentially:

  • What files can I purge from /var/log? Will it lead to any errors?
  • Better yet: Is there a way to make my rootfs any bigger?

Best Answer

In /var/log, you can delete everything that ends with .1 or .2.gz (or 3, or 4, or whatever). Those files are not currently being written to.

However, I may be misreading your output but I don't see how you're sure that it's /var/log that's the problem - I would have guessed it to be the mysql directory.

Instead of increasing your rootfs, I'd advise you to move /var to its own disk. You have a huge /home; you could shrink it a bit and use some of it for /var, or you could just create a directory on that disk to be used for var and then mount it:

mv /var /oldvar
mount --bind /home/vardisk /var 
cd /oldvar
tar cf - . |(cd /var; tar xpf -)

Note that this command needs to be run when the server is booted in single-user state, since /var will be unusable while you're doing this.

Related Question