Fix Broken /lib/modules/ and APT Issues

aptbootdiskupdates

I have a machine that went neglected for some time, and ended up with a full disk and full boot partition .. At that point any apt command would not work, because no space.

Before we start, this server runs a minimal MySQL DB — and /var/log/ was at 14MB and /var/mail/ was at 18kb. I also tried to run apt clean and apt autoclean — Disk was still full .. After Googling and Googling .. And EVERY post saying just run apt clean (of course) I decided to attempt to take matters into my own hand and I FAFO.

So I ran uname -r and it cam out

4.4.0-119-generic

So I went into the boot drive and removed anything not 4.4.0-119-generic. This has always been what I would try in the past and it usually clears up enough for me to run a apt install -f and apt autoremove then update-grub — Usually all is gtg after that.

My issue is that there was not enough space on the hard disk to run any apt install or apt autoremove — So I did the same in /usr/src/ and in lib/modules/

RECAP I cleared the usr/src and lib/modules of "unused" kernels.

I believe this is where I messed up. Now when I try to run apt install -f I get

Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 158 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Setting up linux-image-4.4.0-208-generic (4.4.0-208.240) ...
depmod: ERROR: could not open directory /lib/modules/4.4.0-208-generic: No such file or directory
depmod: FATAL: could not search modules: No such file or directory
dpkg: error processing package linux-image-4.4.0-208-generic (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 linux-image-4.4.0-208-generic
E: Sub-process /usr/bin/dpkg returned an error code (1)

I see the source of the error (which alerted me to my mistake)

depmod: ERROR: could not open directory /lib/modules/4.4.0-208-generic: No such file or directory

I tried to run sudo apt --fix-broken install as well — Same error — No avail.

What can I do to fix this seemingly monstrous conundrum I have cornered myself into?

Best Answer

Give the package manager what it wants for the purges: If it complains about a missing file, use touch and create the file, make missing directories. Package manager is too stupid to realize missing files should be OK when you're trying to remove them anyway, but it really doesn't care if they are empty substitutes.

Safely Removing Old Kernels

For users of LVM systems, encrypted systems or limited-storage systems, the most frequent problem is that the /boot partition is simply full. The package manager cannot install a pending upgrade due to lack of space. Besides, apt-get can not remove a package due to broken dependency.

This problem can be fixed quickly and easily from the shell. Simply identify one or two old kernels to remove manually, which will provide the package manager enough space to install the queued upgrade.


$ sudo rm -rv ${TMPDIR:-/var/tmp}/mkinitramfs-*  
                                  ## In Ubuntu 16.04 and earlier there may be leftover temporary
                                  ## files to delete.
                                  ## See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814345

$ uname -r                        ## This command identifies the currently-running kernel
4.2.0-21-generic                  ## This is the current kernel.
                                  ## DO NOT REMOVE it!

$ dpkg -l | tail -n +6 | grep -E 'linux-image-[0-9]+' | grep -Fv $(uname -r)
                                  ## This command lists all the kernels excluding the booted
                                  ## kernel in the package database, and their status.
rc  linux-image-4.2.0-14-generic  ## The oldest kernel in the database
                                  ## Status 'rc' means it's already been removed
ii  linux-image-4.2.0-15-generic  ## The oldest installed kernel. Eligible for removal.
                                  ## Status 'ii' means Installed.
ii  linux-image-4.2.0-16-generic  ## Another old installed kernel. Eligible for removal
ii  linux-image-4.2.0-18-generic  ## Another old installed kernel. Eligible for removal
ii  linux-image-4.2.0-19-generic  ## The previous good kernel. Keep
iU  linux-image-4.2.0-22-generic  ## DO NOT REMOVE. Status 'iU' means it's not installed,
                                  ## but queued for install in apt.
                                  ## This is the package we want apt to install.

                                  ## Purge the oldest kernel package using dpkg instead of apt.
                                  ## First you need to remove the image initrd.img file manually
                                  ## due to Bug #1678187.
$ sudo update-initramfs -d -k 4.2.0-15-generic
$ sudo dpkg --purge linux-image-4.2.0-15-generic linux-image-extra-4.2.0-15-generic
                                  ## If the previous command fails, some installed package
                                  ## depends on the kernel. The output of dpkg tells the name
                                  ## of the package. Purge it first.

                                  ## Also purge the respective header package.
$ sudo dpkg --purge linux-headers-4.2.0-15-generic
                                  ## Try also purging the common header package.
$ sudo dpkg --purge linux-headers-4.2.0-15
                                  ## Do not worry, if the previous command fails.

$ sudo apt-get -f install         ## Try to fix the broken dependency.

I followed this with:

sudo apt-get autoremove --purge

Best Answer

Related Solutions

Ubuntu 12.04 – Fixing Apt Errors After Upgrade

Fix False Disk Full Error – Apt-Get Unable to Install or Remove

Safely Removing Old Kernels

Related Question