Yesterday, I did a update
/dist-upgrade
. Today, I powered on the machine and it was hanging at the loading screen with logo and cycling dots – I've waited at this screen for about an hour several times with no results. If I interrupt upstart
with ctrl-alt-del, bootup resumes/completes, but it puts me at tty console login. X
does start, a few seconds later, but a dialog about graphics being configured incorrectly is immediately brought up. Update: The X issue was resolved by doing apt-get install nvidia-current
. Interrupt issue still stands.
Unfortunately, every lead I've found as to why this might be happening has turned into a dead end. Here's my boot.log
(from /var/log
) showing where I interrupted the startup. You can see it hangs just as it starts "enable remaining boot-time encrypted block devices" (this is from cryptdisks
), but removing that service makes no difference. I've tried pretty much everything from this Mint bug report, which describes symptoms nearly identical to mine, to no avail. At this point, I'm fairly sure that cryptdisks
is a red herring, and that it's something else entirely.
I've also found that resuming startup from recovery mode seems to load things in a different order. Upstart
still hangs, but not after cryptdisks
. If I ctrl-alt-del, it brings me to the graphical login manager instead of a tty, and I can login successfully. However, the system still isn't fully functional; USB plug and play seems not to work, I can't use my second monitor, and I have to manually do start resolvconf
to access the internet. Here's the boot.log from one of those startups.
I should add that I am encrypting my HDD with LUKS
, and the hang happens after I successfully enter the decryption password. Here's my fstab
, in case it has anything to do with things:
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/mapper/ubuntu--vg-root / ext4 errors=remount-ro 0 1
# /boot was on /dev/sda1 during installation
UUID=9e7c1e90-f3e4-4075-b3b0-e3ccb6d933c7 /boot ext2 defaults 0 2
/dev/mapper/ubuntu--vg-swap_1 none swap sw 0 0
What's going on here?
Best Answer
The root cause was a huge number of files in my
/tmp
directory.I'd used the
/tmp
directory to store millions of files earlier. It turns out that having that many files there causes the service which cleans/tmp
to take a long, long time (duh). After moving the files out of/tmp
, the problem is solved. It had nothing to do with the upgrade; that was just a coincidence.In case it helps anyone later, here's the process I used to figure it out. I enabled the "Magic SysRq key" by changing
etc/sysctl.d/10-magic-sysrq.conf
. Then, I reproduced the problem by rebooting; when startup hung, I hit Alt-SysRq-t. This dumped the following in the kernel buffer, read usingdmesg
:It dumps a lot more than this, but this is the relevant part. This shows the blocked task is
find
. After that, it was just a matter of a knowledgeable friend knowing that the/tmp
cleaning service was a likely culprit.