Today, for the first time I encountered a strange behaviour on a machine with Ubuntu 15.10 . The problem itself was quite strange and rendered the machine unusable with default boot settings.

The problem

kernel panic-not syncing: VFS: unable to mount root fs on unknown block(0,0)

As I went to power up the machine this was the message when booting with default options. The user operating this machine had told me that he had performed an update prior to this problem. So I initially thought it was some kind of incompatibility with a newer kernel. So I went and use the precedent kernel, but nothing. And here the strange thing, I went and boot the third kernel in the list and boom! It booted! Now I decided to take a side trip to the solution and issued:

grub-mkconfig -o /boot/grub/grub.cfg

Everything went fine, however, I noticed the initrd was missing for some entries in the list. Then I tried listing files inside the boot partition and verified what I suspected: the initrd image for the last two kernels was nowhere to be found in the boot partition. That was the problem.

After that I tried to generate the initrd images for the last two kernels, and here I found the real problem: boot partition was full. This machine had been operating for the past three months without maintenance except for the manual updates performed by the user and the boot partition size was set to 500M (a reasonable size).

Now I didn’t even know this could happen since I’m used to Red Hat systems… YUM (and probably DNF, I’m not sure) limits the number of kernels to keep, on top of that the number is tweak-able by easily modifying a configuration file.

The solution

I manually removed all the kernels

sudo apt-get remove linux-image-VERSION

that had been installed over the past months and ran a

sudo apt-get autoremove

to further clean up. However I got astonished that such a bottleneck for beginners is there sitting inside Ubuntu and wondered why APT didn’t have such a useful feature (even for servers). So I started searching and found this article which explain a way that does exactly what I was looking for. The solution is synthesized as a couple of concatenated commands:

dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get -y purge

If you are curious about what this command does you can read this great article.

Be careful: Thanks to /u/elmicha, the commands above will also remove linux-libc-dev and can cause potential issues. You might want to omit the last part of the command | xargs sudo apt-get -y purge in order to see which packages will be removed.

Another solution I’ve found is explained in detail in the Lubuntu documentation.

Conclusion

If you don’t pay attention to cleaning the old kernels, your machine will not boot the next time you boot in case the boot partition is full. APT won’t automatically remove the older kernels, instead it will keep stacking them. If you’re running only a machine then, cleaning manually from time to time is not a problem; if you are running multiple servers it might come in handy adding that little script to cron (if you’re not using a centralized management tool of course). Another useful solution I found is this answer at AskUbuntu.

Image courtesy of Maicon Fonseca Zanco.