The future of the boot system in Debian

To: debian-devel-announce@lists.debian.org

Subject: The future of the boot system in Debian

From: Petter Reinholdtsen <pere@hungry.com>

Date: Sat, 05 Sep 2009 13:21:00 +0200

Message-id: <[🔎] 2fl1vmlwsbn.fsf@login2.uio.no>

Mail-followup-to: debian-devel@lists.debian.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The future of the boot system in Debian ======================================= Over the last few years, the boot system in Debian has progressively deteriorated due to changes in the Linux kernel which make the kernel more and more event based. For example, the kernel and its drivers no longer block all processing while detecting disks, network interfaces and other hardware, making the once trusty old boot system in Debian increasingly fragile. During the current boot sequence device files in /dev/ are often missing when fsck or mount are looking for them, or the network is not available when the boot system tries to mount NFS entries because network interfaces were slow to initialise, or audio devices are missing when audio settings should be set. The problem is fundamental to the way we boot Debian today - sequentially, and a solution needs to address this fundamental problem. We believe the solution is to migrate to an event based boot system. In addition to this, there are long lasting problems with the boot sequence of the existing init.d scripts, for some combination of packages. The boot sequence is wrong in these cases, and to solve it one needs to change the sequence numbers of all the immediate forward and reverse dependencies of the init.d script in question - and their forward and reverse dependencies and so forth until the boot sequence is correct. In some cases the change needs to happen to several scripts in different packages at the same time, which is impossible with the old way of ordering init.d scripts. Previously the ordering was done by asking the package maintainers to guess on and update sequence numbers, a process that tended to introduce new problems and took a long time to be solved properly. The solution to this problem is to change how we order boot scripts. Change it from static sequence numbers to calculate the boot sequence using dependency information provided in the init.d scripts themselves. Since 2009-07-27 this is the default in Debian unstable, and it will be the way init.d scripts are ordered in Squeeze. Switching to a dependency based boot sequencing allows us to ensure its correctness, detect and fix dependency loops, and in general fix a set of bugs in the distribution that have been very hard to fix before. Other solved problems with the new system are incorrect stop sequences (the default should have been S20/K80, not S20/K20), and the misleading runlevels 0 and 6, where start symlinks are called with 'stop' as the argument to the scripts. All of these problems are solved when Debian now moves to dependency based boot sequencing. It will take longer to solve the fundamental problem. It requires a rewrite of how we handle the boot, and can not be done by just modifying the boot sequence framework. Before explaining the current plan, some background information. The current boot system can be seen as consisting of three different parts: 1. The implementation of /sbin/init, reading /etc/inittab and starting the script implementing part 2 (/etc/init.d/rc). This is normally done using the sysvinit package, but other replacements are available, like initng and upstart. 2. The implementation of /etc/init.d/rc, which is responsible for calling the init.d scripts in the correct sequence. This is normally done using the sysv-rc package. An alternative is the file-rc package, which uses a file /etc/runlevel.conf instead of symlinks in /etc/rc?.d/ to decide what to execute and in which order. 3. The individual init.d scripts, taking care of the tasks that need to be done during boot. The basic framework is provided by the initscripts package, and the rest is handled by individual packages like udev, netbase, ifupdown, apache, etc. :) There are approximately 850 packages with init.d scripts in Debian unstable. Part 2 (sysv-rc) and 3 have been changed to use dependency based boot sequencing. This was a release goal for Lenny and the continued work is a release goal for Squeeze. The init.d scripts have seen review with regard to dependency based ordering for more than 3 years. New installs will use dependency based boot sequencing. For upgrades, a critical debconf question will give the default option to migrate if testing find no issues with the existing scripts or for now keep the legacy boot ordering. To solve the fundamental problem, the plan is to replace /sbin/init with an implementation that is able to handle kernel events. It will allow us to modify the boot system for the early boot to become event based, while keeping the existing boot stuff working. We could rewrite sysvinit to become event based, or have a look at the existing boot systems that handle kernel events. After checking the options and the systems used in other distributions, upstart seems like the most promising candidate. It is used by Ubuntu and Fedora at the moment, and solves the problem in a backwards compatible way. The plan is to change upstart to actually use /etc/inittab, to ease the switch between sysvinit and upstart. We will also change the init.d script handling to treat upstart jobs as init.d scripts, to provide an alternative for architectures lacking upstart support. These changes should make it transparent for the users which package provides /sbin/init, and thus make it easier to migrate from sysvinit to upstart. When /sbin/init is changed to an event based framework, the next step is to rewrite the early boot system to use these events when available, and behave the traditional way when there are no events. When this step is finished the fundamental problem will be solved, and the boot will be robust and should work correctly even in edge cases with slow device buses. The planned time frame for this is to replace /sbin/init with upstart for Squeeze, and see if we manage to change the very early boot to become event based in time for Squeeze too, fixing the most pressing of the current boot problems (failing fsck and mount with USB disks). For Squeeze+1, more of the early boot system will be converted, to handle more of the existing problems. According to the Linux Software Base specification, all LSB compliant distributions must handle packages with init.d scripts. As Debian plans to continue to follow the LSB, this mean the boot system needs to continue to handle init.d scripts. Because of this, we need a boot system in debian that is both event based for the early boot, and which also calls init.d scripts at the appropriate time. References: http://wiki.debian.org/LSBInitScripts/DependencyBasedBoot Petter Reinholdtsen, Kel Modderman, Armin Berres -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFKoklo20zMSyow1ykRAkvfAJ4+kofKHP+sYOQndZlBhe/mC7nTvQCdE8af YZEQ67miKPOCeMbC0Ephx1w= =bA+k -----END PGP SIGNATURE-----