A history of modern init systems (1992-2015)

by VR

The subject of process management, supervision and init(8) for Unix-like systems is one plagued by a large degree of ahistoricity and “pop culture” explanations. This leads to a lot of confusion and misunderstanding surrounding feature sets and how one formulates the problems surrounding reliable process management on Unix in general, making it a ripe topic for demagogues of all persuasions.

Where GNU/Linux is concerned, the most common chronology given is that first there was sysvinit. It sucked for a long time, but for whatever reason no one did anything about it until Apple created launchd, which inspired Ubuntu to make (or depending on who you sympathize with, NIH) Upstart only to hamper it with a CLA. Then in 2010 systemd changes everything. Oh, and in the meantime, Gentoo did something called OpenRC or whatever.

Indeed, the timeline of events and solutions is poorly understood, which very often colors perceptions about just how novel existing solutions are. Most critically, without a good historical understanding, you are unable to learn from the mistakes of previous undertakings and perhaps end up unnecessarily making the same errors in your design.

The purpose of this article is to set the record straight on the history of attempts to create “modern” init systems, where we define “modern” somewhat broadly as anything that tries to improve the classical BSD and System V styles of initialization and service management.

This article is not meant to impart any technical judgments, but to simply document what has been done, so that people interested in systems software research and particularly Unix process supervision, can use it as a quick reference for prior art. Just as importantly, it is meant to inform more casual readers as to the history of solutions in this problem space and hopefully broaden their horizons about this controversial issue.

It will not necessarily go into deep technical analysis, but rather provide general overviews and links for further information.

(Before we begin, this article will not cover basic supervisors that cannot be adapted into init(8) daemons, such as supervisord, monit, Circus, God, bluepill and Eye. These are popular with web development crowds considering many of them are based on languages like Python or Ruby, but they are not init systems, and generally not novel, either.)

IBM System Resource Controller (1992)

Probably the first modern init system, written for IBM AIX. It was the first to insist on the now well known supervision semantics of not having services daemonize themselves, but instead have them be daemonized by the supervisor itself.

It was also the first to operate not just on daemons, but “subsystems” - which could be groups of daemons and auxiliary programs. Solaris SMF would reuse the same principle, 13 years later.

SRC was a fully contained framework, having tools for starting, stopping, restarting and obtaining status for subsystems.

SRC did not rely on scripting or even traditional service configuration, but instead relied on using commands to register subsystems and servers inside its own object database. Again, SMF would echo this by introducing a service configuration repository for storing runtime data and persistent configuration, though still retaining user-visible configuration files in the form of XML manifests.

daemontools (1997) + derivatives (1997-2015)

The Maxwell’s equations of Unix process management.

Initially released by Daniel J. Bernstein in 1997, it has since went on to become massively influential and often used to the present day. It has inspired several derivatives and directly influenced systems that have been more deviating from its model such as minit, ninit and depinit.

Virtually all init systems can and have been mapped as supersets of daemontools in one form or another (e.g. systemd => nosh).

See JdeBP’s “The daemontools family”, Wayne Marshall’s daemontools section in “the djb way”and Bernstein’s own page for far better overviews than I could ever give.

rc.d (2000)

rc.d is a modular, dependency-based initscript framework initially adopted by NetBSD circa 2000, primarily designed by Luke Mewburn. It has since spread to the other BSDs, replacing the old, completely flat /etc/rc.

In rc.d, the init(8) daemon executes /etc/rc, which runs the rcorder(8) program for calculating ordering dependencies on the initscripts in /etc/rc.d/. Initscripts are written in a clean, standard format based on a file of common subroutines called /etc/rc.subr, from which all initscripts source. In turn, the global system and service startup behavior from which services to enable/disable, to name service configuration and other modules, is done from the /etc/rc.conf file, a shell script sourced by /etc/rc, working on simple key-value pairs.

rc.d manages to retain a high degree of configurability and flexibility with most common pain points about System V initscript-based systems being completely eradicated, but it is again limited to a basic management framework by design. Arch Linux used an rc.d-like setup before migrating to systemd in 2012.

simpleinit, jinit and the need(8) concept (2001-3)

Envisioned by Richard Gooch in his 2002 paper “Linux Boot Scripts” as the middle ground between SysV and BSD, with a twist.

In short, it was based on a flat directory of short scripts in /sbin/init.d providing services and being synchronized through the use of two small utilities: need(8) and provide(8), which are in turn symlinked to the argv[0] of initctl(8).

These two primitives were meant to obsolete runlevels in favor of state transitions and provide dependency management. provide(8) registers a service based on a name, and need(8) starts or stops a service in a blocking fashion depending on whether the name exists. display-services(8) was present for status listing. Otherwise, the standard SysV inittab(5) was retained. See also the initctl(8) manpage.

It did gain some modest success, being included in several embedded or experimental projects, as well as serving as an early example of dependency-based init. It is still used to this day by at least one active distribution, Source Mage GNU/Linux.

simpleinit directly inspired John Fremlin to write a derivative in C++ entitled jinit, which unusually used System V message queues for its IPC. It has not been updated since 2003.

minit (2001-2)

Designed by Fefe (Felix von Leitner), it can be somewhat described as an inverted (endogenous) daemontools.

Where daemontools spawns a separate supervise process for each service, minit unifies it into a central msvc supervisor. It also has a very basic dependency system through each service directory having a “depends” file, really just a way of calculating service ordering. It can start services both synchronously and asynchronously.

It’s very tiny with a minimal footprint, being designed to link to dietlibc, made by the same author. A 2004 presentation at Linux Kongress goes into more detail about the principles behind minit.

Nikola Vladov later forked it around 2007 to create ninit, which extended minit with more service configuration options, as well as adding sysvinit compatibility.

depinit (2002)

depinit, written by Richard Lightman circa 2002, was a system self-described as “[incorporating] ideas from sysvinit, simpleinit, daemontools and make”.

It supported parallel service startup, a relatively intelligent (for its time) dependency system where the minimum number of dependent services were calculated when having to stop something, rotating loggers through pipes and user-configurable signals for how to operate on processes. It eschewed runlevels in favor of grouping services by name in the file system, and had a fully self-contained shutdown procedure that did not depend on scripts.

It used shell scripts, but these were much more concise due to having a sane process management system. It unfortunately failed to gain significant notice and died out.

daemond (2002-3)

A seldom known but historically interesting system, daemond came with a relatively intricate at the time system for resolving dependencies (including special stanzas for kernel modules), and its own configuration syntax based on blocks that could optionally include fragments from shell scripts (similar to Upstart jobs).

Example:

service "fsck" { description "Check filesystems"; require "lvm"; setup "/sbin/fsck -C -R -A -a"; } service "mount-local" { require "fsck"; description "Mount local filesystems"; setup "/sbin/mount -a -v -t nonfs,nosmbfs"; }

Other than services, it also read an initial configuration file from /etc/daemond.rc and was designed around fast parallel startup.

Its dependency stanzas were as follows:

require "file-or-service"; This states that the service cannot be started at all unless the file is present, or the service has been succesfully started. need "file-or-service"; Same as require, except that if the dependency cannot be satisfied then the entire service is made unavaliable, as though it did not exist (so that services that depend on it will be able to proceed). This is useful when you want a service that must start when some condition is met, but which is optionnal otherwise. want "service"; This is not a proper dependency, but a 'collaborating' service. This directive states that if the service where it appears starts, then service must be attempted as well, but need not succeed. require module "module"; need module "module"; Same as the first two, but for kernel modules. It is usually better to rely on kernel autoloading for the most part. group "group"; This places the service in a group. That group can then be refered to as if it was a service (starting all of the group) and will be deemed successful if all the members of the group are started, unless... require any "group"; ...is used, in which case the group will be deemed succesful if /any/ service in the group is started. mode "mode" { ... }; This defines a target mode (akin to init's runlevels). It can only contain dependencies.

It was written in C++ instead of C, and the author evidently had ambitions for it, saying in his README:

I want this to someday be a mainstream alternative to the antedeluvian SysV >and BSD inits, and it needs to be hammered on by lots of people. If you >created the service definition files to make your system boot right, I almost >certainly want a copy– especially if you are using a standard distribution– >so that I can distribute those as well.

It failed to leave an impression.

GNU dmd (2003)

dmd (daemon managing daemons) is a system initially launched in 2003 by Wolfgang Jährling, most notable for being entirely written and configurable in Guile Scheme.

It was comatose (if not dead) for nearly a decade, it was revived in 2013 as part of the Guix transactional package management system, also written in Guile. Today it is used as the init daemon and service manager for the Guix System Distribution.

It is well documented and generally simple, being based on dependencies in the form of a provides/requires relationship and having service configuration routines be reusable Scheme macros, including so-called constructors for encapsulating various execution disciplines.

Due to the primacy of the underlying Scheme language, it is thus remarkably flexible and extensible. Here is a sample from the GuixSD sources:

(define (root-file-system-service) "Return a service whose sole purpose is to re-mount read-only the root file system upon shutdown (aka. cleanly \"umounting\" root.) This service must be the root of the service dependency graph so that its 'stop' action is invoked when dmd is the only process left." (with-monad %store-monad (return (service (documentation "Take care of the root file system.") (provision '(root-file-system)) (start #~(const #t)) (stop #~(lambda _ ;; Return #f if successfully stopped. (sync) (call-with-blocked-asyncs (lambda () (let ((null (%make-void-port "w"))) ;; Close 'dmd.log'. (display "closing log

") ;; XXX: Ideally we'd use 'stop-logging', but that one ;; doesn't actually close the port as of dmd 0.1. (close-port (@@ (dmd comm) log-output-port)) (set! (@@ (dmd comm) log-output-port) null) ;; Redirect the default output ports.. (set-current-output-port null) (set-current-error-port null) ;; Close /dev/console. (for-each close-fdes '(0 1 2)) ;; At this point, there are no open files left, so the ;; root file system can be re-mounted read-only. (mount #f "/" #f (logior MS_REMOUNT MS_RDONLY) #:update-mtab? #f) #f))))) (respawn? #f)))))

pinit (2003)

pinit was a barely known yet surprisingly important init system designed by Wouter von Klaunen circa 2003.

It was arguably the first system to use XML as its service configuration language, predating even launchd and SMF. These resembled the following:

<?xml version="1.0"?> <command provides="system.swap"> <startup message="Activating all swap partitions..."> /sbin/swapon -a </startup> <shutdown message="Deactivating all swap partitions..."> /sbin/swapoff -a </shutdown> <dependency name="system.checkfs"/> </command>

As can be evidenced, it had a dependency system (more of an ordering-based one than a fully transactional dependency resolver like SMF or systemd). It supported parallel service startup and also might have been the first to have a plugin system. That is to say, various bootup procedures, instead of being hardcoded into the init daemon or launched as scripts, were dynamically loaded and unloaded into/from pinit’s address space as shared objects, though there did not appear to be a formally defined API.

Runlevels were set aside in favor of static profiles listing what to enable/disable for a state transition, not unlike systemd presets.

It was somewhat heavyweight in comparison to its predecessors, making use of libxml and GLib for its utility library.

It was abandoned and ultimately failed to leave an impression.

initng (2005)

Initially released by Jimmy Wennlund in March of 2005 and largely tested on Gentoo systems, initng was one of the more ambitious and complete new-school systems. Besides the usual process management, supervision, service grouping (called runlevels here), dependencies and parallelism, it was most notable for its extremely comprehensive plugin system, having 47 in the base system by its final release. Plugins were capable of hooking into over 20 different segmented subsystems private to initng. These are all documented here, but they effectively turn the init daemon into a dedicate module loader and handler.

Services themselves were configured in a block-based format called ifiles, initng providing many prewritten files, e.g.

service service/aumix { use = service/alsasound; need = system/initial system/bootmisc; stdall = /dev/null; script start = { if [ -f /etc/aumixrc ] then @/usr/bin/aumix@ -f /etc/aumixrc -L else @/usr/bin/aumix@ -v75 -c75 -w75 fi }; exec stop = @/usr/bin/aumix@ -f /etc/aumixrc -S; }

As such, it could be regarded as a comprehensive example of a meta-init.

It was considered by Ubuntu at one point, but they ultimately decided to create Upstart, instead. The initng project has since quietly died, gaining little to no success.

launchd (2005)

Arguably the first “new-school” init system, characterized by having much of the logic all in the init(8) daemon while talking to it with a control utility. Configured through XML plists. In OS X, it’s also the bootstrap daemon (discovery registry) for Mach kernel services. Popularized the buzzword of “socket activation”. Segregates daemons (system-wide) from agents (per-user), the latter being grouped into types welded into OS X-specific subsystems like the loginwindow or the Aqua UI. Revolves around pure lazy loading of services, with no formal dependency model, instead expecting services to synchronize themselves via IPC throughout the rest of the OS X stack. Couples process types to scheduling policies and resource limits, evidently meant to preserve desktop responsiveness.

It was considered for Ubuntu, thrown away for licensing reasons (at the time it used the GPL-incompatible Apple Public Source License). Currently being explored by the NextBSD project and potentially for FreeBSD later down the road.

Service Management Facility (SMF) (2005)

Solaris SMF was likely the first to have a complicated transactional dependency system, these being tracked in an internal graph engine. Designed for complex server management scenarios. Integrates deeply with Solaris Fault Manager for tracking hardware anomalies, each service being identified through an FMRI (Fault Management Resource Identifier). Services are configured using XML manifests, which are then compiled into a database called the service configuration repository (where services may also optionally store runtime data), which can be read from using svcprop(1) and dynamically configured with svccfg(1). Service instances themselves are controlled through svcadm(1) and status obtained with svcs(1).

SMF differentiates between the master restarter (svc.startd, the default dependency manager) and delegated restarters, which export the same service states as the master restarter but have different, application-specific behaviors. Under Solaris, inetd is a delegated restarter.

More information can be found at the Oracle documentation.

eINIT (2006)

Similar to initng in it being heavily plugin-based, the init daemon is only a handler. Also targeted towards Gentoo. Configured in XML. Significantly higher meta-configurability than initng, each module being configurable with XML in the einit.xml manifest. Based on provides/requires type of dependencies and an event subsystem, but more for internal events like monitoring module loads/unloads than something meant for services to register to.

Example:

<einit prefix="services-virtual-module"> <daemon id="daemon-boinc" name="BOINC client" provides="boinc" requires="mount-critical" command="cd /var/lib/boinc; boinc_client" restart="yes" /> </einit>

Failed to gain traction and since abandoned.

Upstart (2006)

Originally designed by Scott James Remnant for Ubuntu. Briefly used in Fedora at one point, still in ChromeOS (Ubuntu itself moving on to systemd).

Upstart revolves around the idea of emitting events and taking actions in response, e.g. start and stop a service. It provides several modules called bridges to translate various kernel or userspace events into its own native queue. The list of built-in events is defined in upstart-events(7).

First to use D-Bus as the communication mechanism inside PID 1.

An event is thus an abstract precondition or postcondition for taking an action with regards to a service. They are just as much about synchronization as they are about dynamism and lazy loading.

The Upstart Cookbook goes in-depth into the architecture.

Asus eeePC fastinit + derivatives (2007-2015)

As part of Asus’s eeePC notebook line around 2007-2008, which had variants preinstalled with a GNU/Linux distribution called Xandros, they wrote a proprietary init(8) replacement entitled fastinit specifically designed for the purpose of… booting really fast, I suppose.

In 2008, it was reverse engineered by Claudio Matsuoka. The reason for its speed was simple. It was a completely self-contained boot logic in a small C program that directly called to POSIX for all operations that would usually be run as a shell script. It was designed for static configurability at compile time by editing the hardcoded C macros. As such, it also wasn’t flexible.

Around 2014-2015, however, it was forked and significantly extended by embedded developer Joachim Nilsson and rechristened finit.

Much like the previously mentioned pinit, finit works on a plugin-based system for hooking into the boot logic dynamically, except it actually has a clearly defined API. It stays compatible with SysV runlevels and comes with an embedded inetd for preopening server sockets, and is configured using a flat /etc/finit.conf file, like so:

user admin host testbed check /dev/vda1 module button module evdev module loop module psmouse runlevel 2 network service networking start tty /dev/tty1 tty /dev/tty2 tty /dev/tty3 # Alternative method instead of runparts #task [S] /etc/init.d/keyboard-setup start -- Setting up preliminary keymap #task [S] /etc/init.d/acpid start -- Starting ACPI Daemon #task [S] /etc/init.d/kbd start -- Preparing console #run [2] /etc/init.d/networking start -- Start networking # Services to be monitored and respawned as needed service [2345] /sbin/klogd -n -- Kernel logging server service [2345] /sbin/syslogd -n -- Syslog server service [3] /usr/sbin/gdm -- GNOME Display Manager # Run start scripts from this directory # runparts /etc/start.d # Inetd services inetd time/udp wait [2345] internal -- UNIX rdate service inetd time/tcp nowait [2345] internal -- UNIX rdate service inetd ssh@eth0:222/tcp nowait [2345] /usr/sbin/sshd -i -- SSH service inetd ssh/tcp nowait [2345] /usr/sbin/sshd -i -- SSH service # For multiple instances of the same service, add :ID somewhere between # the service/run/task keyword and the command. service :1 [2345] /sbin/httpd -f -h /http -p 80 -- Web server service :2 [2345] /sbin/httpd -f -h /http -p 8080 -- Old web server

finit is definitely oriented towards more limited systems, but it manages to impart a lot of flexibility with its small surface.

OpenRC (2007)

Used primarily by Gentoo but also Alpine Linux and others, meant to replace the earlier Gentoo baselayout scripts. OpenRC in fact doesn’t supply an init daemon, but provides a comprehensive process management framework (though little in the way of supervision, as it is meant to integrate with external supervisors, such as its explicit support for the s6 daemontools-like supervisor) heavily influenced by the rc.d of the BSD systems. Hence the name. It has been booted from Busybox init+mdev and sysvinit successfully.

See the Gentoo wiki and ArchWiki overviews.

Android init (2008)

Android init is a specialized init daemon designed to handle platform-specific features like system properties and provide a generic lazy loading system based on an event mechanism called actions, some user-defined, others set by the init daemon itself. It is configured in a monolithic /init.rc using a line-based configuration language.

I have a detailed article about its workings which can be read here. It does not have much in the way of novelty, but rather is intended to be something vendors can configure once in their initramfs and forget afterwards.

systemd (2010)

Originally meant to be called Babykit.

No further clarification necessary.

procd (2012)

procd is a small init daemon with supervision designed specifically for OpenWrt, and thus tailored towards systems like routers. It uses a small, object-oriented message bus implementation called ubus for communication, and supports service sandboxing through namespaces and through syscall filtering like seccomp-bpf.

A special library for init scripts is used, e.g.

START=50 USE_PROCD=1 start_service() { procd_open_instance procd_set_param command /usr/bin/xupnpd procd_append_param command -d /usr/share/xupnpd procd_set_param respawn procd_close_instance }

Where the procd_ routines serialize the arguments into JSON and pass them over ubus.

Epoch (2014)

Epoch is a deliberately minimalist init daemon having full process management and supervision, but executing all processes serially/synchronously. Uses the Object metaphor for services and similarly to Android init, is configured using a central file (here INI-like), e.g.

Hostname=FILE /etc/hostname DefaultRunlevel=boot EnableLogging=true DisableCAD=true BlankLogOnBoot=true MountVirtual=procfs sysfs devpts+ devshm+ ObjectID=sysclock ObjectDescription=Configuring system clock ObjectStartCommand=hwclock -s ObjectStopCommand=hwclock -w ObjectStartPriority=1 ObjectStopPriority=2 ObjectEnabled=true ObjectOptions=RAWDESCRIPTION ObjectRunlevels=boot core ObjectID=mountruntmp ObjectDescription=Mounting /run and /tmp ObjectStartCommand=/etc/epoch/scripts/mountruntmp.sh ObjectStopCommand=NONE ObjectStartPriority=2 ObjectStopPriority=0 ObjectEnabled=true ObjectOptions=RAWDESCRIPTION ObjectRunlevels=boot core hurr ObjectID=rwfs ObjectDescription=root filesystem read-write support ObjectStartCommand=/bin/mount -o remount,rw / ObjectStopCommand=/bin/mount -o remount,ro / ObjectStartPriority=4 ObjectStopPriority=6 ObjectEnabled=true

Uses its own private, basic message bus implementation based on System V shared memory.

As is observed, it has the concept of priorities for ordering as opposed to dependencies. Priorities can also be used for logical groupings, and a priority of 0 is equivalent to a mask or hard disable.

sinit (2014)

Über alles.

# MIT license. #include <sys/types.h> #include <sys/wait.h> #include <signal.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h> #define LEN(x) (sizeof (x) / sizeof *(x)) static void sigpoweroff(void); static void sigreap(void); static void sigreboot(void); static void spawn(char *const []); static struct { int sig; void (*handler)(void); } sigmap[] = { { SIGUSR1, sigpoweroff }, { SIGCHLD, sigreap }, { SIGINT, sigreboot }, }; static char *const rcinitcmd[] = { "/bin/rc.init", NULL }; static char *const rcrebootcmd[] = { "/bin/rc.shutdown", "reboot", NULL }; static char *const rcpoweroffcmd[] = { "/bin/rc.shutdown", "poweroff", NULL }; static sigset_t set; int main(void) { int sig; size_t i; if (getpid() != 1) return 1; chdir("/"); sigfillset(&set); sigprocmask(SIG_BLOCK, &set, NULL); spawn(rcinitcmd); while (1) { sigwait(&set, &sig); for (i = 0; i < LEN(sigmap); i++) { if (sigmap[i].sig == sig) { sigmap[i].handler(); break; } } } /* not reachable */ return 0; } static void sigpoweroff(void) { spawn(rcpoweroffcmd); } static void sigreap(void) { while (waitpid(-1, NULL, WNOHANG) > 0) ; } static void sigreboot(void) { spawn(rcrebootcmd); } static void spawn(char *const argv[]) { pid_t pid; pid = fork(); if (pid < 0) { perror("fork"); } else if (pid == 0) { sigprocmask(SIG_UNBLOCK, &set, NULL); setsid(); execvp(argv[0], argv); perror("execvp"); _exit(1); } }

Final notes

Contact V.R. at Dark n' Edgy forums or leave a comment if there are any inaccuracies. The article’s descriptions are kept general, and further research should be done on a per-project basis if desired.

Comments