e-tinkerer

A Place for my train of thought, mainly electronics, mcus and math

On Linux init managers

In this post let’s look into the first process that is run in the linux userland: the init process. We’ll look briefly at the kernel-userland interface and existing init managers. This post is per se not a tutorial but an overview of concepts relating to init managers.

A brief overview of init managers

A reminder of the kernel boot process: the kernel initializes the system, loads in-tree kernel modules, detects and initializes hardware. After this the kernel is ready to start running userland.

The init process is the first process that is run when the linux kernel enter the userland. For brevity let’s call the init process just init. At this stage the kernel is looking for executable files. A snippet of kernel source shows the exact paths the kernel looks for:

	if (!try_to_run_init_process("/sbin/init") ||
	    !try_to_run_init_process("/etc/init") ||
	    !try_to_run_init_process("/bin/init") ||
	    !try_to_run_init_process("/bin/sh"))
		return 0;

	panic("No working init found.  Try passing init= option to kernel. "
	      "See Linux Documentation/admin-guide/init.rst for guidance.");

If no executable is found the kernel panics and displays a message that no working init is found. You can see that the last fallback is the /bin/sh. The message also denotes that a custom path for init can be given with an option init=/path/to/my/init.

Technically the init process can be any executable script, a schell script or an executable binary. If the init is a shell script, the script must start with a shebang #!/bin/sh for the kernel to know to interpret the file with a shell. If no shebang is found the init is assumed to be an executable binary and in case of a schell script this results in a panic.

If a executalve binary file found it needs to have the int main() function implemented. By default the first process on the system becomes the process with PID number 1.

If all goes well, the kernel finds a proper executable and calls the try_to_run_init_process().

static int try_to_run_init_process(const char *init_filename)
{
	int ret;

	ret = run_init_process(init_filename);

	if (ret && ret != -ENOENT) {
		pr_err("Starting init: %s exists but couldn't execute it (error %d)\n",
		       init_filename, ret);
	}

	return ret;
}

The run_init_process() then calls kernel_execve() with the executable path. The first process in now succesfully running!

As a side thought, if the system had a Python interpreter included in PATH then technically you could run a .py script as the first process in the system with #!/bin/python. That might not be much of use but a fun experiment to try!

The first running process could in theory contain all the application logic. This might make sense if the application logic is brief not that complex. However crashing the PID 1 results in a panic and thus a system crash. In a more complex system separate layers and stacks (network, usb etc.) are needed for better decoupling, maintainability and testability.

Given all this there is a lot of responsibility on the init. In a running system the init should not return nor crash. Modern linux systems have evolved to use more sophisticated init managers such as systemd that runs on Ubuntu, Red Hat and Fedora based systems.

A more seasoned systems developer knows the sysvinit and the tacky syntax init script it uses. On Gentoo there is the OpenRC, a kind of middle-ground between systemd and sysvinit, and some other smaller systems like runit (docs on runit) and minit (post about minit on medium.com). Some init managers like OpenRC have added modularity. Besides using start-stop-daemon it supports separate daemon monitoring processes such like s6 (more info on OpenRC and s6 here).

/etc/inittab, init scripts and daemons

On a linux system running full desktop environment there are a lot of daemons. The system manages internal clock with ntpd, hot-plugged devices with udev, an ssh server with sshd etc. The user on a desktop systems needs not to burden themself of running these manually. systemd in itself is a lot more than an init system that starts and stop services. For more details on systemd features, see here.

As an example let’s look at BusyBox. BusyBox provides an init that supports the /etc/inittab file from System V Release 2 times.

# inittab for linux
id:1:initdefault:
rc::bootwait:/etc/rc
1:1:respawn:/etc/getty 9600 tty1
2:1:respawn:/etc/getty 9600 tty2
3:1:respawn:/etc/getty 9600 tty3
4:1:respawn:/etc/getty 9600 tty4

inittab is written per line with the format: id:runlevels:action:process. In other words every line specifies a guide how to spawn a single process at system start up. The script above tells that:

  • id 1 is the default runlevel
  • run /etc/rc at boot and wait until it returns
  • start four terminal instances on different ttyX devices. If the getty instances return they are respawned

This format proves handy in smaller systems. More lines and runlevels can be appended to the file. You can read more about inittab in docs and how it is parsed by BusyBox.

While the inittab has some process management rules, like respawning mechanism a larger systems with tens or hundreds or processes needs more sophisticated controls. An init manager, like systemd, treats daemons and user processes as services. A service is a just a process that is forked from the PID 1. Systemd initiates services using the same syscalls as the /bin/shell, that is fork() follwoed by execve(). An init manager provides commands like start, stop and restart are provided to manage the running services.

A minimal init manager is an iterator that reads in a list of guide lines, called init scripts, found in /etc/init.d, that describe how to configure and start a service. The init script can define start up and tear down functions. The benefit of separate init scripts is easy to understand. Starting services as a part of a loop already allows a single init script startup to crash. A crashed init script is then cleaned and the init system continues to the next script in the list. Once the init scripts are all run the init system goes to sleep. At this stage the user can give commands to the init manager with system tools like systemctl.

OpenRC, systemd, sysvinit

Now let’s look at some init managers.

For simplicity let’s look at a init script that OpenRC uses. I find the OpenRC style init scripts easiest to read.

#!/sbin/openrc-run

description="Example daemon service"

command="/usr/bin/mydaemon"
command_args="--config /etc/mydaemon.conf"
pidfile="/run/mydaemon.pid"
command_user="mydaemon:mydaemon"

depend() {
    need net
    after firewall
}

start_pre() {
    checkpath --directory --owner mydaemon:mydaemon --mode 0755 /run/mydaemon
}

start() {
    ebegin "Starting mydaemon"
    start-stop-daemon --start \
        --quiet \
        --pidfile "${pidfile}" \
        --make-pidfile \
        --background \
        --user "${command_user}" \
        --exec "${command}" -- ${command_args}
    eend $?
}

stop() {
    ebegin "Stopping mydaemon"
    start-stop-daemon --stop \
        --quiet \
        --pidfile "${pidfile}" \
        --exec "${command}"
    eend $?
}

It is clear that the script runs start_pre() after which start() is run. When either the user or the init manager stops the service then the stop() function is called. OpenRC support custom functions to be implemented in init scripts. The depend() function defines how the service relates to other services. This mainly has to do with the services start sequence order.

Next let’s look at a systemd example. A systemd unit might look like the following:

[Unit]
Description=Example background service
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/example-app --run
Restart=on-failure
User=example
Group=example
WorkingDirectory=/usr/local/bin

# Logging
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

The syntax is less verbose compared to OpenRC but at the same time the function of some lines is not that obvious, like for example the “Type” and “WantedBy” tags.

And finally let’s look at a SysV example:

#!/bin/sh

DAEMON=/usr/local/bin/example-app
DAEMON_OPTS="--run"
NAME=example
PIDFILE=/var/run/$NAME.pid
USER=example

. /lib/lsb/init-functions

start() {
    echo "Starting $NAME..."
    start-stop-daemon --start --quiet --background --pidfile $PIDFILE --make-pidfile \
        --chuid $USER --exec $DAEMON -- $DAEMON_OPTS
    status=$?
    [ $status -eq 0 ] && log_end_msg 0 || log_end_msg 1
}

stop() {
    echo "Stopping $NAME..."
    start-stop-daemon --stop --quiet --pidfile $PIDFILE --retry=TERM/30/KILL/5
    status=$?
    [ $status -eq 0 ] && rm -f $PIDFILE && log_end_msg 0 || log_end_msg 1
}

status() {
    status_of_proc -p $PIDFILE $DAEMON $NAME && exit 0 || exit $?
}

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        sleep 1
        start
        ;;
    status)
        status
        ;;
    *)
        echo "Usage: $0 {start|stop|restart|status}"
        exit 1
esac

exit 0

You can see the SysV init script is just a shell script. This has two clear benefits: the script can be called directly, has a dependency for a shell, and it could be copied onto the system over scp and be ready to use.

The start-stop-daemon has been seen used in several places here. We won’t look into how this shared binary works but feel free to explore the basics of start-stop-daemon here.

Conclusions

In this overview we have briefly looked at init managers and how they handle system process management. We have seen the interface between the linux kernel and userland, and examples of inittab format and OpenRC, systemd and SysV init scripts.

While these topics are often not visible if you’re using Ubuntu and package management on a smaller system you might want to concider which init manager suits the system requirements the best.

Hope you learned something new here on init managers and linux. Have a good day!

–

Further reading