Prerequisites

You should have Python >3.6 installed. If you don't you may download a copy of Python for your platform here.

Let's walk through how to create a daemon manually, without leveraging a third-party library. The only other prerequisite for this is to have the requests Python package installed.

$ mkdir daemons $ cd daemons $ touch daemon_func.py $ pip install requests

Let's create a simple python function. A fictitious news aggregator that pulls the latest top news feeds based on US headlines and writes them to a file every few seconds. The generated file format is as follows:

pid_{pid}_my_daemon_log_{timeref}.txt

This function will be the basis for illustrating how we create a Python daemon. Ensure you generate your own API key. To get your own API key, create a free account on https://newsapi.org and add your key to the code snippet below:

Take note that all import statements have been included within the main function for convenience and code isolation. Execute the above function to make sure everything is working as expected.

Let's create the main function that will be responsible for daemonize our main() function

Forking a daemon on Unix requires a certain specific sequence of system calls.

For starters, Daemon processes must detach from their controlling terminal and process group. This is achieved through a process called forking. When we fork something a copy of itself is created. In Unix we need to fork twice, thus terminating each parent process and letting only the grandchild of the original process run the daemon’s code.

In the above code, this is done by invoking the os.fork() function. This allows us to decouple the daemon process from the calling terminal so that the daemon process can keep running (typically, as a server process without further user interaction, like a web server, for example) even after the calling terminal is closed. Any code added after seconding forking will be executed within the background as a daemon. Executing our code would produce the output below. Note the PID numbers from the parent process to the detached process:

This is the parent PID 39813

Detaching from parent environment

Detached Daemon PID 39815

executing daemon background......

The call to os.setsid() creates a new session. The process becomes the leader of a new session and a new process group, and is disassociated from its controlling terminal.

The call to os.chdir(“/”) changes the current working directory to the root directory. The current working directory inherited from the parent could be on a mounted file system. Since daemons normally exist until the system is rebooted, if the daemon stays on a mounted file system, that file system cannot be unmounted.

To kill your daemon open up your terminal and enter the command kill -9 {PID}

If you’re on a mac, you can type [CMD]+ [space], search for monitor and then manually find the PID and force exit the process. If you run the program multiple types, be sure to kill all the Python processes running your daemon code.

One thing I want to remind you. Creating child processes using a Python multiprocessing module and setting process to daemon=True does not create a Unix detached daemon, as in the example code below. They have different characteristics. Additionally, a daemon is not a service.

Daemon multiprocessing module

The Python multiprocessing module Daemon flag executes a process in the background when the daemon property flag is set to true . The key difference is the daemon process will continue to run as long as the main process is executing. It will terminate after finishing its execution or when the main program is killed. It's easy to assume that using this property achieves the same thing.

Our initial code, with the double forking magic, looks simple enough to integrate with your own Python projects but to the trained eye, it’s missing a lot.

For starters, we don’t have proper PID state management in place. There’s no way to stop or restart the daemon in the event of exceptions. There’s no way to know the state of background daemons. There's no way to know how many demons have been invoked and are executing. There's no way to change the fact that only a single occurrence of the daemon can be invoked. Based onthe pep-3143 specification there are certain conditions and behaviors that should be performed when turning a program into a well-behaved Unix daemon process.

Let's take a look at two libraries that will make it easy to daemonize your Python applications.