Why child process (daemon=True) not exiting when main process exit in python?

daemon, process, python

Solution

Notes:

The use of xrange the implies Python 2

`xrange(1, 4)` will yield 3 values not 4 (so, there will only be 3 children)

This is not quite how things work. The doc ([Python 2.Docs]: multiprocessing - daemon) should probably be more specific.

The thing is that multiprocessing registers a cleanup function to kill all its deamonic children when exiting. That is done via [Python 2.Docs]: atexit - Exit handlers (emphasis is mine):

Note: The functions registered via this module are not called when the program is killed by a signal not handled by Python, when a Python fatal internal error is detected, or when os._exit() is called.

You don't handle the TERM signal (sent by default by the kill command), therefore the cleanup function is not called by the main process (leaving its children running).

I modified your code to better illustrate the behavior.

code00.py:

#!/usr/bin/env python2

import multiprocessing
import os
import sys
import time


print_text_pattern = "Output from process {:s} - pid: {:d}, ppid: {:d}"


def child(name):
    while True:
        print(print_text_pattern.format(name, os.getpid(), os.getppid()))
        time.sleep(1)


def main(*argv):
    procs = list()
    for x in xrange(1, 3):
        proc_name = "Child{:d}".format(x)
        proc = multiprocessing.Process(target=child, args=(proc_name,))
        proc.daemon = True #x % 2 == 0
        print("Process {:s} daemon: {:}".format(proc_name, proc.daemon))
        procs.append(proc)

    for proc in procs:
        proc.start()

    counter = 0
    while counter < 3:
        print(print_text_pattern.format("Main", os.getpid(), os.getppid()))
        time.sleep(1)
        counter += 1


if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.\n")
    sys.exit(rc)

Notes:

Changed the way how children processes are spawned a bit: all of them are created 1st, and only then started

Added some print calls from each process, to track their activity in the StdOut - also added some time.sleep calls (1 second), to avoid producing too much output

Most important - the main process no longer runs forever. At some point it exits gracefully (after 3 cycles - due to counter variable), and there's when the behavior that I mentioned earlier kicks in. This could also have been possible by intercepting the TERM signal (and others that can be explicitly be sent by the kill command) and performing the cleanup then - in that way the children would be killed as well when killing the main process - but that's more complicated

I simplified things a bit so that only 2 children are spawned

Moved everything in a main function (for structure) enclosed in a `if __name__ == "__main__":` conditional, so the processes are not spawned if you import the module

Give different values proc.daemon for each child then monitor the output and `ps -ef | grep "code00.py"` output

Added an argument (name) to child func, but that's only for display purposes

Output:

[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow]> python2 ./code00.py
Python 2.7.12 (default, Oct  8 2019, 14:14:10) [GCC 5.4.0 20160609] 064bit on linux2

Process Child1 daemon: True
Process Child2 daemon: True
Output from process Main - pid: 1433, ppid: 1209
Output from process Child1 - pid: 1434, ppid: 1433
Output from process Child2 - pid: 1435, ppid: 1433
Output from process Main - pid: 1433, ppid: 1209
Output from process Child2 - pid: 1435, ppid: 1433
Output from process Child1 - pid: 1434, ppid: 1433
Output from process Main - pid: 1433, ppid: 1209
Output from process Child1 - pid: 1434, ppid: 1433
Output from process Child2 - pid: 1435, ppid: 1433
Output from process Child1 - pid: 1434, ppid: 1433
Output from process Child2 - pid: 1435, ppid: 1433

Done.

Problem

Here is the official explanation of `daemon` flag in python multiprocessing: When a process exits, it attempts to terminate all of its daemonic child processes. By my understanding, the parent process will kill its children whose daemon flag is set to be True when it exits. Below is the code I used to prove my guess. But the result is different. ``` import multiprocessing def child(): while True: pass for x in xrange(1, 4): proc = multiprocessing.Process(target=child, args=()) proc.daemon=True proc.start() while True: pass ``` The above starts 4 child processes and one main process. I killed the main process but the 4 children did not exit. So why are they not terminated by main since the daemon is set to be true?

Original source