Python multiprocessing daemon vs non-daemon vs main
multiprocessing, python
Solution
When you see
[INFO/MainProcess] calling join() for process p1
it means the main process has not exited yet -- it's in the process of shutting down, but of course will not be done shutting down until that `join` returns... which will happen only once the joined process is done.
So the timeline is indeed as you expect it to be -- but since that `join` to `p1` is the very last think the `main` process ever does, you don't see that in the output or logging from it. (main has taken all the termination-triggered action, but as a process it's still alive until then).
To verify, use (on a Unixy system) `ps` from another terminal while running this (perhaps with slightly longer delays to help you check): you will never see a single Python process running out of this complex -- there will be two (main and p1) until the end.
Problem
I'm learning multiprocessing in Python while I found this odd behaviour between daemon and non-daemon process with respect to the main process. My Code: ``` import multiprocessing import time def worker(name,num): print name, 'Starting' time.sleep(num) print name, 'Exiting' if __name__ == '__main__': print 'main starting' p1=multiprocessing.Process(target=worker, args=("p1",7,)) p2=multiprocessing.Process(target=worker, args=("p2",5,)) p2.daemon=True p1.start() p2.start() time.sleep(4) print 'main exiting' ``` The output I'm getting is: ``` main starting p1 Starting p2 Starting main exiting p1 Exiting ``` Expected Output: ``` main starting p1 Starting p2 Starting main exiting p2 Exiting p1 Exiting ``` After few searches, I found this answer and inserted the following line to my code. ``` logger = multiprocessing.log_to_stderr(logging.INFO) ``` And the output I got is, ``` main starting [INFO/p1] child process calling self.run() p1 Starting [INFO/p2] child process calling self.run() p2 Starting main exiting [INFO/MainProcess] process shutting down [INFO/MainProcess] calling terminate() for daemon p2 [INFO/MainProcess] calling join() for process p2 [INFO/MainProcess] calling join() for process p1 p1 Exiting [INFO/p1] process shutting down [INFO/p1] process exiting with exitcode 0 ``` As per my understanding, - When a main process exits, it terminates all of its child daemon processes. (Which can be seen here) - A main process can not exit before all of its child non-daemon processes exit. But here, why the main process is trying to shut down before p1 exits? - p1 starts and sleeps for 7 seconds. - p2 starts and sleeps for 5 seconds. - main process sleeps for 4 seconds. - main process wakes up after 4 seconds and waits for p1 to exit. - p2 wakes up after 5 seconds and exits. - p1 wakes up after 7 seconds and exits. - main process exits. Wouldn't be the above a normal time-line for the above program? Can anyone please explain what's happening here and why? EDIT After adding the line `p1.join()` at the end of the code, I'm getting the following output: ``` main starting [INFO/Process-1] child process calling self.run() p1 Starting [INFO/Process-2] child process calling self.run() p2 Starting main exiting p2 Exiting [INFO/Process-2] process shutting down [INFO/Process-2] process exiting with exitcode 0 p1 Exiting [INFO/Process-1] process shutting down [INFO/Process-1] process exiting with exitcode 0 [INFO/MainProcess] process shutting down ```