Why child process (daemon=True) not exiting when m

问题:

Here is the official explanation of daemon flag in python multiprocessing:

When a process exits, it attempts to terminate all of its daemonic child processes.

By my understanding, the parent process will kill its children whose daemon flag is set to be True when it exits.

Below is the code I used to prove my guess. But the result is different.

import multiprocessing


def child():
    while True:
        pass


for x in xrange(1, 4):
    proc = multiprocessing.Process(target=child, args=())
    proc.daemon=True
    proc.start()


while True:
    pass

The above starts 4 child processes and one main process. I killed the main process but the 4 children did not exit.

So why are they not terminated by main since the daemon is set to be true?

回答1:

Notes:

• The use of xrange the implies Python 2
• xrange(1, 4) will yield 3 values not 4 (so, there will only be 3 children)

This is not quite how things work. The doc ([Python 2]: daemon) should probably be more specific.

The thing is that multiprocessing registers a cleanup function to kill all its deamonic children when exiting. That is done via [Python 2]: atexit - Exit handlers:

Note: The functions registered via this module are not called when the program is killed by a signal not handled by Python, when a Python fatal internal error is detected, or when os._exit() is called.

You don't handle the TERM signal (sent by default by the kill command), therefore the cleanup function is not called by the main process (leaving its children running).

I modified your code to better illustrate the behavior.

code.py:

#!/usr/bin/env python2

import multiprocessing
import os
import time


print_text = "Output from process {:s} - pid: {:d}, ppid: {:d}"


def child(name):
    while True:
        print(print_text.format(name, os.getpid(), os.getppid()))
        time.sleep(1)


if __name__ == "__main__":
    procs = list()
    for x in xrange(1, 3):
        proc_name = "Child{:d}".format(x)
        proc = multiprocessing.Process(target=child, args=(proc_name,))
        proc.daemon = True #x % 2 == 0
        print("Process {:s} daemon: {:}".format(proc_name, proc.daemon))
        procs.append(proc)

    for proc in procs:
        proc.start()

    counter = 0
    while counter < 3:
        print(print_text.format("Main", os.getpid(), os.getppid()))
        time.sleep(1)
        counter += 1

Notes:

• Changed the way how children processes are spawned a bit: all of them are created 1^st, and only then started
• Added some print calls from each process, to track their activity in the stdout - also added some time.sleep calls (1 second), to avoid producing too much output
• Most important - the main process no longer runs forever. At some point it exits gracefully (after 3 cycles - due to counter variable), and there's when the behavior that I mentioned earlier kicks in.
  This could also have been possible by intercepting the TERM signal (and others that can be explicitly be sent by the kill command) and performing the cleanup then - in that way the children would be killed as well when killing the main process - but that's more complicated
• I simplified things a bit so that only 2 children are spawned
• Enclosed everything in if __name__ == "__main__": so the processes are not spawned if you import the module
• Give different values proc.daemon for each child then monitor the output and ps -ef | grep "code.py" output
• Added an argument (name) to child func, but that's only for display purposes

Output:

[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q049604997]> python code.py
Process Child1 daemon: True
Process Child2 daemon: True
Output from process Main - pid: 20593, ppid: 12794
Output from process Child2 - pid: 20595, ppid: 20593
Output from process Child1 - pid: 20594, ppid: 20593
Output from process Main - pid: 20593, ppid: 12794
Output from process Child2 - pid: 20595, ppid: 20593
Output from process Child1 - pid: 20594, ppid: 20593
Output from process Main - pid: 20593, ppid: 12794
Output from process Child2 - pid: 20595, ppid: 20593
Output from process Child1 - pid: 20594, ppid: 20593

回答2:

Yes, your understanding is correct and your code to test this also works.

I just added some sleep statements to debug the output (without sleep, it is difficult to infer from the huge output of prints):

import multiprocessing
import time
import sys

print("main")

def child():
    while True:
        print("child")
        time.sleep(3)

for x in xrange(1, 4):
    proc = multiprocessing.Process(target=child, args=())
    proc.daemon=True
    proc.start()

time.sleep(7)
print("exit")
sys.exit() # this exits the main process

Now, when I ran this script, and when it was running, I did a ps aux and could see four processes running from this script. After 7 seconds, when I did ps aux again, I could no more see those processes running - which means:

When the main process exited, it terminated all of its daemonic child processes.

After that, I also set the proc.daemon to False, and ran the script once again. This time, even after 7 seconds, when I did a ps aux, I could still see the child processes running (since they are non-daemonic now, they do not exit even after main process terminates).

So this works as expected - let me know if you still have an issue here.

Edit 1: Thanks to @CristiFati for pointing out the original cleanup issue. This code works because calling sys.exit() also registers the atexit callbacks as elaborated here.