Is a logger accessible to both parent and child pr

Question

In python if a logger is configured in the parent process, then will the child process also get that logger? To be more clear, in my application I configure root logger for parent process by doing logger = logging.getlogger() and adding handlers to it. Now when a child process is forked and it does

logger = logging.getlogger()
logger.info("dfsdf")

then all the logs are processed according to the parent's root logger. I didn't configure the root logger for the child. How is it possible? They are two different processes then how can they have same logger?

Answer 1

There are two things that may confuse you.

1. "The same logger object." Object is, of course, not the same, quite like all the objects in the child process are not the same as ones in the parent process. Child process is a full copy of its parent with a separate address space. That means they do not share any memory. If an object is located at some memory address in parent process, after fork the child process will have an identical object located at the same address. However, the addresses are in different address spaces, memory is not shared, and the objects are not related to each other after the forking is done. So, the short answer to your question is: the objects are not the same.

2. The logger object may have a file open before the fork is done. Open file is inherited by child process. That means the two processes have two file handles for a single file. And that is a question of synchronization.

UPD: I have looked through the logging module sources in my Python 2.7 installation. It uses threading.RLock to synchronize log access. That means you can not safely use the same log file in parent and child processes. You have to care about child and parent processes having different logs. Is it a daemon program? If yes, usually parent process does not need much logging and may use a separate logger.

Answer 2

When you fork a process, it 'inherits' the parent process memory, including the logger configuration.

From the Fork Wikipedia page:

The fork operation creates a separate address space for the child. The child process has an exact copy of all the memory segments of the parent process, though if copy-on-write semantics are implemented actual physical memory may not be assigned (i.e., both processes may share the same physical memory segments for a while). Both the parent and child processes possess the same code segments, but execute independently of each other.

This is not unique to Python; this happens for any UNIX process that is forked, be it implemented in C, Perl or Python.

The multiprocessing module uses this (on platforms that support it) to spin up new processes quickly.

Note that inheriting a logger may lead to race conditions; the logging module only knows about thread-safety; it uses a thread lock to serialize access to handlers, but that lock is not shared across processes (everything in the child process is a copy, and not the same object).

This means that when you log messages from both the parent and child at the same time, the log entries can end up mixed together as the OS switches between the two processes while writing the log entries to the file.