How to define a logger in python once for the whol

Question

I want to set up my logger once in my Python project and use that throughout the project.

main.py:

import logging
import test    

hostname = {"hostname": socket.gethostname()}
logger = logging.getLogger()
syslog = logging.StreamHandler()
formatter = logging.Formatter("{\"label\":\"%(name)s\", \"level\":\"%(levelname)s\", \"hostname\":\"%(hostname)s\", \"logEntry\": %(message)s, \"timestamp\", \"%(asctime)s\"}")
syslog.setFormatter(formatter)
logger.setLevel(logging.DEBUG)
logger.addHandler(syslog)
logger = logging.LoggerAdapter(logger, hostname)

def entry_point():
    logger.debug("entry_point")
    test.test_function()

entry_point()

test.py:

import logging

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)

def test_function():
    logger.debug("test_function")

This should give me:

entry_point
test_function

... both formatted with the format provider.

However I actually get an error KeyError: 'hostname' because it would seem the second logger does not know about the hostname format provider. I've also tried initialising both loggers with __name__ but then I get No handlers could be found for logger "test".

Is there a way I can define my logging configuration once and re-use it throughout my application?

Answer 1

A LoggingAdapter is a separate object, it does not replace the result of logging.getLogger() calls. You'll need to store it somewhere that it can be shared between different modules. You could use a separate module that everything else in your project imports from, for example.

I'll detail below how to handle this, but there is also an alternative that doesn't involve adapters at all, and instead uses a filter which is attached to the handler you created, letting you avoid having to deal with adapters altogether. See further down.

I'd separate out configuration and log object handling too; have the main module call the 'setup' function to configure the handlers and log levels, and set up the adapter in the module for others to import:

log.py:

import logging
import socket
def setup_logging():
    """Adds a configured stream handler to the root logger"""
    syslog = logging.StreamHandler()
    formatter = logging.Formatter(
        '{"label":"%(name)s", "level":"%(levelname)s",'
        ' "hostname":"%(hostname)s", "logEntry": %(message)s,'
        ' "timestamp", "%(asctime)s"}')
    syslog.setFormatter(formatter)

    logger = logging.getLogger()
    logger.addHandler(syslog)
    logger.setLevel(logging.DEBUG)


def host_log_adapter(logger):
    hostname = {"hostname": socket.gethostname()}
    return logging.LoggerAdapter(logger, hostname)


logger = host_log_adapter(logging.getLogger())

Then in main do:

import log
import test

log.setup_logging()

def entry_point():
    log.logger.debug("entry_point")
    test.test_function()

entry_point()

and in test:

from log import logger

def test_function():
    logger.debug("test_function")

Alternatively, rather than use an adapter, you could add information to log records by (ab)using a filter:

class HostnameInjectingFilter(logging.Filter):
    def __init__(self):
        self.hostname = socket.gethostname()}
    def filter(self, record):
        record.hostname = self.hostname
        return True

This adds the extra field directly on the record object and always returns True. Then just add this filter to the handler that needs to have the hostname available:

syslog = logging.StreamHandler()
formatter = logging.Formatter(
    '{"label":"%(name)s", "level":"%(levelname)s",'
    ' "hostname":"%(hostname)s", "logEntry": %(message)s,'
    ' "timestamp", "%(asctime)s"}')
syslog.setFormatter(formatter)
syslog.setFilter(HostnameInjectingFilter())

This removes the need to use an adapter entirely, so you can remove the host_log_adapter() function entirely and just use logging.getLogger() everywhere.