How to configure logging to syslog in Python?

Question

I can't get my head around Python's logging module. My needs are very simple: I just want to log everything to syslog. After reading documentation I came up with this simple test script:

import logging
import logging.handlers

my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.DEBUG)

handler = logging.handlers.SysLogHandler()

my_logger.addHandler(handler)

my_logger.debug('this is debug')
my_logger.critical('this is critical')

But this script does not produce any log records in syslog. What's wrong?

Answer 1

I add a little extra comment just in case it helps anyone because I found this exchange useful but needed this little extra bit of info to get it all working.

To log to a specific facility using SysLogHandler you need to specify the facility value. Say for example that you have defined:

local3.* /var/log/mylog

in syslog, then you'll want to use:

handler = logging.handlers.SysLogHandler(address = ('localhost',514), facility=19)

and you also need to have syslog listening on UDP to use localhost instead of /dev/log.

Answer 2

Change the line to this:

handler = SysLogHandler(address='/dev/log')

This works for me

import logging
import logging.handlers

my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.DEBUG)

handler = logging.handlers.SysLogHandler(address = '/dev/log')

my_logger.addHandler(handler)

my_logger.debug('this is debug')
my_logger.critical('this is critical')

Answer 3

Here's the yaml dictConfig way recommended for 3.2 & later.

In log cfg.yml:

version: 1
disable_existing_loggers: true

formatters:
    default:
        format: "[%(process)d] %(name)s(%(funcName)s:%(lineno)s) - %(levelname)s: %(message)s"

handlers:
    syslog:
        class: logging.handlers.SysLogHandler
        level: DEBUG
        formatter: default
        address: /dev/log
        facility: local0

    rotating_file:
        class: logging.handlers.RotatingFileHandler
        level: DEBUG
        formatter: default
        filename: rotating.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8

root:
    level: DEBUG
    handlers: [syslog, rotating_file]
    propogate: yes

loggers:
    main:
        level: DEBUG
        handlers: [syslog, rotating_file]
        propogate: yes

Load the config using:

log_config = yaml.safe_load(open('cfg.yml'))
logging.config.dictConfig(log_config)

Configured both syslog & a direct file. Note that the /dev/log is OS specific.

Answer 4

import syslog
syslog.openlog(ident="LOG_IDENTIFIER",logoption=syslog.LOG_PID, facility=syslog.LOG_LOCAL0)
syslog.syslog('Log processing initiated...')

the above script will log to LOCAL0 facility with our custom "LOG_IDENTIFIER"... you can use LOCAL[0-7] for local purpose.

Answer 5

From https://github.com/luismartingil/per.scripts/tree/master/python_syslog

#!/usr/bin/python
# -*- coding: utf-8 -*-

'''
Implements a new handler for the logging module which uses the pure syslog python module.

@author:  Luis Martin Gil
@year: 2013
'''
import logging
import syslog

class SysLogLibHandler(logging.Handler):
    """A logging handler that emits messages to syslog.syslog."""
    FACILITY = [syslog.LOG_LOCAL0,
                syslog.LOG_LOCAL1,
                syslog.LOG_LOCAL2,
                syslog.LOG_LOCAL3,
                syslog.LOG_LOCAL4,
                syslog.LOG_LOCAL5,
                syslog.LOG_LOCAL6,
                syslog.LOG_LOCAL7]
    def __init__(self, n):
        """ Pre. (0 <= n <= 7) """
        try:
            syslog.openlog(logoption=syslog.LOG_PID, facility=self.FACILITY[n])
        except Exception , err:
            try:
                syslog.openlog(syslog.LOG_PID, self.FACILITY[n])
            except Exception, err:
                try:
                    syslog.openlog('my_ident', syslog.LOG_PID, self.FACILITY[n])
                except:
                    raise
        # We got it
        logging.Handler.__init__(self)

    def emit(self, record):
        syslog.syslog(self.format(record))

if __name__ == '__main__':
    """ Lets play with the log class. """
    # Some variables we need
    _id = 'myproj_v2.0'
    logStr = 'debug'
    logFacilityLocalN = 1

    # Defines a logging level and logging format based on a given string key.
    LOG_ATTR = {'debug': (logging.DEBUG,
                          _id + ' %(levelname)-9s %(name)-15s %(threadName)-14s +%(lineno)-4d %(message)s'),
                'info': (logging.INFO,
                         _id + ' %(levelname)-9s %(message)s'),
                'warning': (logging.WARNING,
                            _id + ' %(levelname)-9s %(message)s'),
                'error': (logging.ERROR,
                          _id + ' %(levelname)-9s %(message)s'),
                'critical': (logging.CRITICAL,
                             _id + ' %(levelname)-9s %(message)s')}
    loglevel, logformat = LOG_ATTR[logStr]

    # Configuring the logger
    logger = logging.getLogger()
    logger.setLevel(loglevel)

    # Clearing previous logs
    logger.handlers = []

    # Setting formaters and adding handlers.
    formatter = logging.Formatter(logformat)
    handlers = []
    handlers.append(SysLogLibHandler(logFacilityLocalN))
    for h in handlers:
        h.setFormatter(formatter)
        logger.addHandler(h)

    # Yep!
    logging.debug('test debug')
    logging.info('test info')
    logging.warning('test warning')
    logging.error('test error')
    logging.critical('test critical')

Answer 6

You should always use the local host for logging, whether to /dev/log or localhost through the TCP stack. This allows the fully RFC compliant and featureful system logging daemon to handle syslog. This eliminates the need for the remote daemon to be functional and provides the enhanced capabilities of syslog daemon's such as rsyslog and syslog-ng for instance. The same philosophy goes for SMTP. Just hand it to the local SMTP software. In this case use 'program mode' not the daemon, but it's the same idea. Let the more capable software handle it. Retrying, queuing, local spooling, using TCP instead of UDP for syslog and so forth become possible. You can also [re-]configure those daemons separately from your code as it should be.

Save your coding for your application, let other software do it's job in concert.