Log Rotating into a Directory using Python

Question

I have a file called Poller.log and it's appended by log details all the time. I want this log file to be rotated everyday and limited by 30 days. Thus, the code works well.

Now I want this logs that has been rotated to be in a folder (i.e. logs/poller.log.2011-03-04_15-36). Is there anyway to direct where this rotated file should be created?

This python script will be executed by Cron.

import logging
import logging.handlers

LOG_FILENAME = '/home/stackoverflow/snmpdata/poller.log'

# Set up a specific logger with our desired output level
poll_logger = logging.getLogger('pollerLog')

# Add the log message handler to the logger
log_rotator = logging.handlers.TimedRotatingFileHandler(LOG_FILENAME, when='d', interval=1, backupCount=30, encoding=None, delay=False, utc=False)
poll_logger.addHandler(log_rotator)

# Roll over on application start
poll_logger.handlers[0].doRollover()

Answer 1

Python logging handler don't allow to do that easily. You might have 2 way of achieve this :

1. The simplest way would to setup LOG_FILENAME to be already in logs/poller.log, and if you want to access to your poller.log anywhere else, use a symlink :)

2. Create your own handler starting from TimedRotatingFileHandler, and copy/paste the doRollover() from /usr/lib/python2.X/logging/handlers.py, TimedRotatingFileHandler class. And change :

dfn = self.baseFilename + "." + time.strftime(self.suffix, timeTuple)

to

dfn = os.path.join('logs', os.path.basename(self.baseFilename)) + "." + time.strftime(self.suffix, timeTuple)

Answer 2

If you don't mind the extra dependency you could always use the rollover logging module in twisted. Twisted has a logfile module that allows for daily logs, weekly logs, or even monthly logs like this situation.