Python Logging - Set Date as Filename

Question

I am working on implementing logging within my Python project and have hit a bit of a snag. I am trying to set up my logging such that the Handlers, and Formatters are all organized into a configuration file. What I am trying to do at the moment is to set up my fileHandler such that it will create a log file that looks something like this: YYYY_MM_DD.log obviously with the Y's representing the year, M's representing the month, and D's representing the day.

This is what I have attempted with my config file:

[loggers]
keys=root,MainLogger

[handlers]
keys=fileHandler, consoleHandler

[formatters]
keys=logFormatter, consoleFormatter

[logger_root]
level=DEBUG
handlers=fileHandler

[logger_MainLogger]
level=DEBUG
handlers=fileHandler, consoleHandler
qualname=MainLogger
propagate=0

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=consoleFormatter
args=(sys.stdout,)

[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=logFormatter
args=(datetime.now().strftime('%Y_%m_%d.log'), 'a')

[formatter_logFormatter]
format=%(asctime)s | %(levelname)-8s | %(lineno)04d | %(message)s

[formatter_consoleFormatter]
format=%(asctime)s | %(levelname)-8s | %(fillname)s-%(funcName)s-%(lineno)04d | %message)s

The file I am using to test the configuration is pretty simple:

import logging
import logging.config

logging.config.fileConfig('logging.conf')
logger = logging.getLogger('MainLogger')
logger.debug("TEST")

The specific error I am getting at the moment is:

configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: "%Y_%m_%d.log'), 'a')"

I've tried changing the %Y, %m, and %d as the error says, but that doesn't fix the problem. How do I go about setting up the config file so that my log files look the way I want them to?

I should note when I change the filename to test.log everything worked fine, so this is the only error I seem to be having with this.

Answer 1

This uses content from your config file, but does not access the file directly. You create your own filehandler and then add it to the logger.

import logging
from datetime import datetime

# Create logger.
logger = logging.getLogger('MainLogger')
logger.setLevel(logging.DEBUG)

# Create filehandler with desired filename.
fh = logging.FileHandler('{}.log'.format(datetime.now().strftime('%Y_%m_%d')))
fh.setLevel(logging.DEBUG)
log_formatter = logging.Formatter('%(asctime)s | %(levelname)-8s | %(lineno)04d | %(message)s')
fh.setFormatter(log_formatter)

# Add filehandler to logger.
logger.addHandler(fh)

Note that a (append) is the default mode parameter for a FileHandler.

Answer 2

Using double '%'-characters in the format string in combination with the approach suggested by Abhishek led to a working solution in my case (Python 3.5):

The filehandler in the config file should then look similar to this one:

[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=defaultFormatter
args=(__import__("datetime").datetime.now().strftime('/your_path/your_file_name_%%Y-%%m-%%d_%%H-%%M-%%S.log'), 'a')

Answer 3

This worked for me.

Update This: args=(datetime.now().strftime('%Y_%m_%d.log'), 'a')

To

args=(__import__("datetime").datetime.now().strftime('%Y_%m_%d.log'), 'a')

Reference (Example no 3): http://python-reference.readthedocs.io/en/latest/docs/functions/eval.html

Answer 4

Maybe try changing the name after you've loaded the config file:

from datetime inport datetime

logging.config.fileConfig('logging.conf')
logging.basicConfig(filename = datetime.now().strftime('%Y_%m_%d.log'))

Answer 5

This also works

from dateime import datetime
log_file = str(datetime.utcnow().strftime('%m_%d_%Y_%I_%M_%S')) + '.log'
logging.basicConfig(filename=log_file, format='%(levelname)s | %(asctime)s | %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p', level=logging.DEBUG)

Logfile name:04_30_2018_10_03_01.log

Answer 6

You can't use datetime in a config file, as it doesn't know what it means. You can however add the Filehandler in the python file itself:

import logging.config
from datetime import datetime

logging.config.fileConfig('aaa.conf')
logger = logging.getLogger('MainLogger')

fh = logging.FileHandler('{:%Y-%m-%d}.log'.format(datetime.now()))
formatter = logging.Formatter('%(asctime)s | %(levelname)-8s | %(lineno)04d | %(message)s')
fh.setFormatter(formatter)

logger.addHandler(fh)
logger.debug("TEST")

This way you can set the date as the file name in the handler.

This is the config file, note that you had a typo in the last formatter, you put fillname instead of filename and you forgot ( in message.

[loggers]
keys=root,MainLogger

[handlers]
keys=consoleHandler

[formatters]
keys=consoleFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler

[logger_MainLogger]
level=DEBUG
handlers=consoleHandler
qualname=MainLogger
propagate=0

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=consoleFormatter
args=(sys.stdout,)

[formatter_consoleFormatter]
format=%(asctime)s | %(levelname)-8s | %(filename)s-%(funcName)s-%(lineno)04d | %(message)s

This Should work just fine.