How to log everything into a file using RotatingFi

Question

I am trying to use RotatingHandler for our logging purpose in Python. I have kept backup files as 500 which means it will create maximum of 500 files I guess and the size that I have set is 2000 Bytes (not sure what is the recommended size limit is).

If I run my below code, it doesn't log everything into a file. I want to log everything into a file -

#!/usr/bin/python

import logging
import logging.handlers

LOG_FILENAME = 'testing.log'

# Set up a specific logger with our desired output level
my_logger = logging.getLogger('agentlogger')

# Add the log message handler to the logger
handler = logging.handlers.RotatingFileHandler(LOG_FILENAME, maxBytes=2000, backupCount=100)

# create a logging format
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)

my_logger.addHandler(handler)

my_logger.debug('debug message')
my_logger.info('info message')
my_logger.warn('warn message')
my_logger.error('error message')
my_logger.critical('critical message')

# Log some messages
for i in range(10):
    my_logger.error('i = %d' % i)

This is what gets printed out in my testing.log file -

2013-11-22 12:59:34,782 - agentlogger - WARNING - warn message
2013-11-22 12:59:34,782 - agentlogger - ERROR - error message
2013-11-22 12:59:34,782 - agentlogger - CRITICAL - critical message
2013-11-22 12:59:34,782 - agentlogger - ERROR - i = 0
2013-11-22 12:59:34,782 - agentlogger - ERROR - i = 1
2013-11-22 12:59:34,783 - agentlogger - ERROR - i = 2
2013-11-22 12:59:34,783 - agentlogger - ERROR - i = 3
2013-11-22 12:59:34,783 - agentlogger - ERROR - i = 4
2013-11-22 12:59:34,783 - agentlogger - ERROR - i = 5
2013-11-22 12:59:34,783 - agentlogger - ERROR - i = 6
2013-11-22 12:59:34,784 - agentlogger - ERROR - i = 7
2013-11-22 12:59:34,784 - agentlogger - ERROR - i = 8
2013-11-22 12:59:34,784 - agentlogger - ERROR - i = 9

It doesn't print out INFO, DEBUG message into the file somehow.. Any thoughts why it is not working out?

And also, right now, I have defined everything in this python file for logging purpose. I want to define above things in the logging conf file and read it using the fileConfig() function. I am not sure how to use the RotatingFileHandler example in the logging.conf file?

UPDATE:-

Below is my updated Python code that I have modified to use with log.conf file -

#!/usr/bin/python

import logging
import logging.handlers

my_logger = logging.getLogger(' ')
my_logger.config.fileConfig('log.conf')

my_logger.debug('debug message')
my_logger.info('info message')
my_logger.warn('warn message')
my_logger.error('error message')
my_logger.critical('critical message')

# Log some messages
for i in range(10):
    my_logger.error('i = %d' % i)

And below is my log.conf file -

[loggers]
keys=root

[handlers]
keys=logfile

[formatters]
keys=logfileformatter

[logger_root]
level=DEBUG
handlers=logfile

[logger_zkagentlogger]
level=DEBUG
handlers=logfile
qualname=zkagentlogger
propagate=0

[formatter_logfileformatter]
format=%(asctime)s %(name)-12s: %(levelname)s %(message)s

[handler_logfile]
class=handlers.RotatingFileHandler
level=NOTSET
args=('testing.log',2000,100)
formatter=logfileformatter

But whenever I compile it, this is the error I got on my console -

$ python logtest3.py
Traceback (most recent call last):
  File "logtest3.py", line 6, in <module>
    my_logger.config.fileConfig('log.conf')
AttributeError: 'Logger' object has no attribute 'config'

Any idea what wrong I am doing here?

Answer 1

It doesn't print out INFO, DEBUG message into the file somehow.. Any thoughts why it is not working out?

you don't seem to set a loglevel, so the default (warning) is used

from http://docs.python.org/2/library/logging.html :

Note that the root logger is created with level WARNING.

as for your second question, something like this should do the trick (I haven't tested it, just adapted from my config which is using the TimedRotatingFileHandler):

[loggers]
keys=root

[handlers]
keys=logfile

[formatters]
keys=logfileformatter

[logger_root]
level=DEBUG
handlers=logfile

[formatter_logfileformatter]
format=%(asctime)s %(name)-12s: %(levelname)s %(message)s

[handler_logfile]
class=handlers.RotatingFileHandler
level=NOTSET
args=('testing.log','a',2000,100)
formatter=logfileformatter

Answer 2

I know, it is very late ,but I just got same error, and while searching that are I got your problem. I am able to resolve my problem, and I thought it might be helpful for some other user also :

you have created a logger object and trying to access my_logger.config.fileConfig('log.conf') which is wrong you should use logger.config.fileConfig('log.conf') as I mention below and need to import logging.config as well :

#!/usr/bin/python

import logging
import logging.handlers
import logging.config

logging.config.fileConfig('log.config',disable_existing_loggers=0)
my_logger = logging.getLogger('you logger name as you mention in your conf file')

my_logger.debug('debug message')
my_logger.info('info message')
my_logger.warn('warn message')
my_logger.error('error message')
my_logger.critical('critical message')

after doing these changes, AttributeError: 'Logger' object has no attribute 'config' error must be gone.