Python logger: won't overwrite the original lo

Question

So, when I copy paste the following x times to the python prompt, it add the log x times to the end of the designated file.

How can I change the code so that each time I copy paste this to the prompt, I simply overwrite the existing file (the code seems to not accept the mode = 'w' option or I do not seem to understand its meaning)

def MinimalLogginf():
    import logging
    import os
    paths = {'work': ''}
    logger = logging.getLogger('oneDayFileLoader')
    LogHandler = logging.FileHandler(os.path.join(paths["work"] , "oneDayFileLoader.log"), mode='w')
    formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
    LogHandler.setFormatter(formatter)
    logger.addHandler(LogHandler) 
    logger.setLevel(logging.DEBUG)
    #Let's say this is an error:
    if(1 == 1):
        logger.error('overwrite')

So I run it once: MinmalLoggingf()

Now, I want the new log file to overwrite the log file created on the previous run:

MinmalLoggingf()

Answer 1

The mode is specified as part of logging.basicConfig and is passed through using filemode.

logging.basicConfig(
    level = logging.DEBUG,
    format = '%(asctime)s %(levelname)s %(message)s',
    filename = 'oneDayFileLoader.log,
    filemode = 'w'
)

https://docs.python.org/3/library/logging.html#simple-examples

Answer 2

If I understand correctly, you're running a certain Python process for days at a time, and want to rotate the log every day. I'd recommend you go a different route, using a handler that automatically rotates the log file, e.g. http://www.blog.pythonlibrary.org/2014/02/11/python-how-to-create-rotating-logs/

But, if you want to control the log using the process in the same method you're comfortable with (Python console, pasting in code.. extremely unpretty and error prone, but sometimes quick-n-dirty is sufficient for the task at hand), well...

Your issue is that you create a new FileHandler each time you paste in the code, and you add it to the Logger object. You end up with a logger that has X FileHandlers attached to it, all of them writing to the same file. Try this:

import logging
paths = {'work': ''}
logger = logging.getLogger('oneDayFileLoader')
if logger.handlers:
   logger.handlers[0].close()
   logger.handlers = []
logHandler = logging.FileHandler(os.path.join(paths["work"] , "oneDayFileLoader.log"), mode='w')
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
logHandler.setFormatter(formatter)
logger.addHandler(logHandler) 
logger.setLevel(logging.DEBUG)
logger.error('overwrite')

Based on your request, I've also added an example using TimedRotatingFileHandler. Note I haven't tested it locally, so if you have issues ping back.

import logging
from logging.handlers import TimedRotatingFileHandler

logPath = os.path.join('', "fileLoaderLog")
logger = logging.getLogger('oneDayFileLoader')
logHandler = TimedRotatingFileHandler(logPath,
                                   when="midnight",
                                   interval=1)
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
logHandler.setFormatter(formatter)
logger.addHandler(logHandler) 
logger.setLevel(logging.DEBUG)
logger.error('overwrite')

Answer 3

Your log messages are being duplicated because you call addHandler more than once. Each call to addHandler adds an additional log handler.

If you want to make sure the file is created from scratch, add an extra line of code to remove it:

os.remove(os.path.join(paths["work"], "oneDayFileLoader.log"))