Python logging only log from script

Question

I'm using the Python logging module in a simple script of mine with the following setup at the moment.

logging.basicConfig(format='%(asctime)s %(message)s', level=logging.INFO, datefmt='%Y-%m-%d %H:%M:%S')
logger = logging.getLogger(__name__)

My problem is that this also catches 3rd party modules like requests and output info() log messages from them. Is there any way to suppress these messages or tell the logging module only to log messages from my own script?

Answer 1

Using named loggers in your modules:

import logging
logger = logging.getLogger(__name__)
logger.info("my info")
logger.error("my error")

you can set the log level for all the other loggers to error and for your loggers to debug:

import logging
logging.basicConfig(level=logging.ERROR)
logging.getLogger(my_module.__name__).setLevel(logging.DEBUG)

Answer 2

The above answer is not really correct - it will just set the bar higher for messages from other modules to be shown.

A very quick approach would be to use this piece of code:

import logging.config
logging.config.dictConfig({
    'version': 1,
    'disable_existing_loggers': True,
})

You have to set this after importing all modules - it will disable all loggers that were created up to this point. This will work most of the time, but some modules create their logger when you make a class instance for example (which would happen later in your code).

---

When you set up loggers according to the basic python tutorial they tell you to use logging.basicConfig(...). This is a problem as this will set the handler (aka where the log will be routed to) to logging.lastResort which is stderr starting with Python 3.2 for all loggers globally in the process. This means you now have enabled full logging for all modules.

So a better approach is to create a different logger only for your modules and give it some handlers of its own instead of using basicConfig().

There are two ways of doing that:

1) All function:

import logging

log = logging.getLogger(__name__)
log.setLevel(logging.DEBUG)
formatter = logging.Formatter(fmt="%(asctime)s %(levelname)s: %(message)s", 
                          datefmt="%Y-%m-%d - %H:%M:%S")
ch = logging.StreamHandler(sys.stdout)
ch.setLevel(logging.DEBUG)
ch.setFormatter(formatter)
fh = logging.FileHandler("mylog.log", "w")
fh.setLevel(logging.DEBUG)
fh.setFormatter(formatter)
log.addHandler(ch)
log.addHandler(fh)

This will give you the logger log that you can then use like log.error("Error found"). It will write to a new file called mylog.log and will also log so sys.stdout. You can change this as you like of course.

2) Using a dict:

import logging
import logging.config

DEFAULT_LOGGING = {
    'version': 1,
    'formatters': { 
        'standard': {
            'format': '%(asctime)s %(levelname)s: %(message)s',
            'datefmt': '%Y-%m-%d - %H:%M:%S' },
    },
    'handlers': {
        'console':  {'class': 'logging.StreamHandler', 
                     'formatter': "standard", 
                     'level': 'DEBUG', 
                     'stream': sys.stdout},
        'file':     {'class': 'logging.FileHandler', 
                     'formatter': "standard", 
                     'level': 'DEBUG', 
                     'filename': 'live_detector.log','mode': 'w'} 
    },
    'loggers': { 
        __name__:   {'level': 'INFO', 
                     'handlers': ['console', 'file'], 
                     'propagate': False },
    }
}

logging.config.dictConfig(DEFAULT_LOGGING)
log = logging.getLogger(__name__)

This will give the same result as the above, a bit longer, but maybe easier to read. This will automatically also set 'disable_existing_loggers': True. If you do not want that, you have to add it and set it to False.