Suppress newline in Python logging module

Question

I'm trying to replace an ad-hoc logging system with Python's logging module. I'm using the logging system to output progress information for a long task on a single line so you can tail the log or watch it in a console. I've done this by having a flag on my logging function which suppresses the newline for that log message and build the line piece by piece.

All the logging is done from a single thread so there's no serialisation issues.

Is it possible to do this with Python's logging module? Is it a good idea?

Answer 1

If you wanted to do this you can change the logging handler terminator. I'm using Python 3.4. This was introduced in Python 3.2 as stated by Ninjakannon.

handler = logging.StreamHandler()
handler.terminator = ""

When the StreamHandler writes it writes the terminator last.

Answer 2

Let's start with your last question: No, I do not believe it's a good idea. IMO, it hurts the readability of the logfile in the long run.

I suggest sticking with the logging module and using the '-f' option on your 'tail' command to watch the output from the console. You will probably end up using the FileHandler. Notice that the default argument for 'delay' is False meaning the output won't be buffered.

If you really needed to suppress newlines, I would recommend creating your own Handler.

Answer 3

The new line, \n, is inserted inside the StreamHandler class.

If you're really set on fixing this behaviour, then here's an example of how I solved this by monkey patching the emit(self, record) method inside the logging.StreamHandler class.

A monkey patch is a way to extend or modify the run-time code of dynamic languages without altering the original source code. This process has also been termed duck punching.

Here is the custom implementation of emit() that omits line breaks:

def customEmit(self, record):
    # Monkey patch Emit function to avoid new lines between records
    try:
        msg = self.format(record)
        if not hasattr(types, "UnicodeType"): #if no unicode support...
            self.stream.write(msg)
        else:
            try:
                if getattr(self.stream, 'encoding', None) is not None:
                    self.stream.write(msg.encode(self.stream.encoding))
                else:
                    self.stream.write(msg)
            except UnicodeError:
                self.stream.write(msg.encode("UTF-8"))
        self.flush()
    except (KeyboardInterrupt, SystemExit):
        raise
    except:
        self.handleError(record)

Then you would make a custom logging class (in this case, subclassing from TimedRotatingFileHandler).

class SniffLogHandler(TimedRotatingFileHandler):
    def __init__(self, filename, when, interval, backupCount=0,
                 encoding=None, delay=0, utc=0):

        # Monkey patch 'emit' method
        setattr(StreamHandler, StreamHandler.emit.__name__, customEmit)

        TimedRotatingFileHandler.__init__(self, filename, when, interval,
                                          backupCount, encoding, delay, utc)

Some people might argue that this type of solution is not Pythonic, or whatever. It might be so, so be careful.

Also, be aware that this will globally patch SteamHandler.emit(...), so if you are using multiple logging classes, then this patch will affect the other logging classes as well!

Check out these for further reading:

• What is monkey-patching?
• Is monkeypatching considered good programming practice?
• Monkeypatching For Humans

Hope that helps.