Best practices for Java logging from multiple thre

Question

I want to have a diagnostic log that is produced by several tasks managing data. These tasks may be in multiple threads. Each task needs to write an element (possibly with subelements) to the log; get in and get out quickly. If this were a single-task situation I'd use XMLStreamWriter as it seems like the best match for simplicity/functionality without having to hold a ballooning XML document in memory.

But it's not a single-task situation, and I'm not sure how to best make sure this is "threadsafe", where "threadsafe" in this application means that each log element should be written to the log correctly and serially (one after the other and not interleaved in any way).

Any suggestions? I have a vague intuition that the way to go is to use a queue of log elements (with each one able to be produced quickly: my application is busy doing real work that's performance-sensitive), and have a separate thread which handles the log elements and sends them to a file so the logging doesn't interrupt the producers.

The logging doesn't necessarily have to be XML, but I do want it to be structured and machine-readable.

edit: I put "threadsafe" in quotes. Log4j seems to be the obvious choice (new to me but old to the community), why reinvent the wheel...

Answer 1

Use a logging framework, such as Log4.

and if you are not happy with the output you can write your own Appender, Filter, whatever to tweak it just write. So you could do even some caching to rearrange the entries, although I am not saying this is a good idea.

Answer 2

log4j is and has been the standard for java logging for many years. But if you don't fancy an external dependency then the java.util.logging package provides an acceptable solution.

Answer 3

Use a logging framework, such as Log4j.

Answer 4

If you had to, you could roll your own .. using single-writer/single-reader FIFO or queues.

Answer 5

Use logback-classic. It is a newer and better implementation of log4j.

Answer 6

I had a similar problem and implementation demands for special logs only. My solution was:

1. I took a blockinglinkedqueue with size of *2 of the app's traffic/min.

2. All threads put the object in the queue and finishes the job.

3. Separate Log-Writer thread taking head object from queue and writing it to log4j file using a separate appender. This appender was not used for systemlogs.

This ensures that logs are written serially and always are in order.

This will not affect performance of the application since log writing is a completely separate process and will not create a bottleneck.

You can also use aysncappender of log4j.