Log4j Logging to a Shared Log File

Question

Is there a way to write log4j logging events to a log file that is also being written to by other applications. The other applications could be non-java applications. What are the drawbacks? Locking issues? Formatting?

Answer 1

Your best bet might be to let each application log separately, then put a scheduled job in place to 'zipper' the files together based on time. If you need really up-to-date access to the full log, you could have this run every hour.

Answer 2

Log4j is flexible enough that you can create log entries in a format that is compatible with the records already in the file. Just look into the appenders for how to format specific data fields.

Your main concern will be with locking, most likely by other applications. Make sure all of the applications do not have an exclusive lock on the file and you should be fine.

Answer 3

I don't think the default log4j appenders do any file locking or synchronization. Without such locking, you're likely to lose log messages or receive them mangled.

I'm not sure how easy it is to do file locking in Java, but to make this work directly, I think you would need to subclass the appropriate Appender and override the logging method, wrapping it with synchronization code that locks and unlocks the file. This might have performance implications, depending on your system load.

Log4perl has a synchronizing Appender, Log::Log4perl::Appender::Synchronized, which wraps an Appender and accomplishes this, and also seems to provide a flag in its Log::Log4perl::Appender::File that prevents interleaved lines. You might look at what those do to see if it could be replicable in Java.

Answer 4

I have a experience with the following two approaches:

1. Use database for logging instead of plain text file - it can be prohibitive because of performance issues, on the other hand it is very easy to analyze logs, create reports. Database takes care for all concurrency problems.
2. The other approach involves usage of JBoss server, which can be used to read logging information from other sources. JBoss can be run in the minimal configuration and thanks to that it is really lightweight (2 seconds startup time). Details can be found here http://docs.jboss.org/process-guide/en/html/logging.html (Logging to a Seperate Server). Log4J takes care of all locking/concurrency issues.

If you are not planning to use JBoss you can use the second approach as a base of your own logging solution.

Answer 5

Log4j has a SocketAppender, which will send events to a service, which you can implement yourself or use the simple implementation bundled with Log4j.

It also supports syslogd and the Windows event log, which may be useful in trying to unify your log output with events from non-Java applications.

If performance is an issue at all, you want a single service writing the log file, rather than trying to coordinate a consistent locking strategy among diverse logging applications.