Log4j dailyrollingfileappender file issues

Question

We are encountering a peculiar problem. Scenario: We have 3 servers which with multiple instances of a component all writing transactional log to a single log file.We use log4j and the servers run in Java 1.3. setAppend() is passed true and implementation is DailyRollingFileAppender

Problem: At midnight, we are expecting the current log file to roll over with a new file name and start writing to a new file. This is working well in our test setup (single server writing logs). In production, at midnight, new file is getting created where new logs are getting written but rolled over file is getting deleted

Any help will be highly appreciated as its been couple of days and we are not able to get any leads for the problem.

Answer 1

Say you have configured DailyRollingFileAppender with daily rotation (It can be configured for rotation every hour, minute etc). Say, it is 31-Dec-2014 today and log file name is sample.log. Log rotation will happen in the following way:

• First log message that is received after midnight (say at 1am on 1-Jan-2015) will trigger log file rotation.
• Log file rotation will first delete any existing file with previous day suffix. (i.e. It will delete any file with name sample-2014-12-31.log. Ideally no such file should exist.).
• It will then rename current file with suffix of previous day. i.e. it renames sample.log to sample-2014-12-31.log.
• It will create new log file without suffix. i.e. new sample.log
• It will start writing into the new file sample.log.

If two instances of Log Manager points to same log file then each instance will independently repeats above steps on the same file. This can happen in any of the following scenarios:

• If two or more WAR file deployed in same container points to same log file.
• If two or more processes points to same log file.
• etc

Such scenario leads to the issue mentioned in the question.

• On a windows machine, once first process has rotated the log file and acquired handle on the new file, second log appender will fail to write logs.
• On a linux machine, Second process will delete the archive file created by first process and rename the new file (currently being used by first process) to previous day file. Thus first process will start writing logs in previous day file, second process will write logs in new file. After midnight, log file used by first process will get deleted. So, logs from first process will keep getting lost.

Answer 2

We ran into the same problem. The underlying problem is that there is no way to coordinate access to log file across multiple processes (in this case running on multiple servers.) This means all sorts of bad things happen: logs gets overwritten, files fail to roll etc...

My suggestion to you is to have each server write to a separate file, and then merge them in a post processing job.

Answer 3

You should not log to the same file from many processes. Log4j is thread-safe all right, but it's not process-safe, if I may say so; it does not work as a shared library among different java processes. Log4j embedded in one java application has no knowledge of any other one.

With rollover it causes the problem you've just discovered: all processes run their own rollover code, blindly overwriting the previous contents (because none of them expects any).

A possible solution here: Log4j Logging to a Shared Log File