Log4j in clustered environment

Question

I am using log4j for a web application running in clustered environment. Under few circumstances the logging is lost for some of the processes (not particular). I am not able to pin point why. When run on single server instance, it works fine. I am using a daily rolling file appender. Any additional attributes to be added/modified ? Will Async Appender help ? Any solution on this ?

EDIT : Following is log4j XML along with AsyncAppender that I am planning to use

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">

<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/" debug="true">
    <appender name="appLog" class="org.apache.log4j.DailyRollingFileAppender">
        <param name="File" value="/logs/app.log" />
        <param name="DatePattern" value="'.'yyyy-MM-dd" />
        <param name="Threshold" value="DEBUG" />
        <param name="Append" value="true" />
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%d [%t] %-5p (%F:%L) - %m%n" />
        </layout>
    </appender>

    <appender name="async" class="org.apache.log4j.AsyncAppender">
        <param name="BufferSize" value="256" />
        <param name="LocationInfo" value="true"/> 
        <appender-ref ref="appLog" />
    </appender>

    <root>
        <priority value="debug" />
        <appender-ref ref="async" />
    </root>
</log4j:configuration>

Answer 1

Without you attaching your Log4J configuration, plus some information about your cluster topology, it is impossible to know for sure, but: what you're describing that could very well happen if there's a mismatch between how you configure Log4J, and how your application is configured to run in a cluster.

For example, if both cluster members run on the same physical machine, and your application is using the very same Log4J configuration file in both instances, then effectively, you have two different JVM's logging into the same files at once. That isn't going to work very well.

Generally speaking, you should avoid a situation in which two different classloaders write into the same physical files.

UPDATE (following attachment of log4j.xml): assuming that your clustered servers run on the same physical machine, the configuration you attached will end up with multiple log4j configurations logging into the same physical files (log4j "lives" in the context of a classloader, and you have multiple servers - hence multiple classloaders, on the same physical machine). This is not going to work, because the two log4j "instances" don't have a way to synchronize writing into that file.

An AsyncAppender isn't going to help you. All that AsyncAppender does is that it buffers the logging requests so your program gets control before the log lines are actually written. It doesn't solve the problem of synchronizing writing into the very same file.

There are two ways to resolve it:

1. Have each JVM log into a different log file. Actually, give it a shot and witness that it works.
2. Use a SocketAppender instead. All your logging (from all server instances) will end up being sent over a network connection, and a server program (provided by log4j) will collect them and write them into a file. Explanation how to use it can be found here.

Answer 2

As per suggestion given by Isaac to avoid a situation in which two different classloaders/jvm write into the same physical log file:

There are two ways to resolve it:

1. Have each JVM log into a different log file. Actually, give it a shot and witness that it works

2. Use a SocketAppender instead. All your logging (from all server instances) will end up being sent over a network connection, and a server program (provided by log4j) will collect them and write them into a file. Explanation how to use it can be found here.

For point#1, below approach can work:

Use:

<param name="File" value="${sys:log4j.logFile}" />

Instead:

 <param name="File" value="/logs/app.log" />

For JVM1 set the JVM property: log4j.logFile=/logs/jvm1/app.log

For JVM2 set the JVM property: log4j.logFile=/logs/jvm2/app.log

For more detail and other options: Refer: link1, link2