Using different Log files for every thread created

Question

I am working on logging with log4j, i have sample code which creats certain number of threads , i want to use different log files for each threads. so I have use the PropertyConfigurator.configure() which takes a property object.

I have configured the property object in my code i.e hardcoded , but I want to load it external to code.

Now I want to provide all the configurations externally through a property file i.e not to hardcode them in code and at runtime add the two properties -

props.setProperty("log4j.logger."+"Thread" +
        Thread.currentThread().getName(),"DEBUG, file");

and

props.setProperty("log4j.appender.file.File",
        "/home/ekhaavi/workspace/TEST_2/ThreadHandler"+
                Thread.currentThread().getName()+".log"); 

since these variables like Thread.currentThread().getName() will get evaluated at runtime.

can anyone suggest me how to do it

I have three class --> MainClass, Parser(Thread class), LoggerClass in package

import java.util.HashMap;
import java.util.Map;

public class MainClass {
private static final org.apache.log4j.Logger log = LoggerClass.getLogger(MainClass.class);

public static void main(String args[]){
    Map map = new HashMap();
    map.put("Subject", "Math");

    log.info("The value of the map is " + map.toString());
    for(int ii=0; ii< 3; ii++){

        Parser th = new Parser(ii);
       }
    }
}

---

package com.test.log4j.conf;


public class Parser implements Runnable{

private org.apache.log4j.Logger log;

Parser(){
    Thread th = new Thread(this);
    th.start();

}


@Override
public void run() {

    log = LoggerClass.getThreadLogger("Thread" + Thread.currentThread().getName());
    log.info("------dagdjlagdjgasdjljgljg");
    System.out.println("The thread is " + Thread.currentThread().getName());
}

}

---

and finally my logger class is -->

import java.util.Properties;
import org.apache.log4j.Logger;
import org.apache.log4j.PropertyConfigurator;

public abstract class LoggerClass {

public static Logger getLogger(Class clazz){

    org.apache.log4j.Logger log = Logger.getLogger(clazz);
    Properties props=new Properties();
    props.setProperty("log4j.appender.file","org.apache.log4j.RollingFileAppender");
     props.setProperty("log4j.appender.logfile","org.apache.log4j.DailyRollingFileAppender");
    props.setProperty("log4j.appender.logfile.DatePattern","'.'yyyy-MM-dd");
    props.setProperty("log4j.appender.logfile.layout","org.apache.log4j.PatternLayout");
    props.setProperty("log4j.appender.logfile.layout.ConversionPattern","%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n");
    props.setProperty("log4j.appender.logfile.File","/home/ekhaavi/workspace/TEST_2/mylogfile.log");
    props.setProperty("log4j.logger.com.com.test.log4j.conf","INFO, logfile");

    PropertyConfigurator.configure(props);
    return log;

}

public static Logger getThreadLogger(String str){

    org.apache.log4j.Logger log = Logger.getLogger(str);
    Properties props=new Properties();
    props.setProperty("log4j.appender.file","org.apache.log4j.DailyRollingFileAppender");
    props.setProperty("log4j.appender.file.DatePattern","'.'yyyy-MM-dd");
    props.setProperty("log4j.appender.file.layout","org.apache.log4j.PatternLayout");
    props.setProperty("log4j.appender.file.File","/home/ekhaavi/workspace/TEST_2/ThreadHandler"+Thread.currentThread().getName()+".log");
    props.setProperty("log4j.appender.file.layout.ConversionPattern","%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n");
    props.setProperty("log4j.logger."+"Thread" + Thread.currentThread().getName(),"DEBUG, file");

    PropertyConfigurator.configure(props);
    return log;

 }

 }

Answer 1

The key 'threadId' [used in <Routes pattern="$${ctx:threadId}"> ] must be added to the thread context map in the run() method of the Runnable class as follows,

public class SomeClass implements Runnable{

 private int threadID;

 public SomeClass(int threadID){
   this.threadID=threadID;
   }
 @Override
 public void run() {

    ThreadContext.put("threadId", threadID);
    //Some code
    ThreadContext.remove("threadId");
   }
}

In addition, the correct log4j packages must be imported, as shown below.

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;

Please note that the following imports will not work. LogManager and Logger must also come from org.apache.logging.log4j.

import org.apache.log4j.LogManager;
import org.apache.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;

Answer 2

For log4j v2 you can use RoutingAppender to dynamically route messages. You can put value for key 'threadId' into the ThreadContext map and then use this id as a part of file name. There is an example which I have easily applied for the same purpose as yours. See http://logging.apache.org/log4j/2.x/faq.html#separate_log_files

Be aware when putting values into ThradContext map: "A child thread automatically inherits a copy of the mapped diagnostic context of its parent." So if you have put a value for key 'threadId' into the parent thread and eventually created multiple threads from it, then all child threads will inherit the value of 'threadId' value. I was no able to simply override this value by using put() one more time - you need to use ThreadContext.clear() or explicitly remove() the value from thread context map.

Here is my working log4j.xml:

<?xml version="1.0" encoding="UTF-8"?>
<configuration status="WARN">
    <properties>
            <property name="logMsgPattern">%d{HH:mm:ss} %-5level - %msg%n</property>
            <property name="logDir">test logs</property><!-- ${sys:testLogDir} -->
    </properties>
    <appenders>
        <Console name="Console" target="SYSTEM_OUT">            
                <PatternLayout pattern="${logMsgPattern}"/>
        </Console>

        <Routing name="Routing">
                    <Routes pattern="$${ctx:threadId}">             
                        <Route>
                            <RollingFile name="RollingFile-${ctx:threadId}" fileName="${logDir}/last-${ctx:threadId}.log" filePattern="${logDir}/%d{yyyy-MM-dd}/archived_%d{HH-mm}-${ctx:threadId}.log">
                                    <PatternLayout pattern="${logMsgPattern}"/>
                                    <Policies>
                                <OnStartupTriggeringPolicy />
                            </Policies> 
                    </RollingFile>
                        </Route>
                    </Routes>
            </Routing>  
        </appenders>

        <loggers>               
            <root level="debug">
                <appender-ref ref="Console" level="debug" />
                <appender-ref ref="Routing" level="debug"/>
            </root>                     
        </loggers>  
</configuration>