can we have level based log files using log4j

Question

I have created an application using log4j for storing all the level based logs in a single file, but in the current case the single file will hold all the level logs, such as ALL, INFO, ERROR, FATAL, WARN, DEBUG etc... since in my application i am having an option say a select box upon which when i select ERROR, then it should show only error logs, if i select INFO then it should only INFO logs.

I am parsing the logs file to get the appropriate logs based on a particular level, but in this case say if the log file is very big then it will take more time.

One solution which i have thought for this is to maintain separate log files for each levels, so that we can avoid parsings

Like to know everyone's thought on this. Still if this is fine how can we redirect level based logs to separate log file like ERROR logs to appError.log, INFO logs to appInfo.log, and a common log file to hold all the level based logs.

My log4j.properties is as given below

# LOG4J configuration
log4j.rootLogger=DEBUG, Appender1, Appender2

log4j.appender.Appender1=org.apache.log4j.ConsoleAppender
log4j.appender.Appender1.layout=org.apache.log4j.PatternLayout
log4j.appender.Appender1.layout.ConversionPattern=%d [%t] %-7p %10c{1} - %m%n

log4j.appender.Appender2=org.apache.log4j.FileAppender
log4j.appender.Appender2.File=${appRootPath}/WEB-INF/logs/LogDemos.log
log4j.appender.Appender2.layout=org.apache.log4j.PatternLayout
log4j.appender.Appender2.layout.ConversionPattern=%d [%t] %-7p %10c{1} - %m%n

The parsing method is like as shown below

Scanner scanner = new Scanner(new File(ctx.getRealPath("/")+"/WEB-INF/logs/LogDemos.log"));
        while(scanner.hasNext())
        {
            String level = request.getParameter("level");
            String logs = scanner.nextLine();

            String regex = "(\\d{4}-\\d{2}-\\d{2}) (\\d{2}:\\d{2}:\\d{2},\\d{3}) \\[(.*)\\] ([^ ]*) +([^ ]*) - (.*)$";

            Pattern p = Pattern.compile(regex);

            Matcher m = p.matcher(logs);

            if (m.matches() && m.groupCount() == 6) {

                String date = m.group(1);
                String time = m.group(2);
                String threadId = m.group(3);
                String priority = m.group(4);
                String category = m.group(5);
                String message = m.group(6);
                if(priority.trim().equalsIgnoreCase(level.trim()))
                {
                    logsList.add(logs);
                    System.out.println("date: " + date);
                    System.out.println("time: " + time);
                    System.out.println("threadId: " + threadId);
                    System.out.println("priority: " + priority);
                    System.out.println("category: " + category);
                    System.out.println("message: " + message);
                }
                else
                {
                    logsList.add(logs);
                }
            }

        }
        session.setAttribute("logs", logsList);
        scanner.close();

Answer 1

If you use org.apache.log4j.varia.LevelRangeFilter, you can do as follows (for INFO and ERROR):

# LOG4J configuration
log4j.rootLogger=DEBUG, OnlyInfo, OnlyError

# Only INFO
log4j.appender.OnlyInfo=org.apache.log4j.FileAppender
log4j.appender.OnlyInfo.File=${appRootPath}/WEB-INF/logs/info.log
log4j.appender.OnlyInfo.File=logs/info.log
log4j.appender.OnlyInfo.layout=org.apache.log4j.PatternLayout
log4j.appender.OnlyInfo.layout.ConversionPattern=%d [%t] %-7p %10c{1} - %m%n
log4j.appender.OnlyInfo.filter.A=org.apache.log4j.varia.LevelRangeFilter
log4j.appender.OnlyInfo.filter.A.LevelMin=INFO    
log4j.appender.OnlyInfo.filter.A.LevelMax=INFO
log4j.appender.OnlyInfo.filter.A.AcceptOnMatch=true

# Only ERROR
log4j.appender.OnlyError=org.apache.log4j.FileAppender
log4j.appender.OnlyError.File=${appRootPath}/WEB-INF/logs/error.log
log4j.appender.OnlyError.layout=org.apache.log4j.PatternLayout
log4j.appender.OnlyError.layout.ConversionPattern=%d [%t] %-7p %10c{1} - %m%n
log4j.appender.OnlyError.filter.A=org.apache.log4j.varia.LevelRangeFilter
log4j.appender.OnlyError.filter.A.LevelMin=ERROR    
log4j.appender.OnlyError.filter.A.LevelMax=ERROR
log4j.appender.OnlyError.filter.A.AcceptOnMatch=true

Answer 2

The default assumption that is wired in to Log4J is that whenever you care about messages at a certain level (e.g. INFO) then you also care about more important messages too (e.g. ERROR). You can set levels on loggers and thresholds on appenders to filter out messages below a certain level of importance but if you want to send (say) INFO messages to a particular appender but not also WARN, ERROR and FATAL, then the only way to achieve that is by applying a filter.

The properties configuration mechanism doesn't support filters, so you will have to switch to the log4j.xml configuration style. The following configuration example is taken from the Log4J wiki:

    <appender name="info-out" class="org.apache.log4j.FileAppender"> 
            <param name="File" value="info.log"/> 
            <layout class="org.apache.log4j.PatternLayout"> 
                    <param name="ConversionPattern" value="%m%n"/> 
            </layout> 
            <filter class="org.apache.log4j.varia.LevelMatchFilter">
                    <param name="LevelToMatch" value="info" />
                    <param name="AcceptOnMatch" value="true"/>
            </filter>
            <filter class="org.apache.log4j.varia.DenyAllFilter" />
    </appender>