Logging to JSON

Question

I'm wondering if there's an implementation of slf4j that logs into a JSON format. Where each log messages would be a JSON object, 1 per row.

e.g. each line of the log file would look like something this:

{"severity":"WARN", "ts":12345678, "host":"myhostname.com", "message":"Failed to do something"}

Answer 1

I was using Log4j2 and I had a similar requirement. This helped me -

https://github.com/maartenbosteels/log4j-jsonevent-layout

log4j2.xml:

<Console name="Console" target="SYSTEM_OUT">
    <JSONEventLayoutV1>
</Console>

I had to add the following execution to my maven-compiler-plugin build plugin:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <version>3.1</version>
  <executions>
    <execution>
      <id>log4j-plugin-processor</id>
      <goals>
        <goal>compile</goal>
      </goals>
      <phase>process-classes</phase>
      <configuration>
        <proc>only</proc>
        <annotationProcessors>
          <annotationProcessor>org.apache.logging.log4j.core.config.plugins.processor.PluginProcessor</annotationProcessor>
        </annotationProcessors>
      </configuration>
    </execution>
  </executions>
</plugin>

Answer 2

If you are using Logback as the backend, check out logstash-logback-encoder. It contains encoders to log in json format.

You can include it as a Maven dependency:

<dependency>
  <groupId>net.logstash.logback</groupId>
  <artifactId>logstash-logback-encoder</artifactId>
  <version>4.4</version>
</dependency>

And in your logback.xml configuration file, include the encoder. RollingFileAppender example used with LogstashEncoder mentioned below (Source - logstash-logback-encoder#encoder):

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <appender name="stash" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
      <level>info</level>
    </filter>
    <file>/some/path/to/your/file.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>/some/path/to/your/file.log.%d{yyyy-MM-dd}</fileNamePattern>
      <maxHistory>30</maxHistory>
    </rollingPolicy>
    <encoder class="net.logstash.logback.encoder.LogstashEncoder" />
  </appender>
  <root level="all">
    <appender-ref ref="stash" />
  </root>
</configuration>

Answer 3

If you are using Log4j as backend:

• https://github.com/michaeltandy/log4j-json
• https://github.com/Aconex/json-log4j-layout
• https://github.com/bfrancojr/Java-JSON-Tools/tree/master/jsontools-log4j

If you are using JUL (java.util.logging) as backend:

• https://github.com/SYNAXON/logstash-util-formatter

Note that they all format a single log event as JSON object. No valid JSON array is created.

Answer 4

Ok, this thread needs to be updated a bit since most of the answers are quite old. All modern java log liibraries support json layout out of the box. With log4j2 you don't even need any additional libs, just jackson in your classpath, which you most likely already have.

If this is not enough, you can also take a look at this repo: https://github.com/savoirtech/slf4j-json-logger

Answer 5

I guess this is what you are looking for: http://jsontools.berlios.de/articles/faq/what-is-in-log4j.html

I haven't used this before but on the page i mentioned says:

It contains a formatter to format log entries in the JSON format. You can install it in your log system and then afterwards parse the log stream using the core tools.

Answer 6

Some time ago I wrote a MongoDBAppender for Logback (see also LBCLASSIC-261). As the name implies it uses MongoDB to store logging events, but since MongoDB facilitates JSON-like document store, you might find it useful:

{
    "_id" : ObjectId("4d9cbcbf7abb3abdaf9679cd"),
    "timeStamp" : ISODate("2011-04-06T19:19:27.006Z"),
    "level" : "ERROR",
    "thread" : "main",
    "logger" : "ch.qos.logback.classic.db.mongo.MongoDBAppenderTest",
    "message" : "D"
}