Secure / Encrypt log4j files

问题:

I have a problem ; security requirement i have java swing app that have logging files generated with log4j for support issues in case of tracking a bug.

I have to ecrypt/cypher/secure the files so the client cant open them and see them (at least not as human readable way) and at the same time when support tech team take these files they will know how to read (decrypt) them .

I did a lot of searches and i tried my best option i found which is build custom appender by extending SkeletonAppender .

Now know that i have log4j working great as below configuration, but i created new class to encrypt it but i cant get it work even with simple setup it dose not create the file , so i can continue in the ecnryption part.

Any help , links are good.

Working...version

<appender name="cache" class="com.MyAppender">  
            <param name="Threshold" value="ALL" />
            <param name="ImmediateFlush" value="true" />  
            <param name="File" value="${home}/logs/cache.log"/> 
            <param name="Append" value="true"/>
            <param name="Threshold" value="ALL" />
            <param name="Encoding" value="UTF-8" />

            <layout class="org.apache.log4j.EnhancedPatternLayout">
            <param name="ConversionPattern" value="%-5p %d{MMM-dd-yyyy HH:mm:ss,SSS} %c{1} - %m%n" />
        </layout>
    </appender>

Not Working...version

   <appender name="cache" class="com.MyAppender">   
            <param name="Threshold" value="ALL" />
            <param name="ImmediateFlush" value="true" />  
            <param name="File" value="${home}/logs/cache.log"/> 
            <param name="Append" value="true"/>
            <param name="Threshold" value="ALL" />
            <param name="Encoding" value="UTF-8" />

            <rollingPolicy class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
                 <param name="FileNamePattern"
            value="${home}/logs/cache.%d{yyyy-MM-dd-HH}.gz" />
                 <param name="ActiveFileName" value="${home}/logs/cache.log" />
             </rollingPolicy> 

            <layout class="org.apache.log4j.EnhancedPatternLayout">
            <param name="ConversionPattern"
                value="%-5p %d{MMM-dd-yyyy HH:mm:ss,SSS} %c{1} - %m%n" />
        </layout>
    </appender>

The simple class test

http://www.javaworld.com/article/2075817/core-java/customized-appender-extending-org-apache-log4j-fileappender.html

package com.MyAppender;

import org.apache.log4j.spi.LoggingEvent;

public class MyAppender extends org.apache.log4j.RollingFileAppender {

    private String file;
    private boolean initialized = false;
    private String baseFileName = null;

    // private static final Log log = LogFactory.getLog(MyAppender.class);

    /**
     * 
     * write to ActivityLog
     * 
     * @param event
     *            logging event invoked.
     * 
     */
    @Override
    protected void subAppend(LoggingEvent event) {
        if (!initialized) {
            createNewFile();
        }
        synchronized (this) {
            super.subAppend(event);
        }
    }

    /**
     * 
     * create a new ActivityLog File
     * 
     */
    public void createNewFile() {
        try {
            baseFileName = file;
            super.setFile(baseFileName);
            super.activateOptions();
            initialized = true;
        } catch (Exception e) {
            // log.error("*#*Error in configuration of log4j params,unable to create ActivityLog file");
        }
    }

    /**
     * 
     * invokes File Appender's activateOptions() which controls the creation of
     * log files.
     * 
     */
    @Override
    public void activateOptions() {
        super.setFile(file);
        super.activateOptions();
    }

    /**
     * 
     * Close and rename the current ActivityLog file and reset counter and
     * timestamp.
     * 
     */
    public void rollOver() {
        closeFile();
        initialized = false;
    }

    @Override
    public void setFile(String file) {
        this.file = file;
    }

}

Then i plan to implement the code in Cipher OutputStream

回答1:

A possible workaround to the problem is to write the logs to an embedded database that supports encryption, e.g. H2 natively supports encryption and SQLite has open source encryption extensions - this way you can just use the JDBCAppender and let the database take care of encryption without having to worry about a custom appender.

---

From this question, SQLite config would look something like

<appender name="jdbcAppender" class="org.apache.log4j.jdbc.JDBCAppender">
    <param name="URL" value="jdbc:sqlite:D:/download/mapLogic/sf_log.db" />
    <param name="user" value="" />
    <param name="password" value="" />
    <param name="driver" value="org.sqlite.JDBC" />
    <param name="sql"
        value="INSERT INTO Log(Message,Priority,Logger,Date) VALUES ('%m','%p','%c','%d{ABSOLUTE}')" />
</appender>

where your log table looks like

CREATE TABLE Log (
    LogId        INTEGER PRIMARY KEY,
    Date         DATETIME NOT NULL,
    Level        VARCHAR(50) NOT NULL,
    Logger       VARCHAR(255) NOT NULL,
    Message      TEXT DEFAULT NULL
);

Documentation on the JDBCAppender can be found here

---

There's an official encryption extension for SQLite as well as at least one third party open source extension; I've never had to encrypt SQLite, but if I had to do so then I'd go with the official extension unless I ran into problems with it.

---

If you're running this on the client, then ideally you'll be able to have the program phone home at boot time to get the database encryption key so that the key never exists on the client's disk drive (ignoring the possibility that it goes to the swap file) - the client could still use a debugger or whatever to try to get the key out of memory, but presumably they're not interested enough in decrypting the logs to go to that amount of trouble. If you've got to store the key on the client side then you can at a minimum obfuscate it by hashing it several times before using it, e.g. hard-code the base_key in the program, then at boot time you create actual_key by running base_key through SHA512 (or whatever) several times; the client could still figure out what you're doing by using a debugger, but again they hopefully won't want to go to the trouble.

回答2:

Option 1: Use a Custom SocketAppender

As an alternative to Zim-Zam's answer about using a JDBC-capable appender (remember to enable secure transport as well, by the way, if you go down this route), you could also look into using a SocketAppender and roll out your own encryption method.

Option 2: Use Flume and a FlumeAppender

Refer to the log4j documentation on appenders and have a look at using a FlumeAppender, which supports event encryption:

A sample FlumeAppender configuration that is configured with a primary and a secondary agent, compresses the body, formats the body using the RFC5424Layout, and persists encrypted events to disk. This sample "compresses the body, formats the body using the RFC5424Layout, and persists encrypted events to disk:"

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn" name="MyApp" packages="">
  <Appenders>
    <Flume name="eventLogger" compress="true" type="persistent" dataDir="./logData">
      <Agent host="192.168.10.101" port="8800"/>
      <RFC5424Layout enterpriseNumber="18060" includeMDC="true" appName="MyApp"/>
      <Property name="keyProvider">MySecretProvider</Property>
    </Flume>
  </Appenders>
  <Loggers>
    <Root level="error">
      <AppenderRef ref="eventLogger"/>
    </Root>
  </Loggers>
</Configuration>

---

Interesting Reads

This does not answer your question directly, but is fairly interesting as well: Creating an encrypted log file

回答3:

I think you are looking something like this. Though I won't suggest to use it. If you are shipping key with the code, there are plenty de-compilers to decompile jar/class files and get "key". Otherwise you should use PKI and all..? I haven't look into that option here.

extend RollingFileAppender class

public class EncryptedRollingFileAppender extends RollingFileAppender

since RollingFileAppender class not final class so you can do.

overwrite method

public synchronized void setFile(String fileName, boolean append,
        boolean bufferedIO, int bufferSize) throws IOException

something like this.

        LogLog.debug("setFile called: " + fileName + ", " + append);

    // It does not make sense to have immediate flush and bufferedIO.
    if (bufferedIO) {
        setImmediateFlush(false);
    }

    reset();
    OutputStream ostream = null;
    try {
        //
        // attempt to create file
        //
        // ostream = new FileOutputStream(fileName, append);
        ostream = this.createEncryptedOutputStream(fileName, append);
    } catch (FileNotFoundException ex) {
        //
        // if parent directory does not exist then
        // attempt to create it and try to create file
        // see bug 9150
        //
        String parentName = new File(fileName).getParent();
        if (parentName != null) {
            File parentDir = new File(parentName);
            if (!parentDir.exists() && parentDir.mkdirs()) {
                // ostream = new FileOutputStream(fileName, append);
                try {
                    ostream = this.createEncryptedOutputStream(fileName, append);
                } catch (Exception e) {
                    e.printStackTrace();
                } 
            } else {
                throw ex;
            }
        } else {
            throw ex;
        }
    } catch (Exception e) {
        throw new FileNotFoundException();
    }
    Writer fw = createWriter(ostream);
    if (bufferedIO) {
        fw = new BufferedWriter(fw, bufferSize);
    }
    this.setQWForFiles(fw);
    this.fileName = fileName;
    this.fileAppend = append;
    this.bufferedIO = bufferedIO;
    this.bufferSize = bufferSize;
    writeHeader();
    LogLog.debug("setFile ended");

    if (append) {
        File f = new File(fileName);
        ((CountingQuietWriter) qw).setCount(f.length());
    }

most of the code is copied from base class.

encryption private method should look something like this

    private OutputStream createEncryptedOutputStream(String filename, boolean append) throws FileNotFoundException, 
                                                                            NoSuchAlgorithmException, 
                                                                            NoSuchPaddingException, 
                                                                            InvalidKeyException, 
                                                                            InvalidAlgorithmParameterException {
    CipherOutputStream cstream = null;

    try {
        byte[] keyBytes = "1234123412341234".getBytes();  //example
        final byte[] ivBytes = new byte[] { 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 
                 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f }; //example

        final SecretKey key = new SecretKeySpec(keyBytes, "AES");
        final IvParameterSpec IV = new IvParameterSpec(ivBytes);
        final Cipher cipher = Cipher.getInstance("AES/CFB8/NoPadding");
        cipher.init(Cipher.ENCRYPT_MODE, key, IV);

        cstream = new CipherOutputStream(new FileOutputStream(filename, append), cipher);
    } catch (FileNotFoundException e) {
        throw e;
    }
    return (cstream);
}

You won't loose any capabilities of RollingFileAppender. You can do similar coding @ other appenders as well.

Disclaimer :- Please don't these keys @ production, if you are using it. Again, you are shipping keys with your jar. A smart hacker can hack things easily.Fine tune of code to be done since it is 'logging'. I haven't tested this code.