Log to different file with log4cxx

Question

I want to log to different files in my code.

How can i do that in Log4cxx with xml configuration or programatically in code...

• Suppose that I have 1.k,k+1,..n components.
• They run in the same application
• I want component k log to Logger-k, k+1 component log to Logger-k+1 at the same time

Update:

Logger.addAppender() approach:

log4cxx::helpers::Pool p; 

std::string paramAppender = "appxNormalAppender";
std::string paramFileName = "\\Logs\\MyLog.txt";

LOG4CXX_DECODE_CHAR(logAppender, paramAppender );
LOG4CXX_DECODE_CHAR(logFileName, paramFileName );


FileAppenderPtr fileAppender = 
logger->getLoggerRepository()->getRootLogger()->getAppender(logAppender);

if(fileAppender!= NULL)
{


    fileAppender->setFile(logFileName);

    fileAppender->activateOptions(p);

}

This does not work because

Suppose that i set FileName to Logger-k for k component, it logs to Logger-k, then i set file name to Logger-k+1 for compoent k+1, then both component k, and k+1 log the same loggerk+1 file. It seems last file name overrides or effects all others...

And

All compoenent compenent 1, ...compoeent k, componentk+1,.... component n are in the same application...

Answer 1

Create Each of the Loggers as normal, and then for each logger add a FileAppender set to the desired file via the Logger.addAppender() method.

Answer 2

This is another solution for dynamic logging using thread context in log4cxx with class MDC, after big frustration with current solutions. It basically allows to have an instanceid prefix added to the File property of a FileAppender. It requires a custom FileAppender to be properly compiled and defined before loading of a configuration file. Works with usual old log4cxx 0.10.0 version. It can be used with the following snippet:

#include <XFileAppender.h>

void PushInstance(const std::string &instanceid)
{
    MDC::put("__ITRInstanceId", value);
}

void PopInstance()
{
    MDC::remove("__ITRInstanceId");
}

int main()
{
    {
        auto logger = Logger::getLogger("Test1");
        PushInstance("user1");
        logger->log(log4cxx::Level::getInfo(), "Info");
        PopInstance();
    }

    {
        auto logger = Logger::getLogger("Test2");
        PushInstance("user1");
        logger->log(log4cxx::Level::getWarn(), "Warning");
        PopInstance();
    }

    // Imagine "Test1" and "Test2" used from different
    // threads acting as "user1"

    // Following line will ensure writers will be closed when "user1"
    // e.g. is logging out
    XFileAppender::CloseIstanceWriters("instance1");
    return 0;
}

Can be used with this property configuration file:

# Default log filename. Can be overridden by system properties
LogFilename=File.log

# Appenders
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d{dd/MM/yyyy HH:mm:ss}] %-6p: %m%n

log4j.appender.USER=org.apache.log4j.XFileAppender
log4j.appender.USER.File=${LogFilename}
log4j.appender.USER.layout=org.apache.log4j.PatternLayout
log4j.appender.USER.layout.ConversionPattern=[%d{dd/MM/yyyy HH:mm:ss}] %-6p: %m%n

# Root
log4j.rootLogger=WARN, stdout

# Classes
log4j.logger.Test1=INFO, USER
log4j.additivity.Test1 = false
log4j.logger.Test2=INFO, USER
log4j.additivity.Test2 = false

XFileAppender header:

// Copyright (c) 2019 Francesco Pretto
// This file is subject to the Apache License Version 2.0

#pragma once

#include "libdefs.h"
#include <memory>
#include <unordered_map>
#include <mutex>
#include <log4cxx/fileappender.h>

namespace log4cxx
{
    class ITR_LOGGING_SHARED_API XFileAppender : public FileAppender
    {
    private:
        struct InstanceWriter
        {
            typedef std::shared_ptr<InstanceWriter> Ptr;
            InstanceWriter(helpers::Pool& p);
            helpers::Mutex Mutex;
            helpers::WriterPtr Writer;
        };
        typedef std::unordered_map<LogString, XFileAppender *> AppenderIdentities;
    public:
        DECLARE_LOG4CXX_OBJECT(XFileAppender)
        BEGIN_LOG4CXX_CAST_MAP()
            LOG4CXX_CAST_ENTRY(XFileAppender)
            LOG4CXX_CAST_ENTRY_CHAIN(FileAppender)
        END_LOG4CXX_CAST_MAP()

        XFileAppender();

    public:
        void close() override;
        void append(const spi::LoggingEventPtr &event, helpers::Pool &p) override;
        void activateOptions(helpers::Pool &p) override;

    public:
        /**
        Clear all registered writers

        NOTE: It doesn't close them. This is useful example when reloading configuration
         */
        static void ClearWriters();

        /**
        Close all writers linked to instance
         */
        static void CloseIstanceWriters(const LogString &instanceid);

    private:
        InstanceWriter::Ptr getInstanceWriter(helpers::Pool &p);
        void CloseWriter(const LogString &istanceid);
        void closeWriter(InstanceWriter &writer);
        void closeWriter(helpers::Writer &writer);
        void closeWriters();
        helpers::WriterPtr createWriter(const LogString &instanceid, helpers::Pool& p);
        helpers::WriterPtr createWriter(const File &file, helpers::Pool& p);
        static void removeAppenderIstances(const LogString &appname, std::vector<LogString> &instanceIds);

    private:
        XFileAppender(const XFileAppender&);
        XFileAppender& operator=(const XFileAppender&);

    private:
        static helpers::Mutex s_mutex;
        static std::unordered_map<LogString, AppenderIdentities> s_appenderIdentities; // NOTE: Guarded by s_mutex
    private:
        bool m_failedWriter;                                                           // NOTE: Guarded by mutex
        helpers::WriterPtr m_writer;                                                   // NOTE: Guarded by mutex
        std::unordered_map<LogString, InstanceWriter::Ptr> m_instanceWriters;          // NOTE: Guarded by mutex

    }; // class XFileAppender
    LOG4CXX_PTR_DEF(XFileAppender);

}  // namespace log4cxx

XFileAppender source:

// Copyright (c) 2019 Francesco Pretto
// This file is subject to the Apache License Version 2.0

#include "XFileAppender.h"
#include <log4cxx/helpers/synchronized.h>
#include <log4cxx/helpers/transcoder.h>
#include <log4cxx/helpers/loglog.h>
#include <log4cxx/helpers/stringhelper.h>
#include <log4cxx/helpers/bufferedwriter.h>
#include <log4cxx/helpers/bytebuffer.h>
#include <log4cxx/helpers/fileoutputstream.h>

using namespace std;
using namespace log4cxx;
using namespace log4cxx::spi;
using namespace log4cxx::helpers;

static Pool s_pool;
Mutex XFileAppender::s_mutex(s_pool);
unordered_map<LogString, XFileAppender::AppenderIdentities> XFileAppender::s_appenderIdentities;

XFileAppender::XFileAppender()
{
    m_failedWriter = false;
}

void XFileAppender::close()
{
    synchronized sync(mutex);
    if (closed)
        return;

    closeWriters();
    closed = true;
}

void XFileAppender::append(const LoggingEventPtr &event, Pool &p)
{
    InstanceWriter::Ptr instance;
    {
        synchronized sync(mutex);
        if (closed)
            return;

        instance = getInstanceWriter(p);
        if (instance == nullptr)
        {
            // Try to use non instanced writer
            if (m_failedWriter)
                return;

            if (m_writer == nullptr)
            {
                m_writer = createWriter(LogString(), p);
                if (m_writer == nullptr)
                {
                    m_failedWriter = true;
                    return;
                }
            }

            LogString msg;
            layout->format(msg, event, p);
            m_writer->write(msg, p);

            if (getImmediateFlush())
                m_writer->flush(p);

            return;
        }
    }

    // NOTE: From now, we can release the appender istance lock

    if (instance->Writer == nullptr)
    {
        // This is a failed writer
        return;
    }

    LogString msg;
    layout->format(msg, event, p);

    synchronized syncWriter(instance->Mutex);
    instance->Writer->write(msg, p);

    if (getImmediateFlush())
        instance->Writer->flush(p);
}

void XFileAppender::activateOptions(helpers::Pool &p)
{
    synchronized syncWriter(mutex);
    closeWriters();
    // Do nothing more. We lazily create writers later
}

void XFileAppender::ClearWriters()
{
    synchronized lock(s_mutex);
    s_appenderIdentities.clear();
}

void XFileAppender::closeWriter(InstanceWriter &writer)
{
    synchronized syncWriter(writer.Mutex);
    // If it's a valid writer. It could be a failed one 
    if (writer.Writer != nullptr)
        closeWriter(*writer.Writer);
}

// Stripped from WriterAppender.cpp
void XFileAppender::closeWriter(Writer &writer)
{
    try
    {
        // before closing we have to output out layout's footer
        // NOTE: Using the object's pool since this is a one-shot operation and
        // pool is likely to be reclaimed soon when appender is destructed.
        if (layout != NULL)
        {
            LogString foot;
            layout->appendFooter(foot, pool);
            writer.write(foot, pool);
        }

        writer.close(pool);
    }
    catch (IOException& e)
    {
        LogLog::error(LogString(LOG4CXX_STR("Could not close writer for WriterAppender named ")) + name, e);
    }
}

void XFileAppender::closeWriters()
{
    vector<LogString> instancesToDelete;
    for (auto &pair : m_instanceWriters)
    {
        auto &writer = pair.second;
        closeWriter(*writer->Writer);
        instancesToDelete.push_back(pair.first);
    }

    removeAppenderIstances(getName(), instancesToDelete);
    m_instanceWriters.clear();

    if (m_writer != nullptr)
    {
        closeWriter(*m_writer);
        m_writer = nullptr;
    }
}

// Stripped from FileAppender.cpp
WriterPtr XFileAppender::createWriter(const LogString &instanceid, helpers::Pool& p)
{
    LogString fileName = getFile();
    if (fileName.empty())
    {
        LogLog::error(LogString(LOG4CXX_STR("File option not set for appender ["))
            + name + LOG4CXX_STR("]."));
        LogLog::warn(LOG4CXX_STR("Are you using FileAppender instead of ConsoleAppender?"));
        return nullptr;
    }

    File file(fileName);
    if (instanceid.length() != 0)
    {
        auto name = file.getName();
        auto parent = file.getParent(p);
#if WIN32
        file = parent + LOG4CXX_STR("\\") + instanceid + LOG4CXX_STR("_") + name;
#else
        file = parent + LOG4CXX_STR("/") + instanceid + LOG4CXX_STR("_") + name;
#endif
    }

    try
    {
        return createWriter(file, p);
    }
    catch (IOException& e)
    {
        LogString msg(LOG4CXX_STR("createWriter("));
        msg.append(fileName);
        msg.append(1, (logchar)0x2C /* ',' */);
        StringHelper::toString(fileAppend, msg);
        msg.append(LOG4CXX_STR(") call failed."));
        errorHandler->error(msg, e, ErrorCode::FILE_OPEN_FAILURE);
        return nullptr;
    }
}

// Stripped from FileAppender.cpp
WriterPtr XFileAppender::createWriter(const File &outFile, helpers::Pool& p)
{
    bool append = getAppend();
    bool writeBOM = false;

    if (StringHelper::equalsIgnoreCase(getEncoding(),
        LOG4CXX_STR("utf-16"), LOG4CXX_STR("UTF-16")))
    {
        // don't want to write a byte order mark if the file exists
        if (append)
        {
            writeBOM = !outFile.exists(p);
        }
        else
        {
            writeBOM = true;
        }
    }

    OutputStreamPtr outStream;

    try
    {
        outStream = new FileOutputStream(outFile.getPath(), append);
    }
    catch (IOException& ex)
    {
        LogString parentName = outFile.getParent(p);

        if (!parentName.empty())
        {
            File parentDir;
            parentDir.setPath(parentName);

            if (!parentDir.exists(p) && parentDir.mkdirs(p))
            {
                outStream = new FileOutputStream(outFile.getPath(), append);
            }
            else
            {
                throw;
            }
        }
        else
        {
            throw;
        }
    }

    // if a new file and UTF-16, then write a BOM
    if (writeBOM)
    {
        char bom[] = { (char)0xFE, (char)0xFF };
        ByteBuffer buf(bom, 2);
        outStream->write(buf, p);
    }

    WriterPtr newWriter(WriterAppender::createWriter(outStream));

    if (getBufferedIO())
    {
        newWriter = new BufferedWriter(newWriter, getBufferSize());
    }

    if (layout != NULL)
    {
        LogString header;
        layout->appendHeader(header, p);
        newWriter->write(header, p);
    }

    return newWriter;
}

void XFileAppender::removeAppenderIstances(const LogString &appname, vector<LogString> &instanceIds)
{
    synchronized lock(s_mutex);
    if (s_appenderIdentities.size() == 0)
        return;

    for (auto &instanceid : instanceIds)
    {
        auto found = s_appenderIdentities.find(instanceid);
        if (found == s_appenderIdentities.end())
            break;

        found->second.erase(appname);
        if (found->second.size() == 0)
        {
            // All appenders for this instance were closed
            s_appenderIdentities.erase(found);
        }
    }
}

XFileAppender::InstanceWriter::Ptr XFileAppender::getInstanceWriter(Pool &p)
{
    LogString instanceid = MDC::get(LOG4CXX_STR("__ITRInstanceId"));
    if (instanceid.length() == 0)
        return nullptr;

    auto &writer = m_instanceWriters[instanceid];
    if (writer == nullptr)
    {
        // NOTE: We must use instance pool here otherwise there are
        // crashes, don't know exactly why
        writer.reset(new InstanceWriter(pool));
        writer->Writer = createWriter(instanceid, p);
        synchronized lock(s_mutex);
        auto &appenders = s_appenderIdentities[instanceid];
        appenders[getName()] = this;
    }

    return writer;
}

void XFileAppender::CloseIstanceWriters(const LogString &instanceid)
{
    synchronized lock(s_mutex);
    auto found = s_appenderIdentities.find(instanceid);
    if (found == s_appenderIdentities.end())
        return;

    for (auto &pair : found->second)
    {
        auto appender = pair.second;
        appender->CloseWriter(instanceid);
    }

    s_appenderIdentities.erase(found);
}

void XFileAppender::CloseWriter(const LogString &istanceid)
{
    synchronized sync(mutex);
    auto found = m_instanceWriters.find(istanceid);
    closeWriter(*found->second);
    m_instanceWriters.erase(found);
}

XFileAppender::InstanceWriter::InstanceWriter(Pool &p)
    : Mutex(p) { }

IMPLEMENT_LOG4CXX_OBJECT(XFileAppender)

Answer 3

For dynamic components try this:

#include <boost/foreach.hpp>
#include <boost/thread.hpp>
#include <boost/bind.hpp>
#include <boost/lexical_cast.hpp>

#include <log4cxx/Logger.h>
#include <log4cxx/LogManager.h>
#include <log4cxx/xml/domconfigurator.h>
#include <log4cxx/FileAppender.h>
#include <log4cxx/SimpleLayout.h>
#include <log4cxx/helpers/transcoder.h>


void func(int k) {
    std::string strName = "Log." + boost::lexical_cast<std::string>(k);
    log4cxx::LoggerPtr log = log4cxx::Logger::getLogger(strName);
    LOG4CXX_DECODE_CHAR(fileName, strName + ".log");
    log4cxx::FileAppenderPtr appender(new log4cxx::FileAppender(new log4cxx::SimpleLayout, fileName, false));
    log->addAppender(appender);

    LOG4CXX_INFO(log, strName);

    log->removeAppender(appender);
}

int main(int argc, char * argv[]) {
    log4cxx::xml::DOMConfigurator::configure("trace.xml");
    if(log4cxx::Logger::getLogger("Log")->getAllAppenders().empty()) {
        std::cout << "failed to config log4cxx" << std::endl;
        return 1;
    }
    log4cxx::LoggerPtr log = log4cxx::Logger::getLogger("Log");

    boost::thread_group threadGroup;
    for(int k = 0; k != 3; ++k) {
        threadGroup.create_thread(boost::bind(func, k));
    }
    threadGroup.join_all();
    return 0;
}

with simple trace.xml

<?xml version="1.0" encoding="UTF-8" ?>
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
    <appender name="ConsoleAppender" class="org.apache.log4j.ConsoleAppender"> 
        <param name="Target" value="System.out"/> 
        <layout class="org.apache.log4j.SimpleLayout"/> 
    </appender>

    <root>
        <level value="all"/>
    </root> 

    <category name="Log">
        <level value="INFO"/>
        <appender-ref ref="ConsoleAppender"/>
    </category>
</log4j:configuration>

Answer 4

You do need to create separate appenders for you loggers. In my example i've created afile1 and afile2 appenders. I have also created two loggers : my.logger1 and my.logger2. When you use my.logger1 it logs to mylog1 file, When you use my.logger2 it logs to mylog2 file.

Here is my log.properties file:

log4cplus.appender.afile1.layout=log4cplus::PatternLayout
log4cplus.appender.afile1.layout.ConversionPattern=%d [ %c{1} ] [ %-5p ] %m%n
log4cplus.appender.afile1=log4cplus::RollingFileAppender
log4cplus.appender.afile1.File=mylog1.log
log4cplus.appender.afile1.MaxFileSize=5MB
log4cplus.appender.afile1.MaxBackupIndex=2

log4cplus.appender.afile2.layout=log4cplus::PatternLayout
log4cplus.appender.afile2.layout.ConversionPattern=%d [ %c{1} ] [ %-5p ] %m%n
log4cplus.appender.afile2=log4cplus::RollingFileAppender
log4cplus.appender.afile2.File=mylog2.log
log4cplus.appender.afile2.MaxFileSize=5MB
log4cplus.appender.afile2.MaxBackupIndex=2

log4cplus.logger.my.logger1=INHERIT, afile1
log4cplus.additivity.my.logger1=false
log4cplus.logger.my.logger2=INHERIT, afile2
log4cplus.additivity.my.logger2=false 

Here is an example programm:
example.cpp:

#include <iostream>
#include <log4cplus/logger.h>
#include <log4cplus/loglevel.h>
#include <log4cplus/configurator.h>
#include <log4cplus/fileappender.h>

#define MYLOG1_INFO(logEvent) LOG4CPLUS_INFO (log4cplus::Logger::getInstance("my.logger1"), logEvent)
#define MYLOG2_INFO(logEvent) LOG4CPLUS_INFO (log4cplus::Logger::getInstance("my.logger2"), logEvent)

int main(int argc, char**argv)
{
    try
    {
        log4cplus::PropertyConfigurator::doConfigure("log.properties");
    }
    catch( ... )
    {
    std::cerr<<"Exception occured while opening log.properties\n";
    return -1;
    }
    MYLOG1_INFO("hello world!");
    MYLOG2_INFO("hello world!");
    return 0;
}

Here is Makefile( i suppose my log4cplus is built in parent dirrectory):

CXXFLAGS+=-I$(abspath ../log4cplus-1.0.4/include)
all: example.o
    $(CXX) $^ $(abspath ../log4cplus-1.0.4/src/.libs/liblog4cplus.a) -lpthread -o test

Try this example and you should understand how appenders works

Log4cplus is mostly like log4j. so you can read basic principles log4j. And To get classes names you got to visit log4cplus.sourceforge.net

About log format. Documentation for log4cplus is available only in doxygen . so here you can read about formating in pattern layout
And if you want to log process id , than you should use %i in your layout conversion pattern
example:

...
log4cplus.appender.afile2.layout.ConversionPattern=[%i] %m%n
...

It will log process id and message