perl: Will Log::Log4perl work in a multi-threaded

Question

I'm using logging vai Log::Log4perl in my perl script; I'm wondering if making multiple calls to write to the same log object will cause incorrect / erroneous behavior.

I'm using a Appender::File to write out the log in the following manner:

$log->info("Launching commands...");
foreach my $param (@params) {
    push @thread_handles, async {
        system("$param");
        $log->info("$param COMPLETE");
        logstatus($?);
    };
}

$_->join() foreach @thread_handles;
$log->info("Commands completed...");

Answer 1

The Log4perl is working in threaded enviroment, however you have to choose the appenders carefully.

Except the synced appenders the logs messages with buffering won't appear to be in order, but this is not a problem.

Either use different files for each thread or add the pid to the logmessage.

The synced appenders may cause a lot of overhead in your threaded application, use it carefully.

I opt for a single logfile with pid or some kind of thread identifier. I am using this kind of logging without a problem.

regards,

Answer 2

Using the Synchronized appender makes a lot of sense. My question is: Will the logger variable be passed around to the threads? According to perlthrtut it will not. I have yet to verify this. Unless somebody's already done it?

You may be tempted to use threads::shared to share the logger variable but the threads::shared documentation specifies that you can only share scalars, arrays, or hashes. I've tried that anyway with perl 5.8.8 and, as expected, it will not work.

The other approach would be to create a separate logger for each subroutine that will be invoked in a thread. The log4perl appenders can be configured to avoid locking and interleaving. But I am seriously concerned about the performance impact incurred with generating a separate logger instance for each active thread.

Updated: It turns out one doesn't have to get complicated. If you initialize log4perl as if you are writing a single thread script and you call the logger objects methods without any special tricks, everything works as advertised. The logger object doesn't have to be passed to thread entry point. The sub invoked in the thread entry point gets access to the logger methods the same way as if it were called the regular way. The Synchronized appender keeps everything lines up.

Answer 3

The Log::Log4perl with the default file based appender will work, but some overlapping may occur in a multi-threaded or multi-processed environment using the same log file.

One solution is to use Log::Log4perl::Appender::Synchronized as an appender. See How can I run Log::Log4perl under mod_perl? in the FAQ for more info.