java logging : multiple small file vs one big

问题:

Using log4j on Unix, which Appender would perform the best to write 1000Meg :

1) Using RollingFileAppender writing 10 file of 100 Meg

or

2) Using a FileAppender and writing a single 1000Meg file

In other words, using java on unix, does the size matter?

Thank you

回答1:

There no Java-side performance difference between writing to a small file or writing to a large file. There might be a small difference at the OS level when a file gets big enough that an extra level of index blocks is required (FS dependent), but it is probably not worth worrying about.

There will be a performance cost in implementing the file rolling behavior. The appender has to:

• test / remember how big the file is,
• close the current one,
• rename it,
• open a new file.

My gut feeling is that this is not likely to be significant. (However, it would be worth measuring to see if the performance impact should be a concern. Also, you should probably ask yourself if you are not doing too much logging.)

You have to compare all of the above against the advantages of file rolling:

• Having a bounded size on log files means that your logging won't fill the disk, causing problems for the application and potentially others on the same machine.
• Smaller log files can make it easier / quicker to do searches for events at specific times. (Running less on a 1000Mb file can be painful ...)

回答2:

They'll both easily write 1000MB files. I don't see why they should perform differently.

You do need the RollingFileAppender though in order to set the total maximum size that the log file(s) can reach. Otherwise you may run out of hard disk space, assuming that you application has got traffic.

回答3:

I think it's always preferable to use small files than large files because they are more manageable. Also, consider that with large files you may have problems in the case of file-system full because of the risk of having to remove the log file when the process is up and running to free up disk space.