Are files defined in the log section of a snakemak

Question

As I understand the documentation for the log section of a snakemake rule, one has to "manually" send things to the log files. It seems to me that one could achieve the same results using files defined in the output section.

What are the important differences between these two possible approaches?

What is the real usefulness of the log section?

Answer 1

For me the best pratice for log is Snakemake is like that :

rule example1:
  input:
    file = <input>

  log: 
    out = '_stdout.log',
    err = '_stderr.err'

  output:
    <output>

  shell: 
    'Script/Tool {input.file} 2> {log.err} 1> {log.out}'

The log section is very useful I think. Most programs or tools produce some logs on standard out and standard error.This is useful for the user to know at which step of the tool or program it fails.

Of course you can do it on the output section like the following code :

rule example2:
  input:
    file = <input>

  output:
    file = <output>
    out = '_stdout.log',
    err = '_stderr.err'

  shell: 
    'Script/Tool {input.file} 2> {output.err} 1> {output.out}'

This will produce the same results as the example1 rule. But the purpose of output section is to make dependencies with other rules or just provide the results files you needed. In most cases, logs aren't these files, unless in a rule to check some parameters or files.

There is one big disadvantage to put the log on output. When a rule in Snakemake fails, Snakemake delete all the output which might be corrupted by the fail. So your log will be deleted too, and you might not be able to see at which step of the program it fails or the reason of it.

Hugo