Save error message of subprocess command

Question

When running bash command using subprocess, I might run into situation where the command is not valid. In this case, bash would return an error messsage. How can we catch this message? I would like to save this message to a log file. The following is an example, where I try to list files in a non-existed directory.

try:
    subprocess.check_call(["ls", "/home/non"]) 
    df = subprocess.Popen(["ls", "/home/non"], stdout=subprocess.PIPE)        
    output, err = df.communicate()
    # process outputs
except Exception as error:        
    print error
    sys.exit(1)

Bash would prints "ls: cannot access /home/non: No such file or directory". How can I get this error message? The error caught by the except line is clearly different, it says "Command '['ls', '/home/non']' returned non-zero exit status 2".

Answer 1

"ls: cannot access /home/non: No such file or directory" is generated by ls command, not bash here.

If you want to handle non-existing files using exception handling then use subprocess.check_output():

#!/usr/bin/env python
from subprocess import check_output, STDOUT, CalledProcessError

try:
    output = check_output(['ls', 'nonexistent'], stderr=STDOUT)
except CalledProcessError as exc:
    print(exc.output)
else:
    assert 0

Output

ls: cannot access nonexistent: No such file or directory

Answer 2

You can redirect stderr to a file object:

from subprocess import PIPE, CalledProcessError, check_call, Popen

with open("log.txt", "w") as f:
    try:
        check_call(["ls", "/home/non"], stderr=f)
        df = Popen(["ls", "/home/non"], stdout=PIPE)
        output, err = df.communicate()
    except CalledProcessError as e:
        print(e)
        exit(1)

Output to log.txt:

ls: cannot access /home/non: No such file or directory

If you want the message in the except:

try:
    check_call(["ls", "/home/non"])
    df = Popen(["ls", "/home/non"], stdout=PIPE)
    output, err = df.communicate()
except CalledProcessError as e:
    print(e.message)

For python 2.6 the e.message won't work. You can use a similar version of python 2.7's check_output that will work with python 2.6:

from subprocess import PIPE, CalledProcessError, Popen

def check_output(*args, **kwargs):
    process = Popen(stdout=PIPE, *args, **kwargs)
    out, err = process.communicate()
    ret = process.poll()
    if ret:
        cmd = kwargs.get("args")
        if cmd is None:
            cmd = args[0]
        error = CalledProcessError(ret, cmd)
        error.out = out
        error.message = err
        raise error
    return out

try:
    out = check_output(["ls", "/home"], stderr=PIPE)
    df = Popen(["ls", "/home/non"], stdout=PIPE)
    output, err = df.communicate()
except CalledProcessError as e:
    print(e.message)
else:
    print(out)