subprocess.check_output(): show output on failure

Question

The output of subprocess.check_output() looks like this at the moment:

CalledProcessError: Command '['foo', ...]' returned non-zero exit status 1

Is there a way to get a better error message?

I want to see stdout and stderr.

Answer 1

Redirect STDERR to STDOUT.

Example from the interpreter:

>>> try:
...   subprocess.check_output(['ls','-j'], stderr=subprocess.STDOUT)
... except subprocess.CalledProcessError as e:
...   print('error>', e.output, '<')
...

Will throw:

error> b"ls: invalid option -- 'j'\nTry `ls --help' for more information.\n" <

Explantion

From check_output documentation:

To also capture standard error in the result, use stderr=subprocess.STDOUT

Answer 2

Don't use check_output(), use Popen and Popen.communicate() instead:

>>> proc = subprocess.Popen(['cmd', '--optional-switch'])
>>> output, errors = proc.communicate()

Here output is data from stdout and errors is data from stderr.

Answer 3

In my opinion that a perfect scenario to use sys.excepthook! You just have to filter what you would like to be formatted as you want in the if statement. With this solution, it will cover every exception of your code without having to refract everything!

#!/usr/bin/env python
import sys
import subprocess

# Create the exception handler function
def my_excepthook(type, value, traceback):
    # Check if the exception type name is CalledProcessError
    if type.__name__ == "CalledProcessError":
        # Format the error properly
        sys.stderr.write("Error: " + type.__name__ + "\nCommand: " + value.cmd + "\nOutput: " + value.output.strip())
    # Else we format the exception normally
    else:
        sys.stderr.write(str(value))

# We attach every exceptions to the function my_excepthook
sys.excepthook = my_excepthook

# We duplicate the exception
subprocess.check_output("dir /f",shell=True,stderr=subprocess.STDOUT)

You can modify the output as you wish, here is the actual ouput:

Error: CalledProcessError
Command: dir /f
Output: Invalid switch - "f".

Answer 4

Since I don't want to write more code, just to get a good error message, I wrote subx

From the docs:

subprocess.check_output() vs subx.call()

Look, compare, think and decide what message helps your more.

subprocess.check_output()::

CalledProcessError: Command '['cat', 'some-file']' returned non-zero exit status 1

sub.call()::

SubprocessError: Command '['cat', 'some-file']' returned non-zero exit status 1:
stdout='' stderr='cat: some-file: No such file or directory'

... especially if the code fails in a production environment where reproducing the error is not easy, subx can call help you to spot the source of the failure.