Python, subprocess, call(), check_call and returnc

Question

I've figured out how to use call() to get my python script to run a command:

import subprocess

mycommandline = ['lumberjack', '-sleep all night', '-work all day']
subprocess.call(mycommandline)

This works but there's a problem, what if users don't have lumberjack in their command path? It would work if lumberjack was put in the same directory as the python script, but how does the script know it should look for lumberjack? I figured if there was a command-not-found error then lumberjack wouldn't be in the command path, the script could try to figure out what its directory is and look for lumberjack there and finally warn the user to copy lumberjack into one of those two places if it wasn't found in either one. How do I find out what the error message is? I read that check_call() can return an error message and something about a returncode attribute. I couldn't find examples on how to use check_call() and returncode, what the message would be or how I could tell if the message is command-not-found.

Am I even going about this the right way?

Answer 1

A simple snippet:

try:
    subprocess.check_call(['executable'])
except subprocess.CalledProcessError:
    pass # handle errors in the called executable
except OSError:
    pass # executable not found

Answer 2

subprocess will raise an exception, OSError, when a command is not found.

When the command is found, and subprocess runs the command for you, the result code is returned from the command. The standard is that code 0 means success, and any failure is some non-zero error code (which varies; check the documentation for the specific command you are running).

So, if you catch OSError you can handle the non-existent command, and if you check the result code you can find out whether the command succeeded or not.

The great thing about subprocess is that you can make it collect all the text from stdout and stderr, and you can then discard it or return it or log it or display it as you like. I often use a wrapper that discards all output from a command, unless the command fails in which case the text from stderr is output.

I agree that you shouldn't be asking users to copy executables around. Programs should be in a directory listed in the PATH variable; if a program is missing it should be installed, or if it is installed in a directory not on the PATH the user should update the PATH to include that directory.

Note that you do have the option of trying subprocess multiple times with various hard-coded paths to executables:

import os
import subprocess as sp

def _run_cmd(s_cmd, tup_args):
    lst_cmd = [s_cmd]
    lst_cmd.extend(tup_args)
    result = sp.call(lst_cmd)
    return result

def run_lumberjack(*tup_args):
    try:
        # try to run from /usr/local/bin
        return _run_cmd("/usr/local/bin/lumberjack", tup_args)
    except OSError:
        pass

    try:
        # try to run from /opt/forest/bin
        return _run_cmd("/opt/forest/bin/lumberjack", tup_args)
    except OSError:
        pass

    try:
        # try to run from "bin" directory in user's home directory
        home = os.getenv("HOME", ".")
        s_cmd = home + "/bin/lumberjack"
        return _run_cmd(s_cmd, tup_args)
    except OSError:
        pass

    # Python 3.x syntax for raising an exception
    # for Python 2.x, use:  raise OSError, "could not find lumberjack in the standard places"
    raise OSError("could not find lumberjack in the standard places")

run_lumberjack("-j")

EDIT: After thinking about it a little bit, I decided to completely rewrite the above. It's much cleaner to just pass a list of locations, and have a loop try the alternative locations until one works. But I didn't want to build the string for the user's home directory if it wasn't needed, so I just made it legal to put a callable into the list of alternatives. If you have any questions about this, just ask.

import os
import subprocess as sp

def try_alternatives(cmd, locations, args):
    """
    Try to run a command that might be in any one of multiple locations.

    Takes a single string argument for the command to run, a sequence
    of locations, and a sequence of arguments to the command.  Tries
    to run the command in each location, in order, until the command
    is found (does not raise OSError on the attempt).
    """
    # build a list to pass to subprocess
    lst_cmd = [None]  # dummy arg to reserve position 0 in the list
    lst_cmd.extend(args)  # arguments come after position 0

    for path in locations:
        # It's legal to put a callable in the list of locations.
        # When this happens, we should call it and use its return
        # value for the path.  It should always return a string.
        if callable(path):
            path = path()

        # put full pathname of cmd into position 0 of list    
        lst_cmd[0] = os.path.join(path, cmd)
        try:
            return sp.call(lst_cmd)
        except OSError:
            pass
    raise OSError('command "{}" not found in locations list'.format(cmd))

def _home_bin():
    home = os.getenv("HOME", ".")
    return os.path.join(home, "bin")

def run_lumberjack(*args):
    locations = [
        "/usr/local/bin",
        "/opt/forest/bin",
        _home_bin, # specify callable that returns user's home directory
    ]
    return try_alternatives("lumberjack", locations, args)

run_lumberjack("-j")

Answer 3

Wow, that was fast! I combined Theodros Zelleke's simple example and steveha's use of functions with abarnert comment about OSError and Lattyware's comment about moving files:

import os, sys, subprocess

def nameandpath():
    try:
        subprocess.call([os.getcwd() + '/lumberjack']) 
        # change the word lumberjack on the line above to get an error
    except OSError:
        print('\nCould not find lumberjack, please reinstall.\n')
        # if you're using python 2.x, change the () to spaces on the line above

try:
    subprocess.call(['lumberjack'])
    # change the word lumberjack on the line above to get an error
except OSError:
    nameandpath()

I tested it on Mac OS-X (6.8/Snow Leopard), Debian (Squeeze) and Windows (7). It seemed to work the way I wanted it to on all three operating systems. I tried using check_call and CalledProcessError but no matter what I did, I seemed to get an error every time and I couldn't get the script to handle the errors. To test the script I changed the name from 'lumberjack' to 'deadparrot', since I had lumberjack in the directory with my script.

Do you see any problems with this script the way it's written?