Printing all shell commands fired from python scri

Question

I have a moderately large python script that executes a lot of shell commands from within itself. I would like to print all these commands to screen before executing it, so that I can trace the control flow of the python script.

Now, the script uses multiple commands to actually execute shell commands, including system() from os module, and call(), popen() and check_output() from subprocess module. What might be the easiest way to do this?

I was thinking of a wrapper function that prints the shell command argument before executing it, but I don't know how to write a generic one that can call the correct call/Popen or other command as per user discretion. And we also have to keep in mind that these calls take in different number and type of arguments.

Thanks!

Answer 1

import commands

commands = r'''find .'''
result = commands.getstatusoutput(command)[0]
print("Command: {}\nresult: {}".format(command,result))

Answer 2

Since you want to start a child process, but log your behavior, you will need to record child process stdout, stderr,

import subprocess as sp
import datetime
dt=datetime.datetime.now().strftime("%Y%m%d%H%M00")
ofd = open("res.sql","w+")
efd = open("log/cmd-{}.log".format(dt),"a+")

Suppose you are constructing a command (example, mysqldump) and you have the database, table, and credentials filenames (db, tbl, cnf) loaded. You want to print the command you are about to executed,

args = ["mysqldump","--defaults-extra-file="+cnf,"--single-transaction","--quick",db,tbl]
print " ".join(args)

Now, assume you have opened output and error files above (ofd, efd),

proc = sp.Popen(args, shell=False, stdin=sp.PIPE, stdout=ofd, stderr=efd)
stdout,stderr = proc.communicate
rc = proc.wait()
if rc>0: print "error",rc,"dbdump failed"
else: print "result",stdout,stderr

Remember to close ofd, efd.

Answer 3

Build and install a decorator for the various calls you wish to intercept. Because even the "builtin" functions are first-class objects, you can replace them with logging versions:

import os

def log_wrap(f):
    def logging_wrapper(*args, **kwargs):
        print("Calling {} with args: {!r}, kwargs: {!r}".format(f.__name__, args, kwargs))
        return f(*args, **kwargs)

    return logging_wrapper

os.system("echo 'Hello, world!'")
os.system = log_wrap(os.system)
os.system("echo 'How do you like me now?!'")

This code, when run (python3) prints:

$ python test.py
Hello, world!
Calling system with args: ("echo 'How do you like me now?!'",), kwargs: {}
How do you like me now?!

Note that between the first and second calls to os.system, I replaced the os.system function with one that prints a log message before passing the arguments along to the original function. You can print your log message to a file, or (better yet) call the logging module, or invoke the pdb debugger, or whatever you like...