Output of subprocess both to PIPE and directly to

Question

I found a number of questions which looks like mine, but which did not produce a solution I can use (closest is: subprocess output to stdout and to PIPE)

The problem: I want to start a process using subprocess which takes a long time. After running the command I need to parse the stdout-output and the stderr-output.

Currently I do it as follows:

p = subprocess.Popen( command_list, stdout=subprocess.PIPE, 
    stderr=subprocess.PIPE )
out, error_msg = p.communicate()
print out + "\n\n" + error_msg

#next comes code in which I check out and error_msg

But the drawback of this method is that the user does not see the output of the process while it is running. Only at the end the output is printed.

Is there a way that the output is printed while the command is running (as if I gave the command without stdout/stderr=subprocess.PIPE) and still have the output via p.communicate in the end?

Note: I'm currently developing on python 2.5 (old software release which uses this python version).

Answer 1

This snippet has helped me once in a similar situation:

process = subprocess.Popen(cmd, bufsize=1, universal_newlines=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line in iter(process.stdout.readline, ''):
    print line,
    sys.stdout.flush() # please see comments regarding the necessity of this line 
process.wait()
errcode = process.returncode