Python IDLE script does not show output of subproc

Question

I'm using Python 2.7.6 and IDLE on Windows 7.

I have 2 Python scripts:

script.py:

import subprocess, os, sys

print("hello 1")

mypath = os.path.abspath(__file__)
mydir = os.path.dirname(mypath)
start = os.path.join(mydir, "script2.py")

subprocess.call([sys.executable, start, "param"])

print("bye 1")

and script2.py that is being called by the previous script:

import sys

print "hello 2"

print (sys.argv[1])

print "bye 2"

If I run script.py with cmd.exe shell I get the expected result:

C:\tests>python ./script.py
hello 1
hello 2
param
bye 2
bye 1

But if I open script.py with the IDLE editor and run it with F5 I get this result:

>>> ================================ RESTART ================================
>>> 
hello 1
bye 1
>>> 

Why is the sub script not writing to the IDLE Python shell?

Answer 1

I verified that abamert's change works in 2.7, on Win7, with Idle started normally from the icon. The slight glitch is that 'print out' inserts an extra blank line. This is easily changed by making print a function with a future import and use of the end parameter.

from __future__ import print_function
...
print(out, end='')

With Python 3, there is an additional issue that 'out' is bytes instead of str, so that it prints as

b'hello 2\r\nparam\r\nbye 2\r\n'

Since your output is all ascii, this can be fixed by changing the print call to

print(out.decode(), end='')

The resulting program works identically in 2.7 and 3.x.

Answer 2

You're running the subprocess without providing any stdout or stderr.

When run in a terminal, the subprocess will inherit your stdout and stderr, so anything it prints will show up intermingled with your output.

When run in IDLE, the subprocess will also inherit your stdout and stderr, but those don't go anywhere. IDLE intercepts the Python-level wrappers sys.stdout and sys.stderr,* so anything you print to them from within Python will end up in the GUI window, but anything that goes to real stdout or stderr—like the output of any subprocess you run that inherits your streams—just goes nowhere.**

The simplest fix is to capture the stdout and stderr from the subprocess and print them yourself. For example:

out = subprocess.check_output([sys.executable, start, "param"],
                              stderr=subprocess.STDOUT)
print out

_{* IDLE is more complicated than it looks. It's actually running separate processes for the GUI window and for running your code, communicating over a socket. The sys.stdout (and likewise for the others) that IDLE provides for your script isn't a file object, it's a custom file-like object that redirects each write to the GUI process via remote procedure call over the socket.}

_{** Actually, if you launched IDLE from a terminal rather than by double-clicking its icon, the subprocess's output may end up there. I'm not sure how it works on Windows. But at any rate, that isn't helpful to you.}