I am starting a subprocess
in python through this command:
result = subprocess.Popen([sys.executable, "/subscript.py"] + parameter,stdout=subprocess.PIPE )
result.wait()
out, err = result.communicate()
for line in out.splitlines():
print line
The problem that I am facing with this approach is that all the print
statements in subscript.py
are kind of buffered till the end and are printed on the terminal
at the end of the execution. So if my subscript.py
takes about 15 minutes in execution then for 15 minutes the terminal
does not shows anything and once the subscript.py
execution ends then all the print statements are printed together. I know this is because of the use of stdout=subprocess.PIPE
and if I remove it then I will get what I want but I cannot remove it as I am using it to get the value of a variable from the subscript.py
(I am printing that variable in subscript.py
and then in parent script I am going through each print to get the print with variable value).
So, what I want is that all the print statements should be printed as they are encountered in the subscript.py
without any buffering till the end of the execution. Is there a way to do that?
UPDATE: I tried to follow J.F. Sebastian's method but I still get the same output. The prints from subprocess are buffered and once the subprocess ends then they are printed to the terminal. The updated code I have is:
result = subprocess.Popen([sys.executable, "/subscript.py"] + parameter,stdout=subprocess.PIPE,bufsize=1)
with result.stdout:
for line in iter(result.stdout.readline, b''):
print line,
result.wait()