I have the following subtle problem: The Python code starts two threads, each creating a subprocess call to an exexutable (in fact written in C). The first executable is passed the argument 10000, meaning a delay of 10s before its exits. Similar for the second executable, but with a delay of 20s.
I observe, that worker1 and worker 2 do their printouts at the same time, namely after 20s, the longer of both delays. Why is wait()
for worker1 somehow "blocked" by the longer delay of the other? Whatever I pass as arguments, both print outputs are right after the longer of both times.
Note: This is a boiled down example for demonstration purposes and not the code I'm working on. Anyway, it is not related to print output, in my original code instead of print I do something more complicated.
import subprocess
import threading
import time
def worker1():
proc = subprocess.Popen(["../testapp1.exe", "10000"], shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
outs, errs = proc.communicate()
ret = proc.wait()
print("result worker1:%s ret:%d" % (outs, ret))
print("done")
def worker2():
proc = subprocess.Popen(["../testapp2.exe", "20000"], shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
outs, errs = proc.communicate()
ret = proc.wait()
print("result worker2:%s ret:%d" % (outs, ret))
print("done")
job_thread1 = threading.Thread(target = worker1)
job_thread1.start()
job_thread2 = threading.Thread(target = worker2)
job_thread2.start()
job_thread1.join()
print ("after join1")
job_thread2.join()
print ("after join2")
# time.sleep(20000) original
The executable:
#include <string.h>
#include <windows.h>
int main(int argc, char *argv[])
{
int count;
//printf ("This program was called with \"%s\".\n",argv[0]);
int sleep = 0;
if (argc > 1)
{
for (count = 1; count < argc; count++)
{
if (count == 1)
{
sleep = atoi(argv[1]);
}
printf("\nargv[%d] = %s\n", count, argv[count]);
}
}
else
{
printf("The command had no other arguments.\n");
}
int fin = sleep;
for (int i=0; i<fin; i++)
{
for (int i2=0; i2<100; i2++);
}
//Sleep(sleep);
return 0;
}
EDIT: when I remove "communicate" it works for some reason. But in this case I cannot get the output of the process...
def worker1():
proc = subprocess.Popen(["../testapp1.exe", "10000"], shell=False,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
# outs, errs = proc.communicate()
ret = proc.wait()
print("result worker1:%s ret:%d" % (outs, ret))
print("done")
EDIT2: Using only wait(), I found, that it is blocking at the marked label, where the output from the pipe is read...
def worker1():
try:
proc = subprocess.Popen(["../testapp.exe", "100000000"], shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
ret = proc.wait()
xread = proc.stdout.read() ### !!!! BLOCKS
print("xread:",xread)
sys.stdout.write("done-1")
sys.stdout.flush()
except ComError as e:
print("got exception")
raise e