Program getting stuck when using subprocess.Popen(

Question

I want to run a program from python and find its memory usage. To do so I am using:

l=['./a.out','<','in.txt','>','out.txt']
p=subprocess.Popen(l,shell=False,stdout = subprocess.PIPE, stderr = subprocess.PIPE)
p.wait()
Res= getrusage(resource.RUSAGE_CHILDREN)
print Res.ru_maxrss

I also tried to use check_call(l,shell=False,stdout = subprocess.PIPE, stderr = subprocess.PIPE) and remove p.wait but the problem is program is getting stuck at p.wait() when using Popen and at check_call() when using check_call(). I am not able to figure out why this is happening. Is my argument list wrong.

The command ./a.out < in.txt > out.txt is working fine on terminal. I am using Ubuntu

Answer 1

There are two issues (at least):

1. <, > redirection is handled by a shell. subprocess doesn't spawn a shell by default (you shouldn't either)
2. If stdout=PIPE, stderr=PIPE then you must read from the pipes otherwise the process may block forever

To make a subprocess read from a file and write to a file:

from subprocess import check_call, STDOUT

with open('in.txt') as file, open('out.txt', 'w') as outfile:
    check_call(["./a.out"], stdin=file, stdout=outfile, stderr=STDOUT)

stderr=STDOUT merges stdout, stderr.

Answer 2

You are using shell redirection characters in your call, but when you use subprocess, and set shell=False, you have to handle those pipes manually. You seem to be passing those redirection characters directly as arguments to the a.out program. Try running this in your shell:

./a.out '<' in.txt '>' out.txt

See if a.out terminates then as well.