subprocess popen stdout locking up?

Question

Hi having some problem with stdout reading when using subprocess.Popen

daniel@desktop:~$ python -V
Python 2.7.3

heres the code: (commented code is some things i've tried)

import subprocess

RUN = './hlds_run -game cstrike -maxplayers 11'

p = subprocess.Popen(RUN.split(), shell=False, stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE)

while 1:
    try:
        out = p.stdout.readline()
        #if out == '':
        #   out = p.stdout.read()
        #p.stdout.flush()
    except: 
        p.terminate()
        break

    try:
        err = p.stderr.readline()
        #if err == '':
        #   err = p.stderr.read()
        #p.stderr.flush()
    except:
        p.terminate()
        break
    if out != '':
        print out.rstrip()

    if err != '':
        print err.rstrip()


    #print '\n' #constantly prints new lines until "Calling BreakpadMiniDumpSystemInit."

and this is the output im getting when connecting to the server and disconnecting:

daniel@desktop:~/hlds$ python hlds.py 
Auto detecting CPU
Using breakpad crash handler
Using Pentium II Optimised binary.
Setting breakpad minidump AppID = 10
Auto-restarting the server on crash
Forcing breakpad minidump interfaces to load

Looking up breakpad interfaces from steamclient
Console initialized.
Calling BreakpadMiniDumpSystemInit
scandir failed:/home/daniel/hlds/./valve/SAVE
Installing breakpad exception handler for appid(10)/version(5447)
scandir failed:/home/daniel/hlds/./platform/SAVE
Looking up breakpad interfaces from steamclient
Protocol version 48
Calling BreakpadMiniDumpSystemInit

the while loop locks up after:

Calling BreakpadMiniDumpSystemInit.

but the server is still running and i can connect, run commands etc...

---

if i run:

 ./hlds_run -game cstrike -maxplayers 11 >> stdout.log 2>&1

i get the following output in stdout.log:

daniel@desktop:~/hlds$ cat stdout.log 
Auto detecting CPU
Using Pentium II Optimised binary.
Auto-restarting the server on crash

Console initialized.
Using breakpad crash handler
Setting breakpad minidump AppID = 10
Forcing breakpad minidump interfaces to load
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
Installing breakpad exception handler for appid(10)/version(5447)
scandir failed:/home/daniel/hlds/./valve/SAVE
scandir failed:/home/daniel/hlds/./platform/SAVE
Protocol version 48
Exe version 1.1.2.6/Stdio (cstrike)
Exe build: 14:06:24 Sep 23 2011 (5447)
STEAM Auth Server
Server IP address 127.0.1.1:27015
couldn't exec listip.cfg
couldn't exec banned.cfg
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
scandir failed:/home/daniel/hlds/./valve/SAVE
scandir failed:/home/daniel/hlds/./platform/SAVE

Could not establish connection to Steam servers.
Reconnected to Steam servers.
   VAC secure mode is activated.
ERROR: couldn't open custom.hpk.
JAMES HETFIELD : hello!
Dropped JAMES HETFIELD from server
Reason:  Client sent 'drop'
Sat Apr 14 00:10:54 CEST 2012: Server Quit

however if i do not do 2>&1 i still get this output in stdout and the rest in stdout.log:

daniel@desktop:~/hlds$ ./hlds_run -game cstrike -maxplayers 11 >> stdout.log
Using breakpad crash handler
Setting breakpad minidump AppID = 10
Forcing breakpad minidump interfaces to load
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
Installing breakpad exception handler for appid(10)/version(5447)
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit

trying to create a servermanager and dispatcher as a subprocess learning experience :]

All help appreciated! :)

Answer 1

Have you been able to resolve this using select? I find that when I start SRCDS even nonblocking it doesn't go past the "Calling BreakpadMiniDumpSystemInit":

p = subprocess.Popen(cmd, shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True)
fcntl.fcntl(p.stdout, fcntl.F_SETFL, fcntl.fcntl(p.stdout, fcntl.F_GETFL) | os.O_NONBLOCK)
print p.stdout.read()

This prints up to the "BreakpadMiniDumpSystemInit" message, further calls to read() throw "Resource temporarily unavailable" until something is written to p.stdin.

Answer 2

What happens is that one of the pipes gets filled up with data, thus blocking the subprocess while your python process is blocked trying to read a line from the other pipe, leading to a deadlock. You probably want to use some kind of polling (select/poll/epoll) instead of doing blocking reads on the pipes.

A quick hack would be to do non-blocking reads in your while-loop, but that will lead to your python process using a lot of CPU.

Take a look at the documentation for the select module for more information on solving the problem the non-hacky way.