Python, Popen and select - waiting for a process t

问题:

I run a subprocess using:

  p = subprocess.Popen("subprocess", 
                       stdout=subprocess.PIPE, 
                       stderr=subprocess.PIPE, 
                       stdin=subprocess.PIPE)

This subprocess could either exit immediately with an error on stderr, or keep running. I want to detect either of these conditions - the latter by waiting for several seconds.

I tried this:

  SECONDS_TO_WAIT = 10
  select.select([], 
                [p.stdout, p.stderr], 
                [p.stdout, p.stderr],
                SECONDS_TO_WAIT)

but it just returns:

  ([],[],[])

on either condition. What can I do?

回答1:

Have you tried using the Popen.Poll() method. You could just do this:

p = subprocess.Popen("subprocess", 
                   stdout=subprocess.PIPE, 
                   stderr=subprocess.PIPE, 
                   stdin=subprocess.PIPE)

time.sleep(SECONDS_TO_WAIT)
retcode = p.poll()
if retcode is not None:
   # process has terminated

This will cause you to always wait 10 seconds, but if the failure case is rare this would be amortized over all the success cases.

---

Edit:

How about:

t_nought = time.time()
seconds_passed = 0

while(p.poll() is not None and seconds_passed < 10):
    seconds_passed = time.time() - t_nought

if seconds_passed >= 10:
   #TIMED OUT

This has the ugliness of being a busy wait, but I think it accomplishes what you want.

Additionally looking at the select call documentation again I think you may want to change it as follows:

SECONDS_TO_WAIT = 10
  select.select([p.stderr], 
                [], 
                [p.stdout, p.stderr],
                SECONDS_TO_WAIT)

Since you would typically want to read from stderr, you want to know when it has something available to read (ie the failure case).

I hope this helps.

回答2:

This is what i came up with. Works when you need and don't need to timeout on thep process, but with a semi-busy loop.

def runCmd(cmd, timeout=None):
    '''
    Will execute a command, read the output and return it back.

    @param cmd: command to execute
    @param timeout: process timeout in seconds
    @return: a tuple of three: first stdout, then stderr, then exit code
    @raise OSError: on missing command or if a timeout was reached
    '''

    ph_out = None # process output
    ph_err = None # stderr
    ph_ret = None # return code

    p = subprocess.Popen(cmd, shell=True,
                         stdout=subprocess.PIPE,
                         stderr=subprocess.PIPE)
    # if timeout is not set wait for process to complete
    if not timeout:
        ph_ret = p.wait()
    else:
        fin_time = time.time() + timeout
        while p.poll() == None and fin_time > time.time():
            time.sleep(1)

        # if timeout reached, raise an exception
        if fin_time < time.time():

            # starting 2.6 subprocess has a kill() method which is preferable
            # p.kill()
            os.kill(p.pid, signal.SIGKILL)
            raise OSError("Process timeout has been reached")

        ph_ret = p.returncode


    ph_out, ph_err = p.communicate()

    return (ph_out, ph_err, ph_ret)

回答3:

Here is a nice example:

from threading import Timer
from subprocess import Popen, PIPE

def kill_proc():
    proc.kill()

proc = Popen("ping 127.0.0.1", shell=True)
t = Timer(60, kill_proc)
t.start()
proc.wait()

回答4:

Using select and sleeping doesn't really make much sense. select (or any kernel polling mechanism) is inherently useful for asynchronous programming, but your example is synchronous. So either rewrite your code to use the normal blocking fashion or consider using Twisted:

from twisted.internet.utils import getProcessOutputAndValue
from twisted.internet import reactor    

def stop(r):
    reactor.stop()
def eb(reason):
    reason.printTraceback()
def cb(result):
    stdout, stderr, exitcode = result
    # do something
getProcessOutputAndValue('/bin/someproc', []
    ).addCallback(cb).addErrback(eb).addBoth(stop)
reactor.run()

Incidentally, there is a safer way of doing this with Twisted by writing your own ProcessProtocol:

http://twistedmatrix.com/projects/core/documentation/howto/process.html

回答5:

If, as you said in the comments above, you're just tweaking the output each time and re-running the command, would something like the following work?

from threading import Timer
import subprocess

WAIT_TIME = 10.0

def check_cmd(cmd):
    p = subprocess.Popen(cmd,
        stdout=subprocess.PIPE, 
            stderr=subprocess.PIPE)
    def _check():
        if p.poll()!=0:
            print cmd+" did not quit within the given time period."

    # check whether the given process has exited WAIT_TIME
    # seconds from now
    Timer(WAIT_TIME, _check).start()

check_cmd('echo')
check_cmd('python')

The code above, when run, outputs:

python did not quit within the given time period.

The only downside of the above code that I can think of is the potentially overlapping processes as you keep running check_cmd.

回答6:

Python 3.3

import subprocess as sp

try:
    sp.check_call(["/subprocess"], timeout=10,
                  stdin=sp.DEVNULL, stdout=sp.DEVNULL, stderr=sp.DEVNULL)
except sp.TimeoutError:
    # timeout (the subprocess is killed at this point)
except sp.CalledProcessError:
    # subprocess failed before timeout
else:
    # subprocess ended successfully before timeout

See TimeoutExpired docs.

回答7:

This is a paraphrase on Evan's answer, but it takes into account the following :

1. Explicitly canceling the Timer object : if the Timer interval would be long and the process will exit by its "own will" , this could hang your script :(

2. There is an intrinsic race in the Timer approach (the timer attempt killing the process just after the process has died and this on Windows will raise an exception).

     DEVNULL = open(os.devnull, "wb")
     process = Popen("c:/myExe.exe", stdout=DEVNULL) # no need for stdout
   
     def kill_process():
     """ Kill process helper"""
     try:
        process.kill()
      except OSError:
        pass  # Swallow the error
   
     timer = Timer(timeout_in_sec, kill_process)
     timer.start()
   
     process.wait()
     timer.cancel()