Python subprocess timeout?

Question

Is there any argument or options to setup a timeout for Python's subprocess.Popen method?

Something like this:

subprocess.Popen(['..'], ..., timeout=20) ?

Answer 1

You could do

from twisted.internet import reactor, protocol, error, defer

class DyingProcessProtocol(protocol.ProcessProtocol):
    def __init__(self, timeout):
        self.timeout = timeout

    def connectionMade(self):
        @defer.inlineCallbacks
        def killIfAlive():
            try:
                yield self.transport.signalProcess('KILL')
            except error.ProcessExitedAlready:
                pass

        d = reactor.callLater(self.timeout, killIfAlive)

reactor.spawnProcess(DyingProcessProtocol(20), ...)

using Twisted's asynchronous process API.

Answer 2

No there is no time out. I guess, what you are looking for is to kill the sub process after some time. Since you are able to signal the subprocess, you should be able to kill it too.

generic approach to sending a signal to subprocess:

proc = subprocess.Popen([command])
time.sleep(1)
print 'signaling child'
sys.stdout.flush()
os.kill(proc.pid, signal.SIGUSR1)

You could use this mechanism to terminate after a time out period.

Answer 3

import subprocess, threading

class Command(object):
    def __init__(self, cmd):
        self.cmd = cmd
        self.process = None

    def run(self, timeout):
        def target():
            print 'Thread started'
            self.process = subprocess.Popen(self.cmd, shell=True)
            self.process.communicate()
            print 'Thread finished'

        thread = threading.Thread(target=target)
        thread.start()

        thread.join(timeout)
        if thread.is_alive():
            print 'Terminating process'
            self.process.terminate()
            thread.join()
        print self.process.returncode

command = Command("echo 'Process started'; sleep 2; echo 'Process finished'")
command.run(timeout=3)
command.run(timeout=1)

The output of this should be:

Thread started
Process started
Process finished
Thread finished
0
Thread started
Process started
Terminating process
Thread finished
-15

where it can be seen that, in the first execution, the process finished correctly (return code 0), while the in the second one the process was terminated (return code -15).

I haven't tested in windows; but, aside from updating the example command, I think it should work since I haven't found in the documentation anything that says that thread.join or process.terminate is not supported.

Answer 4

Unfortunately, there isn't such a solution. I managed to do this using a threaded timer that would launch along with the process that would kill it after the timeout but I did run into some stale file descriptor issues because of zombie processes or some such.

Answer 5

A python subprocess auto-timeout is not built in, so you're going to have to build your own.

This works for me on Ubuntu 12.10 running python 2.7.3

Put this in a file called test.py

#!/usr/bin/python
import subprocess
import threading

class RunMyCmd(threading.Thread):
    def __init__(self, cmd, timeout):
        threading.Thread.__init__(self)
        self.cmd = cmd 
        self.timeout = timeout

    def run(self):
        self.p = subprocess.Popen(self.cmd)
        self.p.wait()

    def run_the_process(self):
        self.start()
        self.join(self.timeout)

        if self.is_alive():
            self.p.terminate()   #if your process needs a kill -9 to make 
                                 #it go away, use self.p.kill() here instead.

            self.join()

RunMyCmd(["sleep", "20"], 3).run_the_process()

Save it, and run it:

python test.py

The sleep 20 command takes 20 seconds to complete. If it doesn't terminate in 3 seconds (it won't) then the process is terminated.

el@apollo:~$  python test.py 
el@apollo:~$ 

There is three seconds between when the process is run, and it is terminated.

Answer 6

I would advise taking a look at the Timer class in the threading module. I used it to implement a timeout for a Popen.

First, create a callback:

    def timeout( p ):
        if p.poll() is None:
            print 'Error: process taking too long to complete--terminating'
            p.kill()

Then open the process:

    proc = Popen( ... )

Then create a timer that will call the callback passing the process to it.

    t = threading.Timer( 10.0, timeout, [proc] )
    t.start()
    t.join()

Somewhere later in the program, you may want to add the line:

    t.cancel()

Otherwise, the python program will keep running until the timer has finished running.

EDIT: I was advised that there is a race condition that the subprocess p may terminate between the p.poll() and p.kill() calls. I believe the following code can fix that:

    import errno

    def timeout( p ):
        if p.poll() is None:
            try:
                p.kill()
                print 'Error: process taking too long to complete--terminating'
            except OSError as e:
                if e.errno != errno.ESRCH:
                    raise

Though you may want to clean the exception handling to specifically handle just the particular exception that occurs when the subprocess has already terminated normally.