How to stop BaseHTTPServer.serve_forever() in a Ba

Question

I am running my HTTPServer in a separate thread (using the threading module which has no way to stop threads...) and want to stop serving requests when the main thread also shuts down.

The Python documentation states that BaseHTTPServer.HTTPServer is a subclass of SocketServer.TCPServer, which supports a shutdown method, but it is missing in HTTPServer.

The whole BaseHTTPServer module has very little documentation :(

Answer 1

I should start by saying that "I probably wouldn't do this myself, but I have in the past". The serve_forever (from SocketServer.py) method looks like this:

def serve_forever(self):
    """Handle one request at a time until doomsday."""
    while 1:
        self.handle_request()

You could replace (in subclass) while 1 with while self.should_be_running, and modify that value from a different thread. Something like:

def stop_serving_forever(self):
    """Stop handling requests"""
    self.should_be_running = 0
    # Make a fake request to the server, to really force it to stop.
    # Otherwise it will just stop on the next request.
    # (Exercise for the reader.)
    self.make_a_fake_request_to_myself()

Edit: I dug up the actual code I used at the time:

class StoppableRPCServer(SimpleXMLRPCServer.SimpleXMLRPCServer):

    stopped = False
    allow_reuse_address = True

    def __init__(self, *args, **kw):
        SimpleXMLRPCServer.SimpleXMLRPCServer.__init__(self, *args, **kw)
        self.register_function(lambda: 'OK', 'ping')

    def serve_forever(self):
        while not self.stopped:
            self.handle_request()

    def force_stop(self):
        self.server_close()
        self.stopped = True
        self.create_dummy_request()

    def create_dummy_request(self):
        server = xmlrpclib.Server('http://%s:%s' % self.server_address)
        server.ping()

Answer 2

Another way to do it, based on http://docs.python.org/2/library/basehttpserver.html#more-examples, is: instead of serve_forever(), keep serving as long as a condition is met, with the server checking the condition before and after each request. For example:

import CGIHTTPServer
import BaseHTTPServer

KEEP_RUNNING = True

def keep_running():
    return KEEP_RUNNING

class Handler(CGIHTTPServer.CGIHTTPRequestHandler):
    cgi_directories = ["/cgi-bin"]

httpd = BaseHTTPServer.HTTPServer(("", 8000), Handler)

while keep_running():
    httpd.handle_request()

Answer 3

I think you can use [serverName].socket.close()

Answer 4

In my python 2.6 installation, I can call it on the underlying TCPServer - it still there inside your HTTPServer:

TCPServer.shutdown


>>> import BaseHTTPServer
>>> h=BaseHTTPServer.HTTPServer(('',5555), BaseHTTPServer.BaseHTTPRequestHandler)
>>> h.shutdown
<bound method HTTPServer.shutdown of <BaseHTTPServer.HTTPServer instance at 0x0100D800>>
>>> 

Answer 5

This method I use successfully (Python 3) to stop the server from the web application itself (a web page):

import http.server
import os
import re

class PatientHTTPRequestHandler(http.server.SimpleHTTPRequestHandler):
    stop_server = False
    base_directory = "/static/"
    # A file to use as an "server stopped user information" page.
    stop_command = "/control/stop.html"
    def send_head(self):
        self.path = os.path.normpath(self.path)
        if self.path == PatientHTTPRequestHandler.stop_command and self.address_string() == "127.0.0.1":
            # I wanted that only the local machine could stop the server.
            PatientHTTPRequestHandler.stop_server = True
            # Allow the stop page to be displayed.
            return http.server.SimpleHTTPRequestHandler.send_head(self)
        if self.path.startswith(PatientHTTPRequestHandler.base_directory):
            return http.server.SimpleHTTPRequestHandler.send_head(self)
        else:
            return self.send_error(404, "Not allowed", "The path you requested is forbidden.")

if __name__ == "__main__":
    httpd = http.server.HTTPServer(("127.0.0.1", 8080), PatientHTTPRequestHandler)
    # A timeout is needed for server to check periodically for KeyboardInterrupt
    httpd.timeout = 1
    while not PatientHTTPRequestHandler.stop_server:
        httpd.handle_request()

This way, pages served via base address http://localhost:8080/static/ (example http://localhost:8080/static/styles/common.css) will be served by the default handler, an access to http://localhost:8080/control/stop.html from the server's computer will display stop.html then stop the server, any other option will be forbidden.

Answer 6

In python 2.7, calling shutdown() works but only if you are serving via serve_forever, because it uses async select and a polling loop. Running your own loop with handle_request() ironically excludes this functionality because it implies a dumb blocking call.

From SocketServer.py's BaseServer:

def serve_forever(self, poll_interval=0.5):
    """Handle one request at a time until shutdown.

    Polls for shutdown every poll_interval seconds. Ignores
    self.timeout. If you need to do periodic tasks, do them in
    another thread.
    """
    self.__is_shut_down.clear()
    try:
        while not self.__shutdown_request:
            # XXX: Consider using another file descriptor or
            # connecting to the socket to wake this up instead of
            # polling. Polling reduces our responsiveness to a
            # shutdown request and wastes cpu at all other times.
            r, w, e = select.select([self], [], [], poll_interval)
            if self in r:
                self._handle_request_noblock()
    finally:
        self.__shutdown_request = False
        self.__is_shut_down.set()

Heres part of my code for doing a blocking shutdown from another thread, using an event to wait for completion:

class MockWebServerFixture(object):
    def start_webserver(self):
        """
        start the web server on a new thread
        """
        self._webserver_died = threading.Event()
        self._webserver_thread = threading.Thread(
                target=self._run_webserver_thread)
        self._webserver_thread.start()

    def _run_webserver_thread(self):
        self.webserver.serve_forever()
        self._webserver_died.set()

    def _kill_webserver(self):
        if not self._webserver_thread:
            return

        self.webserver.shutdown()

        # wait for thread to die for a bit, then give up raising an exception.
        if not self._webserver_died.wait(5):
            raise ValueError("couldn't kill webserver")