Does Python have a module for parsing HTTP request

Question

httplib (now http.client) and friends all have conn.getresponse() and an HTTPResponse class, but the server-side operations of conn.getrequest() and an HTTPRequest class seem to be lacking.

I understand that BaseHTTPServer and BaseHTTPRequestHandler can perform this functionality, but they don't expose these methods for use outside of the module.

Essentially what I want is BaseHTTPRequestHandler#parse_request to be a static method that returns an HTTPRequest object rather than populating member variables.

Answer 1

For server-side processing you want to look at something like wsgiref.

The WSGI standard parses the request into a simple dictionary with all of the relevant headers and elements.

Answer 2

Jeff, to enable parsing I create a small nine-line subclass of the base HTTP request handler:

from BaseHTTPServer import BaseHTTPRequestHandler
from StringIO import StringIO

class HTTPRequest(BaseHTTPRequestHandler):
    def __init__(self, request_text):
        self.rfile = StringIO(request_text)
        self.raw_requestline = self.rfile.readline()
        self.error_code = self.error_message = None
        self.parse_request()

    def send_error(self, code, message):
        self.error_code = code
        self.error_message = message

You can now take a string with the text of an HTTP request inside and parse it by instantiating this class:

# Simply instantiate this class with the request text

request = HTTPRequest(request_text)

print request.error_code       # None  (check this first)
print request.command          # "GET"
print request.path             # "/who/ken/trust.html"
print request.request_version  # "HTTP/1.1"
print len(request.headers)     # 3
print request.headers.keys()   # ['accept-charset', 'host', 'accept']
print request.headers['host']  # "cm.bell-labs.com"

# Parsing can result in an error code and message

request = HTTPRequest('GET\r\nHeader: Value\r\n\r\n')

print request.error_code     # 400
print request.error_message  # "Bad request syntax ('GET')"

Answer 3

You would probably find WebOb useful. Frameworks like Pylons, Turbogears, and Bfg use it as part of their api. It does operate under the assumption that you are working under WSGI though.