get a “raw” request\response from MITM Proxy

Question

i', scripting mitm proxy (http://mitmproxy.org/index.html) to write HTTP and HTTPS request and responses to a file according to their IP (each client can then access it's own requests\responses) for unit tests for mobile.

As far as i can see for now i can't just use str(Flow.request) or repr(Flow.request) to get a "raw" print of the response\request like i get in fiddler, i need to reconstruct it from the internal data of the Request and Response objects.

anyone knows of a better way ? i'm using :

def response(ScriptContext, Flow):
    Flow.request....
    Flow.response....

To access the request or response being intercepted, i'm not changing anything, just observing. For now the proxy is on 8080, later on it's to be transparent proxy on 80 and 443. If anyone has done it before i'll be happy if you can share some info.

Answer 1

couple of things. first youcan build the raw response yourself using str(flow.request.headers) and request.httpversion and the like. however it seems that _assemble() and _assemble_headers() do the trick just fine.

so basically:

def request(context, flow):
req = flow.request;
try:
    print("Request: -----------------");
    print(req._assemble());
    print("--------------------------");
except Exception as ee:
    print(str(ee));

def response(context, flow):
    res = flow.response;
    try:
        print("Response: -----------------");
    print(res._assemble());

    if res.content:
        size = len(res.content);
        size  = min(size, 20);
        if res.content[0:size] != res.get_decoded_content()[0:size]:
            print("\n\n");
            print(res.get_decoded_content());
    print("--------------------------");
except Exception as ee:
    print(str(ee));

as you can see if the decoded body is not similar to the non decoded one (i can check for gzip content type though) i'm printing the decoded message as well. This should be saved to files according to current dates and each file is named after the client ip taken from request\response.client_conn object. This pretty much solved my problem. Some check with fiddler shows that the request are reproducable later on which is just what i needed.