Change URL to another URL using mitmproxy

问题:

I am trying to redirect one page to another by using mitmproxy and Python. I can run my inline script together with mitmproxy without issues, but I am stuck when it comes to changing the URL to another URL. Like if I went to google.com it would redirect to stackoverflow.com

def response(context, flow):
        print("DEBUG")
        if flow.request.url.startswith("http://google.com/"):
            print("It does contain it")
            flow.request.url = "http://stackoverflow/"

This should in theory work. I see http://google.com/ in the GUI of mitmproxy (as GET) but the print("It does contain it") never gets fired.

When I try to just put flow.request.url = "http://stackoverflow.com" right under the print("DEBUG") it won't work neither.

What am I doing wrong? I have also tried if "google.com" in flow.request.url to check if the URL contains google.com but that won't work either.

Thanks

回答1:

The following mitmproxy script will

1. Redirect requesto from mydomain.com to newsite.mydomain.com
2. Change the request method path (supposed to be something like /getjson? to a new one `/getxml
3. Change the destination host scheme
4. Change the destination server port

5. Overwrite the request header Host to pretend to be the origi

   import mitmproxy
   from mitmproxy.models import HTTPResponse
   from netlib.http import Headers
   def request(flow):
   
       if flow.request.pretty_host.endswith("mydomain.com"):
               mitmproxy.ctx.log( flow.request.path )
               method = flow.request.path.split('/')[3].split('?')[0]
               flow.request.host = "newsite.mydomain.com"
               flow.request.port = 8181
               flow.request.scheme = 'http'
               if method == 'getjson':
                   flow.request.path=flow.request.path.replace(method,"getxml")
               flow.request.headers["Host"] = "newsite.mydomain.com"
   

回答2:

You can set .url attribute, which will update the underlying attributes. Looking at your code, your problem is that you change the URL in the response hook, after the request has been done. You need to change the URL in the request hook, so that the change is applied before requesting resources from the upstream server.

回答3:

Setting the url attribute will not help you, as it is merely constructed from underlying data. [EDIT: I was wrong, see Maximilian’s answer. The rest of my answer should still work, though.]

Depending on what exactly you want to accomplish, there are two options.

(1) You can send an actual HTTP redirection response to the client. Assuming that the client understands HTTP redirections, it will submit a new request to the URL you give it.

from mitmproxy.models import HTTPResponse
from netlib.http import Headers

def request(context, flow):
    if flow.request.host == 'google.com':
        flow.reply(HTTPResponse('HTTP/1.1', 302, 'Found',
                                Headers(Location='http://stackoverflow.com/',
                                        Content_Length='0'),
                                b''))

(2) You can silently route the same request to a different host. The client will not see this, it will assume that it’s still talking to google.com.

def request(context, flow):
    if flow.request.url == 'http://google.com/accounts/':
        flow.request.host = 'stackoverflow.com'
        flow.request.path = '/users/'

These snippets were adapted from an example found in mitmproxy’s own GitHub repo. There are many more examples there.

For some reason, I can’t seem to make these snippets work for Firefox when used with TLS (https://), but maybe you don’t need that.