I'm coding on a high scalable web harvester running on top of Eventmachine. All works fine and fast. Recently I'm trying to fire the requests through a bunch of proxies which also works fine, e.g.:
EventMachine.run do
connect_opts = { :proxy => { :host => '11.12.13.14', :port => 3128 } }
request_opts = { :proxy => { :authorization => ['jdoe', 'mysecretpass'] } }
req = EventMachine::HttpRequest.new('http://www.example.com/', connect_opts).get request_opts
req.callback { }
end
I'm iterating over hundreds of proxies and firing several hundred requests per second. What I need now is to know which proxy was used for which request to store this metadata in a db. How would you fetch this information from the req object?
I hoped there's is some method like req.proxy_foo (equivalent to req.response, req.response_header etc.) to get this information but didn't find any appropriate way yet.