I am still trying to use browsermob-proxy
(here) to try to monitor the network traffic during a python selenium test (see browsermobproxypy). I am using the following script to test it:
from browsermobproxy import Server
server = Server("/home/adietz/Projects/Invest/browsermob-proxy/browsermob-proxy-2.1.4/bin/browsermob-proxy")
server.start()
proxy = server.create_proxy()
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_proxy(proxy.selenium_proxy())
driver = webdriver.Firefox(firefox_profile=profile)
proxy.new_har("google")
driver.get("http://www.google.co.in")
proxy.har
server.stop()
driver.quit()
I somehow managed to install a 'browsermob.jar' file (which is not really nicely described in the documentation here; you have to download the jar files from here). I am starting the proxy as follows:
java -jar ./browsermob-proxy-2.1.4/lib/browsermob-dist-2.1.4.jar --port 9090
Running BrowserMob Proxy using LittleProxy implementation. To revert to the legacy implementation, run the proxy with the command-line option '--use-littleproxy false'.
[INFO 2018-01-18T07:47:20,761 net.lightbody.bmp.proxy.Main] (main) Starting BrowserMob Proxy version 2.1.4
[INFO 2018-01-18T07:47:20,782 org.eclipse.jetty.util.log] (main) jetty-7.x.y-SNAPSHOT
[INFO 2018-01-18T07:47:20,802 org.eclipse.jetty.util.log] (main) started o.e.j.s.ServletContextHandler{/,null}
[INFO 2018-01-18T07:47:20,885 org.eclipse.jetty.util.log] (main) Started SelectChannelConnector@0.0.0.0:9090
Then I am running the test script as
python tester1.py
However, in the logs I see errors like follows:
[WARN 2018-01-18T07:28:30,265 org.eclipse.jetty.util.log] (main) FAILED SelectChannelConnector@0.0.0.0:8080: java.net.BindException: Address already in use
[WARN 2018-01-18T07:28:30,266 org.eclipse.jetty.util.log] (main) FAILED org.eclipse.jetty.server.Server@4b41e4dd: java.net.BindException: Address already in use
[ERROR 2018-01-18T07:28:30,266 net.lightbody.bmp.proxy.Main] (main) Failed to start Jetty server. Aborting. java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_151]
at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_151]
at sun.nio.ch.Net.bind(Net.java:425) ~[?:1.8.0_151]
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) ~[?:1.8.0_151]
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) ~[?:1.8.0_151]
at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:162) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:356) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:240) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.server.Server.doStart(Server.java:269) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55) ~[browsermob-dist-2.1.4.jar:?]
at net.lightbody.bmp.proxy.Main.main(Main.java:64) [browsermob-dist-2.1.4.jar:?]
My best guess is that the port is already in use, or something else is not working correctly. Maybe I need to explicitly specify the port in the script? But how to do that? A python documentation is virtually not-existing ...
Addendum
Looking at the source code(!) I figured out how to set the port. I am using the following line to define the server and the port in the python script:
server = Server("/home/adietz/Projects/Invest/browsermob-proxy/browsermob-proxy-2.1.4/bin/browsermob-proxy", {'port':9090})
But no matter what I set as the port I always get the following error:
Traceback (most recent call last):
File "tester1.py", line 5, in <module>
proxy = server.create_proxy()
File "/home/adietz/Projects/Invest/browsermob-proxy/venv_browsermob/local/lib/python2.7/site-packages/browsermobproxy/server.py", line 40, in create_proxy
client = Client(self.url[7:], params)
File "/home/adietz/Projects/Invest/browsermob-proxy/venv_browsermob/local/lib/python2.7/site-packages/browsermobproxy/client.py", line 37, in __init__
"Another server running on this port?\n%s..." % content[:512])
Exception: Could not read Browsermob-Proxy json
Another server running on this port?
java.lang.RuntimeException: java.net.BindException: Address already in use
at org.littleshoot.proxy.impl.DefaultHttpProxyServer.doStart(DefaultHttpProxyServer.java:545)
at org.littleshoot.proxy.impl.DefaultHttpProxyServer.start(DefaultHttpProxyServer.java:489)
at org.littleshoot.proxy.impl.DefaultHttpProxyServer.access$700(DefaultHttpProxyServer.java:69)
at org.littleshoot.proxy.impl.DefaultHttpProxyServer$DefaultHttpProxyServerBootstrap.start(DefaultHttpProxyServer.java:858)
at net.lightbody.bmp.Brows...
So there must be something else working incorrectly...
Maybe I will just code such a proxy on my own. Seems to be the faster way...
We can start the BMP server with required port as given below. server accepts dictionary object for server arguments.
Initialize the server with this -
The internal code in Browsermob-Proxy is looking for key in
'options'
dictionary named'existing_proxy_port_to_use'
, else it uses default i.e. empty dictionary.