I use a simple webdriver phantomjs script to update some adverts on preloved.co.uk. This script worked great until recently, but then started failing with the "Click submitted but load failed" error after the login link was clicked. In accordance with this I updated my version of phantomjs to latest stable, 1.9.7 following the guide here. However, now the login click does not seem to register either, and the page does not reload.
The first step is simply getting to login form page.
from selenium import webdriver
br = webdriver.PhantomJS(service_log_path='/path/to/logfile.log')
url = "http://www.preloved.co.uk"
br.get(url)
# Go to login page
login_button = br.find_element_by_xpath('//div[@id="header-status-login"]/a')
login_button.click()
Normally (and if you replace the browser line with br = webdriver.Firefox()
for example), this results in reloading to login page, and the script proceeds from there, but now it appears the click does not load the new page at all and br.current_url
is still 'http://www.preloved.co.uk/'
Why doesn't this load work?
Even if I extract the href and do an explicit GET it doesn't seem to follow and reload:
newurl=login_button.get_attribute('href')
br.get(newurl)
br.current_url
is still 'http://www.preloved.co.uk/'.
The login page is secured through https. Recently the POODLE vulnerability forced websites to move away from SSLv3 for https, but since PhantomJS uses SSLv3 per default the login page doesn't load. See also this answer.
This can be fixed by passing --ssl-protocol=tlsv1
or --ssl-protocol=any
to PhantomJS or upgrading PhantomJS to at least version 1.9.8. It seems that the service_args
argument could be used for that in the python bindings for Selenium.
It looks like in the current official implementation the service_args
cannot be passed from WebDriver
to the Service
in PhantomJS. You can sub-class it.
from selenium import webdriver
from selenium.webdriver.phantomjs.service import Service
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.remote.webdriver import WebDriver as RemoteWebDriver
class PhantomJSService(webdriver.PhantomJS):
def __init__(self, executable_path="phantomjs", port=0,
desired_capabilities=DesiredCapabilities.PHANTOMJS,
service_args=None, service_log_path=None):
self.service = Service(executable_path,
port=port, service_args=service_args,
log_path=service_log_path)
self.service.start()
try:
RemoteWebDriver.__init__(self,
command_executor=self.service.service_url,
desired_capabilities=desired_capabilities)
except:
self.quit()
raise
It seems that this webdriver fork contains the necessary arguments to set those options.