Trying to login to quora using mechanize

问题:

I'm trying to use mechanize module to login to quora.com.

This is my code:

#!/usr/bin/env python

import mechanize
import cookielib

br = mechanize.Browser() # create a browser object
br.set_handle_equiv(True)
br.set_handle_redirect(True)
br.set_handle_referer(True)
br.set_handle_robots(False)
br.set_headers = [('User-Agent', 'Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)')]

cj = cookielib.LWPCookieJar()
br.set_cookiejar(cj)

res = br.open('http://www.quora.com')
br.select_form(nr = 0)

br.form['email'] = 'uuuu'
br.form['password'] = 'pppp'

res = br.submit()
print res.read()

I get this error:

Traceback (most recent call last):
  File "mech.py", line 29, in <module>
    res = br.submit()
  File "/usr/local/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 541, in submit
    return self.open(self.click(*args, **kwds))
  File "/usr/local/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 203, in open
    return self._mech_open(url, data, timeout=timeout)
  File "/usr/local/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 255, in _mech_open
    raise response
mechanize._response.httperror_seek_wrapper: HTTP Error 500: Internal Server Error

I know that HTTP 500 means there is something wrong server-side, but I have no idea what it is, and how I can debug the issue.

回答1:

I bet mechanize can't handle submitting this particular form. Using browser developer tools you can see that this form is submitted via javascript XHR POST request to https://www.quora.com/webnode2/server_call_POST?__instart__ where request parameters looks like:

json:{"args":[],"kwargs":{"email":"<email>","password":"<password>","passwordless":1}}
formkey:62c4f0d88246bfd81b27cf0dca410d75
window_id:dep4-4597603286175583039
_lm_transaction_id:0.4317954108119011
_lm_window_id:dep4-4597603286175583039
__vcon_json:["hmac","t1cKg1QhQsYPCA"]
__vcon_method:do_login
js_init:{}

FYI, if you turn on debugging via br.set_debug_http(True) you'll see that your script behind the scenes makes a POST request to the main page http://www.quora.com with the following parameters:

email=<email>
allow_passwordless=allow_passwordless
password=<password>

As you can see: wrong URL and different parameters.

I'd use selenium for this particular case.

Hope that helps.