I'm trying to crawl a website (http://www.dataescolabrasil.inep.gov.br/dataEscolaBrasil/home.seam) using mechanize
but I am getting an error I cannot understand (and therefore cannot solve). That's probably due to my poor knowledge of web development.
Here's what I'm trying to do:
import mechanize
# this is the website I want to crawl
LINK = "http://www.dataescolabrasil.inep.gov.br/dataEscolaBrasil/home.seam"
br = mechanize.Browser()
br.open(LINK)
request = mechanize.Request(LINK)
response = mechanize.urlopen(request)
# there're two forms in the page (output ommited), I want the second one.
forms = mechanize.ParseResponse(response, backwards_compat=False)
for form in br.forms():
print "Form name:", form.name
print form
br.select_form(nr=1)
br.form['codEntidadeDecorate:codEntidadeInput'] = '11024968'
response2 = br.submit()
And here's the runtime error I'm getting:
Traceback (most recent call last):
File "C:\test.py", line 19, in <module>
response2 = br.submit()
File "build\bdist.win32\egg\mechanize\_mechanize.py", line 541, in submit
File "build\bdist.win32\egg\mechanize\_mechanize.py", line 530, in click
File "build\bdist.win32\egg\mechanize\_form.py", line 2999, in click
File "build\bdist.win32\egg\mechanize\_form.py", line 3201, in _click
File "build\bdist.win32\egg\mechanize\_form.py", line 2350, in _click
File "build\bdist.win32\egg\mechanize\_form.py", line 3269, in _switch_click
File "build\bdist.win32\egg\mechanize\_form.py", line 3257, in _request_data
ValueError: unknown POST form encoding type ''
I've tried some tweaks to the of encoding the string I pass to the form, tried to understand GET v. POST, but no success.