How to make a Python HTTP Request with POST data a

Question

I am trying to do a HTTP POST using cookies in Python.

I have the values of URL, POST data and cookie.

import urllib2
url="http://localhost/testing/posting.php"
data="subject=Alice-subject&addbbcode18=%23444444&addbbcode20=0&helpbox=Close+all+open+bbCode+tags&message=alice-body&poll_title=&add_poll_option_text=&poll_length=&mode=newtopic&sid=5b2e663a3d724cc873053e7ca0f59bd0&f=1&post=Submit"
cookie = "phpbb2mysql_data=a%3A2%3A%7Bs%3A11%3A%22autologinid%22%3Bs%3A0%3A%22%22%3Bs%3A6%3A%22userid%22%3Bs%3A1%3A%223%22%3B%7D; phpbb2mysql_t=a%3A9%3A%7Bi%3A3%3Bi%3A1330156986%3Bi%3A1%3Bi%3A1330160737%3Bi%3A5%3Bi%3A1330161702%3Bi%3A6%3Bi%3A1330179284%3Bi%3A2%3Bi%3A1330160743%3Bi%3A7%3Bi%3A1330163187%3Bi%3A8%3Bi%3A1330164442%3Bi%3A9%3Bi%3A1330164739%3Bi%3A10%3Bi%3A1330176335%3B%7D; phpbb2mysql_sid=5b2e663a3d724cc873053e7ca0f59bd0"
#creating HTTP Req
req = urllib2.Request(url,data,cookie)

f = urllib2.urlopen(req)
print f.read()

However, if I try to run the program, it is throwing an error:

Traceback (most recent call last):
  File "task-4.py", line 7, in <module>
    req = urllib2.Request(url,data,cookie)
  File "/usr/lib/python2.6/urllib2.py", line 197, in __init__
    for key, value in headers.items():
AttributeError: 'str' object has no attribute 'items'

I have two questions: 1. Is my HTTP POST request proper? (I have properly been able to execute the same thing in Java and got a HTTP 200 with a successful post to phpBB, however, I am new to Python) 2. Can someone show me a toy example of handling HTTP POST with POST data and cookies?

Thanks in advance,

Roy

Answer 1

1. Not really. That error is because urllib2 library is trying to iterate over the items of the cookie string you gave it. Try using:

cookies = urllib.urlencode({'phpbb2mysql_data':'foo', 'autologinid':'blahblah'})
# Can do the same for data, allowing you to store it as a map.
headers = {'Cookie': cookies}
req = urllib2.Request(url, data, headers)

1. See python: urllib2 how to send cookie with urlopen request but your best reference is still really the urllib2 Request docs, but yes it's a tricky (but powerful) library compared to some newer ones.

Answer 2

the 3rd argument you pass is a header and should be a dictionary. This should do it

cookie = {"Cookie" : "phpbb2mysql_data=a%3A2%3A%7Bs%3A11%3A%22autologinid%22%3Bs%3A0%3A%22%22%3Bs%3A6%3A%22userid%22%3Bs%3A1%3A%223%22%3B%7D; phpbb2mysql_t=a%3A9%3A%7Bi%3A3%3Bi%3A1330156986%3Bi%3A1%3Bi%3A1330160737%3Bi%3A5%3Bi%3A1330161702%3Bi%3A6%3Bi%3A1330179284%3Bi%3A2%3Bi%3A1330160743%3Bi%3A7%3Bi%3A1330163187%3Bi%3A8%3Bi%3A1330164442%3Bi%3A9%3Bi%3A1330164739%3Bi%3A10%3Bi%3A1330176335%3B%7D; phpbb2mysql_sid=5b2e663a3d724cc873053e7ca0f59bd0"}

Answer 3

I like httplib:

from urlparse import urlparse
from httplib import HTTPConnection
url = "http://localhost/testing/posting.php"
data = "subject=Alice-subject&addbbcode18=%23444444&addbbcode20=0&helpbox=Close+all+open+bbCode+tags&message=alice-body&poll_title=&add_poll_option_text=&poll_length=&mode=newtopic&sid=5b2e663a3d724cc873053e7ca0f59bd0&f=1&post=Submit"
cookie = "phpbb2mysql_data=a%3A2%3A%7Bs%3A11%3A%22autologinid%22%3Bs%3A0%3A%22%22%3Bs%3A6%3A%22userid%22%3Bs%3A1%3A%223%22%3B%7D; phpbb2mysql_t=a%3A9%3A%7Bi%3A3%3Bi%3A1330156986%3Bi%3A1%3Bi%3A1330160737%3Bi%3A5%3Bi%3A1330161702%3Bi%3A6%3Bi%3A1330179284%3Bi%3A2%3Bi%3A1330160743%3Bi%3A7%3Bi%3A1330163187%3Bi%3A8%3Bi%3A1330164442%3Bi%3A9%3Bi%3A1330164739%3Bi%3A10%3Bi%3A1330176335%3B%7D; phpbb2mysql_sid=5b2e663a3d724cc873053e7ca0f59bd0"
urlparts = urlparse(url)
conn = HTTPConnection(urlparts.netloc, urlparts.port or 80)
conn.request("POST", urlparts.path, data, {'Cookie': cookie})
resp = conn.getresponse()
body = resp.read()

Answer 4

You can try requests, which makes life easier when dealing with HTTP queries.

import requests
url="http://localhost/testing/posting.php"
data= {
    'subject': 'Alice-subject',
    'addbbcode18': '%23444444',
    'addbbcode20': '0',
    'helpbox': 'Close all open bbCode tags',
    'message': 'alice-body',
    'poll_title': '',
    'add_poll_option_text': '',
    'poll_length': '',
    'mode': 'newtopic',
    'sid': '5b2e663a3d724cc873053e7ca0f59bd0',
    'f': '1',
    'post': 'Submit',
    }
 cookies = {'phpbb2mysql_data': 'a%3A2%3A%7Bs%3A11%3A%22autologinid%22%3Bs%3A0%3A%22%22%3Bs%3A6%3A%22userid%22%3Bs%3A1%3A%223%22%3B%7D',
    'phpbb2mysql_t': 'a%3A9%3A%7Bi%3A3%3Bi%3A1330156986%3Bi%3A1%3Bi%3A1330160737%3Bi%3A5%3Bi%3A1330161702%3Bi%3A6%3Bi%3A1330179284%3Bi%3A2%3Bi%3A1330160743%3Bi%3A7%3Bi%3A1330163187%3Bi%3A8%3Bi%3A1330164442%3Bi%3A9%3Bi%3A1330164739%3Bi%3A10%3Bi%3A1330176335%3B%7D', 
    'phpbb2mysql_sid': '5b2e663a3d724cc873053e7ca0f59bd0',
    }
print requests.get(url, data=data, cookies=cookies).text

http://python-requests.org/