Python 3 - Add custom headers to urllib.request Re

Question

In Python 3, the following code obtains the HTML source for a webpage.

import urllib.request
url = "https://docs.python.org/3.4/howto/urllib2.html"
response = urllib.request.urlopen(url)

response.read()

How can I add the following custom header to the request when using urllib.request?

headers = { 'User-Agent' : 'Mozilla/5.0 (Windows NT 6.1; Win64; x64)' }

Answer 1

import urllib.request

opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
urllib.request.install_opener(opener)
response = urllib.request.urlopen("url")
response.read()

Should you wish to learn about the details you can refer to the python documentation: https://docs.python.org/3/library/urllib.request.html

Answer 2

The request headers can be customized by first creating a request object then supplying it to urlopen.

import urllib.request
url = "https://docs.python.org/3.4/howto/urllib2.html"
hdr = { 'User-Agent' : 'Mozilla/5.0 (Windows NT 6.1; Win64; x64)' }

req = urllib.request.Request(url, headers=hdr)
response = urllib.request.urlopen(req)
response.read()

Source: Python 3.4 Documentation