The input url is mixed ascii and multi byte code.
And I can't change this strings.
I would like to download this url, but an error occured ordinal not in range(128).
input_url = "http://sample.jp/api?v1=aaa&v2=日本語&v3=ccc"
req = urllib.request.Request(input_url)
resp = None
try:
resp = urllib.request.urlopen(req)
except UnicodeEncodeError as e:
print(e.reason) # I had an error `ordinal not in range(128)`
return resp
First, I tried urllib.parse.quote() but the result is http%3a%2f%2fsample%2ejp%2fapi%3fv1%3daaa%26v2%3d%93%fa%96%7b%8c%ea%26v3%3dccc. I had another error ValueError: unknown url type.
How can I resolve this problem? Or do you have some ideas?
A combination of urllib, and urlparse should do it for you:
>>> urllib.urlencode(urlparse.parse_qsl(urlparse.urlparse(input_url).query))
'v1=aaa&v2=%E6%97%A5%E6%9C%AC%E8%AA%9E&v3=ccc'
You need to encode those parameters to UTF-8 bytes and the bytes toURL percent encoding. You can do all this with the urllib.parse module:
from urllib.parse import urlparse, parse_qs, urlencode
parsed = urlparse(input_url)
query = parse_qs(parsed.query)
fixed_url = parsed._replace(query=urlencode(query, doseq=True)).geturl()
Demo:
>>> from urllib.parse import urlparse, parse_qs, urlencode
>>> input_url = "http://sample.jp/api?v1=aaa&v2=日本語&v3=ccc"
>>> parsed = urlparse(input_url)
>>> query = parse_qs(parsed.query)
>>> parsed._replace(query=urlencode(query, doseq=True)).geturl()
'http://sample.jp/api?v1=aaa&v2=%E6%97%A5%E6%9C%AC%E8%AA%9E&v3=ccc'
>>> import urllib.request
>>> urllib.request.urlopen(_)
<http.client.HTTPResponse object at 0x108f0f7b8>
