Python Unicode UnicodeEncodeError

Question

I am having issues with trying to convert an UTF-8 string to unicode. I get the error.

UnicodeEncodeError: 'ascii' codec can't encode characters in position 73-75: ordinal not in range(128)

I tried wrapping this in a try/except block but then google was giving me a system administrator error which was one line. Can someone suggest how to catch this error and continue.

Cheers, John.

-- FULL ERROR --

Traceback (most recent call last):
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/webapp/__init__.py", line 501, in __call__
    handler.get(*groups)
  File "/Users/johnb/Sites/hurl/hurl.py", line 153, in get
    self.redirect(url.long_url)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/webapp/__init__.py", line 371, in redirect
    self.response.headers['Location'] = str(absolute_url)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 73-75: ordinal not in range(128)

Answer 1

Try this:

self.response.headers['Location'] = absolute_url.decode("utf-8")
or
self.response.headers['Location'] = unicode(absolute_url, "utf-8")

Answer 2

The location header you are trying to set needs to be an Url, and an Url needs to be in Ascii. Since your Url is not an Ascii string you get the error. Just catching the error won't help since the Location header won't work with an invalid Url.

When you create absolute_url you need to make sure it is encoded properly, best by using urllib.quote and the strings encode() method. You can try this:

self.response.headers['Location'] = urllib.quote(absolute_url.encode('utf-8'))

Answer 3

The correct solution is to do the following:

self.response.headers['Location'] = urllib.quote(absolute_url.encode("utf-8"))

Answer 4

Please edit that mess so that it's legible. Hint: use the "code block" (101010 thingy button).

You say that you are "trying to convert an UTF-8 string to unicode" but str(absolute_url) is a strange way of going about it. Are you sure that absolute_url is UTF-8? Try

print type(absolute_url)
print repr(absolute_url)

If it is UTF-8, you need absolute_url.decode('utf8')