Unsuppress UnicodeEncodeError exceptions when run

Question

The following is a statement that should raise an UnicodeEncodeError exception:

print 'str+{}'.format(u'unicode:\u2019')

In a Python shell, the exception is raised as expected:

>>> print 'str+{}'.format(u'unicode:\u2019')

Traceback (most recent call last):
  File "<pyshell#10>", line 1, in <module>
    print 'str+{}'.format(u'unicode:\u2019')
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 8: ordinal not in range(128)

However, if I place that line at the start of my settings.py and start the Django server from Aptana Studio, no error is raised and this line is printed:

str+unicode:’

But if I execute manage.py runserver from a shell, the exception is raised:

UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 8: ordinal not in range(128)

Is there some sort of Python setting that silently suppresses these unicode errors?

How can I prevent the unicode error from being ignored when I start the Django test server directly from Aptana Studio?

Using

• Python 2.7.3
• Aptana Studio 3.3.2

Answer 1

If you simply cast a bytestring to unicode, like

print unicode(s)

or mix unicode and bytestrings in string formatting operations like your example, Python will fall back on the system default encoding (which is ascii unless it has been changed), and implicitly will try to encode unicode / decode the bytestring using the ascii codec.

The currently active system default encoding can be displayed with

import sys
sys.getdefaultencoding()

Now it seems like Aptana Studio does in fact mess with your interpreters default encoding:

From a blog post by Mikko Ohtamaa:

[...] Looks like the culprint was PyDev (Eclipse Python plug-in). The interfering source code is here. Looks like the reason was to co-operate with Eclipse console. However it has been done incorrectly. Instead of setting the console encoding, the encoding is set to whole Python run-time environment, messing up the target run-time where the development is being done.

There is a possible fix for this problem. In Eclipse Run… dialog settings you can choose Console Encoding on Common tab. There is a possible value US-ASCII. I am not sure what Python 2 thinks “US-ASCII” encoding name, since the default is “ascii”.

So make sure you reset the default to ascii, and you should be good.