BeautifulSoup error in google appengine

Question

I have made an application in Google app engine that is using Beautiful Soup. I'm using the latest version of it, http://www.crummy.com/software/BeautifulSoup/bs4/download/4.0/. Everything worked fine until i uploaded it to GAE. The thing that happend then was that I got this error:

Traceback (most recent call last):
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/_webapp25.py", line 701, in __call__
    handler.get(*groups)
  File "/base/data/home/apps/s~app/1.358226218191077920/main.py", line 285, in get
    self.response.out.write(self.makeQuery("9147094591"))
  File "/base/data/home/apps/s~app/1.358226218191077920/main.py", line 191, in makeQuery
    from bs4 import BeautifulSoup
  File "/base/data/home/apps/s~app/1.358226218191077920/bs4/__init__.py", line 29, in <module>
    from .builder import builder_registry
  File "/base/data/home/apps/s~app/1.358226218191077920/bs4/builder/__init__.py", line 279, in <module>
    from . import _htmlparser
  File "/base/data/home/apps/s~app/1.358226218191077920/bs4/builder/_htmlparser.py", line 23, in <module>
    from bs4.element import (
  File "/base/data/home/apps/s~app/1.358226218191077920/bs4/element.py", line 6, in <module>
    from bs4.dammit import EntitySubstitution
  File "/base/data/home/apps/s~app/1.358226218191077920/bs4/dammit.py", line 254
    smart_quotes_re = b"([\x80-\x9f])"
                                     ^
SyntaxError: invalid syntax

When I look at line 254 in dammit.py I find this:

if (self.smart_quotes_to is not None
    and proposed.lower() in self.ENCODINGS_WITH_SMART_QUOTES):
    smart_quotes_re = b"([\x80-\x9f])"
    smart_quotes_compiled = re.compile(smart_quotes_re)
    markup = smart_quotes_compiled.sub(self._sub_ms_char, markup)

I can't really see what is wrong. I've tried other versions of BS but they don't work because I'm using the soup.select("CSS SELECTOR") which only seems to work in the latest version.

But, as I said earlier, it worked fine on my computer but not in the cloud.

Answer 1

BeautifulSoup 4 needs Python 2.7 or newer, but Appengine has Python 2.5 by default. You can either:

• Use Python 2.7 on Appengine (Edit: doesn't require a paid app. Thanks Grewe.)

• Use BeautifulSoup 3 for now.