Using matplotlib in GAE

Question

My tags and title quite clearly state my problem. I want to use matplotlib to create real-time plots in Google App Engine. I've read the documentation and searched on SO and Google. I found a post, pointing to this working demo. But when I try it on my own, it doesn't work for me.

I created a simple application, consisting only of a handler-script hello_world.py

import numpy as np
import os
import sys
import cStringIO

print "Content-type: image/png\n"

os.environ["MATPLOTLIBDATA"] = os.getcwdu()  # own matplotlib data
os.environ["MPLCONFIGDIR"] = os.getcwdu()    # own matplotlibrc
import matplotlib.pyplot as plt

plt.plot(np.random.random((20))) #imshow(np.random.randint((10,10)))

sio = cStringIO.StringIO()
plt.savefig(sio, format="png")
sys.stdout.write(sio.getvalue())

and a a configuration file app.yaml

application: helloworldtak
version: 1
runtime: python27
api_version: 1
threadsafe: no

handlers:
- url: /.*
  script: hello_world.py

libraries:
- name: numpy
  version: "latest"
- name: matplotlib
  version: "latest"

I want to plot something and then return the content as png-image. This procedure works fine for a normal web-server like Apache or IIS, I did this a million times.

The problem is rather: when I run my script locally within the development server, I get an error that is probably due to my MPL version 1.1.1, which is only "experimental" in GAE. But when I deploy my app to GAE, I get a completely different, uncorrelated error.

Looking at the looks, the traceback is:

Traceback (most recent call last):
  File "/base/data/home/apps/s~helloworldtak/1.364765672279579252/hello_world.py", line 16, in <module>
    import matplotlib.pyplot as plt
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/pyplot.py", line 23, in <module>
    from matplotlib.figure import Figure, figaspect
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/figure.py", line 18, in <module>
    from axes import Axes, SubplotBase, subplot_class_factory
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/axes.py", line 14, in <module>
    import matplotlib.axis as maxis
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/axis.py", line 10, in <module>
    import matplotlib.font_manager as font_manager
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/font_manager.py", line 1324, in <module>
    _rebuild()
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/font_manager.py", line 1278, in _rebuild
    fontManager = FontManager()
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/font_manager.py", line 995, in __init__
    self.defaultFont['ttf'] = self.ttffiles[0]
IndexError: list index out of range

It seems to have to do something with the fonts-cache of MPL. I read in the docs that caching and file-access is one of the problems with MPL in GAE, but obviously, the import works for others.

What am I doing wrong?

Edit Based on the answer below, I changed my code to be

import numpy as np
import cStringIO
import matplotlib.pyplot as plt

import webapp2

class MainPage(webapp2.RequestHandler):
    def get(self):
        plt.plot(np.random.random((20)),"r-")
        sio = cStringIO.StringIO()
        plt.savefig(sio, format="png")
        self.response.headers['Content-Type'] = 'image/png'

        self.response.out.write(sio.getvalue())

app = webapp2.WSGIApplication([('/', MainPage)],
                              debug=True)

and like this, it's working.

Answer 1

The problem was that you were setting the MATPLOTLIBDATA and MPLCONFIGDIR environment variables to your app directory before importing matplotlib. Since you didn't have any fonts in your app directory, it couldn't load any fonts.

Answer 2

I'm not familiar with sys module. To give an answer to the question I prefer using webapp2. This is a working handler:

import webapp2
import StringIO
import numpy as np
import matplotlib.pyplot as plt


class MainPage(webapp2.RequestHandler):
    def get(self):
        plt.plot(np.random.random((20)))
        sio = StringIO.StringIO()
        plt.savefig(sio, format="png")
        img_b64 = sio.getvalue().encode("base64").strip()
        plt.clf()
        sio.close()
        self.response.write("""<html><body>""")
        self.response.write("<img src='data:image/png;base64,%s'/>" % img_b64)
        self.response.write("""</body> </html>""")

app = webapp2.WSGIApplication([('/', MainPage)], debug=True)

Alternatively, you could write the sio.getvalue() in the blobstore with files api and use the method get_serving_url() of images api for avoid to encode in base64.