Caching Matplotlib with Memcache (Wont Pickle)

问题:

I have a chart that is rendered takes 3 seconds and then subcharts that can be made from said chart where things are added to it. I want to cache the axes from the main chart so that I can retrieve it and modify it later when rendering the subcharts. How can I get past this error?

Heres a sample test code:

import pylibmc
cache = pylibmc.Client(["127.0.0.1"], binary=True, behaviors={"tcp_nodelay": True, "ketama": True})
import matplotlib.pyplot as plt


cache_name = 'test'
fig = plt.figure(figsize=(20, 7))
ax = fig.add_axes([0, 0.15, 0.98, 0.85])
cache.set(cache_name, ax, 300)

Which gives the following error:

cPickle.PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

Is there anyway I could get this to work?

回答1:

There are discussion out there regarding the desire for matplotlib figures to be able to be serialized. I haven't seen anything that reports this has been addressed or even accepted as a goal. So if you try to send them over the wire to memcached, its obviously going to fail. The discussions that I have found when searching suggest that the current design of matplotlib doesn't cater to this goal easily, and it would require a refactor of the internals. Reference: http://old.nabble.com/matplotlib-figure-serialization-td28016714.html

What you could do, to dramatically reduce your execution time, is to reorganize your data into a dataset, and only call ax.bar() once. The dataset can then be serialized and stored in whatever format you want (into memcached for instance).

Here is a code example showing the test between your approach, and one that combines them into a dataset. You can view it here more easily if you want: https://gist.github.com/2597804

import matplotlib.pyplot as plt
from random import randint 
from time import time 

DATA = [
    (i, randint(5,30), randint(5,30), randint(30,35), randint(1,5)) \
    for i in xrange(1, 401)
]

def mapValues(group):
    ind, open_, close, high, low = group
    if open_ > close: # if open is higher then close
        height = open_ - close # heigth is drawn at bottom+height
        bottom = close
        yerr = (open_ - low, high - open_)
        color = 'r' # plot as a white barr
    else:
        height = close - open_ # heigth is drawn at bottom+height
        bottom = open_
        yerr = (close - low, high - close)
        color = 'g' # plot as a black bar

    return (ind, height, bottom, yerr, color)

#
# Test 1
#
def test1():
    fig = plt.figure()
    ax = fig.add_subplot(111)

    data = map(mapValues, DATA)

    start = time()

    for group in data: 

        ind, height, bottom, yerr, color = group

        ax.bar(left=ind, height=height, bottom=bottom, yerr=zip(yerr), 
                color=color, ecolor='k', zorder=10,
                error_kw={'barsabove': False, 'zorder': 0, 'capsize': 0}, 
                alpha=1)

    return time()-start

#
# Test 2
#
def test2():
    fig = plt.figure()
    ax = fig.add_subplot(111)

    # plotData can be serialized
    plotData = zip(*map(mapValues, DATA))

    ind, height, bottom, yerr, color = plotData

    start = time()

    ax.bar(left=ind, height=height, bottom=bottom, yerr=zip(*yerr), 
            color=color, ecolor='k', zorder=10,
            error_kw={'barsabove': False, 'zorder': 0, 'capsize': 0}, 
            alpha=1)

    return time()-start


def doTest(fn):
    end = fn()
    print "%s - Sec: %0.3f, ms: %0d" % (fn.__name__, end, end*1000)



if __name__ == "__main__":
    doTest(test1)
    doTest(test2)

    # plt.show()

Results:

python plot.py 
test1 - Sec: 1.592, ms: 1592
test2 - Sec: 0.358, ms: 357

回答2:

As of matplotlib 1.2 you should be able to pickle and unpickle figures.

This is very much an "experimental" feature, but if you do find any issues, please let us know on the mpl mailing list or by raising an issue on github.com/matplotlib/matplotlib

HTH

回答3:

Looking at the documentation, it would appear that fig.add_axes() takes a tuple as an argument, where you're passing a list. As such, it's not returning the Axes object (since it isn't being created), so ax is being assigned the function itself.