Caching Matplotlib with Memcache (Wont Pickle)

matplotlib, memcached, pickle, python

Solution

There are discussion out there regarding the desire for matplotlib figures to be able to be serialized. I haven't seen anything that reports this has been addressed or even accepted as a goal. So if you try to send them over the wire to memcached, its obviously going to fail. The discussions that I have found when searching suggest that the current design of matplotlib doesn't cater to this goal easily, and it would require a refactor of the internals. Reference: http://old.nabble.com/matplotlib-figure-serialization-td28016714.html

What you could do, to dramatically reduce your execution time, is to reorganize your data into a dataset, and only call `ax.bar()` once. The dataset can then be serialized and stored in whatever format you want (into memcached for instance).

Here is a code example showing the test between your approach, and one that combines them into a dataset. You can view it here more easily if you want: https://gist.github.com/2597804

import matplotlib.pyplot as plt
from random import randint 
from time import time 

DATA = [
    (i, randint(5,30), randint(5,30), randint(30,35), randint(1,5)) \
    for i in xrange(1, 401)
]

def mapValues(group):
    ind, open_, close, high, low = group
    if open_ > close: # if open is higher then close
        height = open_ - close # heigth is drawn at bottom+height
        bottom = close
        yerr = (open_ - low, high - open_)
        color = 'r' # plot as a white barr
    else:
        height = close - open_ # heigth is drawn at bottom+height
        bottom = open_
        yerr = (close - low, high - close)
        color = 'g' # plot as a black bar

    return (ind, height, bottom, yerr, color)

#
# Test 1
#
def test1():
    fig = plt.figure()
    ax = fig.add_subplot(111)

    data = map(mapValues, DATA)

    start = time()

    for group in data: 

        ind, height, bottom, yerr, color = group

        ax.bar(left=ind, height=height, bottom=bottom, yerr=zip(yerr), 
                color=color, ecolor='k', zorder=10,
                error_kw={'barsabove': False, 'zorder': 0, 'capsize': 0}, 
                alpha=1)

    return time()-start

#
# Test 2
#
def test2():
    fig = plt.figure()
    ax = fig.add_subplot(111)

    # plotData can be serialized
    plotData = zip(*map(mapValues, DATA))

    ind, height, bottom, yerr, color = plotData

    start = time()

    ax.bar(left=ind, height=height, bottom=bottom, yerr=zip(*yerr), 
            color=color, ecolor='k', zorder=10,
            error_kw={'barsabove': False, 'zorder': 0, 'capsize': 0}, 
            alpha=1)

    return time()-start


def doTest(fn):
    end = fn()
    print "%s - Sec: %0.3f, ms: %0d" % (fn.__name__, end, end*1000)



if __name__ == "__main__":
    doTest(test1)
    doTest(test2)

    # plt.show()

Results:

python plot.py 
test1 - Sec: 1.592, ms: 1592
test2 - Sec: 0.358, ms: 357

Problem

I have a chart that is rendered takes 3 seconds and then subcharts that can be made from said chart where things are added to it. I want to cache the axes from the main chart so that I can retrieve it and modify it later when rendering the subcharts. How can I get past this error? Heres a sample test code: ``` import pylibmc cache = pylibmc.Client(["127.0.0.1"], binary=True, behaviors={"tcp_nodelay": True, "ketama": True}) import matplotlib.pyplot as plt cache_name = 'test' fig = plt.figure(figsize=(20, 7)) ax = fig.add_axes([0, 0.15, 0.98, 0.85]) cache.set(cache_name, ax, 300) ``` Which gives the following error: ``` cPickle.PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed ``` Is there anyway I could get this to work?

Original source