Idiots guide to app engine and memcache

Question

I am struggling to find a good tutorial or best practices document for the use of memcache in app engine.

I'm pretty happy with it on the level presented in the docs. Get an object by ID, checking memcache first, but I'm unclear on things like:

If you cache a query, is there an accepted method for ensuring that the cache is cleared/updated when an object stored in that query is updated.

What are the effects of using ReferenceProperties ? If a cache a Foo object with a Bar reference. Is my foo.bar in memcache too and in need of clearing down if it gets updated from some other part of my application.

I don't expect answers to this here (unless you are feeling particularly generous!), but pointers to things I could read would be very gratefully received.

Answer 1

One way to easily do this is by searching for any datastore put() call and set memcache after it. Next make sure to get data from memcache before attempting a datastore query.

Set memcache after writing to datastore:

data.put()
memcache.set(user_id, data)

Try getting data from memcache before doing a datastore query:

data = memcache.get(user_id)
if data is None:
    data = Data.get_by_key_name(user_id)
    memcache.set(user_id, data)

Using Memcache reduces app engine costs significantly. More details on how I optimized on app engine.

Answer 2

If you cache a query, is there an accepted method for ensuring that the cache is cleared/updated when an object stored in that query is updated.

Typically you wrap your reads with a conditional to retrieve the value from the main DB if it's not in the cache. Just wrap your updates as well to fill the cache whenever you write the data. That's if you need the results to be as up to date as possible - if you're not so bothered about it being out of date just set an expiry time that is low enough for the application to have to re-request the data from the main DB often enough.

Answer 3

About reference properties, let's say you have MainModel and RefModel with a reference property 'ref' that points to a MainModel instance. Whenever you cal ref_model.ref it does a datastore get operation and retrieves the object from datastore. It does not interact with memcache in any way.