How can I have Google App Engine clear memcache ev

Question

The title asks it all. The content on the site I'm building wont change very quickly at all and so Memcache could potentially store data for months except for when I put up an update. Is there a way to make it clear the cache every time I deploy the site? I'm using the Python runtime.

Update 1

Using jldupont's answer I put the following code in my main request handling script...

Update 2

I've switched to the method mentioned by Koen Bok in the selected answer's comments and prefixed all my memcache keys with os.environ['CURRENT_VERSION_ID']/ with the helpful code in the answer's 2nd update. This solution seems to be much more elegant than the function I posted before.

Answer 1

When creating keys for your cached values, include the version of the file that is doing the cache gets/sets in the key. That way, when a new version of the file exists, it will no longer reference the old versions in the cache - they will be left to expire out on their own.

We use CVS and java, so we declare this variable in each file that will do caching:

private static final String CVS_REVISION = "$Revision $";

When you check that file out, you'll get something like this:

private static final String CVS_REVISION = "$Revision: 1.15 $";

You can adapt for your language and version control system if not CVS. Remember to encode special characters out of your keys. We've found that URL Encoding key values works well for memcached.

Answer 2

I have not tested this but perhaps if you insert into memcache a key with version # on instance start.

Then when the next instance is started, aka after a deployment, it would check memcache and its local version, if they differ, flush all and re-initalize the key.

Only flaw is what if the key is evicted, could replace memcache to datastore but then your making datastore calls for every instance start.

=edit=

Add to the top of your called python files from app.yaml

# Check if the version is updated
if memcache.get("static-version") == os.environ["CURRENT_VERSION_ID"]:
    pass
else:
    memcache.flush_all()
    memcache.set(key="static-version", value=os.environ["CURRENT_VERSION_ID"])

Answer 3

You could just create an admin-only path that would flush the cache when it's accessed.

Answer 4

Have you tried flush_all() function? Docs here. You'll need a bit of logic & state to detect a new deployment or have a special script to perform the flushing.

Updated: look at the absolute path of one of your script: this changes on every deployment. You can use http://shell.appspot.com/ to experiment:

  import sys
  sys.path

['/base/python_dist/lib/python25.zip', '/base/python_lib/versions/third_party/django-0.96', '/base/python_dist/lib/python2.5/', '/base/python_dist/lib/python2.5/plat-linux2', '/base/python_dist/lib/python2.5/lib-tk', '/base/python_dist/lib/python2.5/lib-dynload', '/base/python_lib/versions/1', '/base/data/home/apps/shell/1.335852500710379686/']

Look at the line with /shell/1.335852500710379686/.

So, just keep a snapshot (in memcache ;-) of this deployment state variable and compare in order to effect a flushing action.

Updated 2: as suggested by @Koen Bok, the environment variable CURRENT_VERSION_ID can be used also (part of the absolute path to script files also).

 import os
 os.environ["CURRENT_VERSION_ID"]