How to name/arrange my memcached keys when I have

Question

I have an existing Django app that doesn't do any Database caching. I am working to implement memcached to get a performance boost and reduce the amount of expensive database hits.

The strategy I will use is as follows: Every time I am querying the DB with XXX.objects.get() or XXX.objects.filter(), I will first check the cache to see if the result for this same query is already in memcached. If it is, I will just use that. If it is not, I will query the DB and stuff it into memcached with a specifically named key. Any time I update any results of this query, I will invalidate that cache key by using Django's post_save() signal. Sounds fairly simple, right?

Well, I'm struggling with how I should name my cache keys so that this works in an orderly fashion. The issue is that I have Django model objects that have foreign keys to 2 other Django models.

Here are my models:

memCache = pylibmc.Client(["127.0.0.1"])

class myObjectA(models.Model):  
    field1 = models.CharField(max_length=255)

    def getC_Children(self):
        if "SOME_NAME1_%s" % self.pk in memCache:
           return memCache["SOME_NAME1_%s" % self.pk]
        else:
           newCacheEntry = myObjectC.objects.filter(fk_myObjectA=self)
           memCache["SOME_NAME1_%s" % self.pk] = newCacheEntry
           return newCacheEntry




class myObjectB(models.Model):  
    field2 = models.CharField(max_length=255)

    def getC_Children(self):
        if "SOME_NAME2_%s" % self.pk in memCache:
           return memCache["SOME_NAME2_%s" % self.pk]
        else:
           newCacheEntry = myObjectC.objects.filter(fk_myObjectB=self)
           memCache["SOME_NAME2_%s" % self.pk] = newCacheEntry
           return newCacheEntry



class myObjectC(models.Model):  
    fk_myObjectA = models.ForeignKey(myObjectA, related_name="Blah_Blah") 
    fk_myObjectB = models.ForeignKey(myObjectB, related_name="Blah_Blah2") 
    field3 = models.CharField(max_length=255)

In the post_save handler() for myObjectC, I need to invalidate the cache keys SOME_NAME1_X and SOME_NAME2_X because they are now out-of-date. Right? I think that's what I need to do.

But what if there are numerous such keys for each instance of each class? After all, there will be one such key for each XXX.objects.get() or XXX.objects.filter() call per instance. Do I have to invalidate them all manually? Isn't there a systematic way to name and invalidate these keys all at once without having to remember each cache entry myself?

Answer 1

Caching requires a solid and carefully crafted strategy because you might end doing things worse than they are. In most of the projects you actually don't need advanced caching. Why don't you cache the page and delete the page cache each time your DB is updated? this will allow the query to run once but the rest of the times the cache will be retrieved. If you still insist for caching, cache using a unique key, they key combination could be created by the unique objects id and the name of it. Signals are the way to go either way, if you update or create and new object delete the cache, but as you can see from your own example, a page cache is way more easily treated than going through a large number of objects. Apart from that, why don't you use the build in Django cache.get, cache.set, cache.delete function? this way you keep compatibility of your project and decouple your code from the cache engine (maybe tomorrow redis will be more preferred to your needs).