In Django, to cluster memcached nodes, a very simple method is used. Simply list all node address in the settings.py file of all your django servers like so:
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': [
'xxx.xxx.xxx.240:11211',
'xxx.xxx.xxx.242:11211',
...,
]
}
}
Obviously editing the setting.py file of each instance whenever an instance drops out or a new one is added would be painful, how would you go about automagically managing the addition of new nodes to the cluster?
- All instances are behind a load balancer.
Possible non-answers:
- I could also just dedicate one django instance to run a memcached single node since I am only using memcached to store tiny tokens. But the goal is to have all ec2 instances be identical.
- I could also use elasticache but it is expensive (35 bucks/month! :) ) for the smallest version
Note: I use memcached to prevent celeryd workers from accessing the same resources, but its ok if occasionally a resource is double accessed. And my tokens have a short lifespan (less than 15 seconds). So loosing memcached nodes is not a big deal as long as it doesn't happen too frequently.