Redis client times out when connecting

Question

I have a Redis cluster running on ElastiCache.

Multiple processes connect to the Redis cluster. Each process lives within a Docker container. The processes are not all the same -- I have a web process, a worker process, etc.

After a few days of operating normally, some of my web processes begin to time out when connecting to Redis. When I ssh into an affected web container and try to reach the cluster via redis-cli, my connection to the cluster times out. This tells me that the issue affects the entire container, and not just the web process.

When I use redis-cli from any other container, I connect without problems.

My web processes create new connections as needed, and close old connections when they're idle for a long time. My guess is that any given Docker container can open a certain number of connections before reaching some kind of limit. After a few days, my web containers reach that limit.

Any idea how to go about fixing this?

--

One more detail: for some reason, resetting my Redis cluster fixes the issue across all web containers. Maybe the Redis server imposes a limit on how many connections can be opened from a given IP address?

Answer 1

Josiah and I had a long discussion on the Redis Google group which helped solve my issue; hopefully the discussion can help someone else in the future.