How to scale a slack bot to 1000's of teams

Question

To implement a slack bot, i need to deal with 'Real Time Messaging API' of slack. It is a WebSocket-based API that allows you to receive events from Slack in real time and send messages as user. more info: https://api.slack.com/rtm

To create a bot for only one team, i need to open one websocket connection and listen it for events.

To make available the slack bot for another team. I need to open a new websocket connection. So,

• 1 team => 1 websocket connection
• 2 teams => 2 websocket connections
• N teams => N websocket connections

what should i do to scale my websocket connections for endless teams?

What kind of architecture can handle autoscaling of 1000’s of websockets connections?

Answer 1

With slack sockets, you have lots of things to scale:

• Number of sockets. This is easy because even cheap servers can handle thousands of sockets, like more than 50k. But each socket represents a couple other types of load, listed next.
• Amount of memory used per team, which depends on your own server implementation. If you are trying to keep a large amount of message history in memory, you will hit your server's limit faster than if your message processing code is somewhat stateless.
• Amount of I/O, which might make you want to offload any image serving to a separate load balancer.

The other thing to consider is fault-tolerance. Let's say you did sticky load balancing and one of your servers is handling 50 teams. That server is the only one handling those 50 teams so if it goes down then all 50 bots go offline. Alternatively, you can open up multiple sockets per team on separate servers and use a message handling queue so that each message is only responded to once.

So the architecture I would propose is a thin, redundant load balancer for RTM sockets as a first layer, and a reliable message queue underneath that.