Long-polling (user notification) with a RESTful ar

Question

I'm working on a simple RESTful api (on NodeJs). I understand that being restful implies that horizontal scaling will be much easier. Unfortunately, I need some way for clients to be notified of certain events almost instantly. For this reason, I was thinking of doing long-polling. The question I have is how this work with multiple servers. For instance, here's a simplified used case:

Both servers A and B are behind a common load balancer.
User Alice posts a message on Bob's wall (PUT request on server A).
If Bob is online, he should be notified instantly (long-polling on server B).

How can server A send a notification to Bob, or server B know that Bob should be notified?

Answer 1

Long polling approach puts the burden on client to make periodic requests - hence they are separate requests. There is not "notifying the client" with Long polling. The client polls.

You can use distributed cache to store the intermediate state information to which both A and B will have access to. Thus Server A and B can take part in the conversation with the client because it is all kept in that distributed cache.

Client Alpha Request 1 -> Node A =) Cache it as k,v (Aplha, State<-has request info)

Meanwhile one or more Node A,B,C or D works on it and takes its own time.

Client Alpha Request 2 -> Node B =) Retrieve Cache it as k,v (Aplha,State <-not done)
(are we done yet?)

Meanwhile one Work its Done. Your Request is fulfilled :) i.e.

Node X =) Update cache (Aplha, State <- results)

Client Alpha Request 2 -> Node B =) Retrieve Cache it as k,v (Aplha, State<-results)
    (are we done yet?)                      (here you Sir - your results are now ready)

WebSockets: Another options is to go for WebSockets instead of Long Polling

    Connection between Client and Node A is persistent with bidirectional communication.

Answer 2

First off, I don't know why you wouldn't just use something like socket.io on both client and server to handle your notification channel. That will use web sockets if available (more efficient than long polling) and then fallback to long polling if no web sockets.

There are a number of ways to handle the issue of notifying a user that may not be connected to the server that originates the notification:

Connected Database

When a user connects, they are load balanced to a random server, but which server they end up connecting to is stored in a central database (e.g. perhaps a redis store) from which any of your servers can find out which server any user is currently connected to. This gives you the ability to lookup which server any user is currently connected to. Each server that handles user connections just adds each user to the database with a reference to their server id when they connect and removes them from the database when the user disconnects. Note: since this information doesn't need to be stored persistently to disk and the size of the data is small, you can select a database that excels at keeping most or all of the info in memory.

Hashed Connection

Based on some well-known characteristic of the user such as their userID, a hash value is computed and a repeatable algorithm is created to map hash values across the server pool. This is a predictable assignment. Thus, when a server wants to notify Bob, they can call the same function that will figure out which server Bob has connected to. The hash algorithm can be adaptive such that if today you have three servers, the hashed users are spread evenly among the three servers and if tomorrow you have four servers, then the hashed users are spread evenly among the four servers.

Multi-Connection

Since low-activity websockets are pretty scalable these days, all users could connect to all servers and thus each server would have a socket connection to the user. This is simpler infrastructure, but not ultimately as scalable.