How to make a distributed node.js application?

Question

Creating a node.js application is simple enough.

var app = require('express')();
app.get('/',function(req,res){
    res.send("Hello world!");
});

But suppose people became obsessed with your Hello World! application and exhausted your resources. How could this example be scaled up on practice? I don't understand it, because yes, you could open several node.js instance in different computers - but when someone access http://your_site.com/ it aims directly that specific machine, that specific port, that specific node process. So how?

Answer 1

I've been working with node for quite some time but recently got the opportunity to try scaling my node apps and have been researching on the same topic for some time now and have come across following pre-requisites for scaling:

1. My app needs to be available on a distributed system each running multiple instances of node

2. Each system should have a load balancer that helps distribute traffic across the node instances.

3. There should be a master load balancer that should distribute traffic across the node instances on distributed systems.

4. The master balancer should always be running OR should have a dependable restart mechanism to keep the app stable.

For the above requisites I've come across the following:

1. Use modules like cluster to start multiple instances of node in a system.

2. Use nginx always. It's one of the most simplest mechanism for creating a load balancer i've came across so far

3. Use HAProxy to act as a master load balancer. A few pointers on how to use it and keep it forever running.

Useful resources:

1. Horizontal scaling node.js and websockets.
2. Using cluster to take advantages of multiple cores.

I'll keep updating this answer as I progress.

Answer 2

There are many many ways to deal with this, but it boils down to 2 things:

1. being able to use more cores per server
2. being able to scale beyond more than one server.

node-cluster

For the first option, you can user node-cluster or the same solution as for the seconde option. node-cluster (http://nodejs.org/api/cluster.html) essentially is a built in way to fork the node process into one master and multiple workers. Typically, you'd want 1 master and n-1 to n workers (n being your number of available cores).

load balancers

The second option is to use a load balancer that distributes the requests amongst multiple workers (on the same server, or across servers).

Here you have multiple options as well. Here are a few:

• a node based option: Load balancing with node.js using http-proxy
• nginx: Node.js + Nginx - What now? (using more than one upstream server)
• apache: (no clearly helpful link I could use, but a valid option)

One more thing, once you start having multiple processes serving requests, you can no longer use memory to store state, you need an additional service to store shared states, Redis (http://redis.io) is a popular choice, but by no means the only one.

If you use services such as cloudfoundry, heroku, and others, they set it up for you so you only have to worry about your app's logic (and using a service to deal with shared state)

Answer 3

The basic way to use multiple machines is to put them behind a load balancer, and point all your traffic to the load balancer. That way, someone going to http://my_domain.com, and it will point at the load balancer machine. The sole purpose (for this example anyways; in theory more could be done) of the load balancer is to delegate the traffic to a given machine running your application. This means that you can have x number of machines running your application, however an external machine (in this case a browser) can go to the load balancer address and get to one of them. The client doesn't (and doesn't have to) know what machine is actually handling its request. If you are using AWS, it's pretty easy to set up and manage this. Note that Pascal's answer has more detail about your options here.

With Node specifically, you may want to look at the Node Cluster module. I don't really have alot of experience with this module, however it should allow you to spawn multiple process of your application on one machine all sharing the same port. Also node that it's still experimental and I'm not sure how reliably it will be.

Answer 4

I'd recommend to take a look to http://senecajs.org, a microservices toolkit for Node.js. That is a good start point for beginners and to start thinking in "services" instead of monolitic applications.

Having said that, building distributed applcations is hard, take time to learn, take LOT of time to master it, and usually you will face a lot trade-off between performance, reliability, manteinance, etc.