How to elect a master node among the nodes running

Question

I'm writing a managed cloud stack (on top of hardware-level cloud providers like EC2), and a problem I will face soon is:

How do several identical nodes decide which one of them becomes a master? (I.e. think of 5 servers running on EC2. One of them has to become a master, and other ones have to become slaves.)

I read a description of the algorithm used by MongoDB, and it seems pretty complicated, and also depends on a concept of votes — i.e. two nodes left alone won't be able to decide anything. Also their approach has a significant delay before it produces the results.

1. I wonder if there are any less complicated, KISS-embrasing approaches? Are they used widely, or are they risky to adopt?

2. Suppose we already have a list of servers. Then we can just elect the one that is up and has a numerically smallest IP address. What are downsides of this approach?

3. Why is MongoDB's algorithm so complicated?

This is a duplicate of How to elect new Master in Cluster?, which gives less details and has not been answered for 6 months, so I feel it is appropriate to start a new question.

(The stack I'm working on is open-source, but it's on a very early stage of development so not giving a link here.)

UPDATE: based on the answers, I have designed a simple consensus algorithm, you can find a JavaScript (CoffeeScript) implementation on GitHub: majority.js.

Answer 1

Me like this algorithm:

• Each node calculates the lowest known node id and sends a vote for leadership to this node
• If a node receives sufficiently many votes and the node also voted for itself, then it takes on the role of leader and starts publishing cluster state.

and at link below have some many algorithm of election master-node in cluster: https://www.elastic.co/blog/found-leader-election-in-general#the-zen-way

Also can see Raft-algorithm: https://raft.github.io

Answer 2

If your nodes also need to agree on things and their total order, you might want to consider Paxos. It's complicated, but nobody's come up with an easier solution for distributed consensus.

Answer 3

Leader election algorithms typically consider the split brain as a fault case to support. If you assume that it's not the nodes that fail but the networking, you may run into the case where all nodes are up, but fail to talk to each other. Then, you may end up with two masters.

If you can exclude "split brain" from your fault model (i.e. if you consider only node failures), your algorithm (leader is the one with the smallest address) is fine.

Answer 4

Use Apache ZooKeeper. It solves exactly this problem (and many more).