MonogoDB Replica Set Status Not changing from Star

Question

I have setup a MongoDB replica set with 3 nodes(vm's running CentOS). One node became Primary other 2 stuck in Startup. When these 2 nodes will change their states from startup to secondary.

aryabhata:PRIMARY> rs.status()
{
    "set" : "aryabhata",
    "date" : ISODate("2016-04-30T08:10:45.173Z"),
    "myState" : 1,
    "members" : [
            {
                    "_id" : 0,
                    "name" : "localhost.localdomain:27017",
                    "health" : 1,
                    "state" : 1,
                    "stateStr" : "PRIMARY",
                    "uptime" : 69091,
                    "optime" : Timestamp(1461935462, 1),
                    "optimeDate" : ISODate("2016-04-29T13:11:02Z"),
                    "electionTime" : Timestamp(1461934754, 1),
                    "electionDate" : ISODate("2016-04-29T12:59:14Z"),
                    "configVersion" : 459192,
                    "self" : true
            },
            {
                    "_id" : 1,
                    "name" : "repset1.com:27017",
                    "health" : 1,
                    "state" : 0,
                    "stateStr" : "STARTUP",
                    "uptime" : 92,
                    "optime" : Timestamp(0, 0),
                    "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                    "lastHeartbeat" : ISODate("2016-04-30T08:10:44.485Z"),
                    "lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
                    "pingMs" : 0,
                    "configVersion" : -2
            },
            {
                    "_id" : 2,
                    "name" : "repset2.com:27017",
                    "health" : 1,
                    "state" : 0,
                    "stateStr" : "STARTUP",
                    "uptime" : 68382,
                    "lastHeartbeat" : ISODate("2016-04-30T08:10:43.974Z"),
                    "lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
                    "pingMs" : 0,
                    "configVersion" : -2
            }
    ],
    "ok" : 1
}

Answer 1

From primary check whether you are able to connect to secondary

mongo --host repset1.com --port 27017

When the above one fails may be firewall or BindIP issue.

Check bind_ip (should be 0.0.0.0, change in mongodb.conf is it's 127.0.0.1):

netstat -plunt | grep :27017 | grep LISTEN

Look at the log-files of secondaries, why they are stuck. Did they receive the configuration details?

Try to reconfigure, see mongo replicaset reconfigure

Answer 2

For me the problem was that primary had authorization enabled. In this case the secondaries always stayed in STARTUP.

To use authorization you need to set keyFile in configuration file of all nodes (primary and secondary).

---

Create mongodb key file on linux:

openssl rand -base64 741 > mongodb.key
chmod 600 mongodb.key
chown mongod:mongod mongodb.key

mongod.conf file:

replication:
  replSetName: rs0

security:
  authorization: enabled
  keyFile: /home/mongodb.key

Source MongoDB replica set with simple password authentication