Note this looks long, but provides context and lists my main questions at the bottom. I researched all parts and included references.
I used the Google Cloud Launcher to create a Mongo database. This created a replica set of two Mongo servers (primary and secondary) and an arbiter on three separate VMs. I have not changed any VM configurations (other than opening the firewall). I have SSH'd into both of the servers and verified Mongo is working with replication as expected. I'm changing the external IPs from x.x.x.x to FIRST.EXTERNAL.IP and SECOND.EXTERNAL.IP to make the post more readable (and security).
I'm now trying to connect to the database through a Java MongoClient:
MongoClientOptions.Builder mongoClientOptions = MongoClientOptions.builder().connectionsPerHost(40)
.writeConcern(WriteConcern.ACKNOWLEDGED).readPreference(ReadPreference.secondaryPreferred())
.readConcern(ReadConcern.LOCAL).socketTimeout(60000).threadsAllowedToBlockForConnectionMultiplier(1500);
MongoClientURI mongoClientURI = new MongoClientURI("mongodb://FIRST.EXTERNAL.IP:27017,SECOND.EXTERNAL.IP:27017", mongoClientOptions);
mongoClient = new MongoClient(mongoClientURI);
I wanted to verify I should include both the primary external IP and the secondary external IP in the URI. I did not include the arbiter, since it doesn't make sense for the MongoClient to interact directly with the arbiter.
The following shows the Java exception when I attempt a find(). Insert() creates the same error, except with WritePreferenceServerSelector instead of ReadPref..
1449 [main] INFO org.mongodb.driver.cluster - Cluster created with settings {hosts=[SECOND.EXTERNAL.IP:27017, FIRST.EXTERNAL.IP:27017], mode=MULTIPLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=60000}
1449 [main] INFO org.mongodb.driver.cluster - Adding discovered server SECOND.EXTERNAL.IP:27017 to client view of cluster
1558 [main] INFO org.mongodb.driver.cluster - Adding discovered server FIRST.EXTERNAL.IP:27017 to client view of cluster
1579 [main] INFO org.mongodb.driver.cluster - No server chosen by ReadPreferenceServerSelector{readPreference=ReadPreference{name=secondaryPreferred}} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=SECOND.EXTERNAL.IP:27017, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=FIRST.EXTERNAL.IP:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
1743 [cluster-ClusterId{value='588b214418786c23f4fcf845', description='null'}-FIRST.EXTERNAL.IP:27017] INFO org.mongodb.driver.connection - Opened connection [connectionId{localValue:2, serverValue:12}] to FIRST.EXTERNAL.IP:27017
1766 [cluster-ClusterId{value='588b214418786c23f4fcf845', description='null'}-SECOND.EXTERNAL.IP:27017] INFO org.mongodb.driver.connection - Opened connection [connectionId{localValue:1, serverValue:9}] to SECOND.EXTERNAL.IP:27017
1790 [cluster-ClusterId{value='588b214418786c23f4fcf845', description='null'}-FIRST.EXTERNAL.IP:27017] INFO org.mongodb.driver.cluster - Monitor thread successfully connected to server with description ServerDescription{address=FIRST.EXTERNAL.IP:27017, type=REPLICA_SET_PRIMARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 4, 1]}, minWireVersion=0, maxWireVersion=5, maxDocumentSize=16777216, roundTripTimeNanos=41160155, setName='rs0', canonicalAddress=my-project-name-mongo-server-1:27017, hosts=[my-project-name-mongo-server-1:27017, my-project-name-mongo-server-2:27017], passives=[], arbiters=[my-project-name-mongo-arbiter-1:27017], primary='my-project-name-mongo-server-1:27017', tagSet=TagSet{[]}, electionId=7fffffff0000000000000003, setVersion=3, lastWriteDate=Fri Jan 27 02:30:26 PST 2017, lastUpdateTimeNanos=165366848909779}
1791 [cluster-ClusterId{value='588b214418786c23f4fcf845', description='null'}-FIRST.EXTERNAL.IP:27017] INFO org.mongodb.driver.cluster - Discovered cluster type of REPLICA_SET
1791 [cluster-ClusterId{value='588b214418786c23f4fcf845', description='null'}-FIRST.EXTERNAL.IP:27017] INFO org.mongodb.driver.cluster - Adding discovered server my-project-name-mongo-server-1:27017 to client view of cluster
1793 [cluster-ClusterId{value='588b214418786c23f4fcf845', description='null'}-FIRST.EXTERNAL.IP:27017] INFO org.mongodb.driver.cluster - Adding discovered server my-project-name-mongo-server-2:27017 to client view of cluster
1794 [cluster-ClusterId{value='588b214418786c23f4fcf845', description='null'}-FIRST.EXTERNAL.IP:27017] INFO org.mongodb.driver.cluster - Adding discovered server my-project-name-mongo-arbiter-1:27017 to client view of cluster
1796 [cluster-ClusterId{value='588b214418786c23f4fcf845', description='null'}-FIRST.EXTERNAL.IP:27017] INFO org.mongodb.driver.cluster - Server SECOND.EXTERNAL.IP:27017 is no longer a member of the replica set. Removing from client view of cluster.
1797 [cluster-ClusterId{value='588b214418786c23f4fcf845', description='null'}-FIRST.EXTERNAL.IP:27017] INFO org.mongodb.driver.cluster - Server FIRST.EXTERNAL.IP:27017 is no longer a member of the replica set. Removing from client view of cluster.
1798 [cluster-ClusterId{value='588b214418786c23f4fcf845', description='null'}-FIRST.EXTERNAL.IP:27017] INFO org.mongodb.driver.cluster - Canonical address my-project-name-mongo-server-1:27017 does not match server address. Removing FIRST.EXTERNAL.IP:27017 from client view of cluster
1799 [cluster-ClusterId{value='588b214418786c23f4fcf845', description='null'}-my-project-name-mongo-server-1:27017] INFO org.mongodb.driver.cluster - Exception in monitor thread while connecting to server my-project-name-mongo-server-1:27017
com.mongodb.MongoSocketException: my-project-name-mongo-server-1: unknown error
at com.mongodb.ServerAddress.getSocketAddress(ServerAddress.java:188)
at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:57)
at com.mongodb.connection.SocketStream.open(SocketStream.java:58)
at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:115)
at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:113)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.UnknownHostException: my-project-name-mongo-server-1: unknown error
//(then it just repeats the socket exception through connection retries I believe)
My current theory is the hostname configuration originally set by the Google Cloud Launcher needs to be updated with the external IPs. This would mean changing my-project-name-mongo-server-1 to FIRST.EXTERNAL.IP and the same for the second server as seen in the Mongo Documentation. If this is correct, should I just leave the arbiter hostname? (Side note - how does my-project-name-mongo-server-1:27017 resolve? Is it an internal IP to the Google Cloud?)
As far as I can tell, this would fix the my issue and the exception Canonical address my-project-name-mongo-server-1:27017 does not match server address.
To recap each of my questions about my issue:
- Which IPs should I include in the Java MongoClient URI?
- Should I reconfigure the member hostnames to the external IPs? If so, which ones?
- Optionally, it'd be helpful to explain if my-project-name-mongo-server-1:27017 is a local hostname.
- Since you know my application design, would there be a better way to network my external connection?
