Couchbase Connection - External ip instead of inte

Question

In the same data center I have an application server (the client) connecting to a couchbase cluster containing 3 nodes.

I would like the client to connect via the internal IP and not the external for optimized performance. Let's assume these are my IPs:

• node1InternalIP / node1ExternalIP
• node2InternalIP / node2ExternalIP
• node3InternalIP / node3ExternalIP

When creating my couchbase client (java code) im providing my internal IPs but when the connection has been established I get these logs:

2014-07-28 12:33:21.030 INFO net.spy.memcached.auth.AuthThread:  Authenticated to /node1InternalIP :11210
2014-07-28 12:33:21.142 INFO net.spy.memcached.auth.AuthThread:  Authenticated to /node2InternalIP :11210
2014-07-28 12:33:21.253 INFO net.spy.memcached.auth.AuthThread:  Authenticated to /node3InternalIP :11210
2014-07-28 12:33:21.374 INFO com.couchbase.client.vbucket.provider.BucketConfigurationProvider:  Carrier config not available, bootstrapped through HTTP.
2014-07-28 12:33:21.544 INFO com.couchbase.client.CouchbaseConnection:  Added {QA sa=cache2.lac.company.info/node1ExternalIP:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2014-07-28 12:33:21.545 INFO com.couchbase.client.CouchbaseConnection:  Added {QA sa=cache3.lac.company.info/node2ExternalIP:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2014-07-28 12:33:21.545 INFO com.couchbase.client.CouchbaseConnection:  Added {QA sa=cache4.lac.company.info/node3ExternalIP:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue

So it seems that authentication is done against the internal IP but the connection itself goes via the external IP - which concerns me.

In addition, once in a while I'm getting CheckedOperationTimeoutException:

net.spy.memcached.internal.CheckedOperationTimeoutException: Timed out waiting for operation - failing node: cache2.lac.company.info/node1ExternalIP:11210

The exception shows that the client is getting timeout from an external ip connection.

How can I make sure my connection will use the internal IPs?

Answer 1

Couchbase uses a single address (IP or hostname) to identify each cluster node. These addresses are then sent to clients when bootstrapping so they can connect to all the nodes in the cluster. What you are seeing will be due to you initially setting up the cluster nodes using the external IP address as their names, and this is what is sent to clients when they connect.

To achieve what you want to do you will need to use hostnames (Admin guide) instead of IP addresses for identifying each node, then configure your clients' DNS to use the external IP address, but the nodes' DNS (or /etc/hosts) to use the internal IP.