I'm having the hardest time trying to figure this one out. I have a 2 server replica set with one arbiter running alongside the current primary.
When I connect or issue queries, I get intermittent errors which range from any one of the following:
- Could not determine master.
- Could not connect to [put host here].
- Broken pipe exceptions.
- MongoCursorExceptions.
It seems like they happen more often the longer the servers are up. After awhile, no connections can be made and I get the following error when I try and login to the mongo shell:
MongoDB shell version: 1.8.1
connecting to: test
Wed May 11 16:36:50 MessagingPort recv() errno:104 Connection reset by peer 127.0.0.1:27017
Wed May 11 16:36:50 SocketException: remote: error: 9001 socket exception [1]
Wed May 11 16:36:50 DBClientCursor::init call() failed
exception: DBClientBase::findOne: transport error: 127.0.0.1 query: { whatsmyuri: 1 }
(BTW, with verbose (-vvvv) logging, the 9001 socket exception seems to come up with every query that is run.)
When I restart the mongo servers, things come back up and start working, but not for long.
No matter what combination of connection options I used, nothing worked. When I changed both webservers to connect to the primary replica everything started working. No errors. No automatic failover :(. Weird thing was, that the replicaSet option was still set which means the PHP driver still needs to determine whether or not the one it connected to was a master and if not connect to it, if so, do nothing.
This tells me that the issue resides in the PHP driver where it tries to connect to the master after attempting to determine if it is connected to the master or not. Just to try and back up my theory, I told one of the servers to connect to the secondary server. Immediately, I started getting errors from that server. Not the other server which was still connected to the primary. Switch back, no errors.
Can anyone shed any light on this?
My setup: - 2 webservers - Apache/PHP on both - MongoDB Replica-Set span across both - Secondary on one - Primary and Arbiter on the other - MongoDB 1.8.1 - PHP Driver 1.1.4 - PHP 5.3.3 - Apache 2.2.17 - RedHat 5.5 (Tikanga)
