Mongodb service won't start

Question

I am not sure why, and the service was working fine just yesterday, all of a sudden I can not get MongoDB to start up.

[root@purr ~]# service mongod start
Starting mongod: Thu Aug 29 23:26:07.550 
Thu Aug 29 23:26:07.550 warning: 32-bit servers don't have journaling enabled by default. Please use --journal if you want durability.
Thu Aug 29 23:26:07.550 
about to fork child process, waiting until server is ready for connections.
forked process: 5078
all output going to: /var/log/mongo/mongod.log
ERROR: child process failed, exited with error number 100
                                                           [FAILED]

I looked in /var/log/mongo/mongod.log and the following:

***** SERVER RESTARTED *****


Thu Aug 29 23:29:05.874 [initandlisten] MongoDB starting : pid=5118 port=27017 dbpath=/var/lib/mongo 32-bit host=purr.ipetfindr.com
Thu Aug 29 23:29:05.874 [initandlisten]
Thu Aug 29 23:29:05.874 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
Thu Aug 29 23:29:05.874 [initandlisten] **   32 bit builds are limited to less than 2GB of data (or less with --journal).
Thu Aug 29 23:29:05.874 [initandlisten] **   Note that journaling defaults to off for 32 bit and is currently off.
Thu Aug 29 23:29:05.874 [initandlisten] **   See http://dochub.mongodb.org/core/32bit
Thu Aug 29 23:29:05.874 [initandlisten]
Thu Aug 29 23:29:05.874 [initandlisten] ** WARNING: You are running in OpenVZ. This is known to be broken!!!
Thu Aug 29 23:29:05.875 [initandlisten]
Thu Aug 29 23:29:05.875 [initandlisten] db version v2.4.6
Thu Aug 29 23:29:05.875 [initandlisten] git version: b9925db5eac369d77a3a5f5d98a145eaaacd9673
Thu Aug 29 23:29:05.875 [initandlisten] build info: Linux bs-linux32.10gen.cc 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49
Thu Aug 29 23:29:05.875 [initandlisten] allocator: system
Thu Aug 29 23:29:05.875 [initandlisten] options: { config: "/etc/mongod.conf", dbpath: "/var/lib/mongo", fork: "true", logappend: "true", logpath: "/var/log/mongo/mongod.log", pidf$
**************
Unclean shutdown detected.
Please visit http://dochub.mongodb.org/core/repair for recovery instructions.
*************
Thu Aug 29 23:29:05.875 [initandlisten] exception in initAndListen: 12596 old lock file, terminating
Thu Aug 29 23:29:05.875 dbexit:
Thu Aug 29 23:29:05.875 [initandlisten] shutdown: going to close listening sockets...
Thu Aug 29 23:29:05.875 [initandlisten] shutdown: going to flush diaglog...
Thu Aug 29 23:29:05.875 [initandlisten] shutdown: going to close sockets...
Thu Aug 29 23:29:05.875 [initandlisten] shutdown: waiting for fs preallocator...
Thu Aug 29 23:29:05.875 [initandlisten] shutdown: closing all files...
Thu Aug 29 23:29:05.875 [initandlisten] closeAllFiles() finished
Thu Aug 29 23:29:05.875 dbexit: really exiting now

How should I go about fixing this as restarting the server, and service does not seem to work.

ERROR after repair

Not sure what to do it is saying something about error couldn't open file /var/lib/mongo/local.ns terminating

***** SERVER RESTARTED *****


Fri Aug 30 04:49:43.041 [initandlisten] MongoDB starting : pid=3300 port=27017 dbpath=/var/lib/mongo 32-bit host=purr.ipetfindr.com
Fri Aug 30 04:49:43.041 [initandlisten]
Fri Aug 30 04:49:43.041 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
Fri Aug 30 04:49:43.041 [initandlisten] **   32 bit builds are limited to less than 2GB of data (or less with --journal).
Fri Aug 30 04:49:43.041 [initandlisten] **   Note that journaling defaults to off for 32 bit and is currently off.
Fri Aug 30 04:49:43.041 [initandlisten] **   See http://dochub.mongodb.org/core/32bit
Fri Aug 30 04:49:43.041 [initandlisten]
Fri Aug 30 04:49:43.041 [initandlisten] ** WARNING: You are running in OpenVZ. This is known to be broken!!!
Fri Aug 30 04:49:43.041 [initandlisten]
Fri Aug 30 04:49:43.041 [initandlisten] db version v2.4.6
Fri Aug 30 04:49:43.041 [initandlisten] git version: b9925db5eac369d77a3a5f5d98a145eaaacd9673
Fri Aug 30 04:49:43.041 [initandlisten] build info: Linux bs-linux32.10gen.cc 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49
Fri Aug 30 04:49:43.041 [initandlisten] allocator: system
Fri Aug 30 04:49:43.041 [initandlisten] options: { config: "/etc/mongod.conf", dbpath: "/var/lib/mongo", fork: "true", logappend: "true", logpath: "/var/log/mongo/mongod.log", pidf$
Fri Aug 30 04:49:43.058 [initandlisten] couldn't open /var/lib/mongo/local.ns errno:13 Permission denied
Fri Aug 30 04:49:43.058 [initandlisten] error couldn't open file /var/lib/mongo/local.ns terminating
Fri Aug 30 04:49:43.059 dbexit:
Fri Aug 30 04:49:43.059 [initandlisten] shutdown: going to close listening sockets...
Fri Aug 30 04:49:43.059 [initandlisten] shutdown: going to flush diaglog...
Fri Aug 30 04:49:43.059 [initandlisten] shutdown: going to close sockets...
Fri Aug 30 04:49:43.059 [initandlisten] shutdown: waiting for fs preallocator...
Fri Aug 30 04:49:43.059 [initandlisten] shutdown: closing all files...
Fri Aug 30 04:49:43.059 [initandlisten] closeAllFiles() finished
Fri Aug 30 04:49:43.059 [initandlisten] shutdown: removing fs lock...
Fri Aug 30 04:49:43.059 dbexit: really exiting now

Answer 1

For me, the reason for not starting turned out to be an orphaned lock file at /var/lib/mongo/mongo.lock When I deleted that file, mongo would then start up OK. My system had had some messy crashes prior to this. [Fedora 14]

Answer 2

It's all in your error message - seems like unclean shutdown was detected. See http://docs.mongodb.org/manual/tutorial/recover-data-following-unexpected-shutdown/ for detailed information.

In my expirience, usually it helps to run mongod.exe with --repair option ro repair DB.

Answer 3

I can't upvote/comment yet, but +1 for manually removing the lock file haha.

My C9 workspace crashed on me and triggered an unexpected shutdown. The API advises: https://docs.mongodb.com/manual/tutorial/recover-data-following-unexpected-shutdown/

.. but removing data/mongo.lock worked for me :).

Also, just in case you're getting a connection refusal (which happened to me), running the repair command before removing the lock file could solve your problem (it did mine).

sudo -u mongodb mongod --repair --dbpath /var/lib/mongodb/

Answer 4

When I ran mongod.exe from Windows Terminal I got a message Unrecognized option: mp. There was an empty mp: in the end of mongod.cfg. Removing that solved the problem for me.

Answer 5

I verified permissions but all was good (mongod:mongod). As I'm working on a large project and from a similar issue in our dev environment where we had a script consuming all available disk space, I could see in the error messages that mongod needs at least 3.7Gb free disk space to run..

I checked my own disk space only to see that less than 2Gb was remaining. After moving / erasing some data I can successfully start mongod again.

Hope this helps ;-)