I tried to recover from a sudden shutdown of mongo, but I am experiencing some problems. Here is the log file
***** SERVER RESTARTED *****
Thu Apr 25 01:09:17 [initandlisten] MongoDB starting : pid=19996 port=27017 dbpath=/var/lib/mongodb 64-bit host=Vault
Thu Apr 25 01:09:17 [initandlisten] db version v2.2.3, pdfile version 4.5
Thu Apr 25 01:09:17 [initandlisten] git version: f570771a5d8a3846eb7586eaffcf4c2f4a96bf08
Thu Apr 25 01:09:17 [initandlisten] build info: Linux ip-10-2-29-40 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_49
Thu Apr 25 01:09:17 [initandlisten] options: { config: "/etc/mongodb.conf", dbpath: "/var/lib/mongodb", logappend: "true", logpath: "/var/log/mongodb/mongodb.log", nohttpinterface: "true" }
Thu Apr 25 01:09:17 [initandlisten] journal dir=/var/lib/mongodb/journal
Thu Apr 25 01:09:17 [initandlisten] recover begin
Thu Apr 25 01:09:17 [initandlisten] recover lsn: 3778163987
Thu Apr 25 01:09:17 [initandlisten] recover /var/lib/mongodb/journal/j._0
Thu Apr 25 01:09:17 [initandlisten] dbexception during recovery: 13537 journal file header invalid. This could indicate corruption in a journal file, or perhaps a crash where sectors in file header were in flight written out of order at time of crash (unlikely but possible).
Thu Apr 25 01:09:17 [initandlisten] exception in initAndListen: 13537 journal file header invalid. This could indicate corruption in a journal file, or perhaps a crash where sectors in file header were in flight written out of order at time of crash (unlikely but possible)., terminating
Thu Apr 25 01:09:17 dbexit:
Thu Apr 25 01:09:17 [initandlisten] shutdown: going to close listening sockets...
Thu Apr 25 01:09:17 [initandlisten] shutdown: going to flush diaglog...
Thu Apr 25 01:09:17 [initandlisten] shutdown: going to close sockets...
Thu Apr 25 01:09:17 [initandlisten] shutdown: waiting for fs preallocator...
Thu Apr 25 01:09:17 [initandlisten] shutdown: lock for final commit...
Thu Apr 25 01:09:17 [initandlisten] shutdown: final commit...
Thu Apr 25 01:09:17 [initandlisten] shutdown: closing all files...
Thu Apr 25 01:09:17 [initandlisten] closeAllFiles() finished
Thu Apr 25 01:09:17 [initandlisten] shutdown: removing fs lock...
Thu Apr 25 01:09:17 dbexit: really exiting now
Best Answer
Based on your log file, you are using MongoDB version v2.2.3. This version is not officially supported by MongoDB, so I'd recommend you plan to upgrade to a newer version. The latest stable version, MongoDB 4.0.9.
As per the log message, the journal file j._0 in /var/lib/mongodb/journal/ folder is corrupted.
Try, removing the j._0 file and do the recovery process.
Disclaimer: you may lose some data that present in the journal file.