I’m running a simple development server (Ubuntu) on which MySQL and MongoDB sometimes crash. I always restart them with sudo service mysql restart
.
Although I know I need to investigate why they crash—and I will—I’m currently looking for a way to automatically restart them after they've crashed. I guess I need to have some kind of daemon which pings them and restarts them if they are not responsive anymore, but I'm unsure of how to do this.
I read about tools like Nagios, but I guess that’s a bit overkill for my situation.
Does anybody know how I can get started?
Best Answer
Easy. Look into setting up monitoring configurations with Monit. It’s a lightweight and easy to setup system monitoring tool that is very useful to setup in scenarios exactly as you describe; service goes down, restart it and alert me about it.
I’ve mainly use it for Apache web servers, but there are lots of examples of what can be done for other programs/software such as MySQL and such.
Setting up Monit.
The way I set it up is line this. First, install the Monit program itself like this:
Once installed, then edit the config here; I prefer to use
nano
but feel free to use whatever text editor you prefer:Adjust the default daemon values to check services every 60 seconds with a start delay of 120:
Then find the
mailserver
area ofmonitrc
and add the following line. Postfix or an SMTP needs to be active for this to work. I typically have Postfix installed on my servers so I use the following setup:Then I make sure a Monit config directory is setup like this:
Setting up a Monit Apache2 monitoring ruleset.
Now—like I said—I mainly use Monit for Apache monitoring so this is a simple config I like to use but the basic concept is similar for MySQL, MongoDB or other things. I would save it in this file:
And this would be the contents of that file:
The syntax is fairly self-explanatory, but basically:
apache2.pid
; be sure to change that to match the actual location of yourapache2.pid
orhttpd.pid
in your environment.start
andstop
.80
onlocalhost
(127.0.0.1
)Setting up a Monit MySQL monitoring ruleset.
Based on the examples I linked to above, I would assume a config like this would work for MySQL. First, create a file like this:
And I have adapted the example so it—I would assume—behaves similarly to what I have setup for Apache:
Of course that should be tweaked to match your actual working environment—such as adjusting the location of the
mysqld.pid
, the email address and such—but past that it’s fairly generic in ideas/implementation.Once that is set, restart
monit
and all should be good:Setting up a Monit MongoDB monitoring ruleset.
To create a MongoDB monitoring ruleset, create a file like this:
And here is the MongoDB monitoring rule; note this matches the active MongoDB daemon and not a PID (aka:
mongod.lock
) since it didn’t seem to work with that:Of course that should be tweaked to match your actual working environment—such as adjusting the actual path of the
/usr/bin/mongod
binary, the email address and such—but past that it’s fairly generic in ideas/implementation.Once that is set, restart
monit
and all should be good:Monitoring Monit.
You can follow the Monit log to see it in action:
And as a test, you can simply stop the MySQL or MongoDB server and then see what shows up in that log. If all goes well you should see the whole monitoring process and restart happen including an email being sent to the address you have set in the config.