How to know if your pc is being monitored by employer

OK, I've been trawling though the Nagios3 documentation and have answered the port configuration part of my question...

The answer lies in the Object Inheritance model that exists within the Nagios configuration files. Essentially I created a custom variable in each host definition that specifies the unique ssh port on that machine:

define host {
    use              generic-host
    host_name        serverB
    address          10.0.1.3
    _sshport         67382
}

The hosts are grouped together inside the hostgroups_nagios2.cfg file:

# A list of your ssh-accessible servers
define hostgroup {
    hostgroup_name  ssh-servers
    alias           SSH servers
    members         localhost,serverB,serverC
}

This group is referenced inside the services_nagios2.cfg by the block that checks SSH:

# check that ssh services are running
define service {
    hostgroup_name                  ssh-servers
    service_description             SSH
    check_command                   check_ssh_port!$_HOSTSSHPORT
    use                             generic-service
    notification_interval           0 ; set > 0 if you want to be renotified
}

At the end of the check_ssh_port command you can see that I added the sshport variable $_HOSTSSHPORT that is inherited from each host inside the ssh-servers hostgroup as the checks are run.

Now, when adding new servers, I only have to modify my hosts_nagios2.cfg file with the details of the new host.

To enable backward compatibility, I also modified my generic-host_nagios2.cfg file adding the line _sshport 22 so that if for some reason I need to monitor some system running SSH on the default port, the port config will already be inherited from the generic host template.

I hope this helps others who find themselves in the same predicament. I am still trying to understand why the remote checks aren't using the custom config files on the remote servers.

I read about tools like Nagios, but I guess that’s a bit overkill for my situation.

Does anybody know how I can get started?

Easy. Look into setting up monitoring configurations with Monit. It’s a lightweight and easy to setup system monitoring tool that is very useful to setup in scenarios exactly as you describe; service goes down, restart it and alert me about it.

I’ve mainly use it for Apache web servers, but there are lots of examples of what can be done for other programs/software such as MySQL and such.

Setting up Monit.

The way I set it up is line this. First, install the Monit program itself like this:

sudo apt-get install monit

Once installed, then edit the config here; I prefer to use nano but feel free to use whatever text editor you prefer:

sudo nano /etc/monit/monitrc

Adjust the default daemon values to check services every 60 seconds with a start delay of 120:

set daemon 60
with start delay 60

Then find the mailserver area of monitrc and add the following line. Postfix or an SMTP needs to be active for this to work. I typically have Postfix installed on my servers so I use the following setup:

set mailserver localhost

Then I make sure a Monit config directory is setup like this:

sudo mkdir -p /etc/monit/conf.d

Setting up a Monit Apache2 monitoring ruleset.

Now—like I said—I mainly use Monit for Apache monitoring so this is a simple config I like to use but the basic concept is similar for MySQL, MongoDB or other things. I would save it in this file:

sudo nano /etc/monit/conf.d/apache2.conf

And this would be the contents of that file:

check process apache with pidfile /var/run/apache2.pid
  start "/usr/sbin/service apache2 start"
  stop  "/usr/sbin/service apache2 stop"
  if failed host 127.0.0.1 port 80
    with timeout 15 seconds
  then restart
  alert email_address@example.com only on { timeout, nonexist }

The syntax is fairly self-explanatory, but basically:

• The process hinges on the apache2.pid; be sure to change that to match the actual location of your apache2.pid or httpd.pid in your environment.
• Then has commands connected to the processes of start and stop.
• And has logic that monitors the web server on port 80 on localhost (127.0.0.1)
• And only acts of the server is unreachable for 15 seconds.
• An if it has to act, it attempts a restart.
• And then sends an alert to the specified email address on incidents of the server timing out or not existing.

Setting up a Monit MySQL monitoring ruleset.

Based on the examples I linked to above, I would assume a config like this would work for MySQL. First, create a file like this:

sudo nano /etc/monit/conf.d/mysql.conf

And I have adapted the example so it—I would assume—behaves similarly to what I have setup for Apache:

check process mysqld with pidfile /var/run/mysqld/mysqld.pid
  start program = "/usr/sbin/service mysql start"
  stop program = "/usr/sbin/service mysql stop"
  if failed host 127.0.0.1 port 3306 protocol mysql
    with timeout 15 seconds
  then restart
  alert email_address@example.com only on { timeout, nonexist }

Of course that should be tweaked to match your actual working environment—such as adjusting the location of the mysqld.pid, the email address and such—but past that it’s fairly generic in ideas/implementation.

Once that is set, restart monit and all should be good:

sudo service monit restart

Setting up a Monit MongoDB monitoring ruleset.

To create a MongoDB monitoring ruleset, create a file like this:

sudo nano /etc/monit/conf.d/mongod.conf

And here is the MongoDB monitoring rule; note this matches the active MongoDB daemon and not a PID (aka: mongod.lock) since it didn’t seem to work with that:

check process mongod matching "/usr/bin/mongod"
  start program = "/usr/sbin/service mongod start"
  stop program = "/usr/sbin/service mongod stop"
  if failed host 127.0.0.1 port 27017 protocol http
    with timeout 15 seconds
  then restart
  alert email_address@example.com only on { timeout, nonexist }

Of course that should be tweaked to match your actual working environment—such as adjusting the actual path of the /usr/bin/mongod binary, the email address and such—but past that it’s fairly generic in ideas/implementation.

Once that is set, restart monit and all should be good:

sudo service monit restart

Monitoring Monit.

You can follow the Monit log to see it in action:

sudo tail -f -n 200 /var/log/monit.log

And as a test, you can simply stop the MySQL or MongoDB server and then see what shows up in that log. If all goes well you should see the whole monitoring process and restart happen including an email being sent to the address you have set in the config.