Nagios Event Handlers

One of my pet projects is automate some responses to Nagios alerts.  Nagios has the eventhandler functionality built in for this case.  The simple case I had was to restart a service which was always flaky on my Nagios server. 

I had installed the check_system_procs plugin to monitor the service as my first step.  It works well and we usually react quickly.  However it would be really nice to restart the service automatically before an alert is sent.  I followed the example on Nagios.org, but since the event_handlers are executed as the nagios user I had several issues.

The script needed to be run as the "root" user.  Using sudo, I was able to execute the script without an issue from the command line.  However as an event handler, the script never seemed to work.  I added a little output line to create some log data.  The script was definitely running.

One thing that was not mentioned in the several bits I had read was some minor configuration tweaks to sudo.  Specifically the default value to "requiretty".  Took me awhile, but I had solved it.  You can read more about the use of Nagios event handlers to restart a local service here with some additional details.

Next... I need to come up with a system to restart a service on a remote service in my environment.  Shouldn't be too hard now right?