Nagios Event Handling via Script

Nagios Monitoring is a great tool. Anyone reading this page knows that. I use Nagios Event Handlers to automatically take care of some issues that can be identified and dealt with by calling to a script or using long commands piped together for the Event Handler Command.

I was recently shown that when Nagios makes a call to the Event Handler script, it can pass parameters as part of the command that the script can use to perform different actions based on the Service State, State Type, or any Nagios Variable that you want to pass.

So instead of a single command line, we can evaluate the condition and execute different commands based on the current state.

To start, lets define a new command. In the nagios/conf.d/commands.cfg file we can add something like:

 
define command{
        command_name    event-handler-name-goes-here
        command_line    /opt/scripts/eventhandlerscript.sh $SERVICESTATE$ $SERVICESTATETYPE$ $SERVICEATTEMPT$ $HOSTADDRESS$
        }

Here is a skeleton script that we can use to build the event handler script for service conditions based on the variables that we just passed from the Nagios command:

#!/bin/sh

# $1 is $SERVICESTATE$ (OK WARNING UNKNOWN CRITICAL) 
# $2 is $SERVICESTATETYPE$ (SOFT HARD) 
# $3 is $SERVICEATTEMPT$ (1 through 4) 
# $4 is $HOSTADDRESS$ (4.2.2.1 or something.something.com)

case "$1" in
OK)
	;;
WARNING)
	;;
UNKNOWN)
	;;
CRITICAL)
	case "$2" in
	SOFT)
		case "$3" in
		1)
                   # 1st notification, usually watch for a 2nd. 
			;;
		2)
                   # 2nd notification, try to fix with a script
			;;
		3)
		   # Script must have failed, try a 2nd script or send out email notifications.
			;;
			esac
			;;
	HARD)
		 case "$3" in
		1)
                   # 1st notification, usually watch for a 2nd. 
			;;
		2)
                   # 2nd notification, try to fix with a script
			;;
		3)
		   # Script must have failed, try a 2nd script or send out email notifications.
			;;
			esac
			;;
		;;
	esac
	;;
esac
exit 0

Save this file, Make it executable with ‘CHMOD +x filename’. Make sure the filename matches what you have in the Nagios command.

So when Nagios detects a problem, it will call the command_name here which calls the script in the command line. You can have it run more tests and email results, try restarting services, whatever you like.

One last thing. The script is executed with the nagios user id. So for some commands that require root privileges, you may need to add certain commands to sudoers for the nagios account.