nagios-event-triger-AutoRecover
10余年的淮南网站建设经验，针对设计、前端、开发、售后、文案、推广等六对一服务，响应快，48小时及时工作处理。营销型网站的优势是能够根据用户设备显示端的尺寸不同，自动调整淮南建站的显示方式，使网站能够适用不同显示终端，在浏览器中调整网站的宽度，无论在任何一种浏览器上浏览网站，都能展现优雅布局与设计，从而大程度地提升浏览体验。创新互联公司从事“淮南网站设计”,“淮南网站推广”以来，每个客户项目都认真落实执行。
思路：
use NRPE to execute the necessary commands on the remote hosts
In order to adapt the scheme from the Nagios docs to work on remote servers as well three things need to be done:
1.The command that is executed by the event handler script should be changed to use NRPE
2.On the remote machine the nagios user (under which the NRPE service is running) should be given some sudo rights so that it is actually allowed to start a service.
3.The NRPE configuration on the remote machine should of course be changed to include the new command(s) for starting services.
1. nagios manage server
（1）vi localhost.cfg
define service{
use generic-service
host_name test2.bigdata.com
service_description gmond
check_command check_nrpe_eventhandler!check_gmond
notifications_enabled 1
notification_interval 0
max_check_attempts 4
event_handler restart-service!gmond
}
define service{
use generic-service
host_name test2.bigdata.com
service_description MySQLd
check_command check_nrpe_eventhandler!check_mysqld
notifications_enabled 1
notification_interval 0
max_check_attempts 5
event_handler restart-service!mysqld
}（2）vi commands.cfg
define command{
command_name check_nrpe_eventhandler
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -t 40 -c $ARG1$
}
define command {
command_name restart-service
command_line $USER1$/eventhandlers/event_handler_script.sh $SERVICESTATE$ $SERVICESTATETYPE$ $SERVICEATTEMPT$ $HOSTADDRESS$ $ARG1$ $SERVICEDESC$
}利用后面编写的通用的事件处理脚本模块文件 /usr/local/nagios/libexec/eventhandlers/event_handler_script.sh 传递监控服务所需的参数$ARG1$
2. on remote machine with NRPE running
（1）vi nrpe.cfg
command[check_gmond]=/usr/local/nagios/libexec/check_gmond command[restart_gmond]=/usr/bin/sudo /etc/init.d/gmond restart command[check_mysqld]=/usr/local/nagios/libexec/check_mysqld command[restart_mysqld]=/usr/bin/sudo /usr/local/nagios/libexec/restart_mysqld
(2)edit your service manage script on remote machine.
/usr/local/nagios/libexec/check_mysqld
/usr/local/nagios/libexec/restart_mysqld
3. 通用的事件处理脚本模块文件 /usr/local/nagios/libexec/eventhandlers/event_handler_script.sh内容如下
#!/bin/sh # # Event handler script for restarting the web server on the local machine # # Note: This script will only restart the web server if the service is # retried 3 times (in a "soft" state) or if the web service somehow # manages to fall into a "hard" error state. # update 2015/10/23 # version: 0.2 date=`date` # What state is the HTTP service in case "$1" in OK) # The service just came back up, so don't do anything... ;; WARNING) # We don't really care about warning states, since the service is probably still running... ;; UNKNOWN) # We don't know what might be causing an unknown error, so don't do anything... ;; CRITICAL) # We don't really care about warning states, since the service is probably still running... # Aha! The HTTP service appears to have a problem - perhaps we should restart the server... # Is this a "soft" or a "hard" state? case "$2" in # We're in a "soft" state, meaning that Nagios is in the middle of retrying the # check before it turns into a "hard" state and contacts get notified... SOFT) # What check attempt are we on? We don't want to restart the web server on the first # check, because it may just be a fluke! case "$3" in # Wait until the check has been tried 3 times before restarting the web server. # If the check fails on the 4th time (after we restart the web server), the state # type will turn to "hard" and contacts will be notified of the problem. # Hopefully this will restart the web server successfully, so the 4th check will # result in a "soft" recovery. If that happens no one gets notified because we # fixed the problem! 3) echo -n "Restarting service $6 (3rd soft critical state)...\n" # Call NRPE to restart the service on the remote machine /usr/local/nagios/libexec/check_nrpe -H $4 -c restart_$5 echo "$date -restart $6 on server $4 -at retry $3 times -SOFT" >> /tmp/eventhandlers ;; esac ;; # The HTTP service somehow managed to turn into a hard error without getting fixed. # It should have been restarted by the code above, but for some reason it didn't. # Let's give it one last try, shall we? # Note: Contacts have already been notified of a problem with the service at this # point (unless you disabled notifications for this service) HARD) echo -n "Restarting $6 service...\n" # Call the init script to restart the NRPE server echo "$date -restart $6 on server $4 -at retry $3 times -HARD" >> /tmp/eventhandlers /usr/local/nagios/libexec/check_nrpe -H $4 -c restart_$5 ;; esac ;; esac exit 0