centostricks

Just another WordPress.com site

Troubleshooting Nagios/Nrpe Issues


Troubleshooting NRPE (Nagios Remote Plugin Executor) Client

Nagios Server communicates with nrpe via SSL. So, all the communication is encrypted.

Common Errors while configuring NRPE

1. CHECK_NRPE: Error – Could not complete SSL handshake

Solution: 

This error message could be due to several problems:

1. SSL is disabled. Make sure both the NRPE daemon and the check_nrpe plugin were compiled with SSL support (During ./configure)

2. Incorrect file permissions. Make sure the NRPE config file (nrpe.cfg) is readable by the user (i.e. nagios) that executes the NRPE binary from inetd/xinetd.

3. The command that the NRPE daemon was asked to run took longer than 10 seconds to execute. This is the most likely cause if the error message was “CHECK_NRPE: Socket timeout after 10 seconds”. Use the –t command line option to specify a longer timeout for the check_nrpe plugin. The following example will increase the timeout to 30 seconds:
/usr/local/nagios/libexec/check_nrpe -H localhost -c somecommand -t 30

4. The NRPE daemon is not installed or not running on the remote host. Verify that the NRPE daemon is running as a standalone daemon or under inetd/xinetd with one of the following commands:

# ps -ef | grep nrpe
# netstat -at | grep nrpe
5. There is a firewall that is blocking the communication between the monitoring host (which runs the check_nrpe plugin) and the remote host (which runs the NRPE daemon). Verify that the firewall rules ( Eg : iptables) that are running on the remote host allow for communication and make sure there isn’t a physical firewall that is located between the monitoring host and the remote host.

6. There could be a network issue. Check ping on the remote IP address on which you are trying to connect

2. The check_nrpe plugin returns “CHECK_NRPE: Received 0 bytes from daemon”

Solution :

First thing you should do is check the remote server logs for an error message. Seriously. :-) This error could be due to the following problem:

1.  The check_nrpe plugin was unable to complete an SSL handshake with the NRPE daemon. An error message in the logs should indicate whether or not this was the case. Check the versions of OpenSSL that are installed on the monitoring host and remote host. If you’re running a commercial version of SSL on the remote host, there might be some compatibility problems.

3. The check_nrpe plugin returns “NRPE: Unable to read output”

Solution :

This error indicates that the command that was run by the NRPE daemon did not return any character output.  This could be an indication of the following problems:

1. An incorrectly defined command line in the command definition. Verify that the command definition in your NRPE configuration file is correct.

2. The plugin that is specified in the command line is malfunctioning. Run the command line manually to make sure the plugin returns some kind of text output.

3. There should be file permission issue. You need to grant read and execute privileges to the user which runs the nrpe daemon (this can be found in your nrpe config file).

For example : Your plugins are located under /usr/local/nagios/libexec/check_*

You can do this with

# chmod ug+rx /usr/local/nagios/libexec/check_*

# chown  nagios:nagios /usr/local/nagios

# chown –R nagios:nagios /usr/local/nagios/libexec

4. Check the /var/log/messages to find any errors related to host.allow/host.deny file. If there was any permission issue with this file will also result in above error

4. Unable to read output  due to Sudo Issues in CentOS when configuring an nrpe plugin with sudo:

[root@system ~]# /usr/lib/nagios/plugins/check_nrpe -H 3.3.3.3 -c check_dns

NRPE: Unable to read output

Given that check_dns is defined as follows, in nrpe.conf:

command[check_dns]=sudo /usr/local/nagios/libexec/check_dns

Solution :

You should also add its relative /etc/sudoers line as follows:

nagios ALL=(ALL) NOPASSWD:/usr/local/nagios/libexec/check_dns

Then the problem is in the requiretty options in /etc/sudoers, enabled by default on CentOS. Simply comment it as follows:

#Defaults requiretty

Now the plugin should work as expected:

[root@system ~]# /usr/lib/nagios/plugins/check_nrpe -H 3.3.3.3 -c check_dns

DNS Ok

5. NPRE Daemon not shown when checked with netstat –ta

Solution :

Add a line to your /etc/services file as follows (modify the port number as you see fit)

nrpe 5666/tcp # NRPE

6. ERROR: Could not fetch information from server

The most logical first step is to re-verify the Nagios server config file.  Check to make sure DNS resolution is correct.  Second, take a look at the NSC.log on the client system.  In my case, I saw:

2009-03-30 10:52:23: error:.\NSClientListener.cpp:307: Unauthorized access from: 192.168.1.25

Well, that could definitely be a problem.  The allowed_hosts line of:

Edit nsc.ini file and added the below lines

allowed_hosts=192.168.1.25/32

Sometime you should have added the server ip address in the allowed_hosts directive, but still the connection is not happening, Even if the local firewall is allowing you. This may be still the same due to some blockage at firewall or may be your nagios server is coming through a load balancer to your client network to access the client which inturn will result in hitting your client with the load balancer ip which is not allowed in allowed_hosts directive in nsc.ini. Please have a check on nsclient.log or nsc.log file to check what is the issue and added the IP. Once you verify it’s a trusted IP address. You should be all set J

About these ads

2 responses to “Troubleshooting Nagios/Nrpe Issues

  1. rugwiza November 28, 2012 at 12:17 pm

    ure expert. meanwhile i did almost all the troubleshoot u mentioned but nothing is going on.
    , the error
    CHECK_NRPE: Error – Could not complete SSL handshake.
    could not fixed after configuring the nrpe and xinet. I need yr help.

  2. gopinathachari February 16, 2013 at 1:51 pm

    Hi Rugwiza, Thanks much,, Did you check if anything on iptables is blocking the connections ?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: