See part 1 of this blog series for more information if you don’t already have it set up.

v3.x

I forwarded the useful patches from the old NRPE v2.x branch to the new NRPE 3.x branch that were applicable still.

I patched the SystemD startup script to work with CentOS/Red Hat systems.

A sudoer’s file is already to go for those who want to use sudo with their NRPE remote calls.

There is firewall configuration all ready to use with FirewallD.

Best of all, with my RPMs, you can run SELinux in full Enforcing mode for that extra piece of mind from a security standpoint!

The Goods

For those who really don’t care and want to just jump right in with the product. Here you go!

You can download the packages manually if you choose, or reference them using my repository:

Package Download Description nrpe el7.rpm The NRPE Server: This should be installed on any server you want to monitor. The server will allow the machine to respond to requests sent to it via the check_nrpe (Nagios) plugin. nrpe-selinux el7.rpm An SELinux add-on package that allows the NRDP Server to operate under Enforcing Mode.



Note: This RPM is not required by the NRDP server to run correctly. nagios-plugins-nrpe el7.rpm Our NRPE Client; this is the Nagios Plugin used to request information from the NRPE Server(s). You’ll only need to install this on the Nagios server (for the purpose of this blog). The RPM provides a small tool called check_nrpe that gets installed into the Nagios Plugin Directory (/usr/lib64/nagios/plugins). Through some simple configuration; Nagios can use the check_nrpe tool to monitoring anything you want just as long as the NRPE server is (installed and) running at the other end.

Note: The source rpm can be obtained here which builds everything you see in the table above. It’s not required for the application to run, but might be useful for developers or those who want to inspect how I put the package together.

NRPE Server Side Configuration

The NRPE Server is usually installed onto all of the Application servers you want Nagios to monitor remotely. It provides a means of accepting requests to process (such as, what is your system load like?) and handling the request and returning the response back.

Assuming you hooked up to my repository here, the NRDP server can can be easily installed with the following command:

# Install NRPE (Server) on your Application Server yum install nrpe nrpe-selinux

NRPE communicates through TCP port 5666; which means you may need to enable the ports in your firewall first to allow remote connections from Nagios. The below commands open up the protocol to everyone attached to your network. You should only perform this command if your Application Server will be running on a local private network:

# Enable NRPE port (5666) firewall-cmd --permanent --add-service=nrpe firewall-cmd --add-service=nrpe

If you’re opening this up to the internet, then you might want to just open the (NRPE) port exclusively for the Nagios Server. The following presumes you know the IP of the Nagios Server and will open access to ‘JUST’ that system:

# Assuming 5.6.7.8 belongs to the Nagios Server you intend to # allow to monitor you; you might do the following: firewall-cmd --permanent --zone=public \ --add-rich-rule='\ rule family="ipv4" \ source address="5.6.7.8/32" \ port protocol="tcp" \ port="5666" \ accept'

You’ll want to have a look at your NRPE Configuration as you will probably have to update a few lines. See /etc/nagios/nrpe.cfg and have a look for the following directives:

Directive Details allowed_hosts This is added security for NRPE but can come back and haunt you if you ignore it (as nothing will work). You should specify the IP address of your Nagios server here and not allow anyone else! For example, if your Nagios server was 6.7.8.9, then you would put 6.7.8.9/32 here. dont_blame_nrpe This allows you to pass options into your remote checks. I personally think this is awesome, but there is no question that depending on the checks that accept arguments, it could could exploit content from your system you wouldn’t have otherwise wanted to share. This is specially the case if you grant NRPE Sudoers permission (discussed a bit lower). Set this value to one (1) to enable argument passing and zero (0) to disable it. allow_bash_command_substitution Bash substitution is something like $(hostname) (which might return something like node01.myserver). Set this value to one (1) to enable argument passing and zero (0) to disable it. I don’t use personally use this and therefore have it disabled on my system.

Setting up NRPE Tokens

Once your server is set up, there is one last thing you need to do. You need to associate tokens with some of the status checks you want to do. You see, NRPE won’t just execute any program you tell it to, it will only execute programs a specific way that you’ve allowed for. This is purely for security and it makes sense to do so! It’s really not that complicated, consider a token/command mapping like this (as an example):

Token Command check_load # Check the system load:

# – warning if greater than 10.0

# – critical if grater then 20.0

# – we map the check_load token to this command:

/usr/lib64/nagios/plugins/check_load -w 10 -c 20

You pass this information into NRPE by creating a .conf file in /etc/nrpe.d (assuming you’re using my RPMs) with the syntax:

command[ token ]= /path/to/mapped/command

So with respect to the example we started; it would look like this:

command[ check_load ]= /usr/lib64/nagios/plugins/check_load -w 10 -c 20

You can specify as many tokens as you like (each one has to be unique from the other). If you have a look in /etc/nrpd.d/, you’ll see one called common_checks.cfg which has a handful of useful commands to start you off with. You can add to this file or start another if you like (in the same directory with a .conf extension). Each time you make changes to the configuration (or another) you’ll need to signal NRPE to have it load any new changes you provided:

# Restart our NRPE Service systemctl restart nrpe.service

Here is a conceptual diagram that will help illustrate what was just explained here using the check_load example above:

Sudoer’s Permission

Substitute User Do (or sudo) allows you to run commands as other users. Most commonly people use sudo to elevate there permission to the root (superuser) privilege to execute a command. When the command has completed, they return back to their normal privileges.

Some actions require you to have higher system authority to get anything good from them. This includes retrieve certain kinds of system/status information. For example, you can’t check how much mail is spooled and ready for remote delivery (on a Mail Server) unless you reduce the privileges of all of your stored mail (making it accessible by anyone); not a very good idea! But alternatively you can use sudo to grant one user permission to just run a command that can only fetch the number of mail items queued; this is a much better approach! Thus elevating a users permissions for just "special status checking only " commands isn’t so much of a risk. NRPE can be configured to have superuser permissions for it’s status checks which can allow you to monitor a lot more things! Here is how to do it:

# First make sure that the sudoer's configuration doesn't # require tty (teletype terminals) endpoints to use. You # do so by commenting out the line that reads 'Defaults requiretty' # in the /etc/sudoers file (access it with the command visudo) # You can also do this with the below one-liner too sed -i -e 's/^\(Defaults[ \t]\+requiretty.*\)$/#\1/g' \ /etc/sudoers # By doing this, you allow pseudo-teletype (pty) endpoints # to use the sudo command too. NRDP would be a pty service # for example since it's not an actual person running the # command. # Update our configuration to use the sudo command sed -i -e "s|^[ \t#]*\(command_prefix\)=.*|\1=/usr/bin/sudo|g" \ /etc/nagios/nrpe.cfg # Restart our server if it's running systemctl restart nrpe.service

SELinux users will want to also do this:

# Allow NRPE/Nagios calls to run /usr/bin/sudo setsebool -P nagios_run_sudo on

NRPE Client Side Configuration

The client side simply consists of a small application (called check_nrpe) which connects remotely to any NRPE Server you tell it to and hands it a token for processing (provided your NRPE Server is configured correctly). The NRPE Server will take this token and execute a command that was associated with it and return the results back to you.

# You'll want to be connected to my repositories for this to work: # See: https://nuxref.com/repo for more information # Install NRPE on the same server running Nagios yum install nagios-plugins-nrpe

There is configuration already in place for you if you’re using my RPMS located in /etc/nagios/conf.d called nrpe.cfg. But you can feel free to create your own (or over-write it like so:

cat << _EOF > /etc/nagios/conf.d/nrpe.cfg ; simple wrapper to check_nrpe for remote calls to NRPE servers define command{ command_name check_nrpe command_line /usr/lib64/nagios/plugins/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ } ; check_nrpe with arguments enabled (enable the dont_blame_nrpe for this ; to work properly otherwise simply don't use it) define command{ command_name check_nrpe_args command_line /usr/lib64/nagios/plugins/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ -a $ARG2$ }

You can now send command tokens to servers running NRPE via Nagios. A Nagios configuration might look like this:

cat << _EOF > /etc/nagios/conf.d/my.application.server01.cfg ; first we define a host that we want to monitor. ; If you already have a host configured; you can skip this part define host{ ; Name of host template to use. This host definition will inherit ; all variables that are defined in (or inherited by) the ; linux-server host template definition. You can find this ; in /etc/nagios/objects/templates.cfg if you're interested use linux-server ; Now we define the server we're going to monitor host_name my-application-server01.nuxref.com alias my-application-server01.nuxref.com address 192.168.1.2 statusmap_image redhat.png icon_image redhat.png icon_image_alt CentOS 7.x } ; Now we want to define our monitoring service that talks to our ; Application server with NRPE installed on it: define service{ ; Name of service template to use. This service definition will inherit ; all variables that are defined in (or inherited by) the ; local-service service template definition. You can find this ; in /etc/nagios/objects/templates.cfg if you're interested use local-service host_name my-application-server01.nuxref.com service_description Our System Load ; The Exclamation mark (!) lets nagios know that check_load is the argument ; we want to pass into our check_nrpe command we defined earlier (as $ARG1$) check_command check_nrpe!check_load } _EOF

If you want to make sure it’s going to work, before you go any further you can run the same command you just told Nagios to do (above):

# The syntax 'check_nrpe!check_load' gets translated to: # /usr/lib64/nagios/plugins/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ # # Which we can further translate (for testing purposes) to: /usr/lib64/nagios/plugins/check_nrpe -H 192.168.1.2 -c check_load

If everything worked okay you should see something similar to the output:

OK – load average: 0.00, 0.00, 0.00|load1=0.000;10.000;20.000;0; load5=0.000;10.000;20.000;0; load15=0.000;10.000;20.000;0

You’ll want to reload Nagios to pick up on your new configuration so it can start calling this command too:

# But before you reload it; it doesn't hurt to just check and make # sure your configuration is okay; You can test it out with: nagios -v /etc/nagios/nagios.cfg # correct any errors that display and rerun the above command until # everything checks out okay! # Now reload Nagios so it reads in it's new configuration: systemctl reload nagios.service

NRPE vs. NRDP

Which one should you use? Both NRDP and NRPE have their Pro’s and Cons.

Benefits of using… NRPE NRDP You have central control over the checking periods of an application. You only ever need to open 1 TCP port; from a security standpoint; this is awesome! You don’t need Nagios to use this. If used properly, it can provide a great way to access remote servers for information and even execute administrative and maintenance commands. You only need to manage the Nagios configuration when a new server is added. Can send multiple status messages in one single (TCP) transaction. Can reflect a remote application status change immediately oppose to NRPE which one reflect the change until the next status check is performed.

It should be worth noting that nothing is stopping you from using both NRDP and NRPE at the same time. You might choose an NRDP strategy for remote systems while choosing an NRPE strategy for all your systems residing in your private network. NRPE works over the internet as well, but just exercise caution and be sure to have SELinux running in Enforcing mode on all of the server end points.

Credit

This blog took me a very (,very) long time to put together and test! The repository hosting alone accommodates all my blog entries up to this date. All of the custom packaging described here was done by me personally. I took the open source available to me and rebuilt it to make it an easier solution and decided to share it. If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

Sources