If you rely on ssl/tls certificates and you have a slew of services to maintain online, things can quickly get out of hand. If you don’t have the time or the resources to keep up to speed with what ciphers to disable or what techniques to employ serverside, you might quickly fall prey to the next “Exploit with a Logo”. Heartbleed, Beast, Poodle and friends come to mind.

The guys at mozilla have taken measures to give all of us sysadmins more free time by maintaining lists of recommended steps to take serverside.

In addition they’ve also created Cipherscan, which is a great little tool to check if your sites comply with the latest security advisories.

Verify new certificates and ciphersuites on a test instance

One of my favorite features is that I can leverage SNI to check a new certificate on an arbitrary IP. This takes all the tension out of rolling out updated certificates every year or two, because I can verify that the new certificate has a sane certificate chain and does what it’s supposed to before I expose it to production traffic.

$ ./cipherscan -servername web.acme.customer.com 10.10.10.10:443 ....................................... Target: 10.10.10.10:443 prio ciphersuite protocols pfs curves 1 ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 ECDH,P-256,256bits prime256v1 2 ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 ECDH,P-256,256bits prime256v1 3 DHE-RSA-AES128-GCM-SHA256 TLSv1.2 DH,2048bits None 4 DHE-RSA-AES256-GCM-SHA384 TLSv1.2 DH,2048bits None 5 ECDHE-RSA-AES128-SHA256 TLSv1.2 ECDH,P-256,256bits prime256v1 6 ECDHE-RSA-AES256-SHA384 TLSv1.2 ECDH,P-256,256bits prime256v1 7 ECDHE-RSA-AES128-SHA TLSv1,TLSv1.1,TLSv1.2 ECDH,P-256,256bits prime256v1 8 ECDHE-RSA-AES256-SHA TLSv1,TLSv1.1,TLSv1.2 ECDH,P-256,256bits prime256v1 9 DHE-RSA-AES128-SHA256 TLSv1.2 DH,2048bits None 10 DHE-RSA-AES128-SHA TLSv1,TLSv1.1,TLSv1.2 DH,2048bits None 11 DHE-RSA-AES256-SHA256 TLSv1.2 DH,2048bits None 12 DHE-RSA-AES256-SHA TLSv1,TLSv1.1,TLSv1.2 DH,2048bits None 13 ECDHE-RSA-DES-CBC3-SHA TLSv1,TLSv1.1,TLSv1.2 ECDH,P-256,256bits prime256v1 14 EDH-RSA-DES-CBC3-SHA TLSv1,TLSv1.1,TLSv1.2 DH,2048bits None 15 AES128-GCM-SHA256 TLSv1.2 None None 16 AES256-GCM-SHA384 TLSv1.2 None None 17 AES128-SHA256 TLSv1.2 None None 18 AES256-SHA256 TLSv1.2 None None 19 AES128-SHA TLSv1,TLSv1.1,TLSv1.2 None None 20 AES256-SHA TLSv1,TLSv1.1,TLSv1.2 None None 21 DES-CBC3-SHA TLSv1,TLSv1.1,TLSv1.2 None None Certificate: trusted , 2048 bits, sha256WithRSAEncryption signature TLS ticket lifetime hint: 300 NPN protocols: None OCSP stapling: supported Cipher ordering: server Curves ordering: server - fallback: no Server supports secure renegotiation Server supported compression methods: NONE TLS Tolerance: yes

This checks the certificate and ciphers for web.acme.customer.com on a test instance at 10.10.10.10 (of course the test server needs to have the same web server setup as production).

Testing and fixing a bad setup

We have internal-service.example.com serving highly important pictures of lolcats to all employees, and we want to check if it’s up to scratch:

$ ./cipherscan internal-service.example.com:443 ....................................... Target: internal-service.example.com:443 prio ciphersuite protocols pfs curves 1 ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 ECDH,P-256,256bits prime256v1 2 ECDHE-RSA-AES256-SHA384 TLSv1.2 ECDH,P-256,256bits prime256v1 3 ECDHE-RSA-AES256-SHA TLSv1,TLSv1.1,TLSv1.2 ECDH,P-256,256bits prime256v1 4 DHE-RSA-AES256-GCM-SHA384 TLSv1.2 DH,2048bits None 5 DHE-RSA-AES256-SHA256 TLSv1.2 DH,2048bits None 6 DHE-RSA-AES256-SHA TLSv1,TLSv1.1,TLSv1.2 DH,2048bits None 7 DHE-RSA-CAMELLIA256-SHA TLSv1,TLSv1.1,TLSv1.2 DH,2048bits None 8 AES256-GCM-SHA384 TLSv1.2 None None 9 AES256-SHA256 TLSv1.2 None None 10 AES256-SHA TLSv1,TLSv1.1,TLSv1.2 None None 11 CAMELLIA256-SHA TLSv1,TLSv1.1,TLSv1.2 None None 12 ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 ECDH,P-256,256bits prime256v1 13 ECDHE-RSA-AES128-SHA256 TLSv1.2 ECDH,P-256,256bits prime256v1 14 ECDHE-RSA-AES128-SHA TLSv1,TLSv1.1,TLSv1.2 ECDH,P-256,256bits prime256v1 15 DHE-RSA-AES128-GCM-SHA256 TLSv1.2 DH,2048bits None 16 DHE-RSA-AES128-SHA256 TLSv1.2 DH,2048bits None 17 DHE-RSA-AES128-SHA TLSv1,TLSv1.1,TLSv1.2 DH,2048bits None 18 DHE-RSA-SEED-SHA TLSv1,TLSv1.1,TLSv1.2 DH,2048bits None 19 DHE-RSA-CAMELLIA128-SHA TLSv1,TLSv1.1,TLSv1.2 DH,2048bits None 20 AES128-GCM-SHA256 TLSv1.2 None None 21 AES128-SHA256 TLSv1.2 None None 22 AES128-SHA TLSv1,TLSv1.1,TLSv1.2 None None 23 SEED-SHA TLSv1,TLSv1.1,TLSv1.2 None None 24 CAMELLIA128-SHA TLSv1,TLSv1.1,TLSv1.2 None None 25 ECDHE-RSA-RC4-SHA TLSv1,TLSv1.1,TLSv1.2 ECDH,P-256,256bits prime256v1 26 RC4-SHA TLSv1,TLSv1.1,TLSv1.2 None None 27 ECDHE-RSA-DES-CBC3-SHA TLSv1,TLSv1.1,TLSv1.2 ECDH,P-256,256bits prime256v1 28 EDH-RSA-DES-CBC3-SHA TLSv1,TLSv1.1,TLSv1.2 DH,2048bits None 29 DES-CBC3-SHA TLSv1,TLSv1.1,TLSv1.2 None None Certificate: trusted , 2048 bits, sha256WithRSAEncryption signature TLS ticket lifetime hint: 300 NPN protocols: None OCSP stapling: not supported Cipher ordering: client Curves ordering: server - fallback: no Server supports secure renegotiation Server supported compression methods: NONE TLS Tolerance: yes

It’s clear that we are not doing everything right here, but what to do about it? Enter analyze.py , a small utility included with cipherscan that can tell you which knobs to turn. It lets you check your setup against the three defined levels of compliance that mozilla lays out: Modern, Intermediate and Old. For our specific case, intermediate will do the trick. To use it, first you need to output the results from cipherscan as JSON to a file, and run analyze on it:

$ ./cipherscan -j internal-service.example.com:443 > foo $ ./analyze.py foo -l intermediate internal-service.example.com:443 has bad ssl/tls and DOES NOT comply with the 'intermediate' level Things that are bad: * remove cipher ECDHE-RSA-RC4-SHA * remove cipher RC4-SHA Changes needed to match the intermediate level: * remove cipher DHE-RSA-CAMELLIA256-SHA * remove cipher CAMELLIA256-SHA * remove cipher DHE-RSA-SEED-SHA * remove cipher DHE-RSA-CAMELLIA128-SHA * remove cipher SEED-SHA * remove cipher CAMELLIA128-SHA * remove cipher ECDHE-RSA-RC4-SHA * remove cipher RC4-SHA * consider enabling OCSP Stapling * enforce server side ordering

Well then! We are using blacklisted ciphers, and a few other weak ciphers. We are not using OCSP stapling, and the client gets to dictate which ciphers we prefer. Let’s get rid of those blacklisted ciphers. We are using nginx for this, so set it up with the ciphers found here

Also, we should use server side ordering and OCSP stapling, resulting in this config (generated using this nifty generator):

# intermediate configuration. tweak to your needs. ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS'; ssl_prefer_server_ciphers on; # OCSP Stapling ssl_stapling on; ssl_stapling_verify on;

After reloading (i would restart) the nginx, lets see how we perform:

$ ./cipherscan -servername web.acme.customer.com 10.10.10.10:443 ....................................... Target: 10.10.10.10:443 prio ciphersuite protocols pfs curves 1 ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 ECDH,P-256,256bits prime256v1 2 ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 ECDH,P-256,256bits prime256v1 3 DHE-RSA-AES128-GCM-SHA256 TLSv1.2 DH,2048bits None 4 DHE-RSA-AES256-GCM-SHA384 TLSv1.2 DH,2048bits None 5 ECDHE-RSA-AES128-SHA256 TLSv1.2 ECDH,P-256,256bits prime256v1 6 ECDHE-RSA-AES256-SHA384 TLSv1.2 ECDH,P-256,256bits prime256v1 7 ECDHE-RSA-AES128-SHA TLSv1,TLSv1.1,TLSv1.2 ECDH,P-256,256bits prime256v1 8 ECDHE-RSA-AES256-SHA TLSv1,TLSv1.1,TLSv1.2 ECDH,P-256,256bits prime256v1 9 DHE-RSA-AES128-SHA256 TLSv1.2 DH,2048bits None 10 DHE-RSA-AES128-SHA TLSv1,TLSv1.1,TLSv1.2 DH,2048bits None 11 DHE-RSA-AES256-SHA256 TLSv1.2 DH,2048bits None 12 DHE-RSA-AES256-SHA TLSv1,TLSv1.1,TLSv1.2 DH,2048bits None 13 ECDHE-RSA-DES-CBC3-SHA TLSv1,TLSv1.1,TLSv1.2 ECDH,P-256,256bits prime256v1 14 EDH-RSA-DES-CBC3-SHA TLSv1,TLSv1.1,TLSv1.2 DH,2048bits None 15 AES128-GCM-SHA256 TLSv1.2 None None 16 AES256-GCM-SHA384 TLSv1.2 None None 17 AES128-SHA256 TLSv1.2 None None 18 AES256-SHA256 TLSv1.2 None None 19 AES128-SHA TLSv1,TLSv1.1,TLSv1.2 None None 20 AES256-SHA TLSv1,TLSv1.1,TLSv1.2 None None 21 DES-CBC3-SHA TLSv1,TLSv1.1,TLSv1.2 None None Certificate: trusted , 2048 bits, sha256WithRSAEncryption signature TLS ticket lifetime hint: 300 NPN protocols: None OCSP stapling: supported Cipher ordering: server Curves ordering: server - fallback: no Server supports secure renegotiation Server supported compression methods: NONE TLS Tolerance: yes

Much better! Let’s see what analyze says:

$ ./cipherscan -j internal-service.example.com:443 > foo && ./analyze.py foo -l intermediate internal-service.example.com:443 has intermediate ssl/tls and complies with the 'intermediate' level

Compliant!

Using nagios to verify your certificates

I like to let someone else do my work for me and nagios (icinga in this case) is pretty good for that. So I want to set up tests of all my SSL/TLS endpoints everywhere. For nagios, things are pretty easy. analyze.py even has a –nagios mode that produces the right exit codes. Here’s the script I use:

#!/bin/bash set -euo pipefail _fail (){ echo $@ exit 2 } LEVEL = $1 SERVERNAME = $2 HOST = $3 PORT = $4 tmpfile = $( mktemp ) || _fail "error creating temp file" LOCK = /tmp/ $( basename $0 ) - ${ SERVERNAME } _ ${ HOST } .lock trap "rm -f $LOCK $tmpfile " INT HUP TERM EXIT test -f $LOCK && _fail "lockfile found: $LOCK " /opt/cipherscan/cipherscan -j -servername ${ SERVERNAME } ${ HOST } : ${ PORT } > $tmpfile || \ _fail unable to scan ${ SERVERNAME } at ${ HOST } : ${ PORT } /opt/cipherscan/analyze.py $tmpfile -l ${ LEVEL } --nagios exit $?

I call it through nrpe like this:

command[check_cipherscan_someservice_example_com]=/usr/local/lib/monitoring/plugins/nagios_cipherscan intermediate some-service.example.com 10.10.10.10 443

That’s it!