Detecting a SSH Brute Force Attack

If you are under a SSH brute force attack, you will likely see something like this in your logs.

Jan 26 03:46:02 host sshd[22731]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=125.39.22.154 Jan 26 03:46:02 host sshd[22731]: pam_succeed_if(sshd:auth): error retrieving information about user seymour Jan 26 03:46:02 host sshd[22734]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=61.147.103.185 user=root Jan 26 03:46:02 host sshd[22722]: Failed password for root from 61.147.103.185 port 16563 ssh2 Jan 26 03:46:02 host sshd[22723]: Received disconnect from 61.147.103.185: 11: Normal Shutdown, Thank you for playing Jan 26 03:46:03 host sshd[22705]: Received disconnect from 61.147.103.185: 11: Normal Shutdown, Thank you for playing Jan 26 03:46:03 host sshd[22726]: Failed password for invalid user madonna from 125.39.22.154 port 51706 ssh2 Jan 26 03:46:03 host sshd[22917]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=61.147.103.185 user=root Jan 26 03:46:03 host sshd[22727]: Failed password for root from 61.147.103.185 port 16723 ssh2 Jan 26 03:46:03 host sshd[22729]: Failed password for invalid user florin from 125.39.22.154 port 54958 ssh2

This is a bot scanning your server trying to guess passwords.

Methods to Stop SSH Brute Force Attacks

There are basically four approaches to dealing with SSH brute force attacks:

Restrict SSH access by IP address

Change SSH to another Port

Use intrusion prevention tools to dynamically block access

Rates limit SSH sessions using IPTables

All of these approaches have theirs benefits and drawbacks.

While restricting SSH access by IP address is the most secure method, such restrictions are often not possible when dealing with web hosting services as you have multiple users with constantly changing IP addresses.

Changing the SSH port may defeat bot scans but does little against targeted attacks. Also, this usually just frustrates your users.

Intrusion prevention tools like fail2ban and denyhosts have their place but they are subject to log based attacks. These tools essential analyze logs using regular expressions. Hackers have found ways around both of these tools in the past.

Lastly, you have a great tool to block ssh brute force attacks right on your server: IPtables.

Using IPtables to Stop SSH Brute Force Attacks

I like to think of this approach similar to flow rates with pipes. Bigger pipes allow more water to flow. Smaller pipes can handle less water.

To block a SSH brute force attack, we just need to slow down the flow of requests. We can do this by rate-limiting requests to SSH with iptables.

Essentially, we create a smaller pipe for new SSH sessions. This slows brute force attacks to a point where they become ineffective.

The iptables rules are relatively simple.

/usr/sbin/iptables -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent --set /usr/sbin/iptables -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent --update --seconds 60 --hitcount 4 -j DROP

This rule will block an IP if it attempts more than 3 connections per minute to SSH. Notice that the state is set to NEW. This means only new connections not established ones are impacted. Established connections are the result of a successful SSH authentication, so users who authenticate properly will not be blocked.

If you need to see what’s being done, you may want to log these drops. You can do so by setting up a log rule and then using these rules instead.

/sbin/iptables -N LOGDROP /sbin/iptables -A LOGDROP -j LOG /sbin/iptables -A LOGDROP -j DROP iptables -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent --set iptables -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent --update --seconds 60 --hitcount 4 -j LOGDROP

Notice that I’ve changed the rule from DROP to LOGDROP. This way your drops will get logged and you can see the results in your logs:

Jan 27 08:22:29 server kernel: IN=eth1 OUT= MAC=00:30:48:94:fb:21:00:1b:00:00:00:00:00:00 SRC=118.103.140.3 DST=208.43.148.64 LEN=60 TOS=0x00 PREC=0x00 TTL=51 ID=16406 DF PROTO=TCP SPT=31003 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0 Jan 27 08:22:29 server kernel: IN=eth1 OUT= MAC=00:30:48:94:fb:21:00:1b:00:00:00:00:00:00 SRC=118.103.140.3 DST=192.168.1.1 LEN=60 TOS=0x00 PREC=0x00 TTL=51 ID=5076 DF PROTO=TCP SPT=45354 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0 Jan 27 08:22:35 server kernel: IN=eth1 OUT= MAC=00:30:48:94:fb:21:00:1b:00:00:00:00:00:00 SRC=118.103.140.3 DST=208.43.148.66 LEN=60 TOS=0x00 PREC=0x00 TTL=51 ID=37295 DF PROTO=TCP SPT=21077 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0 Jan 27 08:22:35 server kernel: IN=eth1 OUT= MAC=00:30:48:94:fb:21:00:1b:00:00:00:00:00:00 SRC=118.103.140.3 DST=208.43.148.64 LEN=60 TOS=0x00 PREC=0x00 TTL=51 ID=16408 DF PROTO=TCP SPT=31003 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0

Effectively Stopping SSH Brute Force Attacks

I always try to get a sense of effectiveness of any tool or configuration we deploy. I find many “security” tools that are popular among the web hosting crowd provide little to no value. In many cases, appropriate configuration of your server or web application could achieve similar results without the hassle of maintaining a third party product.

Are the IPTables rules effective? In short yes.

During a recent attack on a server, the SSH service remained fully accessible with no service interruption.

Previously such aggressive attack would have caused service interruptions. So on the service side, this approach works. When I dug into the logs, I found three failed user attempts against SSH prior to the rate-limiting kicked in. The attack then sent 67 more attempts before it gave up.

Benefits of Using IPtables to Block SSH Attacks

The benefit of this approach is you don’t need any added software. IPtables is likely sitting on your server already, so you can easily and quickly deploy this solution.

Also, there are no “ban lists” to maintain. People forget passwords or incorrectly setup their SSH/SFTP programs. As a result, they trigger a block and get locked out. You then have to manually edit some ban list to remove them or whitelist IPs. Over time or with multiple servers, this is a time-consuming server management tasks. By using iptables, there’s no list to maintain — leaving you time to work on more important things.

One of the drawbacks is that this approach does not lock accounts. A slow, distributed attack could fall under the radar. If it was a directed attack against a specific user account, the attacker could churn away for days or weeks without detection. For that, you would need something that can lock user accounts after failures. PAM includes a module called pam_tally that does just this. If you fail too many times, an account is locked.

Hardening SSH

In addition to these IPTables settings, there are some things you can do within the SSH configuration to harden SSH from attacks. We commonly apply these settings as part of our server support services.