If you've ever bothered looking at your system logs, you may have noticed repeated (automated) attempts from hosts attempting to gain SSH access to your box, using seemingly random (predictable/common) usernames, and perhaps some default passwords.
Dec 20 06:24:10 router sshd[18755]: Failed password for invalid user patrick from 202.129.204.199 port 53485 ssh2 Dec 20 06:24:15 router sshd[18767]: Failed password for invalid user patrick from 202.129.204.199 port 53683 ssh2 Dec 20 06:24:45 router sshd[18795]: Failed password for invalid user rolo from 202.129.204.199 port 55169 ssh2 Dec 20 06:24:50 router sshd[18797]: Failed password for invalid user iceuser from 202.129.204.199 port 55357 ssh2 Dec 21 05:37:23 router sshd[13305]: Failed password for invalid user user from 203.172.22.84 port 56622 ssh2 Dec 21 05:37:28 router sshd[13307]: Failed password for invalid user admin from 203.172.22.84 port 56746 ssh2 Dec 21 05:37:43 router sshd[13313]: Failed password for invalid user soni from 203.172.22.84 port 57111 ssh2 Dec 21 05:37:55 router sshd[13319]: Failed password for invalid user soni from 203.172.22.84 port 57362 ssh2 Dec 21 05:38:15 router sshd[13347]: Failed password for invalid user test from 203.172.22.84 port 57875 ssh2 Dec 21 05:38:21 router sshd[13349]: Failed password for invalid user admin from 203.172.22.84 port 57993 ssh2 Dec 21 05:38:27 router sshd[13351]: Failed password for invalid user admin from 203.172.22.84 port 58133 ssh2 Dec 21 05:38:44 router sshd[13359]: Failed password for invalid user test from 203.172.22.84 port 58534 ssh2 Dec 21 05:38:49 router sshd[13361]: Failed password for invalid user testuser from 203.172.22.84 port 58670 ssh2 Dec 21 05:38:55 router sshd[13364]: Failed password for invalid user test123 from 203.172.22.84 port 58792 ssh2 Dec 21 05:39:01 router sshd[13366]: Failed password for invalid user toor from 203.172.22.84 port 58927 ssh2 Dec 21 05:39:21 router sshd[13392]: Failed password for invalid user user from 203.172.22.84 port 59440 ssh2 Dec 21 05:39:26 router sshd[13394]: Failed password for invalid user admin from 203.172.22.84 port 59556 ssh2 Dec 21 05:39:32 router sshd[13396]: Failed password for invalid user test from 203.172.22.84 port 59665 ssh2
In the past I have always ignored these since using strong passwords, limiting users with shell access, and not exposing usernames to the outside world should provide a adequate protection. As the numbers of users on my systems has increased, and I as an admin have chosen convenient (sometimes not very obscure usernames), and left it up to users to determine their passwords, I have become more concerned about this type of attack.
Common Approaches
The obvious way to block this kind of attack is to block access to the server from the host machine, via its IP address.
As far as I can tell, their are two main approaches to blocking this kind of attack:
The iptables solution is pretty cool, its kernel level, so a very low overhead, however it cannot differentiate between genuine (successfull) connections and illicit attempts. Depending on your threshold, you can quite easily lock yourself out of your own server (by making a number of sequential SSH connections), or allowing more attempts than you are comfortable with to prevent the previous issue.
Monitoring syslog is a more accurate solution, from the perspective of detecting whether a connection was legitimate or not, however processing log files can be expensive on high load systems, and will either require additional daemons to be run, or be run out of cron, which may mean that the attack is finished by the time the IP is blocked.
An alternative approach - rsyslog
Newer distributions come with a new syslog daemon, rsyslog, which offers some very powerful features we can use:
- On-the-fly message filtering
- Shell-script execution
We can use these features to perform similar function to the retrospective syslog monitoring, however it can be done in real-time, and without such a hefty overhead of additional daemons, and parsing the whole log file on each invocation.
Configuration of rsyslog is a bit tricky, but since its configuration format supports conventional BSD syslog notation, and a couple of other rsyslog extension formats that have been developed over the years.
First thing to note is I already have a firewall blacklist feature on my gateway, so I already have a script to call to perform the blacklist. It just requires an IP address.
For the SSH blocking, I am going to wrap this script up in another shell script, that is able to handle incorrect passwords for valid users in a slightly more gracious way...it gives you 2 chances to get your password right, before the IP is blocked. Attempts to login as an invalid user cause instant blacklist of the IP.
My wrapper script is '/usr/local/bin/sshLoginFail'
It takes two parameters - the source IP of the attempted connection, and the username for the login attempt (if its a valid username for the system)
First thing to do is define a couple of rsyslog templates which can extract the required information for the script parameters out of the log message.
The syslog messages are formatted as follows, and we need to extract the information highlighted:
Failed password for root from 203.172.22.84 port 51920 ssh2 Invalid user oracle from 203.172.22.84
The %msg% token contains the message being processed. The first template is a regex that will extract the IP address from the 'Invalid user' message. The second uses field extraction to extract the username and the IP address.
#Invalid user - We only care about the IP address, - use a regex because there could be spaces in the username
#which would confuse us
$template SSHInvalidUser,"%msg:R,ERE,1,DFLT:from ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})--end%"
#Valid user, wrong password, extract both, to pass to our script
$template SSHValidUserFailedPassword,"%msg:F,32:7% %msg:F,32:5%"
These template are used to create the command-line parameters with which to call our script, we now need to determine when were are processing a message that needs to trigger our script.
There are two mechanisms available in rsyslog, Property-based filters and Expression-based filters. Both will do the job, but with the Expression-based filters, you can explicitly select messages that have come from sshd.
The property-based filters are commented out, but left in for completeness
#Hit on messages that come out of SSHD - ignore the Failed password for invalid user message, - Invalid user is enough #:msg,contains,"Invalid user " ^/usr/local/bin/sshLoginFail;SSHInvalidUser #:msg,contains,"Failed password for invalid user " ~ #:msg,contains,"Failed password for " ^/usr/local/bin/sshLoginFail;SSHValidUserFailedPassword #Same as above but using Expression-based filters, which should be more efficient and accurate, since they will match based on facility #and application if $syslogfacility-text == 'auth' and $app-name == 'sshd' and $msg contains 'Invalid user ' then ^/usr/local/bin/sshLoginFail;SSHInvalidUser if $syslogfacility-text == 'auth' and $app-name == 'sshd' and $msg contains 'Failed password for ' and not ($msg contains 'Failed password for invalid user ') then ^/usr/local/bin/sshLoginFail;SSHValidUserFailedPassword
Categories: Linux
