NEW "SSH Brute Force " ruleset (20050628.0)

NEW "SSH Brute Force " ruleset (20050628.0)

[Taylor, Grant]

Ok, it looks like there is some renewed interest in the "SSH Brute Force" chain as it was.  I have taken requests and recommendations of others on this mail list and have grown the "SSH Brute Force" chain to be a MUCH more complex rule set (it's now more than one chain so I can't call it a chain any more) that has MUCH more capability.  I have put a version number of 20050628.0 in the subject line so that we can keep track of this on the mail list as the last time I posted this it got to be quite a thread with many versions.

The idea behind this version is accomplish the following:
1)  Have a way to detect and block a particular SSH connection based on a duration of time.
2)  Enforce a "Quite Time" of a specified duration before a connection is allowed to be established or restart the counter on the "Quite Time".
3)  Only allow someone to ""trip a trigger for a particular time window a specified number of times or else they ""trip a trigger for the next larger time window.
4)  Provide the capability of "Black Listing" of IPs.  I have not completely implemented this in this version but it would be very easy to have a global Black List based on the "SSH_Blacklist" recent list.
5)  Have a known string to search for in the logs to look for IPs that need to be added to /etc/hosts.deny.  At present I have not written any thing to do this monitoring, I leave that up to you.  That is unless you ask for help.

So with out further a due:

#!/bin/bash
IPTablesPath=/usr/sbin
IPTablesBin=iptables
IPTablesCount=0
IPTablesErrorCount=0

function iptables () {
                IPTablesCommand[IPTablesCount]=`echo ${IPTablesBin} "${@}" | sed -e 's/  //g'`
                IPTablesError[IPTablesCount]=`${IPTablesPath}/${IPTablesBin} "${@}" 2>&1`
                if test $? -eq 0; then {
                        echo -n "."
                }; else {
                        echo -n "x"
                        IPTablesCommand[++IPTablesCount]=""
                }; fi
}



#
# iptables-chainprobe is a small function that I wrote to allow me to write my firewall scripts with out worrying if a chain exists or not, much like modprobe for kernel modules, hens the name.
#
function iptables-chainprobe () {
        if test "${1}" == "-t" -o "${1}" == "--table"; then {
		# Insert some logic to test to make sure that all required parameters were passed.
                if test "${2}" == "filter" -o "${2}" == "nat" -o "${2}" == "mangle" -o "${2}" == "raw"; then {
                        if test -n "${3}"; then {
                                if test -n "${4}"; then {
                                        case ${3} in
                                                "-N"|"--new-chain")
                                                        iptables-save -t ${2} | fgrep ":${4} -" > /dev/null
                                                        if test $? -ne 0; then {
                                                                iptables -t ${2} -N ${4}
                                                        }; else {
                                                                iptables -t ${2} -F ${4}
                                                        }; fi
                                                        ;;
                                                "-F"|"--flush")
                                                        iptables-save -t ${2} | fgrep ":${4} -" > /dev/null
                                                        if test $? -eq 0; then {
                                                                iptables -t ${2} -F ${4}
                                                        }; else {
                                                                iptables -t ${2} -N ${4}
                                                        }; fi
                                                        ;;
                                                # Insert more of the actions that can be taken on various chains,
                                                # -X / --delete-chain, 
                                                # -Z / --zero-chain, 
                                                # -P / --policy
                                                        # Only allow setting policies on non-userdefined chains.
                                                # -E / --rename-chain
                                                # Put these actions in alphabetical order.
                                        esac
                                }; else {
                                        echo -n "x"
                                        IPTablesCommand[IPTablesCount]=`echo iptables-chainprobe "${@}" | sed -e 's/  //g'`
                                        IPTablesError[IPTablesCount]="Error text for not specifying a chain goes here."
                                        IPTablesCommand[++IPTablesCount]=""
                                }; fi
                        }; else {
                                echo -n "x"
                                IPTablesCommand[IPTablesCount]=`echo iptables-chainprobe "${@}" | sed -e 's/  //g'`
                                IPTablesError[IPTablesCount]="Error text for not specifying an action for the chain goes here."
                                IPTablesCommand[++IPTablesCount]=""
                        }; fi
                }; else {
                        echo -n "x"
                        IPTablesCommand[IPTablesCount]=`echo iptables-chainprobe "${@}" | sed -e 's/  //g'`
                        IPTablesError[IPTablesCount]="Error text for not specifying a valid table goes here."
                        IPTablesCommand[++IPTablesCount]=""
                }; fi
        }; else {
                echo -n "x"
                IPTablesCommand[IPTablesCount]=`echo iptables-chainprobe "${@}" | sed -e 's/  //g'`
                IPTablesError[IPTablesCount]="Error text for not specifying \"-t\" or \"--table\" goes here."
                IPTablesCommand[++IPTablesCount]=""
        }; fi
}



function iptables-errors () {
        if test ${IPTablesCount} -gt 0; then {
                if test ${IPTablesCount} -eq 1; then {
                        echo "There was 1 error with your firewall script.  The error is listed below."
                }; else {
                        echo "There were ${IPTablesCount} errors with your firewall script.  The errors are listed below."
                }; fi
                for ((x=0;x<IPTablesCount;x++)); do {
                        echo -e "Command:\t${IPTablesCommand[$x]}"
                        echo ${IPTablesCommand[$x]} | egrep -e "(-m|--match) comment" > /dev/null
                        if test $? -eq 0; then {
                                echo -e "   Note:\tComment match extension comments do not show up with double quotes in the \"Error:\" line below.  (Bug)."
                        }; fi
                        echo -e "  Error:\t${IPTablesError[$x]}"
                }; done
        }; fi
}



echo -n "Setting up the firewall"



#
# Filter Table SSH_Brute_Force_2678400 Chain
#
iptables-chainprobe -t filter -F SSH_Brute_Force_2678400
iptables -t filter -A SSH_Brute_Force_2678400 -m recent --rcheck --seconds 604800 --hitcount 2 --name SSH_Brute_Force_86400 --rsource -j RETURN
iptables -t filter -A SSH_Brute_Force_2678400 -m recent ! --rcheck --seconds 2678400 --hitcount 2 --name SSH_Brute_Force_2678400 --rsource -j RETURN
iptables -t filter -A SSH_Brute_Force_2678400 -m recent --name SSH_Blacklist --set --rsource
iptables -t filter -A SSH_Brute_Force_2678400 -j LOG --log-prefix "SSH Blacklist:  "
iptables -t filter -A SSH_Brute_Force_2678400 -p tcp -j REJECT



#
# Filter Table SSH_Brute_Force_604800 Chain
#
iptables-chainprobe -t filter -F SSH_Brute_Force_604800
iptables -t filter -A SSH_Brute_Force_604800 -m recent --rcheck --seconds 86400 --hitcount 2 --name SSH_Brute_Force_3600 --rsource -j RETURN
iptables -t filter -A SSH_Brute_Force_604800 -m recent ! --rcheck --seconds 604800 --hitcount 2 --name SSH_Brute_Force_86400 --rsource -j RETURN
iptables -t filter -A SSH_Brute_Force_604800 -m recent --name SSH_Brute_Force_2678400 --set --rsource
iptables -t filter -A SSH_Brute_Force_604800 -j LOG --log-prefix "SSH Brute Force (604800):  "
iptables -t filter -A SSH_Brute_Force_604800 -p tcp -j REJECT



#
# Filter Table SSH_Brute_Force_86400 Chain
#
iptables-chainprobe -t filter -F SSH_Brute_Force_86400
iptables -t filter -A SSH_Brute_Force_86400 -m recent --rcheck --seconds 3600 --hitcount 5 --name SSH_Brute_Force_60 --rsource -j RETURN
iptables -t filter -A SSH_Brute_Force_86400 -m recent ! --rcheck --seconds 86400 --hitcount 2 --name SSH_Brute_Force_3600 --rsource -j RETURN
iptables -t filter -A SSH_Brute_Force_86400 -m recent --name SSH_Brute_Force_86400 --set --rsource
iptables -t filter -A SSH_Brute_Force_86400 -j LOG --log-prefix "SSH Brute Force (86400):  "
iptables -t filter -A SSH_Brute_Force_86400 -p tcp -j REJECT



#
# Filter Table SSH_Brute_Force_3600 Chain
#
iptables-chainprobe -t filter -F SSH_Brute_Force_3600
iptables -t filter -A SSH_Brute_Force_3600 -m recent --rcheck --seconds 60 --hitcount 3 --name SSH_Brute_Force --rsource -j RETURN
iptables -t filter -A SSH_Brute_Force_3600 -m recent ! --rcheck --seconds 3600 --hitcount 5 --name SSH_Brute_Force_60 --rsource -j RETURN
iptables -t filter -A SSH_Brute_Force_3600 -m recent --name SSH_Brute_Force_3600 --set --rsource
iptables -t filter -A SSH_Brute_Force_3600 -j LOG --log-prefix "SSH Brute Force (3600):  "
iptables -t filter -A SSH_Brute_Force_3600 -p tcp -j REJECT



#
# Filter Table SSH_Brute_Force_60 Chain
#
iptables-chainprobe -t filter -F SSH_Brute_Force_60
iptables -t filter -A SSH_Brute_Force_60 -m recent ! --rcheck --seconds 60 --hitcount 3 --name SSH_Brute_Force --rsource -j RETURN
iptables -t filter -A SSH_Brute_Force_60 -m recent --name SSH_Brute_Force_60 --set --rsource
iptables -t filter -A SSH_Brute_Force_60 -j LOG --log-prefix "SSH Brute Force (60):  "
iptables -t filter -A SSH_Brute_Force_60 -p tcp -j REJECT



#
# Filter Table SSH_Brute_Force_1 Chain
#
# This is not the cleanest way to handle multiple SSH connections per second, but it will work by artificially tripping SSH_Brute_Force_60.
iptables-chainprobe -t filter -F SSH_Brute_Force_1
iptables -t filter -A SSH_Brute_Force_1 -m recent ! --rcheck --seconds 1 --hitcount 2 --name SSH_Brute_Force --rsource -j RETURN
iptables -t filter -A SSH_Brute_Force_1 -m recent --name SSH_Brute_Force --set --rsource
iptables -t filter -A SSH_Brute_Force_1 -j LOG --log-prefix "SSH Brute Force (1):  "
iptables -t filter -A SSH_Brute_Force_1 -p tcp -j REJECT



#
# Filter Table SSH_Brute_Force_NEW Chain
#
iptables-chainprobe -t filter -F SSH_Brute_Force_NEW
iptables -t filter -A SSH_Brute_Force_NEW -m recent --name SSH_Brute_Force --set --rsource
iptables -t filter -A SSH_Brute_Force_NEW -j SSH_Brute_Force_2678400
iptables -t filter -A SSH_Brute_Force_NEW -j SSH_Brute_Force_604800
iptables -t filter -A SSH_Brute_Force_NEW -j SSH_Brute_Force_86400
iptables -t filter -A SSH_Brute_Force_NEW -j SSH_Brute_Force_3600
iptables -t filter -A SSH_Brute_Force_NEW -j SSH_Brute_Force_60
iptables -t filter -A SSH_Brute_Force_NEW -j SSH_Brute_Force_1



##
## Filter Table SSH_Brute_Force_ESTABLISHED Chain
##
#iptables-chainprobe -t filter -F SSH_Brute_Force_ESTABLISHED
#iptables -t filter -A SSH_Brute_Force_ESTABLISHED -m recent --name SSH_Brute_Force --remove --rsource
#iptables -t filter -A SSH_Brute_Force_ESTABLISHED -m recent --name SSH_Brute_Force_1 --remove --rsource
#iptables -t filter -A SSH_Brute_Force_ESTABLISHED -m recent --name SSH_Brute_Force_60 --remove --rsource
#iptables -t filter -A SSH_Brute_Force_ESTABLISHED -m recent --name SSH_Brute_Force_3600 --remove --rsource
#iptables -t filter -A SSH_Brute_Force_ESTABLISHED -m recent --name SSH_Brute_Force_86400 --remove --rsource
#iptables -t filter -A SSH_Brute_Force_ESTABLISHED -m recent --name SSH_Brute_Force_604800 --remove --rsource
#iptables -t filter -A SSH_Brute_Force_ESTABLISHED -m recent --name SSH_Brute_Force_2678400 --remove --rsource
#iptables -t filter -A SSH_Brute_Force_ESTABLISHED -m recent --name SSH_Blacklist --remove --rsource
#iptables -t filter -A SSH_Brute_Force_ESTABLISHED -m recent --name SSH_Known_Good --set --rsource



#
# Filter Table SSH_Brute_Force Chain
#
iptables-chainprobe -t filter -F SSH_Brute_Force
#iptables -t filter -A SSH_Brute_Force -m recent --name SSH_Known_Good --rcheck --rsource -j RETURN
iptables -t filter -A SSH_Brute_Force -s 208.47.119.113 -j RETURN
iptables -t filter -A SSH_Brute_Force -s 208.47.119.114 -j RETURN
iptables -t filter -A SSH_Brute_Force -s 208.47.119.115 -j RETURN
iptables -t filter -A SSH_Brute_Force -s 208.47.119.116 -j RETURN
iptables -t filter -A SSH_Brute_Force -s 208.47.119.117 -j RETURN
iptables -t filter -A SSH_Brute_Force -m state --state NEW -j SSH_Brute_Force_NEW
# I don't have support for connbytes in my kernel (yet) so I cant run this test.  :(
#iptables -t filter -A SSH_Brute_Force -m state --state ESTABLISHED -m connbytes --connbytes 10000 -j SSH_Brute_Force_ESTABLISHED



echo "done."
iptables-errors

The basic logic behind this script is as follows:
 1)  Jump to the "SSH_Brute_Force" chain from somewhere else in your firewall.
 2)  Jump to the "SSH_Brute_Force_NEW" chain if the connection is considered to be NEW by conntrack.
 3)  Jump to the "SSH_Brute_Force_ESTABLISHED" chain if the connection is considered to be ESTABLISHED by conntrack.
     This chain should clear any and all recent lists related to SSH_Brute_Force for this connection save for the SSH_Known_Good recent list, for which it should set it.
 4)  The "SSH_Brute_Force_NEW" chain will then jump to the various sub chains to determine if a connection is being blocked for any specific duration of time.
 5)  The SSH_Brute_Force_[0-9]* chains will test to see if a connection is being banned for the next lower amount of time and not process if this is the case and defer processing to the chain that is doing the banning.
 6)  If a SSH_Brute_Force_[0-9]* chain does see that a packet has been banned by the next shorter duration chain but is not presently banned check to see how many times it has been banned for the next shorter duration.  If the count of the number of times the next shorter ban is over a specified number ban the connection for a longer duration.
     Translation, if I ban a connection for 60 seconds more than two times an hour ban them for an hour.  If I ban them for an hour more than once a day ban them for a day, etc.

One point of interest would be the use of the "--rttl" option on the recent matches.  I have not tested such tests but plan to do so in the future.  Please reply to the mail list with your experiences.

As always any comments and / or suggestions are most welcome and appreciated.



Grant. . . .

Re: NEW "SSH Brute Force " ruleset (20050628.0)

[Marius Mertens]

Hi everybody,

On Tuesday, June 28, 2005 9:10 AM,
Taylor, Grant wrote:

> [...]
> One point of interest would be the use of the "--rttl" option on the
> recent matches.  I have not tested such tests but plan to do so in
> the future.  Please reply to the mail list with your experiences.
> As always any comments and / or suggestions are most welcome and
> appreciated.

Definitely a very nice piece of work! Though I personally do not use the 
longer banning times, since just banning them for a minute has proven long 
enough to make them move to the next host, I do like the "stacked chains". 
My use for a second level is to log only the first DROP, so the DROPS won't 
flood my logs.

Speaking of flooding: Does anybody know how long IPs are stored in the 
recent list, until they are removed again?

The requested "any comments / suggestions" section:
Maybe also put a timelimit on the whitelist, since IPs added there will be 
most likely dynamic ones, otherwise they would probably have been added to 
another permanent whitelist.
Possibly strange ideas: If a certain IP makes it to a certain level of 
blacklisting, drop everything --state NEW from them, not just ssh (dropping 
also established things might allow a too powerful and easy DOS)
Add a host to a special whiltelist after doing something special, like 
connecting to a certain port, which would lower the risk of a DOS (they can 
still try, but you can override it)
I haven't used the -rttl option either, but I did monitor the TTLs of 
dropped bruteforce attempts. Most different hosts also had different TTLs, 
but packets from a single attacker never had different ones, so adding that 
option should (at least at the moment) not result in significantly more 
successful connection attempts than without.

Marius 

Re: NEW "SSH Brute Force " ruleset (20050628.0)

[curby .]

On 7/1/05, Marius Mertens <marius.mertens@gmx.de> wrote:
> Add a host to a special whiltelist after doing something special, like
> connecting to a certain port, which would lower the risk of a DOS (they can
> still try, but you can override it)

I've been considering this too.  It's actually a simple form of port
knocking that can be implemented exclusively in iptables (without the
need of extra tools).  The primary goal of port knocking is to foil
port scans, but it could be applicable here.

To protect against DoS, is there any easy way of requiring that three
packets be transferred in an SSH connection before it triggers a
recent update?  Since someone spoofing source IPs to DoS would be
unlikely to continue the connection with the server, such DoS attacks
might be foiled more effectively this way than using rttl (which the
attacker can just exhaustively try all values for).

Re: NEW "SSH Brute Force " ruleset (20050628.0)

[Grant Taylor]

> To protect against DoS, is there any easy way of requiring that three
> packets be transferred in an SSH connection before it triggers a
> recent update?  Since someone spoofing source IPs to DoS would be
> unlikely to continue the connection with the server, such DoS attacks
> might be foiled more effectively this way than using rttl (which the
> attacker can just exhaustively try all values for).

(I've not responded to these posts b/c I don't have the needed resources in my kernel (and I'm not running modules) and I have not had the time to recompile for my Cobalt (non standard kernel & boot process).)

I think it would be vary easy to use the connbytes module to test for the number of packets or the amount of data that has been transfered in any given connection.  In fact this is how I was going to determine on the packet level that an SSH connection was good and assume that the user had successfully logged in.  If any given connection has transfered x number of bytes or y number of packets then I'll assume (on the network layer) that they have successfully logged in on the application layer and that they are a valid user.



Grant. . . .

Re: NEW "SSH Brute Force " ruleset (20050628.0)

[/dev/rob0]

I'm glad this came up again, because I have my own, simpler solution 
working. Mine only depends on the -m limit and -m state match 
extensions, both fairly standard and widely used.

My firewalls generally have a "State" chain called first from both 
FORWARD and INPUT, with these rules:
-A State -m state --state INVALID -j DROP
-A State -m state --state RELATED,ESTABLISHED -j ACCEPT

"--state RELATED" lets in the ssh attack bots, so I've diverted SSH 
packets in the State chain:
-A State -m state --state INVALID -j DROP
-A State -p tcp --dport 22 -j Ssh
-A State -m state --state RELATED,ESTABLISHED -j ACCEPT

The new "Ssh" chain does this:
-A Ssh -m state --state ESTABLISHED -j ACCEPT
# [ a rule to allow from internal hosts here, optional ]
-A Ssh -m limit --limit 3/m -j ACCEPT
-A Ssh -m limit --limit 1/m j LOG --log-prefix "SSH attack: "
-A Ssh -j REJECT
# or DROP if you prefer

It does seem to work to repel the attacks. IME of watching a couple 
attacks hit, they quit and move on after the rejections start. Script 
kiddie phishers don't deal well with rejection. :)

However, this might not be practical for a public shell server with a 
lot of users. (I guess. I don't know exactly how the --limit is 
applied. Is that 3/m from the whole Internet, or 3/m from any given 
host?)

Another curious thing about the --limit is that when I stress test it:
for X in `seq 99` ; do
    echo $X | nc target.hostname ssh & echo -n $X
done
with 3/m I get 5 connections through, and usually about 5 logged with 
the 1/m limit. That test just now got 8 connections through (not all 
made it in the first minute.)

I don't think the limit will cause me any problems; even if I had to 
retype a password I get 3 chances before sshd cuts me off, and those 
retries are in "--state ESTABLISHED".
-- 
    mail to this address is discarded unless "/dev/rob0"
    or "not-spam" is in Subject: header

Re: NEW "SSH Brute Force " ruleset (20050628.0)

[/dev/rob0]

On Sunday 2005-September-04 00:01, I wrote:
> The new "Ssh" chain does this:
> -A Ssh -m state --state ESTABLISHED -j ACCEPT
> -A Ssh -m limit --limit 3/m -j ACCEPT
> -A Ssh -m limit --limit 1/m j LOG --log-prefix "SSH attack: "
[ note, a typo here ----------^ "-j" ]
> -A Ssh -j REJECT
[snip]
> Another curious thing about the --limit is that when I stress test
> it: for X in `seq 99` ; do
>     echo $X | nc target.hostname ssh & echo -n $X
> done
> with 3/m I get 5 connections through, and usually about 5 logged with
> the 1/m limit. That test just now got 8 connections through (not all
> made it in the first minute.)

RTFM:
--limit-burst number
     Maximum  initial  number  of  packets to match: this number gets
     recharged by one every time the limit  specified  above  is  not
     reached, up to this number; the default is 5.
So I've changed the above:
-A Ssh -m state --state ESTABLISHED -j ACCEPT
-A Ssh -m limit --limit 3/m --limit-burst 3 -j ACCEPT
-A Ssh -m limit --limit 1/m --limit-burst 1 -j LOG --log-prefix "SSH attack: "
-A Ssh -j REJECT

I still got 8 packets of the 99 through, but only 2 were logged, one
minute apart.
-- 
    mail to this address is discarded unless "/dev/rob0"
    or "not-spam" is in Subject: header

Re: NEW "SSH Brute Force " ruleset (20050628.0)

[/dev/rob0]

On Sunday 2005-September-04 14:59, I wrote:
> -A Ssh -m state --state ESTABLISHED -j ACCEPT
> -A Ssh -m limit --limit 3/m --limit-burst 3 -j ACCEPT
> -A Ssh -m limit --limit 1/m --limit-burst 1 -j LOG --log-prefix "SSH attack: "
> -A Ssh -j REJECT 

Originally when I did this I used DROP. DROP sent the bots away. But
results with REJECT today indicate a sort of tarpitting effect: one bot
took 11 minutes from start to finish, logging 33 attempts to
authenticate and 11 --log-prefix "SSH attack: " entries.

So it depends what your goal is. If you don't feel safe enough with
your sshd, use DROP. If you want to interfere with their operation,
REJECT.

I think that REJECT gives them an opportunity to DoS, to a small
extent. During that 11-minute attack I probably would not have been
able to SSH in. If that's so, DROP should effectively limit the DoS to
one minute per attack.
-- 
    mail to this address is discarded unless "/dev/rob0"
    or "not-spam" is in Subject: header