Rate Limiting After a Threshold

Rate Limiting After a Threshold

[John Jung]

Hi,

   I'm new to IP Tables in general, but I've been able to whack away at the 
rules to get connlimit to do what I want.  Now I'm trying to do something 
more sophisticated, but it doesn't seem to work.

   My ultimate goal is to allow most Web users to access my site, but to slow 
down the abusers.  So, for example, I want to let in the first 10 HTTP 
connections in, and then after that, limit that IP to only 20 connections per 
minute afterwards.  (And then after a certain point, connlimit will block any 
additional connections by that IP.)

   I'm using a vanilla 2.6.21.3 Linux kernel, but I can't figure out how to 
do it.

   I think hashlimit is the key, but it really just doesn't want to work for 
me.  For example, I've tried:

     iptables -A INPUT -p tcp --dport 23 -m hashlimit --hashlimit 1/hour
       --hashlimit-mode srcip --hashlimit-burst 1 --hashlimit-name test
       -j REJECT

   but I can open up more than 1 telnet session in under a minute, let alone 
an hour.

   I've read and re-read the hashlimit man page, tried various arguments that 
I've found on on the Web, all to now avail.

   Any and all suggestions are welcomed.

   Thanks.

						John

-- 
+-------------------------------------+-------------------------------------+
| John Jung (john.j.jung@siemens.com) |    Siemens Automation and Drives    |
|           Support Engineer          |           UGS PLM Software          |
|       Customer Support - GTAC       |     10824 Hope Street, MS: 1177     |
|       Operating Systems Group       |      Cypress, California 90630      |
+--------------------------- +1 (800) 955-0000 -----------------------------+

Re: Rate Limiting After a Threshold

[Ray Leach]

[-- Attachment #1: Type: text/plain, Size: 1666 bytes --]



John Jung wrote:
> Hi,
>
>   I'm new to IP Tables in general, but I've been able to whack away at 
> the rules to get connlimit to do what I want.  Now I'm trying to do 
> something more sophisticated, but it doesn't seem to work.
>
>   My ultimate goal is to allow most Web users to access my site, but 
> to slow down the abusers.  So, for example, I want to let in the first 
> 10 HTTP connections in, and then after that, limit that IP to only 20 
> connections per minute afterwards.  (And then after a certain point, 
> connlimit will block any additional connections by that IP.)
>
>   I'm using a vanilla 2.6.21.3 Linux kernel, but I can't figure out 
> how to do it.
>
>   I think hashlimit is the key, but it really just doesn't want to 
> work for me.  For example, I've tried:
>
>     iptables -A INPUT -p tcp --dport 23 -m hashlimit --hashlimit 1/hour
>       --hashlimit-mode srcip --hashlimit-burst 1 --hashlimit-name test
>       -j REJECT
>
>   but I can open up more than 1 telnet session in under a minute, let 
> alone an hour.
>
>   I've read and re-read the hashlimit man page, tried various 
> arguments that I've found on on the Web, all to now avail.
>
>   Any and all suggestions are welcomed.
If you're using iptables, what OS are you using? Why are you using the 
telnet port (23)? instead of the SSH port (22)?

-- 
<img src='http://www.danasoft.com/sig/spoonssig.jpg' />
--------------------------------------------------
RCHQ Hobbies cc
http://www.rchq.co.za and http://store.rchq.co.za
Fax: +27 86 652 2773       eMail: admin@rchq.co.za
P O Box 10376, Vorna Valley, Midrand, 1686
--------------------------------------------------

Re: Rate Limiting After a Threshold

[Michael Hissler]

John Jung wrote:

[...]

>   I think hashlimit is the key, but it really just doesn't want to work
> for me.  For example, I've tried:
> 
>     iptables -A INPUT -p tcp --dport 23 -m hashlimit --hashlimit 1/hour
>       --hashlimit-mode srcip --hashlimit-burst 1 --hashlimit-name test
>       -j REJECT


The hashlimit match works the other way round. Try '-j ACCEPT' and
append a rule to drop/reject connections to this port.
You should also use the state match, as you want to filter connections,
not packets.

So try this:

iptables -A INPUT -m state --state ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp --dport 23 -m state --state NEW -m hashlimit
--hashlimit 1/hour --hashlimit-mode srcip --hashlimit-burst 1
--hashlimit-name test -j ACCEPT
iptables -A INPUT -p tcp --dport 23 -m state --state NEW -j REJECT

(If you enter the rules in this order, you can omit the '-m state
--state NEW' in the last rule, but OTOH it doesn't hurt.)


michael

Re: Rate Limiting After a Threshold

[John Jung]

Hi Michael,

Michael Hissler wrote:
[...]
> The hashlimit match works the other way round. Try '-j ACCEPT' and
> append a rule to drop/reject connections to this port.
> You should also use the state match, as you want to filter connections,
> not packets.
> 
> So try this:
> 
> iptables -A INPUT -m state --state ESTABLISHED -j ACCEPT
> iptables -A INPUT -p tcp --dport 23 -m state --state NEW -m hashlimit
> --hashlimit 1/hour --hashlimit-mode srcip --hashlimit-burst 1
> --hashlimit-name test -j ACCEPT
> iptables -A INPUT -p tcp --dport 23 -m state --state NEW -j REJECT

   This still doesn't quite do what I want it to do (I'm able to open up more 
than 1 telnet session per IP per hour), but it's close enough for what I need.

   I want to rate limit people using download acceleration programs on my 
Website.  So I'm willing to let the first x connections in completely, then 
slow them down at a rate of y connections per minute up to z total 
connections.  Where x, y and z will be determined at a later date.  :)

   In the past, I've had people open up 8 connections in a second, drop them 
almost immediately, then repeat.  Do this for an hour, or have multiple 
people do this, and the load on my server starts going way up.  (My Web site 
requires a lengthy authentication process that's fairly resource intensive. 
So it's usually a badly configured download acceleration program that cause 
problems.)

   As to answer some other people's questions to me: I'm moving to vanilla 
Linux 2.6.21.3 and I'm only using telnet as a test port against my eventual 
port 80 goal.  If I can rate limit telnet connections then I can rate limit 
http connections.

   If anybody can improve on the above rules, please let me know.  If not, 
like I said, it does enough for what I want.  It's not perfect, but it'll 
work for me.

						John

-- 
+-------------------------------------+-------------------------------------+
| John Jung (john.j.jung@siemens.com) |    Siemens Automation and Drives    |
|           Support Engineer          |           UGS PLM Software          |
|       Customer Support - GTAC       |     10824 Hope Street, MS: 1177     |
|       Operating Systems Group       |      Cypress, California 90630      |
+--------------------------- +1 (800) 955-0000 -----------------------------+

Re: Rate Limiting After a Threshold

[Michael Hissler]

John Jung wrote:
> Hi Michael,
> 
> Michael Hissler wrote:

>> iptables -A INPUT -m state --state ESTABLISHED -j ACCEPT
>> iptables -A INPUT -p tcp --dport 23 -m state --state NEW -m hashlimit
>> --hashlimit 1/hour --hashlimit-mode srcip --hashlimit-burst 1
>> --hashlimit-name test -j ACCEPT
>> iptables -A INPUT -p tcp --dport 23 -m state --state NEW -j REJECT
> 
>   This still doesn't quite do what I want it to do (I'm able to open up
> more than 1 telnet session per IP per hour), but it's close enough for
> what I need.

Sorry, my fault! I forgot to add '--hashlimit-htable-expire 3600000'.
Per default, hashtable entries expire after 10 seconds.
See /proc/net/ipt_hashlimit/test, the first column shows the remaining
time in seconds.

michael