next topic: --limit and --burst-limit

next topic: --limit and --burst-limit

[Robert P. J. Day]

  i've read three different explanations about these options,
and all three disagree with each other.  so let me start with
a simple question or two.

  first, as i read it, you can use the "--limit" option by itself,
without --burst-limit.  if you do that, then this limit is considered
a *hard* limit -- say 5/sec -- and once you hit that limit, you're
done until the next time unit.  is this correct?

  adding "--burst-limit", on the other hand, allows you to exceed
the first limit, but you end up paying for it down the road.
however, i'm still unclear on what the burst-limit represents
and how it gets recharged.  can someone explain this, preferably
with an example containing numbers?  

rday

Re: next topic: --limit and --burst-limit

[Cedric Blancher]

Le mer 30/10/2002 à 19:47, Robert P. J. Day a écrit :
>   first, as i read it, you can use the "--limit" option by itself,
> without --burst-limit.  if you do that, then this limit is considered
> a *hard* limit -- say 5/sec -- and once you hit that limit, you're
> done until the next time unit.  is this correct?

No.
If you do not specify burst, then it has a default value of 5. So

        --limit 5/s

is the same as

        --limit 5/s --bust-limit 5

From libipt_limit.c :
[...]
#define IPT_LIMIT_AVG   "3/hour"
#define IPT_LIMIT_BURST 5
[...]

These are default values, that are used if nothing's specified. To
achieve what you descriobe, you have to explicitly specify a burst of 0.
 
>   adding "--burst-limit", on the other hand, allows you to exceed
> the first limit, but you end up paying for it down the road.

Adding --burst-limit allows you to specify another value than the
default one.

> however, i'm still unclear on what the burst-limit represents
> and how it gets recharged.  can someone explain this, preferably
> with an example containing numbers?  

It is a bit difficult for me to give a more expressive exemple than the
one in Linux 2.4 Packet Filtering HOWTO at :

http://www.netfilter.org/documentation/HOWTO//packet-filtering-HOWTO-7.html

So I'll try to explain it a bit...

Limit is a bucket. This bucket is separated in two parts. The first one
is the burst. The second one is the limit. You do not apply the limit
until burst is full. This bucket is emptying a limit rate.

        |       |
        |       | <---- limit
        |-------|
        |       |
        |       | <---- burst
        |_______|

So, in the HOWTO exemple, (limit 1/s, burst 5, 4 packet/s flow), you're
filling the burst part of the bucket the first second. The second one,
the burst part is filled when receiving the 2nd packet. So, we begin to
fill the limit part of the bucket, and the limit of 1/s is beginning to
apply. I can accept the sixth packet, but not the following one,
arriving at the same second. When the second ends, I can flush the limit
part of the bucket, so I can accept more packets on top of the burst
part, within the limit.

Later, the flow stops. Each second I do not receive a packet, one packet
is taken from the bucket (because it's 1/s limit). For limit part is
empty, then we take it from the burst part. And then, if it goes again,
the burst part is getting filled again, and once filled, the limit
applies. And so on...

-- 
Cédric Blancher  <blancher@cartel-securite.fr>
IT systems and networks security expert  - Cartel Sécurité
Phone : +33 (0)1 44 06 97 87 - Fax: +33 (0)1 44 06 97 99
PGP KeyID:157E98EE  FingerPrint:FA62226DA9E72FA8AECAA240008B480E157E98EE

Re: next topic: --limit and --burst-limit

[Robert P. J. Day]

  ok, i think i've got a handle on the meaning of --limit and
--limit-burst, but there's still one thing that puzzles me.
some of the examples i've seen of this on the net seem to
be backwards.

  from "man iptables", we have that the "limit" module:

  "matches at a limited rate using a token bucket filter.  A rule
   using this extension will *match* [my emphasis] until this
   limit is reached (unless the '!' flag is used)."

this suggests that, if you add a rule incorporating a limit,
a match would imply that you *haven't* exceeded the limit and
that you don't necessarily want to take any countermeasures.

  from one tutorial, here's the lines that deal with 
syn-flooding protection:

  iptables -N syn-flood
  iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood
  iptables -A syn-flood -m limit --limit 1/s --limit-burst 4 -j RETURN
  iptables -A syn-flood -j DROP
  
this seems to match the man page -- in the "syn-flood" user-defined
chain, if you match the limit rule, you're still fine and you return.
otherwise, you drop the packet that forced you to exceed the limit.

  i'm harping on this since i've seen examples in which a matching
rule incorporating limit explicitly drops the packet, or LOGs it
with "Packet dropped, limit exceeded."  which seems backward.
(and no, they didn't have a '!' to negate the rule.)

  so, have i understood this correctly?  thanks for your patience.

rday

Re: next topic: --limit and --burst-limit

[Cedric Blancher]

Le jeu 31/10/2002 à 12:00, Robert P. J. Day a écrit :
> this suggests that, if you add a rule incorporating a limit,
> a match would imply that you *haven't* exceeded the limit and
> that you don't necessarily want to take any countermeasures.

Yes it is. You match packets that are _below_ the limit. When filtering,
you usually specifiy things that are acceptable.
 
>   from one tutorial, here's the lines that deal with 
> syn-flooding protection:
> 
>   iptables -N syn-flood
>   iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood
>   iptables -A syn-flood -m limit --limit 1/s --limit-burst 4 -j RETURN
>   iptables -A syn-flood -j DROP
>   
> this seems to match the man page -- in the "syn-flood" user-defined
> chain, if you match the limit rule, you're still fine and you return.
> otherwise, you drop the packet that forced you to exceed the limit.

That's true. If you don't return, then you're over the limit anbd get
logged.

>   so, have i understood this correctly?  thanks for your patience.

On these two points, you're right.

-- 
Cédric Blancher  <blancher@cartel-securite.fr>
Consultant en sécurité des systèmes et réseaux  - Cartel Sécurité
Tél: +33 (0)1 44 06 97 87 - Fax: +33 (0)1 44 06 97 99
PGP KeyID:157E98EE  FingerPrint:FA62226DA9E72FA8AECAA240008B480E157E98EE

Re: next topic: --limit and --burst-limit

[Robert P. J. Day]

  ok, here it is -- the actual limiting algorithm, as verified by
just setting some limits and watching where the packets go.

  i shut down all firewalling, then just added a rule to watch for
incoming pings, and LOG them with the following limits:

  limit: 6/min
  burst: 5

at that point, i just "# ping 127.0.0.1" and tailed the file
/var/log/messages.  what i saw, at the seconds counter:

1:	Y
2:	Y
3:	Y
4:	Y
5:	Y	(clearly, i've just run out of tokens)
10:	Y	(just got an additional token, so i buy another packet)
20:	Y	(same here)
30:	Y	(and so on)

... and so on.  so there you have it.  clearly, the "burst" represents
the initial number of tokens you get in your bucket to pay for incoming
packets.  and the limit frequency is used to replenish your bucket at
a nice uniform rate, as in "6/min" means every 10 seconds, and not a
handful of 6 every minute.  which makes perfect sense.

lord awmighty, why do all the docs make this sound so incomprehensible?
ya got a bucket, ya got some tokens ... sheesh.  :-)

rday

Re: next topic: --limit and --burst-limit

[Robert P. J. Day]

On 31 Oct 2002, Cedric Blancher wrote:

> So I'll try to explain it a bit...
> 
> Limit is a bucket. This bucket is separated in two parts. The first one
> is the burst. The second one is the limit. You do not apply the limit
> until burst is full. This bucket is emptying a limit rate.
> 
>         |       |
>         |       | <---- limit
>         |-------|
>         |       |
>         |       | <---- burst
>         |_______|
> 
> So, in the HOWTO exemple, (limit 1/s, burst 5, 4 packet/s flow), you're
> filling the burst part of the bucket the first second. The second one,
> the burst part is filled when receiving the 2nd packet. So, we begin to
> fill the limit part of the bucket, and the limit of 1/s is beginning to
> apply. I can accept the sixth packet, but not the following one,
> arriving at the same second. When the second ends, I can flush the limit
> part of the bucket, so I can accept more packets on top of the burst
> part, within the limit.
> 
> Later, the flow stops. Each second I do not receive a packet, one packet
> is taken from the bucket (because it's 1/s limit). For limit part is
> empty, then we take it from the burst part. And then, if it goes again,
> the burst part is getting filled again, and once filled, the limit
> applies. And so on...

ok, i'm almost certainly going to embarrass myself here, but after looking
at the graphical explanation of limiting and bursts, this seems so simple,
i'm at a loss to understand why so many explanations make it seem like
rocket science (witness the phrase "hysteresis door" -- sheesh).
so here's what it looks like (in contrast to cedric's explanation,
i prefer the analogy of *starting* with a bucket of tokens and 
paying them out as packets come in).

example:
	limit: 3/hour
	burst: 5

what does this mean?  it means that i start with a bucket of 5 tokens
and, every time a matching packet comes in, i can accept it by paying
a token.  simple as that.    when i run out of tokens, i can't accept
any more packets until i get more tokens.

and with a limit of 3/hour, every 20 minutes, i get another token dropped
into my bucket, up to a maximum of my burst value of 5, so that i can 
accept more packets.  that's it.  is it really that simple?

this explanation seems to match what i read in the packet filtering HOWTO
at netfilter.org.  it means that, with a full bucket, i can take a burst
of (in this example) 5 incoming packets, but now have to wait until the
next 20-minute point for another token so i can accept just one more
additional packet.  and so on.

have i oversimplified things here?

rday

Re: next topic: --limit and --burst-limit

[Cedric Blancher]

Le jeu 31/10/2002 à 13:15, Robert P. J. Day a écrit :
> example:
> 	limit: 3/hour
> 	burst: 5
> 
> what does this mean?  it means that i start with a bucket of 5 tokens
> and, every time a matching packet comes in, i can accept it by paying
> a token.  simple as that.    when i run out of tokens, i can't accept
> any more packets until i get more tokens.

That's it. It another interesting point of vue ;)
 
> and with a limit of 3/hour, every 20 minutes, i get another token dropped
> into my bucket, up to a maximum of my burst value of 5, so that i can 
> accept more packets.  that's it.  is it really that simple?

You begin with 8 tokens and that's you maximum number of tokens. 5 for
the burst (lower part of my bucket), and 3 that are time dependant
(upper part of the bucket).
Each time you receive a packet, you pay it with a token, so you can pay
for 8 packets within the first _hour_. Then, at the end of this first
hour, you will get three more tokens up to 8 tokens.

I can be wrong, but that's what I understood from the HOWTO, as for 1/s
limit with burst of 5, you first accept 6 packets.

Moreover, limit does not imply average value. You say 3 packets in an
hour. If you loose all your toeksn within the first second, then you
won't be able to match anything till the next hour. That's why limit
match is not really applicable for trafic shaping as an example.

Hope I've been more clear.

-- 
Cédric Blancher  <blancher@cartel-securite.fr>
IT systems and networks security expert  - Cartel Sécurité
Phone : +33 (0)1 44 06 97 87 - Fax: +33 (0)1 44 06 97 99
PGP KeyID:157E98EE  FingerPrint:FA62226DA9E72FA8AECAA240008B480E157E98EE

Re: next topic: --limit and --burst-limit

[Robert P. J. Day]

On 31 Oct 2002, Cedric Blancher wrote:

> Le jeu 31/10/2002 à 13:15, Robert P. J. Day a écrit :
> > example:
> > 	limit: 3/hour
> > 	burst: 5
> > 
> > what does this mean?  it means that i start with a bucket of 5 tokens
> > and, every time a matching packet comes in, i can accept it by paying
> > a token.  simple as that.    when i run out of tokens, i can't accept
> > any more packets until i get more tokens.
> 
> That's it. It another interesting point of vue ;)
>  
> > and with a limit of 3/hour, every 20 minutes, i get another token dropped
> > into my bucket, up to a maximum of my burst value of 5, so that i can 
> > accept more packets.  that's it.  is it really that simple?
> 
> You begin with 8 tokens and that's you maximum number of tokens. 5 for
> the burst (lower part of my bucket), and 3 that are time dependant
> (upper part of the bucket).
> Each time you receive a packet, you pay it with a token, so you can pay
> for 8 packets within the first _hour_. Then, at the end of this first
> hour, you will get three more tokens up to 8 tokens.
> 
> I can be wrong, but that's what I understood from the HOWTO, as for 1/s
> limit with burst of 5, you first accept 6 packets.

ok, you and i still have a fundamental disagreement, but it can be
cleared up pretty quickly.  (sadly, the graphical example in the
HOWTO at www.netfilter.org doesn't use numbers that would distinguish
between your view and mine.  bummer.)

let's go with my analogy of an initial bucket of tokens, with which
you have to pay a token to accept a packet.

Example:
	limit: 3/hour
	burst: 5

so, here are the simple questions that will settle this once and
for all:

1)  how many packets can i accept in an initial burst if they just
    come flooding in?  i assumed 5, you contend 8.  (IOW, i assumed
    that i have just 5 tokens to start with, you seem to think that
    it's 5 + 3 = 8.  so which is it?

2) regardless of the traffic, how frequently and by how much does my
   token bucket get refilled?  it could be adding 3 every hour, or
   it could be more uniform and be adding an extra token every
   20 minutes.  (the latter would produce a more uniform packet
   acceptance behavior.  the average would, of course, turn out to
   be the same, but the distribution would look different.)

so answering these two questions would pretty much settle the issue,
right?

rday
 

Re: next topic: --limit and --burst-limit

[Cedric Blancher]

Le jeu 31/10/2002 à 15:27, Robert P. J. Day a écrit :
> 1)  how many packets can i accept in an initial burst if they just
>     come flooding in?  i assumed 5, you contend 8.  (IOW, i assumed
>     that i have just 5 tokens to start with, you seem to think that
>     it's 5 + 3 = 8.  so which is it?

I assumed this, but I've just finished doing some testings and I must
admit you're right. Burst represent your initial amount of tokens.
So 5.

> 2) regardless of the traffic, how frequently and by how much does my
>    token bucket get refilled?  it could be adding 3 every hour, or
>    it could be more uniform and be adding an extra token every
>    20 minutes.  (the latter would produce a more uniform packet
>    acceptance behavior.  the average would, of course, turn out to
>    be the same, but the distribution would look different.)

Well, I am no sure of anything about this match anymore, so I will let
some developper answer ;)))

-- 
Cédric Blancher  <blancher@cartel-securite.fr>
Consultant en sécurité des systèmes et réseaux  - Cartel Sécurité
Tél: +33 (0)1 44 06 97 87 - Fax: +33 (0)1 44 06 97 99
PGP KeyID:157E98EE  FingerPrint:FA62226DA9E72FA8AECAA240008B480E157E98EE

RE: next topic: --limit and --burst-limit

[mailinglists]

> >   from one tutorial, here's the lines that deal with 
> > syn-flooding protection:
> > 
> >   iptables -N syn-flood
> >   iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood
> >   iptables -A syn-flood -m limit --limit 1/s --limit-burst 
> 4 -j RETURN
> >   iptables -A syn-flood -j DROP

I've just been playing with that, too.

Well, I couldn't stop scans like
nmap -sS -P0 -T Insane ip.address -p 21,22,25,80

However, 'nmap -sS -P0 -T Polite ip.address' works fine with this --limit
specification above.

How am I going to stop something like that?



Thanks,
philipp