[LARTC] Weird quirk with ingress policing

[LARTC] Weird quirk with ingress policing

[David McNab]

Hi,

I notice that if two or more existing connections match an ingress 
policing filter, the input bandwidth does not get evenly divided up 
between the n connections.

Kinda like litters of baby animals, where the stronger babies get more 
access to the mothers teats and grow up bigger and faster than their 
siblings.

The only workaround that's working for me is to set explicit ingress 
policing filters, which match src and dest host and port, so for each 
filter there can only be exactly one connection which matches. Then, 
it's just a matter of evenly dividing up the allocated total input bw 
amongst these n processes. This works, but it's not exactly optimal.

-- 

Kind regards
David

--

leave this line intact so your email gets through my junk mail filter
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Weird quirk with ingress policing

[Roy]

That is normal, policers do not share bandwich, they simply drop packets on
match randomly.
Probably you are already using index feature.

I checked policer source code, and cant understand anything at all,
probabaly everyhing is in other modules.
From that little what I could understand, seems that index means sharing
some parts of data.
probably rate counters, this is quite logical, but than I dont undestand its
behavior,
each policer should see sum of rates, but match on its own rate.

In practice it seems to ignore its own rate and match on sum of rates all at
once
so that one which packet exceed sum of trates will drop its packet.
Maybe you could check if it is corerct?
Since it is hard to believe that policer is intended to work in such way so.

I dont think policers are capable to divide trafic in fair way, they work as
simple one fifo.
(all policers as single fifo at once)


----- Original Message ----- 
From: "David McNab" <david@rebirthing.co.nz>
To: <lartc@mailman.ds9a.nl>
Sent: Sunday, March 14, 2004 1:11 PM
Subject: [LARTC] Weird quirk with ingress policing


> Hi,
>
> I notice that if two or more existing connections match an ingress
> policing filter, the input bandwidth does not get evenly divided up
> between the n connections.
>
> Kinda like litters of baby animals, where the stronger babies get more
> access to the mothers teats and grow up bigger and faster than their
> siblings.
>
> The only workaround that's working for me is to set explicit ingress
> policing filters, which match src and dest host and port, so for each
> filter there can only be exactly one connection which matches. Then,
> it's just a matter of evenly dividing up the allocated total input bw
> amongst these n processes. This works, but it's not exactly optimal.
>
> -- 
>
> Kind regards
> David
>
> --
>
> leave this line intact so your email gets through my junk mail filter
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Weird quirk with ingress policing

[David McNab]

Thanks for that Roy.

I kind of gathered that there's no (sane) way to share bandwidth fairly 
amongst the connections matching an ingress policing filter, and your 
investigations have helped to confirm this.

So what's working well for me, in the case of pyshaper, is to determine 
the number of connections which match a pyshaper class, divide this 
number into the total downstream bw allocated for that class, and spit 
out tc commands on a per-connection basis, matching src/dest ip/port, so 
that for every ingress policing filter, there is exactly one existing 
connection which matches.

So if I've allocated 12kbit for total ingress for a pyshaper class, and 
there are 4 matching connections, pyshaper executes tc commands to set 
up 4 ingress policers, each with 3kbit rate. If one of the connections 
drops out, pyshaper clears these policers, and generates 3 ingress 
policers each with 4kbit rate.

Cheers
David


Roy wrote:
> That is normal, policers do not share bandwich, they simply drop packets on
> match randomly.
> Probably you are already using index feature.
> 
> I checked policer source code, and cant understand anything at all,
> probabaly everyhing is in other modules.
>>From that little what I could understand, seems that index means sharing
> some parts of data.
> probably rate counters, this is quite logical, but than I dont undestand its
> behavior,
> each policer should see sum of rates, but match on its own rate.
> 
> In practice it seems to ignore its own rate and match on sum of rates all at
> once
> so that one which packet exceed sum of trates will drop its packet.
> Maybe you could check if it is corerct?
> Since it is hard to believe that policer is intended to work in such way so.
> 
> I dont think policers are capable to divide trafic in fair way, they work as
> simple one fifo.
> (all policers as single fifo at once)
> 
> 
> ----- Original Message ----- 
> From: "David McNab" <david@rebirthing.co.nz>
> To: <lartc@mailman.ds9a.nl>
> Sent: Sunday, March 14, 2004 1:11 PM
> Subject: [LARTC] Weird quirk with ingress policing
> 
> 
> 
>>Hi,
>>
>>I notice that if two or more existing connections match an ingress
>>policing filter, the input bandwidth does not get evenly divided up
>>between the n connections.
>>
>>Kinda like litters of baby animals, where the stronger babies get more
>>access to the mothers teats and grow up bigger and faster than their
>>siblings.
>>
>>The only workaround that's working for me is to set explicit ingress
>>policing filters, which match src and dest host and port, so for each
>>filter there can only be exactly one connection which matches. Then,
>>it's just a matter of evenly dividing up the allocated total input bw
>>amongst these n processes. This works, but it's not exactly optimal.
>>
>>-- 
>>
>>Kind regards
>>David
>>
>>--
>>
>>leave this line intact so your email gets through my junk mail filter
>>_______________________________________________
>>LARTC mailing list / LARTC@mailman.ds9a.nl
>>http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>>
> 
> 
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> 

-- 

Kind regards
David

--

leave this line intact so your email gets through my junk mail filter
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Weird quirk with ingress policing

[Andy Furniss]

David McNab wrote:
> Hi,
> 
> I notice that if two or more existing connections match an ingress 
> policing filter, the input bandwidth does not get evenly divided up 
> between the n connections.
> 
> Kinda like litters of baby animals, where the stronger babies get more 
> access to the mothers teats and grow up bigger and faster than their 
> siblings.

That's tcp not being fair.

> 
> The only workaround that's working for me is to set explicit ingress 
> policing filters, which match src and dest host and port, so for each 
> filter there can only be exactly one connection which matches. Then, 
> it's just a matter of evenly dividing up the allocated total input bw 
> amongst these n processes. This works, but it's not exactly optimal.
> 

Ingress shaping is alot harder than egress. It is possible to use 
htb/sfq for ingress by either shaping outbound on your LAN interface (if 
you have only one) or by using IMQ on your WAN interface. I found this 
nicer than using an ingress policer, but it's not perfect.

In reality if you want to properly shape ingress from behind the 
bottleneck you need something to keep state on all the TCP connections, 
have the ability to change the advertised window to control them and to 
be aware of what unstoppable packets are due in the future from the WAN 
so that new TCP connections can be started without causing your queues 
to move into your ISPs buffer. Even if you did that you are still up 
against bursty delivery of packets and occilations of TCP caused for 
example, by ack compression.

Andy.




_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/