[LARTC] tc limit

[LARTC] tc limit

[Martin Schiøtz]

Hi 

I have been playing with tc. I'm using htb and sfq. I tried to setup
shaping of every ip address  within a /20.
I found that there must be an upper limit of about 2096 on the number
of rules (classes, qdiscs or filters). I was testing a /20 which gives
4096 classes, qdiscs and filters. I'm using hashkey to optimize
processing of frames.

When executing the script below - it stops when adding rules for "ip
10.0.7.253" with this error:

handle 1:4097
handle 1:4099
handle 1:4101
RTNETLINK answers: File exists
We have an error talking to the kernel
handle 1:4103
RTNETLINK answers: File exists
We have an error talking to the kernel
handle 1:4105
...
...
etc.

Is there a limit with tc or I'm I doing something wrong ?
(I'm using iproute2-2.6.9-041019)

This is the script:
#!/bin/sh

/sbin/tc qdisc add dev eth1 root handle 1: htb
/sbin/tc class add dev eth1 parent 1: classid 1:3 htb rate 34mbit ceil
34mbit burst 0kbit cburst 0kbit
/sbin/tc filter add dev eth1 parent 1:3 protocol ip u32
/sbin/tc filter add dev eth1 parent 1:3 handle 3: protocol ip u32 divisor 256
/sbin/tc filter add dev eth1 protocol ip parent 1: u32 match ip dst
10.0.0.0/20 hashkey mask 0x000000ff at 16 link 3:

echo handle 1:13
/sbin/tc class add dev eth1 parent 1:3 classid 1:13 htb rate 512kbit
ceil 512kbit burst 0k cburst 0k
/sbin/tc qdisc add dev eth1 parent 1:13 handle 13: sfq perturb 4
/sbin/tc filter add dev eth1 protocol ip parent 1:3 u32 ht 3:1: match
ip dst 10.0.0.1 flowid 1:13

echo handle 1:15
/sbin/tc class add dev eth1 parent 1:3 classid 1:15 htb rate 512kbit
ceil 512kbit burst 0k cburst 0k
/sbin/tc qdisc add dev eth1 parent 1:15 handle 15: sfq perturb 4
/sbin/tc filter add dev eth1 protocol ip parent 1:3 u32 ht 3:2: match
ip dst 10.0.0.2 flowid 1:15
...
...
echo handle 1:4101
/sbin/tc class add dev eth1 parent 1:3 classid 1:4101 htb rate 512kbit
ceil 512kbit burst 0k cburst 0k
/sbin/tc qdisc add dev eth1 parent 1:4101 handle 4101: sfq perturb 4
/sbin/tc filter add dev eth1 protocol ip parent 1:3 u32 ht 3:fd: match
ip dst 10.0.7.253 flowid 1:4101
...
...
echo handle 1:8201
/sbin/tc class add dev eth1 parent 1:3 classid 1:8201 htb rate 512kbit
ceil 512kbit burst 0k cburst 0k
/sbin/tc qdisc add dev eth1 parent 1:8201 handle 8201: sfq perturb 4
/sbin/tc filter add dev eth1 protocol ip parent 1:3 u32 ht 3:ff: match
ip dst 10.0.15.255 flowid 1:8201
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] tc limit

[gypsy]

Martin Schiøtz wrote:
> 
> Hi

Hello.
 
> I have been playing with tc. I'm using htb and sfq. I tried to setup
> shaping of every ip address  within a /20.
> I found that there must be an upper limit of about 2096 on the number
> of rules (classes, qdiscs or filters). I was testing a /20 which gives
> 4096 classes, qdiscs and filters. I'm using hashkey to optimize
> processing of frames.

Filters.
 
> Is there a limit with tc or I'm I doing something wrong ?

Yes, sort of.  Yes.

> This is the script:
> #!/bin/sh
> 

> /sbin/tc filter add dev eth1 parent 1:3 protocol ip u32

'prio #' must be included in all 'filter' lines where "#" is not zero
and is always the same; the "zero prio" has a limit and there is a limit
on '#' - but no limit on how many times '#' can be (e.g.) 5

So, e.g.:
tc filter add dev eth1 parent 1: protocol ip prio 5 \
   u32 match ip src $IP flowid 1:40

will run at least 64K times without complaint, but this will fail after
~2K:
tc filter add dev eth1 parent 1: protocol ip prio 0 \
   u32 match ip src $IP flowid 1:40

> /sbin/tc filter add dev eth1 parent 1:3 handle 3: protocol ip u32 divisor 256
> /sbin/tc filter add dev eth1 protocol ip parent 1: u32 match ip dst
> 10.0.0.0/20 hashkey mask 0x000000ff at 16 link 3:

google LARTC please document

gypsy
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] tc limit

[Martin Schiøtz]

Thanks, with "prio #" on all filters - all rules can be added. 
:-)

Just curious - Is there a reason for the ~2K limit on filters with "prio 0"?

Martin

On Wed, 19 Jan 2005 07:12:56 -0800, gypsy <gypsy@iswest.com> wrote:
> Martin Schiøtz wrote:
> >
> > Hi
> 
> Hello.
> 
> > I have been playing with tc. I'm using htb and sfq. I tried to setup
> > shaping of every ip address  within a /20.
> > I found that there must be an upper limit of about 2096 on the number
> > of rules (classes, qdiscs or filters). I was testing a /20 which gives
> > 4096 classes, qdiscs and filters. I'm using hashkey to optimize
> > processing of frames.
> 
> Filters.
> 
> > Is there a limit with tc or I'm I doing something wrong ?
> 
> Yes, sort of.  Yes.
> 
> > This is the script:
> > #!/bin/sh
> >
> 
> > /sbin/tc filter add dev eth1 parent 1:3 protocol ip u32
> 
> 'prio #' must be included in all 'filter' lines where "#" is not zero
> and is always the same; the "zero prio" has a limit and there is a limit
> on '#' - but no limit on how many times '#' can be (e.g.) 5
> 
> So, e.g.:
> tc filter add dev eth1 parent 1: protocol ip prio 5 \
>    u32 match ip src $IP flowid 1:40
> 
> will run at least 64K times without complaint, but this will fail after
> ~2K:
> tc filter add dev eth1 parent 1: protocol ip prio 0 \
>    u32 match ip src $IP flowid 1:40
> 
> > /sbin/tc filter add dev eth1 parent 1:3 handle 3: protocol ip u32 divisor 256
> > /sbin/tc filter add dev eth1 protocol ip parent 1: u32 match ip dst
> > 10.0.0.0/20 hashkey mask 0x000000ff at 16 link 3:
> 
> google LARTC please document
> 
> gypsy
>
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] tc limit

[gypsy]

Martin Schiøtz wrote:
> 
> Thanks, with "prio #" on all filters - all rules can be added.
> :-)

You're welcome.
 
> Just curious - Is there a reason for the ~2K limit on filters with "prio 0"?
> 
> Martin

No reason I'm aware of.  Probably some OOPS! in the code that creates
filters.  Since I've never bothered to learn C, I've also never bothered
to read the source code.  So I'd never spot a programming mistake no
matter how obvious, and I suspect that this one is very subtle.

The real crime here is that the docs do not tell you that there is this
trap that "everyone" with a lot of users falls into.

> On Wed, 19 Jan 2005 07:12:56 -0800, gypsy <gypsy@iswest.com> wrote:
> > Martin Schiøtz wrote:
> > >
> > > Hi
> >
> > Hello.
> >
> > > I have been playing with tc. I'm using htb and sfq. I tried to setup
> > > shaping of every ip address  within a /20.
> > > I found that there must be an upper limit of about 2096 on the number
> > > of rules (classes, qdiscs or filters). I was testing a /20 which gives
> > > 4096 classes, qdiscs and filters. I'm using hashkey to optimize
> > > processing of frames.
> >
> > Filters.
> >
> > > Is there a limit with tc or I'm I doing something wrong ?
> >
> > Yes, sort of.  Yes.
> >
> > > This is the script:
> > > #!/bin/sh
> > >
> >
> > > /sbin/tc filter add dev eth1 parent 1:3 protocol ip u32
> >
> > 'prio #' must be included in all 'filter' lines where "#" is not zero
> > and is always the same; the "zero prio" has a limit and there is a limit
> > on '#' - but no limit on how many times '#' can be (e.g.) 5
> >
> > So, e.g.:
> > tc filter add dev eth1 parent 1: protocol ip prio 5 \
> >    u32 match ip src $IP flowid 1:40
> >
> > will run at least 64K times without complaint, but this will fail after
> > ~2K:
> > tc filter add dev eth1 parent 1: protocol ip prio 0 \
> >    u32 match ip src $IP flowid 1:40
> >
> > > /sbin/tc filter add dev eth1 parent 1:3 handle 3: protocol ip u32 divisor 256
> > > /sbin/tc filter add dev eth1 protocol ip parent 1: u32 match ip dst
> > > 10.0.0.0/20 hashkey mask 0x000000ff at 16 link 3:
> >
> > google LARTC please document
> >
> > gypsy
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/