[LARTC] exploring HTB

[LARTC] exploring HTB

[Sumit Pandya]

Hi All,
	I have few queries in reference of HTB manuals on link
http://luxik.cdi.cz/~devik/qos/htb/htbman.htm.

1> Is "default" is must while attaching queue discipline HTB to device?
	My observation for above question is, when I do following 4 command then I’
m able to reach to all hosts on eth1. But if I remove "default 12" from
second command then I cannot reach to hosts on eth1, but I can reach to the
host on other device (eth0, eth2). This occurs after I create extra class to
serve as the root (i.e. creating class having parent 1:1, 4th command)
tc qdisc del dev eth1 root
tc qdisc add dev eth1 root handle 1: htb default 12
tc class add dev eth1 parent 1: classid 1:1 htb rate 100kbps ceil 100kbps
burst 2k
tc class add dev eth1 parent 1:1 classid 1:11 htb rate 10kbps ceil 100kbps
burst 2k
tc class add dev eth1 parent 1:1 classid 1:12 htb rate 60kbps ceil 100kbps
burst 2k

	Ya, this reachability can be restored only after delition of root class,
i.e. classid 1:1. I'm taking this as a feature but is it a bug?

2> Manual states that "... ... ... Now we can optionally attach queuing
disciplines to the leaf classes... ... ... tc qdisc add dev eth0 parent 1:10
handle 20: pfifo limit 5... ... ...". If I not going to do that still I'm
fine with my rule set, as written "optional". Then is there benefit or
payoff against this leaf-class?

3> A Small but nice document about Cross device sharing on link
http://luxik.cdi.cz/~devik/qos/htb/htbmeas1.htm. Can’t this limitation of
slot be increased?

4> It would be nice if someone help me by providing statistics over number
of queues and filter. What is maximum limit of these two for a PIII 700 with
256 MB RAM? How much maximum bandwidth can be shaped?

Thanks in Advance.
-- Sumit

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] exploring HTB

[Stef Coene]

> 2> Manual states that "... ... ... Now we can optionally attach queuing
> disciplines to the leaf classes... ... ... tc qdisc add dev eth0 parent
> 1:10 handle 20: pfifo limit 5... ... ...". If I not going to do that still
> I'm fine with my rule set, as written "optional". Then is there benefit or
> payoff against this leaf-class?
Each class has a qdisc attached.  By default, this is a FIFO one (First In, 
First Out).  You can replace it with a SFQ qdisc.  A SFQ qdisc will treath 
each flow with the same priority.  So one flow can not eat all bandwidth.

> 3> A Small but nice document about Cross device sharing on link
> http://luxik.cdi.cz/~devik/qos/htb/htbmeas1.htm. Can’t this limitation of
> slot be increased?
Can't find that page.

>
> 4> It would be nice if someone help me by providing statistics over number
> of queues and filter. What is maximum limit of these two for a PIII 700
> with 256 MB RAM? How much maximum bandwidth can be shaped?
I think you can shape a lot of traffic with this setup, but I don't know 
exactly.

Stef

-- 

stef.coene@docum.org
 More QOS info : http://www.docum.org/
 Title : "Using Linux as bandwidth manager"
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

Re: [LARTC] exploring HTB

[Martin Devera]

> 
> 1> Is "default" is must while attaching queue discipline HTB to device?

no it is optional.

> 	My observation for above question is, when I do following 4 command then I’
> m able to reach to all hosts on eth1. But if I remove "default 12" from
> second command then I cannot reach to hosts on eth1, but I can reach to the
> host on other device (eth0, eth2). This occurs after I create extra class to
> serve as the root (i.e. creating class having parent 1:1, 4th command)
> tc qdisc del dev eth1 root
> tc qdisc add dev eth1 root handle 1: htb default 12
> tc class add dev eth1 parent 1: classid 1:1 htb rate 100kbps ceil 100kbps
> burst 2k
> tc class add dev eth1 parent 1:1 classid 1:11 htb rate 10kbps ceil 100kbps
> burst 2k
> tc class add dev eth1 parent 1:1 classid 1:12 htb rate 60kbps ceil 100kbps
> burst 2k
> 
> 	Ya, this reachability can be restored only after delition of root class,
> i.e. classid 1:1. I'm taking this as a feature but is it a bug?

depends on filters you are using. Without "default" all unclassified
traffic is lost (like ARP packets) unless you have some catch-all
filter which cares about such traffic.
The bahaviour changed in htb2 patch - not even without default all
unclassified traffic goes into virtual X:0 class which is pass-thru
internal fifo IIRC.
It is funny that I can't remember how I implemented latest semantic here.
... well I just checked the source and I'm right with the fifo.

You can look at stats from tc -s class ... to see fate of your packets.

> 2> Manual states that "... ... ... Now we can optionally attach queuing
> disciplines to the leaf classes... ... ... tc qdisc add dev eth0 parent 1:10
> handle 20: pfifo limit 5... ... ...". If I not going to do that still I'm
> fine with my rule set, as written "optional". Then is there benefit or
> payoff against this leaf-class?

this general tc issue was replied by Stef to you :) (thanx Stef)

> 3> A Small but nice document about Cross device sharing on link
> http://luxik.cdi.cz/~devik/qos/htb/htbmeas1.htm. Can’t this limitation of
> slot be increased?

the slots was removed from latest code and link above is obsolete (where
you have found it ?) ;)
The slots concept was too much of hack and there was too many races
on SMP (unfixable ones). The IMQ device is superior for Cross device
sharing and it is a LOT simpler ;)
See http://luxik.cdi.cz/~devik/qos/

> 4> It would be nice if someone help me by providing statistics over number
> of queues and filter. What is maximum limit of these two for a PIII 700 with
> 256 MB RAM? How much maximum bandwidth can be shaped?

I've tried on my pII 375, I was able to shape 100Mbit with 5 classes (3
leaves). But both CBQ and HTB has O(N) worst case complexity where N is
no of classes. I just finished new HTB design (with the same interface
of course) and it should be faster in this way.
The I will do some serious testing on ratw vs. no of classes.

Just to note that speed depends ONLY on number of ACTIVE classes (active
flows). Those without data enqueued are idle and don't decrease
throughtput.
Also limiting is packet rate not byte rate .. The limit is somewhere
at 30000 packets per second for both CBQ and HTB with 5 classes which
is cca 500Mbit for 1.5k packets (on my PII).

devik

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

RE: [LARTC] exploring HTB

[Sumit Pandya]

-----Original Message-----
From: Martin Devera [mailto:devik@cdi.cz]

>> 3> A Small but nice document about Cross device sharing on link
>> http://luxik.cdi.cz/~devik/qos/htb/htbmeas1.htm. Can?t this limitation of
>> slot be increased?

> the slots was removed from latest code and link above is obsolete (where
> you have found it ?) ;)
	I can still find this link despite of refreshing page in my browser, May
some cache server running in between.

>The IMQ device is superior for Cross device sharing and it is a LOT simpler
;)
> See http://luxik.cdi.cz/~devik/qos/
	COOL IMQ. How much throughput will be reduced or latency will be added
while using IMQ? As in every flow it adds/increase one hope internally?

>> 4> It would be nice if someone help me by providing statistics over
number
>> of queues and filter. What is maximum limit of these two for a PIII 700
with
>> 256 MB RAM? How much maximum bandwidth can be shaped?

> I've tried on my pII 375, I was able to shape 100Mbit with 5 classes (3
> leaves). But both CBQ and HTB has O(N) worst case complexity where N is
> no of classes. I just finished new HTB design (with the same interface
> of course) and it should be faster in this way.
> The I will do some serious testing on ratw vs. no of classes.

> Just to note that speed depends ONLY on number of ACTIVE classes (active
> flows). Those without data enqueued are idle and don't decrease
> throughtput.
	Presently I'm running setup with 200 queues (200 RED leaves and 200 u32
classifiers), Thus basically a seperate queue for every individual in my
organization. Ya I never knew about the dependency of throughput on active
flows. I'm planing to go for 400 queues. Sometime before Documentations on
Devik's home page stat that RED is not going to work with HTB and will
simply eat CPU cycles. I couldn't locate it now anywhere? Does it works with
new release of HTB?

> Also limiting is packet rate not byte rate .. The limit is somewhere
> at 30000 packets per second for both CBQ and HTB with 5 classes which
> is cca 500Mbit for 1.5k packets (on my PII).
	But ultimately its going to limit on byte transfer, isn't it?

-- Sumit

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

RE: [LARTC] exploring HTB

[Martin Devera]

> > the slots was removed from latest code and link above is obsolete (where
> > you have found it ?) ;)
> 	I can still find this link despite of refreshing page in my browser, May
> some cache server running in between.

No cache .. the page is still on my server but it is treated
as obsolete. I just marked the link to it as such ;)
 
> 	COOL IMQ. How much throughput will be reduced or latency will be added
> while using IMQ? As in every flow it adds/increase one hope internally?

Yes it is one enqueue and dequeue for qdisc. So that is will add
latency as dictated by the qdisc .. So that there should be only
negligible unwanted performance hit. 
 
> > Just to note that speed depends ONLY on number of ACTIVE classes (active
> > flows). Those without data enqueued are idle and don't decrease
> > throughtput.
> 	Presently I'm running setup with 200 queues (200 RED leaves and 200 u32
> classifiers), Thus basically a seperate queue for every individual in my
> organization. Ya I never knew about the dependency of throughput on active
> flows. I'm planing to go for 400 queues. Sometime before Documentations on
> Devik's home page stat that RED is not going to work with HTB and will
> simply eat CPU cycles. I couldn't locate it now anywhere? Does it works with
> new release of HTB?

I can't remember that I would have such note on my pages. But basically
HTB,CBQ and TBF are exclusive options. They don't fit well one into
another. It is because they are all absolute rate dependent and all are
non-work conserving.
RED is also dependant on absolute rate but it is work-conserving. So that
it can be used as CBQ's or HTB's leaf and there should be ho hard problem
with it. However I don't know RED sources well so that I can't predict
some obscure problems.
 
> > Also limiting is packet rate not byte rate .. The limit is somewhere
> 	But ultimately its going to limit on byte transfer, isn't it?

Yes as end effect it is.
devik 

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

RE: [LARTC] exploring HTB

[Don Cohen]

 > Subject: RE: [LARTC] exploring HTB

I expected someone else to say this before, but since nobody has yet ... 

 > > > Just to note that speed depends ONLY on number of ACTIVE classes (active
 > > > flows). Those without data enqueued are idle and don't decrease
 > > > throughtput.
First, the classification time could be pretty significant with
hundreds of filters.  You could minimize this by building a balanced
tree of classes, so that it would only take log(#classes) filters, but
that would be a lot of trouble to build and maintain.

 > > 	Presently I'm running setup with 200 queues (200 RED leaves and 200 u32
 > > classifiers), Thus basically a seperate queue for every individual in my
 > > organization. Ya I never knew about the dependency of throughput on active
 > > flows. I'm planing to go for 400 queues. Sometime before Documentations on
 > > Devik's home page stat that RED is not going to work with HTB and will
 > > simply eat CPU cycles. I couldn't locate it now anywhere? Does it works with
 > > new release of HTB?

I don't think this is the problem HTB was meant to solve.
I think what you really want is the (often suggested/requested)
modified version of sfq that hashes only on source or destination
address (depending on whether you use it for the queue going in to or
out of your local network).  If your addresses are consecutive then
the "hash" function could be particularly good (fast and no
collisions). This would not allow you to offer different amounts of
service to different addresses, but that could also be added easily
(though it would then reintroduce the inconvenience of maintaining
this data in the configuration file).
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

RE: [LARTC] exploring HTB

[Martin Devera]

>  > > > Just to note that speed depends ONLY on number of ACTIVE classes (active
>  > > > flows). Those without data enqueued are idle and don't decrease
>  > > > throughtput.
> First, the classification time could be pretty significant with
> hundreds of filters.  You could minimize this by building a balanced
> tree of classes, so that it would only take log(#classes) filters, but
> that would be a lot of trouble to build and maintain.

I'm not sure. How balanced tree of classes could help you with big number
of filters ? These are unrelated IMHO. When filter determines class number
it is looked using hash table.
If you use u32 with 200 nodes in tree and htb then classification time
will be about 5 steps for each packet...
 
>  > > 	Presently I'm running setup with 200 queues (200 RED leaves and 200 u32
>  > > classifiers), Thus basically a seperate queue for every individual in my
>  > > organization. Ya I never knew about the dependency of throughput on active
> 
> I don't think this is the problem HTB was meant to solve.
> I think what you really want is the (often suggested/requested)
> modified version of sfq that hashes only on source or destination
> address (depending on whether you use it for the queue going in to or

Right. I think so too :) HTB can be useful in this way if you want
different parameters (rate,prio) per host ..
But we definitely need the changed sfq. The change is trivial but
I'm too deep in new htb just now :(
devik

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

RE: [LARTC] exploring HTB

[Don Cohen]

Martin Devera writes:
 > I'm not sure. How balanced tree of classes could help you with big number
 > of filters ? These are unrelated IMHO. When filter determines class number
 > it is looked using hash table.
 > If you use u32 with 200 nodes in tree and htb then classification time
 > will be about 5 steps for each packet...
What do you do if you have 100 rules of form:
 if source ip = ip1 then class1
 if source ip = ip2 then class2
 if source ip = ip3 then class3
 ...
I assume you have to try them until one matches or you run out.

However, I believe that you now support filters at interior classes.
This means that at the top you could do something like
 if source ip in 10.0.0.0 - 10.0.0.50 then class1
 if source ip in 10.0.0.51 - 10.0.0.99 then class2
and then in class1 put filters:
 if source ip in 10.0.0.0 - 10.0.0.25 then class3
 if source ip in 10.0.0.26 - 10.0.0.50 then class3
etc.  Then you'd only have to match o(log(n)) filters.

 > > I don't think this is the problem HTB was meant to solve.
 > > I think what you really want is the (often suggested/requested)
 > > modified version of sfq that hashes only on source or destination
 > > address (depending on whether you use it for the queue going in to or
 > 
 > Right. I think so too :) HTB can be useful in this way if you want
 > different parameters (rate,prio) per host ..
 > But we definitely need the changed sfq. The change is trivial but
 > I'm too deep in new htb just now :(

I can't promise any time soon either.  If someone else wants to work
on this I'd like to send some (reverse engineered) documentation to
include in future sfq - might also help you to make further changes.
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/

RE: [LARTC] exploring HTB

[Martin Devera]

> What do you do if you have 100 rules of form:
>  if source ip = ip1 then class1
>  if source ip = ip2 then class2
>  if source ip = ip3 then class3
>  ...
> I assume you have to try them until one matches or you run out.

If these are different filters then yes. But you can use u32's hash
to match them all in one step.
 
> However, I believe that you now support filters at interior classes.
> This means that at the top you could do something like
>  if source ip in 10.0.0.0 - 10.0.0.50 then class1
>  if source ip in 10.0.0.51 - 10.0.0.99 then class2
> and then in class1 put filters:
>  if source ip in 10.0.0.0 - 10.0.0.25 then class3
>  if source ip in 10.0.0.26 - 10.0.0.50 then class3
> etc.  Then you'd only have to match o(log(n)) filters.

Oh yes. But the you can't use classes for balancing rates - because
you will need balanced binary tree ..
But if you use inner classes then you are right it will be always
better (faster). But for large ip count I'd use u32 hash table.

devik

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/lartc/