HTB - What's the minimal value for 'rate' parameter?

HTB - What's the minimal value for 'rate' parameter?

[Antonio Almeida]

Hi!
I've been using HTB for a while, and we've already sent some e-mails
each other when resolving HTB accuracy issue.
When using HTB, I realised that for some configurations the rate limit
doesn't work.
I suspect that the problem is the minimum value of rate parameter,
which I cant figure out what is.

I simple configuration that turns out to be wrong is as fallows: The
root (1:1) gets the link bandwidth configuration; the second (1:2) is
set to 4096Kbit; then I have two branches (1:10 and 1:11) with rate
1024Kbit and ceil 4096Kbit; and finally a leaf class in each branch
(1:111 below 1:11, and 1:101 below 1:10) with rate 8bit and ceil
4096Kbit, and the same priority.
I don't want to have sustained rate, and since I must configure 'rate'
parameter I decide to set it to 8bits - which is the minimal accepted
value. My cue goes for 'rate' parameter. If I set 'rate' parameter to
1Kbit for instance, the problem disappears and the shaping is done
perfectly.

So, I'm looking for help to find out if the problem is actually in
this parameter configuration or if it's just coincidence and I'll get
the same problem ahead :(
What's the minimal value for 'rate' parameter using HTB qdisc?

Here's the tc command output, using leaves rate set to 8bit:

# tc -s class list dev eth1
class htb 1:101 parent 1:10 leaf 101: prio 3 rate 8bit ceil 4096Kbit
burst 225b cburst 3655b
 Sent 42305702 bytes 27943 pkt (dropped 23031, overlimits 0 requeues 0)
 rate 4036Kbit 333pps backlog 0b 126p requeues 0
 lended: 27817 borrowed: 0 giants: 0
 tokens: 1250000000 ctokens: -39387

class htb 1:11 parent 1:2 rate 1024Kbit ceil 4096Kbit burst 2113b cburst 3655b
 Sent 42170956 bytes 27854 pkt (dropped 0, overlimits 0 requeues 0)
 rate 4035Kbit 333pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: -937499999 ctokens: -42881

class htb 1:10 parent 1:2 rate 1024Kbit ceil 4096Kbit burst 2113b cburst 3655b
 Sent 42114938 bytes 27817 pkt (dropped 0, overlimits 0 requeues 0)
 rate 4035Kbit 333pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: -937499999 ctokens: -39387

class htb 1:1 root rate 1000Mbit ceil 1000Mbit burst 503375b cburst 503375b
 Sent 84285894 bytes 55671 pkt (dropped 0, overlimits 0 requeues 0)
 rate 8071Kbit 666pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: 62750 ctokens: 62750

class htb 1:111 parent 1:11 leaf 111: prio 3 rate 8bit ceil 4096Kbit
burst 225b cburst 3655b
 Sent 42363234 bytes 27981 pkt (dropped 23064, overlimits 0 requeues 0)
 rate 4035Kbit 333pps backlog 0b 127p requeues 0
 lended: 27854 borrowed: 0 giants: 0
 tokens: 1250000000 ctokens: -42881

class htb 1:2 parent 1:1 rate 4096Kbit ceil 4096Kbit burst 3655b cburst 3655b
 Sent 84285894 bytes 55671 pkt (dropped 0, overlimits 0 requeues 0)
 rate 8071Kbit 666pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: -937499999 ctokens: -937499999

class sfq 111:16 parent 111:
 (dropped 0, overlimits 0 requeues 0)
 backlog 0b 127p requeues 0
 allot 1514

class sfq 101:252 parent 101:
 (dropped 0, overlimits 0 requeues 0)
 backlog 0b 126p requeues 0
 allot 1514


Regards
  Antonio Almeida

Re: HTB - What's the minimal value for 'rate' parameter?

[Jarek Poplawski]

Antonio Almeida wrote, On 04/08/2010 01:07 PM:

> Hi!

Hi!

> I've been using HTB for a while, and we've already sent some e-mails
> each other when resolving HTB accuracy issue.
> When using HTB, I realised that for some configurations the rate limit
> doesn't work.
> I suspect that the problem is the minimum value of rate parameter,
> which I cant figure out what is.
> 
> I simple configuration that turns out to be wrong is as fallows: The
> root (1:1) gets the link bandwidth configuration; the second (1:2) is
> set to 4096Kbit; then I have two branches (1:10 and 1:11) with rate
> 1024Kbit and ceil 4096Kbit; and finally a leaf class in each branch
> (1:111 below 1:11, and 1:101 below 1:10) with rate 8bit and ceil
> 4096Kbit, and the same priority.
> I don't want to have sustained rate, and since I must configure 'rate'
> parameter I decide to set it to 8bits - which is the minimal accepted
> value. My cue goes for 'rate' parameter. If I set 'rate' parameter to
> 1Kbit for instance, the problem disappears and the shaping is done
> perfectly.
> 
> So, I'm looking for help to find out if the problem is actually in
> this parameter configuration or if it's just coincidence and I'll get
> the same problem ahead :(
> What's the minimal value for 'rate' parameter using HTB qdisc?


I think "reasonable" or "minimally useful" (for common use) should be
enough, and 8bits meaning one 1500 byte packet per 25 minutes or
something, doesn't look like this to me.

This changelog:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=a4a710c4a7490587406462bf1d54504b7783d7d7
mentions ~2 minutes as max time for accounting, so 1 max packet
per 2 minutes should give such a minimal rate, I guess, but I'd
still multiply it a few times to call it useful.

Regards,
Jarek P.

Re: HTB - What's the minimal value for 'rate' parameter?

[Antonio Almeida]

So, what about the rate limit miss?
As you can see the ceil of class 1:2 is set to 4096Kbit but its
sending rate is actually 8071Kbit!
It looks like classes 1:10 and 1:11 are ignoring hierarchical rate
restrictions of class 1:2
Here:
class htb 1:2 parent 1:1 rate 4096Kbit ceil 4096Kbit burst 3655b cburst 3655b
 Sent 84285894 bytes 55671 pkt (dropped 0, overlimits 0 requeues 0)
 rate 8071Kbit 666pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: -937499999 ctokens: -937499999




On Thu, Apr 8, 2010 at 10:33 PM, Jarek Poplawski <jarkao2@gmail.com> wrote:
> Antonio Almeida wrote, On 04/08/2010 01:07 PM:
>
>> Hi!
>
> Hi!
>
>> I've been using HTB for a while, and we've already sent some e-mails
>> each other when resolving HTB accuracy issue.
>> When using HTB, I realised that for some configurations the rate limit
>> doesn't work.
>> I suspect that the problem is the minimum value of rate parameter,
>> which I cant figure out what is.
>>
>> I simple configuration that turns out to be wrong is as fallows: The
>> root (1:1) gets the link bandwidth configuration; the second (1:2) is
>> set to 4096Kbit; then I have two branches (1:10 and 1:11) with rate
>> 1024Kbit and ceil 4096Kbit; and finally a leaf class in each branch
>> (1:111 below 1:11, and 1:101 below 1:10) with rate 8bit and ceil
>> 4096Kbit, and the same priority.
>> I don't want to have sustained rate, and since I must configure 'rate'
>> parameter I decide to set it to 8bits - which is the minimal accepted
>> value. My cue goes for 'rate' parameter. If I set 'rate' parameter to
>> 1Kbit for instance, the problem disappears and the shaping is done
>> perfectly.
>>
>> So, I'm looking for help to find out if the problem is actually in
>> this parameter configuration or if it's just coincidence and I'll get
>> the same problem ahead :(
>> What's the minimal value for 'rate' parameter using HTB qdisc?
>
>
> I think "reasonable" or "minimally useful" (for common use) should be
> enough, and 8bits meaning one 1500 byte packet per 25 minutes or
> something, doesn't look like this to me.
>
> This changelog:
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=a4a710c4a7490587406462bf1d54504b7783d7d7
> mentions ~2 minutes as max time for accounting, so 1 max packet
> per 2 minutes should give such a minimal rate, I guess, but I'd
> still multiply it a few times to call it useful.
>
> Regards,
> Jarek P.
>
>
>

Re: HTB - What's the minimal value for 'rate' parameter?

[Jarek Poplawski]

On Fri, Apr 09, 2010 at 04:40:44PM +0100, Antonio Almeida wrote:
> So, what about the rate limit miss?
> As you can see the ceil of class 1:2 is set to 4096Kbit but its
> sending rate is actually 8071Kbit!
> It looks like classes 1:10 and 1:11 are ignoring hierarchical rate
> restrictions of class 1:2
> Here:
> class htb 1:2 parent 1:1 rate 4096Kbit ceil 4096Kbit burst 3655b cburst 3655b
>  Sent 84285894 bytes 55671 pkt (dropped 0, overlimits 0 requeues 0)
>  rate 8071Kbit 666pps backlog 0b 0p requeues 0
>  lended: 0 borrowed: 0 giants: 0
>  tokens: -937499999 ctokens: -937499999

Yes, since 1:2 has grandchildren with overflown rate tables, they
could behave as if they had set rates higher than their parents or
grandparent (and HTB doesn't restrict it hierarchically).

Jarek P.

Re: HTB - What's the minimal value for 'rate' parameter?

[Antonio Almeida]

What do you mean with "1:2 has grandchildren with overflown rate tables"?
I couldn't understand your idea. Is there any mistake in the
configuration I sent?
How would you set rates for this particular example?

Regards
  Antonio Almeida



On Fri, Apr 9, 2010 at 10:26 PM, Jarek Poplawski wrote:
> On Fri, Apr 09, 2010 at 04:40:44PM +0100, Antonio Almeida wrote:
>> So, what about the rate limit miss?
>> As you can see the ceil of class 1:2 is set to 4096Kbit but its
>> sending rate is actually 8071Kbit!
>> It looks like classes 1:10 and 1:11 are ignoring hierarchical rate
>> restrictions of class 1:2
>> Here:
>> class htb 1:2 parent 1:1 rate 4096Kbit ceil 4096Kbit burst 3655b cburst 3655b
>>  Sent 84285894 bytes 55671 pkt (dropped 0, overlimits 0 requeues 0)
>>  rate 8071Kbit 666pps backlog 0b 0p requeues 0
>>  lended: 0 borrowed: 0 giants: 0
>>  tokens: -937499999 ctokens: -937499999
>
> Yes, since 1:2 has grandchildren with overflown rate tables, they
> could behave as if they had set rates higher than their parents or
> grandparent (and HTB doesn't restrict it hierarchically).
>
> Jarek P.
>

Re: HTB - What's the minimal value for 'rate' parameter?

[Jarek Poplawski]

Antonio Almeida wrote, On 04/14/2010 12:22 PM:

> What do you mean with "1:2 has grandchildren with overflown rate tables"?
> I couldn't understand your idea. Is there any mistake in the
> configuration I sent?
> How would you set rates for this particular example?


class htb 1:1 root rate 1000Mbit ceil 1000Mbit
class htb 1:2 parent 1:1 rate 4096Kbit ceil 4096Kbit
class htb 1:10 parent 1:2 rate 1024Kbit ceil 4096Kbit
class htb 1:11 parent 1:2 rate 1024Kbit ceil 4096Kbit
class htb 1:101 parent 1:10 prio 3 rate 8bit ceil 4096Kbit
class htb 1:111 parent 1:11 prio 3 rate 8bit ceil 4096Kbit

Classes 1:101 and 1:111 have too low rates, which causes wrong (overflowed!)
values in their rate tables, so their rates could be practically
uncontrollable. They are limited by their ceils instead, so something like:

class htb 1:101 parent 1:10 leaf 101: prio 3 rate 4096Kbit ceil 4096Kbit
class htb 1:111 parent 1:11 leaf 111: prio 3 rate 4096Kbit ceil 4096Kbit

But then their guaranteed rates are higher than their parents, and the
sum is higher than grandparent's rate, which means the config is wrong.
(You have to control these sums - HTB doesn't.)

As I wrote before, the minimal (overflow safe) rate depends on max
packet size, and for 1500 byte it would be something around:
1500b/2min, so if your clients can wait so long, try this:

class htb 1:101 parent 1:10 leaf 101: prio 3 rate 100bit ceil 4096Kbit
class htb 1:111 parent 1:11 leaf 111: prio 3 rate 100bit ceil 4096Kbit

Regards,
Jarek P.

Re: HTB - What's the minimal value for 'rate' parameter?

[Antonio Almeida]

Now I understand. It makes sense - totally! Thanks for your endurance
trying to open my eyes :)
I've been trying rates bigger that 100bit for a while and it's working fine.
Thanks a lot for your illustration!

Regards
  Antonio Almeida



On Wed, Apr 14, 2010 at 10:45 PM, Jarek Poplawski wrote:
> Antonio Almeida wrote, On 04/14/2010 12:22 PM:
>
>> What do you mean with "1:2 has grandchildren with overflown rate tables"?
>> I couldn't understand your idea. Is there any mistake in the
>> configuration I sent?
>> How would you set rates for this particular example?
>
>
> class htb 1:1 root rate 1000Mbit ceil 1000Mbit
> class htb 1:2 parent 1:1 rate 4096Kbit ceil 4096Kbit
> class htb 1:10 parent 1:2 rate 1024Kbit ceil 4096Kbit
> class htb 1:11 parent 1:2 rate 1024Kbit ceil 4096Kbit
> class htb 1:101 parent 1:10 prio 3 rate 8bit ceil 4096Kbit
> class htb 1:111 parent 1:11 prio 3 rate 8bit ceil 4096Kbit
>
> Classes 1:101 and 1:111 have too low rates, which causes wrong (overflowed!)
> values in their rate tables, so their rates could be practically
> uncontrollable. They are limited by their ceils instead, so something like:
>
> class htb 1:101 parent 1:10 leaf 101: prio 3 rate 4096Kbit ceil 4096Kbit
> class htb 1:111 parent 1:11 leaf 111: prio 3 rate 4096Kbit ceil 4096Kbit
>
> But then their guaranteed rates are higher than their parents, and the
> sum is higher than grandparent's rate, which means the config is wrong.
> (You have to control these sums - HTB doesn't.)
>
> As I wrote before, the minimal (overflow safe) rate depends on max
> packet size, and for 1500 byte it would be something around:
> 1500b/2min, so if your clients can wait so long, try this:
>
> class htb 1:101 parent 1:10 leaf 101: prio 3 rate 100bit ceil 4096Kbit
> class htb 1:111 parent 1:11 leaf 111: prio 3 rate 100bit ceil 4096Kbit
>
> Regards,
> Jarek P.
>

Re: HTB - What's the minimal value for 'rate' parameter?

[Benny Amorsen]

Jarek Poplawski <jarkao2@gmail.com> writes:

> As I wrote before, the minimal (overflow safe) rate depends on max
> packet size, and for 1500 byte it would be something around:
> 1500b/2min, so if your clients can wait so long, try this:

Wouldn't it be nice of either tc or the kernel to warn about wrong
configurations, or possibly reject them completely?


/Benny

Re: HTB - What's the minimal value for 'rate' parameter?

[Jarek Poplawski]

Benny Amorsen wrote, On 04/17/2010 11:19 AM:

> Jarek Poplawski <jarkao2@gmail.com> writes:
> 
>> As I wrote before, the minimal (overflow safe) rate depends on max
>> packet size, and for 1500 byte it would be something around:
>> 1500b/2min, so if your clients can wait so long, try this:
> 
> Wouldn't it be nice of either tc or the kernel to warn about wrong
> configurations, or possibly reject them completely?

...or have it documented etc.

Acked-by: Jarek P. ;-)