[LARTC] Weird HTB behaviour in 2.6.17

[LARTC] Weird HTB behaviour in 2.6.17

[Marlon Dutra]

Hello,

I've just compiled the kernel 2.6.17 and noticed an odd HTB behaviour.

For instance:

tc class add dev eth0 parent 1:1 classid 1:30 htb rate 1kbit ceil
750Kbit burst 15k
tc filter add dev eth0 parent 1:0 protocol ip prio 15 u32 match ip dst
192.168.5.1 classid 1:30

The filter works ok and the traffic to 192.168.5.1 flows through the
class 1:30, but the rate gets much higher than 750 Kbps.

As far as I realized, the faster the processor is, the higher the
traffic gets above the ceiling.

I've seen that same behaviour in several machines with this kernel,
both with Intel and AMD processors. With a dual Xeon 3 GHz, when my
ceiling is 750 Kbps, I get the traffic about 5 Mbps.

It seems to be something related to clock.

http://mailman.ds9a.nl/pipermail/lartc/2005q3/016981.html

In that post, there is another guy with the same problem, but with 2.6.11.

Any clues?

-- 
Marlon
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

RE: [LARTC] Weird HTB behaviour in 2.6.17

[Jake Altchill]

What kind of network cards do you have?
What are the tc stats that you get on
the class (anything about giants?)?

There's something funny with the intel gig cards.
Even if the mtu on them is set to 1500
there's something that breaks / confuses htb.
If these are the cards you are using you
need to explicitly set the mtu for your
classes to 16500.


>Hello,
>
>I've just compiled the kernel 2.6.17 and noticed an odd HTB behaviour.
>
>For instance:
>
>tc class add dev eth0 parent 1:1 classid 1:30 htb rate 1kbit ceil
>750Kbit burst 15k
>tc filter add dev eth0 parent 1:0 protocol ip prio 15 u32 match ip dst
>192.168.5.1 classid 1:30
>
>The filter works ok and the traffic to 192.168.5.1 flows through the
>class 1:30, but the rate gets much higher than 750 Kbps.
>
>As far as I realized, the faster the processor is, the higher the
>traffic gets above the ceiling.
>
>I've seen that same behaviour in several machines with this kernel,
>both with Intel and AMD processors. With a dual Xeon 3 GHz, when my
>ceiling is 750 Kbps, I get the traffic about 5 Mbps.
>
>It seems to be something related to clock.
>
>http://mailman.ds9a.nl/pipermail/lartc/2005q3/016981.html
>
>In that post, there is another guy with the same problem, but with 2.6.11.
>
>Any clues?

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today - it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Weird HTB behaviour in 2.6.17

[Marlon Dutra]

On 7/27/06, Jake Altchill <altchill@hotmail.com> wrote:

> What kind of network cards do you have?

When I noticed the problem, I was using a Dell server with an Intel
e1000.

But I reproduced the same problem in my desktop, that's an AMD Semprom
with an nVidia ethernet card. So, different processor, chipset and nic.

> What are the tc stats that you get on the class (anything about
> giants?)?

class htb 1:31 parent 1:1 leaf 31: prio 0 rate 1000bit ceil 750000bit
burst 15Kb cburst 1974b
 Sent 11678670 bytes 2711 pkts (dropped 0, overlimits 0)
 rate 1725Kbit 43pps backlog 13p

As you can see, I have rate in 1725Kbit when my ceiling is 750Kbit. That
difference is low, since I'm testing on my Semprom. If I do the same on
the Dell server, the difference is much higher. Some times over ten
times the ceiling.

With a 2.4 kernel, it works perfectly in both hardwares.

> If these are the cards you are using you need to explicitly set the
> mtu for your classes to 16500.

Sorry, I didn't get it. How should I do that?

-- 
Marlon
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Weird HTB behaviour in 2.6.17

[Andy Furniss]

Marlon Dutra wrote:
  (anything about
>> giants?)?
> 
> 
> class htb 1:31 parent 1:1 leaf 31: prio 0 rate 1000bit ceil 750000bit
> burst 15Kb cburst 1974b
> Sent 11678670 bytes 2711 pkts (dropped 0, overlimits 0)
> rate 1725Kbit 43pps backlog 13p
> 
> As you can see, 

I see you snipped giants even though you were asked about them.

Regardless 11678670/2711 > 1500 so specify your mtu next to every rate 
and ceil.

Andy.
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Weird HTB behaviour in 2.6.17

[Marlon Dutra]

On 7/28/06, Andy Furniss <lists@andyfurniss.entadsl.com> wrote:

> I see you snipped giants even though you were asked about them.

Oh sorry. Here it goes:

class htb 1:31 parent 1:1 leaf 31: prio 0 rate 1000bit ceil 750000bit
burst 15Kb cburst 1875b
 Sent 23991629 bytes 5227 pkts (dropped 0, overlimits 0)
 rate 1567Kbit 44pps
 lended: 150 borrowed: 5077 giants: 3986
 tokens: -14728330 ctokens: -21365

> Regardless 11678670/2711 > 1500 so specify your mtu next to every rate
> and ceil.

I created the class with "mtu 1500" and the result is above, same
behaviour. Afaik, the default mtu is 1600.

By the way, my ethernet MTU is 1500.

How can I prevent those giants?

-- 
Marlon
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Weird HTB behaviour in 2.6.17

[Andy Furniss]

Marlon Dutra wrote:
> On 7/28/06, Andy Furniss <lists@andyfurniss.entadsl.com> wrote:
> 
>> I see you snipped giants even though you were asked about them.
> 
> 
> Oh sorry. Here it goes:
> 
> class htb 1:31 parent 1:1 leaf 31: prio 0 rate 1000bit ceil 750000bit
> burst 15Kb cburst 1875b
> Sent 23991629 bytes 5227 pkts (dropped 0, overlimits 0)
> rate 1567Kbit 44pps
> lended: 150 borrowed: 5077 giants: 3986
> tokens: -14728330 ctokens: -21365
> 
>> Regardless 11678670/2711 > 1500 so specify your mtu next to every rate
>> and ceil.
> 
> 
> I created the class with "mtu 1500" and the result is above, same
> behaviour. Afaik, the default mtu is 1600.
> 
> By the way, my ethernet MTU is 1500.
> 
> How can I prevent those giants?

It seems there is a problem with some nics not obeying mtu then (I mean 
ifconfig/ip mtu) maybe it's to do with tso or something.

Normally you shouldn't have to specify mtu to tc if you are running 1500 
- you only need to if you are bigger, which it looks like you are 
whatever ifconfig/ip says.

I would find/lookup what you are really using and tell htb that and then 
check that the giants count is 0.

Andy.







> 

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Weird HTB behaviour in 2.6.17

[Jake Altchill]

Just append "mtu 16500" at the end of your qdisc
and class (especially class) statements. Like this:

tc class add dev eth0 parent 1:1 classid 1:30 htb rate 1kbit ceil
750Kbit burst 15k mtu 16500

I had a similar problem on Dell 2850s with intel e1000
cards. The mtu on the e1000 interfaces was set to the
default 1500, but the cards didn't obey it all the time.

I suspect that nVidia ethernet driver has similar
problems.

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today - it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Weird HTB behaviour in 2.6.17

[Patrick McHardy]

Andy Furniss wrote:
> Marlon Dutra wrote:
> 
>> lended: 150 borrowed: 5077 giants: 3986
>> tokens: -14728330 ctokens: -21365
>>
>>> Regardless 11678670/2711 > 1500 so specify your mtu next to every rate
>>> and ceil.
>>
>> I created the class with "mtu 1500" and the result is above, same
>> behaviour. Afaik, the default mtu is 1600.
>>
>> By the way, my ethernet MTU is 1500.
>>
>> How can I prevent those giants?
> 
> 
> It seems there is a problem with some nics not obeying mtu then (I mean
> ifconfig/ip mtu) maybe it's to do with tso or something.


Right, the "problem" is related to TCP segmentation offloading (or
GSO in current kernels). The card gets large chunks of data and is
responsible for creating MTU-sized packets, which essentially means
qdiscs get to see those large chunks of data. You can disable TSO
using ethtool (but it will cost you performance) or configure your
qdisc appropriately.


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Weird HTB behaviour in 2.6.17

[Marlon Dutra]

Hello,

On 7/28/06, Patrick McHardy <kaber@trash.net> wrote:

> Right, the "problem" is related to TCP segmentation offloading (or GSO
> in current kernels). The card gets large chunks of data and is
> responsible for creating MTU-sized packets, which essentially means
> qdiscs get to see those large chunks of data. You can disable TSO
> using ethtool (but it will cost you performance) or configure your
> qdisc appropriately.

Thanks a lot. It was exactly that. I turned the tso off and HTB is
working properly.

How can I know the largest chunk of data the kernel sends to the card,
so that I can configure qdisc appropriately?

In the last post, Jake Altchill recommended using mtu 16500 in the
qdiscs, but I'm not sure whether that's a real or just a big number.

Regards.

-- 
Marlon
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc