[LARTC] TC (HTB) doesn't work well when network is congested?

All of lore.kernel.org
 help / color / mirror / Atom feed

* [LARTC] TC (HTB) doesn't work well when network is congested?
@ 2007-10-25 16:38 William Xu
  2007-10-25 17:22 ` Peter Rabbitson
                   ` (6 more replies)
  0 siblings, 7 replies; 8+ messages in thread
From: William Xu @ 2007-10-25 16:38 UTC (permalink / raw)
  To: lartc

Hi,

I have a server and ten clients in a Gigabit network. The server has 
125mbps network bandwidth.
I want that the server has 40Mbps bandwidth reserved for client 1 (IP 
192.168.5.141), and the
rest bandwidth is for all other clients.

My script looks like this (I use IFB for incoming traffic):

#!/bin/bash

export TC="/sbin/tc"

$TC qdisc add dev ifb0 root handle 1: htb default 30 r2q 100

$TC class add dev ifb0 parent 1:0 classid 1:1 htb rate 125mbps mtu 9000
$TC class add dev ifb0 parent 1:1 classid 1:10 htb rate 40mbps ceil 
125mbps mtu 9000 prio 0
$TC class add dev ifb0 parent 1:1 classid 1:30 htb rate 85mbps ceil 
125mbps mtu 9000 prio 1

$TC filter add dev ifb0 parent 1: protocol ip prio 1 u32 match ip src 
192.168.5.141/32 flowid 1:10

$TC qdisc add dev eth2 ingress
$TC filter add dev eth2 parent ffff: protocol ip prio 1 u32 \
        match u32 0 0 flowid 1:1 \
        action mirred egress redirect dev ifb0

$TC qdisc add dev eth2 root handle 1: htb default 30 r2q 100

$TC class add dev eth2 parent 1: classid 1:1 htb rate 125mbps mtu 9000
$TC class add dev eth2 parent 1:1 classid 1:10 htb rate 40mbps ceil 
125mbps mtu 9000 prio 0
$TC class add dev eth2 parent 1:1 classid 1:30 htb rate 85mbps ceil 
125mbps mtu 9000 prio 1

$TC filter add dev eth2 parent 1: protocol ip prio 1 u32 match ip dst 
192.168.5.141/32 classid 1:10

I ran a test in which all 10 clients send/receive packets to/from the 
server simultaneously. But
Client 1 only got 20mbps bandwidth for sending, and 38mpbs for 
receiving. If I limit the rate of
both classes 1:1 to 60mbps instead of 125mbps, Client 1 got 39mbps for 
sending, and 40mbps for
receiving.

I am not sure what might cause this. Is it because TC doesn't work well 
when network is congested?
Or my script is not right?

Thanks a lot,
william

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [LARTC] TC (HTB) doesn't work well when network is congested?
  2007-10-25 16:38 [LARTC] TC (HTB) doesn't work well when network is congested? William Xu
@ 2007-10-25 17:22 ` Peter Rabbitson
  2007-10-25 18:20 ` William Xu
                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Peter Rabbitson @ 2007-10-25 17:22 UTC (permalink / raw)
  To: lartc

William Xu wrote:
> Hi,
> 
> I have a server and ten clients in a Gigabit network. The server has 
> 125mbps network bandwidth.
> I want that the server has 40Mbps bandwidth reserved for client 1 (IP 
> 192.168.5.141), and the
> rest bandwidth is for all other clients.
> 
> <snip>
> 
> I ran a test in which all 10 clients send/receive packets to/from the 
> server simultaneously. But
> Client 1 only got 20mbps bandwidth for sending, and 38mpbs for 
> receiving. If I limit the rate of
> both classes 1:1 to 60mbps instead of 125mbps, Client 1 got 39mbps for 
> sending, and 40mbps for
> receiving.
> 
> I am not sure what might cause this. Is it because TC doesn't work well 
> when network is congested?
> Or my script is not right?
> 

No network will be able to operate at its theoretical maximum. In the 
case of a gigabit network you will be lucky to get consistent 120mbps, 
and it heavily depends on the hardware quality, and the number of 
switches in between. So what you are doing is oversaturating the link, 
the ACK packets can not get through, your speed drops due to 
delays/retransmissions. Perform a test with only two systems sending 
stuff to each other to see what is the actual bandwidth you can hope 
for, and use that number instead of 125mbps.
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [LARTC] TC (HTB) doesn't work well when network is congested?
  2007-10-25 16:38 [LARTC] TC (HTB) doesn't work well when network is congested? William Xu
  2007-10-25 17:22 ` Peter Rabbitson
@ 2007-10-25 18:20 ` William Xu
  2007-10-25 21:13 ` Peter Rabbitson
                   ` (4 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: William Xu @ 2007-10-25 18:20 UTC (permalink / raw)
  To: lartc

Thanks, Peter,

So what you means is that network congestion caused my problem. I test 
the bandwidth
between client1 and server using iperf, SEND and RECV are both 122MB/s.  
I tried
different values for the total bandwidth, and I got the following numbers:

total bandwidth 120MB/s    client1 SEND :10MB/s     RECV :39MB/s
total bandwidth 110MB/s    client1 SEND :16MB/s     RECV :40MB/s
total bandwidth 100MB/s    client1 SEND :30MB/s     RECV :38MB/s
total bandwidth   90MB/s    client1 SEND :39MB/s     RECV :40MB/s
total bandwidth   80MB/s    client1 SEND :39MB/s     RECV :40MB/s
total bandwidth   70MB/s    client1 SEND :40MB/s     RECV :40MB/s
total bandwidth   60MB/s    client1 SEND :40MB/s     RECV :40MB/s

So TC works well as long as total bandwidth is below 90MB/s, which is 
about 70% of the
wise speed. Is it possible that I can use the full bandwidth (122MB/s) 
in my script?

william

Peter Rabbitson wrote:

> William Xu wrote:
>
>> Hi,
>>
>> I have a server and ten clients in a Gigabit network. The server has 
>> 125mbps network bandwidth.
>> I want that the server has 40Mbps bandwidth reserved for client 1 (IP 
>> 192.168.5.141), and the
>> rest bandwidth is for all other clients.
>>
>> <snip>
>>
>> I ran a test in which all 10 clients send/receive packets to/from the 
>> server simultaneously. But
>> Client 1 only got 20mbps bandwidth for sending, and 38mpbs for 
>> receiving. If I limit the rate of
>> both classes 1:1 to 60mbps instead of 125mbps, Client 1 got 39mbps 
>> for sending, and 40mbps for
>> receiving.
>>
>> I am not sure what might cause this. Is it because TC doesn't work 
>> well when network is congested?
>> Or my script is not right?
>>
>
> No network will be able to operate at its theoretical maximum. In the 
> case of a gigabit network you will be lucky to get consistent 120mbps, 
> and it heavily depends on the hardware quality, and the number of 
> switches in between. So what you are doing is oversaturating the link, 
> the ACK packets can not get through, your speed drops due to 
> delays/retransmissions. Perform a test with only two systems sending 
> stuff to each other to see what is the actual bandwidth you can hope 
> for, and use that number instead of 125mbps.


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [LARTC] TC (HTB) doesn't work well when network is congested?
  2007-10-25 16:38 [LARTC] TC (HTB) doesn't work well when network is congested? William Xu
  2007-10-25 17:22 ` Peter Rabbitson
  2007-10-25 18:20 ` William Xu
@ 2007-10-25 21:13 ` Peter Rabbitson
  2007-10-26 13:44 ` William Xu
                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Peter Rabbitson @ 2007-10-25 21:13 UTC (permalink / raw)
  To: lartc

William Xu wrote:
> So TC works well as long as total bandwidth is below 90MB/s, which is 
> about 70% of the
> wise speed. Is it possible that I can use the full bandwidth (122MB/s) 
> in my script?
> 

In order to troubleshoot further more info is needed:

1) execute your script with 120MB/s as limit
2) perform a test transfer for several minutes
3) post back the output of the following commands:
	ip link show
	tc qdisc show
	tc -s -d class show dev ifb0
	tc -s -d class show dev eth2


Peter
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [LARTC] TC (HTB) doesn't work well when network is congested?
  2007-10-25 16:38 [LARTC] TC (HTB) doesn't work well when network is congested? William Xu
                   ` (2 preceding siblings ...)
  2007-10-25 21:13 ` Peter Rabbitson
@ 2007-10-26 13:44 ` William Xu
  2007-10-26 22:39 ` Peter Rabbitson
                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: William Xu @ 2007-10-26 13:44 UTC (permalink / raw)
  To: lartc

Hi Peter, thanks for looking at this.

Here are the information I got after running tests. The client1 got 
7MB/s instead of 40MB/s for SEND,
and 40MB/s for RECV during the test.

Thanks,
william

# ip link show

...
5: eth2: <BROADCAST,MULTICAST,UP,LUP> mtu 9000 qdisc htb qlen 1000
    link/ether 00:e0:ed:04:9f:a2 brd ff:ff:ff:ff:ff:ff
...
12: ifb0: <BROADCAST,NOARP,UP,LUP> mtu 9000 qdisc htb qlen 32
    link/ether f2:f2:77:f9:cf:30 brd ff:ff:ff:ff:ff:ff

#tc qdisc show
qdisc pfifo_fast 0: dev eth0 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
1 1 1 1 1
qdisc ingress ffff: dev eth2 ----------------
qdisc htb 1: dev eth2 r2q 100 default 30 direct_packets_stat 0
qdisc pfifo_fast 0: dev eth3 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
1 1 1 1 1
qdisc pfifo_fast 0: dev eth4 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
1 1 1 1 1
qdisc pfifo_fast 0: dev eth5 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
1 1 1 1 1
qdisc pfifo_fast 0: dev eth6 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
1 1 1 1 1
qdisc pfifo_fast 0: dev eth7 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
1 1 1 1 1
qdisc pfifo_fast 0: dev eth8 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
1 1 1 1 1
qdisc htb 1: dev ifb0 r2q 100 default 30 direct_packets_stat 0

#tc -s -d class show dev ifb0
class htb 1:10 parent 1:1 prio 0 quantum 200000 rate 320000Kbit ceil 
960000Kbit burst 169000b/64 mpu 0b overhead 0b cburst 489000b/64 mpu 0b 
overhead 0b level 0
 Sent 2366125838 bytes 928639 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
 lended: 925807 borrowed: 2832 giants: 0
 tokens: 4224 ctokens: 4075

class htb 1:1 root rate 960000Kbit ceil 960000Kbit burst 489000b/64 mpu 
0b overhead 0b cburst 489000b/64 mpu 0b overhead 0b level 7
 Sent 36927678674 bytes 6132723 pkt (dropped 0, overlimits 0 requeues 0)
 rate 2672bit 1pps backlog 0b 0p requeues 0
 lended: 1131873 borrowed: 0 giants: 0
 tokens: 4074 ctokens: 4074

class htb 1:30 parent 1:1 prio 1 quantum 200000 rate 640000Kbit ceil 
960000Kbit burst 328960b/64 mpu 0b overhead 0b cburst 489000b/64 mpu 0b 
overhead 0b level 0
 Sent 34561552836 bytes 5204084 pkt (dropped 44, overlimits 0 requeues 0)
 rate 528bit 0pps backlog 0b 0p requeues 0
 lended: 4075043 borrowed: 1129041 giants: 0
 tokens: 4108 ctokens: 4074

#tc -s -d class show dev eth2
class htb 1:10 parent 1:1 prio 0 quantum 200000 rate 320000Kbit ceil 
960000Kbit burst 169000b/64 mpu 0b overhead 0b cburst 489000b/64 mpu 0b 
overhead 0b level 0
 Sent 12092794712 bytes 1544210 pkt (dropped 0, overlimits 0 requeues 0)
 rate 56bit 0pps backlog 0b 0p requeues 0
 lended: 1543687 borrowed: 523 giants: 0
 tokens: 4224 ctokens: 4075

class htb 1:1 root rate 960000Kbit ceil 960000Kbit burst 489000b/64 mpu 
0b overhead 0b cburst 489000b/64 mpu 0b overhead 0b level 7
 Sent 36872760531 bytes 7346321 pkt (dropped 0, overlimits 0 requeues 0)
 rate 288bit 0pps backlog 0b 0p requeues 0
 lended: 40477 borrowed: 0 giants: 0
 tokens: 4073 ctokens: 4073

class htb 1:30 parent 1:1 prio 1 quantum 200000 rate 640000Kbit ceil 
960000Kbit burst 328960b/64 mpu 0b overhead 0b cburst 489000b/64 mpu 0b 
overhead 0b level 0
 Sent 24779965819 bytes 5802111 pkt (dropped 0, overlimits 0 requeues 0)
 rate 176bit 0pps backlog 0b 0p requeues 0
 lended: 5762157 borrowed: 39954 giants: 0
 tokens: 4109 ctokens: 4073


Peter Rabbitson wrote:

> William Xu wrote:
>
>> So TC works well as long as total bandwidth is below 90MB/s, which is 
>> about 70% of the
>> wise speed. Is it possible that I can use the full bandwidth 
>> (122MB/s) in my script?
>>
>
> In order to troubleshoot further more info is needed:
>
> 1) execute your script with 120MB/s as limit
> 2) perform a test transfer for several minutes
> 3) post back the output of the following commands:
>     ip link show
>     tc qdisc show
>     tc -s -d class show dev ifb0
>     tc -s -d class show dev eth2
>
>
> Peter


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [LARTC] TC (HTB) doesn't work well when network is congested?
  2007-10-25 16:38 [LARTC] TC (HTB) doesn't work well when network is congested? William Xu
                   ` (3 preceding siblings ...)
  2007-10-26 13:44 ` William Xu
@ 2007-10-26 22:39 ` Peter Rabbitson
  2007-10-31 19:39 ` William Xu
  2007-11-01 12:08 ` Georgi Alexandrov
  6 siblings, 0 replies; 8+ messages in thread
From: Peter Rabbitson @ 2007-10-26 22:39 UTC (permalink / raw)
  To: lartc

William Xu wrote:
> Hi Peter, thanks for looking at this.
> 
> Here are the information I got after running tests. The client1 got 
> 7MB/s instead of 40MB/s for SEND,
> and 40MB/s for RECV during the test.
> 
> Thanks,
> william
> 
> # ip link show
> 
> ...
> 5: eth2: <BROADCAST,MULTICAST,UP,LUP> mtu 9000 qdisc htb qlen 1000
>    link/ether 00:e0:ed:04:9f:a2 brd ff:ff:ff:ff:ff:ff
> ...
> 12: ifb0: <BROADCAST,NOARP,UP,LUP> mtu 9000 qdisc htb qlen 32
>    link/ether f2:f2:77:f9:cf:30 brd ff:ff:ff:ff:ff:ff
> 
> #tc qdisc show
> qdisc pfifo_fast 0: dev eth0 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
> 1 1 1 1 1
> qdisc ingress ffff: dev eth2 ----------------
> qdisc htb 1: dev eth2 r2q 100 default 30 direct_packets_stat 0
> qdisc pfifo_fast 0: dev eth3 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
> 1 1 1 1 1
> qdisc pfifo_fast 0: dev eth4 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
> 1 1 1 1 1
> qdisc pfifo_fast 0: dev eth5 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
> 1 1 1 1 1
> qdisc pfifo_fast 0: dev eth6 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
> 1 1 1 1 1
> qdisc pfifo_fast 0: dev eth7 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
> 1 1 1 1 1
> qdisc pfifo_fast 0: dev eth8 root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 
> 1 1 1 1 1
> qdisc htb 1: dev ifb0 r2q 100 default 30 direct_packets_stat 0
> 
> #tc -s -d class show dev ifb0
> class htb 1:10 parent 1:1 prio 0 quantum 200000 rate 320000Kbit ceil 
> 960000Kbit burst 169000b/64 mpu 0b overhead 0b cburst 489000b/64 mpu 0b 
> overhead 0b level 0
> Sent 2366125838 bytes 928639 pkt (dropped 0, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 0b 0p requeues 0
> lended: 925807 borrowed: 2832 giants: 0
> tokens: 4224 ctokens: 4075
> 
> class htb 1:1 root rate 960000Kbit ceil 960000Kbit burst 489000b/64 mpu 
> 0b overhead 0b cburst 489000b/64 mpu 0b overhead 0b level 7
> Sent 36927678674 bytes 6132723 pkt (dropped 0, overlimits 0 requeues 0)
> rate 2672bit 1pps backlog 0b 0p requeues 0
> lended: 1131873 borrowed: 0 giants: 0
> tokens: 4074 ctokens: 4074
> 
> class htb 1:30 parent 1:1 prio 1 quantum 200000 rate 640000Kbit ceil 
> 960000Kbit burst 328960b/64 mpu 0b overhead 0b cburst 489000b/64 mpu 0b 
> overhead 0b level 0
> Sent 34561552836 bytes 5204084 pkt (dropped 44, overlimits 0 requeues 0)
> rate 528bit 0pps backlog 0b 0p requeues 0
> lended: 4075043 borrowed: 1129041 giants: 0
> tokens: 4108 ctokens: 4074
> 
> #tc -s -d class show dev eth2
> class htb 1:10 parent 1:1 prio 0 quantum 200000 rate 320000Kbit ceil 
> 960000Kbit burst 169000b/64 mpu 0b overhead 0b cburst 489000b/64 mpu 0b 
> overhead 0b level 0
> Sent 12092794712 bytes 1544210 pkt (dropped 0, overlimits 0 requeues 0)
> rate 56bit 0pps backlog 0b 0p requeues 0
> lended: 1543687 borrowed: 523 giants: 0
> tokens: 4224 ctokens: 4075
> 
> class htb 1:1 root rate 960000Kbit ceil 960000Kbit burst 489000b/64 mpu 
> 0b overhead 0b cburst 489000b/64 mpu 0b overhead 0b level 7
> Sent 36872760531 bytes 7346321 pkt (dropped 0, overlimits 0 requeues 0)
> rate 288bit 0pps backlog 0b 0p requeues 0
> lended: 40477 borrowed: 0 giants: 0
> tokens: 4073 ctokens: 4073
> 
> class htb 1:30 parent 1:1 prio 1 quantum 200000 rate 640000Kbit ceil 
> 960000Kbit burst 328960b/64 mpu 0b overhead 0b cburst 489000b/64 mpu 0b 
> overhead 0b level 0
> Sent 24779965819 bytes 5802111 pkt (dropped 0, overlimits 0 requeues 0)
> rate 176bit 0pps backlog 0b 0p requeues 0
> lended: 5762157 borrowed: 39954 giants: 0
> tokens: 4109 ctokens: 4073
> 
> 

The setup looks good. There are only two things that come to mind - you 
have problems with TSO, or your clock is too slow. For the first use 
ethtool -K to disable all 6 offloading parameters. For the second check 
what is the value of CONFIG_HZ in the current kernel config 
(/boot/config-<your-kernel-version>), and if it is less than 1000 this 
might be your problem as well. If none of those help - I am out of 
ideas, hopefully someone else can help you.

Peter
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [LARTC] TC (HTB) doesn't work well when network is congested?
  2007-10-25 16:38 [LARTC] TC (HTB) doesn't work well when network is congested? William Xu
                   ` (4 preceding siblings ...)
  2007-10-26 22:39 ` Peter Rabbitson
@ 2007-10-31 19:39 ` William Xu
  2007-11-01 12:08 ` Georgi Alexandrov
  6 siblings, 0 replies; 8+ messages in thread
From: William Xu @ 2007-10-31 19:39 UTC (permalink / raw)
  To: lartc

Thank you, Peter,

After changing CONFIG_HZ to 1000, TC works much better. I still need to 
limit the total bandwidth
to 110MB/s (about 90% of 125MB/s), but that's normal I guess.

Thanks again,
william
Peter Rabbitson wrote:

> William Xu wrote:
>
>> Hi Peter, thanks for looking at this.
>>
>> Here are the information I got after running tests. The client1 got 
>> 7MB/s instead of 40MB/s for SEND,
>> and 40MB/s for RECV during the test.
>>
>> Thanks,
>> william
>>
>> # ip link show
>>
>> ...
>> 5: eth2: <BROADCAST,MULTICAST,UP,LUP> mtu 9000 qdisc htb qlen 1000
>>    link/ether 00:e0:ed:04:9f:a2 brd ff:ff:ff:ff:ff:ff
>> ...
>> 12: ifb0: <BROADCAST,NOARP,UP,LUP> mtu 9000 qdisc htb qlen 32
>>    link/ether f2:f2:77:f9:cf:30 brd ff:ff:ff:ff:ff:ff
>>
>> #tc qdisc show
>> qdisc pfifo_fast 0: dev eth0 root bands 3 priomap  1 2 2 2 1 2 0 0 1 
>> 1 1 1 1 1 1 1
>> qdisc ingress ffff: dev eth2 ----------------
>> qdisc htb 1: dev eth2 r2q 100 default 30 direct_packets_stat 0
>> qdisc pfifo_fast 0: dev eth3 root bands 3 priomap  1 2 2 2 1 2 0 0 1 
>> 1 1 1 1 1 1 1
>> qdisc pfifo_fast 0: dev eth4 root bands 3 priomap  1 2 2 2 1 2 0 0 1 
>> 1 1 1 1 1 1 1
>> qdisc pfifo_fast 0: dev eth5 root bands 3 priomap  1 2 2 2 1 2 0 0 1 
>> 1 1 1 1 1 1 1
>> qdisc pfifo_fast 0: dev eth6 root bands 3 priomap  1 2 2 2 1 2 0 0 1 
>> 1 1 1 1 1 1 1
>> qdisc pfifo_fast 0: dev eth7 root bands 3 priomap  1 2 2 2 1 2 0 0 1 
>> 1 1 1 1 1 1 1
>> qdisc pfifo_fast 0: dev eth8 root bands 3 priomap  1 2 2 2 1 2 0 0 1 
>> 1 1 1 1 1 1 1
>> qdisc htb 1: dev ifb0 r2q 100 default 30 direct_packets_stat 0
>>
>> #tc -s -d class show dev ifb0
>> class htb 1:10 parent 1:1 prio 0 quantum 200000 rate 320000Kbit ceil 
>> 960000Kbit burst 169000b/64 mpu 0b overhead 0b cburst 489000b/64 mpu 
>> 0b overhead 0b level 0
>> Sent 2366125838 bytes 928639 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 0bit 0pps backlog 0b 0p requeues 0
>> lended: 925807 borrowed: 2832 giants: 0
>> tokens: 4224 ctokens: 4075
>>
>> class htb 1:1 root rate 960000Kbit ceil 960000Kbit burst 489000b/64 
>> mpu 0b overhead 0b cburst 489000b/64 mpu 0b overhead 0b level 7
>> Sent 36927678674 bytes 6132723 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 2672bit 1pps backlog 0b 0p requeues 0
>> lended: 1131873 borrowed: 0 giants: 0
>> tokens: 4074 ctokens: 4074
>>
>> class htb 1:30 parent 1:1 prio 1 quantum 200000 rate 640000Kbit ceil 
>> 960000Kbit burst 328960b/64 mpu 0b overhead 0b cburst 489000b/64 mpu 
>> 0b overhead 0b level 0
>> Sent 34561552836 bytes 5204084 pkt (dropped 44, overlimits 0 requeues 0)
>> rate 528bit 0pps backlog 0b 0p requeues 0
>> lended: 4075043 borrowed: 1129041 giants: 0
>> tokens: 4108 ctokens: 4074
>>
>> #tc -s -d class show dev eth2
>> class htb 1:10 parent 1:1 prio 0 quantum 200000 rate 320000Kbit ceil 
>> 960000Kbit burst 169000b/64 mpu 0b overhead 0b cburst 489000b/64 mpu 
>> 0b overhead 0b level 0
>> Sent 12092794712 bytes 1544210 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 56bit 0pps backlog 0b 0p requeues 0
>> lended: 1543687 borrowed: 523 giants: 0
>> tokens: 4224 ctokens: 4075
>>
>> class htb 1:1 root rate 960000Kbit ceil 960000Kbit burst 489000b/64 
>> mpu 0b overhead 0b cburst 489000b/64 mpu 0b overhead 0b level 7
>> Sent 36872760531 bytes 7346321 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 288bit 0pps backlog 0b 0p requeues 0
>> lended: 40477 borrowed: 0 giants: 0
>> tokens: 4073 ctokens: 4073
>>
>> class htb 1:30 parent 1:1 prio 1 quantum 200000 rate 640000Kbit ceil 
>> 960000Kbit burst 328960b/64 mpu 0b overhead 0b cburst 489000b/64 mpu 
>> 0b overhead 0b level 0
>> Sent 24779965819 bytes 5802111 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 176bit 0pps backlog 0b 0p requeues 0
>> lended: 5762157 borrowed: 39954 giants: 0
>> tokens: 4109 ctokens: 4073
>>
>>
>
> The setup looks good. There are only two things that come to mind - 
> you have problems with TSO, or your clock is too slow. For the first 
> use ethtool -K to disable all 6 offloading parameters. For the second 
> check what is the value of CONFIG_HZ in the current kernel config 
> (/boot/config-<your-kernel-version>), and if it is less than 1000 this 
> might be your problem as well. If none of those help - I am out of 
> ideas, hopefully someone else can help you.
>
> Peter


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [LARTC] TC (HTB) doesn't work well when network is congested?
  2007-10-25 16:38 [LARTC] TC (HTB) doesn't work well when network is congested? William Xu
                   ` (5 preceding siblings ...)
  2007-10-31 19:39 ` William Xu
@ 2007-11-01 12:08 ` Georgi Alexandrov
  6 siblings, 0 replies; 8+ messages in thread
From: Georgi Alexandrov @ 2007-11-01 12:08 UTC (permalink / raw)
  To: lartc


[-- Attachment #1.1: Type: text/plain, Size: 700 bytes --]

William Xu wrote:
> Thank you, Peter,
> 
> After changing CONFIG_HZ to 1000, TC works much better. I still need to
> limit the total bandwidth
> to 110MB/s (about 90% of 125MB/s), but that's normal I guess.
<snip>

Yes, it's normal. Depending on the situation and tests you'll be
allocating about 90% - 99% of the actual link capacity to tc. 100% will
generally never work good as link queues start filling up and you don't
want this. You (tc) should be the bottleneck of the link thus you can
say what and how much goes where.

-- 
regards,
Georgi Alexandrov

key server - pgp.mit.edu :: key id - 0x37B4B3EE
Key fingerprint = E429 BF93 FA67 44E9 B7D4  F89E F990 01C1 37B4 B3EE


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 143 bytes --]

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2007-11-01 12:08 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-10-25 16:38 [LARTC] TC (HTB) doesn't work well when network is congested? William Xu
2007-10-25 17:22 ` Peter Rabbitson
2007-10-25 18:20 ` William Xu
2007-10-25 21:13 ` Peter Rabbitson
2007-10-26 13:44 ` William Xu
2007-10-26 22:39 ` Peter Rabbitson
2007-10-31 19:39 ` William Xu
2007-11-01 12:08 ` Georgi Alexandrov

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.