TC-HTB issue : low throughput

[Jean Tourrilhes <jt@hpl.hp.com>]

	Hi,

	I'm playing with a TC-HTB. I'm noticing that the throughput is
low. As the LARTC mailing list seems to be dead, I would welcome
suggestions...
	The example below may seem contrived, but I reduced it to
a simple testcase to make it easier to reproduce.

System :
------
	Debian Lenny with 2.6.30.9
	Quad Core 2.5 GHz Q9300
	Intel 1Gb/s NIC, e1000e driver

Setup :
-----
# brctl addbr br0
# brctl addif br0 eth5.33
# brctl addif br0 eth6.34
# brctl addif br0 eth7.38

#iptables -A FORWARD -t mangle -s 10.10.32.38 -j CLASSIFY --set-class 1:2

#tc qdisc add dev eth6.34 root handle 1: htb default 3
#tc class add dev eth6.34 parent 1: classid 1:1 htb rate 1000mbit ceil 1000mbit
#tc class add dev eth6.34 parent 1:1 classid 1:2 htb rate 3kbit ceil 1000mbit prio 1
#tc class add dev eth6.34 parent 1:1 classid 1:3 htb rate 3kbit ceil 1000mbit prio 2

Topology :
--------
Four PCs with Linux :
10.10.10.32 on br0
10.10.10.33 connected to eth5.33 (sender)
10.10.10.38 connected to eth7.38 (sender)
10.10.10.34 connected to eth6.34 (receiver)

Behaviour :
---------
	If I set only qdisc 1: and class 1:1, but *NOT* 1:2 and 1:3 :
	Each host independantly : ~935 Mb/s
	Both host together, for 10.10.10.38 : ~527 Mb/s ;
	Both host together, for 10.10.10.33 : ~443 Mb/s

	If I add classes 1:2 and 1:3 :
	Each host independantly : ~170 Mb/s.
	Both host together, for 10.10.10.38 : ~106 Mb/s ;
	Both host together, for 10.10.10.33 : ~135 Mb/s

	So, not only performance did drop significantely, but
prioritisation did not happen as expected.

Weird detail :
------------
	I've noticed that /sbin/tc calculates a very low burst
value. This is due to the content of /proc/net/psched. I'm wondering
if the burst calculation is what causes the issue here.
	However, I tried with "burst 50kb" and saw no difference...

Dump of TC
----------
# tc -s class show dev eth6.34
class htb 1:1 root rate 1000Mbit ceil 1000Mbit burst 1375b cburst 1375b 
 Sent 770653596 bytes 509070 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 82776Kbit 6835pps backlog 0b 0p requeues 0 
 lended: 509026 borrowed: 0 giants: 0
 tokens: 11 ctokens: 11

class htb 1:2 parent 1:1 prio 1 rate 3000bit ceil 1000Mbit burst 1599b cburst 13
75b 
 Sent 369190389 bytes 243876 pkt (dropped 3239, overlimits 0 requeues 0) 
 rate 44781Kbit 3698pps backlog 0b 0p requeues 0 
 lended: 20 borrowed: 243856 giants: 0
 tokens: -1384355 ctokens: 11

class htb 1:3 parent 1:1 prio 2 rate 3000bit ceil 1000Mbit burst 1599b cburst 13
75b 
 Sent 401463207 bytes 265194 pkt (dropped 3005, overlimits 0 requeues 0) 
 rate 37996Kbit 3137pps backlog 0b 0p requeues 0 
 lended: 24 borrowed: 265170 giants: 0
 tokens: -3719389 ctokens: 11

# tc -s qdisc show dev eth6.34
qdisc htb 1: root r2q 10 default 3 direct_packets_stat 0
 Sent 770653852 bytes 509071 pkt (dropped 6244, overlimits 25973 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 


	Thanks in advance for any help...

	Jean