QoS weirdness : HTB accuracy

[Julien Vehent <julien@linuxwall.info>]

[-- Attachment #1: Type: text/plain, Size: 1547 bytes --]

Hello folks,

I observe unused bandwidth on my QoS policy that I cannot explain.
Conditions are: I have a HTB tree with 8 classes and a total rate of
768kbits. I use the ATM option so I assume the real rate to be something
close to 675kbits (about 88% of the ethernet rate).

The sum of my 8 rates is exactly 768kbits. Some have ceil values up to
768kbits.

When class 20 "tcp_acks" starts borrowing, TC reduces the total bandwidth
down to 595kbits/S (minus 79kbits/s). And I can't explain why....

The attached graph "tc_htb_weirdness.png" shows the result: there are
'holes' in the sending rate. 

I tried to play with burst sizes, r2q value and hysteresis mode, but the
results are the same.

System is debian squeeze, kernel version is 2.6.26, iproute2 version is
2.6.26 - 07/25/2008.

I have attached two files:
- "tcrules.txt" : the traffic control rules
- "tc_htb_weirdness.png" : the rrdtool graph, resolution is 1 second.

And here: http://jve.linuxwall.info/ressources/code/tc_hole_analysis.html
a sheet with some of the measures values. I used it to calculate the size
of one of the hole. The last table (with green and red cells) shows that,
when class 20 "tcp_acks" starts sending at unixtime 1265496813, there is a
lot of bandwidth left over (last column is all green). During the 95
seconds while class 20 is sending, 3880776 bits could be sent but are not.
That's about 40kbits/s on average. 

Does anybody observess the same behavior? Any logical explanation to this
or is it a bug ?


Julien

[-- Attachment #2: tcrules.txt --]
[-- Type: text/plain, Size: 2619 bytes --]

#root
tc qdisc add dev eth0 root handle 1: htb default 99 r2q 100

#main branch
tc class add dev eth0 parent 1:0 classid 1:1 htb rate 768kbit ceil 768kbit linklayer atm overhead 5

# class 10 "interactive"
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 128kbit ceil 200kbit burst 5k prio 1 linklayer atm overhead 5
tc qdisc add dev eth0 parent 1:10 handle 110: pfifo limit 1000
tc filter add dev eth0 parent 1:0 protocol ip prio 1 handle 10 fw flowid 1:10

# class 20 "tcp_acks"
tc class add dev eth0 parent 1:1 classid 1:20 htb rate 64kbit ceil 768kbit burst 3k prio 2 linklayer atm overhead 5
tc qdisc add dev eth0 parent 1:20 handle 120: pfifo limit 1000
tc filter add dev eth0 parent 1:0 protocol ip prio 2 handle 20 fw flowid 1:20

# class 30 "ssh"
tc class add dev eth0 parent 1:1 classid 1:30 htb rate 64kbit ceil 300kbit burst 2k prio 3 linklayer atm overhead 5
tc qdisc add dev eth0 parent 1:30 handle 130: sfq perturb 10
tc filter add dev eth0 parent 1:0 protocol ip prio 3 handle 30 fw flowid 1:30

# class 40 "main web"
tc class add dev eth0 parent 1:1 classid 1:40 htb rate 256kbit ceil 768kbit burst 20k prio 4 linklayer atm overhead 5

# class 41 "sub web 1"
tc class add dev eth0 parent 1:40 classid 1:41 htb rate 100kbit ceil 768kbit burst 20k prio 4 linklayer atm overhead 5
tc qdisc add dev eth0 parent 1:41 handle 141: sfq perturb 10
tc filter add dev eth0 parent 1:0 protocol ip prio 4 handle 41 fw flowid 1:41

# class 42 "sub web 2"
tc class add dev eth0 parent 1:40 classid 1:42 htb rate 100kbit ceil 768kbit burst 20k prio 4 linklayer atm overhead 5
tc qdisc add dev eth0 parent 1:42 handle 142: sfq perturb 10
tc filter add dev eth0 parent 1:0 protocol ip prio 5 handle 42 fw flowid 1:42

# class 49 "sub web gen"
tc class add dev eth0 parent 1:40 classid 1:49 htb rate 56kbit ceil 768kbit burst 20k prio 4 linklayer atm overhead 5
tc qdisc add dev eth0 parent 1:49 handle 149: sfq perturb 10
tc filter add dev eth0 parent 1:0 protocol ip prio 6 handle 49 fw flowid 1:49

# class 50 "torrent"
tc class add dev eth0 parent 1:1 classid 1:50 htb rate 128kbit ceil 768kbit burst 2k prio 5 linklayer atm overhead 5
tc qdisc add dev eth0 parent 1:50 handle 150: sfq perturb 10
tc filter add dev eth0 parent 1:0 protocol ip prio 7 handle 50 fw flowid 1:50

# class 99 "default"
tc class add dev eth0 parent 1:1 classid 1:99 htb rate 128kbit ceil 768kbit burst 2k prio 5 linklayer atm overhead 5 mtu 1500
tc qdisc add dev eth0 parent 1:99 handle 199: sfq perturb 10
tc filter add dev eth0 parent 1:0 protocol ip prio 99 handle 99 fw flowid 1:99

[-- Attachment #3: tc_htb_weirdness.png --]
[-- Type: image/png, Size: 29681 bytes --]