[LARTC] HTB traffic shaping odd effects

[LARTC] HTB traffic shaping odd effects

[Oliver Hookins]

Hi,

I'm trying to perform some (what I consider) basic traffic shaping on 
our network utilising HTB. I have mostly reused the example on the 
lartc.org site:

tc qdisc add dev eth0 root handle 1: htb default 10
tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 50mbit ceil
80mbit burst 19k
tc class add dev eth0 parent 1:1 classid 1:20 htb rate 1mbit ceil
1mbit
tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10
tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip src
X.X.X.X flowid 1:20

So we have a total of 100mbit to be used, the default class 1:10 gets 
50mbit and a ceiling of 80mbit whereas my test host X.X.X.X gets only 
1mbit in any situation. Some rate limiting is definitely happening, but 
I am finding the outbound traffic is limited to 2mbit instead of 1mbit. 
If I change the rate (to say 10mbit) the outbound traffic gets up to 
again twice the rate (in this case 20mbit).

Any thoughts? I have had a look at the tc statistics but it doesn't 
appear as I would expect it to. Class 1:10 shows a lot of dropped 
packets but it is only averaging around 30mbit constantly. On the other 
hand class 1:20 doesn't show any dropped packets. Similarly there are no 
packets marked as overlimit for any class. I occasionally see the tokens 
for 1:20 go negative... everything is quite strange.

Any help would be appreciated.

-- 
Regards,
Oliver Hookins
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] HTB traffic shaping odd effects

[Denis Ovsienko]

> tc qdisc add dev eth0 root handle 1: htb default 10
> tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k
Does the following help?
tc qdisc add dev eth0 root handle 1: htb default 1
tc class add dev eth0 parent 1: classid 1:1 htb rate 81mbit burst 24k

-- 
    DO4-UANIC
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] HTB traffic shaping odd effects

[Oliver Hookins]

Denis Ovsienko wrote:
>>tc qdisc add dev eth0 root handle 1: htb default 10
>>tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k
> 
> Does the following help?
> tc qdisc add dev eth0 root handle 1: htb default 1
> tc class add dev eth0 parent 1: classid 1:1 htb rate 81mbit burst 24k

That seems a bit backward, limiting the total available bandwidth to 
enforce child class limits. Or is that how htb works, that the sum of 
the child classes must add up to no less than the parent class?

In any case we have 100mbit available so I don't want to set it any 
lower than that, if possible.

-- 
Regards,
Oliver Hookins
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] HTB traffic shaping odd effects

[Andy Furniss]

Oliver Hookins wrote:
> Hi,
> 
> I'm trying to perform some (what I consider) basic traffic shaping on 
> our network utilising HTB. I have mostly reused the example on the 
> lartc.org site:
> 
> tc qdisc add dev eth0 root handle 1: htb default 10
> tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k
> tc class add dev eth0 parent 1:1 classid 1:10 htb rate 50mbit ceil
> 80mbit burst 19k

burst 19k will limit you unless your HZ\x1000


> tc class add dev eth0 parent 1:1 classid 1:20 htb rate 1mbit ceil
> 1mbit
> tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10
> tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10
> tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip src
> X.X.X.X flowid 1:20
> 
> So we have a total of 100mbit to be used, the default class 1:10 gets 
> 50mbit and a ceiling of 80mbit whereas my test host X.X.X.X gets only 
> 1mbit in any situation. Some rate limiting is definitely happening, but 
> I am finding the outbound traffic is limited to 2mbit instead of 1mbit. 
> If I change the rate (to say 10mbit) the outbound traffic gets up to 
> again twice the rate (in this case 20mbit).

Not sure can you show output of tc -s -d class ls dev eth0 while it's 
running at 2x speed.

> 
> Any thoughts? I have had a look at the tc statistics but it doesn't 
> appear as I would expect it to. Class 1:10 shows a lot of dropped 
> packets but it is only averaging around 30mbit constantly. On the other 
> hand class 1:20 doesn't show any dropped packets. Similarly there are no 
> packets marked as overlimit for any class.

Apart from the 30mbit that would be normal.

Andy.


  I occasionally see the tokens
> for 1:20 go negative... everything is quite strange.
> 
> Any help would be appreciated.
> 

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] HTB traffic shaping odd effects

[Andy Furniss]

Oliver Hookins wrote:
> Denis Ovsienko wrote:
> 
>>> tc qdisc add dev eth0 root handle 1: htb default 10
>>> tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k
>>
>>
>> Does the following help?
>> tc qdisc add dev eth0 root handle 1: htb default 1
>> tc class add dev eth0 parent 1: classid 1:1 htb rate 81mbit burst 24k
> 
> 
> That seems a bit backward, limiting the total available bandwidth to 
> enforce child class limits. Or is that how htb works, that the sum of 
> the child classes must add up to no less than the parent class?
> 
> In any case we have 100mbit available so I don't want to set it any 
> lower than that, if possible.
> 

You haven't got 100mbit once overheads are accounted for and in this 
test using 81 instead of 100 won't affect your bandwidth as all traffic 
is going to one of the two sub classes. I have IIRC spotted differences 
(to do with quantum and sharing excess) when parent is > than ceils - it 
wasn't the same setup as yours though.

Andy.

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] HTB traffic shaping odd effects

[Oliver Hookins]

Andy Furniss wrote:
> Oliver Hookins wrote:
> 
>> Hi,
>>
>> I'm trying to perform some (what I consider) basic traffic shaping on 
>> our network utilising HTB. I have mostly reused the example on the 
>> lartc.org site:
>>
>> tc qdisc add dev eth0 root handle 1: htb default 10
>> tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k
>> tc class add dev eth0 parent 1:1 classid 1:10 htb rate 50mbit ceil
>> 80mbit burst 19k
> 
> 
> burst 19k will limit you unless your HZ\x1000

Our HZ is 512.

>> tc class add dev eth0 parent 1:1 classid 1:20 htb rate 1mbit ceil
>> 1mbit
>> tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10
>> tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10
>> tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip src
>> X.X.X.X flowid 1:20
>>
>> So we have a total of 100mbit to be used, the default class 1:10 gets 
>> 50mbit and a ceiling of 80mbit whereas my test host X.X.X.X gets only 
>> 1mbit in any situation. Some rate limiting is definitely happening, 
>> but I am finding the outbound traffic is limited to 2mbit instead of 
>> 1mbit. If I change the rate (to say 10mbit) the outbound traffic gets 
>> up to again twice the rate (in this case 20mbit).
> 
> 
> Not sure can you show output of tc -s -d class ls dev eth0 while it's 
> running at 2x speed.

class htb 1:1 root rate 100Mbit ceil 100Mbit burst 24Kb/8 mpu 0b cburst 
132644b/8 mpu 0b level 7
  Sent 741267447736 bytes 735629628 pkts (dropped 0, overlimits 0)
  rate 4316188bps 22688pps
  lended: 489885589 borrowed: 0 giants: 0
  tokens: -78 ctokens: 6677

class htb 1:10 parent 1:1 leaf 10: prio 0 quantum 200000 rate 50Mbit 
ceil 80Mbit burst 19Kb/8 mpu 0b cburst 106440b/8 mpu 0b level 0
  Sent 740240463634 bytes 734522966 pkts (dropped 429910165, overlimits 0)
  rate 4233957bps 22358pps
  lended: 244988819 borrowed: 489885528 giants: 0
  tokens: -293 ctokens: 6636

class htb 1:20 parent 1:1 leaf 20: prio 0 quantum 13107 rate 1Mbit ceil 
1Mbit burst 2Kb/8 mpu 0b cburst 2Kb/8 mpu 0b level 0
  Sent 755354078 bytes 755048 pkts (dropped 0, overlimits 0)
  rate 84198bps 321pps backlog 4p
  lended: 755220 borrowed: 61 giants: 0
  tokens: -21517 ctokens: -21517


-- 
Regards,
Oliver Hookins
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] HTB traffic shaping odd effects

[Andy Furniss]

Oliver Hookins wrote:

>> burst 19k will limit you unless your HZ\x1000
> 
> Our HZ is 512.
> 

I don't know if it makes any difference, but I would have chosen 500 so 
that it was 2ms. The default now is 250 and with 19k burst that fits the 
speed you get really well - with 512 it would be around 70meg. I would 
get rid of the 2 burst settings you use.

>> Not sure can you show output of tc -s -d class ls dev eth0 while it's 
>> running at 2x speed.
> 
> 
> class htb 1:1 root rate 100Mbit ceil 100Mbit burst 24Kb/8 mpu 0b cburst 
> 132644b/8 mpu 0b level 7
>  Sent 741267447736 bytes 735629628 pkts (dropped 0, overlimits 0)
>  rate 4316188bps 22688pps
>  lended: 489885589 borrowed: 0 giants: 0
>  tokens: -78 ctokens: 6677
> 
> class htb 1:10 parent 1:1 leaf 10: prio 0 quantum 200000 rate 50Mbit 
> ceil 80Mbit burst 19Kb/8 mpu 0b cburst 106440b/8 mpu 0b level 0
>  Sent 740240463634 bytes 734522966 pkts (dropped 429910165, overlimits 0)
>  rate 4233957bps 22358pps
>  lended: 244988819 borrowed: 489885528 giants: 0
>  tokens: -293 ctokens: 6636
> 
> class htb 1:20 parent 1:1 leaf 20: prio 0 quantum 13107 rate 1Mbit ceil 
> 1Mbit burst 2Kb/8 mpu 0b cburst 2Kb/8 mpu 0b level 0
>  Sent 755354078 bytes 755048 pkts (dropped 0, overlimits 0)
>  rate 84198bps 321pps backlog 4p
>  lended: 755220 borrowed: 61 giants: 0
>  tokens: -21517 ctokens: -21517

84198 bps - tc means bytes/sec by this, so it's < 1mbit. Htb uses really 
long (100 sec) average for its rate so it probably is giving 1mbit.

Andy.


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc