* HTB vs CoDel performance
@ 2012-09-18 9:28 Lin Ming
2012-09-18 9:45 ` Eric Dumazet
0 siblings, 1 reply; 7+ messages in thread
From: Lin Ming @ 2012-09-18 9:28 UTC (permalink / raw)
To: Eric Dumazet; +Cc: networking
Hi,
I'm testing htb performance on a gigabit router running 2.6.32 kernel.
Packet path: PC1 ---> Router LAN port ---> Router WAN port ---> PC2
pfifo_fast: 920Mbps
htb: 750Mbps, ~20% drops compared to pfifo_fast
htb tc commands as below,
# tc qdisc add dev eth10 root handle 20: htb default 1
# tc class add dev eth10 parent 20 classid 20:1 htb prio 2 rate
1024Mbit ceil 1024Mbit burst 1281408b cburst 1281408b
The performance drop seems caused by the complex htb enqueue/dequeue algorithm.
I had a quick look at CoDel code, seems it does not have so complex
data structure as HTB.
I'm going to backport CoDel. Is this a good choice?
Can I gain similar performance as pfifo_fast?
Thanks,
Lin Ming
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: HTB vs CoDel performance
2012-09-18 9:28 HTB vs CoDel performance Lin Ming
@ 2012-09-18 9:45 ` Eric Dumazet
2012-09-18 9:56 ` Lin Ming
0 siblings, 1 reply; 7+ messages in thread
From: Eric Dumazet @ 2012-09-18 9:45 UTC (permalink / raw)
To: Lin Ming; +Cc: networking
On Tue, 2012-09-18 at 17:28 +0800, Lin Ming wrote:
> Hi,
>
> I'm testing htb performance on a gigabit router running 2.6.32 kernel.
> Packet path: PC1 ---> Router LAN port ---> Router WAN port ---> PC2
>
> pfifo_fast: 920Mbps
> htb: 750Mbps, ~20% drops compared to pfifo_fast
>
> htb tc commands as below,
> # tc qdisc add dev eth10 root handle 20: htb default 1
> # tc class add dev eth10 parent 20 classid 20:1 htb prio 2 rate
> 1024Mbit ceil 1024Mbit burst 1281408b cburst 1281408b
>
> The performance drop seems caused by the complex htb enqueue/dequeue algorithm.
>
> I had a quick look at CoDel code, seems it does not have so complex
> data structure as HTB.
> I'm going to backport CoDel. Is this a good choice?
> Can I gain similar performance as pfifo_fast?
codel is quite different than HTB : It has no rate control, so its very
fast. (But it has no prio differentiation as pfifo_fast with its 3
bands)
So what are your exact needs ?
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: HTB vs CoDel performance
2012-09-18 9:45 ` Eric Dumazet
@ 2012-09-18 9:56 ` Lin Ming
2012-09-18 10:15 ` Eric Dumazet
0 siblings, 1 reply; 7+ messages in thread
From: Lin Ming @ 2012-09-18 9:56 UTC (permalink / raw)
To: Eric Dumazet; +Cc: networking
On Tue, Sep 18, 2012 at 5:45 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> On Tue, 2012-09-18 at 17:28 +0800, Lin Ming wrote:
>> Hi,
>>
>> I'm testing htb performance on a gigabit router running 2.6.32 kernel.
>> Packet path: PC1 ---> Router LAN port ---> Router WAN port ---> PC2
>>
>> pfifo_fast: 920Mbps
>> htb: 750Mbps, ~20% drops compared to pfifo_fast
>>
>> htb tc commands as below,
>> # tc qdisc add dev eth10 root handle 20: htb default 1
>> # tc class add dev eth10 parent 20 classid 20:1 htb prio 2 rate
>> 1024Mbit ceil 1024Mbit burst 1281408b cburst 1281408b
>>
>> The performance drop seems caused by the complex htb enqueue/dequeue algorithm.
>>
>> I had a quick look at CoDel code, seems it does not have so complex
>> data structure as HTB.
>> I'm going to backport CoDel. Is this a good choice?
>> Can I gain similar performance as pfifo_fast?
>
> codel is quite different than HTB : It has no rate control, so its very
> fast. (But it has no prio differentiation as pfifo_fast with its 3
> bands)
>
> So what are your exact needs ?
I need traffic priority/traffic shaping/rate control ... actually all
QoS features on the router.
And if I just set the rate to gigabit(no other settings), for example,
# tc qdisc add dev eth10 root handle 20: htb default 1
# tc class add dev eth10 parent 20 classid 20:1 htb prio 2 rate
1024Mbit ceil 1024Mbit burst 1281408b cburst 1281408b
it should gain similar performance as pfifo_fast.
codel has no rate control. So seems I have to find way to optimize htb?
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: HTB vs CoDel performance
2012-09-18 9:56 ` Lin Ming
@ 2012-09-18 10:15 ` Eric Dumazet
2012-09-19 1:26 ` Lin Ming
0 siblings, 1 reply; 7+ messages in thread
From: Eric Dumazet @ 2012-09-18 10:15 UTC (permalink / raw)
To: Lin Ming; +Cc: networking
On Tue, 2012-09-18 at 17:56 +0800, Lin Ming wrote:
> I need traffic priority/traffic shaping/rate control ... actually all
> QoS features on the router.
> And if I just set the rate to gigabit(no other settings), for example,
>
> # tc qdisc add dev eth10 root handle 20: htb default 1
> # tc class add dev eth10 parent 20 classid 20:1 htb prio 2 rate
> 1024Mbit ceil 1024Mbit burst 1281408b cburst 1281408b
>
> it should gain similar performance as pfifo_fast.
>
> codel has no rate control. So seems I have to find way to optimize htb?
Are you really cpu limited ? You might hit some clocks artifacts.
rate limiting to 1Gbps probably need high resolution timers.
HTB is not the only way to rate limit.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: HTB vs CoDel performance
2012-09-18 10:15 ` Eric Dumazet
@ 2012-09-19 1:26 ` Lin Ming
2012-09-19 17:00 ` Rick Jones
0 siblings, 1 reply; 7+ messages in thread
From: Lin Ming @ 2012-09-19 1:26 UTC (permalink / raw)
To: Eric Dumazet; +Cc: networking
On Tue, Sep 18, 2012 at 6:15 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> On Tue, 2012-09-18 at 17:56 +0800, Lin Ming wrote:
>
>> I need traffic priority/traffic shaping/rate control ... actually all
>> QoS features on the router.
>> And if I just set the rate to gigabit(no other settings), for example,
>>
>> # tc qdisc add dev eth10 root handle 20: htb default 1
>> # tc class add dev eth10 parent 20 classid 20:1 htb prio 2 rate
>> 1024Mbit ceil 1024Mbit burst 1281408b cburst 1281408b
>>
>> it should gain similar performance as pfifo_fast.
>>
>> codel has no rate control. So seems I have to find way to optimize htb?
>
> Are you really cpu limited ? You might hit some clocks artifacts.
Did you mean the cpu speed? It's an ARMv5 processor.
BogoMIPS : 1196.03
>
> rate limiting to 1Gbps probably need high resolution timers.
High resolution timer is enabled.
>
> HTB is not the only way to rate limit.
I'll try other classful qdiscs.
Thanks.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: HTB vs CoDel performance
2012-09-19 1:26 ` Lin Ming
@ 2012-09-19 17:00 ` Rick Jones
2012-09-21 7:23 ` Lin Ming
0 siblings, 1 reply; 7+ messages in thread
From: Rick Jones @ 2012-09-19 17:00 UTC (permalink / raw)
To: Lin Ming; +Cc: Eric Dumazet, networking
On 09/18/2012 06:26 PM, Lin Ming wrote:
> On Tue, Sep 18, 2012 at 6:15 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>> Are you really cpu limited ? You might hit some clocks artifacts.
>
> Did you mean the cpu speed? It's an ARMv5 processor.
> BogoMIPS : 1196.03
At the risk of typing into Eric's keyboard, he was asking if you were
saturating the CPU - was it getting to 100% utilization, that sort of thing.
>> rate limiting to 1Gbps probably need high resolution timers.
>
> High resolution timer is enabled.
When you are running your tests, what sort of CPU utilization do you see
on the CPU of your router with HTB on vs off. Some "quick and dirty"
netperf tests on an Centrino-based laptop suggested that as an end
system at least, HTB at 1 GbE (using your tc commands) increases service
demand (what netperf calculates as CPU consumed per unit of work
performed) ~15% for bulk transfer (netperf TCP_STREAM) and ~18% for
small packet request/response (TCP_RR).
rick jones
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: HTB vs CoDel performance
2012-09-19 17:00 ` Rick Jones
@ 2012-09-21 7:23 ` Lin Ming
0 siblings, 0 replies; 7+ messages in thread
From: Lin Ming @ 2012-09-21 7:23 UTC (permalink / raw)
To: Rick Jones; +Cc: Eric Dumazet, networking
On Thu, Sep 20, 2012 at 1:00 AM, Rick Jones <rick.jones2@hp.com> wrote:
> On 09/18/2012 06:26 PM, Lin Ming wrote:
>>
>> On Tue, Sep 18, 2012 at 6:15 PM, Eric Dumazet <eric.dumazet@gmail.com>
>> wrote:
>>>
>>> Are you really cpu limited ? You might hit some clocks artifacts.
>>
>>
>> Did you mean the cpu speed? It's an ARMv5 processor.
>> BogoMIPS : 1196.03
>
>
> At the risk of typing into Eric's keyboard, he was asking if you were
> saturating the CPU - was it getting to 100% utilization, that sort of thing.
Yes, ~98% utilization.
>
>
>>> rate limiting to 1Gbps probably need high resolution timers.
>>
>>
>> High resolution timer is enabled.
>
>
> When you are running your tests, what sort of CPU utilization do you see on
> the CPU of your router with HTB on vs off. Some "quick and dirty" netperf
HTB off: ~85%
HTB on: ~98%
> tests on an Centrino-based laptop suggested that as an end system at least,
> HTB at 1 GbE (using your tc commands) increases service demand (what netperf
> calculates as CPU consumed per unit of work performed) ~15% for bulk
> transfer (netperf TCP_STREAM) and ~18% for small packet request/response
> (TCP_RR).
>
> rick jones
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2012-09-21 7:24 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-09-18 9:28 HTB vs CoDel performance Lin Ming
2012-09-18 9:45 ` Eric Dumazet
2012-09-18 9:56 ` Lin Ming
2012-09-18 10:15 ` Eric Dumazet
2012-09-19 1:26 ` Lin Ming
2012-09-19 17:00 ` Rick Jones
2012-09-21 7:23 ` Lin Ming
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).