Re: Q: cpuidle results in degredation in ethernet performance ?

[Daniel Lezcano <daniel.lezcano@linaro.org>]

On 08/04/2015 05:24 PM, Ran Shalit wrote:
> On Tue, Aug 4, 2015 at 5:28 PM, Daniel Lezcano
> <daniel.lezcano@linaro.org> wrote:
>> On 08/01/2015 09:34 PM, Ran Shalit wrote:
>>>
>>> Hello,
>>>
>>> Maybe if someone here understands cpuidel he can help me...
>>>
>>> I am using omap, with cpuidel and I have some strange behaviour only
>>> when cpuidle is used:
>>> 1.  I test ethernet bandwidth with iperf
>>> 2. With large packet test (16k), I get same bandwidth in version with
>>> cpuidle support as in version without cpuidel supported.
>>> 3. With small packet test (<2000 bytes), I get high degredation in
>>> performance in version with cpuidle. On viewing statistics I see that
>>> the cpu gets into cpu inactive state (C3 state -  not retention yet,
>>> only cpu not active). I guess this is the cuase of the degredation.
>>> With large packets I get no increment in C3 state usage.
>>> 4. I think that menu governer behaviour might explain this, but I
>>> don't understand it deeply enough.
>>>
>>> Can anyone shed a light on this ? Is there anything I can do to overcome
>>> this ?
>>
>>
>> Is an omap3 or omap4 ?
>>
>>
> Hi,
>
> It is omap3530.

Ok, thanks.

I was asking for the omap version because the cpuidle driver is 
different and behaves differently. The core are coupled and the IPI 
enters in the next prediction equation.

What you are observing may be related to the following behavior:

The deeper idle state the cpu is, the longer it takes to wake up.

  * When there are large packets, the packet processing time makes the 
cpu wake up (exit latency) negligible. For example, the test uses 1000 
packets. The time to process a packet is 100us and the exit latency of 
the idle state is 10us.

    (1000 x 100) + (1000 x 10) = 110000 us

  * When there are small packets, the packet processing time is smaller, 
let's say 10us:

    (1000 x 10) + (1000 x 10) = 20000us

In the first case, you have 90% time to process a packet, in the second 
case 50% time to process a packet.

As the iperf program is probably not sending a fixed number of packet 
but counts the maximum number of packet to be sent in a fixed time 
interval, with small packets you spend half of the time to wake up the cpu.

Perhaps the menu governor is doing some bad decisions also but without 
some traces you can't verify this [1]. I am working on another governor 
which could improve the current situation

Other than that, if everything is ok (good idle decision), then you are 
facing a normal behavior, that is save power at the cost of performance 
loss. If you need to add constraints during a specific amount of time, 
you can look at the cpu_dma_latency constraint which will prevent to 
have the cpu to go to deep idle state.

   -- Daniel

[1] 
https://git.linaro.org/people/daniel.lezcano/linux.git/commit/6febff6cdce103fec523be9b66d5021a72d7febf


-- 
  <http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog