Re: TEO as default governor ?

[Christian Loehle <christian.loehle@arm.com>]

On 3/11/25 18:51, Rafael J. Wysocki wrote:
> On Tue, Mar 11, 2025 at 7:42 PM Daniel Lezcano
> <daniel.lezcano@linaro.org> wrote:
>>
>> On 11/03/2025 19:26, Rafael J. Wysocki wrote:
>>> On Tue, Mar 11, 2025 at 7:00 PM Daniel Lezcano
>>> <daniel.lezcano@linaro.org> wrote:
>>>>
>>>> On 11/03/2025 18:34, Rafael J. Wysocki wrote:
>>>>> On Tue, Mar 11, 2025 at 5:47 PM Christian Loehle
>>>>> <christian.loehle@arm.com> wrote:
>>>>>>
>>>>>> On 3/11/25 16:31, Daniel Lezcano wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I think we can agree the teo governor is better then the menu governor.
>>>>>>>
>>>>>>> Would it make sense to make the teo governor the default governor ?
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> Rafael's position seems to be quite conservative here.
>>>>>> Fact is menu is still the default on many systems.
>>>>>> Even worse, the really bad performance disadvantage when
>>>>>> using menu in an intercept-heavy workload has been fixed by Rafael :)
>>>>>> https://lore.kernel.org/lkml/bc7f915b-8d9f-4e05-9939-8b7ecc078f85@arm.com/
>>>>>>
>>>>>> FWIW I proposed this a while ago:
>>>>>> https://lore.kernel.org/lkml/20240905092645.2885200-3-christian.loehle@arm.com/
>>>>>
>>>>> It will help if one can make a really convincing case for this change
>>>>> (that is, show that menu with the most recent fixes included is really
>>>>> significantly worse on their platform).
>>>>
>>>> For all the platforms I've been testing, the teo governor is always the
>>>> best one.
>>>
>>> Great!  Can you please share any numbers?
>>
>> There are some at a publication doing an evaluation of the irq
>> prediction [1]
>>
>>>> Using the menu governor has also an impact on the user experience as it
>>>> lags on mobile.
>>>
>>> Well, I'm not quite sure what you mean here?
>>
>> For example, the user feels the lag when touching the screen on a mobile
>> or scrolling a document. Changing from menu to teo solves this issue.
>>
>>
>>>> After studying the history of the menu governor few years ago, it
>>>> appeared the menu governor was introduced before the SMP was widely
>>>> used. The strength of the menu governor was the ability to find
>>>> repeating intervals but with he multiplication of the cores, the IPIs
>>>> were introduced which increased the entropy of the busy-idle cycles
>>>> duration, thus making the duration much more random and altering the
>>>> menu governor prediction accuracy.
>>>
>>> While this arguably is the case, menu has also been changed quite a
>>> bit since its introduction.
>>>
>>> What I'm looking for really is a set of numbers showing the difference
>>> and clearly pointing out that teo should be preferred.
>>
>> Ok, let me check if I can find a platform doing energy measurement.
>>
>> I guess x86 is not a good target as the firmware overcomes the kernel
>> decisions right ?
> 
> In many cases, yes, it does.

So do we btw to some extent (arm that is).
The kernel currently has no ultimate and general knowledge if anything
beyond WFI a) has been entered (without observing it's side effects) and
b) when this was done (i.e. how long that request was delayed for).
Even for WFI many platforms have essentially different levels of deepness
the kernel has neither knowledge nor control over (in the general case). [0]
Additionally we have the issue Uffe mentioned of cluster-sleeps, which are
manageable in OSI-mode (like he proposes), but many (I'd say most) platforms
out there are still PC-mode.
The entire cpuidle landscape has become quite nondeterministic I'm afraid.

There should still be workloads where teo performs better, but ultimately
there's also systems+workloads combinations where menu would (now) be
preferred.
(Generally teo still tends to prefer shallower idle states, on mobile that
is quite nice because a) we often are intercept-heavy b) we have quite
efficient WFI and c) for longer idle periods we try to use system suspend
anyway.)

Those are my two cents, without the mentioned menu patches teo was far
ahead in many workloads, now I think it's much more balanced.
I'm happy to revisit this too.

[0]
https://developer.arm.com/documentation/101433/0102/Functional-description/Power-management-/Core-power-modes/Core-dynamic-retention-mode
This has been mentioned previously on LKML to illustrate how WFI != WFI.