Re: [PATCH RFC 0/2] sched: Introduce idle SMT priority for asymmetric capacity systems

[Shrikanth Hegde <sshegde@linux.ibm.com>]

Hi Mete,

On 10/13/26 6:10 PM, Mete Durlu wrote:
> [..snip..]
> 
>>>> What you describe really fit into the prefered core that Shrikanth
>>>> Hegde has developped
>>>
>>> I agree, but Shrikanth's solution can not bring some of the
>>> behavior changes required. cpu_preferred_mask can help
>>> direct task placement away from low capacity cores but idle
>>> core first approach still places tasks to fully idle low
>>> capacity cores. This patches proposal tries to move tasks
>>> first to the idle SMT siblings.
>>>
>>> Consider the following;
>>> On a system with 2 high capacity and 4 low capacity cpus.
>>> steal governor of vCPU backoff mechanism calculates 2 low capacity
>>> cpus should be taken off from the cpu_preferred_mask to decrease
>>> experienced steal time. cpu_preferred_mask has 2 high capacity
>>> and 2 low capacity cpus.
>>
>>
>> In this scenario, do you still have reasonable steal time or no?
> 
> steal time should be sitting between low and high steal time
> threshold values of the steal governor. (no adjustments)
> 
>>>
>>> Current behavior tires to place tasks to fully idle cores first
>>> even if they are low capacity cores.
>>>
>>>        c0(high) c1(high)  c2(low)  c3(low)  ...
>>> -------------------------------------------
>>> SMT 0 task     task      task     task
>>> SMT 1
>>>
>>>
>>> With the proposal, the fully idle core preference is circumvented
>>> and the most fitting SMT thread is selected.
>>>
>>>        c0(high) c1(high)  c2(low)  c3(low)  ...
>>> -------------------------------------------
>>> SMT 0 task     task
>>> SMT 1 task     task
>>>
>>> This is beneficial because at the time of contention(system pressure)
>>> low capacity cores could perform worse than SMT siblings of busy
>>> high capacity cores, even if there is not enough steal time for
>>> steal governor of Shrikanth's vCPU backoff to mark said low
>>> capacity cores as non-preferred.
>>>
>>
>> I read this as tunable problem of low,high thresholds to adjust
>> preferred_mask same as high capacity cores. No?
> 
> No, because cpu_preferred_mask would contain some low capacity
> cpus as the experienced steal time is not high enough to exclude
> all low capacity cpus from the mask.
> 
> And any new task could be placed on them like below as they should
> not cause an increase in steal time. See below (for 6 tasks);
> (non preferred cpus marked with X)
> 
>         c0(high) c1(high)  c2(low)  c3(low)  c4(low)  c5(low)
> ------------------------------------------------------------
> SMT 0 task     task       task     task          X        X
> SMT 1 task     task                              X        X
> 
> The soft affinity should come in as full utilization of high capacity
> cores(and their SMT siblings) before placing tasks to low capacity
> cores (for 4 tasks);
> 
>         c0(high) c1(high)  c2(low)  c3(low)  c4(low)  c5(low)
> ------------------------------------------------------------
> SMT 0 task     task                              X        X
> SMT 1 task     task                              X        X
> 
>>> Andrea came up with an idea to extend Shrikanth's mechanism further
>>> by adding a soft affinity to a selected set of CPUs. Task placement
>>> would first fully load these CPUs before spilling load onto rest
>>> of the CPUs on cpu_preferred_mask. That would help achieve the
>>> behavior I describe above if the soft affinity covers high
>>> capacity CPUs.
>>>
>>> The discussion is still on going though. If Shrikanth thinks this is
>>> a worthwhile concept then I'd be glad to work on the implementation
>>> and provide the patches.
>>
>> What i don't understand is.
>> - if there is no system pressure to use those low capacity cores, why
>>    not use them if the task util fits its capacity. If the task utilization
>>    doesn't, it won't land on them anyways today right?
> 
> Current task placement looks like this (5 tasks);
> 
>         c0(high) c1(high)  c2(low)  c3(low)  c4(low)  c5(low)
> ------------------------------------------------------------
> SMT 0  task     task      task     task*         X        X
> SMT 1  task     (**)                             X        X
> 
> Even with lowered capacities task don't migrate away from
> fully idle cores, cpu selection prioritizes fully idle
> cores before everything else if i am not mistaken.
> 
> The proposal with this patch series tries to circumvent exactly that
> idle core prioritization and uses SMT threads first, like below;
> 
>         c0(high) c1(high)  c2(low)  c3(low)  c4(low)  c5(low)
> ------------------------------------------------------------
> SMT 0  task     task      task     (**)          X        X
> SMT 1  task     task*                            X        X
> 
> *the task that got placed to SMT1 of core1 instead of fully idle
>   core3
> **where the next task would likely land.
> 
> After high capacity CPUs are fully saturated, remaining low capacity
> CPUs in cpu_preferred_mask can be utilized with idle core priority
> once again as there are no distinction between them.
> Utilizing high capacity cores(or entitled cores...) should lower
> the footprint of the guest and allow less noise from guests.
> Steal time(or contention) is expected first on low capacity cores
> anyway.
> 
> Does this make sense?

The examples does make sense. But if one configures steal governor
settings correctly, then number of low cores within the preferred mask
must be very few when under physical CPU contention.

So I would recommend to see the reality of this in practice with reasonable
thresholds. (Default may not be the best for all configs) You can also keep
the low threshold to very low value to see if that ensure preferred mask is
same as your number of entitled cores. i.e you don't have any low cores in
the preferred mask. I agree steal governor is a reactive mechanism and it may
have oscillations which might allow a few low cores sometimes.

Also, IMHO soft affinity thing is bit tricky to implement and need
to ensure minimal cost for common cases.

PS: There might be significant delays in my responses.