Re: [PATCH] sched/fair: Stabilize idle SMT core selection with asym-capacity

[Christian Loehle <christian.loehle@arm.com>]

On 7/3/26 10:40, Andrea Righi wrote:
> Hi Prateek,
> 
> On Fri, Jul 03, 2026 at 11:21:57AM +0530, K Prateek Nayak wrote:
>> Hello Andrea,
>>
>> On 6/30/2026 8:57 PM, Andrea Righi wrote:
>>> select_idle_capacity() scans all logical CPUs also when it is looking
>>> for a fully idle SMT core. Two concurrent wakeups can therefore observe
>>> the same core as idle, encounter different siblings first, and place one
>>> task on each sibling while another core remains unused.
>>>
>>> Make every logical CPU of a selected idle core resolve to the same
>>> stable CPU representative within the scan's existing affinity and
>>> scheduling-domain mask. If the first task is enqueued before the next
>>> scan examines the core, that scan rejects the now-busy core. If both
>>> scans observe the core as idle, they select the same runqueue even if
>>> the first enqueue becomes visible before the second scan finishes,
>>> exposing the imbalance to the load balancer.
>>>
>>> The symmetric CPU idle selection path is subject to the same race, but
>>> normally returns as soon as select_idle_core() finds a fully idle core,
>>> reducing the conflict window. The per-CPU capacity scan can retain an
>>> idle-core candidate while evaluating other CPUs, giving concurrent
>>> wakeups more opportunity to select different siblings of the same SMT
>>> core. Therefore, limit the normalization to the asym-capacity path,
>>> where this behavior has a measurable impact.
>>>
>>> On NVIDIA Vera Rubin (arm64, 176 CPUs/88 cores per NUMA node), a
>>> CPU-intensive NVPL SGEMM workload restricted to 88 threads (one per
>>> core) showed a consistent 23% increase in mean throughput across
>>> multiple runs.
>>
>> Interesting! This reads like active balance across cores is not aggressive
>> enough for this workload and, as a result, stacking somehow helps.
>>
>> I would have expected balance within the core would trigger first and that
>> would just lead to the same scenario as both sibling sibling busy but I
>> guess there is a higher order effect of stacking.
> 
> I think the key here is that temporary runqueue stacking is preferable to
> consuming both SMT siblings when fully-idle SMT cores are available, more than
> having benfits from the stacking itself.
> 
I'm curious now, as a not at all SMT expert, this is super counterintuitive to me,
am I missing something? How does this happen?
The SMT-switch should be way less overhead than the task context-switch, no?