Re: [PATCH] Revert "sched/fair: Add SIS_UTIL support to select_idle_capacity()"

[Andrea Righi <arighi@nvidia.com>]

On Sat, Nov 14, 2026 at 09:18:30AM +0100, Vincent Guittot wrote:
> On Fri, 13 Nov 2026 at 20:32, Rik van Riel <riel@surriel.com> wrote:
> >
> > On Fri, 2026-11-13 at 07:31 -0800, Usama Arif wrote:
> > >
> > > On a 72-CPU NVIDIA grace arm64 server, small reported capacity
> > > differences
> > > (1013-1024) within the single LLC activate asymmetric selection. Both
> > > the reference and affected kernels were observed with a zero scan
> > > hint. The reference kernel ignored it and still searched the domain.
> > > A latency-sensitive, wakeup-heavy production workload saw its
> > > reported
> > > minute p50 latencies average 129 ms on the reference kernel and 223
> > > ms
> > > on the affected kernel.
> > >
> > > Disabling SIS_UTIL reduced the aggregate involuntary context switch
> > > rate of worker main threads from 71.6k/s to 13.3k/s; re-enabling it
> > > raised the rate to 73.7k/s.
> >
> > Afaik, the asymmetrical selection is supposed
> > to solve a very specific problem: when a task
> > requires X amount of CPU capacity to run, we
> > try to avoid scheduling it on a CPU that has
> > only a fraction of X available.
> >
> > This creates the issue that a task that appears
> > to "need" exactly X, because it's been running
> > on a CPU with full available capacity, ends up
> > not being scheduled at all when the available
> > idle core has X-3% available capacity, because
> > that CPU happens to spend 3% of its time handling
> > interrupts, or running something else.
> 
> The task will be scheduled on this idle cpu/core of (X-3% capacity) if
> no other idle CPU with X capacity is available.
> The selection in select_idle_capacity is:
> - Idle core with enough capacity
> - Idle core with enough capacity but doesn't fullfill uclamp_min
> - Idle core that doesn't fit
> - Idle CPU with enough capacity
> - Idle CPU with enough capacity but doesn't fullfill uclamp_min
> - Idle CPU that doesn't fit

Right, and when the scan budget is exhausted select_idle_capacity()
returns the best idle candidate found so far, even if it doesn't fit.

However, in the zero-budget case, we return before examining any
candidates in the scan domain. Relaxing the capacity-fit rule inside
that scan wouldn't change the outcome, because we never reach that
check.

Maybe Rik's suggestion could instead help with the earlier checks in
select_idle_sibling()? Accepting an idle target, previous or recently
used CPU whose capacity is "close enough" could avoid reaching the
zero-budget bailout.

The tradeoff is that we'd accept a potentially weaker CPU immediately,
without searching for a better one.

Thanks,
-Andrea

> 
> >
> > In that situation, it will be better to run the
> > task on the CPU with 97% capacity right now,
> > than to not run the task until a CPU frees up
> > later.
> >
> > Things like SMT and power-driven hardware
> > scaling complicate the matter further, because
> > we simply cannot see the full amount of capacity
> > that is available on each CPU.
> >
> > That makes me wonder if we should simply accept
> > any CPU that's "close enough to X" when dealing
> > with asymmetrical placement.
> >
> > Say, anything that's above 3/4 of the capacity
> > of X would be considered good enough to run the task.
> >
> > That still prevents us from placing a CPU intensive
> > task on a little core in a big.LITTLE system, or
> > from placing a task on a CPU that spends 80% of its
> > time handling interrupts, but it will allow a task
> > to run right now, when the capacity imbalance is small.
> >
> > Does that make sense, or am I forgetting something
> > important?
> >
> > --
> > All Rights Reversed.