From: Chen Yu <chen.yu@linux.dev>
To: Andrea Righi <arighi@nvidia.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>,
Rik van Riel <riel@surriel.com>,
Usama Arif <usama.arif@linux.dev>,
peterz@infradead.org, mingo@redhat.com, kprateek.nayak@amd.com,
juri.lelli@redhat.com, dietmar.eggemann@arm.com,
rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
vschneid@redhat.com, samuelcrossley@gmail.com,
linux-kernel@vger.kernel.org, stable@vger.kernel.org,
kas@kernel.org, Arnav Aghav <aghav@meta.com>
Subject: Re: [PATCH] Revert "sched/fair: Add SIS_UTIL support to select_idle_capacity()"
Date: Mon, 16 Nov 2026 21:00:14 +0800 [thread overview]
Message-ID: <avr-XumgYsrnA6vZ@three-body> (raw)
In-Reply-To: <avglp8nnc4u9OCb5@gpd4>
On Sat, Nov 14, 2026 at 10:11:03AM +0100, Andrea Righi wrote:
> Date: Sat, 14 Nov 2026 10:11:03 +0100
> From: Andrea Righi <arighi@nvidia.com>
> To: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: Rik van Riel <riel@surriel.com>, Usama Arif <usama.arif@linux.dev>,
> peterz@infradead.org, mingo@redhat.com, kprateek.nayak@amd.com,
> juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org,
> bsegall@google.com, mgorman@suse.de, vschneid@redhat.com,
> samuelcrossley@gmail.com, linux-kernel@vger.kernel.org,
> stable@vger.kernel.org, kas@kernel.org, Arnav Aghav <aghav@meta.com>
> Subject: Re: [PATCH] Revert "sched/fair: Add SIS_UTIL support to
> select_idle_capacity()"
>
> On Sat, Nov 14, 2026 at 09:18:30AM +0100, Vincent Guittot wrote:
> > On Fri, 13 Nov 2026 at 20:32, Rik van Riel <riel@surriel.com> wrote:
> > >
> > > On Fri, 2026-11-13 at 07:31 -0800, Usama Arif wrote:
> > > >
> > > > On a 72-CPU NVIDIA grace arm64 server, small reported capacity
> > > > differences
> > > > (1013-1024) within the single LLC activate asymmetric selection. Both
> > > > the reference and affected kernels were observed with a zero scan
> > > > hint. The reference kernel ignored it and still searched the domain.
> > > > A latency-sensitive, wakeup-heavy production workload saw its
> > > > reported
> > > > minute p50 latencies average 129 ms on the reference kernel and 223
> > > > ms
> > > > on the affected kernel.
> > > >
> > > > Disabling SIS_UTIL reduced the aggregate involuntary context switch
> > > > rate of worker main threads from 71.6k/s to 13.3k/s; re-enabling it
> > > > raised the rate to 73.7k/s.
> > >
> > > Afaik, the asymmetrical selection is supposed
> > > to solve a very specific problem: when a task
> > > requires X amount of CPU capacity to run, we
> > > try to avoid scheduling it on a CPU that has
> > > only a fraction of X available.
> > >
> > > This creates the issue that a task that appears
> > > to "need" exactly X, because it's been running
> > > on a CPU with full available capacity, ends up
> > > not being scheduled at all when the available
> > > idle core has X-3% available capacity, because
> > > that CPU happens to spend 3% of its time handling
> > > interrupts, or running something else.
> >
> > The task will be scheduled on this idle cpu/core of (X-3% capacity) if
> > no other idle CPU with X capacity is available.
> > The selection in select_idle_capacity is:
> > - Idle core with enough capacity
> > - Idle core with enough capacity but doesn't fullfill uclamp_min
> > - Idle core that doesn't fit
> > - Idle CPU with enough capacity
> > - Idle CPU with enough capacity but doesn't fullfill uclamp_min
> > - Idle CPU that doesn't fit
>
> Right, and when the scan budget is exhausted select_idle_capacity()
> returns the best idle candidate found so far, even if it doesn't fit.
>
> However, in the zero-budget case, we return before examining any
> candidates in the scan domain. Relaxing the capacity-fit rule inside
> that scan wouldn't change the outcome, because we never reach that
> check.
>
> Maybe Rik's suggestion could instead help with the earlier checks in
> select_idle_sibling()? Accepting an idle target, previous or recently
> used CPU whose capacity is "close enough" could avoid reaching the
> zero-budget bailout.
>
> The tradeoff is that we'd accept a potentially weaker CPU immediately,
> without searching for a better one.
>
Do you something similar to cluster wakeup, give the wakeup a second chance to find an idle
CPU in the LLC. I suppose what you suggested is:
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 470ba0dc6719..d1f3b179013e 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9141,6 +9141,24 @@ static inline bool asym_fits_cpu(unsigned long util,
return true;
}
+static int asym_idle_retry(struct task_struct *p, int prev, int target,
+ int recent)
+{
+ if (choose_idle_cpu(target, p))
+ return target;
+
+ if (prev != target && cpus_share_cache(prev, target) &&
+ choose_idle_cpu(prev, p))
+ return prev;
+
+ if (recent != prev && recent != target &&
+ cpus_share_cache(recent, target) &&
+ choose_idle_cpu(recent, p))
+ return recent;
+
+ return target;
+}
+
/*
* Try and locate an idle core/thread in the LLC cache domain.
*/
@@ -9149,7 +9167,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
bool has_idle_core = false;
struct sched_domain *sd;
unsigned long task_util, util_min, util_max;
- int i, recent_used_cpu, prev_aff = -1;
+ int i, recent_used_cpu, recent, prev_aff = -1;
/*
* On asymmetric system, update task utilization because we will check
@@ -9206,6 +9224,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
/* Check a recently used CPU as a potential idle candidate: */
recent_used_cpu = p->recent_used_cpu;
+ recent = recent_used_cpu;
p->recent_used_cpu = prev;
if (recent_used_cpu != prev &&
recent_used_cpu != target &&
@@ -9244,7 +9263,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
target = i;
goto select_smt_priority;
}
- return target;
+ return asym_idle_retry(p, prev, target, recent);
}
}
--
2.43.0
> Thanks,
> -Andrea
>
> >
> > >
> > > In that situation, it will be better to run the
> > > task on the CPU with 97% capacity right now,
> > > than to not run the task until a CPU frees up
> > > later.
> > >
> > > Things like SMT and power-driven hardware
> > > scaling complicate the matter further, because
> > > we simply cannot see the full amount of capacity
> > > that is available on each CPU.
> > >
> > > That makes me wonder if we should simply accept
> > > any CPU that's "close enough to X" when dealing
> > > with asymmetrical placement.
> > >
> > > Say, anything that's above 3/4 of the capacity
> > > of X would be considered good enough to run the task.
> > >
> > > That still prevents us from placing a CPU intensive
> > > task on a little core in a big.LITTLE system, or
> > > from placing a task on a CPU that spends 80% of its
> > > time handling interrupts, but it will allow a task
> > > to run right now, when the capacity imbalance is small.
> > >
> > > Does that make sense, or am I forgetting something
> > > important?
> > >
> > > --
> > > All Rights Reversed.
prev parent reply other threads:[~2026-11-16 12:59 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-13 15:31 [PATCH] Revert "sched/fair: Add SIS_UTIL support to select_idle_capacity()" Usama Arif
2026-11-13 17:28 ` Andrea Righi
2026-11-13 18:26 ` Usama Arif
2026-11-13 21:32 ` Andrea Righi
2026-11-14 9:06 ` Vincent Guittot
2026-11-16 15:37 ` Chen Yu
2026-11-13 19:31 ` Rik van Riel
2026-11-14 8:18 ` Vincent Guittot
2026-11-14 9:11 ` Andrea Righi
2026-11-16 13:00 ` Chen Yu [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=avr-XumgYsrnA6vZ@three-body \
--to=chen.yu@linux.dev \
--cc=aghav@meta.com \
--cc=arighi@nvidia.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=kas@kernel.org \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=riel@surriel.com \
--cc=rostedt@goodmis.org \
--cc=samuelcrossley@gmail.com \
--cc=stable@vger.kernel.org \
--cc=usama.arif@linux.dev \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.