* Re: [PATCH v4] sched/fair: Prefer fully idle cores for NOHZ balancing
2026-08-04 15:13 [PATCH v4] sched/fair: Prefer fully idle cores for NOHZ balancing Andrea Righi
@ 2026-08-04 15:19 ` Shrikanth Hegde
2026-08-04 18:09 ` Andrea Righi
2026-08-05 10:31 ` Vincent Guittot
2026-08-06 10:26 ` Shrikanth Hegde
2 siblings, 1 reply; 7+ messages in thread
From: Shrikanth Hegde @ 2026-08-04 15:19 UTC (permalink / raw)
To: Andrea Righi, Ingo Molnar, Peter Zijlstra, Juri Lelli,
Vincent Guittot
Cc: Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
Valentin Schneider, K Prateek Nayak, Christian Loehle, Phil Auld,
Mete Durlu, linux-kernel
Hi Andrea,
On 8/4/26 8:43 PM, Andrea Righi wrote:
> find_new_ilb() selects the first idle housekeeping CPU without
> considering whether another thread is running on the same physical core.
> On an SMT system, the idle load balancer can therefore activate both
> siblings even when another housekeeping CPU has an entirely idle core.
>
> On most SMT systems, this is not problematic because the idle load
> balancer is a short-lived activity and the transient wakeup of a sibling
> has negligible performance impact.
>
> However, this can be particularly costly on NVIDIA Olympus cores used in
> Vera. Briefly activating an otherwise idle sibling can reduce the
> performance available to the other sibling and this effect does not
> necessarily end once the activated sibling becomes idle: after the ILB
> finishes and its CPU enters WFI, full single-thread performance is
> restored only after the sibling has remained idle for a qualification
> interval (10 Ki cycles on the tested Vera system). Repeated short
> sibling wakeups can therefore sustain the interference even with little
> actual overlap.
Is this hardware cost has been accounted in things like cpufreq or cpuidle?
or have you set it already to max performance.
>
> Prevent this by preferring an idle housekeeping CPU whose entire SMT
> core is idle. Retain the first idle CPU as a fallback when no fully idle
> core is available, so NOHZ balancing continues to make forward progress.
> Once a partially busy core has been examined, skip its remaining SMT
> siblings to avoid repeating the core-idle check on wide SMT systems.
>
Just Curious, making nohz_full=<except first core> yeilds similar numbers?
> Tests performed using an ad hoc GEMM benchmark running one CPU-intensive
> task per SMT core within its CPU affinity mask improved from
> approximately 6.2 TFLOP/s to 9.4 TFLOP/s.
>
> Note that this preference may wake a fully idle physical core instead of
> using an idle sibling of an active core, potentially increasing ILB
> wakeup latency or energy consumption on some architectures. It may also
> scan additional CPUs before selecting the one to run the ILB. The
> selection falls back to the first idle CPU when no fully idle SMT core
> is available. Non-SMT systems continue to select the first idle
> housekeeping CPU.
>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: K Prateek Nayak <kprateek.nayak@amd.com>
> Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
> Signed-off-by: Andrea Righi <arighi@nvidia.com>
> Reviewed-by: Mete Durlu <meted@linux.ibm.com>
> ---
> Changes in v4:
> - Remove redundant this_cpu check (Prateek Nayak, Vincent Guittot)
> - Link to v3: https://lore.kernel.org/all/20260731191957.3199642-1-arighi@nvidia.com/
>
> Changes in v3:
> - After finding an idle fallback, skip all siblings when a busy CPU is
> encountered, avoiding per-CPU traversal of known-busy cores (Mete Durlu)
> - Link to v2: https://lore.kernel.org/all/20260729163225.1987068-1-arighi@nvidia.com/
>
> Changes in v2:
> - Avoid repeated is_core_idle() checks on wide SMT systems by pruning
> the remaining siblings of a partially busy core (Prateek Nayak)
> - Link to v1: https://lore.kernel.org/r/20260728214442.1648483-1-arighi@nvidia.com/
>
> kernel/sched/fair.c | 55 ++++++++++++++++++++++++++++++++++++---------
> 1 file changed, 44 insertions(+), 11 deletions(-)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 37001c63452e5..9a975a684b487 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -13964,29 +13964,62 @@ static inline int on_null_domain(struct rq *rq)
> */
> static inline int find_new_ilb(void)
> {
> - int this_cpu = smp_processor_id();
> - const struct cpumask *hk_mask;
> - int ilb_cpu;
> + struct cpumask *ilb_cpus;
> + int ilb_cpu, fallback = -1;
> +
> + lockdep_assert_irqs_disabled();
> +
> + /*
> + * Reuse the per-CPU select_rq_mask, which is protected from concurrent
> + * use on this CPU by having interrupts disabled.
> + */
> + ilb_cpus = this_cpu_cpumask_var_ptr(select_rq_mask);
> + cpumask_and(ilb_cpus, nohz.idle_cpus_mask,
> + housekeeping_cpumask(HK_TYPE_KERNEL_NOISE));
> +
> + for_each_cpu(ilb_cpu, ilb_cpus) {
> + if (!idle_cpu(ilb_cpu)) {
> + /*
> + * Once an idle fallback exists, a busy CPU proves that
> + * this core cannot be fully idle. Skip its siblings.
> + */
> + if (sched_smt_active() && fallback >= 0)
> + cpumask_andnot(ilb_cpus, ilb_cpus, cpu_smt_mask(ilb_cpu));
> + continue;
> + }
>
> - hk_mask = housekeeping_cpumask(HK_TYPE_KERNEL_NOISE);
> + /*
> + * Running the idle load balancer on an idle sibling of a busy
> + * SMT core can reduce the capacity available to its sibling. Prefer
> + * a CPU whose entire core is idle, but retain the first idle CPU as
> + * a fallback so idle balancing can still make progress when no fully
> + * idle core exists.
> + */
> + if (sched_smt_active() && !is_core_idle(ilb_cpu)) {
> + if (fallback < 0)
> + fallback = ilb_cpu;
>
> - for_each_cpu_and(ilb_cpu, nohz.idle_cpus_mask, hk_mask) {
> - if (ilb_cpu == this_cpu)
> + /*
> + * The core is not idle, so there is no need to check
> + * any of its other SMT siblings.
> + */
> + cpumask_andnot(ilb_cpus, ilb_cpus,
> + cpu_smt_mask(ilb_cpu));
> continue;
> + }
>
> - if (idle_cpu(ilb_cpu))
> - return ilb_cpu;
> + return ilb_cpu;
> }
>
> - return -1;
> + return fallback;
> }
>
> /*
> * Kick a CPU to do the NOHZ balancing, if it is time for it, via a cross-CPU
> * SMP function call (IPI).
> *
> - * We pick the first idle CPU in the HK_TYPE_KERNEL_NOISE housekeeping set
> - * (if there is one).
> + * Prefer a CPU on a fully idle core in the HK_TYPE_KERNEL_NOISE housekeeping
> + * set. Fall back to the first idle CPU when no fully idle core exists.
> */
> static void kick_ilb(unsigned int flags)
> {
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v4] sched/fair: Prefer fully idle cores for NOHZ balancing
2026-08-04 15:19 ` Shrikanth Hegde
@ 2026-08-04 18:09 ` Andrea Righi
2026-08-04 21:18 ` Shrikanth Hegde
0 siblings, 1 reply; 7+ messages in thread
From: Andrea Righi @ 2026-08-04 18:09 UTC (permalink / raw)
To: Shrikanth Hegde
Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
Valentin Schneider, K Prateek Nayak, Christian Loehle, Phil Auld,
Mete Durlu, linux-kernel
Hi Shrikanth,
On Tue, Aug 04, 2026 at 08:49:01PM +0530, Shrikanth Hegde wrote:
> Hi Andrea,
>
> On 8/4/26 8:43 PM, Andrea Righi wrote:
> > find_new_ilb() selects the first idle housekeeping CPU without
> > considering whether another thread is running on the same physical core.
> > On an SMT system, the idle load balancer can therefore activate both
> > siblings even when another housekeeping CPU has an entirely idle core.
> >
> > On most SMT systems, this is not problematic because the idle load
> > balancer is a short-lived activity and the transient wakeup of a sibling
> > has negligible performance impact.
> >
> > However, this can be particularly costly on NVIDIA Olympus cores used in
> > Vera. Briefly activating an otherwise idle sibling can reduce the
> > performance available to the other sibling and this effect does not
> > necessarily end once the activated sibling becomes idle: after the ILB
> > finishes and its CPU enters WFI, full single-thread performance is
> > restored only after the sibling has remained idle for a qualification
> > interval (10 Ki cycles on the tested Vera system). Repeated short
> > sibling wakeups can therefore sustain the interference even with little
> > actual overlap.
>
> Is this hardware cost has been accounted in things like cpufreq or cpuidle?
> or have you set it already to max performance.
All the CPUs were using the cppc_cpufreq performance governor during the tests.
The scaling minimum frequency was between 95.5% and 97% of the maximum
frequency, depending on the CPU. The CPUs were not strictly pinned to their
maximum frequency, but the performance governor was in use.
The acpi_idle driver was enabled, with both LPI-0 and LPI-1 available. I didn't
disable the idle states.
>
> >
> > Prevent this by preferring an idle housekeeping CPU whose entire SMT
> > core is idle. Retain the first idle CPU as a fallback when no fully idle
> > core is available, so NOHZ balancing continues to make forward progress.
> > Once a partially busy core has been examined, skip its remaining SMT
> > siblings to avoid repeating the core-idle check on wide SMT systems.
> >
>
> Just Curious, making nohz_full=<except first core> yeilds similar numbers?
Yes, provided that the housekeeping core is also excluded from the workload.
I tested this booting with:
nohz_full=1-175,177-351
This leaves CPUs 0 and 176 for housekeeping. I excluded that core from the
workload and ran 87 OpenMP tasks on the remaining node-0 cores (1 task per core,
excluding the housekeeping one):
$ env OMP_NUM_THREADS=87 \
OMP_DYNAMIC=false \
OPENBLAS_LOOPS=10 \
OPENBLAS_PARAM_M=16384 \
OPENBLAS_PARAM_N=16384 \
OPENBLAS_PARAM_K=16384 \
numactl -C 1-87,177-263 --membind=0 \
./benchmark/sgemm.goto 1 1 1
Results:
unpatched : 5.246 TFLOP/s
unpatched+nohz_full : 6.953 TFLOP/s
patched : 6.861 TFLOP/s
patched+nohz_full : 6.984 TFLOP/s
So, nohz_full seems to prevent the problematic ILB wakeups and can produce
similar results for this CPU-bound workload. However, it shouldn't be considered
a sobstiute for the ILB fix, since it requires explicit partitioning and
reserved housekeeping CPUs. Full dyntick also enables context tracking on the
isolated CPUs, adding kernel entry/exit overhead.
Thanks,
-Andrea
>
> > Tests performed using an ad hoc GEMM benchmark running one CPU-intensive
> > task per SMT core within its CPU affinity mask improved from
> > approximately 6.2 TFLOP/s to 9.4 TFLOP/s.
> >
> > Note that this preference may wake a fully idle physical core instead of
> > using an idle sibling of an active core, potentially increasing ILB
> > wakeup latency or energy consumption on some architectures. It may also
> > scan additional CPUs before selecting the one to run the ILB. The
> > selection falls back to the first idle CPU when no fully idle SMT core
> > is available. Non-SMT systems continue to select the first idle
> > housekeeping CPU.
> >
> > Cc: Vincent Guittot <vincent.guittot@linaro.org>
> > Cc: K Prateek Nayak <kprateek.nayak@amd.com>
> > Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
> > Signed-off-by: Andrea Righi <arighi@nvidia.com>
> > Reviewed-by: Mete Durlu <meted@linux.ibm.com>
> > ---
> > Changes in v4:
> > - Remove redundant this_cpu check (Prateek Nayak, Vincent Guittot)
> > - Link to v3: https://lore.kernel.org/all/20260731191957.3199642-1-arighi@nvidia.com/
> >
> > Changes in v3:
> > - After finding an idle fallback, skip all siblings when a busy CPU is
> > encountered, avoiding per-CPU traversal of known-busy cores (Mete Durlu)
> > - Link to v2: https://lore.kernel.org/all/20260729163225.1987068-1-arighi@nvidia.com/
> >
> > Changes in v2:
> > - Avoid repeated is_core_idle() checks on wide SMT systems by pruning
> > the remaining siblings of a partially busy core (Prateek Nayak)
> > - Link to v1: https://lore.kernel.org/r/20260728214442.1648483-1-arighi@nvidia.com/
> >
> > kernel/sched/fair.c | 55 ++++++++++++++++++++++++++++++++++++---------
> > 1 file changed, 44 insertions(+), 11 deletions(-)
> >
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index 37001c63452e5..9a975a684b487 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -13964,29 +13964,62 @@ static inline int on_null_domain(struct rq *rq)
> > */
> > static inline int find_new_ilb(void)
> > {
> > - int this_cpu = smp_processor_id();
> > - const struct cpumask *hk_mask;
> > - int ilb_cpu;
> > + struct cpumask *ilb_cpus;
> > + int ilb_cpu, fallback = -1;
> > +
> > + lockdep_assert_irqs_disabled();
> > +
> > + /*
> > + * Reuse the per-CPU select_rq_mask, which is protected from concurrent
> > + * use on this CPU by having interrupts disabled.
> > + */
> > + ilb_cpus = this_cpu_cpumask_var_ptr(select_rq_mask);
> > + cpumask_and(ilb_cpus, nohz.idle_cpus_mask,
> > + housekeeping_cpumask(HK_TYPE_KERNEL_NOISE));
> > +
> > + for_each_cpu(ilb_cpu, ilb_cpus) {
> > + if (!idle_cpu(ilb_cpu)) {
> > + /*
> > + * Once an idle fallback exists, a busy CPU proves that
> > + * this core cannot be fully idle. Skip its siblings.
> > + */
> > + if (sched_smt_active() && fallback >= 0)
> > + cpumask_andnot(ilb_cpus, ilb_cpus, cpu_smt_mask(ilb_cpu));
> > + continue;
> > + }
> > - hk_mask = housekeeping_cpumask(HK_TYPE_KERNEL_NOISE);
> > + /*
> > + * Running the idle load balancer on an idle sibling of a busy
> > + * SMT core can reduce the capacity available to its sibling. Prefer
> > + * a CPU whose entire core is idle, but retain the first idle CPU as
> > + * a fallback so idle balancing can still make progress when no fully
> > + * idle core exists.
> > + */
> > + if (sched_smt_active() && !is_core_idle(ilb_cpu)) {
> > + if (fallback < 0)
> > + fallback = ilb_cpu;
> > - for_each_cpu_and(ilb_cpu, nohz.idle_cpus_mask, hk_mask) {
> > - if (ilb_cpu == this_cpu)
> > + /*
> > + * The core is not idle, so there is no need to check
> > + * any of its other SMT siblings.
> > + */
> > + cpumask_andnot(ilb_cpus, ilb_cpus,
> > + cpu_smt_mask(ilb_cpu));
> > continue;
> > + }
> > - if (idle_cpu(ilb_cpu))
> > - return ilb_cpu;
> > + return ilb_cpu;
> > }
> > - return -1;
> > + return fallback;
> > }
> > /*
> > * Kick a CPU to do the NOHZ balancing, if it is time for it, via a cross-CPU
> > * SMP function call (IPI).
> > *
> > - * We pick the first idle CPU in the HK_TYPE_KERNEL_NOISE housekeeping set
> > - * (if there is one).
> > + * Prefer a CPU on a fully idle core in the HK_TYPE_KERNEL_NOISE housekeeping
> > + * set. Fall back to the first idle CPU when no fully idle core exists.
> > */
> > static void kick_ilb(unsigned int flags)
> > {
>
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v4] sched/fair: Prefer fully idle cores for NOHZ balancing
2026-08-04 18:09 ` Andrea Righi
@ 2026-08-04 21:18 ` Shrikanth Hegde
0 siblings, 0 replies; 7+ messages in thread
From: Shrikanth Hegde @ 2026-08-04 21:18 UTC (permalink / raw)
To: Andrea Righi
Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
Valentin Schneider, K Prateek Nayak, Christian Loehle, Phil Auld,
Mete Durlu, linux-kernel
On 8/4/26 11:39 PM, Andrea Righi wrote:
> Hi Shrikanth,
>
>> Just Curious, making nohz_full=<except first core> yeilds similar numbers?
>
> Yes, provided that the housekeeping core is also excluded from the workload.
>
> I tested this booting with:
>
> nohz_full=1-175,177-351
>
> This leaves CPUs 0 and 176 for housekeeping. I excluded that core from the
> workload and ran 87 OpenMP tasks on the remaining node-0 cores (1 task per core,
> excluding the housekeeping one):
>
> $ env OMP_NUM_THREADS=87 \
> OMP_DYNAMIC=false \
> OPENBLAS_LOOPS=10 \
> OPENBLAS_PARAM_M=16384 \
> OPENBLAS_PARAM_N=16384 \
> OPENBLAS_PARAM_K=16384 \
> numactl -C 1-87,177-263 --membind=0 \
> ./benchmark/sgemm.goto 1 1 1
>
> Results:
>
> unpatched : 5.246 TFLOP/s
> unpatched+nohz_full : 6.953 TFLOP/s
> patched : 6.861 TFLOP/s
> patched+nohz_full : 6.984 TFLOP/s
>
> So, nohz_full seems to prevent the problematic ILB wakeups and can produce
> similar results for this CPU-bound workload. However, it shouldn't be considered
> a sobstiute for the ILB fix, since it requires explicit partitioning and
> reserved housekeeping CPUs. Full dyntick also enables context tracking on the
> isolated CPUs, adding kernel entry/exit overhead.
>
That's good to know. Thanks for running them.
I put "just curious", knowing nohz_full is not a solution.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v4] sched/fair: Prefer fully idle cores for NOHZ balancing
2026-08-04 15:13 [PATCH v4] sched/fair: Prefer fully idle cores for NOHZ balancing Andrea Righi
2026-08-04 15:19 ` Shrikanth Hegde
@ 2026-08-05 10:31 ` Vincent Guittot
2026-08-06 10:26 ` Shrikanth Hegde
2 siblings, 0 replies; 7+ messages in thread
From: Vincent Guittot @ 2026-08-05 10:31 UTC (permalink / raw)
To: Andrea Righi
Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Dietmar Eggemann,
Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider,
K Prateek Nayak, Christian Loehle, Shrikanth Hegde, Phil Auld,
Mete Durlu, linux-kernel
On Tue, 4 Aug 2026 at 17:13, Andrea Righi <arighi@nvidia.com> wrote:
>
> find_new_ilb() selects the first idle housekeeping CPU without
> considering whether another thread is running on the same physical core.
> On an SMT system, the idle load balancer can therefore activate both
> siblings even when another housekeeping CPU has an entirely idle core.
>
> On most SMT systems, this is not problematic because the idle load
> balancer is a short-lived activity and the transient wakeup of a sibling
> has negligible performance impact.
>
> However, this can be particularly costly on NVIDIA Olympus cores used in
> Vera. Briefly activating an otherwise idle sibling can reduce the
> performance available to the other sibling and this effect does not
> necessarily end once the activated sibling becomes idle: after the ILB
> finishes and its CPU enters WFI, full single-thread performance is
> restored only after the sibling has remained idle for a qualification
> interval (10 Ki cycles on the tested Vera system). Repeated short
> sibling wakeups can therefore sustain the interference even with little
> actual overlap.
>
> Prevent this by preferring an idle housekeeping CPU whose entire SMT
> core is idle. Retain the first idle CPU as a fallback when no fully idle
> core is available, so NOHZ balancing continues to make forward progress.
> Once a partially busy core has been examined, skip its remaining SMT
> siblings to avoid repeating the core-idle check on wide SMT systems.
>
> Tests performed using an ad hoc GEMM benchmark running one CPU-intensive
> task per SMT core within its CPU affinity mask improved from
> approximately 6.2 TFLOP/s to 9.4 TFLOP/s.
>
> Note that this preference may wake a fully idle physical core instead of
> using an idle sibling of an active core, potentially increasing ILB
> wakeup latency or energy consumption on some architectures. It may also
> scan additional CPUs before selecting the one to run the ILB. The
> selection falls back to the first idle CPU when no fully idle SMT core
> is available. Non-SMT systems continue to select the first idle
> housekeeping CPU.
>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: K Prateek Nayak <kprateek.nayak@amd.com>
> Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
> Signed-off-by: Andrea Righi <arighi@nvidia.com>
> Reviewed-by: Mete Durlu <meted@linux.ibm.com>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
> ---
> Changes in v4:
> - Remove redundant this_cpu check (Prateek Nayak, Vincent Guittot)
> - Link to v3: https://lore.kernel.org/all/20260731191957.3199642-1-arighi@nvidia.com/
>
> Changes in v3:
> - After finding an idle fallback, skip all siblings when a busy CPU is
> encountered, avoiding per-CPU traversal of known-busy cores (Mete Durlu)
> - Link to v2: https://lore.kernel.org/all/20260729163225.1987068-1-arighi@nvidia.com/
>
> Changes in v2:
> - Avoid repeated is_core_idle() checks on wide SMT systems by pruning
> the remaining siblings of a partially busy core (Prateek Nayak)
> - Link to v1: https://lore.kernel.org/r/20260728214442.1648483-1-arighi@nvidia.com/
>
> kernel/sched/fair.c | 55 ++++++++++++++++++++++++++++++++++++---------
> 1 file changed, 44 insertions(+), 11 deletions(-)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 37001c63452e5..9a975a684b487 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -13964,29 +13964,62 @@ static inline int on_null_domain(struct rq *rq)
> */
> static inline int find_new_ilb(void)
> {
> - int this_cpu = smp_processor_id();
> - const struct cpumask *hk_mask;
> - int ilb_cpu;
> + struct cpumask *ilb_cpus;
> + int ilb_cpu, fallback = -1;
> +
> + lockdep_assert_irqs_disabled();
> +
> + /*
> + * Reuse the per-CPU select_rq_mask, which is protected from concurrent
> + * use on this CPU by having interrupts disabled.
> + */
> + ilb_cpus = this_cpu_cpumask_var_ptr(select_rq_mask);
> + cpumask_and(ilb_cpus, nohz.idle_cpus_mask,
> + housekeeping_cpumask(HK_TYPE_KERNEL_NOISE));
> +
> + for_each_cpu(ilb_cpu, ilb_cpus) {
> + if (!idle_cpu(ilb_cpu)) {
> + /*
> + * Once an idle fallback exists, a busy CPU proves that
> + * this core cannot be fully idle. Skip its siblings.
> + */
> + if (sched_smt_active() && fallback >= 0)
> + cpumask_andnot(ilb_cpus, ilb_cpus, cpu_smt_mask(ilb_cpu));
> + continue;
> + }
>
> - hk_mask = housekeeping_cpumask(HK_TYPE_KERNEL_NOISE);
> + /*
> + * Running the idle load balancer on an idle sibling of a busy
> + * SMT core can reduce the capacity available to its sibling. Prefer
> + * a CPU whose entire core is idle, but retain the first idle CPU as
> + * a fallback so idle balancing can still make progress when no fully
> + * idle core exists.
> + */
> + if (sched_smt_active() && !is_core_idle(ilb_cpu)) {
> + if (fallback < 0)
> + fallback = ilb_cpu;
>
> - for_each_cpu_and(ilb_cpu, nohz.idle_cpus_mask, hk_mask) {
> - if (ilb_cpu == this_cpu)
> + /*
> + * The core is not idle, so there is no need to check
> + * any of its other SMT siblings.
> + */
> + cpumask_andnot(ilb_cpus, ilb_cpus,
> + cpu_smt_mask(ilb_cpu));
> continue;
> + }
>
> - if (idle_cpu(ilb_cpu))
> - return ilb_cpu;
> + return ilb_cpu;
> }
>
> - return -1;
> + return fallback;
> }
>
> /*
> * Kick a CPU to do the NOHZ balancing, if it is time for it, via a cross-CPU
> * SMP function call (IPI).
> *
> - * We pick the first idle CPU in the HK_TYPE_KERNEL_NOISE housekeeping set
> - * (if there is one).
> + * Prefer a CPU on a fully idle core in the HK_TYPE_KERNEL_NOISE housekeeping
> + * set. Fall back to the first idle CPU when no fully idle core exists.
> */
> static void kick_ilb(unsigned int flags)
> {
> --
> 2.55.0
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v4] sched/fair: Prefer fully idle cores for NOHZ balancing
2026-08-04 15:13 [PATCH v4] sched/fair: Prefer fully idle cores for NOHZ balancing Andrea Righi
2026-08-04 15:19 ` Shrikanth Hegde
2026-08-05 10:31 ` Vincent Guittot
@ 2026-08-06 10:26 ` Shrikanth Hegde
2026-08-06 13:18 ` Andrea Righi
2 siblings, 1 reply; 7+ messages in thread
From: Shrikanth Hegde @ 2026-08-06 10:26 UTC (permalink / raw)
To: Andrea Righi
Cc: Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
Valentin Schneider, K Prateek Nayak, Christian Loehle, Phil Auld,
Mete Durlu, linux-kernel, Ingo Molnar, Peter Zijlstra, Juri Lelli,
Vincent Guittot
Hi Andrea,
On 8/4/26 8:43 PM, Andrea Righi wrote:
> find_new_ilb() selects the first idle housekeeping CPU without
> considering whether another thread is running on the same physical core.
> On an SMT system, the idle load balancer can therefore activate both
> siblings even when another housekeeping CPU has an entirely idle core.
>
> On most SMT systems, this is not problematic because the idle load
> balancer is a short-lived activity and the transient wakeup of a sibling
> has negligible performance impact.
>
> However, this can be particularly costly on NVIDIA Olympus cores used in
> Vera. Briefly activating an otherwise idle sibling can reduce the
> performance available to the other sibling and this effect does not
> necessarily end once the activated sibling becomes idle: after the ILB
> finishes and its CPU enters WFI, full single-thread performance is
> restored only after the sibling has remained idle for a qualification
> interval (10 Ki cycles on the tested Vera system). Repeated short
> sibling wakeups can therefore sustain the interference even with little
> actual overlap.
>
> Prevent this by preferring an idle housekeeping CPU whose entire SMT
> core is idle. Retain the first idle CPU as a fallback when no fully idle
> core is available, so NOHZ balancing continues to make forward progress.
> Once a partially busy core has been examined, skip its remaining SMT
> siblings to avoid repeating the core-idle check on wide SMT systems.
>
> Tests performed using an ad hoc GEMM benchmark running one CPU-intensive
> task per SMT core within its CPU affinity mask improved from
> approximately 6.2 TFLOP/s to 9.4 TFLOP/s.
>
> Note that this preference may wake a fully idle physical core instead of
> using an idle sibling of an active core, potentially increasing ILB
> wakeup latency or energy consumption on some architectures. It may also
> scan additional CPUs before selecting the one to run the ILB. The
> selection falls back to the first idle CPU when no fully idle SMT core
> is available. Non-SMT systems continue to select the first idle
> housekeeping CPU.
>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: K Prateek Nayak <kprateek.nayak@amd.com>
> Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
> Signed-off-by: Andrea Righi <arighi@nvidia.com>
> Reviewed-by: Mete Durlu <meted@linux.ibm.com>
> ---
> Changes in v4:
> - Remove redundant this_cpu check (Prateek Nayak, Vincent Guittot)
> - Link to v3: https://lore.kernel.org/all/20260731191957.3199642-1-arighi@nvidia.com/
>
> Changes in v3:
> - After finding an idle fallback, skip all siblings when a busy CPU is
> encountered, avoiding per-CPU traversal of known-busy cores (Mete Durlu)
> - Link to v2: https://lore.kernel.org/all/20260729163225.1987068-1-arighi@nvidia.com/
>
> Changes in v2:
> - Avoid repeated is_core_idle() checks on wide SMT systems by pruning
> the remaining siblings of a partially busy core (Prateek Nayak)
> - Link to v1: https://lore.kernel.org/r/20260728214442.1648483-1-arighi@nvidia.com/
>
> kernel/sched/fair.c | 55 ++++++++++++++++++++++++++++++++++++---------
> 1 file changed, 44 insertions(+), 11 deletions(-)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 37001c63452e5..9a975a684b487 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -13964,29 +13964,62 @@ static inline int on_null_domain(struct rq *rq)
> */
> static inline int find_new_ilb(void)
> {
> - int this_cpu = smp_processor_id();
> - const struct cpumask *hk_mask;
> - int ilb_cpu;
> + struct cpumask *ilb_cpus;
> + int ilb_cpu, fallback = -1;
> +
> + lockdep_assert_irqs_disabled();
> +
> + /*
> + * Reuse the per-CPU select_rq_mask, which is protected from concurrent
> + * use on this CPU by having interrupts disabled.
> + */
> + ilb_cpus = this_cpu_cpumask_var_ptr(select_rq_mask);
> + cpumask_and(ilb_cpus, nohz.idle_cpus_mask,
> + housekeeping_cpumask(HK_TYPE_KERNEL_NOISE));
> +
> + for_each_cpu(ilb_cpu, ilb_cpus) {
> + if (!idle_cpu(ilb_cpu)) {
> + /*
> + * Once an idle fallback exists, a busy CPU proves that
> + * this core cannot be fully idle. Skip its siblings.
> + */
> + if (sched_smt_active() && fallback >= 0)
> + cpumask_andnot(ilb_cpus, ilb_cpus, cpu_smt_mask(ilb_cpu));
> + continue;
> + }
>
> - hk_mask = housekeeping_cpumask(HK_TYPE_KERNEL_NOISE);
> + /*
> + * Running the idle load balancer on an idle sibling of a busy
> + * SMT core can reduce the capacity available to its sibling. Prefer
> + * a CPU whose entire core is idle, but retain the first idle CPU as
> + * a fallback so idle balancing can still make progress when no fully
> + * idle core exists.
> + */
> + if (sched_smt_active() && !is_core_idle(ilb_cpu)) {
> + if (fallback < 0)
> + fallback = ilb_cpu;
>
> - for_each_cpu_and(ilb_cpu, nohz.idle_cpus_mask, hk_mask) {
> - if (ilb_cpu == this_cpu)
> + /*
> + * The core is not idle, so there is no need to check
> + * any of its other SMT siblings.
> + */
> + cpumask_andnot(ilb_cpus, ilb_cpus,
> + cpu_smt_mask(ilb_cpu));
> continue;
> + }
>
> - if (idle_cpu(ilb_cpu))
> - return ilb_cpu;
> + return ilb_cpu;
> }
>
> - return -1;
> + return fallback;
> }
>
> /*
> * Kick a CPU to do the NOHZ balancing, if it is time for it, via a cross-CPU
> * SMP function call (IPI).
> *
> - * We pick the first idle CPU in the HK_TYPE_KERNEL_NOISE housekeeping set
> - * (if there is one).
> + * Prefer a CPU on a fully idle core in the HK_TYPE_KERNEL_NOISE housekeeping
> + * set. Fall back to the first idle CPU when no fully idle core exists.
> */
> static void kick_ilb(unsigned int flags)
> {
I have only a small system at this moment.
I don't see any noticeable regression/improvement in SMT4.
FWIW, patch looks okay to me.
Reviewed-by: Shrikanth Hegde <sshegde@linux.ibm.com>
BTW, Andrea, are you planning to do that refactoring of select_rq_mask
or you want me to pick it up?
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v4] sched/fair: Prefer fully idle cores for NOHZ balancing
2026-08-06 10:26 ` Shrikanth Hegde
@ 2026-08-06 13:18 ` Andrea Righi
0 siblings, 0 replies; 7+ messages in thread
From: Andrea Righi @ 2026-08-06 13:18 UTC (permalink / raw)
To: Shrikanth Hegde
Cc: Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
Valentin Schneider, K Prateek Nayak, Christian Loehle, Phil Auld,
Mete Durlu, linux-kernel, Ingo Molnar, Peter Zijlstra, Juri Lelli,
Vincent Guittot
Hi Shrikanth,
On Thu, Aug 06, 2026 at 03:56:20PM +0530, Shrikanth Hegde wrote:
...
> I have only a small system at this moment.
> I don't see any noticeable regression/improvement in SMT4.
>
> FWIW, patch looks okay to me.
> Reviewed-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Thanks for testing!
>
>
> BTW, Andrea, are you planning to do that refactoring of select_rq_mask
> or you want me to pick it up?
I was waiting for this patch to land before looking at the select_rq_mask
refactoring. If you have time and would like to pick it up, please go ahead,
I'll be happy to review it.
-Andrea
^ permalink raw reply [flat|nested] 7+ messages in thread