Re: [PATCH v3] sched/fair: Prefer fully idle cores for NOHZ balancing

[Andrea Righi <arighi@nvidia.com>]

Hi Mete,

On Tue, Aug 04, 2026 at 02:36:48PM +0200, Mete Durlu wrote:
> Hi,
> 
> > find_new_ilb() selects the first idle housekeeping CPU without
> > considering whether another thread is running on the same physical core.
> > On an SMT system, the idle load balancer can therefore activate both
> > siblings even when another housekeeping CPU has an entirely idle core.
> > 
> > On most SMT systems, this is not problematic because the idle load
> > balancer is a short-lived activity and the transient wakeup of a sibling
> > has negligible performance impact.
> > 
> > However, this can be particularly costly on NVIDIA Olympus cores used in
> > Vera. Briefly activating an otherwise idle sibling can reduce the
> > performance available to the other sibling and this effect does not
> > necessarily end once the activated sibling becomes idle: after the ILB
> > finishes and its CPU enters WFI, full single-thread performance is
> > restored only after the sibling has remained idle for a qualification
> > interval (10 Ki cycles on the tested Vera system). Repeated short
> > sibling wakeups can therefore sustain the interference even with little
> > actual overlap.
> > 
> > Prevent this by preferring an idle housekeeping CPU whose entire SMT
> > core is idle. Retain the first idle CPU as a fallback when no fully idle
> > core is available, so NOHZ balancing continues to make forward progress.
> > Once a partially busy core has been examined, skip its remaining SMT
> > siblings to avoid repeating the core-idle check on wide SMT systems.
> > 
> > Tests performed using an ad hoc GEMM benchmark running one CPU-intensive
> > task per SMT core within its CPU affinity mask improved from
> > approximately 6.2 TFLOP/s to 9.4 TFLOP/s.
> 
> Although what you describe above with siblings suffering interference
> does not really fit to s390, I'd like to hear more about what sort
> of GEMM (general matrix multiplication) tests you did.
> 
> I tested this patch with a couple of different tools
> - perf bench sched pipe
> - hackbench
> - uperf
> - cyclictest
> - stress-ng (3d-matrix and cyclic)
> 
> Didn't come across any meaningful difference in any of them on multiple
> runs each. So I was curious about the exact sort of benchmark you
> mention here.

Thanks for testing on s390!

The original benchmark I used is based on an internal NVPL container that I
can't share publicly. However, I tried with the public OpenBLAS SGEMM benchmark
and I can see exactly the same behavior and effects:

  https://github.com/OpenMathLib/OpenBLAS

My test machine has the following topology (arm64):

    CPUs:                   352
    Sockets:                2
    Cores per socket:       88
    Threads per core:       2
    NUMA node 0 CPUs:       0-87,176-263
    NUMA node 1 CPUs:       88-175,264-351

The SMT sibling pairs on NUMA node 0 are (0,176), (1,177), ..., (87,263). I only
used NUMA node 0, to prevent adding potential NUMA side effects.

I used OpenBLAS v0.3.33, built using GCC 13.3.0 with the ARMv8 SVE kernels and
OpenMP threading:

 $ make -j176 \
        TARGET=ARMV8SVE \
        USE_OPENMP=1 \
        NUM_THREADS=176 \
        NOFORTRAN=1

 $ make -C benchmark sgemm.goto \
        TARGET=ARMV8SVE \
        USE_OPENMP=1 \
        NUM_THREADS=176 \
        NOFORTRAN=1

The unpatched kernel was tip/master, while the patched kernel used the same base
with only this change applied.

A 10-loop runs produced:

  unpatched:  5.088 TFLOP/s
  patched:    7.143 TFLOP/s

Two additional 5-loop runs produced:

  unpatched:  5.338, 5.246 TFLOP/s
  patched:    7.082, 7.144 TFLOP/s

The median across all the measurements increased from 5.246 TFLOP/s to 7.143
TFLOP/s, an improvement of approximately 36.2%.

The absolute throughput is lower than my initial NVPL benchmark, as expected
from the different GEMM implementations, but the relative behavior seems to be
consistent.

> 
> One minor nit for the diff below;
> 
> > 
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index 37001c63452e5..574b6b3ee922a 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -13965,28 +13965,66 @@ static inline int on_null_domain(struct rq *rq)
> >   static inline int find_new_ilb(void)
> >   {
> >   	int this_cpu = smp_processor_id();
> > -	const struct cpumask *hk_mask;
> > -	int ilb_cpu;
> > +	struct cpumask *ilb_cpus;
> > +	int ilb_cpu, fallback = -1;
> > +
> > +	lockdep_assert_irqs_disabled();
> > -	hk_mask = housekeeping_cpumask(HK_TYPE_KERNEL_NOISE);
> > +	/*
> > +	 * Reuse the per-CPU select_rq_mask, which is protected from concurrent
> > +	 * use on this CPU by having interrupts disabled.
> > +	 */
> > +	ilb_cpus = this_cpu_cpumask_var_ptr(select_rq_mask);
> > +	cpumask_and(ilb_cpus, nohz.idle_cpus_mask,
> > +		    housekeeping_cpumask(HK_TYPE_KERNEL_NOISE));
> > -	for_each_cpu_and(ilb_cpu, nohz.idle_cpus_mask, hk_mask) {
> > +	for_each_cpu(ilb_cpu, ilb_cpus) {
> >   		if (ilb_cpu == this_cpu)
> >   			continue;
> > -		if (idle_cpu(ilb_cpu))
> > -			return ilb_cpu;
> > +		if (!idle_cpu(ilb_cpu)) {
> > +			/*
> > +			 * Once an idle fallback exists, a busy CPU proves that
> > +			 * this core cannot be fully idle. Skip its siblings.
> > +			 */
> > +			if (sched_smt_active() && fallback >= 0)
> > +				cpumask_andnot(ilb_cpus, ilb_cpus,
> > +					       cpu_smt_mask(ilb_cpu));
> 
> nit;
> With line break this if block is now taking multiple lines and deserves
> its own curly braces.

The cpumask_andnot() invocation fits within the line-length limit, so I'll move
it on the same line.

> 
> With or without the nit, feel free to add my r-b to v4, I doubt removal
> of the "this_cpu" check will change anything as it is a dud.
> 
> Reviewed By: Mete Durlu <meted@linux.ibm.com>
> 

Thanks,
-Andrea