Re: [PATCH 2/2] sched_ext: Introduce per-NUMA idle cpumasks

[Andrea Righi <arighi@nvidia.com>]

Hi Yuri,

On Fri, Nov 29, 2024 at 11:38:53AM -0800, Yury Norov wrote:
...
> > @@ -3156,39 +3186,48 @@ static bool test_and_clear_cpu_idle(int cpu)
> >  	 */
> >  	if (sched_smt_active()) {
> >  		const struct cpumask *smt = cpu_smt_mask(cpu);
> > +		struct cpumask *idle_smt = get_idle_smtmask(cpu);
> >  
> >  		/*
> >  		 * If offline, @cpu is not its own sibling and
> >  		 * scx_pick_idle_cpu() can get caught in an infinite loop as
> > -		 * @cpu is never cleared from idle_masks.smt. Ensure that @cpu
> > -		 * is eventually cleared.
> > +		 * @cpu is never cleared from the idle SMT mask. Ensure that
> > +		 * @cpu is eventually cleared.
> >  		 */
> > -		if (cpumask_intersects(smt, idle_masks.smt))
> > -			cpumask_andnot(idle_masks.smt, idle_masks.smt, smt);
> > -		else if (cpumask_test_cpu(cpu, idle_masks.smt))
> > -			__cpumask_clear_cpu(cpu, idle_masks.smt);
> > +		cpumask_andnot(idle_smt, idle_smt, smt);
> > +		__cpumask_clear_cpu(cpu, idle_smt);
> >  	}

these cpumask_intersects() and cpumask_test_cpu() seem to help instead.

I think the reason is that they can help reduce some memory writes and
therefore alleviate cache coherence pressure. So, even though we could
logically remove them, they seem to offer practical benefits. I'll
re-introduce them and add a comment to clarify this in the next patch
set version.

-Andrea