Re: [PATCH 2/2] selftests/sched_ext: Make allowed_cpus idle validation race-free

[Andrea Righi <arighi@nvidia.com>]

On Fri, Jul 31, 2026 at 11:12:09AM +0000, Kuba Piecuch wrote:
> Hi Andrea,
> 
> On Fri Jul 31, 2026 at 8:59 AM UTC, Andrea Righi wrote:
> > A remotely selected CPU can be re-advertised as idle by an idle-to-idle
> > re-pick before the BPF program validates the selection. Checking that
> > the selected CPU remains absent from the idle mask is therefore
> > inherently racy.
> >
> > Validate the stable local invariant instead: a CPU running a non-idle
> > scheduling context in ops.select_cpu() must not be advertised as idle.
> 
> That invariant sounds like it should hold in many contexts, not just in
> ops.select_cpu(). Is there something preventing us from checking it in
> ops.enqueue() as well?

Yes, nothing prevents checking the local invariant from ops.enqueue() as well.
The CPU running the callback shouldn't be advertised as idle. I'll add this in
v3.

> 
> > Also validate both the requested domain and task affinity for selected
> > CPUs.
> >
> > Signed-off-by: Andrea Righi <arighi@nvidia.com>
> > ---
> >  .../selftests/sched_ext/allowed_cpus.bpf.c    | 43 ++++++++++++++++---
> >  1 file changed, 36 insertions(+), 7 deletions(-)
> >
> > diff --git a/tools/testing/selftests/sched_ext/allowed_cpus.bpf.c b/tools/testing/selftests/sched_ext/allowed_cpus.bpf.c
> > index 35923e74a2ec3..411a7edcb9605 100644
> > --- a/tools/testing/selftests/sched_ext/allowed_cpus.bpf.c
> > +++ b/tools/testing/selftests/sched_ext/allowed_cpus.bpf.c
> > @@ -15,15 +15,43 @@ UEI_DEFINE(uei);
> >  private(PREF_CPUS) struct bpf_cpumask __kptr * allowed_cpumask;
> >  
> >  static void
> > -validate_idle_cpu(const struct task_struct *p, const struct cpumask *allowed, s32 cpu)
> > +validate_local_idle_state(void)
> >  {
> > -	if (scx_bpf_test_and_clear_cpu_idle(cpu))
> > -		scx_bpf_error("CPU %d should be marked as busy", cpu);
> > +	struct task_struct *curr;
> > +	s32 cpu = bpf_get_smp_processor_id();
> > +	bool curr_is_idle;
> >  
> > -	if (bpf_cpumask_subset(allowed, p->cpus_ptr) &&
> > -	    !bpf_cpumask_test_cpu(cpu, allowed))
> > +	bpf_rcu_read_lock();
> > +	curr = scx_bpf_cpu_curr(cpu);
> > +	curr_is_idle = curr && (curr->flags & PF_IDLE);
> > +	bpf_rcu_read_unlock();
> > +
> > +	/*
> > +	 * Unlike a remote selected CPU, the local CPU cannot go through an
> > +	 * idle re-pick while this callback is running. If it is running a
> > +	 * non-idle scheduling context, it must not be advertised as idle.
> > +	 */
> > +	if (!curr_is_idle && scx_bpf_test_and_clear_cpu_idle(cpu))
> > +		scx_bpf_error("running CPU %d should be marked as busy", cpu);
> 
> Could we check a stronger invariant by also checking that the bit in the idle
> mask is set if we're running an idle task? We can get the idle cpumask through
> scx_bpf_get_idle_cpumask() and check bits without clearing them using
> bpf_cpumask_test_cpu().

I don't think the other direction always holds: an idle CPU can be claimed by
another BPF idle CPU selection helper, which can clear the idle bit before the
CPU necessarily stops running the idle task. In that case, observing an idle
task with a clear idle bit is legitimate.

However, we can safely use scx_bpf_get_idle_cpumask() to perform the existing
check without modifying the mask.

Thanks,
-Andrea