Re: [PATCH] sched_ext: Fix missing rq lock in scx_bpf_cpuperf_set()

[Andrea Righi <arighi@nvidia.com>]

On Thu, Mar 27, 2025 at 07:09:25AM -1000, Tejun Heo wrote:
> Hello,
> 
> On Thu, Mar 27, 2025 at 10:53:39AM +0100, Andrea Righi wrote:
> ...
> > > Hm... that's right, it looks like this requires a bit more work than
> > > expected, but saving the currently locked rq might be helpful also for
> > > other kfuncs, I'll take a look at this.
> > 
> > What if we lock the rq in the scx_kf_allowed_if_unlocked() case, and for
> > all the other cases we ignore locking if rq == this_rq(). If we need to
> > operate on a different rq than the current one we could either defer the
> > work or just trigger an ops error. Something like:
> > 
> > 	if (scx_kf_allowed_if_unlocked()) {
> > 		rq_lock_irqsave(rq, &rf);
> > 		update_rq_clock(rq);
> > 	} else if (rq != this_rq()) {
> > 		// defer work or ops error
> > 		return;
> > 	}
> > 
> > 	lockdep_assert_rq_held(rq);
> > 	rq->scx.cpuperf_target = perf;
> > 	cpufreq_update_util(rq, 0);
> > 
> > 	if (scx_kf_allowed_if_unlocked())
> > 		rq_unlock_irqrestore(rq, &rf);
> > 
> > AFAICS all the current scx schedulers call scx_bpf_cpuperf_set() from
> > ops.running(), ops.tick() or ops.init(), so even with the ops error we
> > should cover all the existent cases.
> > 
> > The only unsupported scenario is calling scx_bpf_cpuperf_set() from
> > ops.enqueue() / ops.select_cpu(), but maybe we could add the deferred work
> > later to handle that if needed.
> 
> balance_one() can be called from a sibling CPU when core sched is enabled,
> so ttwu isn't the only path where this_rq() test wouldn't work. Even if we
> plug all the existing holes and make it work, it feels a bit too fragile to
> me. It's something which can easily break inadvertently and cause subtle
> failures.
> 
> If we don't want to do locked rq tracking, we can always use
> schedule_deferred() when any rq is locked too. That's a bit more expensive
> tho.

Yeah, I'm a bit worried that locked rq tracking might introduce overhead to
all the scx callbacks, just to address this issue.

Perhaps schedule_deferred() is a reasonable compromise and we can limit the
overhead just to scx_bpf_cpuperf_set().

-Andrea