Re: [PATCH v2 sched_ext/for-7.1] sched_ext: Invalidate dispatch decisions on CPU affinity changes

[Andrea Righi <arighi@nvidia.com>]

On Thu, Mar 19, 2026 at 10:31:30AM +0000, Kuba Piecuch wrote:
> Hi Andrea,
> 
> On Thu Mar 19, 2026 at 8:35 AM UTC, Andrea Righi wrote:
> > @@ -2043,6 +2041,13 @@ static void ops_dequeue(struct rq *rq, struct task_struct *p, u64 deq_flags)
> >  		 */
> >  		BUG();
> >  	case SCX_OPSS_QUEUED:
> > +		/*
> > +		 * Invalidate any in-flight dispatches for this task. The
> > +		 * task is leaving the runqueue, so any dispatch decision
> > +		 * made while it was queued is stale.
> > +		 */
> > +		rq->scx.ops_qseq++;
> 
> I'm not sure why this is necessary. Isn't setting the ops_state to
> SCX_OPSS_NONE enough to invalidate in-flight dispatches? Could you describe
> a scenario where incrementing qseq on dequeue is necessary?

I'm looking back at the code and I think you're right, ops_qseq is already
incremented by the new enqueue, so setting the state to NONE on dequeue
should be enough to drop the in-flight dispatch.

I did some quick tests and everything seems to work also without this
increment. Thanks for noticing this!

> 
> > @@ -2537,9 +2546,26 @@ static void dispatch_to_local_dsq(struct scx_sched *sch, struct rq *rq,
> >  	}
> >  
> >  	if (src_rq != dst_rq &&
> > -	    unlikely(!task_can_run_on_remote_rq(sch, p, dst_rq, true))) {
> > -		dispatch_enqueue(sch, rq, find_global_dsq(sch, task_cpu(p)), p,
> > -				 enq_flags | SCX_ENQ_CLEAR_OPSS | SCX_ENQ_GDSQ_FALLBACK);
> > +	    unlikely(!task_can_run_on_remote_rq(sch, p, dst_rq, false))) {
> > +		/*
> > +		 * Affinity changed after dispatch decision and the task
> > +		 * can't run anymore on the destination rq.
> 
> More of a nitpick, but this doesn't necessarily mean that the affinity changed.
> The scheduler could have also issued an invalid dispatch to a CPU outside of
> the task's cpumask (e.g. due to a bug), in which case the task won't be
> re-enqueued if we simply drop the dispatch, correct?

That's right, the scheduler could have issues an invalid dispatch and in
that case we would just drop the task on the floor, which is not really
nice, it'd be better to immediately error in this case. And we don't need
the global DSQ fallback, since we're erroring anyway.

I need to rethink this part...

> 
> > +		 *
> > +		 * Drop the dispatch, the task will be re-enqueued. Set the
> 
> Just to clarify, is this referring to the enqueue that happens in
> do_set_cpus_allowed(), immediately after the actual cpumask change?

Correct, it's the enqueue that happens from sched_change_end() in
do_set_cpus_allowed().

Thanks,
-Andrea