Re: [PATCH v2 sched_ext/for-7.1] sched_ext: Invalidate dispatch decisions on CPU affinity changes

[Kuba Piecuch <jpiecuch@google.com>]

Hi Andrea,

On Thu Mar 19, 2026 at 8:35 AM UTC, Andrea Righi wrote:
> @@ -2043,6 +2041,13 @@ static void ops_dequeue(struct rq *rq, struct task_struct *p, u64 deq_flags)
>  		 */
>  		BUG();
>  	case SCX_OPSS_QUEUED:
> +		/*
> +		 * Invalidate any in-flight dispatches for this task. The
> +		 * task is leaving the runqueue, so any dispatch decision
> +		 * made while it was queued is stale.
> +		 */
> +		rq->scx.ops_qseq++;

I'm not sure why this is necessary. Isn't setting the ops_state to
SCX_OPSS_NONE enough to invalidate in-flight dispatches? Could you describe
a scenario where incrementing qseq on dequeue is necessary?

> @@ -2537,9 +2546,26 @@ static void dispatch_to_local_dsq(struct scx_sched *sch, struct rq *rq,
>  	}
>  
>  	if (src_rq != dst_rq &&
> -	    unlikely(!task_can_run_on_remote_rq(sch, p, dst_rq, true))) {
> -		dispatch_enqueue(sch, rq, find_global_dsq(sch, task_cpu(p)), p,
> -				 enq_flags | SCX_ENQ_CLEAR_OPSS | SCX_ENQ_GDSQ_FALLBACK);
> +	    unlikely(!task_can_run_on_remote_rq(sch, p, dst_rq, false))) {
> +		/*
> +		 * Affinity changed after dispatch decision and the task
> +		 * can't run anymore on the destination rq.

More of a nitpick, but this doesn't necessarily mean that the affinity changed.
The scheduler could have also issued an invalid dispatch to a CPU outside of
the task's cpumask (e.g. due to a bug), in which case the task won't be
re-enqueued if we simply drop the dispatch, correct?

> +		 *
> +		 * Drop the dispatch, the task will be re-enqueued. Set the

Just to clarify, is this referring to the enqueue that happens in
do_set_cpus_allowed(), immediately after the actual cpumask change?

> +		 * task back to QUEUED so dequeue (if waiting) can proceed
> +		 * using current qseq from the task's rq.
> +		 */
> +		if (src_rq != rq) {
> +			raw_spin_rq_unlock(rq);
> +			raw_spin_rq_lock(src_rq);
> +		}
> +		atomic_long_set_release(&p->scx.ops_state,
> +			       SCX_OPSS_QUEUED |
> +			       (src_rq->scx.ops_qseq << SCX_OPSS_QSEQ_SHIFT));
> +		if (src_rq != rq) {
> +			raw_spin_rq_unlock(src_rq);
> +			raw_spin_rq_lock(rq);
> +		}
>  		return;
>  	}

My understanding is that task_can_run_on_remote_rq() can run without src_rq
locked, so it's possible that @p's cpumask changes after the check, isn't it?
In that case, I think it's still possible to move the task to the local DSQ
of a CPU that is outside of its cpumask, triggering a warning in
move_remote_task_to_local_dsq().

Thanks,
Kuba