Re: [PATCH v5 sched_ext/for-7.3 18/33] sched_ext: Add reject DSQ for cap-rejected dispatches

[Andrea Righi <arighi@nvidia.com>]

Hi Tejun,

On Thu, Jul 09, 2026 at 12:50:26PM -1000, Tejun Heo wrote:
> When a sub-scheduler dispatches a task to a CPU it lacks the required
> capability on, the task must be rejected rather than allowed to run.
> 
> Add the machinery for that. Each rq gets a reject DSQ, a kernel-internal
> holding queue that is never run and that the BPF scheduler cannot reach. An
> insert that must be refused is diverted there instead of the local DSQ, and
> a deferred requeue then hands the parked tasks back to the BPF scheduler to
> re-decide. A cap revoke extends this to already-queued tasks. When the
> revoke reaches the cpu's effective caps, the cpu scans its local DSQ and
> reenqueues the tasks that no longer qualify.
> 
> A migration-disabled task must run on its cpu, so a capless one is admitted
> anyway and counted in the new SCX_EV_SUB_FORCED_ADMIT event.
> 
> This is preparation for the actual sub-sched cap enforcement. The divert is
> wired but inert here.
> 
> v2: Admit offline-rq and migration_pending inserts to local, not reject. (sashiko AI)
> 
> Signed-off-by: Tejun Heo <tj@kernel.org>
> ---

...

> @@ -1344,7 +1351,12 @@ static void scx_dispatch_enqueue(struct scx_sched *sch, struct rq *rq,
>  				 struct scx_dispatch_q *dsq, struct task_struct *p,
>  				 u64 enq_flags)
>  {
> -	bool is_rq_owned = dsq_is_rq_owned(dsq);
> +	bool is_rq_owned = false;
> +
> +	if (dsq->id == SCX_DSQ_LOCAL) {
> +		dsq = scx_local_or_reject_dsq(sch, rq, p, &enq_flags);
> +		is_rq_owned = true;
> +	}

With CONFIG_EXT_SUB_SCHED=y, this calls scx_local_or_reject_dsq() for every
local DSQ insertion even when there are no sub-schedulers attached. It shouldn't
be too bad, but the hot path still pays for the out-of-line call and the
CID/capability bookkeeping.

Maybe the same static key that I suggested for the ops.update_idle() case
(PATCH 28/33) can be used here as well and we could do something like:

    if (!static_branch_unlikely(&scx_has_sub_scheds))
        return &rq->scx.local_dsq;

Thanks,
-Andrea