[PATCHSET sched_ext/for-7.3] sched_ext: Sub-scheduler and cid fixes

[PATCHSET sched_ext/for-7.3] sched_ext: Sub-scheduler and cid fixes

[Tejun Heo]

Hello,

A few fixes for the sub-scheduler and cid-form code on for-7.3.

- 0001 attributes a runnable stall to the scheduler that owns the DSQ the
  task is stuck on rather than the task's owner, so a bypassing sub's task
  parked on an ancestor's bypass DSQ blames the ancestor, not the sub.

- 0002 skips the default CPU selection while bypassing. The pick is discarded
  by the bypass enqueue anyway, and it consults idle masks a scheduler doing
  its own idle tracking leaves frozen.

- 0003 guards the cid kfuncs against the cid tables not being allocated yet,
  which a program can hit while racing the first scheduler enable.

Based on sched_ext/for-7.3 (b20dfde5ec54).

Tejun Heo (3):
  sched_ext: Blame the DSQ's owning scheduler for a runnable stall
  sched_ext: Skip the default CPU selection while bypassing
  sched_ext: Guard the cid kfuncs against unallocated cid tables

Git tree: git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext.git scx-sub-fixes

 kernel/sched/ext/cid.c |  5 +++--
 kernel/sched/ext/cid.h | 23 +++++++++++++++--------
 kernel/sched/ext/ext.c | 26 ++++++++++++++++++++++++--
 3 files changed, 42 insertions(+), 12 deletions(-)

--
tejun

[PATCH 1/3] sched_ext: Blame the DSQ's owning scheduler for a runnable stall

[Tejun Heo]

check_rq_for_timeouts() blames a runnable stall on the task's owner. Under
a sub-scheduler hierarchy the stalled task can be sitting on a DSQ that a
different scheduler has to drain, e.g. an ancestor's bypass DSQ while the
owner is bypassing. The drainer then escapes blame while the owner is
exited, and when the owner's exit is already claimed, nothing actionable is
reported at all.

Blame the DSQ's owning scheduler instead. The local DSQ is consumed by the
cpu itself and keeps blame on the owner. Detection keeps the owner's timeout
and single-scheduler behavior is unchanged.

Signed-off-by: Tejun Heo <tj@kernel.org>
---
 kernel/sched/ext/ext.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index b730eac4b13f..96389ec31ab8 100644
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -3391,8 +3391,19 @@ static bool check_rq_for_timeouts(struct rq *rq)
 
 		if (unlikely(time_after(jiffies,
 					last_runnable + READ_ONCE(sch->watchdog_timeout)))) {
+			struct scx_dispatch_q *dsq = READ_ONCE(p->scx.dsq);
 			u32 dur_ms = jiffies_to_msecs(jiffies - last_runnable);
 
+			/*
+			 * A task can be stuck on a DSQ that a sched other than
+			 * its owner is responsible for draining, e.g. an
+			 * ancestor's bypass DSQ while the owner is bypassing.
+			 * Blame the drainer. The local DSQ is consumed by the
+			 * cpu itself and keeps blame on the owner.
+			 */
+			if (dsq && dsq->sched && dsq->id != SCX_DSQ_LOCAL)
+				sch = dsq->sched;
+
 			__scx_exit(sch, SCX_EXIT_ERROR_STALL, 0, cpu_of(rq),
 				   "%s[%d] failed to run for %u.%03us",
 				   p->comm, p->pid, dur_ms / 1000,
-- 
2.55.0

[PATCH 2/3] sched_ext: Skip the default CPU selection while bypassing

[Tejun Heo]

select_task_rq_scx() falls into the default path when the scheduler has no
ops.select_cpu or is bypassing. There it calls scx_select_cpu_dfl() and
direct-dispatches to the picked CPU's local DSQ.

While bypassing, neither does anything: the enqueue path routes the task to
a bypass DSQ before consulting the direct-dispatch target, so the direct
dispatch never happens, and the CPU pick at most shifts which CPU's bypass
DSQ receives the task. Worse, when the scheduler does its own idle tracking,
the built-in idle cpumasks the pick consults are not even updated, so it
doesn't work anyway.

Return prev_cpu without the default selection while bypassing and let the
bypass enqueue place the task.

Signed-off-by: Tejun Heo <tj@kernel.org>
---
 kernel/sched/ext/ext.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index 96389ec31ab8..e6c4cbe1f182 100644
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -3277,6 +3277,19 @@ static int select_task_rq_scx(struct task_struct *p, int prev_cpu, int wake_flag
 	} else {
 		s32 cpu;
 
+		/*
+		 * While bypassing, the enqueue path routes @p to a bypass DSQ
+		 * without consulting the direct-dispatch target, making the
+		 * default selection pointless. It doesn't work anyway when the
+		 * scheduler does its own idle tracking and the built-in idle
+		 * cpumasks are not updated. Leave @p on @prev_cpu.
+		 */
+		if (bypassing) {
+			__scx_add_event(sch, SCX_EV_BYPASS_DISPATCH, 1);
+			p->scx.selected_cpu = prev_cpu;
+			return prev_cpu;
+		}
+
 		cpu = scx_select_cpu_dfl(p, prev_cpu, wake_flags, NULL, 0);
 		if (cpu >= 0) {
 			refill_task_slice_dfl(sch, p);
@@ -3286,8 +3299,6 @@ static int select_task_rq_scx(struct task_struct *p, int prev_cpu, int wake_flag
 		}
 		p->scx.selected_cpu = cpu;
 
-		if (bypassing)
-			__scx_add_event(sch, SCX_EV_BYPASS_DISPATCH, 1);
 		return cpu;
 	}
 }
-- 
2.55.0

[PATCH 3/3] sched_ext: Guard the cid kfuncs against unallocated cid tables

[Tejun Heo]

The cid tables are allocated by the first enable's scx_cid_init(), which
runs after scx_alloc_and_add_sched() has published ops->priv. A TRACING or
SYSCALL program associated with the enabling struct_ops map gets a non-NULL
sched from scx_prog_sched() as soon as ops->priv is set, so a cid kfunc
called in that window dereferences the still-NULL table pointer. Only the
first enable since boot is exposed as the tables are never freed.

scx_bpf_this_cid() and scx_bpf_task_cid() already handle the window by
testing the table pointer. Do the same in scx_cid_to_cpu(),
scx_cpu_to_cid() and scx_bpf_cid_topo(), returning -EINVAL / all-(-1) topo
as before any scheduler is enabled. __scx_cid_to_cpu() and
__scx_cpu_to_cid() stay unchecked for callers with the tables guaranteed
allocated - ops invocations on a live scheduler and the enable path itself.

Fixes: e9b55af47edf ("sched_ext: Add topological CPU IDs (cids)")
Signed-off-by: Tejun Heo <tj@kernel.org>
---
 kernel/sched/ext/cid.c |  5 +++--
 kernel/sched/ext/cid.h | 23 +++++++++++++++--------
 2 files changed, 18 insertions(+), 10 deletions(-)

diff --git a/kernel/sched/ext/cid.c b/kernel/sched/ext/cid.c
index 9dfd242be34f..699cb8db228d 100644
--- a/kernel/sched/ext/cid.c
+++ b/kernel/sched/ext/cid.c
@@ -875,17 +875,18 @@ bool scx_cmask_empty(const struct scx_cmask *m)
 __bpf_kfunc void scx_bpf_cid_topo(s32 cid, struct scx_cid_topo *out__uninit,
 				  const struct bpf_prog_aux *aux)
 {
+	struct scx_cid_topo *topo = READ_ONCE(scx_cid_topo);
 	struct scx_sched *sch;
 
 	guard(rcu)();
 
 	sch = scx_prog_sched(aux);
-	if (unlikely(!sch) || !cid_valid(sch, cid)) {
+	if (unlikely(!sch) || !cid_valid(sch, cid) || unlikely(!topo)) {
 		*out__uninit = SCX_CID_TOPO_NEG;
 		return;
 	}
 
-	*out__uninit = READ_ONCE(scx_cid_topo)[cid];
+	*out__uninit = topo[cid];
 }
 
 __bpf_kfunc_end_defs();
diff --git a/kernel/sched/ext/cid.h b/kernel/sched/ext/cid.h
index b36a1a28eac8..1498efb81549 100644
--- a/kernel/sched/ext/cid.h
+++ b/kernel/sched/ext/cid.h
@@ -90,9 +90,10 @@ static inline bool cid_valid(struct scx_sched *sch, s32 cid)
  * __scx_cid_to_cpu - Unchecked cid->cpu table lookup
  * @cid: cid to look up. Must be in [0, num_possible_cpus()).
  *
- * Intended for callsites that have already validated @cid and that hold a
- * non-NULL @sch from scx_prog_sched() - a live sched implies the table has
- * been allocated, so no NULL check is needed here.
+ * Intended for callsites that have already validated @cid and where the
+ * tables are guaranteed allocated - ops invocations on a live scheduler or
+ * the enable path itself. Prog-facing kfuncs, which can run while the first
+ * enable is still allocating the tables, use the checked wrappers instead.
  */
 static inline s32 __scx_cid_to_cpu(s32 cid)
 {
@@ -119,13 +120,17 @@ static inline s32 __scx_cpu_to_cid(s32 cpu)
  * Return the cpu for @cid or a negative errno on failure. Invalid cid triggers
  * scx_error() on @sch. The cid arrays are allocated on first scheduler enable
  * and never freed, so the returned cpu is stable for the lifetime of the loaded
- * scheduler.
+ * scheduler. Return -EINVAL without triggering scx_error() if the tables are
+ * not allocated yet, which a prog-facing kfunc can observe while racing the
+ * first enable.
  */
 static inline s32 scx_cid_to_cpu(struct scx_sched *sch, s32 cid)
 {
-	if (!cid_valid(sch, cid))
+	s16 *tbl = READ_ONCE(scx_cid_to_cpu_tbl);
+
+	if (!cid_valid(sch, cid) || unlikely(!tbl))
 		return -EINVAL;
-	return __scx_cid_to_cpu(cid);
+	return tbl[cid];
 }
 
 /**
@@ -138,9 +143,11 @@ static inline s32 scx_cid_to_cpu(struct scx_sched *sch, s32 cid)
  */
 static inline s32 scx_cpu_to_cid(struct scx_sched *sch, s32 cpu)
 {
-	if (!scx_cpu_valid(sch, cpu, NULL))
+	s16 *tbl = READ_ONCE(scx_cpu_to_cid_tbl);
+
+	if (!scx_cpu_valid(sch, cpu, NULL) || unlikely(!tbl))
 		return -EINVAL;
-	return __scx_cpu_to_cid(cpu);
+	return tbl[cpu];
 }
 
 /**
-- 
2.55.0

Re: [PATCH 3/3] sched_ext: Guard the cid kfuncs against unallocated cid tables

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

New issues:
- [Low] `__scx_cid_to_cpu()` is dead code after this patch.

Pre-existing issues:
- [High] Deterministic NULL pointer dereference in `__scx_cpu_to_cid()` during the scheduler error disable path if CID tables fail to allocate. This problem wasn't introduced by this patch, but the commit message explicitly and incorrectly claims the tables are guaranteed to be allocated on the enable path.
- [High] Information leak and data races due to early publication and non-atomic writes to CID tables. This problem wasn't introduced by this patch, but the patch's attempt to fix the concurrent access window in `scx_bpf_cid_topo()` is incomplete.
--

--- Patch [3]: [PATCH 3/3] sched_ext: Guard the cid kfuncs against unallocated cid tables ---
commit 88c849437e9df48602b5417fe50640fa8c0c6abe
Author: Tejun Heo <tj@kernel.org>

sched_ext: Guard the cid kfuncs against unallocated cid tables

This commit adds NULL checks for the global CID tables in
scx_cid_to_cpu(), scx_cpu_to_cid(), and scx_bpf_cid_topo() to prevent
NULL pointer dereferences when BPF tracing programs execute during the
enable window.

> sched_ext: Guard the cid kfuncs against unallocated cid tables
> 
> The cid tables are allocated by the first enable's scx_cid_init(), which
> runs after scx_alloc_and_add_sched() has published ops->priv. A TRACING or
> SYSCALL program associated with the enabling struct_ops map gets a non-NULL
> sched from scx_prog_sched() as soon as ops->priv is set, so a cid kfunc
> called in that window dereferences the still-NULL table pointer. Only the
> first enable since boot is exposed as the tables are never freed.
> 
> scx_bpf_this_cid() and scx_bpf_task_cid() already handle the window by
> testing the table pointer. Do the same in scx_cid_to_cpu(),
> scx_cpu_to_cid() and scx_bpf_cid_topo(), returning -EINVAL / all-(-1) topo
> as before any scheduler is enabled. __scx_cid_to_cpu() and
> __scx_cpu_to_cid() stay unchecked for callers with the tables guaranteed
> allocated - ops invocations on a live scheduler and the enable path itself.

[Severity: High]
This isn't a bug introduced by this patch, but the commit message explicitly
claims the tables are guaranteed to be allocated on the enable path. Is this
fully accurate if memory allocation fails?

If memory allocation fails in scx_cid_arrays_alloc() during scx_cid_init(),
it leaves the table pointers as NULL and returns -ENOMEM:

kernel/sched/ext/cid.c:scx_cid_arrays_alloc() {
    if (!cid_to_cpu || !cpu_to_cid || !cid_topo) {
        kfree(cid_to_cpu);
        kfree(cpu_to_cid);
        kfree(cid_topo);
        return -ENOMEM;
    }
}

The error handling path jumps to err_disable which calls scx_error(). This
queues an IRQ work that executes scx_dump_state(), which iterates over all
CPUs and calls scx_dump_cpu().

scx_dump_cpu() invokes the scheduler's dump_cpu op:

kernel/sched/ext/ext.c:scx_dump_cpu() {
    SCX_CALL_OP(sch, dump_cpu, rq, dctx, scx_cpu_arg(cpu), idle);
}

Because __scx_is_cid_type was already enabled via static_branch_enable()
prior to the failed initialization, scx_cpu_arg() calls __scx_cpu_to_cid(),
which unconditionally dereferences the NULL scx_cpu_to_cid_tbl.

Could this lead to a deterministic NULL pointer dereference during the
scheduler error disable path?

> diff --git a/kernel/sched/ext/cid.c b/kernel/sched/ext/cid.c
> index af83084ec740a..712920b79c9c7 100644
> --- a/kernel/sched/ext/cid.c
> +++ b/kernel/sched/ext/cid.c
> @@ -665,17 +665,18 @@ bool scx_cmask_empty(const struct scx_cmask *m)
>  __bpf_kfunc void scx_bpf_cid_topo(s32 cid, struct scx_cid_topo *out__uninit,
>  				  const struct bpf_prog_aux *aux)
>  {
> +	struct scx_cid_topo *topo = READ_ONCE(scx_cid_topo);
>  	struct scx_sched *sch;
>  
>  	guard(rcu)();
>  
>  	sch = scx_prog_sched(aux);
> -	if (unlikely(!sch) || !cid_valid(sch, cid)) {
> +	if (unlikely(!sch) || !cid_valid(sch, cid) || unlikely(!topo)) {
>  		*out__uninit = SCX_CID_TOPO_NEG;
>  		return;
>  	}
>  
> -	*out__uninit = READ_ONCE(scx_cid_topo)[cid];
> +	*out__uninit = topo[cid];

[Severity: High]
This isn't a bug introduced by this patch, but does the patch's attempt to
fix the concurrent access window in scx_bpf_cid_topo() expose uninitialized
memory?

scx_cid_arrays_alloc() allocates scx_cid_topo without zeroing the memory
and immediately publishes it globally:

kernel/sched/ext/cid.c:scx_cid_arrays_alloc() {
    WRITE_ONCE(scx_cid_topo, cid_topo);
}

If a TRACING or SYSCALL BPF program executes concurrently before the array
is fully populated, the !topo check will pass, but topo[cid] may still
contain uninitialized kernel heap memory which is then copied to the BPF
output structure.

Additionally, scx_bpf_cid_override() modifies this structure:

kernel/sched/ext/cid.c:scx_bpf_cid_override() {
    scx_cid_topo[cid] = SCX_CID_TOPO_NEG;
}

Using non-atomic 16-byte writes while concurrent lockless readers can access
it could cause data races or tearing.

Should the array be zero-initialized or its population synchronized before
publishing?

>  }
>  
>  __bpf_kfunc_end_defs();
> diff --git a/kernel/sched/ext/cid.h b/kernel/sched/ext/cid.h
> index 9c4f4b907f12d..4ac79f6d70566 100644
> --- a/kernel/sched/ext/cid.h
> +++ b/kernel/sched/ext/cid.h

[ ... ]

> @@ -117,13 +118,17 @@ static inline s32 __scx_cpu_to_cid(s32 cpu)
>   * Return the cpu for @cid or a negative errno on failure. Invalid cid triggers
>   * scx_error() on @sch. The cid arrays are allocated on first scheduler enable
>   * and never freed, so the returned cpu is stable for the lifetime of the loaded
> - * scheduler.
> + * scheduler. Return -EINVAL without triggering scx_error() if the tables are
> + * not allocated yet, which a prog-facing kfunc can observe while racing the
> + * first enable.
>   */
>  static inline s32 scx_cid_to_cpu(struct scx_sched *sch, s32 cid)
>  {
> -	if (!cid_valid(sch, cid))
> +	s16 *tbl = READ_ONCE(scx_cid_to_cpu_tbl);
> +
> +	if (!cid_valid(sch, cid) || unlikely(!tbl))
>  		return -EINVAL;
> -	return __scx_cid_to_cpu(cid);
> +	return tbl[cid];

[Severity: Low]
Does replacing this call to __scx_cid_to_cpu() with a direct array read
leave the __scx_cid_to_cpu() inline function completely unused in the tree?
Could it be safely removed as dead code?

>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260720082605.1451945-1-tj@kernel.org?part=3

Re: [PATCH 3/3] sched_ext: Guard the cid kfuncs against unallocated cid tables

[Andrea Righi]

Hi Tejun,

On Sun, Jul 19, 2026 at 10:26:05PM -1000, Tejun Heo wrote:
> The cid tables are allocated by the first enable's scx_cid_init(), which
> runs after scx_alloc_and_add_sched() has published ops->priv. A TRACING or
> SYSCALL program associated with the enabling struct_ops map gets a non-NULL
> sched from scx_prog_sched() as soon as ops->priv is set, so a cid kfunc
> called in that window dereferences the still-NULL table pointer. Only the
> first enable since boot is exposed as the tables are never freed.
> 
> scx_bpf_this_cid() and scx_bpf_task_cid() already handle the window by
> testing the table pointer. Do the same in scx_cid_to_cpu(),
> scx_cpu_to_cid() and scx_bpf_cid_topo(), returning -EINVAL / all-(-1) topo
> as before any scheduler is enabled. __scx_cid_to_cpu() and
> __scx_cpu_to_cid() stay unchecked for callers with the tables guaranteed
> allocated - ops invocations on a live scheduler and the enable path itself.
> 
> Fixes: e9b55af47edf ("sched_ext: Add topological CPU IDs (cids)")
> Signed-off-by: Tejun Heo <tj@kernel.org>
> ---
>  kernel/sched/ext/cid.c |  5 +++--
>  kernel/sched/ext/cid.h | 23 +++++++++++++++--------
>  2 files changed, 18 insertions(+), 10 deletions(-)
> 
> diff --git a/kernel/sched/ext/cid.c b/kernel/sched/ext/cid.c
> index 9dfd242be34f..699cb8db228d 100644
> --- a/kernel/sched/ext/cid.c
> +++ b/kernel/sched/ext/cid.c
> @@ -875,17 +875,18 @@ bool scx_cmask_empty(const struct scx_cmask *m)
>  __bpf_kfunc void scx_bpf_cid_topo(s32 cid, struct scx_cid_topo *out__uninit,
>  				  const struct bpf_prog_aux *aux)
>  {
> +	struct scx_cid_topo *topo = READ_ONCE(scx_cid_topo);

I think a non-NULL pointer here doesn't necessarily mean that the table is
initialized. IIUC scx_cid_arrays_alloc() publishes these pointers before
scx_cid_init() populates them, so a racing program can still observe zero-filled
uninitialized cid_topo entries.

This also applies to the other enables, where the tables renamin non-NULL while
being rewritten. Should we add a defer publication until initialization is
complete or a readiness flag?

Thanks,
-Andrea

>  	struct scx_sched *sch;
>  
>  	guard(rcu)();
>  
>  	sch = scx_prog_sched(aux);
> -	if (unlikely(!sch) || !cid_valid(sch, cid)) {
> +	if (unlikely(!sch) || !cid_valid(sch, cid) || unlikely(!topo)) {
>  		*out__uninit = SCX_CID_TOPO_NEG;
>  		return;
>  	}
>  
> -	*out__uninit = READ_ONCE(scx_cid_topo)[cid];
> +	*out__uninit = topo[cid];
>  }
>  
>  __bpf_kfunc_end_defs();
> diff --git a/kernel/sched/ext/cid.h b/kernel/sched/ext/cid.h
> index b36a1a28eac8..1498efb81549 100644
> --- a/kernel/sched/ext/cid.h
> +++ b/kernel/sched/ext/cid.h
> @@ -90,9 +90,10 @@ static inline bool cid_valid(struct scx_sched *sch, s32 cid)
>   * __scx_cid_to_cpu - Unchecked cid->cpu table lookup
>   * @cid: cid to look up. Must be in [0, num_possible_cpus()).
>   *
> - * Intended for callsites that have already validated @cid and that hold a
> - * non-NULL @sch from scx_prog_sched() - a live sched implies the table has
> - * been allocated, so no NULL check is needed here.
> + * Intended for callsites that have already validated @cid and where the
> + * tables are guaranteed allocated - ops invocations on a live scheduler or
> + * the enable path itself. Prog-facing kfuncs, which can run while the first
> + * enable is still allocating the tables, use the checked wrappers instead.
>   */
>  static inline s32 __scx_cid_to_cpu(s32 cid)
>  {
> @@ -119,13 +120,17 @@ static inline s32 __scx_cpu_to_cid(s32 cpu)
>   * Return the cpu for @cid or a negative errno on failure. Invalid cid triggers
>   * scx_error() on @sch. The cid arrays are allocated on first scheduler enable
>   * and never freed, so the returned cpu is stable for the lifetime of the loaded
> - * scheduler.
> + * scheduler. Return -EINVAL without triggering scx_error() if the tables are
> + * not allocated yet, which a prog-facing kfunc can observe while racing the
> + * first enable.
>   */
>  static inline s32 scx_cid_to_cpu(struct scx_sched *sch, s32 cid)
>  {
> -	if (!cid_valid(sch, cid))
> +	s16 *tbl = READ_ONCE(scx_cid_to_cpu_tbl);
> +
> +	if (!cid_valid(sch, cid) || unlikely(!tbl))
>  		return -EINVAL;
> -	return __scx_cid_to_cpu(cid);
> +	return tbl[cid];
>  }
>  
>  /**
> @@ -138,9 +143,11 @@ static inline s32 scx_cid_to_cpu(struct scx_sched *sch, s32 cid)
>   */
>  static inline s32 scx_cpu_to_cid(struct scx_sched *sch, s32 cpu)
>  {
> -	if (!scx_cpu_valid(sch, cpu, NULL))
> +	s16 *tbl = READ_ONCE(scx_cpu_to_cid_tbl);
> +
> +	if (!scx_cpu_valid(sch, cpu, NULL) || unlikely(!tbl))
>  		return -EINVAL;
> -	return __scx_cpu_to_cid(cpu);
> +	return tbl[cpu];
>  }
>  
>  /**
> -- 
> 2.55.0
>