[PATCHSET sched_ext/for-7.3] sched_ext: Sub-scheduler and cid fixes

[PATCHSET sched_ext/for-7.3] sched_ext: Sub-scheduler and cid fixes

[Tejun Heo]

Hello,

A few fixes for the sub-scheduler and cid-form code on for-7.3.

- 0001 attributes a runnable stall to the scheduler that owns the DSQ the
  task is stuck on rather than the task's owner, so a bypassing sub's task
  parked on an ancestor's bypass DSQ blames the ancestor, not the sub.

- 0002 skips the default CPU selection while bypassing. The pick is discarded
  by the bypass enqueue anyway, and it consults idle masks a scheduler doing
  its own idle tracking leaves frozen.

- 0003 guards the cid kfuncs against the cid tables not being allocated yet,
  which a program can hit while racing the first scheduler enable.

Based on sched_ext/for-7.3 (b20dfde5ec54).

Tejun Heo (3):
  sched_ext: Blame the DSQ's owning scheduler for a runnable stall
  sched_ext: Skip the default CPU selection while bypassing
  sched_ext: Guard the cid kfuncs against unallocated cid tables

Git tree: git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext.git scx-sub-fixes

 kernel/sched/ext/cid.c |  5 +++--
 kernel/sched/ext/cid.h | 23 +++++++++++++++--------
 kernel/sched/ext/ext.c | 26 ++++++++++++++++++++++++--
 3 files changed, 42 insertions(+), 12 deletions(-)

--
tejun

[PATCH 1/3] sched_ext: Blame the DSQ's owning scheduler for a runnable stall

[Tejun Heo]

check_rq_for_timeouts() blames a runnable stall on the task's owner. Under
a sub-scheduler hierarchy the stalled task can be sitting on a DSQ that a
different scheduler has to drain, e.g. an ancestor's bypass DSQ while the
owner is bypassing. The drainer then escapes blame while the owner is
exited, and when the owner's exit is already claimed, nothing actionable is
reported at all.

Blame the DSQ's owning scheduler instead. The local DSQ is consumed by the
cpu itself and keeps blame on the owner. Detection keeps the owner's timeout
and single-scheduler behavior is unchanged.

Signed-off-by: Tejun Heo <tj@kernel.org>
---
 kernel/sched/ext/ext.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index b730eac4b13f..96389ec31ab8 100644
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -3391,8 +3391,19 @@ static bool check_rq_for_timeouts(struct rq *rq)
 
 		if (unlikely(time_after(jiffies,
 					last_runnable + READ_ONCE(sch->watchdog_timeout)))) {
+			struct scx_dispatch_q *dsq = READ_ONCE(p->scx.dsq);
 			u32 dur_ms = jiffies_to_msecs(jiffies - last_runnable);
 
+			/*
+			 * A task can be stuck on a DSQ that a sched other than
+			 * its owner is responsible for draining, e.g. an
+			 * ancestor's bypass DSQ while the owner is bypassing.
+			 * Blame the drainer. The local DSQ is consumed by the
+			 * cpu itself and keeps blame on the owner.
+			 */
+			if (dsq && dsq->sched && dsq->id != SCX_DSQ_LOCAL)
+				sch = dsq->sched;
+
 			__scx_exit(sch, SCX_EXIT_ERROR_STALL, 0, cpu_of(rq),
 				   "%s[%d] failed to run for %u.%03us",
 				   p->comm, p->pid, dur_ms / 1000,
-- 
2.55.0

[PATCH 2/3] sched_ext: Skip the default CPU selection while bypassing

[Tejun Heo]

select_task_rq_scx() falls into the default path when the scheduler has no
ops.select_cpu or is bypassing. There it calls scx_select_cpu_dfl() and
direct-dispatches to the picked CPU's local DSQ.

While bypassing, neither does anything: the enqueue path routes the task to
a bypass DSQ before consulting the direct-dispatch target, so the direct
dispatch never happens, and the CPU pick at most shifts which CPU's bypass
DSQ receives the task. Worse, when the scheduler does its own idle tracking,
the built-in idle cpumasks the pick consults are not even updated, so it
doesn't work anyway.

Return prev_cpu without the default selection while bypassing and let the
bypass enqueue place the task.

Signed-off-by: Tejun Heo <tj@kernel.org>
---
 kernel/sched/ext/ext.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index 96389ec31ab8..e6c4cbe1f182 100644
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -3277,6 +3277,19 @@ static int select_task_rq_scx(struct task_struct *p, int prev_cpu, int wake_flag
 	} else {
 		s32 cpu;
 
+		/*
+		 * While bypassing, the enqueue path routes @p to a bypass DSQ
+		 * without consulting the direct-dispatch target, making the
+		 * default selection pointless. It doesn't work anyway when the
+		 * scheduler does its own idle tracking and the built-in idle
+		 * cpumasks are not updated. Leave @p on @prev_cpu.
+		 */
+		if (bypassing) {
+			__scx_add_event(sch, SCX_EV_BYPASS_DISPATCH, 1);
+			p->scx.selected_cpu = prev_cpu;
+			return prev_cpu;
+		}
+
 		cpu = scx_select_cpu_dfl(p, prev_cpu, wake_flags, NULL, 0);
 		if (cpu >= 0) {
 			refill_task_slice_dfl(sch, p);
@@ -3286,8 +3299,6 @@ static int select_task_rq_scx(struct task_struct *p, int prev_cpu, int wake_flag
 		}
 		p->scx.selected_cpu = cpu;
 
-		if (bypassing)
-			__scx_add_event(sch, SCX_EV_BYPASS_DISPATCH, 1);
 		return cpu;
 	}
 }
-- 
2.55.0

[PATCH 3/3] sched_ext: Guard the cid kfuncs against unallocated cid tables

[Tejun Heo]

The cid tables are allocated by the first enable's scx_cid_init(), which
runs after scx_alloc_and_add_sched() has published ops->priv. A TRACING or
SYSCALL program associated with the enabling struct_ops map gets a non-NULL
sched from scx_prog_sched() as soon as ops->priv is set, so a cid kfunc
called in that window dereferences the still-NULL table pointer. Only the
first enable since boot is exposed as the tables are never freed.

scx_bpf_this_cid() and scx_bpf_task_cid() already handle the window by
testing the table pointer. Do the same in scx_cid_to_cpu(),
scx_cpu_to_cid() and scx_bpf_cid_topo(), returning -EINVAL / all-(-1) topo
as before any scheduler is enabled. __scx_cid_to_cpu() and
__scx_cpu_to_cid() stay unchecked for callers with the tables guaranteed
allocated - ops invocations on a live scheduler and the enable path itself.

Fixes: e9b55af47edf ("sched_ext: Add topological CPU IDs (cids)")
Signed-off-by: Tejun Heo <tj@kernel.org>
---
 kernel/sched/ext/cid.c |  5 +++--
 kernel/sched/ext/cid.h | 23 +++++++++++++++--------
 2 files changed, 18 insertions(+), 10 deletions(-)

diff --git a/kernel/sched/ext/cid.c b/kernel/sched/ext/cid.c
index 9dfd242be34f..699cb8db228d 100644
--- a/kernel/sched/ext/cid.c
+++ b/kernel/sched/ext/cid.c
@@ -875,17 +875,18 @@ bool scx_cmask_empty(const struct scx_cmask *m)
 __bpf_kfunc void scx_bpf_cid_topo(s32 cid, struct scx_cid_topo *out__uninit,
 				  const struct bpf_prog_aux *aux)
 {
+	struct scx_cid_topo *topo = READ_ONCE(scx_cid_topo);
 	struct scx_sched *sch;
 
 	guard(rcu)();
 
 	sch = scx_prog_sched(aux);
-	if (unlikely(!sch) || !cid_valid(sch, cid)) {
+	if (unlikely(!sch) || !cid_valid(sch, cid) || unlikely(!topo)) {
 		*out__uninit = SCX_CID_TOPO_NEG;
 		return;
 	}
 
-	*out__uninit = READ_ONCE(scx_cid_topo)[cid];
+	*out__uninit = topo[cid];
 }
 
 __bpf_kfunc_end_defs();
diff --git a/kernel/sched/ext/cid.h b/kernel/sched/ext/cid.h
index b36a1a28eac8..1498efb81549 100644
--- a/kernel/sched/ext/cid.h
+++ b/kernel/sched/ext/cid.h
@@ -90,9 +90,10 @@ static inline bool cid_valid(struct scx_sched *sch, s32 cid)
  * __scx_cid_to_cpu - Unchecked cid->cpu table lookup
  * @cid: cid to look up. Must be in [0, num_possible_cpus()).
  *
- * Intended for callsites that have already validated @cid and that hold a
- * non-NULL @sch from scx_prog_sched() - a live sched implies the table has
- * been allocated, so no NULL check is needed here.
+ * Intended for callsites that have already validated @cid and where the
+ * tables are guaranteed allocated - ops invocations on a live scheduler or
+ * the enable path itself. Prog-facing kfuncs, which can run while the first
+ * enable is still allocating the tables, use the checked wrappers instead.
  */
 static inline s32 __scx_cid_to_cpu(s32 cid)
 {
@@ -119,13 +120,17 @@ static inline s32 __scx_cpu_to_cid(s32 cpu)
  * Return the cpu for @cid or a negative errno on failure. Invalid cid triggers
  * scx_error() on @sch. The cid arrays are allocated on first scheduler enable
  * and never freed, so the returned cpu is stable for the lifetime of the loaded
- * scheduler.
+ * scheduler. Return -EINVAL without triggering scx_error() if the tables are
+ * not allocated yet, which a prog-facing kfunc can observe while racing the
+ * first enable.
  */
 static inline s32 scx_cid_to_cpu(struct scx_sched *sch, s32 cid)
 {
-	if (!cid_valid(sch, cid))
+	s16 *tbl = READ_ONCE(scx_cid_to_cpu_tbl);
+
+	if (!cid_valid(sch, cid) || unlikely(!tbl))
 		return -EINVAL;
-	return __scx_cid_to_cpu(cid);
+	return tbl[cid];
 }
 
 /**
@@ -138,9 +143,11 @@ static inline s32 scx_cid_to_cpu(struct scx_sched *sch, s32 cid)
  */
 static inline s32 scx_cpu_to_cid(struct scx_sched *sch, s32 cpu)
 {
-	if (!scx_cpu_valid(sch, cpu, NULL))
+	s16 *tbl = READ_ONCE(scx_cpu_to_cid_tbl);
+
+	if (!scx_cpu_valid(sch, cpu, NULL) || unlikely(!tbl))
 		return -EINVAL;
-	return __scx_cpu_to_cid(cpu);
+	return tbl[cpu];
 }
 
 /**
-- 
2.55.0

Re: [PATCH 3/3] sched_ext: Guard the cid kfuncs against unallocated cid tables

[Andrea Righi]

Hi Tejun,

On Sun, Jul 19, 2026 at 10:26:05PM -1000, Tejun Heo wrote:
> The cid tables are allocated by the first enable's scx_cid_init(), which
> runs after scx_alloc_and_add_sched() has published ops->priv. A TRACING or
> SYSCALL program associated with the enabling struct_ops map gets a non-NULL
> sched from scx_prog_sched() as soon as ops->priv is set, so a cid kfunc
> called in that window dereferences the still-NULL table pointer. Only the
> first enable since boot is exposed as the tables are never freed.
> 
> scx_bpf_this_cid() and scx_bpf_task_cid() already handle the window by
> testing the table pointer. Do the same in scx_cid_to_cpu(),
> scx_cpu_to_cid() and scx_bpf_cid_topo(), returning -EINVAL / all-(-1) topo
> as before any scheduler is enabled. __scx_cid_to_cpu() and
> __scx_cpu_to_cid() stay unchecked for callers with the tables guaranteed
> allocated - ops invocations on a live scheduler and the enable path itself.
> 
> Fixes: e9b55af47edf ("sched_ext: Add topological CPU IDs (cids)")
> Signed-off-by: Tejun Heo <tj@kernel.org>
> ---
>  kernel/sched/ext/cid.c |  5 +++--
>  kernel/sched/ext/cid.h | 23 +++++++++++++++--------
>  2 files changed, 18 insertions(+), 10 deletions(-)
> 
> diff --git a/kernel/sched/ext/cid.c b/kernel/sched/ext/cid.c
> index 9dfd242be34f..699cb8db228d 100644
> --- a/kernel/sched/ext/cid.c
> +++ b/kernel/sched/ext/cid.c
> @@ -875,17 +875,18 @@ bool scx_cmask_empty(const struct scx_cmask *m)
>  __bpf_kfunc void scx_bpf_cid_topo(s32 cid, struct scx_cid_topo *out__uninit,
>  				  const struct bpf_prog_aux *aux)
>  {
> +	struct scx_cid_topo *topo = READ_ONCE(scx_cid_topo);

I think a non-NULL pointer here doesn't necessarily mean that the table is
initialized. IIUC scx_cid_arrays_alloc() publishes these pointers before
scx_cid_init() populates them, so a racing program can still observe zero-filled
uninitialized cid_topo entries.

This also applies to the other enables, where the tables renamin non-NULL while
being rewritten. Should we add a defer publication until initialization is
complete or a readiness flag?

Thanks,
-Andrea

>  	struct scx_sched *sch;
>  
>  	guard(rcu)();
>  
>  	sch = scx_prog_sched(aux);
> -	if (unlikely(!sch) || !cid_valid(sch, cid)) {
> +	if (unlikely(!sch) || !cid_valid(sch, cid) || unlikely(!topo)) {
>  		*out__uninit = SCX_CID_TOPO_NEG;
>  		return;
>  	}
>  
> -	*out__uninit = READ_ONCE(scx_cid_topo)[cid];
> +	*out__uninit = topo[cid];
>  }
>  
>  __bpf_kfunc_end_defs();
> diff --git a/kernel/sched/ext/cid.h b/kernel/sched/ext/cid.h
> index b36a1a28eac8..1498efb81549 100644
> --- a/kernel/sched/ext/cid.h
> +++ b/kernel/sched/ext/cid.h
> @@ -90,9 +90,10 @@ static inline bool cid_valid(struct scx_sched *sch, s32 cid)
>   * __scx_cid_to_cpu - Unchecked cid->cpu table lookup
>   * @cid: cid to look up. Must be in [0, num_possible_cpus()).
>   *
> - * Intended for callsites that have already validated @cid and that hold a
> - * non-NULL @sch from scx_prog_sched() - a live sched implies the table has
> - * been allocated, so no NULL check is needed here.
> + * Intended for callsites that have already validated @cid and where the
> + * tables are guaranteed allocated - ops invocations on a live scheduler or
> + * the enable path itself. Prog-facing kfuncs, which can run while the first
> + * enable is still allocating the tables, use the checked wrappers instead.
>   */
>  static inline s32 __scx_cid_to_cpu(s32 cid)
>  {
> @@ -119,13 +120,17 @@ static inline s32 __scx_cpu_to_cid(s32 cpu)
>   * Return the cpu for @cid or a negative errno on failure. Invalid cid triggers
>   * scx_error() on @sch. The cid arrays are allocated on first scheduler enable
>   * and never freed, so the returned cpu is stable for the lifetime of the loaded
> - * scheduler.
> + * scheduler. Return -EINVAL without triggering scx_error() if the tables are
> + * not allocated yet, which a prog-facing kfunc can observe while racing the
> + * first enable.
>   */
>  static inline s32 scx_cid_to_cpu(struct scx_sched *sch, s32 cid)
>  {
> -	if (!cid_valid(sch, cid))
> +	s16 *tbl = READ_ONCE(scx_cid_to_cpu_tbl);
> +
> +	if (!cid_valid(sch, cid) || unlikely(!tbl))
>  		return -EINVAL;
> -	return __scx_cid_to_cpu(cid);
> +	return tbl[cid];
>  }
>  
>  /**
> @@ -138,9 +143,11 @@ static inline s32 scx_cid_to_cpu(struct scx_sched *sch, s32 cid)
>   */
>  static inline s32 scx_cpu_to_cid(struct scx_sched *sch, s32 cpu)
>  {
> -	if (!scx_cpu_valid(sch, cpu, NULL))
> +	s16 *tbl = READ_ONCE(scx_cpu_to_cid_tbl);
> +
> +	if (!scx_cpu_valid(sch, cpu, NULL) || unlikely(!tbl))
>  		return -EINVAL;
> -	return __scx_cpu_to_cid(cpu);
> +	return tbl[cpu];
>  }
>  
>  /**
> -- 
> 2.55.0
>