[PATCHSET sched_ext/for-7.5] sched_ext: scx_qmap: Drop the compat code and fix a lowpri starvation

[PATCHSET sched_ext/for-7.5] sched_ext: scx_qmap: Drop the compat code and fix a lowpri starvation

[Tejun Heo]

Hello,

scx_qmap is often used to exercise sched_ext features as they land and
already requires the kernel it ships with, so its compat code only keeps
untested fallbacks around. The first patch drops it.

The second fixes scx_qmap's lowpri path. Nice 19 tasks coming back from
LOWPRI_DSQ go to SHARED_DSQ, which is consumed ahead of the FIFOs, so enough
nice 19 CPU hogs starve every other task until the watchdog ejects the
scheduler. They now go to their FIFO instead.

Tested with root and sub-scheduler instances under nice 0 and nice 19 CPU
hogs. Without the fix, the root is ejected within seconds. With it, the runs
complete and the FIFOs keep being dispatched.

Based on sched_ext/for-7.4-fixes (ff343a94e44d).

This patchset contains the following 2 patches.

 0001 sched_ext: scx_qmap: Drop the compat code
 0002 sched_ext: scx_qmap: Queue re-enqueued lowpri tasks on their FIFO

The patchset is also available in the following git branch:

 git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext.git qmap-nocompat

diffstat follows. Thanks.

 tools/sched_ext/scx_qmap.bpf.c | 40 +++++++++++++++++++++++-----------------
 tools/sched_ext/scx_qmap.c     | 14 +++++---------
 2 files changed, 28 insertions(+), 26 deletions(-)

--
tejun

[PATCH 1/2] sched_ext: scx_qmap: Drop the compat code

[Tejun Heo]

scx_qmap demonstrates and tests the sched_ext features of the kernel it
ships with. It already requires that kernel, as it implements
ops.sub_cid_sched_updated() and ops.sub_child_ecaps_updated(), so its compat
checks only keep untested fallbacks around. Drop them, use the kernel
interfaces directly, and note the coupling in the top comment. No functional
change on a current kernel.

Signed-off-by: Tejun Heo <tj@kernel.org>
---
 tools/sched_ext/scx_qmap.bpf.c | 31 ++++++++++++++++---------------
 tools/sched_ext/scx_qmap.c     | 14 +++++---------
 2 files changed, 21 insertions(+), 24 deletions(-)

diff --git a/tools/sched_ext/scx_qmap.bpf.c b/tools/sched_ext/scx_qmap.bpf.c
index 63e3c9080e77..77f02fb70a86 100644
--- a/tools/sched_ext/scx_qmap.bpf.c
+++ b/tools/sched_ext/scx_qmap.bpf.c
@@ -10,6 +10,10 @@
  * - Core-sched support.
  * - Hierarchical sub-scheduling: delegating cpus to child cgroup schedulers.
  *
+ * scx_qmap is often used to exercise sched_ext features as they land. It is
+ * coupled with the kernel it ships with and uses its sched_ext interfaces
+ * directly. It isn't expected to load on other kernels.
+ *
  * Base design: Five FIFOs (arena-backed doubly-linked lists through per-task
  * context). A task is assigned to a FIFO by its compound weight. Each cpu
  * round-robins the FIFOs, dispatching more from higher ones.
@@ -583,14 +587,13 @@ void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags)
 	}
 
 	/* see lowpri_timerfn() */
-	if (__COMPAT_has_generic_reenq() &&
-	    p->scx.weight < 2 && !(p->flags & PF_KTHREAD) && !(enq_flags & SCX_ENQ_REENQ)) {
+	if (p->scx.weight < 2 && !(p->flags & PF_KTHREAD) && !(enq_flags & SCX_ENQ_REENQ)) {
 		scx_bpf_dsq_insert(p, LOWPRI_DSQ, slice_ns, enq_flags);
 		return;
 	}
 
 	/* if select_cid() wasn't called, try direct dispatch */
-	if (!__COMPAT_is_enq_cpu_selected(enq_flags) &&
+	if (!(enq_flags & SCX_ENQ_CPU_SELECTED) &&
 	    (cid = pick_direct_dispatch_cid(p, scx_bpf_task_cid(p), taskc)) >= 0) {
 		__sync_fetch_and_add(&qa.nr_ddsp_from_enq, 1);
 		scx_bpf_dsq_insert(p, SCX_DSQ_LOCAL_ON | cid, slice_ns,
@@ -1316,7 +1319,7 @@ static int monitor_timerfn(void *map, int *key, struct bpf_timer *timer)
 
 		dump_shared_dsq();
 
-		__COMPAT_scx_bpf_events(&events, sizeof(events));
+		scx_bpf_events(&events, sizeof(events));
 
 		bpf_printk("%35s: %lld", "SCX_EV_SELECT_CPU_FALLBACK",
 			   scx_read_event(&events, SCX_EV_SELECT_CPU_FALLBACK));
@@ -2097,17 +2100,15 @@ s32 BPF_STRUCT_OPS_SLEEPABLE(qmap_init)
 	if (ret)
 		return ret;
 
-	if (__COMPAT_has_generic_reenq()) {
-		/* see lowpri_timerfn() */
-		timer = bpf_map_lookup_elem(&lowpri_timer, &key);
-		if (!timer)
-			return -ESRCH;
-		bpf_timer_init(timer, &lowpri_timer, CLOCK_MONOTONIC);
-		bpf_timer_set_callback(timer, lowpri_timerfn);
-		ret = bpf_timer_start(timer, LOWPRI_INTV_NS, 0);
-		if (ret)
-			return ret;
-	}
+	/* see lowpri_timerfn() */
+	timer = bpf_map_lookup_elem(&lowpri_timer, &key);
+	if (!timer)
+		return -ESRCH;
+	bpf_timer_init(timer, &lowpri_timer, CLOCK_MONOTONIC);
+	bpf_timer_set_callback(timer, lowpri_timerfn);
+	ret = bpf_timer_start(timer, LOWPRI_INTV_NS, 0);
+	if (ret)
+		return ret;
 
 	/* sub-sched: drive the boundary-cid round-robin from a bpf timer */
 	timer = bpf_map_lookup_elem(&round_robin_timer, &key);
diff --git a/tools/sched_ext/scx_qmap.c b/tools/sched_ext/scx_qmap.c
index 17cf584a2a3d..9b1cbe187ebd 100644
--- a/tools/sched_ext/scx_qmap.c
+++ b/tools/sched_ext/scx_qmap.c
@@ -294,7 +294,7 @@ int main(int argc, char **argv)
 	optind = 1;
 	skel = SCX_OPS_CID_OPEN(qmap_ops, scx_qmap);
 
-	skel->rodata->slice_ns = __COMPAT_ENUM_OR_ZERO("scx_public_consts", "SCX_SLICE_DFL");
+	skel->rodata->slice_ns = skel->rodata->__SCX_SLICE_DFL;
 	skel->rodata->max_tasks = 16384;
 
 	while ((opt = getopt(argc, argv,
@@ -512,14 +512,10 @@ int main(int argc, char **argv)
 		       (unsigned long long)qa->nr_expedited_remote,
 		       (unsigned long long)qa->nr_expedited_from_timer,
 		       (unsigned long long)qa->nr_expedited_lost);
-		if (__COMPAT_has_ksym("scx_bpf_cidperf_cur"))
-			printf("cpuperf: cur min/avg/max=%u/%u/%u target min/avg/max=%u/%u/%u\n",
-			       qa->cpuperf_min,
-			       qa->cpuperf_avg,
-			       qa->cpuperf_max,
-			       qa->cpuperf_target_min,
-			       qa->cpuperf_target_avg,
-			       qa->cpuperf_target_max);
+		printf("cpuperf: cur min/avg/max=%u/%u/%u target min/avg/max=%u/%u/%u\n",
+		       qa->cpuperf_min, qa->cpuperf_avg, qa->cpuperf_max,
+		       qa->cpuperf_target_min, qa->cpuperf_target_avg,
+		       qa->cpuperf_target_max);
 
 		invoke_flush_alloc(skel);
 		print_hier(qa, &hprev, own_cgid);
-- 
2.56.0

[PATCH 2/2] sched_ext: scx_qmap: Queue re-enqueued lowpri tasks on their FIFO

[Tejun Heo]

scx_qmap holds nice 19 tasks on LOWPRI_DSQ, and lowpri_timerfn() re-enqueues
them every 10ms. The re-enqueued tasks take qmap_enqueue()'s SCX_ENQ_REENQ
path to SHARED_DSQ, which qmap_dispatch() consumes before popping the FIFOs,
so enough nice 19 tasks starve every other task:

1. A nice 19 task's slice expires and qmap_enqueue() puts it on LOWPRI_DSQ.
2. Within 10ms, lowpri_timerfn() re-enqueues it onto SHARED_DSQ.
3. qmap_dispatch() finds SHARED_DSQ non-empty and doesn't pop the FIFOs.

With a dozen nice 19 CPU hogs, no FIFO was popped for over 5 seconds, and
the root was ejected as the watchdog work, whose kworker waited on a FIFO,
failed to check in.

Queue a re-enqueued lowpri task on its FIFO instead, where it is dispatched
in its FIFO's turn after its LOWPRI_DSQ wait.

Fixes: 84b1a0ea0b7c ("sched_ext: Implement scx_bpf_dsq_reenq() for user DSQs")
Signed-off-by: Tejun Heo <tj@kernel.org>
---
 tools/sched_ext/scx_qmap.bpf.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/tools/sched_ext/scx_qmap.bpf.c b/tools/sched_ext/scx_qmap.bpf.c
index 77f02fb70a86..3951f1f6e3c6 100644
--- a/tools/sched_ext/scx_qmap.bpf.c
+++ b/tools/sched_ext/scx_qmap.bpf.c
@@ -427,6 +427,7 @@ void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags)
 	static u32 user_cnt, kernel_cnt;
 	task_ctx_t *taskc;
 	int idx = weight_to_idx(p->scx.weight);
+	bool lowpri = p->scx.weight < 2 && !(p->flags & PF_KTHREAD);
 	s32 cid;
 
 	if (enq_flags & SCX_ENQ_REENQ) {
@@ -587,7 +588,7 @@ void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags)
 	}
 
 	/* see lowpri_timerfn() */
-	if (p->scx.weight < 2 && !(p->flags & PF_KTHREAD) && !(enq_flags & SCX_ENQ_REENQ)) {
+	if (lowpri && !(enq_flags & SCX_ENQ_REENQ)) {
 		scx_bpf_dsq_insert(p, LOWPRI_DSQ, slice_ns, enq_flags);
 		return;
 	}
@@ -606,8 +607,12 @@ void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags)
 	 * higher priority scheduling class, just re-enqueue the task directly
 	 * on the global DSQ. As we want another CPU to pick it up, find and
 	 * kick an idle cid.
+	 *
+	 * A lowpri task goes to its FIFO instead. qmap_dispatch() consumes
+	 * SHARED_DSQ ahead of the FIFOs, and enough lowpri tasks coming back
+	 * from LOWPRI_DSQ would keep it non-empty and starve every other task.
 	 */
-	if (enq_flags & SCX_ENQ_REENQ) {
+	if ((enq_flags & SCX_ENQ_REENQ) && !lowpri) {
 		s32 cid;
 
 		scx_bpf_dsq_insert(p, SHARED_DSQ, 0, enq_flags);
@@ -1357,7 +1362,7 @@ struct {
 
 /*
  * Nice 19 tasks are put into the lowpri DSQ. Every 10ms, reenq is triggered and
- * the tasks are transferred to SHARED_DSQ.
+ * qmap_enqueue() queues the tasks on their FIFO.
  */
 static int lowpri_timerfn(void *map, int *key, struct bpf_timer *timer)
 {
-- 
2.56.0

Re: [PATCHSET sched_ext/for-7.5] sched_ext: scx_qmap: Drop the compat code and fix a lowpri starvation

[Andrea Righi]

On Mon, Oct 19, 2026 at 11:32:19AM -1000, Tejun Heo wrote:
> Hello,
> 
> scx_qmap is often used to exercise sched_ext features as they land and
> already requires the kernel it ships with, so its compat code only keeps
> untested fallbacks around. The first patch drops it.
> 
> The second fixes scx_qmap's lowpri path. Nice 19 tasks coming back from
> LOWPRI_DSQ go to SHARED_DSQ, which is consumed ahead of the FIFOs, so enough
> nice 19 CPU hogs starve every other task until the watchdog ejects the
> scheduler. They now go to their FIFO instead.
> 
> Tested with root and sub-scheduler instances under nice 0 and nice 19 CPU
> hogs. Without the fix, the root is ejected within seconds. With it, the runs
> complete and the FIFOs keep being dispatched.
> 
> Based on sched_ext/for-7.4-fixes (ff343a94e44d).
> 
> This patchset contains the following 2 patches.
> 
>  0001 sched_ext: scx_qmap: Drop the compat code
>  0002 sched_ext: scx_qmap: Queue re-enqueued lowpri tasks on their FIFO

Both look good to me.

Reviewed-by: Andrea Righi <arighi@nvidia.com>

Thanks,
-Andrea

Re: [PATCHSET sched_ext/for-7.5] sched_ext: scx_qmap: Drop the compat code and fix a lowpri starvation

[Andrea Righi]

On Tue, Oct 20, 2026 at 12:00:29AM +0200, Andrea Righi wrote:
> On Mon, Oct 19, 2026 at 11:32:19AM -1000, Tejun Heo wrote:
> > Hello,
> > 
> > scx_qmap is often used to exercise sched_ext features as they land and
> > already requires the kernel it ships with, so its compat code only keeps
> > untested fallbacks around. The first patch drops it.
> > 
> > The second fixes scx_qmap's lowpri path. Nice 19 tasks coming back from
> > LOWPRI_DSQ go to SHARED_DSQ, which is consumed ahead of the FIFOs, so enough
> > nice 19 CPU hogs starve every other task until the watchdog ejects the
> > scheduler. They now go to their FIFO instead.
> > 
> > Tested with root and sub-scheduler instances under nice 0 and nice 19 CPU
> > hogs. Without the fix, the root is ejected within seconds. With it, the runs
> > complete and the FIFOs keep being dispatched.
> > 
> > Based on sched_ext/for-7.4-fixes (ff343a94e44d).
> > 
> > This patchset contains the following 2 patches.
> > 
> >  0001 sched_ext: scx_qmap: Drop the compat code
> >  0002 sched_ext: scx_qmap: Queue re-enqueued lowpri tasks on their FIFO
> 
> Both look good to me.
> 
> Reviewed-by: Andrea Righi <arighi@nvidia.com>

Oh... and as I sent this, I just noticed a small nit, there's still one
compat in scx_qmap.c:

    ppt = __COMPAT_ENUM_OR_ZERO("scx_consts", "SCX_RESCUE_DISABLE");

We can probably drop this as well.

Thanks,
-Andrea