Sched_ext development
 help / color / mirror / Atom feed
* [PATCH] tools/sched_ext: Handle pinned tasks in scx_central
@ 2026-04-13 11:04 Cheng-Yang Chou
  2026-04-13 15:29 ` Kuba Piecuch
  0 siblings, 1 reply; 10+ messages in thread
From: Cheng-Yang Chou @ 2026-04-13 11:04 UTC (permalink / raw)
  To: sched-ext, Tejun Heo, David Vernet, Andrea Righi, Changwoo Min
  Cc: Ching-Chun Huang, Chia-Ping Tsai, yphbchou0911

scx_central only fast-pathed per-cpu kthreads (PF_KTHREAD &&
nr_cpus_allowed == 1) directly to the local DSQ, leaving non-kthread
tasks pinned to a single CPU to go through the central queue. When
dispatch_to_cpu() later dequeued such a task and found it couldn't run
on the current CPU, it could trigger scx_error() and disable the
scheduler:

  sched_ext: central: SCX_DSQ_LOCAL[_ON] cannot move migration disabled qemu-system-x86[223212] from CPU 1 to 0

Though scx_central is a demo scheduler and meant to dispatch all tasks
from a central CPU, it should remain robust under any circumstances.

Extend the handling to all tasks with nr_cpus_allowed == 1:
- select_cpu() and enqueue(): return the task's current CPU and dispatch
  directly to local DSQ for pinned tasks.
- dispatch_to_cpu(): use SCX_KICK_PREEMPT when kicking a task's home
  CPU after bouncing to the fallback DSQ, so the CPU preempts
  immediately rather than waiting for the next timer tick.

Test plan:
- run scx_central on the host, then launch virtme-ng in a separate
  terminal to create pinned tasks (qemu-system-x86), and verify stable
  operation.

Signed-off-by: Cheng-Yang Chou <yphbchou0911@gmail.com>
---
 tools/sched_ext/scx_central.bpf.c | 24 +++++++++++++++++++-----
 1 file changed, 19 insertions(+), 5 deletions(-)

diff --git a/tools/sched_ext/scx_central.bpf.c b/tools/sched_ext/scx_central.bpf.c
index 4efcce099bd5..5a02669483cc 100644
--- a/tools/sched_ext/scx_central.bpf.c
+++ b/tools/sched_ext/scx_central.bpf.c
@@ -96,7 +96,13 @@ s32 BPF_STRUCT_OPS(central_select_cpu, struct task_struct *p,
 	 * disturbing other CPUs. It's safe to blindly return the central cpu as
 	 * select_cpu() is a hint and if @p can't be on it, the kernel will
 	 * automatically pick a fallback CPU.
+	 *
+	 * For tasks pinned to a single CPU, return the current CPU to avoid
+	 * an unnecessary fallback.
 	 */
+	if (p->nr_cpus_allowed == 1)
+		return scx_bpf_task_cpu(p);
+
 	return central_cpu;
 }
 
@@ -107,12 +113,13 @@ void BPF_STRUCT_OPS(central_enqueue, struct task_struct *p, u64 enq_flags)
 	__sync_fetch_and_add(&nr_total, 1);
 
 	/*
-	 * Push per-cpu kthreads at the head of local dsq's and preempt the
-	 * corresponding CPU. This ensures that e.g. ksoftirqd isn't blocked
-	 * behind other threads which is necessary for forward progress
-	 * guarantee as we depend on the BPF timer which may run from ksoftirqd.
+	 * Push tasks pinned to a single CPU at the head of local dsq's and
+	 * preempt the corresponding CPU. This ensures that e.g. ksoftirqd
+	 * isn't blocked behind other threads which is necessary for forward
+	 * progress guarantee as we depend on the BPF timer which may run
+	 * from ksoftirqd.
 	 */
-	if ((p->flags & PF_KTHREAD) && p->nr_cpus_allowed == 1) {
+	if (p->nr_cpus_allowed == 1) {
 		__sync_fetch_and_add(&nr_locals, 1);
 		scx_bpf_dsq_insert(p, SCX_DSQ_LOCAL, SCX_SLICE_INF,
 				   enq_flags | SCX_ENQ_PREEMPT);
@@ -155,6 +162,13 @@ static bool dispatch_to_cpu(s32 cpu)
 		if (!bpf_cpumask_test_cpu(cpu, p->cpus_ptr)) {
 			__sync_fetch_and_add(&nr_mismatches, 1);
 			scx_bpf_dsq_insert(p, FALLBACK_DSQ_ID, SCX_SLICE_INF, 0);
+
+			/*
+			 * Kick the task's home CPU to pick it up from the
+			 * fallback DSQ promptly.
+			 */
+			scx_bpf_kick_cpu(scx_bpf_task_cpu(p), SCX_KICK_PREEMPT);
+
 			bpf_task_release(p);
 			/*
 			 * We might run out of dispatch buffer slots if we continue dispatching
-- 
2.48.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2026-04-17 18:38 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-04-13 11:04 [PATCH] tools/sched_ext: Handle pinned tasks in scx_central Cheng-Yang Chou
2026-04-13 15:29 ` Kuba Piecuch
2026-04-14  3:45   ` Cheng-Yang Chou
2026-04-14 17:19     ` Tejun Heo
2026-04-15  6:17       ` Cheng-Yang Chou
2026-04-15  9:59         ` Kuba Piecuch
2026-04-15 10:07           ` Cheng-Yang Chou
2026-04-17 18:38         ` Tejun Heo
2026-04-14 21:01     ` Andrea Righi
2026-04-15  6:02       ` Cheng-Yang Chou

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox