* [PATCH sched_ext/for-7.1] tools/sched_ext: Kick idle CPU for pinned tasks in scx_qmap @ 2026-04-11 11:33 Tejun Heo 2026-04-11 12:57 ` Cheng-Yang Chou 2026-04-11 15:03 ` Andrea Righi 0 siblings, 2 replies; 4+ messages in thread From: Tejun Heo @ 2026-04-11 11:33 UTC (permalink / raw) To: David Vernet, Andrea Righi, Changwoo Min Cc: Emil Tsalapatis, sched-ext, linux-kernel scx_qmap uses global BPF queue maps for task dispatch. A task pinned to a single CPU can only be dispatched by its home CPU's ops.dispatch(), but an idle CPU won't call ops.dispatch() on its own. This leaves per-CPU kthreads like ksoftirqd stranded, causing NOHZ tick-stop warnings from pending softirqs. Kick the target CPU with SCX_KICK_IDLE when enqueueing a pinned task. Signed-off-by: Tejun Heo <tj@kernel.org> --- tools/sched_ext/scx_qmap.bpf.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tools/sched_ext/scx_qmap.bpf.c b/tools/sched_ext/scx_qmap.bpf.c index f3587fb709c9..09d1624fb869 100644 --- a/tools/sched_ext/scx_qmap.bpf.c +++ b/tools/sched_ext/scx_qmap.bpf.c @@ -314,6 +314,14 @@ void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags) __sync_fetch_and_add(&nr_highpri_queued, 1); } __sync_fetch_and_add(&nr_enqueued, 1); + + /* + * Kick idle target CPU for pinned tasks. Without this, the CPU can + * idle while ksoftirqd is pending in the BPF queue, triggering NOHZ + * tick-stop warnings. + */ + if (p->nr_cpus_allowed == 1) + scx_bpf_kick_cpu(scx_bpf_task_cpu(p), SCX_KICK_IDLE); } /* -- 2.53.0 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH sched_ext/for-7.1] tools/sched_ext: Kick idle CPU for pinned tasks in scx_qmap 2026-04-11 11:33 [PATCH sched_ext/for-7.1] tools/sched_ext: Kick idle CPU for pinned tasks in scx_qmap Tejun Heo @ 2026-04-11 12:57 ` Cheng-Yang Chou 2026-04-11 14:27 ` Cheng-Yang Chou 2026-04-11 15:03 ` Andrea Righi 1 sibling, 1 reply; 4+ messages in thread From: Cheng-Yang Chou @ 2026-04-11 12:57 UTC (permalink / raw) To: Tejun Heo Cc: David Vernet, Andrea Righi, Changwoo Min, Emil Tsalapatis, sched-ext, linux-kernel Hi Tejun, On Sat, Apr 11, 2026 at 01:33:56AM -1000, Tejun Heo wrote: > scx_qmap uses global BPF queue maps for task dispatch. A task pinned to a > single CPU can only be dispatched by its home CPU's ops.dispatch(), but an > idle CPU won't call ops.dispatch() on its own. This leaves per-CPU kthreads > like ksoftirqd stranded, causing NOHZ tick-stop warnings from pending > softirqs. > > Kick the target CPU with SCX_KICK_IDLE when enqueueing a pinned task. > > Signed-off-by: Tejun Heo <tj@kernel.org> > --- > tools/sched_ext/scx_qmap.bpf.c | 8 ++++++++ > 1 file changed, 8 insertions(+) > > diff --git a/tools/sched_ext/scx_qmap.bpf.c b/tools/sched_ext/scx_qmap.bpf.c > index f3587fb709c9..09d1624fb869 100644 > --- a/tools/sched_ext/scx_qmap.bpf.c > +++ b/tools/sched_ext/scx_qmap.bpf.c > @@ -314,6 +314,14 @@ void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags) > __sync_fetch_and_add(&nr_highpri_queued, 1); > } > __sync_fetch_and_add(&nr_enqueued, 1); > + > + /* > + * Kick idle target CPU for pinned tasks. Without this, the CPU can > + * idle while ksoftirqd is pending in the BPF queue, triggering NOHZ > + * tick-stop warnings. > + */ > + if (p->nr_cpus_allowed == 1) > + scx_bpf_kick_cpu(scx_bpf_task_cpu(p), SCX_KICK_IDLE); > } > > /* > -- > 2.53.0 Looks good to me! The same issue exists in scx_userland where pinned tasks can be dispatched to SCX_DSQ_GLOBAL without kicking the idle target CPU. I'll follow a patch to add the same fix there! Reviewed-by: Cheng-Yang Chou <yphbchou0911@gmail.com> -- Thanks, Cheng-Yang ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH sched_ext/for-7.1] tools/sched_ext: Kick idle CPU for pinned tasks in scx_qmap 2026-04-11 12:57 ` Cheng-Yang Chou @ 2026-04-11 14:27 ` Cheng-Yang Chou 0 siblings, 0 replies; 4+ messages in thread From: Cheng-Yang Chou @ 2026-04-11 14:27 UTC (permalink / raw) To: Tejun Heo Cc: David Vernet, Andrea Righi, Changwoo Min, Emil Tsalapatis, sched-ext, linux-kernel, Ching-Chun Huang, Chia-Ping Tsai Hi Tejun, On Sat, Apr 11, 2026 at 08:57:42PM +0800, Cheng-Yang Chou wrote: > On Sat, Apr 11, 2026 at 01:33:56AM -1000, Tejun Heo wrote: > > scx_qmap uses global BPF queue maps for task dispatch. A task pinned to a > > single CPU can only be dispatched by its home CPU's ops.dispatch(), but an > > idle CPU won't call ops.dispatch() on its own. This leaves per-CPU kthreads > > like ksoftirqd stranded, causing NOHZ tick-stop warnings from pending > > softirqs. > > > > Kick the target CPU with SCX_KICK_IDLE when enqueueing a pinned task. > > > > Signed-off-by: Tejun Heo <tj@kernel.org> > > --- > > tools/sched_ext/scx_qmap.bpf.c | 8 ++++++++ > > 1 file changed, 8 insertions(+) > > > > diff --git a/tools/sched_ext/scx_qmap.bpf.c b/tools/sched_ext/scx_qmap.bpf.c > > index f3587fb709c9..09d1624fb869 100644 > > --- a/tools/sched_ext/scx_qmap.bpf.c > > +++ b/tools/sched_ext/scx_qmap.bpf.c > > @@ -314,6 +314,14 @@ void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags) > > __sync_fetch_and_add(&nr_highpri_queued, 1); > > } > > __sync_fetch_and_add(&nr_enqueued, 1); > > + > > + /* > > + * Kick idle target CPU for pinned tasks. Without this, the CPU can > > + * idle while ksoftirqd is pending in the BPF queue, triggering NOHZ > > + * tick-stop warnings. > > + */ > > + if (p->nr_cpus_allowed == 1) > > + scx_bpf_kick_cpu(scx_bpf_task_cpu(p), SCX_KICK_IDLE); > > } > > > > /* > > -- > > 2.53.0 > > Looks good to me! The same issue exists in scx_userland where pinned > tasks can be dispatched to SCX_DSQ_GLOBAL without kicking the idle > target CPU. I'll follow a patch to add the same fix there! > > Reviewed-by: Cheng-Yang Chou <yphbchou0911@gmail.com> Actually, would it make more sense to fold this directly into the same patch? diff --git a/tools/sched_ext/scx_userland.bpf.c b/tools/sched_ext/scx_userland.bpf.c index f29862b89386..56c53d457f45 100644 --- a/tools/sched_ext/scx_userland.bpf.c +++ b/tools/sched_ext/scx_userland.bpf.c @@ -195,6 +195,14 @@ static void enqueue_task_in_user_space(struct task_struct *p, u64 enq_flags) */ __sync_fetch_and_add(&nr_failed_enqueues, 1); scx_bpf_dsq_insert(p, SCX_DSQ_GLOBAL, SCX_SLICE_DFL, enq_flags); + + /* + * Kick idle target CPU for pinned tasks. Without this, the CPU can + * idle while ksoftirqd is pending in the BPF queue, triggering NOHZ + * tick-stop warnings. + */ + if (p->nr_cpus_allowed == 1) + scx_bpf_kick_cpu(scx_bpf_task_cpu(p), SCX_KICK_IDLE); } else { __sync_fetch_and_add(&nr_user_enqueues, 1); set_usersched_needed(); -- Thanks, Cheng-Yang ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH sched_ext/for-7.1] tools/sched_ext: Kick idle CPU for pinned tasks in scx_qmap 2026-04-11 11:33 [PATCH sched_ext/for-7.1] tools/sched_ext: Kick idle CPU for pinned tasks in scx_qmap Tejun Heo 2026-04-11 12:57 ` Cheng-Yang Chou @ 2026-04-11 15:03 ` Andrea Righi 1 sibling, 0 replies; 4+ messages in thread From: Andrea Righi @ 2026-04-11 15:03 UTC (permalink / raw) To: Tejun Heo Cc: David Vernet, Changwoo Min, Emil Tsalapatis, sched-ext, linux-kernel Hi Tejun, On Sat, Apr 11, 2026 at 01:33:56AM -1000, Tejun Heo wrote: > scx_qmap uses global BPF queue maps for task dispatch. A task pinned to a > single CPU can only be dispatched by its home CPU's ops.dispatch(), but an > idle CPU won't call ops.dispatch() on its own. This leaves per-CPU kthreads > like ksoftirqd stranded, causing NOHZ tick-stop warnings from pending > softirqs. > > Kick the target CPU with SCX_KICK_IDLE when enqueueing a pinned task. > > Signed-off-by: Tejun Heo <tj@kernel.org> > --- > tools/sched_ext/scx_qmap.bpf.c | 8 ++++++++ > 1 file changed, 8 insertions(+) > > diff --git a/tools/sched_ext/scx_qmap.bpf.c b/tools/sched_ext/scx_qmap.bpf.c > index f3587fb709c9..09d1624fb869 100644 > --- a/tools/sched_ext/scx_qmap.bpf.c > +++ b/tools/sched_ext/scx_qmap.bpf.c > @@ -314,6 +314,14 @@ void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags) > __sync_fetch_and_add(&nr_highpri_queued, 1); > } > __sync_fetch_and_add(&nr_enqueued, 1); > + > + /* > + * Kick idle target CPU for pinned tasks. Without this, the CPU can > + * idle while ksoftirqd is pending in the BPF queue, triggering NOHZ > + * tick-stop warnings. > + */ > + if (p->nr_cpus_allowed == 1) > + scx_bpf_kick_cpu(scx_bpf_task_cpu(p), SCX_KICK_IDLE); I think we should kick the task's CPU in general, also when p->nr_cpus_allowed == N, with N < nr_cpus_ids, otherwise we can have the same problem if one of the N allowed CPUs is never awakened. Moreover, tasks will have a better chance to keep running on the same CPU, which is nice, unless we want to limit the amount of CPU wakeups. If we want to be fancy we could even do something like this: if (!__COMPAT_is_enq_cpu_selected(enq_flags) && !scx_bpf_task_running(p)) scx_bpf_kick_cpu(scx_bpf_task_cpu(p), SCX_KICK_IDLE); In practice: if ops.select_cpu() was called there's no reason to do the kick, because it's supposed to be done already in ops.select_cpu(). Simiarly, if we've a queued wakeup event, ops.select_cpu() was skipped, so we should explicitly kick the CPU if the task was enqueued and it wasn't running already. Thanks, -Andrea ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-04-11 15:03 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-04-11 11:33 [PATCH sched_ext/for-7.1] tools/sched_ext: Kick idle CPU for pinned tasks in scx_qmap Tejun Heo 2026-04-11 12:57 ` Cheng-Yang Chou 2026-04-11 14:27 ` Cheng-Yang Chou 2026-04-11 15:03 ` Andrea Righi
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox