[PATCH 0/2] sched_ext: two more context-safety fixes found in the NMI kfunc audit

[PATCH 0/2] sched_ext: two more context-safety fixes found in the NMI kfunc audit

[Wanwu Li]

Continuing the audit that the sashiko bot kicked off on the NMI-reject
series, I went through every kfunc exposed to BPF_PROG_TYPE_TRACING
(the any / idle / cid context-filter sets). Two more context-safety
issues came out of it; this pair addresses both.

 - Patch 1 fixes the scx_locked_rq() class (three "any"-category kfuncs
   read scx_locked_rq() and take an unsafe fast path on a non-NULL
   return from NMI). scx_locked_rq() gains an in_nmi() test at the
   source that returns NULL from NMI, so the three callers fall onto
   their unlocked paths. This supersedes guarding them individually.

 - Patch 2 fixes an IRQ re-entrancy race on the per-CPU
   per_cpu_unvisited nodemask scratch in the idle path. The search
   scratch is protected only by preempt_disable(), which doesn't mask
   IRQs; switch to irqsave instead. The NMI case is left as-is (a
   misuse with no legitimate use and no crash risk).

Wanwu Li (2):
  sched_ext: Make scx_locked_rq() return NULL from NMI
  sched_ext: Protect the idle-search scratch nodemask with irqsave

 kernel/sched/ext/idle.c     | 14 ++++++++++++--
 kernel/sched/ext/internal.h |  9 +++++++++
 2 files changed, 21 insertions(+), 2 deletions(-)

-- 
2.25.1

[PATCH 1/2] sched_ext: Make scx_locked_rq() return NULL from NMI

[Wanwu Li]

scx_locked_rq() reads the per-CPU scx_locked_rq_state, which tracks the
rq locked by the context running on the CPU. Tracing progs can call
kfuncs from NMI, and an NMI interrupts - rather than replaces - the
context that set scx_locked_rq_state, so a non-NULL read from NMI
falsely tells the caller that it holds the interrupted context's rq
lock.

Three "any"-category kfuncs read scx_locked_rq() on their success path
and take an unsafe fast path on a non-NULL return:

 - scx_bpf_task_set_slice() writes p->scx.slice directly, racing
   update_curr_scx()'s non-atomic read-modify-write of the same field.
 - scx_bpf_dsq_nr_queued() resolves %SCX_DSQ_LOCAL to
   (scx_locked_rq() ?: this_rq()) and can report the interrupted
   context's local DSQ length instead of the caller's.
 - scx_bpf_locked_rq() hands the interrupted context's rq to the BPF
   program, which may then operate on it as if it owned the rq lock.

Make scx_locked_rq() return NULL from NMI so that all three take their
unlocked paths: scx_bpf_task_set_slice() stashes the request into the
atomic p->scx.slice_oob for application under the rq lock,
scx_bpf_dsq_nr_queued() falls back to this_rq(), and
scx_bpf_locked_rq() reports an error and aborts the scheduler through
the NMI-safe exit path.

The kfuncs that take scheduler locks reject NMI calls through
scx_kf_allowed_ctx() before reaching scx_locked_rq(), and the internal
callers only run from struct_ops callbacks, which never run in NMI, so
no other caller is affected.

Suggested-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/r/d84b31727f04e1ed0d40042ba1c09e61@kernel.org
Signed-off-by: Wanwu Li <liwanwu@kylinos.cn>
---
 kernel/sched/ext/internal.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/kernel/sched/ext/internal.h b/kernel/sched/ext/internal.h
index a67277b0fee6..809e0ee0fd5f 100644
--- a/kernel/sched/ext/internal.h
+++ b/kernel/sched/ext/internal.h
@@ -2218,6 +2218,15 @@ static inline void scx_schedule_reenq_local(struct rq *rq, u64 reenq_flags)
  */
 static inline struct rq *scx_locked_rq(void)
 {
+	/*
+	 * Tracing progs can call kfuncs from NMI. scx_locked_rq_state tracks
+	 * the rq locked by the interrupted context, so a non-NULL read from
+	 * NMI would falsely claim its lock. Return NULL from NMI so that
+	 * callers take their unlocked paths.
+	 */
+	if (unlikely(in_nmi()))
+		return NULL;
+
 	return __this_cpu_read(scx_locked_rq_state);
 }
 
-- 
2.25.1

Re: [PATCH 1/2] sched_ext: Make scx_locked_rq() return NULL from NMI

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [High] scx_bpf_cpuperf_set() and scx_bpf_cidperf_set() deadlock in NMI when scx_locked_rq() returns NULL.

Pre-existing issues:
- [High] Tracing BPF programs executing in NMI context can deadlock the system by calling pre-existing 'any' category kfuncs that acquire raw spinlocks.
--

commit e3a4b62825e5b60c4d87fa4b8dd2f4b8dee65bbf
Author: Wanwu Li <liwanwu@kylinos.cn>

sched_ext: Make scx_locked_rq() return NULL from NMI

scx_locked_rq() reads the per-CPU scx_locked_rq_state, which tracks the
rq locked by the context running on the CPU. Tracing progs can call
kfuncs from NMI, and an NMI interrupts - rather than replaces - the
context that set scx_locked_rq_state, so a non-NULL read from NMI
falsely tells the caller that it holds the interrupted context's rq lock.

Link: https://lore.kernel.org/r/d84b31727f04e1ed0d40042ba1c09e61@kernel.org

> diff --git a/kernel/sched/ext/internal.h b/kernel/sched/ext/internal.h
> index 27bbf5e04d900..faa80ac5a653a 100644
> --- a/kernel/sched/ext/internal.h
> +++ b/kernel/sched/ext/internal.h
> @@ -2181,6 +2181,15 @@ static inline void scx_schedule_reenq_local(struct rq *rq, u64 reenq_flags)
>   */
>  static inline struct rq *scx_locked_rq(void)
>  {
> +	/*
> +	 * Tracing progs can call kfuncs from NMI. scx_locked_rq_state tracks
> +	 * the rq locked by the interrupted context, so a non-NULL read from
> +	 * NMI would falsely claim its lock. Return NULL from NMI so that
> +	 * callers take their unlocked paths.
> +	 */
> +	if (unlikely(in_nmi()))
> +		return NULL;
> +

[Severity: High]
Does returning NULL in NMI cause a deadlock in scx_cpuperf_set() and
scx_cidperf_set()?

When a tracing BPF program executing in NMI context calls
scx_bpf_cpuperf_set(), it now sees locked_rq as NULL:

kernel/sched/ext/ext.c:scx_cpuperf_set() {
    [ ... ]
    locked_rq = scx_locked_rq();

    /*
     * If no rq lock is held, allow to operate on any CPU by acquiring
     * the corresponding rq lock.
     */
    if (!locked_rq) {
        rq_lock_irqsave(rq, &rf);
        update_rq_clock(rq);
    }
    [ ... ]
}

If the interrupted kernel context was already holding this CPU's rq->lock,
wouldn't attempting to unconditionally acquire it here cause the NMI handler
to spin forever and deadlock the system?

[Severity: High]
This isn't a bug introduced by this patch, but several any category kfuncs
unconditionally acquire raw spinlocks. Can these cause similar deadlocks if
invoked from NMI tracing programs?

For example, scx_bpf_destroy_dsq() calls destroy_dsq():

kernel/sched/ext/ext.c:destroy_dsq() {
    [ ... ]
    raw_spin_lock_irqsave(&dsq->lock, flags);
    [ ... ]
}

Similarly, bpf_iter_scx_dsq_next() unconditionally takes the dsq lock:

kernel/sched/ext/ext.c:bpf_iter_scx_dsq_next() {
    [ ... ]
    guard(raw_spinlock_irqsave)(&kit->dsq->lock);
    [ ... ]
}

And scx_bpf_dsq_reenq() calls schedule_dsq_reenq() which also takes a raw
spinlock:

kernel/sched/ext/ext.c:schedule_dsq_reenq() {
    [ ... ]
    guard(raw_spinlock_irqsave)(&rq->scx.deferred_reenq_lock);
    [ ... ]
}

If an NMI interrupts a kernel path holding one of these locks and the tracing
program invokes the corresponding kfunc, will the NMI handler spin forever
trying to acquire the held lock?

>  	return __this_cpu_read(scx_locked_rq_state);
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260903032953.659847-1-liwanwu@kylinos.cn?part=1

Re: [PATCH 1/2] sched_ext: Make scx_locked_rq() return NULL from NMI

[liwanwu]

在 2026/9/3 12:18, sashiko-bot@kernel.org 写道:
> Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> 
> New issues:
> - [High] scx_bpf_cpuperf_set() and scx_bpf_cidperf_set() deadlock in NMI when scx_locked_rq() returns NULL.
> 
> Pre-existing issues:
> - [High] Tracing BPF programs executing in NMI context can deadlock the system by calling pre-existing 'any' category kfuncs that acquire raw spinlocks.
> --
> 
> commit e3a4b62825e5b60c4d87fa4b8dd2f4b8dee65bbf
> Author: Wanwu Li <liwanwu@kylinos.cn>
> 
> sched_ext: Make scx_locked_rq() return NULL from NMI
> 
> scx_locked_rq() reads the per-CPU scx_locked_rq_state, which tracks the
> rq locked by the context running on the CPU. Tracing progs can call
> kfuncs from NMI, and an NMI interrupts - rather than replaces - the
> context that set scx_locked_rq_state, so a non-NULL read from NMI
> falsely tells the caller that it holds the interrupted context's rq lock.
> 
> Link: https://lore.kernel.org/r/d84b31727f04e1ed0d40042ba1c09e61@kernel.org
> 
>> diff --git a/kernel/sched/ext/internal.h b/kernel/sched/ext/internal.h
>> index 27bbf5e04d900..faa80ac5a653a 100644
>> --- a/kernel/sched/ext/internal.h
>> +++ b/kernel/sched/ext/internal.h
>> @@ -2181,6 +2181,15 @@ static inline void scx_schedule_reenq_local(struct rq *rq, u64 reenq_flags)
>>    */
>>   static inline struct rq *scx_locked_rq(void)
>>   {
>> +	/*
>> +	 * Tracing progs can call kfuncs from NMI. scx_locked_rq_state tracks
>> +	 * the rq locked by the interrupted context, so a non-NULL read from
>> +	 * NMI would falsely claim its lock. Return NULL from NMI so that
>> +	 * callers take their unlocked paths.
>> +	 */
>> +	if (unlikely(in_nmi()))
>> +		return NULL;
>> +
> 
Already fixed in sched_ext: Reject NMI calls to lock-taking kfuncs.
> [Severity: High]
> Does returning NULL in NMI cause a deadlock in scx_cpuperf_set() and
> scx_cidperf_set()?
> 
> When a tracing BPF program executing in NMI context calls
> scx_bpf_cpuperf_set(), it now sees locked_rq as NULL:
> 
> kernel/sched/ext/ext.c:scx_cpuperf_set() {
>      [ ... ]
>      locked_rq = scx_locked_rq();
> 
>      /*
>       * If no rq lock is held, allow to operate on any CPU by acquiring
>       * the corresponding rq lock.
>       */
>      if (!locked_rq) {
>          rq_lock_irqsave(rq, &rf);
>          update_rq_clock(rq);
>      }
>      [ ... ]
> }
> 
> If the interrupted kernel context was already holding this CPU's rq->lock,
> wouldn't attempting to unconditionally acquire it here cause the NMI handler
> to spin forever and deadlock the system?
> 
> [Severity: High]
> This isn't a bug introduced by this patch, but several any category kfuncs
> unconditionally acquire raw spinlocks. Can these cause similar deadlocks if
> invoked from NMI tracing programs?
> 
> For example, scx_bpf_destroy_dsq() calls destroy_dsq():
> 
> kernel/sched/ext/ext.c:destroy_dsq() {
>      [ ... ]
>      raw_spin_lock_irqsave(&dsq->lock, flags);
>      [ ... ]
> }
> 
> Similarly, bpf_iter_scx_dsq_next() unconditionally takes the dsq lock:
> 
> kernel/sched/ext/ext.c:bpf_iter_scx_dsq_next() {
>      [ ... ]
>      guard(raw_spinlock_irqsave)(&kit->dsq->lock);
>      [ ... ]
> }
> 
> And scx_bpf_dsq_reenq() calls schedule_dsq_reenq() which also takes a raw
> spinlock:
> 
> kernel/sched/ext/ext.c:schedule_dsq_reenq() {
>      [ ... ]
>      guard(raw_spinlock_irqsave)(&rq->scx.deferred_reenq_lock);
>      [ ... ]
> }
> 
Same as above.
> If an NMI interrupts a kernel path holding one of these locks and the tracing
> program invokes the corresponding kfunc, will the NMI handler spin forever
> trying to acquire the held lock?
> 
>>   	return __this_cpu_read(scx_locked_rq_state);
>>   }
> 
Here only read the lock-holding state without acquiring the lock.

[PATCH 2/2] sched_ext: Protect the idle-search scratch nodemask with irqsave

[Wanwu Li]

pick_idle_cpu_from_online_nodes() uses the per-CPU per_cpu_unvisited
nodemask as scratch while walking the online nodes, protected only by
preempt_disable(). preempt_disable() does not mask IRQs and the idle
kfuncs are callable from IRQ-enabled contexts, so a nested invocation
on the same CPU can overwrite the mask with nodes_copy() while the
interrupted invocation is still iterating it, leading to a wrong node
traversal and a wrong idle CPU pick.

Switch to irqsave so a nested invocation can't run on the same CPU.
A stack-allocated nodemask would also close the race, but that would
enlarge the diff to fix a race that is already rare (per-node idle,
CONFIG_NUMA and a cross-node search all at once); irqsave is the
minimal fix for the context that actually triggers it. The NMI case is
deliberately not addressed: there is no legitimate reason to
call pick_idle from NMI and doing so poses no crash risk, so such a
caller is on its own.

Suggested-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/r/d84b31727f04e1ed0d40042ba1c09e61@kernel.org
Signed-off-by: Wanwu Li <liwanwu@kylinos.cn>
---
 kernel/sched/ext/idle.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/ext/idle.c b/kernel/sched/ext/idle.c
index d2973fb3af6d..93e2e0b2d1f8 100644
--- a/kernel/sched/ext/idle.c
+++ b/kernel/sched/ext/idle.c
@@ -153,7 +153,18 @@ static s32 pick_idle_cpu_from_online_nodes(const struct cpumask *cpus_allowed, i
 	nodemask_t *unvisited;
 	s32 cpu = -EBUSY;
 
-	preempt_disable();
+	/*
+	 * @per_cpu_unvisited is per-CPU scratch and the idle kfuncs can be
+	 * called from IRQ-enabled contexts, so mask IRQs to keep a nested
+	 * invocation from clobbering the mask an outer invocation is still
+	 * iterating.
+	 *
+	 * NMI nesting is not handled: there is no legitimate reason to call
+	 * pick_idle from NMI and doing so poses no crash risk, so such a
+	 * caller is on its own.
+	 */
+	guard(irqsave)();
+
 	unvisited = this_cpu_ptr(&per_cpu_unvisited);
 
 	/*
@@ -183,7 +194,6 @@ static s32 pick_idle_cpu_from_online_nodes(const struct cpumask *cpus_allowed, i
 		if (cpu >= 0)
 			break;
 	}
-	preempt_enable();
 
 	return cpu;
 }
-- 
2.25.1

Re: [PATCH 0/2] sched_ext: two more context-safety fixes found in the NMI kfunc audit

[Tejun Heo]

> Wanwu Li (2):
>   sched_ext: Make scx_locked_rq() return NULL from NMI
>   sched_ext: Protect the idle-search scratch nodemask with irqsave

Applied 1-2 to sched_ext/for-7.4.

Thanks.

--
tejun