[PATCH net] net: yield the CPU on every exit of the threaded NAPI poll loop

[PATCH net] net: yield the CPU on every exit of the threaded NAPI poll loop

[Vitaliy Sochnev]

napi_threaded_poll_loop() only reaches cond_resched() when it is about to
iterate again. When __napi_poll() clears repoll the loop breaks first, so
the last iteration returns without any voluntary preemption point.

Control then goes back to napi_threaded_poll(), which calls
napi_thread_wait(). If work is already pending that helper returns without
ever calling schedule(). Under a receive load that keeps arriving at least
as fast as it is drained, this repeats indefinitely and the kthread holds
its CPU without a single reschedule. On CONFIG_PREEMPT_NONE nothing else
on that CPU gets to run.

Everything that waits for deferred work on that CPU then blocks: RCU grace
periods, per-CPU work items and RCU callbacks. Deleting a network device
hits all three - synchronize_net(), flush_all_backlogs() -> flush_work()
and rcu_barrier() from netdev_run_todo() - which is how this was found.
The backlog kthread runs the same loop via run_backlog_napi(), so it can
be held off in the same way.

rcu_softirq_qs_periodic() does not help here. It reports a quiescent state
but does not schedule, so the work items and the callbacks still wait, and
it is skipped on the exit path anyway.

Yield on both exits. rcu_softirq_qs_periodic() keeps its place on the
iterating path, where the RCU annotation is what is needed.

Measured on a Nokia XG-040G-MF (Airoha AN7583, dual core Cortex-A53,
CONFIG_PREEMPT_NONE, HZ=100, airoha_eth with threaded NAPI) while the
board terminates a 985 Mbit/s TCP receive load. 60 minute runs, timing
"ip link del" of a dummy interface:

                    before        after
  mean               1.22 s       0.10 s
  worst             60.32 s       0.16 s
  over 1 s        7 of 172      0 of 179
  RCU stalls             2            0
  receive       985 Mbit/s   984 Mbit/s
  packet rate   82094 p/s    82027 p/s

The packet rate is the control: the same work is done in both runs, so
the difference is not a lighter load.

The test kernel was 6.18, where this loop has no busy_poll_last_qs
parameter. With that pointer NULL the two versions of the loop are the
same code - the initialiser falls back to jiffies, the gro_flush_normal()
call and the write-back are skipped, and the tail condition reduces to
"if (repoll)" - so the change under test is this one. The busy-poll path
is not covered by these runs; this patch does not alter it, as
cond_resched() was already reached there.

An earlier variant that only moved the quiescent-state report, without
changing where the CPU is yielded, was not enough: the delay simply
migrated from synchronize_net() to flush_work() and rcu_barrier().

Without the patch the kernel reports the thread holding the CPU:

  rcu: INFO: rcu_sched self-detected stall on CPU
  rcu:  0-....: (5999 ticks this GP) ... (t=6001 jiffies g=50657 q=400)
  CPU: 0 UID: 0 PID: 241 Comm: napi/qdma_eth-0 Tainted: G W O 6.18.41 #0
  Tainted: [W]=WARN, [O]=OOT_MODULE
  Hardware name: Nokia XG-040G-MF (DT)
  pc : gro_receive_skb+0x0/0x1b8
  lr : gro_cell_poll+0x58/0xc0

Reproducing it needs the loop to be re-entered tens of thousands of times
a second, so it depends on the driver and on the shape of the load. It
did not reproduce on mtk_eth_soc, whose net_dim moderation coalesces the
same packet rate into far fewer interrupts, nor at loads where the queue
drains completely on each poll and the thread sleeps.

The taint is an out-of-tree GPIO button module and an earlier PHY warning,
both unrelated to the networking path above.

Fixes: 29863d41bb6e ("net: implement threaded-able napi poll loop support")
Signed-off-by: Vitaliy Sochnev <sochnev.v.74@gmail.com>
---
 net/core/dev.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/net/core/dev.c b/net/core/dev.c
index ece6700536d9..884aa2dcadbe 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -7876,11 +7876,14 @@ static void napi_threaded_poll_loop(struct napi_struct *napi,
 			gro_flush_normal(&napi->gro, HZ >= 1000);
 		local_bh_enable();
 
-		/* Call cond_resched here to avoid watchdog warnings. */
-		if (repoll || busy_poll_last_qs) {
+		if (repoll || busy_poll_last_qs)
 			rcu_softirq_qs_periodic(last_qs);
-			cond_resched();
-		}
+
+		/* Yield on every exit from the loop, not only when it iterates:
+		 * napi_thread_wait() can return without scheduling, so a thread
+		 * that keeps finding work would never give up the CPU.
+		 */
+		cond_resched();
 
 		if (!repoll)
 			break;
-- 
2.55.0

Re: [PATCH net] net: yield the CPU on every exit of the threaded NAPI poll loop

[Jakub Kicinski]

On Fri, 14 Aug 2026 23:04:27 +0100 Vitaliy Sochnev wrote:
>   CPU: 0 UID: 0 PID: 241 Comm: napi/qdma_eth-0 Tainted: G W O 6.18.41 #0

Please try to repro this on Linus's tree and repost if you can.
Don't recall the exact details but IIRC the preemption is now
more aggressive.
-- 
pw-bot: cr

Re: [PATCH net] net: yield the CPU on every exit of the threaded NAPI poll loop

[Vitaliy Sochnev]

> Please try to repro this on Linus's tree and repost if you can.
> Don't recall the exact details but IIRC the preemption is now
> more aggressive.

The patch itself is against current net/master already - that is the tree
I generated and build-tested it on, and the loop there is unchanged:

        if (repoll || busy_poll_last_qs) {
                rcu_softirq_qs_periodic(last_qs);
                cond_resched();
        }

        if (!repoll)
                break;

The exit path still reaches neither call, which is the case the patch is
about. So there is nothing to change in the posting itself; it is the
numbers that come from 6.18.

They come from 6.18 because that is the only kernel this board runs.
Mainline carries just en7581-evb - the AN7581/AN7583 SoC dtsi and the
board DTS exist only in OpenWrt, along with the airoha_eth changes that
have not landed upstream yet.

On preemption - I did measure that. Same board, same load, plain OpenWrt
without the patch, the two halves differing only in the preemption model
(verified in the built kernel .config):

  PREEMPT_NONE  worst "ip link del" 248.48 s, 8 of 29 samples over 1 s,
                8 classic + 48 expedited RCU stalls
  PREEMPT_LAZY  worst 0.39 s, 0 of 177 samples, no stalls

at the same packet rate, 80862 vs 80036 pkt/s. So you remember right:
with lazy preemption the symptom is gone. What is left is PREEMPT_NONE and
PREEMPT_VOLUNTARY builds.

It is not explained by the NAPI thread being preempted more, though -
nonvoluntary_ctxt_switches on that thread is 4.6/s under LAZY against
13.9/s under PREEMPT_NONE. The interrupt and batching pattern changes
instead. I could not pin the mechanism down, so I am reporting the
measurement rather than a conclusion.

For what it is worth, the reproduction is not specific to this hardware:
threaded NAPI can be turned on for any driver through
/sys/class/net/<dev>/threaded, and what the bug needs on top of that is
little or no interrupt coalescing, so the poll loop is re-entered tens of
thousands of times a second.

If a mainline repro is a hard requirement here, I understand - the 6.18
data and the unchanged code path is what I have.