Re: [PATCH net v2] net: yield the CPU on every exit of the threaded NAPI poll loop

[Vitaliy Sochnev <sochnev.v.74@gmail.com>]

> Rebase your OOT patches on latest and test the tree you're targeting
> or keep this patch in your downstream kernel as well.

I could not produce a repro on the target tree, and the reason turns out
to matter more than my board being out of tree.

Since v7.0 the non-preempting models are gated:

  kernel/Kconfig.preempt
    config PREEMPT_NONE       depends on ARCH_NO_PREEMPT
    config PREEMPT_VOLUNTARY  depends on !ARCH_HAS_PREEMPT_LAZY

  kernel/sched/core.c, sched_dynamic_mode()
    "none" and "voluntary" are compiled out under
    !(PREEMPT_RT || ARCH_HAS_PREEMPT_LAZY)

ARCH_NO_PREEMPT is selected by m68k, hexagon and alpha only, and
ARCH_HAS_PREEMPT_LAZY by x86, arm64, powerpc, s390, riscv and loongarch.
So on those six there is no way to build a kernel, or boot one, in which
cond_resched() is anything but a nop. I checked it on an x86 guest built
from net/main: the set offered by /sys/kernel/debug/sched/preempt is
"full (lazy)", and none and voluntary are not in it.

This patch therefore does nothing on those six, and I should have said
that in v2 instead of describing PREEMPT_NONE and PREEMPT_VOLUNTARY as
what is left. What is left is PREEMPT_VOLUNTARY on arc, arm, csky,
microblaze, mips, nios2, openrisc, parisc, sh, sparc and xtensa,
PREEMPT_NONE on the three above, and 6.18 and earlier everywhere, which
is what the affected devices run and where the Fixes tag applies.

Since every number in v2 came from PREEMPT_NONE, which those
architectures can no longer select, I measured VOLUNTARY as well. Same
board, 6.18 where it is still selectable, untainted, no out-of-tree
module, 30 minute runs, the two halves differing only in this patch:

                          before        after
  mean                    13.38 s       0.19 s
  worst                  135.95 s       0.51 s
  over 1 s              10 of 55      0 of 89
  RCU stalls, classic          12            0
  RCU stalls, expedited        20            0
  packet rate           80312 p/s    81532 p/s

add, mtu and a no-op netlink call stayed at 0.01-0.41 s throughout both
halves, so the delay is the grace period and not rtnl or scheduling. The
splat names the model itself:

  rcu: INFO: rcu_sched self-detected stall on CPU
  rcu:  0-....: (5999 ticks this GP) ... (t=6001 jiffies g=861 q=830)
  CPU: 0 PID: 200 Comm: napi/qdma_eth-0 Not tainted 6.18.44 #0 VOLUNTARY

For completeness, I did try to reproduce it on net/main in a VM, with
threaded NAPI on a veth pair and the ARCH_HAS_PREEMPT_LAZY select
dropped from arch/x86/Kconfig so that voluntary preemption was
reachable at all. The worst "ip link del" went to 0.19 s against 0.01 s
at rest, but never to a stall: below the band the thread sleeps between
batches, above it repoll stays set and the existing cond_resched() runs.
I could not hold it in between, so I am not offering that as a
reproduction.

So this fixes the architectures that still have a non-preempting model,
and the stable kernels, but not the tree it is posted against. If that
makes it not worth carrying in net, say so and I will keep it
downstream. If it is worth carrying, I will send a v3 with the numbers
above in the commit message.