[PATCH] softirq: allow raising NET_RX_SOFTIRQ from SMP call on RT

[PATCH] softirq: allow raising NET_RX_SOFTIRQ from SMP call on RT

[wen.yang]

From: Wen Yang <wen.yang@linux.dev>

RPS distributes network processing load across CPUs by enqueuing
packets on a remote CPU's backlog and raising NET_RX_SOFTIRQ to
process them. This is done via an smp_call_function.

On PREEMPT_RT kernels, raising a softirq from an SMP call function
can increase preemption-off latencies. A WARN_ON_ONCE() in
do_softirq_post_smp_call_flush() enforces this.

A previous commit 6675ce20046d ("softirq: Allow raising SCHED_SOFTIRQ from SMP-call-function on RT kernel")
allowed SCHED_SOFTIRQ to be raised for idle load balancing purposes.
A similar issue exists for RPS. When RPS targets a remote CPU, it uses
an SMP call to raise NET_RX_SOFTIRQ, which triggers the warning.

This patch extends the exception to also allow NET_RX_SOFTIRQ, preventing
the warning when RPS is active on an RT kernel.

Signed-off-by: Wen Yang <wen.yang@linux.dev>
Cc: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Ryo Takakura <ryotkkr98@gmail.com>
Cc: K Prateek Nayak <kprateek.nayak@amd.com>
Cc: linux-kernel@vger.kernel.org
---
 kernel/softirq.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/softirq.c b/kernel/softirq.c
index 5f435c1e48d8..53cccdb29a8f 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -341,15 +341,15 @@ static inline void invoke_softirq(void)
 		wakeup_softirqd();
 }
 
-#define SCHED_SOFTIRQ_MASK	BIT(SCHED_SOFTIRQ)
+#define SCHED_SOFTIRQ_MASK	(BIT(SCHED_SOFTIRQ) | BIT(NET_RX_SOFTIRQ))
 
 /*
  * flush_smp_call_function_queue() can raise a soft interrupt in a function
  * call. On RT kernels this is undesired and the only known functionalities
  * are in the block layer which is disabled on RT, and in the scheduler for
  * idle load balancing. If soft interrupts get raised which haven't been
- * raised before the flush, warn if it is not a SCHED_SOFTIRQ so it can be
- * investigated.
+ * raised before the flush, warn if it is not SCHED_SOFTIRQ or NET_RX_SOFTIRQ
+ * so it can be investigated.
  */
 void do_softirq_post_smp_call_flush(unsigned int was_pending)
 {
-- 
2.25.1

Re: [PATCH] softirq: allow raising NET_RX_SOFTIRQ from SMP call on RT

[K Prateek Nayak]

Hello Wen,

On 12/19/2025 11:28 PM, wen.yang@linux.dev wrote:
> From: Wen Yang <wen.yang@linux.dev>
> 
> RPS distributes network processing load across CPUs by enqueuing
> packets on a remote CPU's backlog and raising NET_RX_SOFTIRQ to
> process them. This is done via an smp_call_function.

I'm assuming net_rps_send_ipi() is the function responsible for IPIs
and the only two callers - net_rps_action_and_irq_enable() and
dev_cpu_dead() - first check for "!use_backlog_threads()" before
sending the IPI which is always false in case of PREEMPT_RT.

What is the path where you end up sending this IPI on PREEMPT_RT?
Can you please trace the path of this problematic IPI on the latest
upstream and prevent that instead?

> 
> On PREEMPT_RT kernels, raising a softirq from an SMP call function
> can increase preemption-off latencies. A WARN_ON_ONCE() in
> do_softirq_post_smp_call_flush() enforces this.
> 
> A previous commit 6675ce20046d ("softirq: Allow raising SCHED_SOFTIRQ from SMP-call-function on RT kernel")

There is also a good justification in 6675ce20046d that idle load
balancing periodically checks need_resched() and will bail out
immediately if it detects a wakeup but the same cannot be expected of
all NET_RX_SOFTIRQ handlers so ignoring the warning isn't a solution.

-- 
Thanks and Regards,
Prateek

Re: [PATCH] softirq: allow raising NET_RX_SOFTIRQ from SMP call on RT

[Wen Yang]

On 12/22/25 14:03, K Prateek Nayak wrote:
> Hello Wen,
> 
> On 12/19/2025 11:28 PM, wen.yang@linux.dev wrote:
>> From: Wen Yang <wen.yang@linux.dev>
>>
>> RPS distributes network processing load across CPUs by enqueuing
>> packets on a remote CPU's backlog and raising NET_RX_SOFTIRQ to
>> process them. This is done via an smp_call_function.
> 
> I'm assuming net_rps_send_ipi() is the function responsible for IPIs
> and the only two callers - net_rps_action_and_irq_enable() and
> dev_cpu_dead() - first check for "!use_backlog_threads()" before
> sending the IPI which is always false in case of PREEMPT_RT.
> 
> What is the path where you end up sending this IPI on PREEMPT_RT?
> Can you please trace the path of this problematic IPI on the latest
> upstream and prevent that instead?
> 

Thank you for your suggestion.
Our production environment uses 6.1 and 6.6 kernels, and after porting 
commit dad6b9770263 ("net: Allow to use SMP threads for backlog NAPI.") 
and its dependent patches, this issue has indeed been fixed.

--
Best wishes,
Wen

>>
>> On PREEMPT_RT kernels, raising a softirq from an SMP call function
>> can increase preemption-off latencies. A WARN_ON_ONCE() in
>> do_softirq_post_smp_call_flush() enforces this.
>>
>> A previous commit 6675ce20046d ("softirq: Allow raising SCHED_SOFTIRQ from SMP-call-function on RT kernel")
> 
> There is also a good justification in 6675ce20046d that idle load
> balancing periodically checks need_resched() and will bail out
> immediately if it detects a wakeup but the same cannot be expected of
> all NET_RX_SOFTIRQ handlers so ignoring the warning isn't a solution.
>