Re: [PATCH 6.1] softirq: Allow raising SCHED_SOFTIRQ from SMP-call-function on RT kernel

[Greg KH <gregkh@linuxfoundation.org>]

On Wed, Jan 29, 2025 at 04:47:59PM +0100, Greg KH wrote:
> On Wed, Jan 29, 2025 at 04:32:26PM +0100, Florian Bezdeka wrote:
> > From: K Prateek Nayak <kprateek.nayak@amd.com>
> > 
> > commit 6675ce20046d149e1e1ffe7e9577947dee17aad5 upstream.
> > 
> > do_softirq_post_smp_call_flush() on PREEMPT_RT kernels carries a
> > WARN_ON_ONCE() for any SOFTIRQ being raised from an SMP-call-function.
> > Since do_softirq_post_smp_call_flush() is called with preempt disabled,
> > raising a SOFTIRQ during flush_smp_call_function_queue() can lead to
> > longer preempt disabled sections.
> > 
> > Since commit b2a02fc43a1f ("smp: Optimize
> > send_call_function_single_ipi()") IPIs to an idle CPU in
> > TIF_POLLING_NRFLAG mode can be optimized out by instead setting
> > TIF_NEED_RESCHED bit in idle task's thread_info and relying on the
> > flush_smp_call_function_queue() in the idle-exit path to run the
> > SMP-call-function.
> > 
> > To trigger an idle load balancing, the scheduler queues
> > nohz_csd_function() responsible for triggering an idle load balancing on
> > a target nohz idle CPU and sends an IPI. Only now, this IPI is optimized
> > out and the SMP-call-function is executed from
> > flush_smp_call_function_queue() in do_idle() which can raise a
> > SCHED_SOFTIRQ to trigger the balancing.
> > 
> > So far, this went undetected since, the need_resched() check in
> > nohz_csd_function() would make it bail out of idle load balancing early
> > as the idle thread does not clear TIF_POLLING_NRFLAG before calling
> > flush_smp_call_function_queue(). The need_resched() check was added with
> > the intent to catch a new task wakeup, however, it has recently
> > discovered to be unnecessary and will be removed in the subsequent
> > commit after which nohz_csd_function() can raise a SCHED_SOFTIRQ from
> > flush_smp_call_function_queue() to trigger an idle load balance on an
> > idle target in TIF_POLLING_NRFLAG mode.
> > 
> > nohz_csd_function() bails out early if "idle_cpu()" check for the
> > target CPU, and does not lock the target CPU's rq until the very end,
> > once it has found tasks to run on the CPU and will not inhibit the
> > wakeup of, or running of a newly woken up higher priority task. Account
> > for this and prevent a WARN_ON_ONCE() when SCHED_SOFTIRQ is raised from
> > flush_smp_call_function_queue().
> > 
> > Signed-off-by: K Prateek Nayak <kprateek.nayak@amd.com>
> > Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> > Link: https://lore.kernel.org/r/20241119054432.6405-2-kprateek.nayak@amd.com
> > Tested-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
> > ---
> > 
> > Newer stable branches (6.12, 6.6) got this already, 5.10 and lower are
> > not affected.
> > 
> > The warning triggered for SCHED_SOFTIRQ under high network load while
> > testing.
> 
> But RT is not in the 6.1.y tree, right?  Or is it?  Why was it only
> backported to 6.6.y and 6.12.y?

And see:
	https://lore.kernel.org/r/d21a8129-e982-463f-af8b-07a14b6a674a@amd.com
for why we added it to 6.12.y in the first place (I don't know why Sasha
added it to 6.6.y...)

thanks,

greg k-h