From: Peter Zijlstra <peterz@infradead.org>
To: Chen Jinghuang <chenjinghuang2@huawei.com>
Cc: mingo@redhat.com, juri.lelli@redhat.com,
vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
vschneid@redhat.com, linux-kernel@vger.kernel.org
Subject: Re: [RESEND] sched/rt: Skip currently executing CPU in rto_next_cpu()
Date: Fri, 23 Jan 2026 11:06:25 +0100 [thread overview]
Message-ID: <20260123100625.GK171111@noisy.programming.kicks-ass.net> (raw)
In-Reply-To: <20260122012533.673768-1-chenjinghuang2@huawei.com>
On Thu, Jan 22, 2026 at 01:25:33AM +0000, Chen Jinghuang wrote:
> CPU0 becomes overloaded when hosting a CPU-bound RT task, a non-CPU-bound
> RT task, and a CFS task stuck in kernel space. When other CPUs switch from
> RT to non-RT tasks, RT load balancing (LB) is triggered; with
> HAVE_RT_PUSH_IPI enabled, they send IPIs to CPU0 to drive the execution
> of rto_push_irq_work_func. During push_rt_task on CPU0,
> if next_task->prio < rq->donor->prio, resched_curr() sets NEED_RESCHED
> and after the push operation completes, CPU0 calls rto_next_cpu().
> Since only CPU0 is overloaded in this scenario, rto_next_cpu() should
> ideally return -1 (no further IPI needed).
>
> However, multiple CPUs invoking tell_cpu_to_push() during LB increments
> rd->rto_loop_next. Even when rd->rto_cpu is set to -1, the mismatch between
> rd->rto_loop and rd->rto_loop_next forces rto_next_cpu() to restart its
> search from -1. With CPU0 remaining overloaded (satisfying rt_nr_migratory
> && rt_nr_total > 1), it gets reselected, causing CPU0 to queue irq_work to
> itself and send self-IPIs repeatedly. As long as CPU0 stays overloaded and
> other CPUs run pull_rt_tasks(), it falls into an infinite self-IPI loop,
> which triggers a CPU hardlockup due to continuous self-interrupts.
>
> The trigging scenario is as follows:
>
> cpu0 cpu1 cpu2
> pull_rt_task
> tell_cpu_to_push
> <------------irq_work_queue_on
> rto_push_irq_work_func
> push_rt_task
> resched_curr(rq) pull_rt_task
> rto_next_cpu tell_cpu_to_push
> <-------------------------- atomic_inc(rto_loop_next)
> rd->rto_loop != next
> rto_next_cpu
> irq_work_queue_on
> rto_push_irq_work_func
>
> Fix redundant self-IPI by filtering the initiating CPU in rto_next_cpu().
> This solution has been verified to effectively eliminate spurious self-IPIs
> and prevent CPU hardlockup scenarios.
>
> Fixes: 4bdced5c9a29 ("sched/rt: Simplify the IPI based RT balancing logic")
> Suggested-by: Steven Rostedt (Google) <rostedt@goodmis.org>
> Suggested-by: K Prateek Nayak <kprateek.nayak@amd.com>
> Signed-off-by: Chen Jinghuang <chenjinghuang2@huawei.com>
> Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>
> Reviewed-by: Valentin Schneider <vschneid@redhat.com>
> ---
Thanks!
next prev parent reply other threads:[~2026-01-23 10:06 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-01-22 1:25 [RESEND] sched/rt: Skip currently executing CPU in rto_next_cpu() Chen Jinghuang
2026-01-23 10:06 ` Peter Zijlstra [this message]
2026-02-03 11:18 ` [tip: sched/core] " tip-bot2 for Chen Jinghuang
-- strict thread matches above, loose matches on Subject: below --
2026-01-15 6:13 [RESEND] " Chen Jinghuang
2026-01-05 3:40 Chen Jinghuang
2026-01-05 21:25 ` Steven Rostedt
2026-01-06 8:41 ` K Prateek Nayak
2026-01-06 15:49 ` Steven Rostedt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260123100625.GK171111@noisy.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=bsegall@google.com \
--cc=chenjinghuang2@huawei.com \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=rostedt@goodmis.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox