From: Frederic Weisbecker <frederic@kernel.org>
To: "Christoph Lameter (Ampere)" <cl@gentwo.org>
Cc: Shubhang Kaushik <shubhang@os.amperecomputing.com>,
Anna-Maria Behnsen <anna-maria@linutronix.de>,
Ingo Molnar <mingo@kernel.org>, Thomas Gleixner <tglx@kernel.org>,
Vincent Guittot <vincent.guittot@linaro.org>,
Valentin Schneider <vschneid@redhat.com>,
dietmar.eggemann@arm.com, bsegall@google.com, mgorman@suse.de,
rostedt@goodmis.org, linux-kernel@vger.kernel.org,
Adam Li <adamli@os.amperecomputing.com>
Subject: Re: [RESEND PATCH] tick/nohz: Fix wrong NOHZ idle CPU state
Date: Wed, 11 Mar 2026 12:06:22 +0100 [thread overview]
Message-ID: <abFMrqIlx-ObX_BD@localhost.localdomain> (raw)
In-Reply-To: <8ae7f176-2e1c-2390-a7da-a694df4b551e@gentwo.org>
Le Fri, Feb 13, 2026 at 10:15:15AM -0800, Christoph Lameter (Ampere) a écrit :
> On Fri, 13 Feb 2026, Frederic Weisbecker wrote:
>
> > Then there seem to be something else going on that we don't fully understand
> > because isolated CPUs run 1 pinned task per CPU and the only housekeeping CPU
> > is CPU 0. So there is nothing to balance here.
> >
> > Perhaps some CPUs spend too much time scanning through all isolated CPUs to
> > see if there is balancing to do. I don't know, this needs further investigation.
> > But if the nohz_full CPUs are correctly domain isolated as they should
> > (through isolcpus=domain or cpuset isolated partitions), they should be
> > invisible to ilb anyway.
>
>
> "balancing" would mean moving tasks from busy cpus (that are not in
> NOHZ_FULL state) to idle cpus that can then be in NOHZ_FULL state.
>
> If the move to from a busy cpu to an idle cpu succeeds then both cpus may
> only run one process and be able to enter NOHZ_FULL.
>
> This is f.e. the caser with threadpools used by certain AI apps. Before
> the app starts numactl is used to setup a group of cpus that the app can use.
>
> One may optimize and allow NOHZ_FULL for these cpus.
>
> The app will then create a number of threads during its startup phase.
> These should be all placed on idle cpus in the allowed cpu range.
>
> If this is configured the right way then each thread is on a different cpu
> and there is one thread per cpu so that we can use NOHZ_FULL.
>
> This is sometimes broken because not all idle cpus are used. Instead some
> cpus get two threads and other cpus stay idle. That is why idle load
> balancing is needed.
Which means you guys eventually rely on load balancing...
So I can only repeat what I said there:
https://lore.kernel.org/lkml/aY3k1_JJjPFUhPd4@localhost.localdomain/
> There is no cpu isolation/cgroups or other black magic involved here.
Too bad, static task placement would fix your issue and domain isolation
would improve your workload.
Thanks.
--
Frederic Weisbecker
SUSE Labs
prev parent reply other threads:[~2026-03-11 11:06 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-02-04 0:49 [RESEND PATCH] tick/nohz: Fix wrong NOHZ idle CPU state Shubhang Kaushik
2026-02-12 14:33 ` Frederic Weisbecker
2026-02-12 19:36 ` Shubhang Kaushik
2026-02-12 20:04 ` Shubhang Kaushik
2026-02-13 13:11 ` Frederic Weisbecker
2026-02-13 12:56 ` Frederic Weisbecker
2026-02-13 18:15 ` Christoph Lameter (Ampere)
2026-03-11 11:06 ` Frederic Weisbecker [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=abFMrqIlx-ObX_BD@localhost.localdomain \
--to=frederic@kernel.org \
--cc=adamli@os.amperecomputing.com \
--cc=anna-maria@linutronix.de \
--cc=bsegall@google.com \
--cc=cl@gentwo.org \
--cc=dietmar.eggemann@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@kernel.org \
--cc=rostedt@goodmis.org \
--cc=shubhang@os.amperecomputing.com \
--cc=tglx@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox