From: Frederic Weisbecker <frederic@kernel.org>
To: "Christoph Lameter (Ampere)" <cl@gentwo.org>
Cc: Shubhang Kaushik <shubhang@os.amperecomputing.com>,
Anna-Maria Behnsen <anna-maria@linutronix.de>,
Ingo Molnar <mingo@kernel.org>, Thomas Gleixner <tglx@kernel.org>,
Vincent Guittot <vincent.guittot@linaro.org>,
Valentin Schneider <vschneid@redhat.com>,
dietmar.eggemann@arm.com, bsegall@google.com, mgorman@suse.de,
rostedt@goodmis.org, linux-kernel@vger.kernel.org,
Adam Li <adamli@os.amperecomputing.com>
Subject: Re: [RESEND PATCH] tick/nohz: Fix wrong NOHZ idle CPU state
Date: Wed, 11 Mar 2026 12:06:22 +0100 [thread overview]
Message-ID: <abFMrqIlx-ObX_BD@localhost.localdomain> (raw)
In-Reply-To: <8ae7f176-2e1c-2390-a7da-a694df4b551e@gentwo.org>
Le Fri, Feb 13, 2026 at 10:15:15AM -0800, Christoph Lameter (Ampere) a écrit :
> On Fri, 13 Feb 2026, Frederic Weisbecker wrote:
>
> > Then there seem to be something else going on that we don't fully understand
> > because isolated CPUs run 1 pinned task per CPU and the only housekeeping CPU
> > is CPU 0. So there is nothing to balance here.
> >
> > Perhaps some CPUs spend too much time scanning through all isolated CPUs to
> > see if there is balancing to do. I don't know, this needs further investigation.
> > But if the nohz_full CPUs are correctly domain isolated as they should
> > (through isolcpus=domain or cpuset isolated partitions), they should be
> > invisible to ilb anyway.
>
>
> "balancing" would mean moving tasks from busy cpus (that are not in
> NOHZ_FULL state) to idle cpus that can then be in NOHZ_FULL state.
>
> If the move to from a busy cpu to an idle cpu succeeds then both cpus may
> only run one process and be able to enter NOHZ_FULL.
>
> This is f.e. the caser with threadpools used by certain AI apps. Before
> the app starts numactl is used to setup a group of cpus that the app can use.
>
> One may optimize and allow NOHZ_FULL for these cpus.
>
> The app will then create a number of threads during its startup phase.
> These should be all placed on idle cpus in the allowed cpu range.
>
> If this is configured the right way then each thread is on a different cpu
> and there is one thread per cpu so that we can use NOHZ_FULL.
>
> This is sometimes broken because not all idle cpus are used. Instead some
> cpus get two threads and other cpus stay idle. That is why idle load
> balancing is needed.
Which means you guys eventually rely on load balancing...
So I can only repeat what I said there:
https://lore.kernel.org/lkml/aY3k1_JJjPFUhPd4@localhost.localdomain/
> There is no cpu isolation/cgroups or other black magic involved here.
Too bad, static task placement would fix your issue and domain isolation
would improve your workload.
Thanks.
--
Frederic Weisbecker
SUSE Labs
next prev parent reply other threads:[~2026-03-11 11:06 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-02-04 0:49 [RESEND PATCH] tick/nohz: Fix wrong NOHZ idle CPU state Shubhang Kaushik
2026-02-12 14:33 ` Frederic Weisbecker
2026-02-12 19:36 ` Shubhang Kaushik
2026-02-12 20:04 ` Shubhang Kaushik
2026-02-13 13:11 ` Frederic Weisbecker
2026-02-13 12:56 ` Frederic Weisbecker
2026-02-13 18:15 ` Christoph Lameter (Ampere)
2026-03-11 11:06 ` Frederic Weisbecker [this message]
2026-04-15 18:01 ` Christoph Lameter (Ampere)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=abFMrqIlx-ObX_BD@localhost.localdomain \
--to=frederic@kernel.org \
--cc=adamli@os.amperecomputing.com \
--cc=anna-maria@linutronix.de \
--cc=bsegall@google.com \
--cc=cl@gentwo.org \
--cc=dietmar.eggemann@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@kernel.org \
--cc=rostedt@goodmis.org \
--cc=shubhang@os.amperecomputing.com \
--cc=tglx@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.