public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: Frederic Weisbecker <frederic@kernel.org>
To: "Christoph Lameter (Ampere)" <cl@gentwo.org>
Cc: Shubhang Kaushik <shubhang@os.amperecomputing.com>,
	Anna-Maria Behnsen <anna-maria@linutronix.de>,
	Ingo Molnar <mingo@kernel.org>, Thomas Gleixner <tglx@kernel.org>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Valentin Schneider <vschneid@redhat.com>,
	dietmar.eggemann@arm.com, bsegall@google.com, mgorman@suse.de,
	rostedt@goodmis.org, linux-kernel@vger.kernel.org,
	Adam Li <adamli@os.amperecomputing.com>
Subject: Re: [RESEND PATCH] tick/nohz: Fix wrong NOHZ idle CPU state
Date: Wed, 11 Mar 2026 12:06:22 +0100	[thread overview]
Message-ID: <abFMrqIlx-ObX_BD@localhost.localdomain> (raw)
In-Reply-To: <8ae7f176-2e1c-2390-a7da-a694df4b551e@gentwo.org>

Le Fri, Feb 13, 2026 at 10:15:15AM -0800, Christoph Lameter (Ampere) a écrit :
> On Fri, 13 Feb 2026, Frederic Weisbecker wrote:
> 
> > Then there seem to be something else going on that we don't fully understand
> > because isolated CPUs run 1 pinned task per CPU and the only housekeeping CPU
> > is CPU 0. So there is nothing to balance here.
> >
> > Perhaps some CPUs spend too much time scanning through all isolated CPUs to
> > see if there is balancing to do. I don't know, this needs further investigation.
> > But if the nohz_full CPUs are correctly domain isolated as they should
> > (through isolcpus=domain or cpuset isolated partitions), they should be
> > invisible to ilb anyway.
> 
> 
> "balancing" would mean moving tasks from busy cpus (that are not in
> NOHZ_FULL state) to idle cpus that can then be in NOHZ_FULL state.
> 
> If the move to from a busy cpu to an idle cpu succeeds then both cpus may
> only run one process and be able to enter NOHZ_FULL.
> 
> This is f.e. the caser with threadpools used by certain AI apps. Before
> the app starts numactl is used to setup a group of cpus that the app can use.
> 
> One may optimize and allow NOHZ_FULL for these cpus.
> 
> The app will then create a number of threads during its startup phase.
> These should be all placed on idle cpus in the allowed cpu range.
> 
> If this is configured the right way then each thread is on a different cpu
> and there is one thread per cpu so that we can use NOHZ_FULL.
> 
> This is sometimes broken because not all idle cpus are used. Instead some
> cpus get two threads and other cpus stay idle. That is why idle load
> balancing is needed.

Which means you guys eventually rely on load balancing...
So I can only repeat what I said there:

https://lore.kernel.org/lkml/aY3k1_JJjPFUhPd4@localhost.localdomain/

> There is no cpu isolation/cgroups or other black magic involved here.

Too bad, static task placement would fix your issue and domain isolation
would improve your workload.

Thanks.

-- 
Frederic Weisbecker
SUSE Labs

      reply	other threads:[~2026-03-11 11:06 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-02-04  0:49 [RESEND PATCH] tick/nohz: Fix wrong NOHZ idle CPU state Shubhang Kaushik
2026-02-12 14:33 ` Frederic Weisbecker
2026-02-12 19:36   ` Shubhang Kaushik
2026-02-12 20:04     ` Shubhang Kaushik
2026-02-13 13:11       ` Frederic Weisbecker
2026-02-13 12:56     ` Frederic Weisbecker
2026-02-13 18:15       ` Christoph Lameter (Ampere)
2026-03-11 11:06         ` Frederic Weisbecker [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=abFMrqIlx-ObX_BD@localhost.localdomain \
    --to=frederic@kernel.org \
    --cc=adamli@os.amperecomputing.com \
    --cc=anna-maria@linutronix.de \
    --cc=bsegall@google.com \
    --cc=cl@gentwo.org \
    --cc=dietmar.eggemann@arm.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@suse.de \
    --cc=mingo@kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=shubhang@os.amperecomputing.com \
    --cc=tglx@kernel.org \
    --cc=vincent.guittot@linaro.org \
    --cc=vschneid@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox