public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: "Guo, Wangyang" <wangyang.guo@intel.com>
To: Shubhang Kaushik Prasanna Kumar <shkaushik@amperecomputing.com>
Cc: benjamin.lei@intel.com, bsegall@google.com,
	dietmar.eggemann@arm.com, juri.lelli@redhat.com,
	linux-kernel@vger.kernel.org, mgorman@suse.de, mingo@redhat.com,
	peterz@infradead.org, rostedt@goodmis.org, tianyou.li@intel.com,
	tim.c.chen@linux.intel.com, vincent.guittot@linaro.org,
	vschneid@redhat.com
Subject: Re: [PATCH] sched/fair: Avoid false sharing in nohz struct
Date: Fri, 19 Dec 2025 21:56:18 +0800	[thread overview]
Message-ID: <23a1e55f-bf03-4c05-a954-28326b59b06b@intel.com> (raw)
In-Reply-To: <F272503E-9E67-411B-B8E6-1B31D954FC92@os.amperecomputing.com>

On 12/19/2025 9:38 AM, Shubhang Kaushik Prasanna Kumar wrote:

> While the intuition behind isolating the `nr_cpus` counter seems correct, could you please justify the added padding ? As this is a high contention path in the scheduler, we shouldn't be inflating global structures with padding on logic alone. I’d like to see some benchmarking such as `perf c2c` results from a multi-core system proving the `false sharing` scenario as a measurable bottleneck.

Here is the cache line view in c2c report:
     Num RmtHitm LclHitm  Offset records             Symbol
   6.25%   0.00%   0.00%   0x0       4   [k] _nohz_idle_balance.isra.0
  18.75% 100.00%   0.00%   0x8      14   [k] nohz_balance_exit_idle
   6.25%   0.00%   0.00%   0x8       8   [k] nohz_balance_enter_idle
   6.25%   0.00%   0.00%   0xc       8   [k] sched_balance_newidle
   6.25%   0.00%   0.00%  0x10      31   [k] nohz_balancer_kick
   6.25%   0.00%   0.00%  0x20      16   [k] sched_balance_newidle
  37.50%   0.00%   0.00%  0x38      50   [k] irqtime_account_irq
   6.25%   0.00%   0.00%  0x38      47   [k] account_process_tick
   6.25%   0.00%   0.00%  0x38      12   [k] account_idle_ticks

Offsets:
*  0x0 -- nohz.idle_cpu_mask (r)
*  0x8 -- nohz.nr_cpus (w)
* 0x38 -- sched_clock_irqtime (r), not in nohz, but share cacheline

The layout in /proc/kallsyms can also confirm that:
ffffffff88600d40 b nohz
ffffffff88600d68 B arch_needs_tick_broadcast
ffffffff88600d6c b __key.264
ffffffff88600d6c b __key.265
ffffffff88600d70 b dl_generation
ffffffff88600d78 b sched_clock_irqtime

In perf cycle hotspots, we noticed that irqtime_account_irq has 3% 
overhead caused by false sharing. With the patch applied, that hotspot 
disappears.

> I am also concerned about the internal layout. By sandwiching the timer fields between two `__cacheline_aligned` boundaries, we might just be shifting the contention rather than fixing it.  See to it that fields like `next_balance` aren't being squeezed into a new conflict zone. Would like to review the benchmark data and the struct layout before we move forward.

Before the patch, all the nohz members stay in the same cache line. 
After layout fix, fields like `next_balance` still in the same cache 
line, no new conflict zone created.

Since nohz is global, the size inflating is minimal - less than 64 bytes 
in total.

BR
Wangyang


  reply	other threads:[~2025-12-19 13:56 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-12-19  1:38 [PATCH] sched/fair: Avoid false sharing in nohz struct Shubhang Kaushik Prasanna Kumar
2025-12-19 13:56 ` Guo, Wangyang [this message]
  -- strict thread matches above, loose matches on Subject: below --
2025-12-11  5:56 Wangyang Guo
2025-12-21 13:05 ` Shrikanth Hegde
2025-12-22  2:21   ` Guo, Wangyang
2025-12-23  7:27     ` Shrikanth Hegde
2025-12-23  8:03       ` Guo, Wangyang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=23a1e55f-bf03-4c05-a954-28326b59b06b@intel.com \
    --to=wangyang.guo@intel.com \
    --cc=benjamin.lei@intel.com \
    --cc=bsegall@google.com \
    --cc=dietmar.eggemann@arm.com \
    --cc=juri.lelli@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@suse.de \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=shkaushik@amperecomputing.com \
    --cc=tianyou.li@intel.com \
    --cc=tim.c.chen@linux.intel.com \
    --cc=vincent.guittot@linaro.org \
    --cc=vschneid@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox