From: "Guo, Wangyang" <wangyang.guo@intel.com>
To: Shubhang Kaushik Prasanna Kumar <shkaushik@amperecomputing.com>
Cc: benjamin.lei@intel.com, bsegall@google.com,
dietmar.eggemann@arm.com, juri.lelli@redhat.com,
linux-kernel@vger.kernel.org, mgorman@suse.de, mingo@redhat.com,
peterz@infradead.org, rostedt@goodmis.org, tianyou.li@intel.com,
tim.c.chen@linux.intel.com, vincent.guittot@linaro.org,
vschneid@redhat.com
Subject: Re: [PATCH] sched/fair: Avoid false sharing in nohz struct
Date: Fri, 19 Dec 2025 21:56:18 +0800 [thread overview]
Message-ID: <23a1e55f-bf03-4c05-a954-28326b59b06b@intel.com> (raw)
In-Reply-To: <F272503E-9E67-411B-B8E6-1B31D954FC92@os.amperecomputing.com>
On 12/19/2025 9:38 AM, Shubhang Kaushik Prasanna Kumar wrote:
> While the intuition behind isolating the `nr_cpus` counter seems correct, could you please justify the added padding ? As this is a high contention path in the scheduler, we shouldn't be inflating global structures with padding on logic alone. I’d like to see some benchmarking such as `perf c2c` results from a multi-core system proving the `false sharing` scenario as a measurable bottleneck.
Here is the cache line view in c2c report:
Num RmtHitm LclHitm Offset records Symbol
6.25% 0.00% 0.00% 0x0 4 [k] _nohz_idle_balance.isra.0
18.75% 100.00% 0.00% 0x8 14 [k] nohz_balance_exit_idle
6.25% 0.00% 0.00% 0x8 8 [k] nohz_balance_enter_idle
6.25% 0.00% 0.00% 0xc 8 [k] sched_balance_newidle
6.25% 0.00% 0.00% 0x10 31 [k] nohz_balancer_kick
6.25% 0.00% 0.00% 0x20 16 [k] sched_balance_newidle
37.50% 0.00% 0.00% 0x38 50 [k] irqtime_account_irq
6.25% 0.00% 0.00% 0x38 47 [k] account_process_tick
6.25% 0.00% 0.00% 0x38 12 [k] account_idle_ticks
Offsets:
* 0x0 -- nohz.idle_cpu_mask (r)
* 0x8 -- nohz.nr_cpus (w)
* 0x38 -- sched_clock_irqtime (r), not in nohz, but share cacheline
The layout in /proc/kallsyms can also confirm that:
ffffffff88600d40 b nohz
ffffffff88600d68 B arch_needs_tick_broadcast
ffffffff88600d6c b __key.264
ffffffff88600d6c b __key.265
ffffffff88600d70 b dl_generation
ffffffff88600d78 b sched_clock_irqtime
In perf cycle hotspots, we noticed that irqtime_account_irq has 3%
overhead caused by false sharing. With the patch applied, that hotspot
disappears.
> I am also concerned about the internal layout. By sandwiching the timer fields between two `__cacheline_aligned` boundaries, we might just be shifting the contention rather than fixing it. See to it that fields like `next_balance` aren't being squeezed into a new conflict zone. Would like to review the benchmark data and the struct layout before we move forward.
Before the patch, all the nohz members stay in the same cache line.
After layout fix, fields like `next_balance` still in the same cache
line, no new conflict zone created.
Since nohz is global, the size inflating is minimal - less than 64 bytes
in total.
BR
Wangyang
next prev parent reply other threads:[~2025-12-19 13:56 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-12-19 1:38 [PATCH] sched/fair: Avoid false sharing in nohz struct Shubhang Kaushik Prasanna Kumar
2025-12-19 13:56 ` Guo, Wangyang [this message]
-- strict thread matches above, loose matches on Subject: below --
2025-12-11 5:56 Wangyang Guo
2025-12-21 13:05 ` Shrikanth Hegde
2025-12-22 2:21 ` Guo, Wangyang
2025-12-23 7:27 ` Shrikanth Hegde
2025-12-23 8:03 ` Guo, Wangyang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=23a1e55f-bf03-4c05-a954-28326b59b06b@intel.com \
--to=wangyang.guo@intel.com \
--cc=benjamin.lei@intel.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=shkaushik@amperecomputing.com \
--cc=tianyou.li@intel.com \
--cc=tim.c.chen@linux.intel.com \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox