RE: [RFC PATCH v2 08/23] sched/topology: Introduce a per-CPU tasks NUMA preferred counter

[Jianyong Wu <wujianyong@hygon.cn>]

> -----Original Message-----
> From: Peter Zijlstra <peterz@infradead.org>
> Sent: Tuesday, September 1, 2026 6:21 PM
> To: Jianyong Wu <wujianyong@hygon.cn>
> Cc: Ingo Molnar <mingo@redhat.com>; Juri Lelli <juri.lelli@redhat.com>;
> Vincent Guittot <vincent.guittot@linaro.org>; Chen Yu
> <yu.c.chen@intel.com>; Tim Chen <tim.c.chen@linux.intel.com>; Dietmar
> Eggemann <dietmar.eggemann@arm.com>; Steven Rostedt
> <rostedt@goodmis.org>; Ben Segall <bsegall@google.com>; Mel Gorman
> <mgorman@suse.de>; Valentin Schneider <vschneid@redhat.com>; K
> Prateek Nayak <kprateek.nayak@amd.com>; Shrikanth Hegde
> <sshegde@linux.ibm.com>; Phil Auld <pauld@redhat.com>; Andrew
> Morton <akpm@linux-foundation.org>; David Hildenbrand
> <david@kernel.org>; linux-kernel@vger.kernel.org; linux-mm@kvack.org;
> jianyong.wu@outlook.com; Yuan Zhong <zhongyuan@hygon.cn>; Huangsj
> <huangsj@hygon.cn>; Fengyu Wang <wangfengyu@hygon.cn>; Zhiwei Ying
> <yingzhiwei@hygon.cn>; justin.he@arm.com
> Subject: Re: [RFC PATCH v2 08/23] sched/topology: Introduce a per-CPU
> tasks NUMA preferred counter
> 
> On Tue, Sep 01, 2026 at 08:31:35AM +0000, Jianyong Wu wrote:
> 
> > > On Thu, Aug 27, 2026 at 08:28:01PM +0800, Jianyong Wu wrote:
> > > > Like the existed task LLC preferred counter, sd->llc_counts, introduce
> > > > sd->numa_counts to denotes the task number that prefer each
> NUMA node in
> > > > a certain rq.
> > > >
> > >
> > > To what purpose? Is not the node occupancy the direct sum of its
> > > constituent llc occupancy?
> >
> > numa_counts is used by the affinity score in patch 12. It holds, for each
> > NUMA node, the number of tasks on this rq whose preferred LLC falls in
> > that node.
> >
> > You're right that numerically it equals the per-node sum of llc_counts.
> > The reason to keep it is granularity: NUMA-level load balance scores at
> > node granularity (numa_counts[node] * distance_delta), while
> NODE-level
> > balancing scores at LLC granularity (llc_counts[llc] * ...). Maintaining
> > the node aggregate costs only a single llc_to_node() lookup (an O(1)
> > array read) plus one increment in enqueue/dequeue, whereas
> recomputing
> > the per-node sum would have to walk every LLC of a node each time the
> > NUMA-level score is evaluated.
> 
> Consider the trade-off. Adding the accounting adds a cache-miss to every
> enqueue/dequeue, while re-computing the value on-demand adds some
> little
> cost to the balancing (slow) path.
> 
> If the llc_counts[] array is dense in node order, then it is likely all
> relevant numbers are in a single line and computing the sum is in fact
> fairly cheap.

This is a good point. I was considering the number of operations, but not the fact that the additional accounting is performed on every enqueue/dequeue and may touch another cache line, while the summation is only needed on the load balancing path.

Since numa_counts[] contains no information that cannot be derived from llc_counts[], I will rework the NUMA-level score to sum the constituent LLC counters on demand and remove the additional numa_counts[] accounting.

Thanks
Jianyong