Re: [PATCH v3 1/4] sched/fair: add SD_CLUSTER in comments

Ricardo Neri
2024-02-06T21:55:44Z

On Tue, Feb 06, 2024 at 02:16:06PM +0100, Valentin Schneider wrote:
> On 05/02/24 18:46, Ricardo Neri wrote:
> > On Fri, Feb 02, 2024 at 03:27:32PM +0100, Valentin Schneider wrote:
> >>
> >> Subject nit: the prefix should be sched/topology
> >>
> >> On 01/02/24 19:54, alexs@kernel.org wrote:
> >> > From: Alex Shi <alexs@kernel.org>
> >> >
> >> > The description of SD_CLUSTER is missing. Add it.
> >> >
> >> > Signed-off-by: Alex Shi <alexs@kernel.org>
> >> > To: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
> >> > To: Valentin Schneider <vschneid@redhat.com>
> >> > To: Vincent Guittot <vincent.guittot@linaro.org>
> >> > To: Juri Lelli <juri.lelli@redhat.com>
> >> > To: Peter Zijlstra <peterz@infradead.org>
> >> > To: Ingo Molnar <mingo@redhat.com>
> >> > ---
> >> > kernel/sched/topology.c | 1 +
> >> > 1 file changed, 1 insertion(+)
> >> >
> >> > diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> >> > index 10d1391e7416..8b45f16a1890 100644
> >> > --- a/kernel/sched/topology.c
> >> > +++ b/kernel/sched/topology.c
> >> > @@ -1554,6 +1554,7 @@ static struct cpumask ***sched_domains_numa_masks;
> >> > * function:
> >> > *
> >> > * SD_SHARE_CPUCAPACITY - describes SMT topologies
> >> > + * SD_CLUSTER - describes CPU Cluster topologies
> >>
> >> So I know this is the naming we've gone for the "Cluster" naming, but this
> >> comment isn't really explaining anything.
> >>
> >> include/linux/sched/sd_flags.h has a bit more info already:
> >> * Domain members share CPU cluster (LLC tags or L2 cache)
> >
> > I also thought of this, but I didn't want to suggest to repeat in topolog.c
> > what is described in sd_flags.h.
> >
> > Maybe it is better to remove the descriptions of all flags here and instead
> > direct the reader to sd_flags.h?
> >
>
> Yeah I agree on less duplication.
>
> >>
> >> I had to go through a bit of git history to remember what the CLUSTER thing
> >> was about, how about this:
> >>
> >> * SD_CLUSTER - describes shared shared caches, cache tags or busses
> >
> > AFAIK, this describes a subset of CPUs in the package that share a
> > resource, likely L2 cache.
> >
> >> * SD_SHARE_PKG_RESOURCES - describes shared LLC cache
> >>
> >> And looking at this it would make sense to:
> >> rename SD_CLUSTER into SD_SHARE_PKG_RESOURCES
> >
> > but not all CPUs in the package share the resource
>
> But SD_CLUSTER never expands beyond the package, right?

Correct.

>
> Regardless, my main point is that having both SD_CLUSTER and
> SD_SHARE_PKG_RESOURCES is a source of confusion (at the very least for
> myself),

Agreed!

> and given SD_SHARE_PKG_RESOURCES is really used to mean "shares
> LLC" (see update_top_cache_domain()), we could make that flag more explicit
> and lift some ambiguity with SD_CLUSTER.

As Yicong stated, cluster topology should mean CPUs beyond SMT that share
some resource but not LLC.

It makes sense to me to keep SD_CLUSTER name as it is today and rename
SD_SHARE_PKG_RESOURCES as SD_SHARE_LLC.