Re: [PATCH v18 00/17] Add support for Sub-NUMA cluster (SNC) systems

Maciej Wieczor-Retman
2024-05-16T09:28:49Z

On 2024-05-15 at 15:23:08 -0700, Tony Luck wrote:
>This series based on top of Linus upstream commit 33e02dc69afb ("Merge
>tag 'sound-6.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound")
>
>The Sub-NUMA cluster feature on some Intel processors partitions the CPUs
>that share an L3 cache into two or more sets. This plays havoc with the
>Resource Director Technology (RDT) monitoring features. Prior to this
>patch Intel has advised that SNC and RDT are incompatible.
>
>Some of these CPUs support an MSR that can partition the RMID counters
>in the same way. This allows monitoring features to be used. Legacy
>monitoring files provide the sum of counters from each SNC node for
>backwards compatibility. Additional files per SNC node provide details
>per node.
>
>Cache and memory bandwidth allocation features continue to operate at
>the scope of the L3 cache.
>
>Signed-off-by: Tony Luck <tony.luck@intel.com>
>
>---
>Changes since v17: https://lore.kernel.org/all/20240503203325.21512-1-tony.luck@intel.com/
>
>Reinette: This is still using the per-domain display_id field as
>discussed. Would a better name make the intent clearer?
>
>Patch 7 in previous version included virtually all of the new changes.
>But that meant it was doing a lot of thinngs in a single patch
>(including reverting a dozen lines from patch 6!)
>
>So this series breaks patch 7 into nine pieces (0007..0015) for
>better documentation in commit comments of the changes, and hopefully
>easier review.
>
>Patches 0001 ... 0005: Unchanged
>Patch 0006: Dropped change that was reverted in v17.0007
>
>Next nine are the split of the original patch v17.0007
>Patch 0007: Added bigger commit comment describing where
> this part of the series is heading and why.
>Patch 0008: Added justification for new display_id field in struct rdt_mon_domain
>Patch 0009: Split out a helper from mkdir_mondata_subdir()
> so real changes in patch 0011 are easier to see.
>Patch 0010: Comment stealing a bit from union mon_data_bits.evtid
>Patch 0011: Save display_id instead of a random d->id in
> meta data for monitor files that must sum SNC nodes
> Don't call mon_event_read() to initialize "sum" files
>Patch 0012: Set domid for "sum" files to the display id, not
> to whatever SNC domain ID is in use here. Don't
> call mon_event_read() for "sum" files.
>Patch 0013: No change (apart from being split out from old patch 7)
>Patch 0014: Because of change in patch 0011 to save the
> display_id can no longer look up a domain using
> rdt_find_domain(). Instead search r->mon_domains
> for a match with d->display_id or d->hdr.id
> Drop extra arg to ___mon_event_count() also
> the "tmp" variable in __mon_event_count()
>Patch 0015: Put #include <linux/cacheinfo.h> in alphabetical order
> When SNC is disabled, keep the old check that
> the current CPU is in the domain being read.
> For the SNC case add comment about reading
> monitor values from any CPU in the same L3 domain.
>
>Patch 0016: Took alternate SNC detection algorithm from:
> https://lore.kernel.org/all/20240327200352.236835-11-tony.luck@intel.com/
> as it is simpler. But merged in the sanity
> checks that make sense.
> Converted the X86_MATCH*() usage to new model
> that supports Intel families other than "6".
>Patch 0017: No change
>
>
>Tony Luck (17):
> x86/resctrl: Prepare for new domain scope
> x86/resctrl: Prepare to split rdt_domain structure
> x86/resctrl: Prepare for different scope for control/monitor
> operations
> x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
> x86/resctrl: Add node-scope to the options for feature scope
> x86/resctrl: Introduce snc_nodes_per_l3_cache
> x86/resctrl: Prepare for new Sub-NUMA (SNC) cluster monitor files
> x86/resctrl: Add and initialize display_id field to struct
> rdt_mon_domain
> x86/resctrl: Add new fields to struct rmid_read for summation of
> domains
> x86/resctrl: Refactor mkdir_mondata_subdir() with a helper function
> x86/resctrl: Allocate a new bit in union mon_data_bits
> x86/resctrl: Create Sub-NUMA (SNC) monitor files
> x86/resctrl: Handle removing directories in Sub-NUMA (SNC) mode
> x86/resctrl: Sum monitor data acrss Sub-NUMA (SNC) nodes when needed
> x86/resctrl: Fix RMID reading sanity check for Sub-NUMA (SNC) mode
> x86/resctrl: Sub NUMA Cluster detection and enable
> x86/resctrl: Update documentation with Sub-NUMA cluster changes
>
> Documentation/arch/x86/resctrl.rst | 17 +
> include/linux/resctrl.h | 89 +++--
> arch/x86/include/asm/msr-index.h | 1 +
> arch/x86/kernel/cpu/resctrl/internal.h | 78 ++--
> arch/x86/kernel/cpu/resctrl/core.c | 422 ++++++++++++++++++----
> arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 70 ++--
> arch/x86/kernel/cpu/resctrl/monitor.c | 106 ++++--
> arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 26 +-
> arch/x86/kernel/cpu/resctrl/rdtgroup.c | 267 +++++++++-----
> 9 files changed, 779 insertions(+), 297 deletions(-)
>
>
>base-commit: 33e02dc69afbd8f1b85a51d74d72f139ba4ca623
>--
>2.44.0
>

Tested-by: Maciej Wieczor-Retman <maciej.wieczor-retman@intel.com>

--
Kind regards
Maciej Wieczór-Retman