[PATCH v18 00/17] Add support for Sub-NUMA cluster (SNC) systems

Tony Luck
2024-05-15T22:23:32Z

This series based on top of Linus upstream commit 33e02dc69afb ("Merge
tag 'sound-6.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound")

The Sub-NUMA cluster feature on some Intel processors partitions the CPUs
that share an L3 cache into two or more sets. This plays havoc with the
Resource Director Technology (RDT) monitoring features. Prior to this
patch Intel has advised that SNC and RDT are incompatible.

Some of these CPUs support an MSR that can partition the RMID counters
in the same way. This allows monitoring features to be used. Legacy
monitoring files provide the sum of counters from each SNC node for
backwards compatibility. Additional files per SNC node provide details
per node.

Cache and memory bandwidth allocation features continue to operate at
the scope of the L3 cache.

Signed-off-by: Tony Luck <tony.luck@intel.com>

---
Changes since v17: https://lore.kernel.org/all/20240503203325.21512-1-tony.luck@intel.com/

Reinette: This is still using the per-domain display_id field as
discussed. Would a better name make the intent clearer?

Patch 7 in previous version included virtually all of the new changes.
But that meant it was doing a lot of thinngs in a single patch
(including reverting a dozen lines from patch 6!)

So this series breaks patch 7 into nine pieces (0007..0015) for
better documentation in commit comments of the changes, and hopefully
easier review.

Patches 0001 ... 0005: Unchanged
Patch 0006: Dropped change that was reverted in v17.0007

Next nine are the split of the original patch v17.0007
Patch 0007: Added bigger commit comment describing where
this part of the series is heading and why.
Patch 0008: Added justification for new display_id field in struct rdt_mon_domain
Patch 0009: Split out a helper from mkdir_mondata_subdir()
so real changes in patch 0011 are easier to see.
Patch 0010: Comment stealing a bit from union mon_data_bits.evtid
Patch 0011: Save display_id instead of a random d->id in
meta data for monitor files that must sum SNC nodes
Don't call mon_event_read() to initialize "sum" files
Patch 0012: Set domid for "sum" files to the display id, not
to whatever SNC domain ID is in use here. Don't
call mon_event_read() for "sum" files.
Patch 0013: No change (apart from being split out from old patch 7)
Patch 0014: Because of change in patch 0011 to save the
display_id can no longer look up a domain using
rdt_find_domain(). Instead search r->mon_domains
for a match with d->display_id or d->hdr.id
Drop extra arg to ___mon_event_count() also
the "tmp" variable in __mon_event_count()
Patch 0015: Put #include <linux/cacheinfo.h> in alphabetical order
When SNC is disabled, keep the old check that
the current CPU is in the domain being read.
For the SNC case add comment about reading
monitor values from any CPU in the same L3 domain.

Patch 0016: Took alternate SNC detection algorithm from:
https://lore.kernel.org/all/20240327200352.236835-11-tony.luck@intel.com/
as it is simpler. But merged in the sanity
checks that make sense.
Converted the X86_MATCH*() usage to new model
that supports Intel families other than "6".
Patch 0017: No change

Tony Luck (17):
x86/resctrl: Prepare for new domain scope
x86/resctrl: Prepare to split rdt_domain structure
x86/resctrl: Prepare for different scope for control/monitor
operations
x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
x86/resctrl: Add node-scope to the options for feature scope
x86/resctrl: Introduce snc_nodes_per_l3_cache
x86/resctrl: Prepare for new Sub-NUMA (SNC) cluster monitor files
x86/resctrl: Add and initialize display_id field to struct
rdt_mon_domain
x86/resctrl: Add new fields to struct rmid_read for summation of
domains
x86/resctrl: Refactor mkdir_mondata_subdir() with a helper function
x86/resctrl: Allocate a new bit in union mon_data_bits
x86/resctrl: Create Sub-NUMA (SNC) monitor files
x86/resctrl: Handle removing directories in Sub-NUMA (SNC) mode
x86/resctrl: Sum monitor data acrss Sub-NUMA (SNC) nodes when needed
x86/resctrl: Fix RMID reading sanity check for Sub-NUMA (SNC) mode
x86/resctrl: Sub NUMA Cluster detection and enable
x86/resctrl: Update documentation with Sub-NUMA cluster changes

Documentation/arch/x86/resctrl.rst | 17 +
include/linux/resctrl.h | 89 +++--
arch/x86/include/asm/msr-index.h | 1 +
arch/x86/kernel/cpu/resctrl/internal.h | 78 ++--
arch/x86/kernel/cpu/resctrl/core.c | 422 ++++++++++++++++++----
arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 70 ++--
arch/x86/kernel/cpu/resctrl/monitor.c | 106 ++++--
arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 26 +-
arch/x86/kernel/cpu/resctrl/rdtgroup.c | 267 +++++++++-----
9 files changed, 779 insertions(+), 297 deletions(-)

base-commit: 33e02dc69afbd8f1b85a51d74d72f139ba4ca623
--
2.44.0