[PATCH v6 0/5] CXL: Support region granularities finer than the root

[Alison Schofield <alison.schofield@intel.com>]

*** Use to be cxl: Support mixed-granularity region interleaves *** 
Long long time ago, I can still remember when this issue first arose.
I simply called it mixed, because previously region and root decoders
had to match. Well, mixed was too simple and too broad. So rather than
continuing to qualify it, drop it and just state what the set actually
enables: region grans finer than root grans.

It's not just a new title. It reframes a lot of language throughout the
series, from the cover letter, commit messages, documentation, and
code naming.

Changes in v6:
Series wide:
- Reframe the series around what it enables: a region granularity finer
  than the root granularity, with monotonic refinement as the supported
  model. Drop the term "mixed-granularity", which read as support for
  arbitrary granularity arrangements. (Jonathan, Robert)
- Rebase onto the cxl_switch_port_setup_targets() refactor [1].
- Drop "cxl/region: Warn on user region position mismatch", merged in cxl/next
- Drop "cxl/region: Name the interleave locals in cxl_port_setup_targets()". 
Patch 1: Documentation/cxl: Describe region granularity finer than the root
- Separate what the CXL Spec permits from what Linux supports (Robert)
- Describe the resulting support rather than the prior restriction (Robert)
- Define region position by address translation (Robert)
- Define region granularity before the span equation (Robert)
- Condense the Mod3 section, keeping the annotated tables (Robert)
- Reword the refining and coarsening weight explanation
Patch 2: cxl/region: Generalize endpoint position mapping
- Pass the region to cxl_calc_interleave_pos() (Robert)
- Fold the per-level target lookup and weighting into cxl_port_interleave_pos() (Robert)
- Define region position by address translation in the kernel-doc (Robert)
- Drop the stale Normalized Addressing rationale (Robert)
- Name parent_pos in the commit message to match the code (Jonathan)
- Document the negative errno return of cxl_port_interleave_pos()
Patch 3: cxl/region: Support auto region granularity finer than the root
- Put peer distance walk and the iw/ig calculation in helpers (Robert)
- Return the peer distance of 1 explicitly for regions finer than the root,
  and explain why in the comment (Jonathan, Robert)
- Rename cxl_region_validate_interleave() to check_root_interleave() and
  move it next to check_interleave_cap() (Robert)
- Reword the span exemption comment to describe only the same-granularity
  power-of-two exception (Robert)
- Simplify the commit message description of the finer-than-region check (Robert)
- Describe the span identity as a requirement of the supported layouts (Robert)
Patch 4: cxl/region: Support user region granularity finer than the root
- Comment why a region granularity coarser than the root is rejected (Robert)
Patch 5: cxl/test: Add a topology for region granularity finer than the root
- Map the mock memdevs to online NUMA nodes, like other mock topologies
- Skip unpopulated entries in the finer-gran memdev cleanup loops
v5: https://lore.kernel.org/linux-cxl/cover.1788475206.git.alison.schofield@intel.com/


Begin Cover Letter:

Linux requires a region's interleave granularity to equal the interleave
granularity of its interleaving root decoder. This series relaxes that one
restriction: a region granularity may be finer than the root granularity,
provided the interleave levels between them refine monotonically toward
the endpoints.

This series does not add general support for every granularity arrangement
the CXL Specification permits. A region granularity coarser than the root
remains invalid, and configurations whose granularity refines and then
coarsens along the decoder path remain unsupported. Of the Mod3
configurations listed in CXL 4.0 Section 9.13.1.1, all are supported
except Table 9-6 row 6, the refine-then-coarsen 12-way composition.

The use case driving this series is 6- and 12-way regions behind a 3-way
host bridge interleave, CXL 4.0 Tables 9-6 and 9-7, which have no
same-granularity alternative. Power-of-two regions finer than the root
fall out of the same implementation.

Support applies to both firmware programmed auto regions and user created
regions.

Two prior proposals addressed parts of this gap:

AlisonS introduced position arithmetic and sysfs gating for auto and
user-created regions to support the 6- and 12-way interleave configurations
without a same-granularity alternative [2].

RobertR introduced an HPA selector-bit model for auto regions that allows
multi-level power-of-two interleaves regardless of granularity ordering [3].

Series structure in v6:
-----------------------
Patch 1 documents which region granularities Linux supports relative to the
root and the monotonic refinement model.

Patch 2 generalizes endpoint position calculation to account for granularity
changes between decoder levels.

Patch 3 enables auto regions with a granularity finer than the root and
validates their decoder geometry.

Patch 4 enables user created regions with a granularity finer than the root
by relaxing the existing root/region granularity restriction.

Patch 5 adds a cxl_test topology and coverage for region granularity finer
than the root.

Dependencies:
-------------
This series is built upon cxl/next, plus the refactor series [1].

The companion NDCTL patchset that allows these regions in 'cxl create-region'
and adds the unit test support is posted [4].


[1] https://lore.kernel.org/linux-cxl/cover.1791860305.git.alison.schofield@intel.com/
[2] https://lore.kernel.org/all/20250306232239.2609017-1-alison.schofield@intel.com/
[3] https://lore.kernel.org/all/20251028094754.72816-1-rrichter@amd.com/
[4] https://lore.kernel.org/linux-cxl/cover.1791863817.git.alison.schofield@intel.com/


Alison Schofield (5):
  Documentation/cxl: Describe region granularity finer than the root
  cxl/region: Generalize endpoint position mapping
  cxl/region: Support auto region granularity finer than the root
  cxl/region: Support user region granularity finer than the root
  cxl/test: Add a topology for region granularity finer than the root

 .../driver-api/cxl/linux/cxl-driver.rst       | 119 +++++
 drivers/cxl/core/region.c                     | 220 ++++++---
 tools/testing/cxl/test/cxl.c                  | 420 +++++++++++++++++-
 3 files changed, 681 insertions(+), 78 deletions(-)

-- 
2.37.3