[PATCH v5 0/7] cxl: Support mixed-granularity region interleaves

[PATCH v5 0/7] cxl: Support mixed-granularity region interleaves

[Alison Schofield]

JonathanC, I kept your tags on Patches 2,5.

RobertR, Same comment as in v4. I've dropped the Originally-by tags since the
implementation no longer carries the selector-bit approach from your original
patch. Let me know what authorship tags you'd like.


Changes in v5:
Series wide:
- Rebase onto 7.3-rc1
- Move the documentation patch to the front of the series (Jonathan)
- Reword commit messages to use new terms of documentation (Jonathan)
- Wrap code and code comments closer to 80 columns (Jonathan)
- Rewrap all commit messages to 75 columns (Jonathan)
- Split combined declarations (Jonathan)
Patch 1: Documentation/cxl: Describe mixed-granularity regions
- Define a mixed-granularity region and monotonic requirement (Jonathan)
- Define span, region position, and weight (Jonathan)
- Explain refining granularity and cross-link first (Jonathan)
- Use an 8-way example interleaved at the host bridge w one switch (Jonathan)
- Clarify config 'legal per the CXL Spec but unsupported by Linux' (Jonathan)
Patch 2: cxl/region: Warn on user region position mismatch
- Log the position mismatch against the region device and name the decoder
Patch 3: cxl/region: Generalize endpoint position mapping
- Pass the region interleave granularity into cxl_calc_interleave_pos() instead
  of using the endpoint decoder granularity (RichardC)
- Rewrite the example comment to define a region position and each decoder
  level's weight, rather than what it spans (Jonathan)
New Patch 4: cxl/region: Name the interleave locals in cxl_port_setup_targets()
- Rename iw and ig to child_iw and child_ig (Jonathan)
Patch 5: cxl/region: Support mixed-granularity auto regions
- Restore the topology-derived peer distance, using 1 only for mixed-granularity
  regions, avoiding a NULL deref on the auto path
- Reject a layout whose derived granularity comes out finer than the region
  granularity, which the topology-derived distance used to catch
- Define what a span is where the root and region spans are compared (Jonathan)
- Add comment that the power-of-two exemption is for same-granularity regions
  that span multiple root interleaves, and that Mod3 roots are width-matched
  instead (Jonathan, Sashiko)
- Report both spans in the span-check dev_dbg
Patch 6: cxl/region: Support mixed-granularity user created regions
- Add a dev_dbg naming both granularities to the coarser-than-root rejection
Patch 7: cxl/test: Add a topology to test mixed-granularity regions
- Rebuild the topology as cross-HB, then HB, then switch (Jonathan)
- Add a third window to support a 12-way region at 256
- Drop the cascade of two switch levels (Jonathan)
- Replace bare '2' with NR_CXL_MIX_GRAN_RPS_PER_HB
- Create and use a chbs_range() helper
Link to v4:
https://lore.kernel.org/linux-cxl/cover.1787255388.git.alison.schofield@intel.com/

Changes in v4:
- Rework port decoder setup around parent granularity and target count,
  dropping the selector-walk and selector-derived granularity machinery
  from v3 (RobertR)
- Replace the mixed-granularity-specific endpoint position handling with
  one weighted position calculation for existing and mixed-gran regions
- Derive mixed-gran port decoder settings directly from the parent interleave
  geometry
- Promote the user-region position self-test from dev_dbg() to dev_warn()
- Update the documentation patch
- Update the cover letter summary and series structure
Link to v3:
https://lore.kernel.org/linux-cxl/cover.1785444498.git.alison.schofield@intel.com/

Changes in v3:
- Reduce scope to coarse-to-fine ordering (RichardC, Sashiko)
- Reject out-of-order layouts (RichardC, Sashiko)
- P1: Factor the selector walk and per-port checks into helpers (RobertR)
  That pre-work is the new Patch 1/8 and the Series Structure section of
  this cover letter describe same below.
- P3: Derive granularity from the highest available selector span (RobertR)
- P3: Validate auto-programmed granularity against the derived value
- P4: Rename root_pos_stride() to root_positions_per_target() (RobertR)
- P4: Drop the "stride" terminology throughout (RobertR)
- The complex Patch 4 of v2 is split into patches 4-6 in v3.
  Also noted in the Series Structure section of cover letter below.
- P9: Move the peer-distance walk and per-function walkthroughs from the doc
  into in-code comments and kernel-doc (RobertR)
- Drop Reviewed-by tags
Link to v2:
https://lore.kernel.org/linux-cxl/cover.1781199122.git.alison.schofield@intel.com/

Changes in v2:
- Patch 1,2: Defer the unused selector var store to keep P1 bisectable (Sashiko)
- Patch 1: Make divide by 3 in get_selctor() work on 32-bit builds) (lkp)
- Patch 4: Use local vars in cxl_region_attach() for readability (DaveJ)
- Patch 5: Add NULL checks on unused mock arrays (Sashiko)
- Resolved errant err_rch unwind with rc7 merge (DaveJ)
- Rebase onto 7.1-rc7
- Update commit logs in 1,2,5 to align w changes in v2
Link to v1:
https://lore.kernel.org/all/cover.1780095671.git.alison.schofield@intel.com/


Begin Cover Letter:

Linux has required that a region's interleave granularity equal the interleave
granularity of its interleaving root decoder. A mixed-granularity region lifts
that restriction: the region granularity may be finer than the root's, so the
interleave refines from the root toward the endpoints. The mix is between the
root and the region, not granularity varying arbitrarily down the hierarchy.

A CXL region may interleave across multiple decoder levels: root, optional
switches, and endpoint. The equal-granularity requirement blocks legal
mixed-granularity arrangements permitted by CXL 4.0 Section 9.13.1 and makes
some 6-way and 12-way configurations defined in Section 9.13.1.1 (Tables 9-6,
9-7) impossible to create.

Two prior proposals addressed parts of this gap:

AlisonS introduced position arithmetic and sysfs gating for auto and
user-created regions to support the 6- and 12-way interleave configurations
without a same-granularity alternative:
https://lore.kernel.org/all/20250306232239.2609017-1-alison.schofield@intel.com/

RobertR introduced an HPA selector-bit model for auto regions that allows
multi-level power-of-two interleaves regardless of granularity ordering:
https://lore.kernel.org/all/20251028094754.72816-1-rrichter@amd.com/

This series adds support for CXL regions where interleave granularity differs
between levels of the decoder hierarchy. The support applies to both firmware
programmed auto regions and user created regions.

Linux supports mixed-granularity configurations that are monotonic from the
interleaving root toward the endpoints. Granularity may remain the same or
become finer at each interleaving level. Configurations that refine and then
become coarser are permitted by the CXL Specification but are not supported
by Linux.

This support covers power-of-two and Mod3 interleaves at any depth. Of the Mod3
configs listed in CXL 4.0 Section 9.13.1.1, all are supported except Table 9-6
row 6, the refine-then-coarsen 12-way composition. See the Patch 1 documentation
update for additional info.


Series structure in v5:
-----------------------
Patch 1 documents the mixed-granularity model and Linux support policy.
It comes first so that the terms the rest of the series uses - span,
region position, weight - are defined before they are used.

Patch 2 promotes the existing user-region endpoint position self-test to
a visible diagnostic.

Patch 3 generalizes endpoint position calculation to account for
granularity changes between decoder levels.

Patch 4 names the interleave locals in cxl_port_setup_targets(). It is a
mechanical rename, carried separately so it does not muddy Patch 5.

Patch 5 enables mixed-granularity auto regions and validates their
coarse-to-fine decoder geometry.

Patch 6 enables mixed-granularity user created regions by relaxing the
existing root/region granularity restriction.

Patch 7 adds a cxl_test topology and coverage for mixed-granularity
regions.


The companion NDCTL patchset that allows mixed-granularity 'cxl create-region'
and adds the unit test is posted here:
https://lore.kernel.org/nvdimm/cover.1788469004.git.alison.schofield@intel.com/


Alison Schofield (7):
  Documentation/cxl: Describe mixed-granularity regions
  cxl/region: Warn on user region position mismatch
  cxl/region: Generalize endpoint position mapping
  cxl/region: Name the interleave locals in cxl_port_setup_targets()
  cxl/region: Support mixed-granularity auto regions
  cxl/region: Support mixed-granularity user created regions
  cxl/test: Add a topology to test mixed-granularity regions

 .../driver-api/cxl/linux/cxl-driver.rst       | 153 +++++++
 drivers/cxl/core/region.c                     | 247 +++++++----
 tools/testing/cxl/test/cxl.c                  | 410 +++++++++++++++++-
 3 files changed, 698 insertions(+), 112 deletions(-)


base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
-- 
2.37.3

[PATCH v5 1/7] Documentation/cxl: Describe mixed-granularity regions

[Alison Schofield]

Linux has required that a region's interleave granularity equal the
interleave granularity of its interleaving root decoder. A mixed-
granularity region lifts that restriction: the region granularity may be
finer than the root's, so the interleave refines from the root toward the
endpoints.

Mixed-granularity region support introduces interleave relationships that
are not obvious from the existing CXL documentation. This is particularly
true for the 3-, 6-, and 12-way configurations and for understanding which
configurations permitted by the CXL Specification are supported by Linux.

Document the mixed-granularity model and Linux's coarse-to-fine
restriction. Include the relevant configurations from CXL 4.0 Section
9.13.1.1 and annotate their Linux support status.

This intentionally repeats information from the CXL Specification. The
specification remains authoritative, but showing the Linux support policy
alongside the legal configurations ensures readers do not mistake an
unsupported Linux setup for a configuration unsupported by the CXL
Specification.

Assisted-by: Claude:Opus-5
Signed-off-by: Alison Schofield <alison.schofield@intel.com>
---
 .../driver-api/cxl/linux/cxl-driver.rst       | 153 ++++++++++++++++++
 1 file changed, 153 insertions(+)

diff --git a/Documentation/driver-api/cxl/linux/cxl-driver.rst b/Documentation/driver-api/cxl/linux/cxl-driver.rst
index dd6dd17dc536..a61a20c8bb6d 100644
--- a/Documentation/driver-api/cxl/linux/cxl-driver.rst
+++ b/Documentation/driver-api/cxl/linux/cxl-driver.rst
@@ -602,6 +602,11 @@ derived from their upstream port connections. In `Cross-Link First` interleave
 configurations, the :code:`interleave_granularity` of a decoder is equal to
 :code:`parent_interleave_granularity * parent_interleave_ways`.
 
+When the region granularity is finer than the granularity of an interleaving
+root decoder, the relation inverts: the :code:`interleave_granularity` of a
+decoder is equal to :code:`parent_interleave_granularity / interleave_ways`.
+See `Mixed Granularity`_.
+
 At Endpoint
 ~~~~~~~~~~~
 `Endpoint Decoders` are programmed similar to Host Bridge and Switch decoders,
@@ -619,6 +624,154 @@ from HPA to DPA.  This is why they must be aware of the entire interleave set.
 Linux does not support unbalanced interleave configurations.  As a result, all
 endpoints in an interleave set must have the same ways and granularity.
 
+Mixed Granularity
+~~~~~~~~~~~~~~~~~
+Linux has required that a region's :code:`interleave_granularity` equal the
+:code:`interleave_granularity` of its interleaving root decoder.  A
+*mixed-granularity* region lifts that restriction: the region granularity may
+be finer than the root's, so the interleave refines from the root toward the
+endpoints.  The mix is between the root and the region, not granularity varying
+arbitrarily down the hierarchy.  Linux requires granularity to change
+monotonically, either coarsening or refining, across the decoders that route
+the request: the interleaving root, the host bridge, and any switches.  The
+endpoint decoder is not part of that walk, it carries the region ways and
+granularity so that it can translate, as `At Endpoint`_ describes above.  A
+mixed-granularity region requires an interleaving root decoder.
+
+Every decoder advances one target every multiple of its own granularity, and
+the decoders below it subdivide the address range their parent assigns to a
+single target.  The `Cross-Link First` example above shows the other ordering,
+where the region granularity equals the granularity of the root decoder and
+granularity coarsens toward the endpoints.  A mixed-granularity region refines
+instead, reaching the region granularity at the innermost interleaving decoder.
+
+Refining is not a preference.  Once the region granularity is finer than the
+root's, Linux gives each level below the root a single granularity, its
+parent's divided by its own ways, so the ordering follows from the region and
+root settings rather than being chosen.  The CXL Specification permits other
+orderings and does not require a monotonic one, see `Mod3 Interleave
+Configurations`_.  Reaching a fine granularity as early as possible remains
+available through `Cross-Link First`, which this does not change.
+
+For an 8-way mixed-granularity region at 1024 below a 2-way interleaving root
+decoder at 4096, where each host bridge interleaves across two root ports and
+each root port hosts a 2-way switch, Linux programs::
+
+  Level             Ways    Granularity
+  -----             ----    -----------
+  Root                 2         4096
+  Host bridge          2         2048
+  Switch               2         1024
+  Endpoint             8         1024
+
+A *region position* is the index of a region-granularity chunk within one full
+pass of the region interleave, and each endpoint of the region backs one
+position.  Each decoder contributes to an endpoint's region position in
+proportion to its granularity::
+
+  position += target_position *
+              decoder_granularity / region_granularity
+
+That multiplier is the *weight* of a level: how many region positions pass
+between successive advances of its target index.  The root above selects a host
+bridge every 4096 bytes, so it advances one target every four region positions,
+weight 4, while the switch advances one target every position, weight 1.
+
+The two orderings weight the levels in opposite directions.  Refining, a
+level's weight is the product of the ways of the interleaving levels below it,
+so the innermost interleaving level has weight 1.  Coarsening, as in
+`Cross-Link First`, it is the product of the ways of the levels above it, so
+the root has weight 1 and advances one target per region position.
+
+The ways and granularity of a mixed-granularity region must describe the same
+interleave *span* as the root decoder, the address range in which the
+interleave pattern completes one full pass::
+
+  root_ways * root_granularity == region_ways * region_granularity
+
+A region that does not describe that span either leaves part of the range
+unclaimed or reaches beyond it, and Linux rejects it.  A same-granularity
+region below a power-of-two root decoder is exempt: it may be wider than the
+root and repeat the root targets, so its span is a multiple of the root's.
+
+Mod3 Interleave Configurations
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+A 3-way, 6-way, or 12-way interleave, known as a Mod3 interleave, selects its
+target with a factor-of-three selection rather than from binary interleave
+selector bits alone.  CXL 4.0 Section 9.13.1.1 defines the 3-way selection as
+the address above the decoder granularity taken modulo 3.  A 6-way selection
+claims one binary HPA bit at the decoder granularity and takes the modulo 3 of
+the address above that bit, and a 12-way selection claims two.
+
+The factor-of-three selection is not an additional binary selector bit, so a
+Mod3 interleave distributes across the hierarchy as::
+
+  3  = 3
+  6  = 3 * 2
+  12 = 3 * 4
+
+The cross-host bridge selection carries the factor of three, and the remaining
+x2 or x4 is binary interleave selection below it.  A 6-way region at IGB across
+three host bridges is therefore::
+
+  Device-level region:   6-way @ IGB
+  Cross-host bridge:     3-way @ 2*IGB
+  Below the root:        2-way @ IGB
+
+where both levels describe the same interleave span::
+
+  3 * (2 * IGB) == 6 * IGB
+
+The CXL Specification defines the legal Mod3 compositions and is normative.
+CXL 4.0 Section 9.13.1.1, "Legal Interleaving Configurations: 12-way, 6-way,
+and 3-way", Tables 9-6, 9-7, and 9-8 list them for a 12-way, 6-way, and 3-way
+device-level interleave at IGB.  Those tables are summarized below, annotated
+with the subset Linux supports.
+
+CXL 4.0 Table 9-8, 3-way device-level interleave at IGB::
+
+  Row  Cross-host bridge  Host bridge    Switch         Linux
+  ---  -----------------  -----------    ------         -----
+   1   3-way @ IGB        none           none           supported
+
+CXL 4.0 Table 9-7, 6-way device-level interleave at IGB::
+
+  Row  Cross-host bridge  Host bridge    Switch         Linux
+  ---  -----------------  -----------    ------         -----
+   1   6-way @ IGB        none           none           supported
+   2   3-way @ 2*IGB      2-way @ IGB    none           supported
+   3   3-way @ 2*IGB      none           2-way @ IGB    supported
+
+CXL 4.0 Table 9-6, 12-way device-level interleave at IGB::
+
+  Row  Cross-host bridge  Host bridge    Switch         Linux
+  ---  -----------------  -----------    ------         -----
+   1   12-way @ IGB       none           none           supported
+   2   6-way @ 2*IGB      2-way @ IGB    none           supported
+   3   6-way @ 2*IGB      none           2-way @ IGB    supported
+   4   3-way @ 4*IGB      4-way @ IGB    none           supported
+   5   3-way @ 4*IGB      none           4-way @ IGB    supported
+   6   3-way @ 4*IGB      2-way @ IGB    2-way @ 2*IGB  unsupported
+   7   3-way @ 4*IGB      2-way @ 2*IGB  2-way @ IGB    supported
+
+Table 9-6 row 6 is legal per the CXL Specification but is unsupported by Linux.
+Walking it from the root toward the endpoints, granularity goes::
+
+  4*IGB -> IGB -> 2*IGB
+
+which refines and then coarsens.  Row 7 interleaves the same 12 endpoints at
+the same granularity with those two levels exchanged::
+
+  4*IGB -> 2*IGB -> IGB
+
+which is monotonic.  Linux programs row 7 for a user region and assembles an
+auto region whose decoders are programmed that way.  An auto region matching
+row 6 is not assembled.
+
+Leaving row 6 unsupported does not prevent a 12-way device-level interleave.
+The specification defines six other legal compositions, all monotonic and
+supported by Linux.
+
 Example Configurations
 ======================
 .. toctree::
-- 
2.37.3

[PATCH v5 2/7] cxl/region: Warn on user region position mismatch

[Alison Schofield]

User region creation includes a self-test that checks the assigned endpoint
positions against the position calculation used by auto region creation.
The result, matching or not, is reported with dev_dbg().

Promote that to a dev_warn() that fires only on a mismatch, so bugs in the
auto region position calculation are easier to catch as that calculation is
extended to support more region configurations.

The matching case no longer prints, so dynamic debug on region.c no longer
confirms that the self-test ran.

Signed-off-by: Alison Schofield <alison.schofield@intel.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>
---
 drivers/cxl/core/region.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index 27e63e6dab7c..6a698f36aa6d 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -2215,10 +2215,11 @@ static int cxl_region_attach(struct cxl_region *cxlr,
 		int test_pos;
 
 		test_pos = cxl_calc_interleave_pos(target, &cxlr->hpa_range);
-		dev_dbg(&target->cxld.dev,
-			"Test cxl_calc_interleave_pos(): %s test_pos:%d target->pos:%d\n",
-			(test_pos == target->pos) ? "success" : "fail",
-			test_pos, target->pos);
+		if (test_pos != target->pos)
+			dev_warn(&cxlr->dev,
+				 "%s: position mismatch: calculated:%d assigned:%d\n",
+				 dev_name(&target->cxld.dev), test_pos,
+				 target->pos);
 	}
 
 	return 0;
-- 
2.37.3

[PATCH v5 3/7] cxl/region: Generalize endpoint position mapping

[Alison Schofield]

Endpoint position calculation currently relies on the requirement that an
interleaving root have the same granularity as the region. Auto region
creation builds the position by multiplying by the parent decoders' ways,
while user region creation selects the root target with 'pos % ways'. Those
calculations are sufficient under the current granularity restriction.

In order to support mixed-granularity regions, that granularity restriction
will need to be removed so decoder granularity can change between levels of
the interleave hierarchy. The position calculation needs to account for
those changes to produce the correct endpoint ordering.

Change the position calculation so each decoder's contribution is weighted
by its granularity relative to the region granularity:

    position += target_pos *
                (decoder_granularity / region_granularity)

Use the same relationship to select the root target during user region
creation.

Weight each level by the region granularity, passed in by the caller,
rather than by the granularity programmed in the endpoint decoder. The two
are equal for most configurations, but not when Normalized Addressing
leaves the endpoint decoder programmed passthrough while the region
interleaves.

For currently supported regions, the new weighted calculation reduces to
the existing position calculation and produces identical endpoint
positions. The exception is a region wider than a same-granularity Mod3
root, where address bit routing programs the level below the root at the
region granularity, so the ratio derives a weight of one rather than three.
A later patch in this series rejects that layout.

Signed-off-by: Alison Schofield <alison.schofield@intel.com>
---
 drivers/cxl/core/region.c | 69 ++++++++++++++++++++++++---------------
 1 file changed, 42 insertions(+), 27 deletions(-)

diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index 6a698f36aa6d..506b1cba1a92 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -1806,9 +1806,15 @@ static int cxl_region_attach_position(struct cxl_region *cxlr,
 	struct cxl_decoder *cxld = &cxlsd->cxld;
 	int iw = cxld->interleave_ways;
 	struct cxl_port *iter;
+	int root_pos = pos;
 	int rc;
 
-	if (dport != cxlrd->cxlsd.target[pos % iw]) {
+	/* Root target selection advances at root-granularity intervals */
+	if (iw > 1)
+		root_pos = pos * cxlr->params.interleave_granularity /
+			   cxld->interleave_granularity;
+
+	if (dport != cxlrd->cxlsd.target[root_pos % iw]) {
 		dev_dbg(&cxlr->dev, "%s:%s invalid target position for %s\n",
 			dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev),
 			dev_name(&cxlrd->cxlsd.cxld.dev));
@@ -1909,12 +1915,13 @@ static int match_switch_decoder_by_range(struct device *dev,
 	return (r1->start == r2->start && r1->end == r2->end);
 }
 
-static int find_pos_and_ways(struct cxl_port *port, struct range *range,
-			     int *pos, int *ways)
+static int find_pos_and_gran(struct cxl_port *port, struct range *range,
+			     int *pos, int *gran)
 {
 	struct cxl_switch_decoder *cxlsd;
 	struct cxl_port *parent;
 	int rc = -ENXIO;
+	int ways;
 
 	parent = parent_port_of(port);
 	if (!parent)
@@ -1929,9 +1936,10 @@ static int find_pos_and_ways(struct cxl_port *port, struct range *range,
 		return rc;
 	}
 	cxlsd = to_cxl_switch_decoder(dev);
-	*ways = cxlsd->cxld.interleave_ways;
+	ways = cxlsd->cxld.interleave_ways;
+	*gran = cxlsd->cxld.interleave_granularity;
 
-	for (int i = 0; i < *ways; i++) {
+	for (int i = 0; i < ways; i++) {
 		if (cxlsd->target[i] == port->parent_dport) {
 			*pos = i;
 			rc = 0;
@@ -1951,24 +1959,29 @@ static int find_pos_and_ways(struct cxl_port *port, struct range *range,
  * cxl_calc_interleave_pos() - calculate an endpoint position in a region
  * @cxled: endpoint decoder member of given region
  * @hpa_range: translated HPA range of the endpoint
+ * @region_gran: interleave granularity of the region
  *
- * The endpoint position is calculated by traversing the topology from
- * the endpoint to the root decoder and iteratively applying this
- * calculation:
+ * A region position is the index of a region-granularity chunk within one full
+ * pass of the region interleave. The position is calculated by traversing the
+ * topology from the endpoint to the root decoder and accumulating the
+ * contribution of each decoder level:
  *
- *    position = position * parent_ways + parent_pos;
+ *    position += parent_pos * (parent_granularity / region_gran);
  *
- * ...where @position is inferred from switch and root decoder target lists.
+ * ...where @parent_pos is inferred from switch and root decoder target lists.
+ * The multiplier is the weight of that level: how many region positions pass
+ * between successive advances of the level's target index. A level that selects
+ * a single target contributes nothing.
  *
  * Return: position >= 0 on success
  *	   -ENXIO on failure
  */
 static int cxl_calc_interleave_pos(struct cxl_endpoint_decoder *cxled,
-				   struct range *hpa_range)
+				   struct range *hpa_range, int region_gran)
 {
 	struct cxl_port *iter, *port = cxled_to_port(cxled);
 	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
-	int parent_ways = 0, parent_pos = 0, pos = 0;
+	int parent_gran = 0, parent_pos = 0, pos = 0;
 	int rc;
 
 	/*
@@ -1981,20 +1994,20 @@ static int cxl_calc_interleave_pos(struct cxl_endpoint_decoder *cxled,
 	 *        |    |           |    |
 	 *       mem0 mem1        mem2 mem3
 	 *
-	 * In the example the calculator will iterate twice. The first iteration
-	 * uses the mem position in the host-bridge and the ways of the host-
-	 * bridge to generate the first, or local, position. The second
-	 * iteration uses the host-bridge position in the root_port and the ways
-	 * of the root_port to refine the position.
+	 * The region and the root decoder interleave at the region
+	 * granularity, so each host-bridge decoder interleaves at twice that.
+	 * The host-bridge decoders advance one target every two region
+	 * positions, weight 2, and the root decoder advances one target every
+	 * region position, weight 1.
 	 *
 	 * A trace of the calculation per endpoint looks like this:
-	 * mem0: pos = 0 * 2 + 0    mem2: pos = 0 * 2 + 0
-	 *       pos = 0 * 2 + 0          pos = 0 * 2 + 1
+	 * mem0: pos += 0 * 2       mem2: pos += 0 * 2
+	 *       pos += 0 * 1             pos += 1 * 1
 	 *       pos: 0                   pos: 1
 	 *
-	 * mem1: pos = 0 * 2 + 1    mem3: pos = 0 * 2 + 1
-	 *       pos = 1 * 2 + 0          pos = 1 * 2 + 1
-	 *       pos: 2                   pos = 3
+	 * mem1: pos += 1 * 2       mem3: pos += 1 * 2
+	 *       pos += 0 * 1             pos += 1 * 1
+	 *       pos: 2                   pos: 3
 	 *
 	 * Note that while this example is simple, the method applies to more
 	 * complex topologies, including those with switches.
@@ -2005,12 +2018,12 @@ static int cxl_calc_interleave_pos(struct cxl_endpoint_decoder *cxled,
 		if (is_cxl_root(iter))
 			break;
 
-		rc = find_pos_and_ways(iter, hpa_range, &parent_pos,
-				       &parent_ways);
+		rc = find_pos_and_gran(iter, hpa_range, &parent_pos,
+				       &parent_gran);
 		if (rc)
 			return rc;
 
-		pos = pos * parent_ways + parent_pos;
+		pos += parent_pos * (parent_gran / region_gran);
 	}
 
 	dev_dbg(&cxlmd->dev,
@@ -2029,7 +2042,8 @@ static int cxl_region_sort_targets(struct cxl_region *cxlr)
 	for (i = 0; i < p->nr_targets; i++) {
 		struct cxl_endpoint_decoder *cxled = p->targets[i];
 
-		cxled->pos = cxl_calc_interleave_pos(cxled, &cxlr->hpa_range);
+		cxled->pos = cxl_calc_interleave_pos(cxled, &cxlr->hpa_range,
+						     p->interleave_granularity);
 		/*
 		 * Record that sorting failed, but still continue to calc
 		 * cxled->pos so that cxl_calc_interleave_pos() emits its
@@ -2214,7 +2228,8 @@ static int cxl_region_attach(struct cxl_region *cxlr,
 		struct cxl_endpoint_decoder *target = p->targets[i];
 		int test_pos;
 
-		test_pos = cxl_calc_interleave_pos(target, &cxlr->hpa_range);
+		test_pos = cxl_calc_interleave_pos(target, &cxlr->hpa_range,
+						   p->interleave_granularity);
 		if (test_pos != target->pos)
 			dev_warn(&cxlr->dev,
 				 "%s: position mismatch: calculated:%d assigned:%d\n",
-- 
2.37.3

[PATCH v5 4/7] cxl/region: Name the interleave locals in cxl_port_setup_targets()

[Alison Schofield]

cxl_port_setup_targets() derives the interleave granularity of one decoder
from the granularity and ways of the decoder above it, and takes that
decoder's own ways from the topology below it. The values from the level
above are named parent_iw and parent_ig, but the values being derived are
the bare iw and ig, so the reader has to work out which level each one
describes.

Rename iw and ig to child_iw and child_ig to match the parent_ prefix used
for the level above.

No functional change.

Suggested-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>
Signed-off-by: Alison Schofield <alison.schofield@intel.com>
---
 drivers/cxl/core/region.c | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index 506b1cba1a92..116f84bb0745 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -1440,7 +1440,7 @@ static int cxl_port_setup_targets(struct cxl_port *port,
 				  struct cxl_endpoint_decoder *cxled)
 {
 	struct cxl_root_decoder *cxlrd = cxlr->cxlrd;
-	int parent_iw, parent_ig, ig, iw, rc, pos = cxled->pos;
+	int parent_iw, parent_ig, child_iw, child_ig, rc, pos = cxled->pos;
 	struct cxl_port *parent_port = to_cxl_port(port->dev.parent);
 	struct cxl_region_ref *cxl_rr = cxl_rr_load(port, cxlr);
 	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
@@ -1538,11 +1538,11 @@ static int cxl_port_setup_targets(struct cxl_port *port,
 		return rc;
 	}
 
-	iw = cxl_rr->nr_targets;
-	rc = ways_to_eiw(iw, &eiw);
+	child_iw = cxl_rr->nr_targets;
+	rc = ways_to_eiw(child_iw, &eiw);
 	if (rc) {
 		dev_dbg(&cxlr->dev, "%s:%s: invalid port interleave: %d\n",
-			dev_name(port->uport_dev), dev_name(&port->dev), iw);
+			dev_name(port->uport_dev), dev_name(&port->dev), child_iw);
 		return rc;
 	}
 
@@ -1558,7 +1558,7 @@ static int cxl_port_setup_targets(struct cxl_port *port,
 		return rc;
 	}
 
-	rc = eig_to_granularity(eig, &ig);
+	rc = eig_to_granularity(eig, &child_ig);
 	if (rc) {
 		dev_dbg(&cxlr->dev, "%s:%s: invalid interleave: %d\n",
 			dev_name(port->uport_dev), dev_name(&port->dev),
@@ -1566,23 +1566,23 @@ static int cxl_port_setup_targets(struct cxl_port *port,
 		return rc;
 	}
 
-	if (iw > 8 || iw > cxlsd->nr_targets) {
+	if (child_iw > 8 || child_iw > cxlsd->nr_targets) {
 		dev_dbg(&cxlr->dev,
 			"%s:%s:%s: ways: %d overflows targets: %d\n",
 			dev_name(port->uport_dev), dev_name(&port->dev),
-			dev_name(&cxld->dev), iw, cxlsd->nr_targets);
+			dev_name(&cxld->dev), child_iw, cxlsd->nr_targets);
 		return -ENXIO;
 	}
 
 	if (test_bit(CXL_REGION_F_AUTO, &cxlr->flags)) {
-		if (cxld->interleave_ways != iw ||
-		    (iw > 1 && cxld->interleave_granularity != ig) ||
+		if (cxld->interleave_ways != child_iw ||
+		    (child_iw > 1 && cxld->interleave_granularity != child_ig) ||
 		    !spa_maps_hpa(p, &cxld->hpa_range) ||
 		    ((cxld->flags & CXL_DECODER_F_ENABLE) == 0)) {
 			dev_err(&cxlr->dev,
 				"%s:%s %s expected iw: %d ig: %d %pr\n",
 				dev_name(port->uport_dev), dev_name(&port->dev),
-				__func__, iw, ig, p->res);
+				__func__, child_iw, child_ig, p->res);
 			dev_err(&cxlr->dev,
 				"%s:%s %s got iw: %d ig: %d state: %s %#llx:%#llx\n",
 				dev_name(port->uport_dev), dev_name(&port->dev),
@@ -1593,24 +1593,24 @@ static int cxl_port_setup_targets(struct cxl_port *port,
 			return -ENXIO;
 		}
 	} else {
-		rc = check_interleave_cap(cxld, iw, ig);
+		rc = check_interleave_cap(cxld, child_iw, child_ig);
 		if (rc) {
 			dev_dbg(&cxlr->dev,
 				"%s:%s iw: %d ig: %d is not supported\n",
 				dev_name(port->uport_dev),
-				dev_name(&port->dev), iw, ig);
+				dev_name(&port->dev), child_iw, child_ig);
 			return rc;
 		}
 
-		cxld->interleave_ways = iw;
-		cxld->interleave_granularity = ig;
+		cxld->interleave_ways = child_iw;
+		cxld->interleave_granularity = child_ig;
 		cxld->hpa_range = (struct range) {
 			.start = p->res->start,
 			.end = p->res->end,
 		};
 	}
 	dev_dbg(&cxlr->dev, "%s:%s iw: %d ig: %d\n", dev_name(port->uport_dev),
-		dev_name(&port->dev), iw, ig);
+		dev_name(&port->dev), child_iw, child_ig);
 add_target:
 	if (cxl_rr->nr_targets_set == cxl_rr->nr_targets) {
 		dev_dbg(&cxlr->dev,
-- 
2.37.3

[PATCH v5 5/7] cxl/region: Support mixed-granularity auto regions

[Alison Schofield]

The CXL Specification permits a region's interleave granularity to differ
between levels of the decoder hierarchy. For an auto region, the CXL driver
reconstructs that hierarchy from the decoder configuration programmed by
platform firmware.

The CXL driver currently requires the interleaving root to have the same
granularity as the region, with each successive interleaving level becoming
coarser toward the endpoints. A mixed-granularity region lifts that
requirement: its interleave granularity is finer than the granularity of
its interleaving root decoder. The driver cannot assemble such a region
today.

Support mixed-granularity auto regions in the CXL driver, restricted to
coarse-to-fine layouts. Derive each interleaving decoder's granularity from
its parent:

        child_ig = parent_ig / child_iw

A same-granularity region keeps the existing derivation, which coarsens
toward the endpoints:

        child_ig = parent_ig * parent_iw

The topology determines each decoder's interleave ways, so the parent
granularity and child ways determine the child granularity. Use that
relationship to validate the decoder geometry while assembling an auto
region. Only the interleaving levels are compared; a passthrough level,
child_iw == 1, keeps its granularity exemption.

Refuse a derived granularity finer than the region granularity. Dividing at
every level lands the deepest interleaving level exactly on the region
granularity, so a finer value means the topology below one port is deeper
than the region interleave has positions for, which happens only when
another branch is too shallow to cover its share. That is what rejects an
unbalanced tree below the interleaving root.

Require the root and region to interleave over the same span, the address
range in which the interleave pattern completes one full pass:

        root_iw * root_ig == region_iw * region_ig

The same span relationship covers the CXL Specification's Mod3
configurations. For example, a 6-way region at IGB across three host
bridges uses a 3-way root interleave at 2 * IGB and a 2-way interleave at
IGB below it, as described in CXL 4.0 Section 9.13.1.1.

Signed-off-by: Alison Schofield <alison.schofield@intel.com>
---
 drivers/cxl/core/region.c | 128 ++++++++++++++++++++++++++------------
 1 file changed, 89 insertions(+), 39 deletions(-)

diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index 116f84bb0745..5b7104a55fdd 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -1435,6 +1435,15 @@ static int check_interleave_cap(struct cxl_decoder *cxld, int iw, int ig)
 	return 0;
 }
 
+/* Mixed granularity has a region IG finer than the interleaving root IG */
+static bool cxl_region_is_mixed_gran(struct cxl_region *cxlr)
+{
+	struct cxl_decoder *cxld = &cxlr->cxlrd->cxlsd.cxld;
+
+	return cxld->interleave_ways > 1 &&
+	       cxld->interleave_granularity > cxlr->params.interleave_granularity;
+}
+
 static int cxl_port_setup_targets(struct cxl_port *port,
 				  struct cxl_region *cxlr,
 				  struct cxl_endpoint_decoder *cxled)
@@ -1448,7 +1457,6 @@ static int cxl_port_setup_targets(struct cxl_port *port,
 	struct cxl_region_params *p = &cxlr->params;
 	struct cxl_decoder *cxld = cxl_rr->decoder;
 	struct cxl_switch_decoder *cxlsd;
-	struct cxl_port *iter = port;
 	u16 eig, peig;
 	u8 eiw, peiw;
 
@@ -1464,26 +1472,26 @@ static int cxl_port_setup_targets(struct cxl_port *port,
 	}
 
 	cxlsd = to_cxl_switch_decoder(&cxld->dev);
+	child_iw = cxl_rr->nr_targets;
+
 	if (cxl_rr->nr_targets_set) {
 		int i, distance = 1;
-		struct cxl_region_ref *cxl_rr_iter;
 
 		/*
-		 * The "distance" between peer downstream ports represents which
-		 * endpoint positions in the region interleave a given port can
-		 * host.
-		 *
-		 * For example, at the root of a hierarchy the distance is
-		 * always 1 as every index targets a different host-bridge. At
-		 * each subsequent switch level those ports map every Nth region
-		 * position where N is the width of the switch == distance.
+		 * @distance is the spacing between region positions sharing this
+		 * dport. Mixed-granularity regions place them contiguously.
 		 */
-		do {
-			cxl_rr_iter = cxl_rr_load(iter, cxlr);
-			distance *= cxl_rr_iter->nr_targets;
-			iter = to_cxl_port(iter->dev.parent);
-		} while (!is_cxl_root(iter));
-		distance *= cxlrd->cxlsd.cxld.interleave_ways;
+		if (!cxl_region_is_mixed_gran(cxlr)) {
+			struct cxl_region_ref *cxl_rr_iter;
+			struct cxl_port *iter = port;
+
+			do {
+				cxl_rr_iter = cxl_rr_load(iter, cxlr);
+				distance *= cxl_rr_iter->nr_targets;
+				iter = to_cxl_port(iter->dev.parent);
+			} while (!is_cxl_root(iter));
+			distance *= cxlrd->cxlsd.cxld.interleave_ways;
+		}
 
 		for (i = 0; i < cxl_rr->nr_targets_set; i++)
 			if (ep->dport == cxlsd->target[i]) {
@@ -1497,15 +1505,15 @@ static int cxl_port_setup_targets(struct cxl_port *port,
 	}
 
 	if (is_cxl_root(parent_port)) {
-		/*
-		 * Root decoder IG is always set to value in CFMWS which
-		 * may be different than this region's IG.  We can use the
-		 * region's IG here since interleave_granularity_store()
-		 * does not allow interleaved host-bridges with
-		 * root IG != region IG.
-		 */
-		parent_ig = p->interleave_granularity;
 		parent_iw = cxlrd->cxlsd.cxld.interleave_ways;
+		/*
+		 * A non-interleaving root does not contribute to the region
+		 * interleave.
+		 */
+		if (parent_iw > 1)
+			parent_ig = cxlrd->cxlsd.cxld.interleave_granularity;
+		else
+			parent_ig = p->interleave_granularity;
 		/*
 		 * For purposes of address bit routing, use power-of-2 math for
 		 * switch ports.
@@ -1538,7 +1546,6 @@ static int cxl_port_setup_targets(struct cxl_port *port,
 		return rc;
 	}
 
-	child_iw = cxl_rr->nr_targets;
 	rc = ways_to_eiw(child_iw, &eiw);
 	if (rc) {
 		dev_dbg(&cxlr->dev, "%s:%s: invalid port interleave: %d\n",
@@ -1546,23 +1553,25 @@ static int cxl_port_setup_targets(struct cxl_port *port,
 		return rc;
 	}
 
-	/*
-	 * Interleave granularity is a multiple of @parent_port granularity.
-	 * Multiplier is the parent port interleave ways.
-	 */
-	rc = granularity_to_eig(parent_ig * parent_iw, &eig);
-	if (rc) {
-		dev_dbg(&cxlr->dev,
-			"%s: invalid granularity calculation (%d * %d)\n",
-			dev_name(&parent_port->dev), parent_ig, parent_iw);
-		return rc;
+	/* Only mixed-gran regions refine granularity toward the endpoints */
+	if (cxl_region_is_mixed_gran(cxlr)) {
+		child_ig = parent_ig / child_iw;
+
+		if (child_ig < p->interleave_granularity) {
+			dev_dbg(&cxlr->dev,
+				"%s:%s: granularity %d is finer than region granularity %d\n",
+				dev_name(port->uport_dev), dev_name(&port->dev),
+				child_ig, p->interleave_granularity);
+			return -ENXIO;
+		}
+	} else {
+		child_ig = parent_ig * parent_iw;
 	}
 
-	rc = eig_to_granularity(eig, &child_ig);
+	rc = granularity_to_eig(child_ig, &eig);
 	if (rc) {
-		dev_dbg(&cxlr->dev, "%s:%s: invalid interleave: %d\n",
-			dev_name(port->uport_dev), dev_name(&port->dev),
-			256 << eig);
+		dev_dbg(&cxlr->dev, "%s:%s: invalid granularity: %d\n",
+			dev_name(port->uport_dev), dev_name(&port->dev), child_ig);
 		return rc;
 	}
 
@@ -2061,6 +2070,43 @@ static int cxl_region_sort_targets(struct cxl_region *cxlr)
 	return rc;
 }
 
+static int cxl_region_validate_interleave(struct cxl_region *cxlr)
+{
+	struct cxl_decoder *cxld = &cxlr->cxlrd->cxlsd.cxld;
+	struct cxl_region_params *p = &cxlr->params;
+	int root_iw = cxld->interleave_ways;
+	int root_ig = cxld->interleave_granularity;
+
+	if (root_iw == 1)
+		return 0;
+
+	if (p->interleave_granularity > root_ig) {
+		dev_dbg(&cxlr->dev,
+			"granularity %d exceeds root decoder granularity %d\n",
+			p->interleave_granularity, root_ig);
+		return -ENXIO;
+	}
+
+	/*
+	 * Same-granularity regions below a power-of-two root may span multiple root
+	 * interleaves. Mod3 roots are width-matched instead.
+	 */
+	if (is_power_of_2(root_iw) && p->interleave_granularity == root_ig)
+		return 0;
+
+	/* Span is one complete pass through the interleave pattern. */
+	if (root_iw * root_ig != p->interleave_ways * p->interleave_granularity) {
+		dev_dbg(&cxlr->dev,
+			"region span %d (%d ways at %d) must equal root span %d (%d ways at %d)\n",
+			p->interleave_ways * p->interleave_granularity,
+			p->interleave_ways, p->interleave_granularity,
+			root_iw * root_ig, root_iw, root_ig);
+		return -ENXIO;
+	}
+
+	return 0;
+}
+
 static int cxl_region_attach(struct cxl_region *cxlr,
 			     struct cxl_endpoint_decoder *cxled, int pos)
 {
@@ -2103,6 +2149,10 @@ static int cxl_region_attach(struct cxl_region *cxlr,
 		return -ENXIO;
 	}
 
+	rc = cxl_region_validate_interleave(cxlr);
+	if (rc)
+		return rc;
+
 	if (p->nr_targets >= p->interleave_ways) {
 		dev_dbg(&cxlr->dev, "region already has %d endpoints\n",
 			p->nr_targets);
-- 
2.37.3

[PATCH v5 6/7] cxl/region: Support mixed-granularity user created regions

[Alison Schofield]

User region creation currently requires an interleaving root to have the
same granularity as the region.

That rejects mixed-granularity layouts, those where the region granularity
is finer than the granularity of the interleaving root decoder. A region
granularity coarser than the root remains invalid because it produces
incorrect DPA translations.

Allow a region granularity finer than an interleaving root and continue to
reject a coarser granularity. Validate the complete ways/granularity
relationship when the region endpoints are attached.

For example, CXL 4.0 Section 9.13.1.1 Table 9-7 describes a 6-way region at
IGB across three host bridges as a 3-way root interleave at 2 * IGB with a
2-way interleave at IGB below it.

Signed-off-by: Alison Schofield <alison.schofield@intel.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>
---
 drivers/cxl/core/region.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index 5b7104a55fdd..374d8c2c32d1 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -572,16 +572,13 @@ static int set_interleave_granularity(struct cxl_region *cxlr, int val)
 	if (rc)
 		return rc;
 
-	/*
-	 * When the host-bridge is interleaved, disallow region granularity !=
-	 * root granularity. Regions with a granularity less than the root
-	 * interleave result in needing multiple endpoints to support a single
-	 * slot in the interleave (possible to support in the future). Regions
-	 * with a granularity greater than the root interleave result in invalid
-	 * DPA translations (invalid to support).
-	 */
-	if (cxld->interleave_ways > 1 && val != cxld->interleave_granularity)
+	/* Region granularity must not be coarser than an interleaving root's */
+	if (cxld->interleave_ways > 1 && val > cxld->interleave_granularity) {
+		dev_dbg(&cxlr->dev,
+			"granularity %d exceeds root decoder granularity %d\n",
+			val, cxld->interleave_granularity);
 		return -EINVAL;
+	}
 
 	lockdep_assert_held_write(&cxl_rwsem.region);
 
-- 
2.37.3

[PATCH v5 7/7] cxl/test: Add a topology to test mixed-granularity regions

[Alison Schofield]

Add a cxl_test topology for mixed-granularity region testing.

modprobe cxl_test mixed_gran_regions=1 creates three host bridges, two
root ports per host bridge, one switch per root port, and two endpoints
per switch, for twelve endpoints total.

Two root ports per host bridge is what makes the interleaving levels
selectable. A region built from one endpoint per root port interleaves
at the host bridge and passes through the switch; a region built from
both endpoints below a single root port passes through the host bridge
and interleaves at the switch; a region using every endpoint
interleaves at both. The CXL Specification lists those as distinct
legal configurations, so the choice of member endpoints, rather than a
second topology, selects which one is under test.

Three CFMWS windows are added:

1) A 2-way window at 4K over two host bridges, hosting an 8-way region
at 1K and a 4-way region at 2K.

2) A 3-way window at 512 over three host bridges, hosting a 6-way
region at 256: CXL 4.0 Table 9-7 rows 2 and 3.

3) A 3-way window at 1K over three host bridges, hosting a 12-way
region at 256: CXL 4.0 Table 9-6 row 7. That row is the composition
the documentation contrasts with the unsupported row 6, and reaching it
needs a root granularity four times the region granularity.

Each window also hosts a same-granularity region at its own ways and
granularity. The 2-way window at 4K hosts 4-way and 8-way regions at 4K as
well; the span identity allows the 3-way windows only their own width.

Register every mock device through cxl_mock_platform_device_add() so
the mock array is populated before platform_device_add(), matching the
ordering the rest of cxl_test relies on to avoid the endpoint-probe
race in commit d90f236f8b9e ("cxl/test: Update mock dev array before
calling platform_device_add()").

NULL-check the legacy topology arrays in the mock device and mock port
lookups, which are left empty when mixed_gran_regions=1 selects the new
topology.

The default and existing special module params are unaffected.

Signed-off-by: Alison Schofield <alison.schofield@intel.com>
---
 tools/testing/cxl/test/cxl.c | 410 +++++++++++++++++++++++++++++++++--
 1 file changed, 390 insertions(+), 20 deletions(-)

diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c
index 62bd92b3be45..75e9a4a3d865 100644
--- a/tools/testing/cxl/test/cxl.c
+++ b/tools/testing/cxl/test/cxl.c
@@ -18,6 +18,7 @@ static int interleave_arithmetic;
 static bool extended_linear_cache;
 static bool fail_autoassemble;
 static bool type2_test;
+static bool mixed_gran_regions;
 
 #define FAKE_QTG_ID	42
 
@@ -27,9 +28,32 @@ static bool type2_test;
 #define NR_CXL_ROOT_PORTS 2
 #define NR_CXL_SWITCH_PORTS 2
 #define NR_CXL_PORT_DECODERS 8
-#define NR_BRIDGES (NR_CXL_HOST_BRIDGES + NR_CXL_SINGLE_HOST + NR_CXL_RCH)
 #define NR_CXL_TYPE2_ACCEL 1
 
+/*
+ * mixed_gran_regions=1: three host bridges, two root ports each, and one
+ * switch per root port, reaching twelve endpoints. Two root ports per host
+ * bridge let a region interleave at the host bridge, at the switch, or at
+ * both, so a single topology covers the power-of-2 and 3-way-family region
+ * layouts that the CXL Specification permits.
+ */
+#define NR_CXL_MIX_GRAN_HB 3
+#define NR_CXL_MIX_GRAN_RPS_PER_HB 2
+#define NR_CXL_MIX_GRAN_RPS (NR_CXL_MIX_GRAN_HB * NR_CXL_MIX_GRAN_RPS_PER_HB)
+#define NR_CXL_MIX_GRAN_SWITCHES NR_CXL_MIX_GRAN_RPS /* 1 per RP */
+#define NR_CXL_MIX_GRAN_DPORTS \
+	(NR_CXL_MIX_GRAN_SWITCHES * NR_CXL_SWITCH_PORTS)
+
+#define NR_DEFAULT_BRIDGES \
+	(NR_CXL_HOST_BRIDGES + NR_CXL_SINGLE_HOST + NR_CXL_RCH)
+#define NR_BRIDGES (NR_DEFAULT_BRIDGES + NR_CXL_MIX_GRAN_HB)
+
+/* CHBS index ranges within mock_cedt.chbs[] */
+#define CHBS_DEFAULT_START 0
+#define CHBS_DEFAULT_END (NR_DEFAULT_BRIDGES - 1)
+#define CHBS_MIX_GRAN_START NR_DEFAULT_BRIDGES
+#define CHBS_MIX_GRAN_END (NR_BRIDGES - 1)
+
 #define MOCK_AUTO_REGION_SIZE_DEFAULT SZ_512M
 static int mock_auto_region_size = MOCK_AUTO_REGION_SIZE_DEFAULT;
 
@@ -54,6 +78,12 @@ struct platform_device *cxl_mem_single[NR_MEM_SINGLE];
 static struct platform_device *cxl_rch[NR_CXL_RCH];
 static struct platform_device *cxl_rcd[NR_CXL_RCH];
 
+static struct platform_device *cxl_mix_gran_hb[NR_CXL_MIX_GRAN_HB];
+static struct platform_device *cxl_mix_gran_root_port[NR_CXL_MIX_GRAN_RPS];
+static struct platform_device *cxl_mix_gran_uport[NR_CXL_MIX_GRAN_SWITCHES];
+static struct platform_device *cxl_mix_gran_dport[NR_CXL_MIX_GRAN_DPORTS];
+static struct platform_device *cxl_mem_mix_gran[NR_CXL_MIX_GRAN_DPORTS];
+
 /*
  * Decoder registry
  *
@@ -117,6 +147,19 @@ static struct acpi_device host_bridge[NR_BRIDGES] = {
 		.handle = &host_bridge[3],
 		.pnp.unique_id = "3",
 	},
+	/* Dedicated mixed-granularity topology */
+	[4] = {
+		.handle = &host_bridge[4],
+		.pnp.unique_id = "4",
+	},
+	[5] = {
+		.handle = &host_bridge[5],
+		.pnp.unique_id = "5",
+	},
+	[6] = {
+		.handle = &host_bridge[6],
+		.pnp.unique_id = "6",
+	},
 };
 
 static bool is_mock_dev(struct device *dev)
@@ -124,15 +167,18 @@ static bool is_mock_dev(struct device *dev)
 	int i;
 
 	for (i = 0; i < ARRAY_SIZE(cxl_mem); i++)
-		if (dev == &cxl_mem[i]->dev)
+		if (cxl_mem[i] && dev == &cxl_mem[i]->dev)
 			return true;
 	for (i = 0; i < ARRAY_SIZE(cxl_mem_single); i++)
-		if (dev == &cxl_mem_single[i]->dev)
+		if (cxl_mem_single[i] && dev == &cxl_mem_single[i]->dev)
 			return true;
 	for (i = 0; i < ARRAY_SIZE(cxl_rcd); i++)
-		if (dev == &cxl_rcd[i]->dev)
+		if (cxl_rcd[i] && dev == &cxl_rcd[i]->dev)
 			return true;
-	if (dev == &cxl_acpi->dev)
+	for (i = 0; i < ARRAY_SIZE(cxl_mem_mix_gran); i++)
+		if (cxl_mem_mix_gran[i] && dev == &cxl_mem_mix_gran[i]->dev)
+			return true;
+	if (cxl_acpi && dev == &cxl_acpi->dev)
 		return true;
 	return false;
 }
@@ -190,6 +236,18 @@ static struct {
 		struct acpi_cedt_cfmws cfmws;
 		u32 target[3];
 	} cfmws8;
+	struct {
+		struct acpi_cedt_cfmws cfmws;
+		u32 target[2];
+	} cfmws9;
+	struct {
+		struct acpi_cedt_cfmws cfmws;
+		u32 target[3];
+	} cfmws10;
+	struct {
+		struct acpi_cedt_cfmws cfmws;
+		u32 target[3];
+	} cfmws11;
 	struct {
 		struct acpi_cedt_cxims cxims;
 		u64 xormap_list[2];
@@ -234,6 +292,31 @@ static struct {
 		.uid = 3,
 		.cxl_version = ACPI_CEDT_CHBS_VERSION_CXL11,
 	},
+	/* Dedicated mixed-granularity topology */
+	.chbs[4] = {
+		.header = {
+			.type = ACPI_CEDT_TYPE_CHBS,
+			.length = sizeof(mock_cedt.chbs[0]),
+		},
+		.uid = 4,
+		.cxl_version = ACPI_CEDT_CHBS_VERSION_CXL20,
+	},
+	.chbs[5] = {
+		.header = {
+			.type = ACPI_CEDT_TYPE_CHBS,
+			.length = sizeof(mock_cedt.chbs[0]),
+		},
+		.uid = 5,
+		.cxl_version = ACPI_CEDT_CHBS_VERSION_CXL20,
+	},
+	.chbs[6] = {
+		.header = {
+			.type = ACPI_CEDT_TYPE_CHBS,
+			.length = sizeof(mock_cedt.chbs[0]),
+		},
+		.uid = 6,
+		.cxl_version = ACPI_CEDT_CHBS_VERSION_CXL20,
+	},
 	.cfmws0 = {
 		.cfmws = {
 			.header = {
@@ -373,6 +456,51 @@ static struct {
 		},
 		.target = { 0, 1, 2, },
 	},
+	.cfmws9 = {
+		.cfmws = {
+			.header = {
+				.type = ACPI_CEDT_TYPE_CFMWS,
+				.length = sizeof(mock_cedt.cfmws9),
+			},
+			.interleave_ways = 1,
+			.granularity = 4,
+			.restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYMEM |
+					ACPI_CEDT_CFMWS_RESTRICT_PMEM,
+			.qtg_id = FAKE_QTG_ID,
+			.window_size = SZ_512M * 4UL,
+		},
+		.target = { 4, 5, },
+	},
+	.cfmws10 = {
+		.cfmws = {
+			.header = {
+				.type = ACPI_CEDT_TYPE_CFMWS,
+				.length = sizeof(mock_cedt.cfmws10),
+			},
+			.interleave_ways = 8,
+			.granularity = 1,
+			.restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYMEM |
+					ACPI_CEDT_CFMWS_RESTRICT_PMEM,
+			.qtg_id = FAKE_QTG_ID,
+			.window_size = SZ_512M * 6UL,
+		},
+		.target = { 4, 5, 6, },
+	},
+	.cfmws11 = {
+		.cfmws = {
+			.header = {
+				.type = ACPI_CEDT_TYPE_CFMWS,
+				.length = sizeof(mock_cedt.cfmws11),
+			},
+			.interleave_ways = 8,
+			.granularity = 2,
+			.restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYMEM |
+					ACPI_CEDT_CFMWS_RESTRICT_PMEM,
+			.qtg_id = FAKE_QTG_ID,
+			.window_size = SZ_512M * 6UL,
+		},
+		.target = { 4, 5, 6, },
+	},
 	.cxims0 = {
 		.cxims = {
 			.header = {
@@ -410,6 +538,10 @@ struct acpi_cedt_cfmws *mock_cfmws[] = {
 	[6] = &mock_cedt.cfmws6.cfmws,
 	[7] = &mock_cedt.cfmws7.cfmws,
 	[8] = &mock_cedt.cfmws8.cfmws,
+	/* Dedicated mixed-granularity topology */
+	[9] = &mock_cedt.cfmws9.cfmws,
+	[10] = &mock_cedt.cfmws10.cfmws,
+	[11] = &mock_cedt.cfmws11.cfmws,
 };
 
 static int cfmws_start;
@@ -418,6 +550,8 @@ static int cfmws_end;
 #define CFMWS_MOD_ARRAY_END   5
 #define CFMWS_XOR_ARRAY_START 6
 #define CFMWS_XOR_ARRAY_END   8
+#define CFMWS_MIX_GRAN_ARRAY_START 9
+#define CFMWS_MIX_GRAN_ARRAY_END   11
 
 struct acpi_cedt_cxims *mock_cxims[1] = {
 	[0] = &mock_cedt.cxims0.cxims,
@@ -496,12 +630,26 @@ static void update_type2_cfmws(void)
 	memcpy(&mock_cedt.cfmws0.cfmws, &type2_cfmws0, sizeof(type2_cfmws0));
 }
 
+/* CHBS entries belonging to the topology this load selected */
+static void chbs_range(int *lo, int *hi)
+{
+	if (mixed_gran_regions) {
+		*lo = CHBS_MIX_GRAN_START;
+		*hi = CHBS_MIX_GRAN_END;
+	} else {
+		*lo = CHBS_DEFAULT_START;
+		*hi = CHBS_DEFAULT_END;
+	}
+}
+
 static int populate_cedt(void)
 {
 	struct cxl_mock_res *res;
+	int chbs_lo, chbs_hi;
 	int i;
 
-	for (i = 0; i < ARRAY_SIZE(mock_cedt.chbs); i++) {
+	chbs_range(&chbs_lo, &chbs_hi);
+	for (i = chbs_lo; i <= chbs_hi; i++) {
 		struct acpi_cedt_chbs *chbs = &mock_cedt.chbs[i];
 		resource_size_t size;
 
@@ -562,12 +710,16 @@ static int mock_acpi_table_parse_cedt(enum acpi_cedt_type id,
 	if (!is_mock_port(dev) && !is_mock_dev(dev))
 		return acpi_table_parse_cedt(id, handler_arg, arg);
 
-	if (id == ACPI_CEDT_TYPE_CHBS)
-		for (i = 0; i < ARRAY_SIZE(mock_cedt.chbs); i++) {
+	if (id == ACPI_CEDT_TYPE_CHBS) {
+		int chbs_lo, chbs_hi;
+
+		chbs_range(&chbs_lo, &chbs_hi);
+		for (i = chbs_lo; i <= chbs_hi; i++) {
 			h = (union acpi_subtable_headers *)&mock_cedt.chbs[i];
 			end = (unsigned long)&mock_cedt.chbs[i + 1];
 			handler_arg(h, arg, end);
 		}
+	}
 
 	if (id == ACPI_CEDT_TYPE_CFMWS)
 		for (i = cfmws_start; i <= cfmws_end; i++) {
@@ -591,13 +743,16 @@ static bool is_mock_bridge(struct device *dev)
 	int i;
 
 	for (i = 0; i < ARRAY_SIZE(cxl_host_bridge); i++)
-		if (dev == &cxl_host_bridge[i]->dev)
+		if (cxl_host_bridge[i] && dev == &cxl_host_bridge[i]->dev)
 			return true;
 	for (i = 0; i < ARRAY_SIZE(cxl_hb_single); i++)
-		if (dev == &cxl_hb_single[i]->dev)
+		if (cxl_hb_single[i] && dev == &cxl_hb_single[i]->dev)
 			return true;
 	for (i = 0; i < ARRAY_SIZE(cxl_rch); i++)
-		if (dev == &cxl_rch[i]->dev)
+		if (cxl_rch[i] && dev == &cxl_rch[i]->dev)
+			return true;
+	for (i = 0; i < ARRAY_SIZE(cxl_mix_gran_hb); i++)
+		if (cxl_mix_gran_hb[i] && dev == &cxl_mix_gran_hb[i]->dev)
 			return true;
 
 	return false;
@@ -611,27 +766,40 @@ static bool is_mock_port(struct device *dev)
 		return true;
 
 	for (i = 0; i < ARRAY_SIZE(cxl_root_port); i++)
-		if (dev == &cxl_root_port[i]->dev)
+		if (cxl_root_port[i] && dev == &cxl_root_port[i]->dev)
 			return true;
 
 	for (i = 0; i < ARRAY_SIZE(cxl_switch_uport); i++)
-		if (dev == &cxl_switch_uport[i]->dev)
+		if (cxl_switch_uport[i] && dev == &cxl_switch_uport[i]->dev)
 			return true;
 
 	for (i = 0; i < ARRAY_SIZE(cxl_switch_dport); i++)
-		if (dev == &cxl_switch_dport[i]->dev)
+		if (cxl_switch_dport[i] && dev == &cxl_switch_dport[i]->dev)
 			return true;
 
 	for (i = 0; i < ARRAY_SIZE(cxl_root_single); i++)
-		if (dev == &cxl_root_single[i]->dev)
+		if (cxl_root_single[i] && dev == &cxl_root_single[i]->dev)
 			return true;
 
 	for (i = 0; i < ARRAY_SIZE(cxl_swu_single); i++)
-		if (dev == &cxl_swu_single[i]->dev)
+		if (cxl_swu_single[i] && dev == &cxl_swu_single[i]->dev)
 			return true;
 
 	for (i = 0; i < ARRAY_SIZE(cxl_swd_single); i++)
-		if (dev == &cxl_swd_single[i]->dev)
+		if (cxl_swd_single[i] && dev == &cxl_swd_single[i]->dev)
+			return true;
+
+	for (i = 0; i < ARRAY_SIZE(cxl_mix_gran_root_port); i++)
+		if (cxl_mix_gran_root_port[i] &&
+		    dev == &cxl_mix_gran_root_port[i]->dev)
+			return true;
+
+	for (i = 0; i < ARRAY_SIZE(cxl_mix_gran_uport); i++)
+		if (cxl_mix_gran_uport[i] && dev == &cxl_mix_gran_uport[i]->dev)
+			return true;
+
+	for (i = 0; i < ARRAY_SIZE(cxl_mix_gran_dport); i++)
+		if (cxl_mix_gran_dport[i] && dev == &cxl_mix_gran_dport[i]->dev)
 			return true;
 
 	if (is_cxl_memdev(dev))
@@ -702,7 +870,15 @@ static struct acpi_pci_root mock_pci_root[ARRAY_SIZE(mock_pci_bus)] = {
 	[3] = {
 		.bus = &mock_pci_bus[3],
 	},
-
+	[4] = {
+		.bus = &mock_pci_bus[4],
+	},
+	[5] = {
+		.bus = &mock_pci_bus[5],
+	},
+	[6] = {
+		.bus = &mock_pci_bus[6],
+	},
 };
 
 static bool is_mock_bus(struct pci_bus *bus)
@@ -1343,7 +1519,8 @@ static bool mock_init_hdm_decoder(struct cxl_decoder *cxld)
 		/* check is endpoint is attach to host-bridge0 */
 		port = cxled_to_port(cxled);
 		do {
-			if (port->uport_dev == &cxl_host_bridge[0]->dev) {
+			if (cxl_host_bridge[0] &&
+			    port->uport_dev == &cxl_host_bridge[0]->dev) {
 				hb0 = true;
 				break;
 			}
@@ -1391,6 +1568,8 @@ static int mock_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm,
 
 	if (is_cxl_endpoint(port))
 		target_count = 0;
+	else if (mixed_gran_regions && is_cxl_root(parent_port))
+		target_count = NR_CXL_MIX_GRAN_RPS_PER_HB;
 	else if (is_cxl_root(parent_port))
 		target_count = NR_CXL_ROOT_PORTS;
 	else
@@ -1490,6 +1669,24 @@ static int get_port_array(struct cxl_port *port,
 	struct platform_device **array;
 	int array_size;
 
+	if (mixed_gran_regions) {
+		if (port->depth == 1) {
+			array_size = ARRAY_SIZE(cxl_mix_gran_root_port);
+			array = cxl_mix_gran_root_port;
+		} else if (port->depth == 2) {
+			array_size = ARRAY_SIZE(cxl_mix_gran_dport);
+			array = cxl_mix_gran_dport;
+		} else {
+			dev_WARN_ONCE(&port->dev, 1, "unexpected depth %d\n",
+				      port->depth);
+			return -ENXIO;
+		}
+
+		*port_array = array;
+		*port_array_size = array_size;
+		return 0;
+	}
+
 	if (port->depth == 1) {
 		if (is_multi_bridge(port->uport_dev)) {
 			array_size = ARRAY_SIZE(cxl_root_port);
@@ -1883,11 +2080,137 @@ static void cxl_single_topo_exit(void)
 	}
 }
 
+#define MIX_GRAN_RP_ID_BASE	(NR_MULTI_ROOT + NR_CXL_SINGLE_HOST)
+#define MIX_GRAN_USP_ID_BASE	MIX_GRAN_RP_ID_BASE
+#define MIX_GRAN_DSP_ID_BASE	(NR_MEM_MULTI + NR_MEM_SINGLE)
+#define MIX_GRAN_MEM_ID_BASE	(NR_MEM_MULTI + NR_MEM_SINGLE + NR_CXL_RCH)
+
+static void cxl_mix_gran_topo_exit(void)
+{
+	int i;
+
+	for (i = NR_CXL_MIX_GRAN_DPORTS - 1; i >= 0; i--) {
+		platform_device_unregister(cxl_mix_gran_dport[i]);
+		cxl_mix_gran_dport[i] = NULL;
+	}
+	for (i = NR_CXL_MIX_GRAN_SWITCHES - 1; i >= 0; i--) {
+		platform_device_unregister(cxl_mix_gran_uport[i]);
+		cxl_mix_gran_uport[i] = NULL;
+	}
+	for (i = NR_CXL_MIX_GRAN_RPS - 1; i >= 0; i--) {
+		platform_device_unregister(cxl_mix_gran_root_port[i]);
+		cxl_mix_gran_root_port[i] = NULL;
+	}
+	for (i = NR_CXL_MIX_GRAN_HB - 1; i >= 0; i--) {
+		struct platform_device *pdev = cxl_mix_gran_hb[i];
+
+		if (!pdev)
+			continue;
+		sysfs_remove_link(&pdev->dev.kobj, "physical_node");
+		platform_device_unregister(pdev);
+		cxl_mix_gran_hb[i] = NULL;
+	}
+}
+
+static __init int cxl_mix_gran_topo_init(void)
+{
+	int i, rc;
+
+	for (i = 0; i < NR_CXL_MIX_GRAN_HB; i++) {
+		struct acpi_device *adev = &host_bridge[NR_DEFAULT_BRIDGES + i];
+		struct platform_device *pdev;
+
+		pdev = platform_device_alloc("cxl_host_bridge",
+					     NR_DEFAULT_BRIDGES + i);
+		if (!pdev) {
+			rc = -ENOMEM;
+			goto err;
+		}
+
+		mock_companion(adev, &pdev->dev);
+		rc = cxl_mock_platform_device_add(pdev, &cxl_mix_gran_hb[i]);
+		if (rc)
+			goto err;
+
+		mock_pci_bus[NR_DEFAULT_BRIDGES + i].bridge = &pdev->dev;
+		rc = sysfs_create_link(&pdev->dev.kobj, &pdev->dev.kobj,
+				       "physical_node");
+		if (rc)
+			goto err;
+	}
+
+	/* 2 root ports per host bridge */
+	for (i = 0; i < NR_CXL_MIX_GRAN_RPS; i++) {
+		struct platform_device *bridge =
+			cxl_mix_gran_hb[i / NR_CXL_MIX_GRAN_RPS_PER_HB];
+		struct platform_device *pdev;
+
+		pdev = platform_device_alloc("cxl_root_port",
+					     MIX_GRAN_RP_ID_BASE + i);
+		if (!pdev) {
+			rc = -ENOMEM;
+			goto err;
+		}
+		pdev->dev.parent = &bridge->dev;
+
+		rc = cxl_mock_platform_device_add(pdev, &cxl_mix_gran_root_port[i]);
+		if (rc)
+			goto err;
+	}
+
+	/* 1 switch uport per root port */
+	for (i = 0; i < NR_CXL_MIX_GRAN_SWITCHES; i++) {
+		struct platform_device *pdev;
+
+		pdev = platform_device_alloc("cxl_switch_uport",
+					     MIX_GRAN_USP_ID_BASE + i);
+		if (!pdev) {
+			rc = -ENOMEM;
+			goto err;
+		}
+		pdev->dev.parent = &cxl_mix_gran_root_port[i]->dev;
+
+		rc = cxl_mock_platform_device_add(pdev, &cxl_mix_gran_uport[i]);
+		if (rc)
+			goto err;
+	}
+
+	/* 2 dports per switch */
+	for (i = 0; i < NR_CXL_MIX_GRAN_DPORTS; i++) {
+		struct platform_device *uport =
+			cxl_mix_gran_uport[i / NR_CXL_SWITCH_PORTS];
+		struct platform_device *pdev;
+
+		pdev = platform_device_alloc("cxl_switch_dport",
+					     MIX_GRAN_DSP_ID_BASE + i);
+		if (!pdev) {
+			rc = -ENOMEM;
+			goto err;
+		}
+		pdev->dev.parent = &uport->dev;
+
+		rc = cxl_mock_platform_device_add(pdev, &cxl_mix_gran_dport[i]);
+		if (rc)
+			goto err;
+	}
+
+	return 0;
+err:
+	cxl_mix_gran_topo_exit();
+	return rc;
+}
+
 static void cxl_type3_mem_exit(void)
 {
 	struct platform_device *pdev;
 	int i;
 
+	if (mixed_gran_regions) {
+		for (i = NR_CXL_MIX_GRAN_DPORTS - 1; i >= 0; i--)
+			platform_device_unregister(cxl_mem_mix_gran[i]);
+		return;
+	}
+
 	for (i = ARRAY_SIZE(cxl_rcd) - 1; i >= 0; i--) {
 		pdev = cxl_rcd[i];
 		if (!pdev)
@@ -1960,10 +2283,42 @@ static int cxl_type2_mem_init(void)
 	return rc;
 }
 
+static int cxl_mem_mix_gran_init(void)
+{
+	int i, rc;
+
+	for (i = 0; i < NR_CXL_MIX_GRAN_DPORTS; i++) {
+		struct platform_device *dport = cxl_mix_gran_dport[i];
+		struct platform_device *pdev;
+
+		pdev = platform_device_alloc("cxl_mem",
+					     MIX_GRAN_MEM_ID_BASE + i);
+		if (!pdev) {
+			rc = -ENOMEM;
+			goto err;
+		}
+		pdev->dev.parent = &dport->dev;
+		set_dev_node(&pdev->dev, i % 2);
+
+		rc = cxl_mock_platform_device_add(pdev, &cxl_mem_mix_gran[i]);
+		if (rc)
+			goto err;
+	}
+
+	return 0;
+err:
+	for (i = NR_CXL_MIX_GRAN_DPORTS - 1; i >= 0; i--)
+		platform_device_unregister(cxl_mem_mix_gran[i]);
+	return rc;
+}
+
 static int cxl_type3_mem_init(void)
 {
 	int i, rc;
 
+	if (mixed_gran_regions)
+		return cxl_mem_mix_gran_init();
+
 	for (i = 0; i < ARRAY_SIZE(cxl_mem); i++) {
 		struct platform_device *dport = cxl_switch_dport[i];
 		struct platform_device *pdev;
@@ -2076,6 +2431,8 @@ static bool __init have_multiple_modparms(void)
 		count++;
 	if (type2_test)
 		count++;
+	if (mixed_gran_regions)
+		count++;
 
 	return count > 1;
 }
@@ -2345,6 +2702,11 @@ static int cxl_type3_topo_init(void)
 
 static void cxl_topo_exit(void)
 {
+	if (mixed_gran_regions) {
+		cxl_mix_gran_topo_exit();
+		return;
+	}
+
 	if (type2_test) {
 		cxl_type2_topo_exit();
 		return;
@@ -2355,6 +2717,8 @@ static void cxl_topo_exit(void)
 
 static int cxl_topo_init(void)
 {
+	if (mixed_gran_regions)
+		return cxl_mix_gran_topo_init();
 	if (type2_test)
 		return cxl_type2_topo_init();
 	return cxl_type3_topo_init();
@@ -2397,7 +2761,10 @@ static __init int cxl_test_init(void)
 	if (rc)
 		goto err_gen_pool_add;
 
-	if (interleave_arithmetic == 1) {
+	if (mixed_gran_regions) {
+		cfmws_start = CFMWS_MIX_GRAN_ARRAY_START;
+		cfmws_end = CFMWS_MIX_GRAN_ARRAY_END;
+	} else if (interleave_arithmetic == 1) {
 		cfmws_start = CFMWS_XOR_ARRAY_START;
 		cfmws_end = CFMWS_XOR_ARRAY_END;
 	} else {
@@ -2484,6 +2851,9 @@ module_param(fail_autoassemble, bool, 0444);
 MODULE_PARM_DESC(fail_autoassemble, "Simulate missing member of an auto-region");
 module_param(type2_test, bool, 0444);
 MODULE_PARM_DESC(type2_test, "Enable type 2 support testing");
+module_param(mixed_gran_regions, bool, 0444);
+MODULE_PARM_DESC(mixed_gran_regions,
+		 "Topology supporting mixed granularity regions");
 module_init(cxl_test_init);
 module_exit(cxl_test_exit);
 MODULE_LICENSE("GPL v2");
-- 
2.37.3

Re: [PATCH v5 1/7] Documentation/cxl: Describe mixed-granularity regions

[Jonathan Cameron]

On Thu,  3 Sep 2026 16:23:43 -0700
Alison Schofield <alison.schofield@intel.com> wrote:

> Linux has required that a region's interleave granularity equal the
> interleave granularity of its interleaving root decoder. A mixed-
> granularity region lifts that restriction: the region granularity may be
> finer than the root's, so the interleave refines from the root toward the
> endpoints.
> 
> Mixed-granularity region support introduces interleave relationships that
> are not obvious from the existing CXL documentation. This is particularly
> true for the 3-, 6-, and 12-way configurations and for understanding which
> configurations permitted by the CXL Specification are supported by Linux.
> 
> Document the mixed-granularity model and Linux's coarse-to-fine
> restriction. Include the relevant configurations from CXL 4.0 Section
> 9.13.1.1 and annotate their Linux support status.
> 
> This intentionally repeats information from the CXL Specification. The
> specification remains authoritative, but showing the Linux support policy
> alongside the legal configurations ensures readers do not mistake an
> unsupported Linux setup for a configuration unsupported by the CXL
> Specification.
> 
> Assisted-by: Claude:Opus-5
> Signed-off-by: Alison Schofield <alison.schofield@intel.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>

Re: [PATCH v5 3/7] cxl/region: Generalize endpoint position mapping

[Jonathan Cameron]

On Thu,  3 Sep 2026 16:23:45 -0700
Alison Schofield <alison.schofield@intel.com> wrote:

> Endpoint position calculation currently relies on the requirement that an
> interleaving root have the same granularity as the region. Auto region
> creation builds the position by multiplying by the parent decoders' ways,
> while user region creation selects the root target with 'pos % ways'. Those
> calculations are sufficient under the current granularity restriction.
> 
> In order to support mixed-granularity regions, that granularity restriction
> will need to be removed so decoder granularity can change between levels of
> the interleave hierarchy. The position calculation needs to account for
> those changes to produce the correct endpoint ordering.
> 
> Change the position calculation so each decoder's contribution is weighted
> by its granularity relative to the region granularity:
> 
>     position += target_pos *
>                 (decoder_granularity / region_granularity)

In the code target_pos becomes parent_pos. The parent_pos naming
seems more logical to me but maybe I'm missing something! (more
that likely given it is interleave maths!)

> 
> Use the same relationship to select the root target during user region
> creation.
> 
> Weight each level by the region granularity, passed in by the caller,
> rather than by the granularity programmed in the endpoint decoder. The two
> are equal for most configurations, but not when Normalized Addressing
> leaves the endpoint decoder programmed passthrough while the region
> interleaves.
> 
> For currently supported regions, the new weighted calculation reduces to
> the existing position calculation and produces identical endpoint
> positions. The exception is a region wider than a same-granularity Mod3
> root, where address bit routing programs the level below the root at the
> region granularity, so the ratio derives a weight of one rather than three.
> A later patch in this series rejects that layout.
> 
> Signed-off-by: Alison Schofield <alison.schofield@intel.com>
I'll assume you'll clarify the naming thing.  Otherwise this LGTM
Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>

> ---
>  drivers/cxl/core/region.c | 69 ++++++++++++++++++++++++---------------
>  1 file changed, 42 insertions(+), 27 deletions(-)
> 
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index 6a698f36aa6d..506b1cba1a92 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c

> @@ -1951,24 +1959,29 @@ static int find_pos_and_ways(struct cxl_port *port, struct range *range,
>   * cxl_calc_interleave_pos() - calculate an endpoint position in a region
>   * @cxled: endpoint decoder member of given region
>   * @hpa_range: translated HPA range of the endpoint
> + * @region_gran: interleave granularity of the region
>   *
> - * The endpoint position is calculated by traversing the topology from
> - * the endpoint to the root decoder and iteratively applying this
> - * calculation:
> + * A region position is the index of a region-granularity chunk within one full
> + * pass of the region interleave. The position is calculated by traversing the
> + * topology from the endpoint to the root decoder and accumulating the
> + * contribution of each decoder level:
>   *
> - *    position = position * parent_ways + parent_pos;
> + *    position += parent_pos * (parent_granularity / region_gran);

Here is the naming difference from the patch description.

>   *
> - * ...where @position is inferred from switch and root decoder target lists.
> + * ...where @parent_pos is inferred from switch and root decoder target lists.
> + * The multiplier is the weight of that level: how many region positions pass
> + * between successive advances of the level's target index. A level that selects
> + * a single target contributes nothing.
>   *
>   * Return: position >= 0 on success
>   *	   -ENXIO on failure
>   */
>  static int cxl_calc_interleave_pos(struct cxl_endpoint_decoder *cxled,
> -				   struct range *hpa_range)
> +				   struct range *hpa_range, int region_gran)
>  {
>  	struct cxl_port *iter, *port = cxled_to_port(cxled);
>  	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
> -	int parent_ways = 0, parent_pos = 0, pos = 0;
> +	int parent_gran = 0, parent_pos = 0, pos = 0;
>  	int rc;
>  

Re: [PATCH v5 4/7] cxo_ol/region: Name the interleave locals in cxl_port_setup_targets()

[Jonathan Cameron]

On Thu,  3 Sep 2026 16:23:46 -0700
Alison Schofield <alison.schofield@intel.com> wrote:

> cxl_port_setup_targets() derives the interleave granularity of one decoder
> from the granularity and ways of the decoder above it, and takes that
> decoder's own ways from the topology below it. The values from the level
> above are named parent_iw and parent_ig, but the values being derived are
> the bare iw and ig, so the reader has to work out which level each one
> describes.
> 
> Rename iw and ig to child_iw and child_ig to match the parent_ prefix used
> for the level above.
> 
> No functional change.
> 
> Suggested-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>
> Signed-off-by: Alison Schofield <alison.schofield@intel.com>

I was wondering a bit about having ceig and ceiw but those would be
very local so I guess not worth it.

Given how often we use ways_to_eiw() not to make the conversion
but just to check if it is possible, I wonder if a ways_valid()
helper is useful?

static inline int ways_valid(unsigned int ways)
{
	u8 eiw;

	return ways_to_eiw(ways, &eiw);
}

Maybe similar for granularity.  Would remove some confusion over whether
we actually want the encoded values or not, making this function more
readable. I didn't look for whether it was helpful elsewhere.

Anyhow this change is good

Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>

Re: [PATCH v5 5/7] cxl/region: Support mixed-granularity auto regions

[Jonathan Cameron]

On Thu,  3 Sep 2026 16:23:47 -0700
Alison Schofield <alison.schofield@intel.com> wrote:

> The CXL Specification permits a region's interleave granularity to differ
> between levels of the decoder hierarchy. For an auto region, the CXL driver
> reconstructs that hierarchy from the decoder configuration programmed by
> platform firmware.
> 
> The CXL driver currently requires the interleaving root to have the same
> granularity as the region, with each successive interleaving level becoming
> coarser toward the endpoints. A mixed-granularity region lifts that
> requirement: its interleave granularity is finer than the granularity of
> its interleaving root decoder. The driver cannot assemble such a region
> today.
> 
> Support mixed-granularity auto regions in the CXL driver, restricted to
> coarse-to-fine layouts. Derive each interleaving decoder's granularity from
> its parent:
> 
>         child_ig = parent_ig / child_iw
> 
> A same-granularity region keeps the existing derivation, which coarsens
> toward the endpoints:
> 
>         child_ig = parent_ig * parent_iw
> 
> The topology determines each decoder's interleave ways, so the parent
> granularity and child ways determine the child granularity. Use that
> relationship to validate the decoder geometry while assembling an auto
> region. Only the interleaving levels are compared; a passthrough level,
> child_iw == 1, keeps its granularity exemption.
> 
> Refuse a derived granularity finer than the region granularity. Dividing at
> every level lands the deepest interleaving level exactly on the region
> granularity, so a finer value means the topology below one port is deeper
> than the region interleave has positions for, which happens only when
> another branch is too shallow to cover its share. That is what rejects an
> unbalanced tree below the interleaving root.
> 
> Require the root and region to interleave over the same span, the address
> range in which the interleave pattern completes one full pass:
> 
>         root_iw * root_ig == region_iw * region_ig
> 
> The same span relationship covers the CXL Specification's Mod3
> configurations. For example, a 6-way region at IGB across three host
> bridges uses a 3-way root interleave at 2 * IGB and a 2-way interleave at
> IGB below it, as described in CXL 4.0 Section 9.13.1.1.
> 
> Signed-off-by: Alison Schofield <alison.schofield@intel.com>

A couple of comments inline, but I'm fine with it as you have it here
if you prefer to not take those suggestions up.
Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>

> ---
>  drivers/cxl/core/region.c | 128 ++++++++++++++++++++++++++------------
>  1 file changed, 89 insertions(+), 39 deletions(-)
> 
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index 116f84bb0745..5b7104a55fdd 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c


>  static int cxl_port_setup_targets(struct cxl_port *port,
>  				  struct cxl_region *cxlr,
>  				  struct cxl_endpoint_decoder *cxled)
> @@ -1448,7 +1457,6 @@ static int cxl_port_setup_targets(struct cxl_port *port,
>  	struct cxl_region_params *p = &cxlr->params;
>  	struct cxl_decoder *cxld = cxl_rr->decoder;
>  	struct cxl_switch_decoder *cxlsd;
> -	struct cxl_port *iter = port;
>  	u16 eig, peig;
>  	u8 eiw, peiw;
>  
> @@ -1464,26 +1472,26 @@ static int cxl_port_setup_targets(struct cxl_port *port,
>  	}
>  
>  	cxlsd = to_cxl_switch_decoder(&cxld->dev);
> +	child_iw = cxl_rr->nr_targets;
> +
>  	if (cxl_rr->nr_targets_set) {
>  		int i, distance = 1;
> -		struct cxl_region_ref *cxl_rr_iter;
>  
>  		/*
> -		 * The "distance" between peer downstream ports represents which
> -		 * endpoint positions in the region interleave a given port can
> -		 * host.
> -		 *
> -		 * For example, at the root of a hierarchy the distance is
> -		 * always 1 as every index targets a different host-bridge. At
> -		 * each subsequent switch level those ports map every Nth region
> -		 * position where N is the width of the switch == distance.
> +		 * @distance is the spacing between region positions sharing this
> +		 * dport. Mixed-granularity regions place them contiguously.

Maybe throw in "Supported" given there are things that match your def
of mixed-granularity that we don't support!

>  		 */

...

...

> @@ -2061,6 +2070,43 @@ static int cxl_region_sort_targets(struct cxl_region *cxlr)
>  	return rc;
>  }
>  
> +static int cxl_region_validate_interleave(struct cxl_region *cxlr)
> +{
> +	struct cxl_decoder *cxld = &cxlr->cxlrd->cxlsd.cxld;
> +	struct cxl_region_params *p = &cxlr->params;

I know this is consistent with elsewhere, but to me would be nice
if this were region_p or something like that so we'd immediately know
at point of use that this was region stuff.

> +	int root_iw = cxld->interleave_ways;
> +	int root_ig = cxld->interleave_granularity;
> +
> +	if (root_iw == 1)
> +		return 0;
> +
> +	if (p->interleave_granularity > root_ig) {
> +		dev_dbg(&cxlr->dev,
> +			"granularity %d exceeds root decoder granularity %d\n",
> +			p->interleave_granularity, root_ig);

In the patch description I think you describe this the other way around and
to me that makes more sense.  No where in the topology can the granularity be
less than that of the region.
	if (root_ig < p->interleave_granularity) {
		dev_dbg(&cxlr->dev,
			"Granularity %d, of root decoder is less than granularity %d of region\n"),

Obviously this is just my mental model being reverse of what is here and
they are mathematically the same, so feel free to ignore this one if you
think of it the other way around.

> +		return -ENXIO;

> +	}

Re: [PATCH v5 6/7] cxl/region: Support mixed-granularity user created regions

[Jonathan Cameron]

On Thu,  3 Sep 2026 16:23:48 -0700
Alison Schofield <alison.schofield@intel.com> wrote:

> User region creation currently requires an interleaving root to have the
> same granularity as the region.
> 
> That rejects mixed-granularity layouts, those where the region granularity
> is finer than the granularity of the interleaving root decoder. A region
> granularity coarser than the root remains invalid because it produces
> incorrect DPA translations.
> 
> Allow a region granularity finer than an interleaving root and continue to
> reject a coarser granularity. Validate the complete ways/granularity
> relationship when the region endpoints are attached.
> 
> For example, CXL 4.0 Section 9.13.1.1 Table 9-7 describes a 6-way region at
> IGB across three host bridges as a 3-way root interleave at 2 * IGB with a
> 2-way interleave at IGB below it.
> 
> Signed-off-by: Alison Schofield <alison.schofield@intel.com>
> Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>

Follow on for earlier comment. Not important though.

> ---
>  drivers/cxl/core/region.c | 15 ++++++---------
>  1 file changed, 6 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index 5b7104a55fdd..374d8c2c32d1 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -572,16 +572,13 @@ static int set_interleave_granularity(struct cxl_region *cxlr, int val)
>  	if (rc)
>  		return rc;
>  
> -	/*
> -	 * When the host-bridge is interleaved, disallow region granularity !=
> -	 * root granularity. Regions with a granularity less than the root
> -	 * interleave result in needing multiple endpoints to support a single
> -	 * slot in the interleave (possible to support in the future). Regions
> -	 * with a granularity greater than the root interleave result in invalid
> -	 * DPA translations (invalid to support).
> -	 */
> -	if (cxld->interleave_ways > 1 && val != cxld->interleave_granularity)
> +	/* Region granularity must not be coarser than an interleaving root's */

This is same as I raised earlier. To me the description of it the other way
around as any given decoder can't have granularity finer than the root is easier
to understand but meh, up to you.

> +	if (cxld->interleave_ways > 1 && val > cxld->interleave_granularity) {
> +		dev_dbg(&cxlr->dev,
> +			"granularity %d exceeds root decoder granularity %d\n",
> +			val, cxld->interleave_granularity);
>  		return -EINVAL;
> +	}
>  
>  	lockdep_assert_held_write(&cxl_rwsem.region);
>  

Re: [PATCH v5 7/7] cxl/test: Add a topology to test mixed-granularity regions

[Jonathan Cameron]

On Thu,  3 Sep 2026 16:23:49 -0700
Alison Schofield <alison.schofield@intel.com> wrote:

> Add a cxl_test topology for mixed-granularity region testing.
> 
> modprobe cxl_test mixed_gran_regions=1 creates three host bridges, two
> root ports per host bridge, one switch per root port, and two endpoints
> per switch, for twelve endpoints total.
> 
> Two root ports per host bridge is what makes the interleaving levels
> selectable. A region built from one endpoint per root port interleaves
> at the host bridge and passes through the switch; a region built from
> both endpoints below a single root port passes through the host bridge
> and interleaves at the switch; a region using every endpoint
> interleaves at both. The CXL Specification lists those as distinct
> legal configurations, so the choice of member endpoints, rather than a
> second topology, selects which one is under test.
> 
> Three CFMWS windows are added:
> 
> 1) A 2-way window at 4K over two host bridges, hosting an 8-way region
> at 1K and a 4-way region at 2K.
> 
> 2) A 3-way window at 512 over three host bridges, hosting a 6-way
> region at 256: CXL 4.0 Table 9-7 rows 2 and 3.
> 
> 3) A 3-way window at 1K over three host bridges, hosting a 12-way
> region at 256: CXL 4.0 Table 9-6 row 7. That row is the composition
> the documentation contrasts with the unsupported row 6, and reaching it
> needs a root granularity four times the region granularity.
> 
> Each window also hosts a same-granularity region at its own ways and
> granularity. The 2-way window at 4K hosts 4-way and 8-way regions at 4K as
> well; the span identity allows the 3-way windows only their own width.
> 
> Register every mock device through cxl_mock_platform_device_add() so
> the mock array is populated before platform_device_add(), matching the
> ordering the rest of cxl_test relies on to avoid the endpoint-probe
> race in commit d90f236f8b9e ("cxl/test: Update mock dev array before
> calling platform_device_add()").
> 
> NULL-check the legacy topology arrays in the mock device and mock port
> lookups, which are left empty when mixed_gran_regions=1 selects the new
> topology.
> 
> The default and existing special module params are unaffected.
> 
> Signed-off-by: Alison Schofield <alison.schofield@intel.com>
Hi Alison,

Sorry, no tag for this, but just because I can't remember enough about
how the tests work (if I ever really understood!) and don't want to
spend the rest of today working it out again :(  
FWIW, I didn't spot anything wrong from a quick look but I'm not feeling
confident to review this.

Jonathan

Re: [PATCH v5 1/7] Documentation/cxl: Describe mixed-granularity regions

[Robert Richter]

On 03.09.26 16:23:43, Alison Schofield wrote:
> Linux has required that a region's interleave granularity equal the
> interleave granularity of its interleaving root decoder. A mixed-
> granularity region lifts that restriction: the region granularity may be
> finer than the root's, so the interleave refines from the root toward the
> endpoints.
> 
> Mixed-granularity region support introduces interleave relationships that
> are not obvious from the existing CXL documentation. This is particularly
> true for the 3-, 6-, and 12-way configurations and for understanding which
> configurations permitted by the CXL Specification are supported by Linux.
> 
> Document the mixed-granularity model and Linux's coarse-to-fine
> restriction. Include the relevant configurations from CXL 4.0 Section
> 9.13.1.1 and annotate their Linux support status.
> 
> This intentionally repeats information from the CXL Specification. The
> specification remains authoritative, but showing the Linux support policy
> alongside the legal configurations ensures readers do not mistake an
> unsupported Linux setup for a configuration unsupported by the CXL
> Specification.

This doc is too long. It should only contain the following:

1) Describe a brief description of how the specification defines
interleaving along with references to it. Only add what is needed to
describe the cxl driver specifics in 2).

2) Limitations and implementation specifics of the cxl driver compared
to the specification.

This document is ambiguous on what the specs defines and that the
driver provides. This is blurred in this doc.

> 
> Assisted-by: Claude:Opus-5

I am fine with that, but we shouldn't use it to create tons of docs
which we will need another AI assistant again to read. Though, I read
it on my own. :-)

> Signed-off-by: Alison Schofield <alison.schofield@intel.com>
> ---
>  .../driver-api/cxl/linux/cxl-driver.rst       | 153 ++++++++++++++++++
>  1 file changed, 153 insertions(+)
> 
> diff --git a/Documentation/driver-api/cxl/linux/cxl-driver.rst b/Documentation/driver-api/cxl/linux/cxl-driver.rst
> index dd6dd17dc536..a61a20c8bb6d 100644
> --- a/Documentation/driver-api/cxl/linux/cxl-driver.rst
> +++ b/Documentation/driver-api/cxl/linux/cxl-driver.rst
> @@ -602,6 +602,11 @@ derived from their upstream port connections. In `Cross-Link First` interleave
>  configurations, the :code:`interleave_granularity` of a decoder is equal to
>  :code:`parent_interleave_granularity * parent_interleave_ways`.
>  
> +When the region granularity is finer than the granularity of an interleaving
> +root decoder, the relation inverts: the :code:`interleave_granularity` of a
> +decoder is equal to :code:`parent_interleave_granularity / interleave_ways`.
> +See `Mixed Granularity`_.

This introduces another unnecessary restriction: granularity must
decrease from root down to the EP. E.g. The following config would not
be supported:

a)

 Level             Ways    Granularity
 -----             ----    -----------
 Root                 2         4096
 Host bridge          2         1024
 Switch               2         2048
 Endpoint             8         1024

Or:

b)

 Level             Ways    Granularity
 -----             ----    -----------
 Root                 3         4096
 Host bridge          2         1024
 Switch               2         2048
 Endpoint             12        1024

In selector bits this is for the variants a/b:

root:   bits 12/-
hb:     bits 10
switch: bits 11

So the actual requirement is that the combined mask is consecutive.
The lowest bit marks the EP granularity. The bit weight marks the
2-factor of the EP (total) ways.

This series should implement all cases as at some point the issue will
pop up again.

> +
>  At Endpoint
>  ~~~~~~~~~~~
>  `Endpoint Decoders` are programmed similar to Host Bridge and Switch decoders,
> @@ -619,6 +624,154 @@ from HPA to DPA.  This is why they must be aware of the entire interleave set.
>  Linux does not support unbalanced interleave configurations.  As a result, all
>  endpoints in an interleave set must have the same ways and granularity.
>  
> +Mixed Granularity
> +~~~~~~~~~~~~~~~~~
> +Linux has required that a region's :code:`interleave_granularity` equal the
> +:code:`interleave_granularity` of its interleaving root decoder.  A

This statement comments on the current kernel implementation which
this series aims to change. It will be misleading to document that
here. I rather would not document an implementation state that will be
changed.

> +*mixed-granularity* region lifts that restriction: the region granularity may

This term is not a spec definition. It just describes a special case
that may happen. I would avoid introducing it here. If the granularity
implementation ware generic, there would be no need to describe that
special case. So better describe the issue of current implementation
and how it could be fixed.

> +be finer than the root's, so the interleave refines from the root toward the
> +endpoints.  The mix is between the root and the region, not granularity varying
> +arbitrarily down the hierarchy.  Linux requires granularity to change
> +monotonically, either coarsening or refining, across the decoders that route

Why that? It is not a requirement, but a limitation.

> +the request: the interleaving root, the host bridge, and any switches.  The
> +endpoint decoder is not part of that walk, it carries the region ways and
> +granularity so that it can translate, as `At Endpoint`_ describes above.  A
> +mixed-granularity region requires an interleaving root decoder.

Again, this is current. But it does not need to be documented here, as
it is changed later.

> +
> +Every decoder advances one target every multiple of its own granularity, and
> +the decoders below it subdivide the address range their parent assigns to a
> +single target.  The `Cross-Link First` example above shows the other ordering,
> +where the region granularity equals the granularity of the root decoder and
> +granularity coarsens toward the endpoints.  A mixed-granularity region refines
> +instead, reaching the region granularity at the innermost interleaving decoder.
> +
> +Refining is not a preference.  Once the region granularity is finer than the
> +root's, Linux gives each level below the root a single granularity, its
> +parent's divided by its own ways, so the ordering follows from the region and
> +root settings rather than being chosen.  The CXL Specification permits other
> +orderings and does not require a monotonic one, see `Mod3 Interleave
> +Configurations`_.  Reaching a fine granularity as early as possible remains
> +available through `Cross-Link First`, which this does not change.
> +
> +For an 8-way mixed-granularity region at 1024 below a 2-way interleaving root
> +decoder at 4096, where each host bridge interleaves across two root ports and
> +each root port hosts a 2-way switch, Linux programs::
> +
> +  Level             Ways    Granularity
> +  -----             ----    -----------
> +  Root                 2         4096
> +  Host bridge          2         2048
> +  Switch               2         1024
> +  Endpoint             8         1024
> +
> +A *region position* is the index of a region-granularity chunk within one full
> +pass of the region interleave, and each endpoint of the region backs one
> +position.  Each decoder contributes to an endpoint's region position in
> +proportion to its granularity::
> +
> +  position += target_position *
> +              decoder_granularity / region_granularity

The position is used to decode the DPA to SPA, and that is the actual
definition of the position: The position bit mask are stripped off bye
the endpoint to determine the DPA. Vice versa, it can be used to
calculate the SPA by inserting the position bits (mask size depends on
ways) at the granularity boundary into the DPA and adding the HPA
offset.

The above definition is just misleading and only reflects this special
case and implementation.

It is also odd to use division when bit positions can be used. Same
with the multiplier below.

> +
> +That multiplier is the *weight* of a level: how many region positions pass
> +between successive advances of its target index.  The root above selects a host
> +bridge every 4096 bytes, so it advances one target every four region positions,
> +weight 4, while the switch advances one target every position, weight 1.
> +
> +The two orderings weight the levels in opposite directions.  Refining, a
> +level's weight is the product of the ways of the interleaving levels below it,
> +so the innermost interleaving level has weight 1.  Coarsening, as in
> +`Cross-Link First`, it is the product of the ways of the levels above it, so
> +the root has weight 1 and advances one target per region position.
> +
> +The ways and granularity of a mixed-granularity region must describe the same
> +interleave *span* as the root decoder, the address range in which the
> +interleave pattern completes one full pass::
> +
> +  root_ways * root_granularity == region_ways * region_granularity

How is region_granularity defined?

In fact this describes, that the root selector bits must be the upper
bits in the selector mask. Even that is an unnecessary limitation to
implement support of "mixed" granularities.

> +
> +A region that does not describe that span either leaves part of the range
> +unclaimed or reaches beyond it, and Linux rejects it.  A same-granularity
> +region below a power-of-two root decoder is exempt: it may be wider than the
> +root and repeat the root targets, so its span is a multiple of the root's.
> +
> +Mod3 Interleave Configurations
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +A 3-way, 6-way, or 12-way interleave, known as a Mod3 interleave, selects its
> +target with a factor-of-three selection rather than from binary interleave
> +selector bits alone.  CXL 4.0 Section 9.13.1.1 defines the 3-way selection as
> +the address above the decoder granularity taken modulo 3.  A 6-way selection
> +claims one binary HPA bit at the decoder granularity and takes the modulo 3 of
> +the address above that bit, and a 12-way selection claims two.
> +
> +The factor-of-three selection is not an additional binary selector bit, so a
> +Mod3 interleave distributes across the hierarchy as::
> +
> +  3  = 3
> +  6  = 3 * 2
> +  12 = 3 * 4
> +
> +The cross-host bridge selection carries the factor of three, and the remaining
> +x2 or x4 is binary interleave selection below it.  A 6-way region at IGB across
> +three host bridges is therefore::
> +
> +  Device-level region:   6-way @ IGB
> +  Cross-host bridge:     3-way @ 2*IGB
> +  Below the root:        2-way @ IGB
> +
> +where both levels describe the same interleave span::
> +
> +  3 * (2 * IGB) == 6 * IGB
> +
> +The CXL Specification defines the legal Mod3 compositions and is normative.
> +CXL 4.0 Section 9.13.1.1, "Legal Interleaving Configurations: 12-way, 6-way,
> +and 3-way", Tables 9-6, 9-7, and 9-8 list them for a 12-way, 6-way, and 3-way
> +device-level interleave at IGB.  Those tables are summarized below, annotated
> +with the subset Linux supports.
> +
> +CXL 4.0 Table 9-8, 3-way device-level interleave at IGB::
> +
> +  Row  Cross-host bridge  Host bridge    Switch         Linux
> +  ---  -----------------  -----------    ------         -----
> +   1   3-way @ IGB        none           none           supported
> +
> +CXL 4.0 Table 9-7, 6-way device-level interleave at IGB::
> +
> +  Row  Cross-host bridge  Host bridge    Switch         Linux
> +  ---  -----------------  -----------    ------         -----
> +   1   6-way @ IGB        none           none           supported
> +   2   3-way @ 2*IGB      2-way @ IGB    none           supported
> +   3   3-way @ 2*IGB      none           2-way @ IGB    supported
> +
> +CXL 4.0 Table 9-6, 12-way device-level interleave at IGB::
> +
> +  Row  Cross-host bridge  Host bridge    Switch         Linux
> +  ---  -----------------  -----------    ------         -----
> +   1   12-way @ IGB       none           none           supported
> +   2   6-way @ 2*IGB      2-way @ IGB    none           supported
> +   3   6-way @ 2*IGB      none           2-way @ IGB    supported
> +   4   3-way @ 4*IGB      4-way @ IGB    none           supported
> +   5   3-way @ 4*IGB      none           4-way @ IGB    supported
> +   6   3-way @ 4*IGB      2-way @ IGB    2-way @ 2*IGB  unsupported
> +   7   3-way @ 4*IGB      2-way @ 2*IGB  2-way @ IGB    supported
> +
> +Table 9-6 row 6 is legal per the CXL Specification but is unsupported by Linux.

Exactly, unsupported config as described above.

I am good with the 3-way section above, though it just reflects, what
the spec describes and I wouldn't be that detailed here.

> +Walking it from the root toward the endpoints, granularity goes::
> +
> +  4*IGB -> IGB -> 2*IGB
> +
> +which refines and then coarsens.  Row 7 interleaves the same 12 endpoints at
> +the same granularity with those two levels exchanged::
> +
> +  4*IGB -> 2*IGB -> IGB
> +
> +which is monotonic.  Linux programs row 7 for a user region and assembles an
> +auto region whose decoders are programmed that way.  An auto region matching
> +row 6 is not assembled.
> +
> +Leaving row 6 unsupported does not prevent a 12-way device-level interleave.
> +The specification defines six other legal compositions, all monotonic and
> +supported by Linux.

Again, we should (and can) avoid this limitation. 

Suppose the following: This is a support cxl driver config:

4-way: 2-way @ IGB    2-way @ 2*IGB

Root ports and switches are configured correctly. Now, you just want
to enable 3-way which could be done without reprogramming the HDM
decoder (accept for the endpoints). Those configs become unsupported
and granularities need to be reprogrammed. This is very unexpected.

Changing implementation to use selector bit logic will solve this
limitation and make code (and doc) much easier.

Thanks,

-Robert

> +
>  Example Configurations
>  ======================
>  .. toctree::
> -- 
> 2.37.3
> 

Re: [PATCH v5 2/7] cxl/region: Warn on user region position mismatch

[Robert Richter]

On 03.09.26 16:23:44, Alison Schofield wrote:
> User region creation includes a self-test that checks the assigned endpoint
> positions against the position calculation used by auto region creation.
> The result, matching or not, is reported with dev_dbg().

As commented to the doc, the position is clearly defined as the bit
mask that is striped off by the endpoint. That should be documented in
the code to describe, what position means in that context, esp. how
the position in the target list of each level adds to the position.
That clarity is missing.

> 
> Promote that to a dev_warn() that fires only on a mismatch, so bugs in the
> auto region position calculation are easier to catch as that calculation is
> extended to support more region configurations.

dev_warn() change looks good to me.

> 
> The matching case no longer prints, so dynamic debug on region.c no longer
> confirms that the self-test ran.
> 
> Signed-off-by: Alison Schofield <alison.schofield@intel.com>
> Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>

Reviewed-by: Robert Richter <rrichter@amd.com>

Re: [PATCH v5 3/7] cxl/region: Generalize endpoint position mapping

[Robert Richter]

On 03.09.26 16:23:45, Alison Schofield wrote:
> Endpoint position calculation currently relies on the requirement that an
> interleaving root have the same granularity as the region. Auto region
> creation builds the position by multiplying by the parent decoders' ways,
> while user region creation selects the root target with 'pos % ways'. Those
> calculations are sufficient under the current granularity restriction.
> 
> In order to support mixed-granularity regions, that granularity restriction
> will need to be removed so decoder granularity can change between levels of
> the interleave hierarchy. The position calculation needs to account for
> those changes to produce the correct endpoint ordering.
> 
> Change the position calculation so each decoder's contribution is weighted
> by its granularity relative to the region granularity:
> 
>     position += target_pos *
>                 (decoder_granularity / region_granularity)

Yes, that reflects the stripped off bits the endpoint does to
determine the DPA. Actually this reflects bit positions of the
selector bits in the SPA:

|XXX...XXX|SSS|GGGGGGGG|

S ... Selector bit (width is defined by region's ways without factor 3)
G ... Granularity bit (defined by region's gran which has the smallest
      size of all decoders)

Position can be calculated using bit logic:

  target_shift = dec_gran_bits - reg_gran_bits
  pos &= ~(target_ways_mask << target_shift)
  pos |= target << target_shift

Assuming target < target_ways and reg_gran_bits <= dec_gran_bits.

Note: the 3-factor part must be added too.

A rather prefer the bit logic here.

> 
> Use the same relationship to select the root target during user region
> creation.
> 
> Weight each level by the region granularity, passed in by the caller,
> rather than by the granularity programmed in the endpoint decoder. The two
> are equal for most configurations, but not when Normalized Addressing
> leaves the endpoint decoder programmed passthrough while the region
> interleaves.

In case of Normalized Addressing, only the Endpoints have different
configurations. Root decoders, host bridges and switches keep the
config as is, thus the region's granularity is still the smallest
granularity programmed in the associated HDM decoders or described in
CFMWS.

The address translation code handles the region detection before the
targets are enumerated. Normalized Addressing should not affect the
target and position setup.

> 
> For currently supported regions, the new weighted calculation reduces to
> the existing position calculation and produces identical endpoint
> positions. The exception is a region wider than a same-granularity Mod3
> root, where address bit routing programs the level below the root at the
> region granularity, so the ratio derives a weight of one rather than three.
> A later patch in this series rejects that layout.
> 
> Signed-off-by: Alison Schofield <alison.schofield@intel.com>
> ---
>  drivers/cxl/core/region.c | 69 ++++++++++++++++++++++++---------------
>  1 file changed, 42 insertions(+), 27 deletions(-)
> 
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index 6a698f36aa6d..506b1cba1a92 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -1806,9 +1806,15 @@ static int cxl_region_attach_position(struct cxl_region *cxlr,
>  	struct cxl_decoder *cxld = &cxlsd->cxld;
>  	int iw = cxld->interleave_ways;
>  	struct cxl_port *iter;
> +	int root_pos = pos;
>  	int rc;
>  
> -	if (dport != cxlrd->cxlsd.target[pos % iw]) {
> +	/* Root target selection advances at root-granularity intervals */
> +	if (iw > 1)
> +		root_pos = pos * cxlr->params.interleave_granularity /
> +			   cxld->interleave_granularity;

Use shift right here.

So root_pos is defined as the Nth host-bridge in the cxlsd.target[]
list.

> +
> +	if (dport != cxlrd->cxlsd.target[root_pos % iw]) {

Isn't root_pos always smaller that iw? Better check for this
assumption and use root_pos to directly access the array.

>  		dev_dbg(&cxlr->dev, "%s:%s invalid target position for %s\n",
>  			dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev),
>  			dev_name(&cxlrd->cxlsd.cxld.dev));
> @@ -1909,12 +1915,13 @@ static int match_switch_decoder_by_range(struct device *dev,
>  	return (r1->start == r2->start && r1->end == r2->end);
>  }
>  
> -static int find_pos_and_ways(struct cxl_port *port, struct range *range,
> -			     int *pos, int *ways)
> +static int find_pos_and_gran(struct cxl_port *port, struct range *range,
> +			     int *pos, int *gran)
>  {
>  	struct cxl_switch_decoder *cxlsd;
>  	struct cxl_port *parent;
>  	int rc = -ENXIO;
> +	int ways;
>  
>  	parent = parent_port_of(port);
>  	if (!parent)
> @@ -1929,9 +1936,10 @@ static int find_pos_and_ways(struct cxl_port *port, struct range *range,
>  		return rc;
>  	}
>  	cxlsd = to_cxl_switch_decoder(dev);
> -	*ways = cxlsd->cxld.interleave_ways;
> +	ways = cxlsd->cxld.interleave_ways;
> +	*gran = cxlsd->cxld.interleave_granularity;
>  
> -	for (int i = 0; i < *ways; i++) {
> +	for (int i = 0; i < ways; i++) {
>  		if (cxlsd->target[i] == port->parent_dport) {
>  			*pos = i;
>  			rc = 0;
> @@ -1951,24 +1959,29 @@ static int find_pos_and_ways(struct cxl_port *port, struct range *range,
>   * cxl_calc_interleave_pos() - calculate an endpoint position in a region
>   * @cxled: endpoint decoder member of given region
>   * @hpa_range: translated HPA range of the endpoint
> + * @region_gran: interleave granularity of the region
>   *
> - * The endpoint position is calculated by traversing the topology from
> - * the endpoint to the root decoder and iteratively applying this
> - * calculation:
> + * A region position is the index of a region-granularity chunk within one full
> + * pass of the region interleave. The position is calculated by traversing the
> + * topology from the endpoint to the root decoder and accumulating the
> + * contribution of each decoder level:
>   *
> - *    position = position * parent_ways + parent_pos;
> + *    position += parent_pos * (parent_granularity / region_gran);
>   *
> - * ...where @position is inferred from switch and root decoder target lists.
> + * ...where @parent_pos is inferred from switch and root decoder target lists.
> + * The multiplier is the weight of that level: how many region positions pass
> + * between successive advances of the level's target index. A level that selects
> + * a single target contributes nothing.
>   *
>   * Return: position >= 0 on success
>   *	   -ENXIO on failure
>   */
>  static int cxl_calc_interleave_pos(struct cxl_endpoint_decoder *cxled,
> -				   struct range *hpa_range)
> +				   struct range *hpa_range, int region_gran)

Better pass cxlr here. That covers both, hpa_range and region_gran.

>  {
>  	struct cxl_port *iter, *port = cxled_to_port(cxled);
>  	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
> -	int parent_ways = 0, parent_pos = 0, pos = 0;
> +	int parent_gran = 0, parent_pos = 0, pos = 0;
>  	int rc;
>  
>  	/*
> @@ -1981,20 +1994,20 @@ static int cxl_calc_interleave_pos(struct cxl_endpoint_decoder *cxled,
>  	 *        |    |           |    |
>  	 *       mem0 mem1        mem2 mem3
>  	 *
> -	 * In the example the calculator will iterate twice. The first iteration
> -	 * uses the mem position in the host-bridge and the ways of the host-
> -	 * bridge to generate the first, or local, position. The second
> -	 * iteration uses the host-bridge position in the root_port and the ways
> -	 * of the root_port to refine the position.
> +	 * The region and the root decoder interleave at the region
> +	 * granularity, so each host-bridge decoder interleaves at twice that.
> +	 * The host-bridge decoders advance one target every two region
> +	 * positions, weight 2, and the root decoder advances one target every
> +	 * region position, weight 1.
>  	 *
>  	 * A trace of the calculation per endpoint looks like this:
> -	 * mem0: pos = 0 * 2 + 0    mem2: pos = 0 * 2 + 0
> -	 *       pos = 0 * 2 + 0          pos = 0 * 2 + 1
> +	 * mem0: pos += 0 * 2       mem2: pos += 0 * 2
> +	 *       pos += 0 * 1             pos += 1 * 1
>  	 *       pos: 0                   pos: 1
>  	 *
> -	 * mem1: pos = 0 * 2 + 1    mem3: pos = 0 * 2 + 1
> -	 *       pos = 1 * 2 + 0          pos = 1 * 2 + 1
> -	 *       pos: 2                   pos = 3
> +	 * mem1: pos += 1 * 2       meam3: pos += 1 * 2
> +	 *       pos += 0 * 1             pos += 1 * 1
> +	 *       pos: 2                   pos: 3
>  	 *
>  	 * Note that while this example is simple, the method applies to more
>  	 * complex topologies, including those with switches.
> @@ -2005,12 +2018,12 @@ static int cxl_calc_interleave_pos(struct cxl_endpoint_decoder *cxled,
>  		if (is_cxl_root(iter))
>  			break;
>  
> -		rc = find_pos_and_ways(iter, hpa_range, &parent_pos,
> -				       &parent_ways);
> +		rc = find_pos_and_gran(iter, hpa_range, &parent_pos,
> +				       &parent_gran);
>  		if (rc)
>  			return rc;
>  
> -		pos = pos * parent_ways + parent_pos;
> +		pos += parent_pos * (parent_gran / region_gran);

How about squashing all the logic into a function and just return the
positional offset caused by the decoder? Sort of cxl_port_interleave_pos?

>  	}
>  
>  	dev_dbg(&cxlmd->dev,
> @@ -2029,7 +2042,8 @@ static int cxl_region_sort_targets(struct cxl_region *cxlr)
>  	for (i = 0; i < p->nr_targets; i++) {
>  		struct cxl_endpoint_decoder *cxled = p->targets[i];
>  
> -		cxled->pos = cxl_calc_interleave_pos(cxled, &cxlr->hpa_range);
> +		cxled->pos = cxl_calc_interleave_pos(cxled, &cxlr->hpa_range,
> +						     p->interleave_granularity);

That interface could be simplified passing cxlr, see above.

-Robert

>  		/*
>  		 * Record that sorting failed, but still continue to calc
>  		 * cxled->pos so that cxl_calc_interleave_pos() emits its
> @@ -2214,7 +2228,8 @@ static int cxl_region_attach(struct cxl_region *cxlr,
>  		struct cxl_endpoint_decoder *target = p->targets[i];
>  		int test_pos;
>  
> -		test_pos = cxl_calc_interleave_pos(target, &cxlr->hpa_range);
> +		test_pos = cxl_calc_interleave_pos(target, &cxlr->hpa_range,
> +						   p->interleave_granularity);
>  		if (test_pos != target->pos)
>  			dev_warn(&cxlr->dev,
>  				 "%s: position mismatch: calculated:%d assigned:%d\n",
> -- 
> 2.37.3
> 

Re: [PATCH v5 4/7] cxl/region: Name the interleave locals in cxl_port_setup_targets()

[Robert Richter]

On 03.09.26 16:23:46, Alison Schofield wrote:
> cxl_port_setup_targets() derives the interleave granularity of one decoder
> from the granularity and ways of the decoder above it, and takes that
> decoder's own ways from the topology below it. The values from the level
> above are named parent_iw and parent_ig, but the values being derived are
> the bare iw and ig, so the reader has to work out which level each one
> describes.
> 
> Rename iw and ig to child_iw and child_ig to match the parent_ prefix used
> for the level above.

Aren't my parents my child's grandparents. :-)

I fear that does not improve it. iw_up/_down? Unsure.

-Robert

> 
> No functional change.

Re: [PATCH v5 4/7] cxl/region: Name the interleave locals in cxl_port_setup_targets()

[Robert Richter]

On 16.09.26 16:34:49, Robert Richter wrote:
> On 03.09.26 16:23:46, Alison Schofield wrote:
> > cxl_port_setup_targets() derives the interleave granularity of one decoder
> > from the granularity and ways of the decoder above it, and takes that
> > decoder's own ways from the topology below it. The values from the level
> > above are named parent_iw and parent_ig, but the values being derived are
> > the bare iw and ig, so the reader has to work out which level each one
> > describes.
> > 
> > Rename iw and ig to child_iw and child_ig to match the parent_ prefix used
> > for the level above.
> 
> Aren't my parents my child's grandparents. :-)
> 
> I fear that does not improve it. iw_up/_down? Unsure.

In the code it is finaly used for auto-config checks or else for this:

	cxld->interleave_ways = child_iw;       
	cxld->interleave_granularity = child_ig;

That looks odd now as it is not the port's or decoder's child iw. It
is just the port's decoder config for that particular endpoint.

> 
> -Robert
> 
> > 
> > No functional change.

Re: [PATCH v5 5/7] cxl/region: Support mixed-granularity auto regions

[Robert Richter]

On 03.09.26 16:23:47, Alison Schofield wrote:
> The CXL Specification permits a region's interleave granularity to differ
> between levels of the decoder hierarchy. For an auto region, the CXL driver
> reconstructs that hierarchy from the decoder configuration programmed by
> platform firmware.
> 
> The CXL driver currently requires the interleaving root to have the same
> granularity as the region, with each successive interleaving level becoming
> coarser toward the endpoints. A mixed-granularity region lifts that
> requirement: its interleave granularity is finer than the granularity of
> its interleaving root decoder. The driver cannot assemble such a region
> today.
> 
> Support mixed-granularity auto regions in the CXL driver, restricted to
> coarse-to-fine layouts. Derive each interleaving decoder's granularity from
> its parent:
> 
>         child_ig = parent_ig / child_iw
> 
> A same-granularity region keeps the existing derivation, which coarsens
> toward the endpoints:
> 
>         child_ig = parent_ig * parent_iw

This is the implementation specific limitation. Either buttom-up or
top-down.

> 
> The topology determines each decoder's interleave ways, so the parent
> granularity and child ways determine the child granularity. Use that
> relationship to validate the decoder geometry while assembling an auto
> region. Only the interleaving levels are compared; a passthrough level,
> child_iw == 1, keeps its granularity exemption.

Tracking selector bits is order independent.

> 
> Refuse a derived granularity finer than the region granularity. Dividing at

Yes, that is a general requirement as otherwise the combined
interleaving configs would never match the region.

> every level lands the deepest interleaving level exactly on the region
> granularity, so a finer value means the topology below one port is deeper
> than the region interleave has positions for, which happens only when
> another branch is too shallow to cover its share. That is what rejects an
> unbalanced tree below the interleaving root.

In general, unbalanced trees could work if all endpoints of the same
region end at the same level. Else, Endpoints would have different
granularities depending on their levels.

> 
> Require the root and region to interleave over the same span, the address
> range in which the interleave pattern completes one full pass:
> 
>         root_iw * root_ig == region_iw * region_ig
> 
> The same span relationship covers the CXL Specification's Mod3
> configurations. For example, a 6-way region at IGB across three host
> bridges uses a 3-way root interleave at 2 * IGB and a 2-way interleave at
> IGB below it, as described in CXL 4.0 Section 9.13.1.1.

Looks correct for Mod3 configs. Then, all the hpa bits above are used
to determine the host bridge interleave position. And else, the same
bits would be used again for subsequent interleaving levels causing an
invalid position (in the CXL spec, CFMWS Structure, Interleave Target
List).

But for factor-of-2 only configs that requirement is not valid. Bits
above HBIG and ENIW could be used at lower levels. Let's see the
checks added...

> 
> Signed-off-by: Alison Schofield <alison.schofield@intel.com>
> ---
>  drivers/cxl/core/region.c | 128 ++++++++++++++++++++++++++------------
>  1 file changed, 89 insertions(+), 39 deletions(-)
> 
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index 116f84bb0745..5b7104a55fdd 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -1435,6 +1435,15 @@ static int check_interleave_cap(struct cxl_decoder *cxld, int iw, int ig)
>  	return 0;
>  }
>  
> +/* Mixed granularity has a region IG finer than the interleaving root IG */
> +static bool cxl_region_is_mixed_gran(struct cxl_region *cxlr)
> +{
> +	struct cxl_decoder *cxld = &cxlr->cxlrd->cxlsd.cxld;
> +
> +	return cxld->interleave_ways > 1 &&
> +	       cxld->interleave_granularity > cxlr->params.interleave_granularity;
> +}
> +
>  static int cxl_port_setup_targets(struct cxl_port *port,
>  				  struct cxl_region *cxlr,
>  				  struct cxl_endpoint_decoder *cxled)
> @@ -1448,7 +1457,6 @@ static int cxl_port_setup_targets(struct cxl_port *port,
>  	struct cxl_region_params *p = &cxlr->params;
>  	struct cxl_decoder *cxld = cxl_rr->decoder;
>  	struct cxl_switch_decoder *cxlsd;
> -	struct cxl_port *iter = port;
>  	u16 eig, peig;
>  	u8 eiw, peiw;
>  
> @@ -1464,26 +1472,26 @@ static int cxl_port_setup_targets(struct cxl_port *port,
>  	}
>  
>  	cxlsd = to_cxl_switch_decoder(&cxld->dev);
> +	child_iw = cxl_rr->nr_targets;
> +
>  	if (cxl_rr->nr_targets_set) {
>  		int i, distance = 1;
> -		struct cxl_region_ref *cxl_rr_iter;
>  
>  		/*
> -		 * The "distance" between peer downstream ports represents which
> -		 * endpoint positions in the region interleave a given port can
> -		 * host.
> -		 *
> -		 * For example, at the root of a hierarchy the distance is
> -		 * always 1 as every index targets a different host-bridge. At
> -		 * each subsequent switch level those ports map every Nth region
> -		 * position where N is the width of the switch == distance.
> +		 * @distance is the spacing between region positions sharing this
> +		 * dport. Mixed-granularity regions place them contiguously.
>  		 */
> -		do {
> -			cxl_rr_iter = cxl_rr_load(iter, cxlr);
> -			distance *= cxl_rr_iter->nr_targets;
> -			iter = to_cxl_port(iter->dev.parent);
> -		} while (!is_cxl_root(iter));
> -		distance *= cxlrd->cxlsd.cxld.interleave_ways;
> +		if (!cxl_region_is_mixed_gran(cxlr)) {
> +			struct cxl_region_ref *cxl_rr_iter;
> +			struct cxl_port *iter = port;
> +
> +			do {
> +				cxl_rr_iter = cxl_rr_load(iter, cxlr);
> +				distance *= cxl_rr_iter->nr_targets;
> +				iter = to_cxl_port(iter->dev.parent);
> +			} while (!is_cxl_root(iter));
> +			distance *= cxlrd->cxlsd.cxld.interleave_ways;
> +		}

Worth moving that to a helper.

distance == 1 in the other case? Add to comment...

>  
>  		for (i = 0; i < cxl_rr->nr_targets_set; i++)
>  			if (ep->dport == cxlsd->target[i]) {
> @@ -1497,15 +1505,15 @@ static int cxl_port_setup_targets(struct cxl_port *port,
>  	}
>  
>  	if (is_cxl_root(parent_port)) {
> -		/*
> -		 * Root decoder IG is always set to value in CFMWS which
> -		 * may be different than this region's IG.  We can use the
> -		 * region's IG here since interleave_granularity_store()
> -		 * does not allow interleaved host-bridges with
> -		 * root IG != region IG.
> -		 */
> -		parent_ig = p->interleave_granularity;
>  		parent_iw = cxlrd->cxlsd.cxld.interleave_ways;
> +		/*
> +		 * A non-interleaving root does not contribute to the region
> +		 * interleave.
> +		 */
> +		if (parent_iw > 1)
> +			parent_ig = cxlrd->cxlsd.cxld.interleave_granularity;
> +		else
> +			parent_ig = p->interleave_granularity;

Finaly this calculates iw and ig. Can that code (including the next
part below) moved out to a function. Possibly the various code paths
could be simplified for the different cases and checks be centralized
for all of them.

>  		/*
>  		 * For purposes of address bit routing, use power-of-2 math for
>  		 * switch ports.
> @@ -1538,7 +1546,6 @@ static int cxl_port_setup_targets(struct cxl_port *port,
>  		return rc;
>  	}
>  
> -	child_iw = cxl_rr->nr_targets;
>  	rc = ways_to_eiw(child_iw, &eiw);
>  	if (rc) {
>  		dev_dbg(&cxlr->dev, "%s:%s: invalid port interleave: %d\n",
> @@ -1546,23 +1553,25 @@ static int cxl_port_setup_targets(struct cxl_port *port,
>  		return rc;
>  	}
>  
> -	/*
> -	 * Interleave granularity is a multiple of @parent_port granularity.
> -	 * Multiplier is the parent port interleave ways.
> -	 */
> -	rc = granularity_to_eig(parent_ig * parent_iw, &eig);
> -	if (rc) {
> -		dev_dbg(&cxlr->dev,
> -			"%s: invalid granularity calculation (%d * %d)\n",
> -			dev_name(&parent_port->dev), parent_ig, parent_iw);
> -		return rc;
> +	/* Only mixed-gran regions refine granularity toward the endpoints */
> +	if (cxl_region_is_mixed_gran(cxlr)) {
> +		child_ig = parent_ig / child_iw;
> +
> +		if (child_ig < p->interleave_granularity) {
> +			dev_dbg(&cxlr->dev,
> +				"%s:%s: granularity %d is finer than region granularity %d\n",
> +				dev_name(port->uport_dev), dev_name(&port->dev),
> +				child_ig, p->interleave_granularity);
> +			return -ENXIO;
> +		}
> +	} else {
> +		child_ig = parent_ig * parent_iw;
>  	}
>  
> -	rc = eig_to_granularity(eig, &child_ig);
> +	rc = granularity_to_eig(child_ig, &eig);
>  	if (rc) {
> -		dev_dbg(&cxlr->dev, "%s:%s: invalid interleave: %d\n",
> -			dev_name(port->uport_dev), dev_name(&port->dev),
> -			256 << eig);
> +		dev_dbg(&cxlr->dev, "%s:%s: invalid granularity: %d\n",
> +			dev_name(port->uport_dev), dev_name(&port->dev), child_ig);
>  		return rc;
>  	}
>  
> @@ -2061,6 +2070,43 @@ static int cxl_region_sort_targets(struct cxl_region *cxlr)
>  	return rc;
>  }
>  
> +static int cxl_region_validate_interleave(struct cxl_region *cxlr)
> +{
> +	struct cxl_decoder *cxld = &cxlr->cxlrd->cxlsd.cxld;
> +	struct cxl_region_params *p = &cxlr->params;
> +	int root_iw = cxld->interleave_ways;
> +	int root_ig = cxld->interleave_granularity;
> +
> +	if (root_iw == 1)
> +		return 0;
> +
> +	if (p->interleave_granularity > root_ig) {
> +		dev_dbg(&cxlr->dev,
> +			"granularity %d exceeds root decoder granularity %d\n",
> +			p->interleave_granularity, root_ig);

This is a valid case for the non-Mod3 case.

> +		return -ENXIO;
> +	}
> +
> +	/*
> +	 * Same-granularity regions below a power-of-two root may span multiple root
> +	 * interleaves. Mod3 roots are width-matched instead.

What about the other power-of-two root configs? Comment suggests the
check is just for Mod3, but those other configs "fall through" too.

> +	 */
> +	if (is_power_of_2(root_iw) && p->interleave_granularity == root_ig)
> +		return 0;
> +
> +	/* Span is one complete pass through the interleave pattern. */
> +	if (root_iw * root_ig != p->interleave_ways * p->interleave_granularity) {
> +		dev_dbg(&cxlr->dev,
> +			"region span %d (%d ways at %d) must equal root span %d (%d ways at %d)\n",
> +			p->interleave_ways * p->interleave_granularity,
> +			p->interleave_ways, p->interleave_granularity,
> +			root_iw * root_ig, root_iw, root_ig);
> +		return -ENXIO;
> +	}
> +
> +	return 0;
> +}
> +
>  static int cxl_region_attach(struct cxl_region *cxlr,
>  			     struct cxl_endpoint_decoder *cxled, int pos)
>  {
> @@ -2103,6 +2149,10 @@ static int cxl_region_attach(struct cxl_region *cxlr,
>  		return -ENXIO;
>  	}
>  
> +	rc = cxl_region_validate_interleave(cxlr);

maybe name this check_root_interleave and pair it with
check_interleave_cap? Function is "check" only. Will be easier to get
the function's purpose. Maybe, move it up.

I rather like to have a check close to the code it is actually needed
for. That makes it more reasonable.

-Robert

> +	if (rc)
> +		return rc;
> +
>  	if (p->nr_targets >= p->interleave_ways) {
>  		dev_dbg(&cxlr->dev, "region already has %d endpoints\n",
>  			p->nr_targets);
> -- 
> 2.37.3
> 

Re: [PATCH v5 6/7] cxl/region: Support mixed-granularity user created regions

[Robert Richter]

On 03.09.26 16:23:48, Alison Schofield wrote:
> User region creation currently requires an interleaving root to have the
> same granularity as the region.
> 
> That rejects mixed-granularity layouts, those where the region granularity
> is finer than the granularity of the interleaving root decoder. A region
> granularity coarser than the root remains invalid because it produces
> incorrect DPA translations.
> 
> Allow a region granularity finer than an interleaving root and continue to
> reject a coarser granularity. Validate the complete ways/granularity
> relationship when the region endpoints are attached.
> 
> For example, CXL 4.0 Section 9.13.1.1 Table 9-7 describes a 6-way region at
> IGB across three host bridges as a 3-way root interleave at 2 * IGB with a
> 2-way interleave at IGB below it.
> 
> Signed-off-by: Alison Schofield <alison.schofield@intel.com>
> Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>
> ---
>  drivers/cxl/core/region.c | 15 ++++++---------
>  1 file changed, 6 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index 5b7104a55fdd..374d8c2c32d1 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -572,16 +572,13 @@ static int set_interleave_granularity(struct cxl_region *cxlr, int val)
>  	if (rc)
>  		return rc;
>  
> -	/*
> -	 * When the host-bridge is interleaved, disallow region granularity !=
> -	 * root granularity. Regions with a granularity less than the root
> -	 * interleave result in needing multiple endpoints to support a single
> -	 * slot in the interleave (possible to support in the future). Regions
> -	 * with a granularity greater than the root interleave result in invalid
> -	 * DPA translations (invalid to support).
> -	 */
> -	if (cxld->interleave_ways > 1 && val != cxld->interleave_granularity)
> +	/* Region granularity must not be coarser than an interleaving root's */
> +	if (cxld->interleave_ways > 1 && val > cxld->interleave_granularity) {
> +		dev_dbg(&cxlr->dev,
> +			"granularity %d exceeds root decoder granularity %d\n",
> +			val, cxld->interleave_granularity);

The spec only restricts that for Mod3 configs. Else, there is no
driver support. Could that added as a comment?

>  		return -EINVAL;
> +	}
>  
>  	lockdep_assert_held_write(&cxl_rwsem.region);
>  
> -- 
> 2.37.3
> 

Re: [PATCH v5 0/7] cxl: Support mixed-granularity region interleaves

[Robert Richter]

On 03.09.26 16:23:42, Alison Schofield wrote:
> 
> JonathanC, I kept your tags on Patches 2,5.
> 
> RobertR, Same comment as in v4. I've dropped the Originally-by tags since the
> implementation no longer carries the selector-bit approach from your original
> patch. Let me know what authorship tags you'd like.

Fine with me.

The series is much simpler than v3 now. Though, I believe the selector
bit approach removes the limitations described, makes implemenation
easier and removes additional checks needed. Will take a look how that
could work and also give this series some testing.

Thanks for driving this.

-Robertx

> Alison Schofield (7):
>   Documentation/cxl: Describe mixed-granularity regions
>   cxl/region: Warn on user region position mismatch
>   cxl/region: Generalize endpoint position mapping
>   cxl/region: Name the interleave locals in cxl_port_setup_targets()
>   cxl/region: Support mixed-granularity auto regions
>   cxl/region: Support mixed-granularity user created regions
>   cxl/test: Add a topology to test mixed-granularity regions
> 
>  .../driver-api/cxl/linux/cxl-driver.rst       | 153 +++++++
>  drivers/cxl/core/region.c                     | 247 +++++++----
>  tools/testing/cxl/test/cxl.c                  | 410 +++++++++++++++++-
>  3 files changed, 698 insertions(+), 112 deletions(-)
> 
> 
> base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
> -- 
> 2.37.3
> 

Re: [PATCH v5 0/7] cxl: Support mixed-granularity region interleaves

[Alison Schofield]

On Wed, Sep 16, 2026 at 08:19:01PM +0200, Robert Richter wrote:
> On 03.09.26 16:23:42, Alison Schofield wrote:
> > 
> > JonathanC, I kept your tags on Patches 2,5.
> > 
> > RobertR, Same comment as in v4. I've dropped the Originally-by tags since the
> > implementation no longer carries the selector-bit approach from your original
> > patch. Let me know what authorship tags you'd like.
> 
> Fine with me.
> 
> The series is much simpler than v3 now. Though, I believe the selector
> bit approach removes the limitations described, makes implemenation
> easier and removes additional checks needed. Will take a look how that
> could work and also give this series some testing.
> 
> Thanks for driving this.

Hi Robert,

Thanks for reviewing!

Yes, the series is much simpler now. In v3 we explored the more general selector-bit
model, but as you pointed out then, there isn't a single decoder granularity to derive
for the general case without defining stricter assignment rules. That led to making
the restriction explicit in v4 and limiting support to the monotonic coarse-to-fine
configurations.

The actual use case driving this series is 6- and 12-way regions behind a 3-way host
bridge. The monotonic power-of-two cases fall naturally out of the same implementation.

The CXL driver does not need to implement every configuration the spec permits, but 
rather adds support as use cases appear.

Do you know of a platform or user requirement that needs these configurations?

A few of your responses in this set appear to expand the supported configs beyond the
current scope. I'll respond to those individually, but wanted to raise the general
concern here first. We need to bottom out on the requirements before adding more
cases to the series.


-- Alison

> 
> -Robertx
> 
> > Alison Schofield (7):
> >   Documentation/cxl: Describe mixed-granularity regions
> >   cxl/region: Warn on user region position mismatch
> >   cxl/region: Generalize endpoint position mapping
> >   cxl/region: Name the interleave locals in cxl_port_setup_targets()
> >   cxl/region: Support mixed-granularity auto regions
> >   cxl/region: Support mixed-granularity user created regions
> >   cxl/test: Add a topology to test mixed-granularity regions
> > 
> >  .../driver-api/cxl/linux/cxl-driver.rst       | 153 +++++++
> >  drivers/cxl/core/region.c                     | 247 +++++++----
> >  tools/testing/cxl/test/cxl.c                  | 410 +++++++++++++++++-
> >  3 files changed, 698 insertions(+), 112 deletions(-)
> > 
> > 
> > base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
> > -- 
> > 2.37.3
> > 

Re: [PATCH v5 1/7] Documentation/cxl: Describe mixed-granularity regions

[Alison Schofield]

On Wed, Sep 16, 2026 at 01:51:00PM +0200, Robert Richter wrote:
> On 03.09.26 16:23:43, Alison Schofield wrote:
> > Linux has required that a region's interleave granularity equal the
> > interleave granularity of its interleaving root decoder. A mixed-
> > granularity region lifts that restriction: the region granularity may be
> > finer than the root's, so the interleave refines from the root toward the
> > endpoints.
> > 
> > Mixed-granularity region support introduces interleave relationships that
> > are not obvious from the existing CXL documentation. This is particularly
> > true for the 3-, 6-, and 12-way configurations and for understanding which
> > configurations permitted by the CXL Specification are supported by Linux.
> > 
> > Document the mixed-granularity model and Linux's coarse-to-fine
> > restriction. Include the relevant configurations from CXL 4.0 Section
> > 9.13.1.1 and annotate their Linux support status.
> > 
> > This intentionally repeats information from the CXL Specification. The
> > specification remains authoritative, but showing the Linux support policy
> > alongside the legal configurations ensures readers do not mistake an
> > unsupported Linux setup for a configuration unsupported by the CXL
> > Specification.
> 
> This doc is too long. It should only contain the following:

Sure I will take another pass at trimming. I'll respond piece by piece
below.

> 
> 1) Describe a brief description of how the specification defines
> interleaving along with references to it. Only add what is needed to
> describe the cxl driver specifics in 2).
> 
> 2) Limitations and implementation specifics of the cxl driver compared
> to the specification.
> 
> This document is ambiguous on what the specs defines and that the
> driver provides. This is blurred in this doc.
> 
Let's go thru that point by point because I think that was the whole
point of the doc ;)


> > 
> > Assisted-by: Claude:Opus-5
> 
> I am fine with that, but we shouldn't use it to create tons of docs
> which we will need another AI assistant again to read. Though, I read
> it on my own. :-)

I find AI is particularly useful for the mechanical .rst work: building tables
and using correct formatting, while keeping the result readable both as rendered
HTML and plain-text patch review.

> 
> > Signed-off-by: Alison Schofield <alison.schofield@intel.com>
> > ---
> >  .../driver-api/cxl/linux/cxl-driver.rst       | 153 ++++++++++++++++++
> >  1 file changed, 153 insertions(+)
> > 
> > diff --git a/Documentation/driver-api/cxl/linux/cxl-driver.rst b/Documentation/driver-api/cxl/linux/cxl-driver.rst
> > index dd6dd17dc536..a61a20c8bb6d 100644
> > --- a/Documentation/driver-api/cxl/linux/cxl-driver.rst
> > +++ b/Documentation/driver-api/cxl/linux/cxl-driver.rst
> > @@ -602,6 +602,11 @@ derived from their upstream port connections. In `Cross-Link First` interleave
> >  configurations, the :code:`interleave_granularity` of a decoder is equal to
> >  :code:`parent_interleave_granularity * parent_interleave_ways`.
> >  
> > +When the region granularity is finer than the granularity of an interleaving
> > +root decoder, the relation inverts: the :code:`interleave_granularity` of a
> > +decoder is equal to :code:`parent_interleave_granularity / interleave_ways`.
> > +See `Mixed Granularity`_.
> 
> This introduces another unnecessary restriction: granularity must
> decrease from root down to the EP. E.g. The following config would not
> be supported:

Yes, those configurations are intentionally outside the supported subset.
Refer to the cover-letter requirements discussion.

> 
> a)
> 
>  Level             Ways    Granularity
>  -----             ----    -----------
>  Root                 2         4096
>  Host bridge          2         1024
>  Switch               2         2048
>  Endpoint             8         1024
> 
> Or:
> 
> b)
> 
>  Level             Ways    Granularity
>  -----             ----    -----------
>  Root                 3         4096
>  Host bridge          2         1024
>  Switch               2         2048
>  Endpoint             12        1024
> 
> In selector bits this is for the variants a/b:
> 
> root:   bits 12/-
> hb:     bits 10
> switch: bits 11
> 
> So the actual requirement is that the combined mask is consecutive.
> The lowest bit marks the EP granularity. The bit weight marks the
> 2-factor of the EP (total) ways.

Yes, that's the architectural requirement for the selector mask. The
restriction here is on the subset of those configs supported by Linux.


> 
> This series should implement all cases as at some point the issue will
> pop up again.

Implementing all cases is not the goal of this set, and I don't think
that is just me being gunshy. We add support to the kernel as use cases
appear.


> 
> > +
> >  At Endpoint
> >  ~~~~~~~~~~~
> >  `Endpoint Decoders` are programmed similar to Host Bridge and Switch decoders,
> > @@ -619,6 +624,154 @@ from HPA to DPA.  This is why they must be aware of the entire interleave set.
> >  Linux does not support unbalanced interleave configurations.  As a result, all
> >  endpoints in an interleave set must have the same ways and granularity.
> >  
> > +Mixed Granularity
> > +~~~~~~~~~~~~~~~~~
> > +Linux has required that a region's :code:`interleave_granularity` equal the
> > +:code:`interleave_granularity` of its interleaving root decoder.  A
> 
> This statement comments on the current kernel implementation which
> this series aims to change. It will be misleading to document that
> here. I rather would not document an implementation state that will be
> changed.

Agree. I'll remove the historical statement and describe the resulting
support only.

> 
> > +*mixed-granularity* region lifts that restriction: the region granularity may
> 
> This term is not a spec definition. It just describes a special case
> that may happen. I would avoid introducing it here. If the granularity
> implementation ware generic, there would be no need to describe that
> special case. So better describe the issue of current implementation
> and how it could be fixed.

Agree it's not CXL terminology, but it's useful terminology for the Linux
feature/subset this series implements

> 
> > +be finer than the root's, so the interleave refines from the root toward the
> > +endpoints.  The mix is between the root and the region, not granularity varying
> > +arbitrarily down the hierarchy.  Linux requires granularity to change
> > +monotonically, either coarsening or refining, across the decoders that route
> 
> Why that? It is not a requirement, but a limitation.

Yes, it is a Linux limitation. It is also a requirement for a configuration to be
supported by Linux, which is what this section is documenting.

> 
> > +the request: the interleaving root, the host bridge, and any switches.  The
> > +endpoint decoder is not part of that walk, it carries the region ways and
> > +granularity so that it can translate, as `At Endpoint`_ describes above.  A
> > +mixed-granularity region requires an interleaving root decoder.
> 
> Again, this is current. But it does not need to be documented here, as
> it is changed later.

I'll need to inspect what truly becomes obsolete. If this describes an
intermediate state superseded by the series, I'll trim it. If parts describe
the final Linux model, I'll keep those.


> 
> > +
> > +Every decoder advances one target every multiple of its own granularity, and
> > +the decoders below it subdivide the address range their parent assigns to a
> > +single target.  The `Cross-Link First` example above shows the other ordering,
> > +where the region granularity equals the granularity of the root decoder and
> > +granularity coarsens toward the endpoints.  A mixed-granularity region refines
> > +instead, reaching the region granularity at the innermost interleaving decoder.
> > +
> > +Refining is not a preference.  Once the region granularity is finer than the
> > +root's, Linux gives each level below the root a single granularity, its
> > +parent's divided by its own ways, so the ordering follows from the region and
> > +root settings rather than being chosen.  The CXL Specification permits other
> > +orderings and does not require a monotonic one, see `Mod3 Interleave
> > +Configurations`_.  Reaching a fine granularity as early as possible remains
> > +available through `Cross-Link First`, which this does not change.
> > +
> > +For an 8-way mixed-granularity region at 1024 below a 2-way interleaving root
> > +decoder at 4096, where each host bridge interleaves across two root ports and
> > +each root port hosts a 2-way switch, Linux programs::
> > +
> > +  Level             Ways    Granularity
> > +  -----             ----    -----------
> > +  Root                 2         4096
> > +  Host bridge          2         2048
> > +  Switch               2         1024
> > +  Endpoint             8         1024
> > +
> > +A *region position* is the index of a region-granularity chunk within one full
> > +pass of the region interleave, and each endpoint of the region backs one
> > +position.  Each decoder contributes to an endpoint's region position in
> > +proportion to its granularity::
> > +
> > +  position += target_position *
> > +              decoder_granularity / region_granularity
> 
> The position is used to decode the DPA to SPA, and that is the actual
> definition of the position: The position bit mask are stripped off bye
> the endpoint to determine the DPA. Vice versa, it can be used to
> calculate the SPA by inserting the position bits (mask size depends on
> ways) at the granularity boundary into the DPA and adding the HPA
> offset.
> 
> The above definition is just misleading and only reflects this special
> case and implementation.

I think I get you point, conceptually. I can define what position means
first using address translation. Then explain how this implementation
calcs it - if that needs documenting. I'll take a deeper look at this.

> 
> It is also odd to use division when bit positions can be used. Same
> with the multiplier below.

Same result. For me, division better communicates the relationship being
documented.

> 
> > +
> > +That multiplier is the *weight* of a level: how many region positions pass
> > +between successive advances of its target index.  The root above selects a host
> > +bridge every 4096 bytes, so it advances one target every four region positions,
> > +weight 4, while the switch advances one target every position, weight 1.
> > +
> > +The two orderings weight the levels in opposite directions.  Refining, a
> > +level's weight is the product of the ways of the interleaving levels below it,
> > +so the innermost interleaving level has weight 1.  Coarsening, as in
> > +`Cross-Link First`, it is the product of the ways of the levels above it, so
> > +the root has weight 1 and advances one target per region position.
> > +
> > +The ways and granularity of a mixed-granularity region must describe the same
> > +interleave *span* as the root decoder, the address range in which the
> > +interleave pattern completes one full pass::
> > +
> > +  root_ways * root_granularity == region_ways * region_granularity
> 
> How is region_granularity defined?

I'll addxregion_granularity definition before this equation.

> 
> In fact this describes, that the root selector bits must be the upper
> bits in the selector mask. Even that is an unnecessary limitation to
> implement support of "mixed" granularities.

Another scope issue.


> 
> > +
> > +A region that does not describe that span either leaves part of the range
> > +unclaimed or reaches beyond it, and Linux rejects it.  A same-granularity
> > +region below a power-of-two root decoder is exempt: it may be wider than the
> > +root and repeat the root targets, so its span is a multiple of the root's.
> > +
> > +Mod3 Interleave Configurations
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +A 3-way, 6-way, or 12-way interleave, known as a Mod3 interleave, selects its
> > +target with a factor-of-three selection rather than from binary interleave
> > +selector bits alone.  CXL 4.0 Section 9.13.1.1 defines the 3-way selection as
> > +the address above the decoder granularity taken modulo 3.  A 6-way selection
> > +claims one binary HPA bit at the decoder granularity and takes the modulo 3 of
> > +the address above that bit, and a 12-way selection claims two.
> > +
> > +The factor-of-three selection is not an additional binary selector bit, so a
> > +Mod3 interleave distributes across the hierarchy as::
> > +
> > +  3  = 3
> > +  6  = 3 * 2
> > +  12 = 3 * 4
> > +
> > +The cross-host bridge selection carries the factor of three, and the remaining
> > +x2 or x4 is binary interleave selection below it.  A 6-way region at IGB across
> > +three host bridges is therefore::
> > +
> > +  Device-level region:   6-way @ IGB
> > +  Cross-host bridge:     3-way @ 2*IGB
> > +  Below the root:        2-way @ IGB
> > +
> > +where both levels describe the same interleave span::
> > +
> > +  3 * (2 * IGB) == 6 * IGB
> > +
> > +The CXL Specification defines the legal Mod3 compositions and is normative.
> > +CXL 4.0 Section 9.13.1.1, "Legal Interleaving Configurations: 12-way, 6-way,
> > +and 3-way", Tables 9-6, 9-7, and 9-8 list them for a 12-way, 6-way, and 3-way
> > +device-level interleave at IGB.  Those tables are summarized below, annotated
> > +with the subset Linux supports.
> > +
> > +CXL 4.0 Table 9-8, 3-way device-level interleave at IGB::
> > +
> > +  Row  Cross-host bridge  Host bridge    Switch         Linux
> > +  ---  -----------------  -----------    ------         -----
> > +   1   3-way @ IGB        none           none           supported
> > +
> > +CXL 4.0 Table 9-7, 6-way device-level interleave at IGB::
> > +
> > +  Row  Cross-host bridge  Host bridge    Switch         Linux
> > +  ---  -----------------  -----------    ------         -----
> > +   1   6-way @ IGB        none           none           supported
> > +   2   3-way @ 2*IGB      2-way @ IGB    none           supported
> > +   3   3-way @ 2*IGB      none           2-way @ IGB    supported
> > +
> > +CXL 4.0 Table 9-6, 12-way device-level interleave at IGB::
> > +
> > +  Row  Cross-host bridge  Host bridge    Switch         Linux
> > +  ---  -----------------  -----------    ------         -----
> > +   1   12-way @ IGB       none           none           supported
> > +   2   6-way @ 2*IGB      2-way @ IGB    none           supported
> > +   3   6-way @ 2*IGB      none           2-way @ IGB    supported
> > +   4   3-way @ 4*IGB      4-way @ IGB    none           supported
> > +   5   3-way @ 4*IGB      none           4-way @ IGB    supported
> > +   6   3-way @ 4*IGB      2-way @ IGB    2-way @ 2*IGB  unsupported
> > +   7   3-way @ 4*IGB      2-way @ 2*IGB  2-way @ IGB    supported
> > +
> > +Table 9-6 row 6 is legal per the CXL Specification but is unsupported by Linux.
> 
> Exactly, unsupported config as described above.
> 
> I am good with the 3-way section above, though it just reflects, what
> the spec describes and I wouldn't be that detailed here.

Will trim that.


> 
> > +Walking it from the root toward the endpoints, granularity goes::
> > +
> > +  4*IGB -> IGB -> 2*IGB
> > +
> > +which refines and then coarsens.  Row 7 interleaves the same 12 endpoints at
> > +the same granularity with those two levels exchanged::
> > +
> > +  4*IGB -> 2*IGB -> IGB
> > +
> > +which is monotonic.  Linux programs row 7 for a user region and assembles an
> > +auto region whose decoders are programmed that way.  An auto region matching
> > +row 6 is not assembled.
> > +
> > +Leaving row 6 unsupported does not prevent a 12-way device-level interleave.
> > +The specification defines six other legal compositions, all monotonic and
> > +supported by Linux.
> 
> Again, we should (and can) avoid this limitation. 

Scope again. Why? Use case? Let's keep that discussion in cover letter
thread.

> 
> Suppose the following: This is a support cxl driver config:
> 
> 4-way: 2-way @ IGB    2-way @ 2*IGB
> 
> Root ports and switches are configured correctly. Now, you just want
> to enable 3-way which could be done without reprogramming the HDM
> decoder (accept for the endpoints). Those configs become unsupported
> and granularities need to be reprogrammed. This is very unexpected.
> 
> Changing implementation to use selector bit logic will solve this
> limitation and make code (and doc) much easier.

v3 had selector-derived machinery. The simplification in v4 and v5 came
after narrowing the supported set and removing it.


> 
> Thanks,
> 
> -Robert
> 
> > +
> >  Example Configurations
> >  ======================
> >  .. toctree::
> > -- 
> > 2.37.3
> > 

Re: [PATCH v5 2/7] cxl/region: Warn on user region position mismatch

[Alison Schofield]

On Wed, Sep 16, 2026 at 02:05:11PM +0200, Robert Richter wrote:
> On 03.09.26 16:23:44, Alison Schofield wrote:
> > User region creation includes a self-test that checks the assigned endpoint
> > positions against the position calculation used by auto region creation.
> > The result, matching or not, is reported with dev_dbg().
> 
> As commented to the doc, the position is clearly defined as the bit
> mask that is striped off by the endpoint. That should be documented in
> the code to describe, what position means in that context, esp. how
> the position in the target list of each level adds to the position.
> That clarity is missing.


Agreed that position could use a clearer definition. Patch 3 updates the
cxl_calc_interleave_pos() documentation, so I'll address this there in v6,
along with the documentation changes discussed in patch 1

-- Alison

> 
> > 
> > Promote that to a dev_warn() that fires only on a mismatch, so bugs in the
> > auto region position calculation are easier to catch as that calculation is
> > extended to support more region configurations.
> 
> dev_warn() change looks good to me.
> 
> > 
> > The matching case no longer prints, so dynamic debug on region.c no longer
> > confirms that the self-test ran.
> > 
> > Signed-off-by: Alison Schofield <alison.schofield@intel.com>
> > Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>
> 
> Reviewed-by: Robert Richter <rrichter@amd.com>

Re: [PATCH v5 3/7] cxl/region: Generalize endpoint position mapping

[Alison Schofield]

On Tue, Sep 08, 2026 at 12:42:07AM +0100, Jonathan Cameron wrote:
> On Thu,  3 Sep 2026 16:23:45 -0700
> Alison Schofield <alison.schofield@intel.com> wrote:
> 
> > Endpoint position calculation currently relies on the requirement that an
> > interleaving root have the same granularity as the region. Auto region
> > creation builds the position by multiplying by the parent decoders' ways,
> > while user region creation selects the root target with 'pos % ways'. Those
> > calculations are sufficient under the current granularity restriction.
> > 
> > In order to support mixed-granularity regions, that granularity restriction
> > will need to be removed so decoder granularity can change between levels of
> > the interleave hierarchy. The position calculation needs to account for
> > those changes to produce the correct endpoint ordering.
> > 
> > Change the position calculation so each decoder's contribution is weighted
> > by its granularity relative to the region granularity:
> > 
> >     position += target_pos *
> >                 (decoder_granularity / region_granularity)
> 
> In the code target_pos becomes parent_pos. The parent_pos naming
> seems more logical to me but maybe I'm missing something! (more
> that likely given it is interleave maths!)

Agree. I'll change the messaging above to be parent_pos.

-- Alison
> 
> > 
> > Use the same relationship to select the root target during user region
> > creation.
> > 
> > Weight each level by the region granularity, passed in by the caller,
> > rather than by the granularity programmed in the endpoint decoder. The two
> > are equal for most configurations, but not when Normalized Addressing
> > leaves the endpoint decoder programmed passthrough while the region
> > interleaves.
> > 
> > For currently supported regions, the new weighted calculation reduces to
> > the existing position calculation and produces identical endpoint
> > positions. The exception is a region wider than a same-granularity Mod3
> > root, where address bit routing programs the level below the root at the
> > region granularity, so the ratio derives a weight of one rather than three.
> > A later patch in this series rejects that layout.
> > 
> > Signed-off-by: Alison Schofield <alison.schofield@intel.com>
> I'll assume you'll clarify the naming thing.  Otherwise this LGTM
> Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>
> 
> > ---
> >  drivers/cxl/core/region.c | 69 ++++++++++++++++++++++++---------------
> >  1 file changed, 42 insertions(+), 27 deletions(-)
> > 
> > diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> > index 6a698f36aa6d..506b1cba1a92 100644
> > --- a/drivers/cxl/core/region.c
> > +++ b/drivers/cxl/core/region.c
> 
> > @@ -1951,24 +1959,29 @@ static int find_pos_and_ways(struct cxl_port *port, struct range *range,
> >   * cxl_calc_interleave_pos() - calculate an endpoint position in a region
> >   * @cxled: endpoint decoder member of given region
> >   * @hpa_range: translated HPA range of the endpoint
> > + * @region_gran: interleave granularity of the region
> >   *
> > - * The endpoint position is calculated by traversing the topology from
> > - * the endpoint to the root decoder and iteratively applying this
> > - * calculation:
> > + * A region position is the index of a region-granularity chunk within one full
> > + * pass of the region interleave. The position is calculated by traversing the
> > + * topology from the endpoint to the root decoder and accumulating the
> > + * contribution of each decoder level:
> >   *
> > - *    position = position * parent_ways + parent_pos;
> > + *    position += parent_pos * (parent_granularity / region_gran);
> 
> Here is the naming difference from the patch description.
> 
> >   *
> > - * ...where @position is inferred from switch and root decoder target lists.
> > + * ...where @parent_pos is inferred from switch and root decoder target lists.
> > + * The multiplier is the weight of that level: how many region positions pass
> > + * between successive advances of the level's target index. A level that selects
> > + * a single target contributes nothing.
> >   *
> >   * Return: position >= 0 on success
> >   *	   -ENXIO on failure
> >   */
> >  static int cxl_calc_interleave_pos(struct cxl_endpoint_decoder *cxled,
> > -				   struct range *hpa_range)
> > +				   struct range *hpa_range, int region_gran)
> >  {
> >  	struct cxl_port *iter, *port = cxled_to_port(cxled);
> >  	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
> > -	int parent_ways = 0, parent_pos = 0, pos = 0;
> > +	int parent_gran = 0, parent_pos = 0, pos = 0;
> >  	int rc;
> >  

Re: [PATCH v5 3/7] cxl/region: Generalize endpoint position mapping

[Alison Schofield]

On Wed, Sep 16, 2026 at 04:15:53PM +0200, Robert Richter wrote:
> On 03.09.26 16:23:45, Alison Schofield wrote:
> > Endpoint position calculation currently relies on the requirement that an
> > interleaving root have the same granularity as the region. Auto region
> > creation builds the position by multiplying by the parent decoders' ways,
> > while user region creation selects the root target with 'pos % ways'. Those
> > calculations are sufficient under the current granularity restriction.
> > 
> > In order to support mixed-granularity regions, that granularity restriction
> > will need to be removed so decoder granularity can change between levels of
> > the interleave hierarchy. The position calculation needs to account for
> > those changes to produce the correct endpoint ordering.
> > 
> > Change the position calculation so each decoder's contribution is weighted
> > by its granularity relative to the region granularity:
> > 
> >     position += target_pos *
> >                 (decoder_granularity / region_granularity)
> 
> Yes, that reflects the stripped off bits the endpoint does to
> determine the DPA. Actually this reflects bit positions of the
> selector bits in the SPA:
> 
> |XXX...XXX|SSS|GGGGGGGG|
> 
> S ... Selector bit (width is defined by region's ways without factor 3)
> G ... Granularity bit (defined by region's gran which has the smallest
>       size of all decoders)
> 
> Position can be calculated using bit logic:
> 
>   target_shift = dec_gran_bits - reg_gran_bits
>   pos &= ~(target_ways_mask << target_shift)
>   pos |= target << target_shift
> 
> Assuming target < target_ways and reg_gran_bits <= dec_gran_bits.
> 
> Note: the 3-factor part must be added too.
> 
> A rather prefer the bit logic here.

I think you are suggesting replacing:

	parent_pos * (parent_granularity / region_granularity)

with something like this:

	target_shift = dec_gran_bits - reg_gran_bits;
	pos &= ~(target_ways_mask << target_shift);
	pos |= target << target_shift;

	Plus the additional handling needed for the 3-factor case.


The current expression says directly that a position at this decoder represents
parent_granularity / region_granularity region positions. I find that easier
to read than converting the granularities to bit positions and manipulating a
selector mask, particularly when the bit version still needs separate handling
for Mod3. Also it is using the objects and concepts already present, ie.
parent position, parent granularity, region granularity.

> 
> > 
> > Use the same relationship to select the root target during user region
> > creation.
> > 
> > Weight each level by the region granularity, passed in by the caller,
> > rather than by the granularity programmed in the endpoint decoder. The two
> > are equal for most configurations, but not when Normalized Addressing
> > leaves the endpoint decoder programmed passthrough while the region
> > interleaves.
> 
> In case of Normalized Addressing, only the Endpoints have different
> configurations. Root decoders, host bridges and switches keep the
> config as is, thus the region's granularity is still the smallest
> granularity programmed in the associated HDM decoders or described in
> CFMWS.
> 
> The address translation code handles the region detection before the
> targets are enumerated. Normalized Addressing should not affect the
> target and position setup.

You're right. Normalized Addressing isn't the reason to pass the region
granularity here. The position is in region-granularity units, so the
per-level weights need to use that granularity.

The NA explanation is stale from an earlier version. I'll drop it in v6.

> 
> > 
> > For currently supported regions, the new weighted calculation reduces to
> > the existing position calculation and produces identical endpoint
> > positions. The exception is a region wider than a same-granularity Mod3
> > root, where address bit routing programs the level below the root at the
> > region granularity, so the ratio derives a weight of one rather than three.
> > A later patch in this series rejects that layout.
> > 
> > Signed-off-by: Alison Schofield <alison.schofield@intel.com>
> > ---
> >  drivers/cxl/core/region.c | 69 ++++++++++++++++++++++++---------------
> >  1 file changed, 42 insertions(+), 27 deletions(-)
> > 
> > diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> > index 6a698f36aa6d..506b1cba1a92 100644
> > --- a/drivers/cxl/core/region.c
> > +++ b/drivers/cxl/core/region.c
> > @@ -1806,9 +1806,15 @@ static int cxl_region_attach_position(struct cxl_region *cxlr,
> >  	struct cxl_decoder *cxld = &cxlsd->cxld;
> >  	int iw = cxld->interleave_ways;
> >  	struct cxl_port *iter;
> > +	int root_pos = pos;
> >  	int rc;
> >  
> > -	if (dport != cxlrd->cxlsd.target[pos % iw]) {
> > +	/* Root target selection advances at root-granularity intervals */
> > +	if (iw > 1)
> > +		root_pos = pos * cxlr->params.interleave_granularity /
> > +			   cxld->interleave_granularity;
> 
> Use shift right here.


I prefer keeping this interms of the granularity values. The division
operation make the relationship being calc'd explicit.


> 
> So root_pos is defined as the Nth host-bridge in the cxlsd.target[]
> list.
> 
> > +
> > +	if (dport != cxlrd->cxlsd.target[root_pos % iw]) {
> 
> Isn't root_pos always smaller that iw? Better check for this
> assumption and use root_pos to directly access the array.
> 

root_pos is not always less than iw.
It is the root-granularity chunk containing the region position, not yet the root target index.

For example, an existing supported configuration can have:

root:    2-way @ 4K
region:  4-way @ 4K

pos:       0  1  2  3
root_pos:  0  1  2  3
target:    0  1  0  1

The region makes multiple passes through the root interleave,
so % iw maps root_pos to the root target. Removing it would also allow an
out-of-bounds access to the root target array for positions 2 and 3.

The new mixed-granularity configurations happen to have `root_pos < iw`,
but that isn't true for all config this function supports.



> >  		dev_dbg(&cxlr->dev, "%s:%s invalid target position for %s\n",
> >  			dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev),
> >  			dev_name(&cxlrd->cxlsd.cxld.dev));
> > @@ -1909,12 +1915,13 @@ static int match_switch_decoder_by_range(struct device *dev,
> >  	return (r1->start == r2->start && r1->end == r2->end);
> >  }
> >  
> > -static int find_pos_and_ways(struct cxl_port *port, struct range *range,
> > -			     int *pos, int *ways)
> > +static int find_pos_and_gran(struct cxl_port *port, struct range *range,
> > +			     int *pos, int *gran)
> >  {
> >  	struct cxl_switch_decoder *cxlsd;
> >  	struct cxl_port *parent;
> >  	int rc = -ENXIO;
> > +	int ways;
> >  
> >  	parent = parent_port_of(port);
> >  	if (!parent)
> > @@ -1929,9 +1936,10 @@ static int find_pos_and_ways(struct cxl_port *port, struct range *range,
> >  		return rc;
> >  	}
> >  	cxlsd = to_cxl_switch_decoder(dev);
> > -	*ways = cxlsd->cxld.interleave_ways;
> > +	ways = cxlsd->cxld.interleave_ways;
> > +	*gran = cxlsd->cxld.interleave_granularity;
> >  
> > -	for (int i = 0; i < *ways; i++) {
> > +	for (int i = 0; i < ways; i++) {
> >  		if (cxlsd->target[i] == port->parent_dport) {
> >  			*pos = i;
> >  			rc = 0;
> > @@ -1951,24 +1959,29 @@ static int find_pos_and_ways(struct cxl_port *port, struct range *range,
> >   * cxl_calc_interleave_pos() - calculate an endpoint position in a region
> >   * @cxled: endpoint decoder member of given region
> >   * @hpa_range: translated HPA range of the endpoint
> > + * @region_gran: interleave granularity of the region
> >   *
> > - * The endpoint position is calculated by traversing the topology from
> > - * the endpoint to the root decoder and iteratively applying this
> > - * calculation:
> > + * A region position is the index of a region-granularity chunk within one full
> > + * pass of the region interleave. The position is calculated by traversing the
> > + * topology from the endpoint to the root decoder and accumulating the
> > + * contribution of each decoder level:
> >   *
> > - *    position = position * parent_ways + parent_pos;
> > + *    position += parent_pos * (parent_granularity / region_gran);
> >   *
> > - * ...where @position is inferred from switch and root decoder target lists.
> > + * ...where @parent_pos is inferred from switch and root decoder target lists.
> > + * The multiplier is the weight of that level: how many region positions pass
> > + * between successive advances of the level's target index. A level that selects
> > + * a single target contributes nothing.
> >   *
> >   * Return: position >= 0 on success
> >   *	   -ENXIO on failure
> >   */
> >  static int cxl_calc_interleave_pos(struct cxl_endpoint_decoder *cxled,
> > -				   struct range *hpa_range)
> > +				   struct range *hpa_range, int region_gran)
> 
> Better pass cxlr here. That covers both, hpa_range and region_gran.


Agree. Will use cxlr as param in v6


> 
> >  {
> >  	struct cxl_port *iter, *port = cxled_to_port(cxled);
> >  	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
> > -	int parent_ways = 0, parent_pos = 0, pos = 0;
> > +	int parent_gran = 0, parent_pos = 0, pos = 0;
> >  	int rc;
> >  
> >  	/*
> > @@ -1981,20 +1994,20 @@ static int cxl_calc_interleave_pos(struct cxl_endpoint_decoder *cxled,
> >  	 *        |    |           |    |
> >  	 *       mem0 mem1        mem2 mem3
> >  	 *
> > -	 * In the example the calculator will iterate twice. The first iteration
> > -	 * uses the mem position in the host-bridge and the ways of the host-
> > -	 * bridge to generate the first, or local, position. The second
> > -	 * iteration uses the host-bridge position in the root_port and the ways
> > -	 * of the root_port to refine the position.
> > +	 * The region and the root decoder interleave at the region
> > +	 * granularity, so each host-bridge decoder interleaves at twice that.
> > +	 * The host-bridge decoders advance one target every two region
> > +	 * positions, weight 2, and the root decoder advances one target every
> > +	 * region position, weight 1.
> >  	 *
> >  	 * A trace of the calculation per endpoint looks like this:
> > -	 * mem0: pos = 0 * 2 + 0    mem2: pos = 0 * 2 + 0
> > -	 *       pos = 0 * 2 + 0          pos = 0 * 2 + 1
> > +	 * mem0: pos += 0 * 2       mem2: pos += 0 * 2
> > +	 *       pos += 0 * 1             pos += 1 * 1
> >  	 *       pos: 0                   pos: 1
> >  	 *
> > -	 * mem1: pos = 0 * 2 + 1    mem3: pos = 0 * 2 + 1
> > -	 *       pos = 1 * 2 + 0          pos = 1 * 2 + 1
> > -	 *       pos: 2                   pos = 3
> > +	 * mem1: pos += 1 * 2       meam3: pos += 1 * 2
> > +	 *       pos += 0 * 1             pos += 1 * 1
> > +	 *       pos: 2                   pos: 3
> >  	 *
> >  	 * Note that while this example is simple, the method applies to more
> >  	 * complex topologies, including those with switches.
> > @@ -2005,12 +2018,12 @@ static int cxl_calc_interleave_pos(struct cxl_endpoint_decoder *cxled,
> >  		if (is_cxl_root(iter))
> >  			break;
> >  
> > -		rc = find_pos_and_ways(iter, hpa_range, &parent_pos,
> > -				       &parent_ways);
> > +		rc = find_pos_and_gran(iter, hpa_range, &parent_pos,
> > +				       &parent_gran);
> >  		if (rc)
> >  			return rc;
> >  
> > -		pos = pos * parent_ways + parent_pos;
> > +		pos += parent_pos * (parent_gran / region_gran);
> 
> How about squashing all the logic into a function and just return the
> positional offset caused by the decoder? Sort of cxl_port_interleave_pos?
> 

Maybe - I'll see how this actually looks when doing v6 changes.


> >  	}
> >  
> >  	dev_dbg(&cxlmd->dev,
> > @@ -2029,7 +2042,8 @@ static int cxl_region_sort_targets(struct cxl_region *cxlr)
> >  	for (i = 0; i < p->nr_targets; i++) {
> >  		struct cxl_endpoint_decoder *cxled = p->targets[i];
> >  
> > -		cxled->pos = cxl_calc_interleave_pos(cxled, &cxlr->hpa_range);
> > +		cxled->pos = cxl_calc_interleave_pos(cxled, &cxlr->hpa_range,
> > +						     p->interleave_granularity);
> 
> That interface could be simplified passing cxlr, see above.

Will do.

> 
> -Robert
> 
> >  		/*
> >  		 * Record that sorting failed, but still continue to calc
> >  		 * cxled->pos so that cxl_calc_interleave_pos() emits its
> > @@ -2214,7 +2228,8 @@ static int cxl_region_attach(struct cxl_region *cxlr,
> >  		struct cxl_endpoint_decoder *target = p->targets[i];
> >  		int test_pos;
> >  
> > -		test_pos = cxl_calc_interleave_pos(target, &cxlr->hpa_range);
> > +		test_pos = cxl_calc_interleave_pos(target, &cxlr->hpa_range,
> > +						   p->interleave_granularity);
> >  		if (test_pos != target->pos)
> >  			dev_warn(&cxlr->dev,
> >  				 "%s: position mismatch: calculated:%d assigned:%d\n",
> > -- 
> > 2.37.3
> > 

Re: [PATCH v5 4/7] cxo_ol/region: Name the interleave locals in cxl_port_setup_targets()

[Alison Schofield]

On Tue, Sep 08, 2026 at 12:50:49AM +0100, Jonathan Cameron wrote:
> On Thu,  3 Sep 2026 16:23:46 -0700
> Alison Schofield <alison.schofield@intel.com> wrote:
> 
> > cxl_port_setup_targets() derives the interleave granularity of one decoder
> > from the granularity and ways of the decoder above it, and takes that
> > decoder's own ways from the topology below it. The values from the level
> > above are named parent_iw and parent_ig, but the values being derived are
> > the bare iw and ig, so the reader has to work out which level each one
> > describes.
> > 
> > Rename iw and ig to child_iw and child_ig to match the parent_ prefix used
> > for the level above.
> > 
> > No functional change.
> > 
> > Suggested-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>
> > Signed-off-by: Alison Schofield <alison.schofield@intel.com>
> 
> I was wondering a bit about having ceig and ceiw but those would be
> very local so I guess not worth it.
> 
> Given how often we use ways_to_eiw() not to make the conversion
> but just to check if it is possible, I wonder if a ways_valid()
> helper is useful?
> 
> static inline int ways_valid(unsigned int ways)
> {
> 	u8 eiw;
> 
> 	return ways_to_eiw(ways, &eiw);
> }
> 
> Maybe similar for granularity.  Would remove some confusion over whether
> we actually want the encoded values or not, making this function more
> readable. I didn't look for whether it was helpful elsewhere.

I like your suggestion!

Since it makes this patchsets code more beautiful, I'm going to add it
as a pre-patch in this series. However, I will watch it, along with
the other prepatch, and will ask DaveJ to merge it ahead of the series
if it gathers tags.

There are 7 check-only call sites, 3 want ways_valid(), 3 want
granularity-valid() and one needs a eig_valid().

The impact is where you were reading. cxl_port_setup_targets() has
4 check only sites, and we can clean out 4 locals by using the new
helpers!


> Anyhow this change is good
> 
> Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>

Re: [PATCH v5 4/7] cxl/region: Name the interleave locals in cxl_port_setup_targets()

[Alison Schofield]

On Wed, Sep 16, 2026 at 04:34:44PM +0200, Robert Richter wrote:
> On 03.09.26 16:23:46, Alison Schofield wrote:
> > cxl_port_setup_targets() derives the interleave granularity of one decoder
> > from the granularity and ways of the decoder above it, and takes that
> > decoder's own ways from the topology below it. The values from the level
> > above are named parent_iw and parent_ig, but the values being derived are
> > the bare iw and ig, so the reader has to work out which level each one
> > describes.
> > 
> > Rename iw and ig to child_iw and child_ig to match the parent_ prefix used
> > for the level above.
> 
> Aren't my parents my child's grandparents. :-)
> 
> I fear that does not improve it. iw_up/_down? Unsure.


Yeah, child_* isn't a great name here. Those are the iw and ig being calc'd for cxld,
while parent_* really does come from the decoder at parent_port.

How about keeping parent_iw / parent_ig and using iw / ig for the values being calc'd
for the current decoder? I'll try that in v6.

-- Alison


> 
> -Robert
> 
> > 
> > No functional change.

Re: [PATCH v5 4/7] cxl/region: Name the interleave locals in cxl_port_setup_targets()

[Alison Schofield]

On Wed, Sep 16, 2026 at 07:45:51PM +0200, Robert Richter wrote:
> On 16.09.26 16:34:49, Robert Richter wrote:
> > On 03.09.26 16:23:46, Alison Schofield wrote:
> > > cxl_port_setup_targets() derives the interleave granularity of one decoder
> > > from the granularity and ways of the decoder above it, and takes that
> > > decoder's own ways from the topology below it. The values from the level
> > > above are named parent_iw and parent_ig, but the values being derived are
> > > the bare iw and ig, so the reader has to work out which level each one
> > > describes.
> > > 
> > > Rename iw and ig to child_iw and child_ig to match the parent_ prefix used
> > > for the level above.
> > 
> > Aren't my parents my child's grandparents. :-)
> > 
> > I fear that does not improve it. iw_up/_down? Unsure.
> 
> In the code it is finaly used for auto-config checks or else for this:
> 
> 	cxld->interleave_ways = child_iw;       
> 	cxld->interleave_granularity = child_ig;
> 
> That looks odd now as it is not the port's or decoder's child iw. It
> is just the port's decoder config for that particular endpoint.

Agreed. This makes it clearer that child_* is misleading.
I'll keep parent_iw / parent_ig for the values from the decoder above
and use iw / ig for the values being calculated for this cxld.


> 
> > 
> > -Robert
> > 
> > > 
> > > No functional change.

Re: [PATCH v5 2/7] cxl/region: Warn on user region position mismatch

[Alison Schofield]

On Thu, Sep 03, 2026 at 04:23:44PM -0700, Alison Schofield wrote:
> User region creation includes a self-test that checks the assigned endpoint
> positions against the position calculation used by auto region creation.
> The result, matching or not, is reported with dev_dbg().
> 
> Promote that to a dev_warn() that fires only on a mismatch, so bugs in the
> auto region position calculation are easier to catch as that calculation is
> extended to support more region configurations.
> 
> The matching case no longer prints, so dynamic debug on region.c no longer
> confirms that the self-test ran.
> 
> Signed-off-by: Alison Schofield <alison.schofield@intel.com>
> Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>

DaveJ,

RobertR also tagged this one.

I'm requesting you merge this ahead of the rest of the series.

It is independent of the rest of this work, and landing it sooner
helps me with a unit test that wants that message to always be present.

Thanks for taking a look!

-- Alison


> ---
>  drivers/cxl/core/region.c | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index 27e63e6dab7c..6a698f36aa6d 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -2215,10 +2215,11 @@ static int cxl_region_attach(struct cxl_region *cxlr,
>  		int test_pos;
>  
>  		test_pos = cxl_calc_interleave_pos(target, &cxlr->hpa_range);
> -		dev_dbg(&target->cxld.dev,
> -			"Test cxl_calc_interleave_pos(): %s test_pos:%d target->pos:%d\n",
> -			(test_pos == target->pos) ? "success" : "fail",
> -			test_pos, target->pos);
> +		if (test_pos != target->pos)
> +			dev_warn(&cxlr->dev,
> +				 "%s: position mismatch: calculated:%d assigned:%d\n",
> +				 dev_name(&target->cxld.dev), test_pos,
> +				 target->pos);
>  	}
>  
>  	return 0;
> -- 
> 2.37.3
> 

Re: [PATCH v5 0/7] cxl: Support mixed-granularity region interleaves

[Dave Jiang]

On 9/3/26 4:23 PM, Alison Schofield wrote:
> 
> JonathanC, I kept your tags on Patches 2,5.
> 
> RobertR, Same comment as in v4. I've dropped the Originally-by tags since the
> implementation no longer carries the selector-bit approach from your original
> patch. Let me know what authorship tags you'd like.
> 
> 
> Changes in v5:
> Series wide:
> - Rebase onto 7.3-rc1
> - Move the documentation patch to the front of the series (Jonathan)
> - Reword commit messages to use new terms of documentation (Jonathan)
> - Wrap code and code comments closer to 80 columns (Jonathan)
> - Rewrap all commit messages to 75 columns (Jonathan)
> - Split combined declarations (Jonathan)
> Patch 1: Documentation/cxl: Describe mixed-granularity regions
> - Define a mixed-granularity region and monotonic requirement (Jonathan)
> - Define span, region position, and weight (Jonathan)
> - Explain refining granularity and cross-link first (Jonathan)
> - Use an 8-way example interleaved at the host bridge w one switch (Jonathan)
> - Clarify config 'legal per the CXL Spec but unsupported by Linux' (Jonathan)
> Patch 2: cxl/region: Warn on user region position mismatch
> - Log the position mismatch against the region device and name the decoder
> Patch 3: cxl/region: Generalize endpoint position mapping
> - Pass the region interleave granularity into cxl_calc_interleave_pos() instead
>   of using the endpoint decoder granularity (RichardC)
> - Rewrite the example comment to define a region position and each decoder
>   level's weight, rather than what it spans (Jonathan)
> New Patch 4: cxl/region: Name the interleave locals in cxl_port_setup_targets()
> - Rename iw and ig to child_iw and child_ig (Jonathan)
> Patch 5: cxl/region: Support mixed-granularity auto regions
> - Restore the topology-derived peer distance, using 1 only for mixed-granularity
>   regions, avoiding a NULL deref on the auto path
> - Reject a layout whose derived granularity comes out finer than the region
>   granularity, which the topology-derived distance used to catch
> - Define what a span is where the root and region spans are compared (Jonathan)
> - Add comment that the power-of-two exemption is for same-granularity regions
>   that span multiple root interleaves, and that Mod3 roots are width-matched
>   instead (Jonathan, Sashiko)
> - Report both spans in the span-check dev_dbg
> Patch 6: cxl/region: Support mixed-granularity user created regions
> - Add a dev_dbg naming both granularities to the coarser-than-root rejection
> Patch 7: cxl/test: Add a topology to test mixed-granularity regions
> - Rebuild the topology as cross-HB, then HB, then switch (Jonathan)
> - Add a third window to support a 12-way region at 256
> - Drop the cascade of two switch levels (Jonathan)
> - Replace bare '2' with NR_CXL_MIX_GRAN_RPS_PER_HB
> - Create and use a chbs_range() helper
> Link to v4:
> https://lore.kernel.org/linux-cxl/cover.1787255388.git.alison.schofield@intel.com/
> 
> Changes in v4:
> - Rework port decoder setup around parent granularity and target count,
>   dropping the selector-walk and selector-derived granularity machinery
>   from v3 (RobertR)
> - Replace the mixed-granularity-specific endpoint position handling with
>   one weighted position calculation for existing and mixed-gran regions
> - Derive mixed-gran port decoder settings directly from the parent interleave
>   geometry
> - Promote the user-region position self-test from dev_dbg() to dev_warn()
> - Update the documentation patch
> - Update the cover letter summary and series structure
> Link to v3:
> https://lore.kernel.org/linux-cxl/cover.1785444498.git.alison.schofield@intel.com/
> 
> Changes in v3:
> - Reduce scope to coarse-to-fine ordering (RichardC, Sashiko)
> - Reject out-of-order layouts (RichardC, Sashiko)
> - P1: Factor the selector walk and per-port checks into helpers (RobertR)
>   That pre-work is the new Patch 1/8 and the Series Structure section of
>   this cover letter describe same below.
> - P3: Derive granularity from the highest available selector span (RobertR)
> - P3: Validate auto-programmed granularity against the derived value
> - P4: Rename root_pos_stride() to root_positions_per_target() (RobertR)
> - P4: Drop the "stride" terminology throughout (RobertR)
> - The complex Patch 4 of v2 is split into patches 4-6 in v3.
>   Also noted in the Series Structure section of cover letter below.
> - P9: Move the peer-distance walk and per-function walkthroughs from the doc
>   into in-code comments and kernel-doc (RobertR)
> - Drop Reviewed-by tags
> Link to v2:
> https://lore.kernel.org/linux-cxl/cover.1781199122.git.alison.schofield@intel.com/
> 
> Changes in v2:
> - Patch 1,2: Defer the unused selector var store to keep P1 bisectable (Sashiko)
> - Patch 1: Make divide by 3 in get_selctor() work on 32-bit builds) (lkp)
> - Patch 4: Use local vars in cxl_region_attach() for readability (DaveJ)
> - Patch 5: Add NULL checks on unused mock arrays (Sashiko)
> - Resolved errant err_rch unwind with rc7 merge (DaveJ)
> - Rebase onto 7.1-rc7
> - Update commit logs in 1,2,5 to align w changes in v2
> Link to v1:
> https://lore.kernel.org/all/cover.1780095671.git.alison.schofield@intel.com/
> 
> 
> Begin Cover Letter:
> 
> Linux has required that a region's interleave granularity equal the interleave
> granularity of its interleaving root decoder. A mixed-granularity region lifts
> that restriction: the region granularity may be finer than the root's, so the
> interleave refines from the root toward the endpoints. The mix is between the
> root and the region, not granularity varying arbitrarily down the hierarchy.
> 
> A CXL region may interleave across multiple decoder levels: root, optional
> switches, and endpoint. The equal-granularity requirement blocks legal
> mixed-granularity arrangements permitted by CXL 4.0 Section 9.13.1 and makes
> some 6-way and 12-way configurations defined in Section 9.13.1.1 (Tables 9-6,
> 9-7) impossible to create.
> 
> Two prior proposals addressed parts of this gap:
> 
> AlisonS introduced position arithmetic and sysfs gating for auto and
> user-created regions to support the 6- and 12-way interleave configurations
> without a same-granularity alternative:
> https://lore.kernel.org/all/20250306232239.2609017-1-alison.schofield@intel.com/
> 
> RobertR introduced an HPA selector-bit model for auto regions that allows
> multi-level power-of-two interleaves regardless of granularity ordering:
> https://lore.kernel.org/all/20251028094754.72816-1-rrichter@amd.com/
> 
> This series adds support for CXL regions where interleave granularity differs
> between levels of the decoder hierarchy. The support applies to both firmware
> programmed auto regions and user created regions.
> 
> Linux supports mixed-granularity configurations that are monotonic from the
> interleaving root toward the endpoints. Granularity may remain the same or
> become finer at each interleaving level. Configurations that refine and then
> become coarser are permitted by the CXL Specification but are not supported
> by Linux.
> 
> This support covers power-of-two and Mod3 interleaves at any depth. Of the Mod3
> configs listed in CXL 4.0 Section 9.13.1.1, all are supported except Table 9-6
> row 6, the refine-then-coarsen 12-way composition. See the Patch 1 documentation
> update for additional info.
> 
> 
> Series structure in v5:
> -----------------------
> Patch 1 documents the mixed-granularity model and Linux support policy.
> It comes first so that the terms the rest of the series uses - span,
> region position, weight - are defined before they are used.
> 
> Patch 2 promotes the existing user-region endpoint position self-test to
> a visible diagnostic.
> 
> Patch 3 generalizes endpoint position calculation to account for
> granularity changes between decoder levels.
> 
> Patch 4 names the interleave locals in cxl_port_setup_targets(). It is a
> mechanical rename, carried separately so it does not muddy Patch 5.
> 
> Patch 5 enables mixed-granularity auto regions and validates their
> coarse-to-fine decoder geometry.
> 
> Patch 6 enables mixed-granularity user created regions by relaxing the
> existing root/region granularity restriction.
> 
> Patch 7 adds a cxl_test topology and coverage for mixed-granularity
> regions.
> 
> 
> The companion NDCTL patchset that allows mixed-granularity 'cxl create-region'
> and adds the unit test is posted here:
> https://lore.kernel.org/nvdimm/cover.1788469004.git.alison.schofield@intel.com/
> 
> 
> Alison Schofield (7):
>   Documentation/cxl: Describe mixed-granularity regions
>   cxl/region: Warn on user region position mismatch
>   cxl/region: Generalize endpoint position mapping
>   cxl/region: Name the interleave locals in cxl_port_setup_targets()
>   cxl/region: Support mixed-granularity auto regions
>   cxl/region: Support mixed-granularity user created regions
>   cxl/test: Add a topology to test mixed-granularity regions
> 
>  .../driver-api/cxl/linux/cxl-driver.rst       | 153 +++++++
>  drivers/cxl/core/region.c                     | 247 +++++++----
>  tools/testing/cxl/test/cxl.c                  | 410 +++++++++++++++++-
>  3 files changed, 698 insertions(+), 112 deletions(-)
> 
> 
> base-commit: cee9395acd8043be0644b25c34bfa86623f2b935

2/7 merged to cxl/next:
682d3772be80