* [PATCH v5 2/3] cxl, doc: Moving conventions in separate files
2026-02-03 17:35 [PATCH v5 1/3] cxl, doc: Remove isonum.txt inclusion Robert Richter
@ 2026-02-03 17:35 ` Robert Richter
2026-02-03 17:36 ` [PATCH v5 3/3] Documentation/driver-api/cxl: ACPI PRM Address Translation Support and AMD Zen5 enablement Robert Richter
1 sibling, 0 replies; 5+ messages in thread
From: Robert Richter @ 2026-02-03 17:35 UTC (permalink / raw)
To: Alison Schofield, Vishal Verma, Ira Weiny, Dan Williams,
Jonathan Cameron, Dave Jiang, Davidlohr Bueso, Jonathan Corbet
Cc: linux-cxl, linux-kernel, Gregory Price, Fabio M. De Francesco,
Terry Bowman, Joshua Hahn, Robert Richter, linux-doc
Moving conventions in separate files.
Cc: Jonathan Corbet <corbet@lwn.net>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Alison Schofield <alison.schofield@intel.com>
Signed-off-by: Robert Richter <rrichter@amd.com>
---
v4:
* updated sob-chain,
v3:
* updated sob-chain,
* move isonum.txt removal to separate patch (Jonathan Cameron),
* kept intro in conventions.rst (Jonathan Cameron),
* removed additional blank line in template (Jonathan Cameron),
v2:
* removed include:: <isonum.txt> line (Jonathan Corbet).
---
Documentation/driver-api/cxl/conventions.rst | 176 +-----------------
.../driver-api/cxl/conventions/cxl-lmh.rst | 135 ++++++++++++++
.../driver-api/cxl/conventions/template.rst | 37 ++++
3 files changed, 178 insertions(+), 170 deletions(-)
create mode 100644 Documentation/driver-api/cxl/conventions/cxl-lmh.rst
create mode 100644 Documentation/driver-api/cxl/conventions/template.rst
diff --git a/Documentation/driver-api/cxl/conventions.rst b/Documentation/driver-api/cxl/conventions.rst
index ed4237583d36..9267a697b2fe 100644
--- a/Documentation/driver-api/cxl/conventions.rst
+++ b/Documentation/driver-api/cxl/conventions.rst
@@ -1,8 +1,7 @@
.. SPDX-License-Identifier: GPL-2.0
-=======================================
Compute Express Link: Linux Conventions
-=======================================
+#######################################
There exists shipping platforms that bend or break CXL specification
expectations. Record the details and the rationale for those deviations.
@@ -10,172 +9,9 @@ Borrow the ACPI Code First template format to capture the assumptions
and tradeoffs such that multiple platform implementations can follow the
same convention.
-<(template) Title>
-==================
+.. toctree::
+ :maxdepth: 1
+ :caption: Contents
-Document
---------
-CXL Revision <rev>, Version <ver>
-
-License
--------
-SPDX-License Identifier: CC-BY-4.0
-
-Creator/Contributors
---------------------
-
-Summary of the Change
----------------------
-
-<Detail the conflict with the specification and where available the
-assumptions and tradeoffs taken by the hardware platform.>
-
-
-Benefits of the Change
-----------------------
-
-<Detail what happens if platforms and Linux do not adopt this
-convention.>
-
-References
-----------
-
-Detailed Description of the Change
-----------------------------------
-
-<Propose spec language that corrects the conflict.>
-
-
-Resolve conflict between CFMWS, Platform Memory Holes, and Endpoint Decoders
-============================================================================
-
-Document
---------
-
-CXL Revision 3.2, Version 1.0
-
-License
--------
-
-SPDX-License Identifier: CC-BY-4.0
-
-Creator/Contributors
---------------------
-
-- Fabio M. De Francesco, Intel
-- Dan J. Williams, Intel
-- Mahesh Natu, Intel
-
-Summary of the Change
----------------------
-
-According to the current Compute Express Link (CXL) Specifications (Revision
-3.2, Version 1.0), the CXL Fixed Memory Window Structure (CFMWS) describes zero
-or more Host Physical Address (HPA) windows associated with each CXL Host
-Bridge. Each window represents a contiguous HPA range that may be interleaved
-across one or more targets, including CXL Host Bridges. Each window has a set
-of restrictions that govern its usage. It is the Operating System-directed
-configuration and Power Management (OSPM) responsibility to utilize each window
-for the specified use.
-
-Table 9-22 of the current CXL Specifications states that the Window Size field
-contains the total number of consecutive bytes of HPA this window describes.
-This value must be a multiple of the Number of Interleave Ways (NIW) * 256 MB.
-
-Platform Firmware (BIOS) might reserve physical addresses below 4 GB where a
-memory gap such as the Low Memory Hole for PCIe MMIO may exist. In such cases,
-the CFMWS Range Size may not adhere to the NIW * 256 MB rule.
-
-The HPA represents the actual physical memory address space that the CXL devices
-can decode and respond to, while the System Physical Address (SPA), a related
-but distinct concept, represents the system-visible address space that users can
-direct transaction to and so it excludes reserved regions.
-
-BIOS publishes CFMWS to communicate the active SPA ranges that, on platforms
-with LMH's, map to a strict subset of the HPA. The SPA range trims out the hole,
-resulting in lost capacity in the Endpoints with no SPA to map to that part of
-the HPA range that intersects the hole.
-
-E.g, an x86 platform with two CFMWS and an LMH starting at 2 GB:
-
- +--------+------------+-------------------+------------------+-------------------+------+
- | Window | CFMWS Base | CFMWS Size | HDM Decoder Base | HDM Decoder Size | Ways |
- +========+============+===================+==================+===================+======+
- | 0 | 0 GB | 2 GB | 0 GB | 3 GB | 12 |
- +--------+------------+-------------------+------------------+-------------------+------+
- | 1 | 4 GB | NIW*256MB Aligned | 4 GB | NIW*256MB Aligned | 12 |
- +--------+------------+-------------------+------------------+-------------------+------+
-
-HDM decoder base and HDM decoder size represent all the 12 Endpoint Decoders of
-a 12 ways region and all the intermediate Switch Decoders. They are configured
-by the BIOS according to the NIW * 256MB rule, resulting in a HPA range size of
-3GB. Instead, the CFMWS Base and CFMWS Size are used to configure the Root
-Decoder HPA range that results smaller (2GB) than that of the Switch and
-Endpoint Decoders in the hierarchy (3GB).
-
-This creates 2 issues which lead to a failure to construct a region:
-
-1) A mismatch in region size between root and any HDM decoder. The root decoders
- will always be smaller due to the trim.
-
-2) The trim causes the root decoder to violate the (NIW * 256MB) rule.
-
-This change allows a region with a base address of 0GB to bypass these checks to
-allow for region creation with the trimmed root decoder address range.
-
-This change does not allow for any other arbitrary region to violate these
-checks - it is intended exclusively to enable x86 platforms which map CXL memory
-under 4GB.
-
-Despite the HDM decoders covering the PCIE hole HPA region, it is expected that
-the platform will never route address accesses to the CXL complex because the
-root decoder only covers the trimmed region (which excludes this). This is
-outside the ability of Linux to enforce.
-
-On the example platform, only the first 2GB will be potentially usable, but
-Linux, aiming to adhere to the current specifications, fails to construct
-Regions and attach Endpoint and intermediate Switch Decoders to them.
-
-There are several points of failure that due to the expectation that the Root
-Decoder HPA size, that is equal to the CFMWS from which it is configured, has
-to be greater or equal to the matching Switch and Endpoint HDM Decoders.
-
-In order to succeed with construction and attachment, Linux must construct a
-Region with Root Decoder HPA range size, and then attach to that all the
-intermediate Switch Decoders and Endpoint Decoders that belong to the hierarchy
-regardless of their range sizes.
-
-Benefits of the Change
-----------------------
-
-Without the change, the OSPM wouldn't match intermediate Switch and Endpoint
-Decoders with Root Decoders configured with CFMWS HPA sizes that don't align
-with the NIW * 256MB constraint, and so it leads to lost memdev capacity.
-
-This change allows the OSPM to construct Regions and attach intermediate Switch
-and Endpoint Decoders to them, so that the addressable part of the memory
-devices total capacity is made available to the users.
-
-References
-----------
-
-Compute Express Link Specification Revision 3.2, Version 1.0
-<https://www.computeexpresslink.org/>
-
-Detailed Description of the Change
-----------------------------------
-
-The description of the Window Size field in table 9-22 needs to account for
-platforms with Low Memory Holes, where SPA ranges might be subsets of the
-endpoints HPA. Therefore, it has to be changed to the following:
-
-"The total number of consecutive bytes of HPA this window represents. This value
-shall be a multiple of NIW * 256 MB.
-
-On platforms that reserve physical addresses below 4 GB, such as the Low Memory
-Hole for PCIe MMIO on x86, an instance of CFMWS whose Base HPA range is 0 might
-have a size that doesn't align with the NIW * 256 MB constraint.
-
-Note that the matching intermediate Switch Decoders and the Endpoint Decoders
-HPA range sizes must still align to the above-mentioned rule, but the memory
-capacity that exceeds the CFMWS window size won't be accessible.".
+ conventions/cxl-lmh.rst
+ conventions/template.rst
diff --git a/Documentation/driver-api/cxl/conventions/cxl-lmh.rst b/Documentation/driver-api/cxl/conventions/cxl-lmh.rst
new file mode 100644
index 000000000000..baece5c35345
--- /dev/null
+++ b/Documentation/driver-api/cxl/conventions/cxl-lmh.rst
@@ -0,0 +1,135 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Resolve conflict between CFMWS, Platform Memory Holes, and Endpoint Decoders
+============================================================================
+
+Document
+--------
+
+CXL Revision 3.2, Version 1.0
+
+License
+-------
+
+SPDX-License Identifier: CC-BY-4.0
+
+Creator/Contributors
+--------------------
+
+- Fabio M. De Francesco, Intel
+- Dan J. Williams, Intel
+- Mahesh Natu, Intel
+
+Summary of the Change
+---------------------
+
+According to the current Compute Express Link (CXL) Specifications (Revision
+3.2, Version 1.0), the CXL Fixed Memory Window Structure (CFMWS) describes zero
+or more Host Physical Address (HPA) windows associated with each CXL Host
+Bridge. Each window represents a contiguous HPA range that may be interleaved
+across one or more targets, including CXL Host Bridges. Each window has a set
+of restrictions that govern its usage. It is the Operating System-directed
+configuration and Power Management (OSPM) responsibility to utilize each window
+for the specified use.
+
+Table 9-22 of the current CXL Specifications states that the Window Size field
+contains the total number of consecutive bytes of HPA this window describes.
+This value must be a multiple of the Number of Interleave Ways (NIW) * 256 MB.
+
+Platform Firmware (BIOS) might reserve physical addresses below 4 GB where a
+memory gap such as the Low Memory Hole for PCIe MMIO may exist. In such cases,
+the CFMWS Range Size may not adhere to the NIW * 256 MB rule.
+
+The HPA represents the actual physical memory address space that the CXL devices
+can decode and respond to, while the System Physical Address (SPA), a related
+but distinct concept, represents the system-visible address space that users can
+direct transaction to and so it excludes reserved regions.
+
+BIOS publishes CFMWS to communicate the active SPA ranges that, on platforms
+with LMH's, map to a strict subset of the HPA. The SPA range trims out the hole,
+resulting in lost capacity in the Endpoints with no SPA to map to that part of
+the HPA range that intersects the hole.
+
+E.g, an x86 platform with two CFMWS and an LMH starting at 2 GB:
+
+ +--------+------------+-------------------+------------------+-------------------+------+
+ | Window | CFMWS Base | CFMWS Size | HDM Decoder Base | HDM Decoder Size | Ways |
+ +========+============+===================+==================+===================+======+
+ | 0 | 0 GB | 2 GB | 0 GB | 3 GB | 12 |
+ +--------+------------+-------------------+------------------+-------------------+------+
+ | 1 | 4 GB | NIW*256MB Aligned | 4 GB | NIW*256MB Aligned | 12 |
+ +--------+------------+-------------------+------------------+-------------------+------+
+
+HDM decoder base and HDM decoder size represent all the 12 Endpoint Decoders of
+a 12 ways region and all the intermediate Switch Decoders. They are configured
+by the BIOS according to the NIW * 256MB rule, resulting in a HPA range size of
+3GB. Instead, the CFMWS Base and CFMWS Size are used to configure the Root
+Decoder HPA range that results smaller (2GB) than that of the Switch and
+Endpoint Decoders in the hierarchy (3GB).
+
+This creates 2 issues which lead to a failure to construct a region:
+
+1) A mismatch in region size between root and any HDM decoder. The root decoders
+ will always be smaller due to the trim.
+
+2) The trim causes the root decoder to violate the (NIW * 256MB) rule.
+
+This change allows a region with a base address of 0GB to bypass these checks to
+allow for region creation with the trimmed root decoder address range.
+
+This change does not allow for any other arbitrary region to violate these
+checks - it is intended exclusively to enable x86 platforms which map CXL memory
+under 4GB.
+
+Despite the HDM decoders covering the PCIE hole HPA region, it is expected that
+the platform will never route address accesses to the CXL complex because the
+root decoder only covers the trimmed region (which excludes this). This is
+outside the ability of Linux to enforce.
+
+On the example platform, only the first 2GB will be potentially usable, but
+Linux, aiming to adhere to the current specifications, fails to construct
+Regions and attach Endpoint and intermediate Switch Decoders to them.
+
+There are several points of failure that due to the expectation that the Root
+Decoder HPA size, that is equal to the CFMWS from which it is configured, has
+to be greater or equal to the matching Switch and Endpoint HDM Decoders.
+
+In order to succeed with construction and attachment, Linux must construct a
+Region with Root Decoder HPA range size, and then attach to that all the
+intermediate Switch Decoders and Endpoint Decoders that belong to the hierarchy
+regardless of their range sizes.
+
+Benefits of the Change
+----------------------
+
+Without the change, the OSPM wouldn't match intermediate Switch and Endpoint
+Decoders with Root Decoders configured with CFMWS HPA sizes that don't align
+with the NIW * 256MB constraint, and so it leads to lost memdev capacity.
+
+This change allows the OSPM to construct Regions and attach intermediate Switch
+and Endpoint Decoders to them, so that the addressable part of the memory
+devices total capacity is made available to the users.
+
+References
+----------
+
+Compute Express Link Specification Revision 3.2, Version 1.0
+<https://www.computeexpresslink.org/>
+
+Detailed Description of the Change
+----------------------------------
+
+The description of the Window Size field in table 9-22 needs to account for
+platforms with Low Memory Holes, where SPA ranges might be subsets of the
+endpoints HPA. Therefore, it has to be changed to the following:
+
+"The total number of consecutive bytes of HPA this window represents. This value
+shall be a multiple of NIW * 256 MB.
+
+On platforms that reserve physical addresses below 4 GB, such as the Low Memory
+Hole for PCIe MMIO on x86, an instance of CFMWS whose Base HPA range is 0 might
+have a size that doesn't align with the NIW * 256 MB constraint.
+
+Note that the matching intermediate Switch Decoders and the Endpoint Decoders
+HPA range sizes must still align to the above-mentioned rule, but the memory
+capacity that exceeds the CFMWS window size won't be accessible.".
diff --git a/Documentation/driver-api/cxl/conventions/template.rst b/Documentation/driver-api/cxl/conventions/template.rst
new file mode 100644
index 000000000000..ff2fcf1b5e24
--- /dev/null
+++ b/Documentation/driver-api/cxl/conventions/template.rst
@@ -0,0 +1,37 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. :: Template Title here:
+
+Template File
+=============
+
+Document
+--------
+CXL Revision <rev>, Version <ver>
+
+License
+-------
+SPDX-License Identifier: CC-BY-4.0
+
+Creator/Contributors
+--------------------
+
+Summary of the Change
+---------------------
+
+<Detail the conflict with the specification and where available the
+assumptions and tradeoffs taken by the hardware platform.>
+
+Benefits of the Change
+----------------------
+
+<Detail what happens if platforms and Linux do not adopt this
+convention.>
+
+References
+----------
+
+Detailed Description of the Change
+----------------------------------
+
+<Propose spec language that corrects the conflict.>
--
2.47.3
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH v5 3/3] Documentation/driver-api/cxl: ACPI PRM Address Translation Support and AMD Zen5 enablement
2026-02-03 17:35 [PATCH v5 1/3] cxl, doc: Remove isonum.txt inclusion Robert Richter
2026-02-03 17:35 ` [PATCH v5 2/3] cxl, doc: Moving conventions in separate files Robert Richter
@ 2026-02-03 17:36 ` Robert Richter
2026-02-03 18:50 ` dan.j.williams
1 sibling, 1 reply; 5+ messages in thread
From: Robert Richter @ 2026-02-03 17:36 UTC (permalink / raw)
To: Alison Schofield, Vishal Verma, Ira Weiny, Dan Williams,
Jonathan Cameron, Dave Jiang, Davidlohr Bueso, Jonathan Corbet
Cc: linux-cxl, linux-kernel, Gregory Price, Fabio M. De Francesco,
Terry Bowman, Joshua Hahn, Robert Richter, linux-doc
This adds a convention document for the following patch series:
cxl: ACPI PRM Address Translation Support and AMD Zen5 enablement
Version 7 and later:
https://lore.kernel.org/linux-cxl/20251114213931.30754-1-rrichter@amd.com/
Link: https://lore.kernel.org/linux-cxl/20251114213931.30754-1-rrichter@amd.com/
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Alison Schofield <alison.schofield@intel.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Robert Richter <rrichter@amd.com>
---
v5:
* updated sob-chain (Dan),
* removed specification of _DSM,
* added PRM handler definition to the specification,
* made PRM handler mandatory including the GUID handler id to be used,
* added references to PRM spec
* revised the entire doc and made small format and grammar fixups and
wording improvements,
v4:
* updated sob-chain (Dave, Alison, Jonathan, Dan),
* added additional spec change for Normalized addressing and _DSM
definition for address translation (Dan),
* revised the entire doc and made small format and grammar fixups and
wording improvements,
v3:
* updated link in descriptions and add Link: tag (Jonathan Cameron),
* clarified CFMWS and HPA range description (Jonathan Cameron),
* use 80 chars line limit (Jonathan Cameron),
* added block diagram to illustrate the example address mappings (Dave),
v2:
* updated sob-chain,
* spell fix in patch description (Randy),
* made small changes as suggested by Randy,
* Removed include:: <isonum.txt> line (Jon).
Signed-off-by: Robert Richter <rrichter@amd.com>
---
Documentation/driver-api/cxl/conventions.rst | 1 +
.../driver-api/cxl/conventions/cxl-atl.rst | 304 ++++++++++++++++++
2 files changed, 305 insertions(+)
create mode 100644 Documentation/driver-api/cxl/conventions/cxl-atl.rst
diff --git a/Documentation/driver-api/cxl/conventions.rst b/Documentation/driver-api/cxl/conventions.rst
index 9267a697b2fe..0d2e07279ad9 100644
--- a/Documentation/driver-api/cxl/conventions.rst
+++ b/Documentation/driver-api/cxl/conventions.rst
@@ -14,4 +14,5 @@ same convention.
:caption: Contents
conventions/cxl-lmh.rst
+ conventions/cxl-atl.rst
conventions/template.rst
diff --git a/Documentation/driver-api/cxl/conventions/cxl-atl.rst b/Documentation/driver-api/cxl/conventions/cxl-atl.rst
new file mode 100644
index 000000000000..3a36a84743d0
--- /dev/null
+++ b/Documentation/driver-api/cxl/conventions/cxl-atl.rst
@@ -0,0 +1,304 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+ACPI PRM CXL Address Translation
+================================
+
+Document
+--------
+
+CXL Revision 3.2, Version 1.0
+
+License
+-------
+
+SPDX-License Identifier: CC-BY-4.0
+
+Creator/Contributors
+--------------------
+
+- Robert Richter, AMD et al.
+
+Summary of the Change
+---------------------
+
+The CXL Fixed Memory Window Structures (CFMWS) describe zero or more Host
+Physical Address (HPA) windows associated with one or more CXL Host Bridges.
+Each HPA range of a CXL Host Bridge is represented by a CFMWS entry. An HPA
+range may include addresses currently assigned to CXL.mem devices, or an OS may
+assign ranges from an address window to a device.
+
+Host-managed Device Memory is Device-attached memory that is mapped to system
+coherent address space and accessible to the Host using standard write-back
+semantics. The managed address range is configured in the CXL HDM Decoder
+registers of the device. An HDM Decoder in a device is responsible for
+converting HPA into DPA by stripping off specific address bits.
+
+CXL devices and CXL bridges use the same HPA space. It is common across all
+components that belong to the same host domain. The view of the address region
+must be consistent on the CXL.mem path between the Host and the Device.
+
+This is described in the *CXL 3.2 specification* (Table 1-1, 3.3.1,
+8.2.4.20, 9.13.1, 9.18.1.3). [#cxl-spec-3.2]_
+
+Depending on the interconnect architecture of the platform, components attached
+to a host may not share the same host physical address space. Those platforms
+need address translation to convert an HPA between the host and the attached
+component, such as a CXL device. The translation mechanism is host-specific and
+implementation dependent.
+
+For example, x86 AMD platforms use a Data Fabric that manages access to physical
+memory. Devices have their own memory space and can be configured to use
+'Normalized addresses' different from System Physical Addresses (SPA). Address
+translation is then needed. For details, see
+:doc:`x86 AMD Address Translation </admin-guide/RAS/address-translation>`.
+
+Those AMD platforms provide PRM [#prm-spec]_ handlers in firmware to perform
+various types of address translation, including for CXL endpoints. AMD Zen5
+systems implement the ACPI PRM CXL Address Translation firmware call. The ACPI
+PRM handler has a specific GUID to uniquely identify platforms with support for
+Normalized addressing. This is documented in the *ACPI v6.5 Porting Guide*
+(Address Translation - CXL DPA to System Physical Address). [#amd-ppr-58088]_
+
+When in Normalized address mode, HDM decoder address ranges must be configured
+and handled differently. Hardware addresses used in the HDM decoder
+configurations of an endpoint are not SPA and need to be translated from the
+address range of the endpoint to that of the CXL host bridge. This is especially
+important for finding an endpoint's associated CXL Host Bridge and HPA window
+described in the CFMWS. Additionally, the interleave decoding is done by the
+Data Fabric and the endpoint does not perform decoding when converting HPA to
+DPA. Instead, interleaving is switched off for the endpoint (1-way). Finally,
+address translation might also be needed to inspect the endpoint's hardware
+addresses, such as during profiling, tracing, or error handling.
+
+For example, with Normalized addressing the HDM decoders could look as follows::
+
+ -------------------------------
+ | Root Decoder (CFMWS) |
+ | SPA Range: 0x850000000 |
+ | Size: 0x8000000000 (512 GB) |
+ | Interleave Ways: 1 |
+ -------------------------------
+ |
+ v
+ -------------------------------
+ | Host Bridge Decoder (HDM) |
+ | SPA Range: 0x850000000 |
+ | Size: 0x8000000000 (512 GB) |
+ | Interleave Ways: 4 |
+ | Targets: endpoint5,8,11,13 |
+ | Granularity: 256 |
+ -------------------------------
+ |
+ -----------------------------+------------------------------
+ | | | |
+ v v v v
+ ------------------- ------------------- ------------------- -------------------
+ | endpoint5 | | endpoint8 | | endpoint11 | | endpoint13 |
+ | decoder5.0 | | decoder8.0 | | decoder11.0 | | decoder13.0 |
+ | PCIe: | | PCIe: | | PCIe: | | PCIe: |
+ | 0000:e2:00.0 | | 0000:e3:00.0 | | 0000:e4:00.0 | | 0000:e1:00.0 |
+ | DPA: | | DPA: | | DPA: | | DPA: |
+ | Start: 0x0 | | Start: 0x0 | | Start: 0x0 | | Start: 0x0 |
+ | Size: | | Size: | | Size: | | Size: |
+ | 0x2000000000 | | 0x2000000000 | | 0x2000000000 | | 0x2000000000 |
+ | (128 GB) | | (128 GB) | | (128 GB) | | (128 GB) |
+ | Interleaving: | | Interleaving: | | Interleaving: | | Interleaving: |
+ | Ways: 1 | | Ways: 1 | | Ways: 1 | | Ways: 1 |
+ | Gran: 256 | | Gran: 256 | | Gran: 256 | | Gran: 256 |
+ ------------------- ------------------- ------------------- -------------------
+ | | | |
+ v v v v
+ DPA DPA DPA DPA
+
+This shows the representation in sysfs:
+
+.. code-block:: none
+
+ /sys/bus/cxl/devices/endpoint5/decoder5.0/interleave_granularity:256
+ /sys/bus/cxl/devices/endpoint5/decoder5.0/interleave_ways:1
+ /sys/bus/cxl/devices/endpoint5/decoder5.0/size:0x2000000000
+ /sys/bus/cxl/devices/endpoint5/decoder5.0/start:0x0
+ /sys/bus/cxl/devices/endpoint8/decoder8.0/interleave_granularity:256
+ /sys/bus/cxl/devices/endpoint8/decoder8.0/interleave_ways:1
+ /sys/bus/cxl/devices/endpoint8/decoder8.0/size:0x2000000000
+ /sys/bus/cxl/devices/endpoint8/decoder8.0/start:0x0
+ /sys/bus/cxl/devices/endpoint11/decoder11.0/interleave_granularity:256
+ /sys/bus/cxl/devices/endpoint11/decoder11.0/interleave_ways:1
+ /sys/bus/cxl/devices/endpoint11/decoder11.0/size:0x2000000000
+ /sys/bus/cxl/devices/endpoint11/decoder11.0/start:0x0
+ /sys/bus/cxl/devices/endpoint13/decoder13.0/interleave_granularity:256
+ /sys/bus/cxl/devices/endpoint13/decoder13.0/interleave_ways:1
+ /sys/bus/cxl/devices/endpoint13/decoder13.0/size:0x2000000000
+ /sys/bus/cxl/devices/endpoint13/decoder13.0/start:0x0
+
+Note that the endpoint interleaving configurations use direct mapping (1-way).
+
+With PRM calls, the kernel can determine the following mappings:
+
+.. code-block:: none
+
+ cxl decoder5.0: address mapping found for 0000:e2:00.0 (hpa -> spa):
+ 0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
+ cxl decoder8.0: address mapping found for 0000:e3:00.0 (hpa -> spa):
+ 0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
+ cxl decoder11.0: address mapping found for 0000:e4:00.0 (hpa -> spa):
+ 0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
+ cxl decoder13.0: address mapping found for 0000:e1:00.0 (hpa -> spa):
+ 0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
+
+The corresponding CXL host bridge (HDM) decoders and root decoder (CFMWS) match
+the calculated endpoint mappings shown:
+
+.. code-block:: none
+
+ /sys/bus/cxl/devices/port1/decoder1.0/interleave_granularity:256
+ /sys/bus/cxl/devices/port1/decoder1.0/interleave_ways:4
+ /sys/bus/cxl/devices/port1/decoder1.0/size:0x8000000000
+ /sys/bus/cxl/devices/port1/decoder1.0/start:0x850000000
+ /sys/bus/cxl/devices/port1/decoder1.0/target_list:0,1,2,3
+ /sys/bus/cxl/devices/port1/decoder1.0/target_type:expander
+ /sys/bus/cxl/devices/root0/decoder0.0/interleave_granularity:256
+ /sys/bus/cxl/devices/root0/decoder0.0/interleave_ways:1
+ /sys/bus/cxl/devices/root0/decoder0.0/size:0x8000000000
+ /sys/bus/cxl/devices/root0/decoder0.0/start:0x850000000
+ /sys/bus/cxl/devices/root0/decoder0.0/target_list:7
+
+The following changes to the specification are needed:
+
+* Allow a CXL device to be in an HPA space other than the host's address space.
+
+* Allow the platform to use implementation-specific address translation when
+ crossing memory domains on the CXL.mem path between the host and the device.
+
+* Define a PRM handler method for converting device addresses to SPAs.
+
+* Specify that the platform shall provide the PRM handler method to the
+ Operating System to detect Normalized addressing and for determining Endpoint
+ SPA ranges and interleaving configurations.
+
+* Add reference to:
+
+ | Platform Runtime Mechanism Specification, Version 1.1 – November 2020
+ | https://uefi.org/sites/default/files/resources/PRM_Platform_Runtime_Mechanism_1_1_release_candidate.pdf
+
+Benefits of the Change
+----------------------
+
+Without the change, the Operating System may be unable to determine the memory
+region and Root Decoder for an Endpoint and its corresponding HDM decoder.
+Region creation would fail. Platforms with a different interconnect architecture
+would fail to set up and use CXL.
+
+References
+----------
+
+.. [#cxl-spec-3.2] Compute Express Link Specification, Revision 3.2, Version 1.0,
+ https://www.computeexpresslink.org/
+
+.. [#amd-ppr-58088] AMD Family 1Ah Models 00h–0Fh and Models 10h–1Fh,
+ ACPI v6.5 Porting Guide, Publication # 58088,
+ https://www.amd.com/en/search/documentation/hub.html
+
+.. [#prm-spec] Platform Runtime Mechanism, Version: 1.1,
+ https://uefi.org/sites/default/files/resources/PRM_Platform_Runtime_Mechanism_1_1_release_candidate.pdf
+
+Detailed Description of the Change
+----------------------------------
+
+The following describes the necessary changes to the *CXL 3.2 specification*
+[#cxl-spec-3.2]_:
+
+Add the following reference to the table:
+
+Table 1-2. Reference Documents
+
++----------------------------+-------------------+---------------------------+
+| Document | Chapter Reference | Document No./Location |
++============================+===================+===========================+
+| Platform Runtime Mechanism | Chapter 8, 9 | https://www.uefi.org/acpi |
+| Version: 1.1 | | |
++----------------------------+-------------------+---------------------------+
+
+Add the following paragraphs to the end of the section:
+
+**8.2.4.20 CXL HDM Decoder Capability Structure**
+
+"A device may use an HPA space that is not common to other components of the
+host domain. The platform is responsible for address translation when crossing
+HPA spaces. The Operating System must determine the interleaving configuration
+and perform address translation to the HPA ranges of the HDM decoders as needed.
+The translation mechanism is host-specific and implementation dependent.
+
+The platform indicates support of independent HPA spaces and the need for
+address translation by providing a Platform Runtime Mechanism (PRM) handler. The
+OS shall use that handler to perform the necessary translations from the DPA
+space to the HPA space. The handler is defined in Section 9.18.4 *PRM Handler
+for CXL DPA to System Physical Address Translation*."
+
+Add the following section and sub-section including tables:
+
+**9.18.4 PRM Handler for CXL DPA to System Physical Address Translation**
+
+"A platform may be configured to use 'Normalized addresses'. Host physical
+address (HPA) spaces are component-specific and differ from system physical
+addresses (SPAs). The endpoint has its own physical address space. All requests
+presented to the device already use Device Physical Addresses (DPAs). The CXL
+endpoint decoders have interleaving disabled (1-way interleaving) and the device
+does not perform HPA decoding to determine a DPA.
+
+The platform provides a PRM handler for CXL DPA to System Physical Address
+Translation. The PRM handler translates a Device Physical Address (DPA) to a
+System Physical Address (SPA) for a specified CXL endpoint. In the address space
+of the host, SPA and HPA are equivalent, and the OS shall use this handler to
+determine the HPA that corresponds to a device address, for example when
+configuring HDM decoders on platforms with Normalized addressing. The GUID and
+the parameter buffer format of the handler are specified in section 9.18.4.1. If
+the OS identifies the PRM handler, the platform supports Normalized addressing
+and the OS must perform DPA address translation as needed."
+
+**9.18.4.1 PRM Handler Invocation**
+
+"The OS calls the PRM handler for CXL DPA to System Physical Address Translation
+using the direct invocation mechanism. Details of calling a PRM handler are
+described in the Platform Runtime Mechanism (PRM) specification.
+
+The PRM handler is identified by the following GUID:
+
+ EE41B397-25D4-452C-AD54-48C6E3480B94
+
+The caller allocates and prepares a Parameter Buffer, then passes the PRM
+handler GUID and a pointer to the Parameter Buffer to invoke the handler. The
+Parameter Buffer is described in Table 9-32."
+
+**Table 9-32. PRM Parameter Buffer used for CXL DPA to System Physical Address Translation**
+
++-------------+-----------+------------------------------------------------------------------------+
+| Byte Offset | Length in | Description |
+| | Bytes | |
++=============+===========+========================================================================+
+| 00h | 8 | **CXL Device Physical Address (DPA)**: CXL DPA (e.g., from |
+| | | CXL Component Event Log) |
++-------------+-----------+------------------------------------------------------------------------+
+| 08h | 4 | **CXL Endpoint SBDF**: |
+| | | |
+| | | - Byte 3 - PCIe Segment |
+| | | - Byte 2 - Bus Number |
+| | | - Byte 1: |
+| | | - Device Number Bits[7:3] |
+| | | - Function Number Bits[2:0] |
+| | | - Byte 0 - RESERVED (MBZ) |
+| | | |
++-------------+-----------+------------------------------------------------------------------------+
+| 0Ch | 8 | **Output Buffer**: Virtual Address Pointer to the buffer, |
+| | | as defined in Table 9-33. |
++-------------+-----------+------------------------------------------------------------------------+
+
+**Table 9-33. PRM Output Buffer used for CXL DPA to System Physical Address Translation**
+
++-------------+-----------+------------------------------------------------------------------------+
+| Byte Offset | Length in | Description |
+| | Bytes | |
++=============+===========+========================================================================+
+| 00h | 8 | **System Physical Address (SPA)**: The SPA converted |
+| | | from the CXL DPA. |
++-------------+-----------+------------------------------------------------------------------------+
--
2.47.3
^ permalink raw reply related [flat|nested] 5+ messages in thread