[PATCH v1 1/2] cxl, doc: Moving conventions in separate files

[PATCH v1 1/2] cxl, doc: Moving conventions in separate files

[Robert Richter]

Moving conventions in separate files.

Signed-off-by: Robert Richter <rrichter@amd.com>
---
 Documentation/driver-api/cxl/conventions.rst  | 183 +-----------------
 .../driver-api/cxl/conventions/cxl-lmh.rst    | 136 +++++++++++++
 .../driver-api/cxl/conventions/template.rst   |  39 ++++
 3 files changed, 181 insertions(+), 177 deletions(-)
 create mode 100644 Documentation/driver-api/cxl/conventions/cxl-lmh.rst
 create mode 100644 Documentation/driver-api/cxl/conventions/template.rst

diff --git a/Documentation/driver-api/cxl/conventions.rst b/Documentation/driver-api/cxl/conventions.rst
index e37336d7b116..53f31a229c8d 100644
--- a/Documentation/driver-api/cxl/conventions.rst
+++ b/Documentation/driver-api/cxl/conventions.rst
@@ -1,182 +1,11 @@
 .. SPDX-License-Identifier: GPL-2.0
-.. include:: <isonum.txt>
 
-=======================================
 Compute Express Link: Linux Conventions
-=======================================
+#######################################
 
-There exists shipping platforms that bend or break CXL specification
-expectations. Record the details and the rationale for those deviations.
-Borrow the ACPI Code First template format to capture the assumptions
-and tradeoffs such that multiple platform implementations can follow the
-same convention.
+.. toctree::
+   :maxdepth: 1
+   :caption: Contents
 
-<(template) Title>
-==================
-
-Document
---------
-CXL Revision <rev>, Version <ver>
-
-License
--------
-SPDX-License Identifier: CC-BY-4.0
-
-Creator/Contributors
---------------------
-
-Summary of the Change
----------------------
-
-<Detail the conflict with the specification and where available the
-assumptions and tradeoffs taken by the hardware platform.>
-
-
-Benefits of the Change
-----------------------
-
-<Detail what happens if platforms and Linux do not adopt this
-convention.>
-
-References
-----------
-
-Detailed Description of the Change
-----------------------------------
-
-<Propose spec language that corrects the conflict.>
-
-
-Resolve conflict between CFMWS, Platform Memory Holes, and Endpoint Decoders
-============================================================================
-
-Document
---------
-
-CXL Revision 3.2, Version 1.0
-
-License
--------
-
-SPDX-License Identifier: CC-BY-4.0
-
-Creator/Contributors
---------------------
-
-- Fabio M. De Francesco, Intel
-- Dan J. Williams, Intel
-- Mahesh Natu, Intel
-
-Summary of the Change
----------------------
-
-According to the current Compute Express Link (CXL) Specifications (Revision
-3.2, Version 1.0), the CXL Fixed Memory Window Structure (CFMWS) describes zero
-or more Host Physical Address (HPA) windows associated with each CXL Host
-Bridge. Each window represents a contiguous HPA range that may be interleaved
-across one or more targets, including CXL Host Bridges. Each window has a set
-of restrictions that govern its usage. It is the Operating System-directed
-configuration and Power Management (OSPM) responsibility to utilize each window
-for the specified use.
-
-Table 9-22 of the current CXL Specifications states that the Window Size field
-contains the total number of consecutive bytes of HPA this window describes.
-This value must be a multiple of the Number of Interleave Ways (NIW) * 256 MB.
-
-Platform Firmware (BIOS) might reserve physical addresses below 4 GB where a
-memory gap such as the Low Memory Hole for PCIe MMIO may exist. In such cases,
-the CFMWS Range Size may not adhere to the NIW * 256 MB rule.
-
-The HPA represents the actual physical memory address space that the CXL devices
-can decode and respond to, while the System Physical Address (SPA), a related
-but distinct concept, represents the system-visible address space that users can
-direct transaction to and so it excludes reserved regions.
-
-BIOS publishes CFMWS to communicate the active SPA ranges that, on platforms
-with LMH's, map to a strict subset of the HPA. The SPA range trims out the hole,
-resulting in lost capacity in the Endpoints with no SPA to map to that part of
-the HPA range that intersects the hole.
-
-E.g, an x86 platform with two CFMWS and an LMH starting at 2 GB:
-
- +--------+------------+-------------------+------------------+-------------------+------+
- | Window | CFMWS Base |    CFMWS Size     | HDM Decoder Base |  HDM Decoder Size | Ways |
- +========+============+===================+==================+===================+======+
- |   0    |   0 GB     |       2 GB        |      0 GB        |       3 GB        |  12  |
- +--------+------------+-------------------+------------------+-------------------+------+
- |   1    |   4 GB     | NIW*256MB Aligned |      4 GB        | NIW*256MB Aligned |  12  |
- +--------+------------+-------------------+------------------+-------------------+------+
-
-HDM decoder base and HDM decoder size represent all the 12 Endpoint Decoders of
-a 12 ways region and all the intermediate Switch Decoders. They are configured
-by the BIOS according to the NIW * 256MB rule, resulting in a HPA range size of
-3GB. Instead, the CFMWS Base and CFMWS Size are used to configure the Root
-Decoder HPA range that results smaller (2GB) than that of the Switch and
-Endpoint Decoders in the hierarchy (3GB).
-
-This creates 2 issues which lead to a failure to construct a region:
-
-1) A mismatch in region size between root and any HDM decoder. The root decoders
-   will always be smaller due to the trim.
-
-2) The trim causes the root decoder to violate the (NIW * 256MB) rule.
-
-This change allows a region with a base address of 0GB to bypass these checks to
-allow for region creation with the trimmed root decoder address range.
-
-This change does not allow for any other arbitrary region to violate these
-checks - it is intended exclusively to enable x86 platforms which map CXL memory
-under 4GB.
-
-Despite the HDM decoders covering the PCIE hole HPA region, it is expected that
-the platform will never route address accesses to the CXL complex because the
-root decoder only covers the trimmed region (which excludes this). This is
-outside the ability of Linux to enforce.
-
-On the example platform, only the first 2GB will be potentially usable, but
-Linux, aiming to adhere to the current specifications, fails to construct
-Regions and attach Endpoint and intermediate Switch Decoders to them.
-
-There are several points of failure that due to the expectation that the Root
-Decoder HPA size, that is equal to the CFMWS from which it is configured, has
-to be greater or equal to the matching Switch and Endpoint HDM Decoders.
-
-In order to succeed with construction and attachment, Linux must construct a
-Region with Root Decoder HPA range size, and then attach to that all the
-intermediate Switch Decoders and Endpoint Decoders that belong to the hierarchy
-regardless of their range sizes.
-
-Benefits of the Change
-----------------------
-
-Without the change, the OSPM wouldn't match intermediate Switch and Endpoint
-Decoders with Root Decoders configured with CFMWS HPA sizes that don't align
-with the NIW * 256MB constraint, and so it leads to lost memdev capacity.
-
-This change allows the OSPM to construct Regions and attach intermediate Switch
-and Endpoint Decoders to them, so that the addressable part of the memory
-devices total capacity is made available to the users.
-
-References
-----------
-
-Compute Express Link Specification Revision 3.2, Version 1.0
-<https://www.computeexpresslink.org/>
-
-Detailed Description of the Change
-----------------------------------
-
-The description of the Window Size field in table 9-22 needs to account for
-platforms with Low Memory Holes, where SPA ranges might be subsets of the
-endpoints HPA. Therefore, it has to be changed to the following:
-
-"The total number of consecutive bytes of HPA this window represents. This value
-shall be a multiple of NIW * 256 MB.
-
-On platforms that reserve physical addresses below 4 GB, such as the Low Memory
-Hole for PCIe MMIO on x86, an instance of CFMWS whose Base HPA range is 0 might
-have a size that doesn't align with the NIW * 256 MB constraint.
-
-Note that the matching intermediate Switch Decoders and the Endpoint Decoders
-HPA range sizes must still align to the above-mentioned rule, but the memory
-capacity that exceeds the CFMWS window size won't be accessible.".
+   conventions/cxl-lmh.rst
+   conventions/template.rst
diff --git a/Documentation/driver-api/cxl/conventions/cxl-lmh.rst b/Documentation/driver-api/cxl/conventions/cxl-lmh.rst
new file mode 100644
index 000000000000..3049c1f58f7d
--- /dev/null
+++ b/Documentation/driver-api/cxl/conventions/cxl-lmh.rst
@@ -0,0 +1,136 @@
+.. SPDX-License-Identifier: GPL-2.0
+.. include:: <isonum.txt>
+
+Resolve conflict between CFMWS, Platform Memory Holes, and Endpoint Decoders
+============================================================================
+
+Document
+--------
+
+CXL Revision 3.2, Version 1.0
+
+License
+-------
+
+SPDX-License Identifier: CC-BY-4.0
+
+Creator/Contributors
+--------------------
+
+- Fabio M. De Francesco, Intel
+- Dan J. Williams, Intel
+- Mahesh Natu, Intel
+
+Summary of the Change
+---------------------
+
+According to the current Compute Express Link (CXL) Specifications (Revision
+3.2, Version 1.0), the CXL Fixed Memory Window Structure (CFMWS) describes zero
+or more Host Physical Address (HPA) windows associated with each CXL Host
+Bridge. Each window represents a contiguous HPA range that may be interleaved
+across one or more targets, including CXL Host Bridges. Each window has a set
+of restrictions that govern its usage. It is the Operating System-directed
+configuration and Power Management (OSPM) responsibility to utilize each window
+for the specified use.
+
+Table 9-22 of the current CXL Specifications states that the Window Size field
+contains the total number of consecutive bytes of HPA this window describes.
+This value must be a multiple of the Number of Interleave Ways (NIW) * 256 MB.
+
+Platform Firmware (BIOS) might reserve physical addresses below 4 GB where a
+memory gap such as the Low Memory Hole for PCIe MMIO may exist. In such cases,
+the CFMWS Range Size may not adhere to the NIW * 256 MB rule.
+
+The HPA represents the actual physical memory address space that the CXL devices
+can decode and respond to, while the System Physical Address (SPA), a related
+but distinct concept, represents the system-visible address space that users can
+direct transaction to and so it excludes reserved regions.
+
+BIOS publishes CFMWS to communicate the active SPA ranges that, on platforms
+with LMH's, map to a strict subset of the HPA. The SPA range trims out the hole,
+resulting in lost capacity in the Endpoints with no SPA to map to that part of
+the HPA range that intersects the hole.
+
+E.g, an x86 platform with two CFMWS and an LMH starting at 2 GB:
+
+ +--------+------------+-------------------+------------------+-------------------+------+
+ | Window | CFMWS Base |    CFMWS Size     | HDM Decoder Base |  HDM Decoder Size | Ways |
+ +========+============+===================+==================+===================+======+
+ |   0    |   0 GB     |       2 GB        |      0 GB        |       3 GB        |  12  |
+ +--------+------------+-------------------+------------------+-------------------+------+
+ |   1    |   4 GB     | NIW*256MB Aligned |      4 GB        | NIW*256MB Aligned |  12  |
+ +--------+------------+-------------------+------------------+-------------------+------+
+
+HDM decoder base and HDM decoder size represent all the 12 Endpoint Decoders of
+a 12 ways region and all the intermediate Switch Decoders. They are configured
+by the BIOS according to the NIW * 256MB rule, resulting in a HPA range size of
+3GB. Instead, the CFMWS Base and CFMWS Size are used to configure the Root
+Decoder HPA range that results smaller (2GB) than that of the Switch and
+Endpoint Decoders in the hierarchy (3GB).
+
+This creates 2 issues which lead to a failure to construct a region:
+
+1) A mismatch in region size between root and any HDM decoder. The root decoders
+   will always be smaller due to the trim.
+
+2) The trim causes the root decoder to violate the (NIW * 256MB) rule.
+
+This change allows a region with a base address of 0GB to bypass these checks to
+allow for region creation with the trimmed root decoder address range.
+
+This change does not allow for any other arbitrary region to violate these
+checks - it is intended exclusively to enable x86 platforms which map CXL memory
+under 4GB.
+
+Despite the HDM decoders covering the PCIE hole HPA region, it is expected that
+the platform will never route address accesses to the CXL complex because the
+root decoder only covers the trimmed region (which excludes this). This is
+outside the ability of Linux to enforce.
+
+On the example platform, only the first 2GB will be potentially usable, but
+Linux, aiming to adhere to the current specifications, fails to construct
+Regions and attach Endpoint and intermediate Switch Decoders to them.
+
+There are several points of failure that due to the expectation that the Root
+Decoder HPA size, that is equal to the CFMWS from which it is configured, has
+to be greater or equal to the matching Switch and Endpoint HDM Decoders.
+
+In order to succeed with construction and attachment, Linux must construct a
+Region with Root Decoder HPA range size, and then attach to that all the
+intermediate Switch Decoders and Endpoint Decoders that belong to the hierarchy
+regardless of their range sizes.
+
+Benefits of the Change
+----------------------
+
+Without the change, the OSPM wouldn't match intermediate Switch and Endpoint
+Decoders with Root Decoders configured with CFMWS HPA sizes that don't align
+with the NIW * 256MB constraint, and so it leads to lost memdev capacity.
+
+This change allows the OSPM to construct Regions and attach intermediate Switch
+and Endpoint Decoders to them, so that the addressable part of the memory
+devices total capacity is made available to the users.
+
+References
+----------
+
+Compute Express Link Specification Revision 3.2, Version 1.0
+<https://www.computeexpresslink.org/>
+
+Detailed Description of the Change
+----------------------------------
+
+The description of the Window Size field in table 9-22 needs to account for
+platforms with Low Memory Holes, where SPA ranges might be subsets of the
+endpoints HPA. Therefore, it has to be changed to the following:
+
+"The total number of consecutive bytes of HPA this window represents. This value
+shall be a multiple of NIW * 256 MB.
+
+On platforms that reserve physical addresses below 4 GB, such as the Low Memory
+Hole for PCIe MMIO on x86, an instance of CFMWS whose Base HPA range is 0 might
+have a size that doesn't align with the NIW * 256 MB constraint.
+
+Note that the matching intermediate Switch Decoders and the Endpoint Decoders
+HPA range sizes must still align to the above-mentioned rule, but the memory
+capacity that exceeds the CFMWS window size won't be accessible.".
diff --git a/Documentation/driver-api/cxl/conventions/template.rst b/Documentation/driver-api/cxl/conventions/template.rst
new file mode 100644
index 000000000000..ced7493f4bb2
--- /dev/null
+++ b/Documentation/driver-api/cxl/conventions/template.rst
@@ -0,0 +1,39 @@
+.. SPDX-License-Identifier: GPL-2.0
+.. include:: <isonum.txt>
+
+.. :: Template Title here:
+
+Template File
+=============
+
+Document
+--------
+CXL Revision <rev>, Version <ver>
+
+License
+-------
+SPDX-License Identifier: CC-BY-4.0
+
+Creator/Contributors
+--------------------
+
+Summary of the Change
+---------------------
+
+<Detail the conflict with the specification and where available the
+assumptions and tradeoffs taken by the hardware platform.>
+
+
+Benefits of the Change
+----------------------
+
+<Detail what happens if platforms and Linux do not adopt this
+convention.>
+
+References
+----------
+
+Detailed Description of the Change
+----------------------------------
+
+<Propose spec language that corrects the conflict.>
-- 
2.47.3

[PATCH v1 2/2] Documentation/driver-api/cxl: ACPI PRM Address Translation Support and AMD Zen5 enablement

[Robert Richter]

This adds a convetion document for the following patch series:

 [v7,00/11] cxl: ACPI PRM Address Translation Support and AMD Zen5 enablement
 https://patchwork.kernel.org/project/cxl/cover/20251114213931.30754-1-rrichter@amd.com/

Signed-off-by: Robert Richter <rrichter@amd.com>
---
 Documentation/driver-api/cxl/conventions.rst  |   1 +
 .../driver-api/cxl/conventions/cxl-atl.rst    | 175 ++++++++++++++++++
 2 files changed, 176 insertions(+)
 create mode 100644 Documentation/driver-api/cxl/conventions/cxl-atl.rst

diff --git a/Documentation/driver-api/cxl/conventions.rst b/Documentation/driver-api/cxl/conventions.rst
index 53f31a229c8d..cf427afac58b 100644
--- a/Documentation/driver-api/cxl/conventions.rst
+++ b/Documentation/driver-api/cxl/conventions.rst
@@ -8,4 +8,5 @@ Compute Express Link: Linux Conventions
    :caption: Contents
 
    conventions/cxl-lmh.rst
+   conventions/cxl-atl.rst
    conventions/template.rst
diff --git a/Documentation/driver-api/cxl/conventions/cxl-atl.rst b/Documentation/driver-api/cxl/conventions/cxl-atl.rst
new file mode 100644
index 000000000000..9c14856e1782
--- /dev/null
+++ b/Documentation/driver-api/cxl/conventions/cxl-atl.rst
@@ -0,0 +1,175 @@
+.. SPDX-License-Identifier: GPL-2.0
+.. include:: <isonum.txt>
+
+ACPI PRM CXL Address Translation
+================================
+
+Document
+--------
+
+CXL Revision 3.2, Version 1.0
+
+License
+-------
+
+SPDX-License Identifier: CC-BY-4.0
+
+Creator/Contributors
+--------------------
+
+- Robert Richter, AMD
+
+Summary of the Change
+---------------------
+
+The CXL Fixed Memory Window Structure (CFMWS) describes zero or more
+Host Physical Address (HPA) windows that are associated with each CXL
+Host Bridge. The HPA ranges of an CFMWS may include addresses that are
+currently assigned to CXL.mem devices, or an OS may assign ranges from
+an address window to a device.
+
+Host-managed Device Memory is Device-attached memory that is mapped to
+system coherent address space and accessible to the Host using
+standard write-back semantics. The managed address range is configured
+in the CXL HDM Decoder registers of the device. An HDM Decoder in a
+device is responsible for converting HPA into DPA by stripping off
+specific address bits.
+
+CXL devices and CXL bridges use the same HPA space. It is common
+across all components that belong to the same host domain. The view of
+the address region must be consistent on the CXL.mem path between the
+Host and the Device.
+
+This is described in the current CXL specification (Table 1-1, 3.3.1,
+8.2.4.20, 9.13.1, 9.18.1.3). [#cxl-spec-3.2]_
+
+Depending on the interconnect architecture of the platform, components
+attached to a host may not share the same host physical address space.
+Those platforms need address translation to convert an HPA between the
+host and the attached component, such as a CXL device. The translation
+mechanism is host-specific and implementation dependent.
+
+E.g., x86 AMD platforms use a Data Fabric that manages access to
+physical memory. Devices have an own memory space and can be
+configured to use 'Normalized addresses' different to System Physical
+Addresses (SPA). Address translation is needed then. Details are
+described also under x86 AMD
+Documentation/admin-guide/RAS/address-translation.rst.
+
+Those AMD platforms provide PRM handlers in firmware to perform
+various types of address translation, including for CXL endpoints.
+AMD Zen5 systems implement the ACPI PRM CXL Address Translation
+firmware call. The ACPI PRM handler has a specific GUID to uniquely
+identify platforms with support of Normalized addressing. This is
+documented in the ACPI v6.5 Porting Guide, Address Translation - CXL
+DPA to System Physical Address.  [#amd-ppr-58088]_
+
+When in Normalized address mode, HDM decoder address ranges must be
+configured and handled differently. Hardware addresses used in the HDM
+decoder configurations of an endpoint are not SPA and need to be
+translated from the endpoint's to its CXL host bridge's address range.
+This is esp. important to find an endpoint's associated CXL Host
+Bridge and HPA window described in the CFMWS. Also, the interleave
+decoding is done by the Data Fabric and the endpoint does not perform
+decoding when converting HPA to DPA. Instead, interleaving is switched
+off for the endpoint (1 way). Finally, address translation might also
+be needed to inspect the Endpoint's hardware addresses, such as during
+profiling, tracing or error handling.
+
+For example, with Normalized addressing the HDM decoders could look as
+following:
+
+.. code-block:: none
+
+ /sys/bus/cxl/devices/endpoint5/decoder5.0/interleave_granularity:256
+ /sys/bus/cxl/devices/endpoint5/decoder5.0/interleave_ways:1
+ /sys/bus/cxl/devices/endpoint5/decoder5.0/size:0x2000000000
+ /sys/bus/cxl/devices/endpoint5/decoder5.0/start:0x0
+ /sys/bus/cxl/devices/endpoint8/decoder8.0/interleave_granularity:256
+ /sys/bus/cxl/devices/endpoint8/decoder8.0/interleave_ways:1
+ /sys/bus/cxl/devices/endpoint8/decoder8.0/size:0x2000000000
+ /sys/bus/cxl/devices/endpoint8/decoder8.0/start:0x0
+ /sys/bus/cxl/devices/endpoint11/decoder11.0/interleave_granularity:256
+ /sys/bus/cxl/devices/endpoint11/decoder11.0/interleave_ways:1
+ /sys/bus/cxl/devices/endpoint11/decoder11.0/size:0x2000000000
+ /sys/bus/cxl/devices/endpoint11/decoder11.0/start:0x0
+ /sys/bus/cxl/devices/endpoint13/decoder13.0/interleave_granularity:256
+ /sys/bus/cxl/devices/endpoint13/decoder13.0/interleave_ways:1
+ /sys/bus/cxl/devices/endpoint13/decoder13.0/size:0x2000000000
+ /sys/bus/cxl/devices/endpoint13/decoder13.0/start:0x0
+
+Note the endpoint interleaving configurations with a direct mapping
+(1-way).
+
+With PRM calls, the kernel can determine the following mappings:
+
+.. code-block:: none
+
+ cxl decoder5.0: address mapping found for 0000:e2:00.0 (hpa -> spa):
+   0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
+ cxl decoder8.0: address mapping found for 0000:e3:00.0 (hpa -> spa):
+   0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
+ cxl decoder11.0: address mapping found for 0000:e4:00.0 (hpa -> spa):
+   0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
+ cxl decoder13.0: address mapping found for 0000:e1:00.0 (hpa -> spa):
+   0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
+
+The corresponding CXL host bridge (HDM) decoders and root decoder
+(CFMWS) show and match with the calculated endpoint mappings:
+
+.. code-block:: none
+
+ /sys/bus/cxl/devices/port1/decoder1.0/interleave_granularity:256
+ /sys/bus/cxl/devices/port1/decoder1.0/interleave_ways:4
+ /sys/bus/cxl/devices/port1/decoder1.0/size:0x8000000000
+ /sys/bus/cxl/devices/port1/decoder1.0/start:0x850000000
+ /sys/bus/cxl/devices/port1/decoder1.0/target_list:0,1,2,3
+ /sys/bus/cxl/devices/port1/decoder1.0/target_type:expander
+ /sys/bus/cxl/devices/root0/decoder0.0/interleave_granularity:256
+ /sys/bus/cxl/devices/root0/decoder0.0/interleave_ways:1
+ /sys/bus/cxl/devices/root0/decoder0.0/size:0x8000000000
+ /sys/bus/cxl/devices/root0/decoder0.0/start:0x850000000
+ /sys/bus/cxl/devices/root0/decoder0.0/target_list:7
+
+The following changes of the specification are needed:
+
+* Allow a CXL device to be in a different HPA space other than the
+  host's space.
+
+* The platform can use implementation-specific address translation
+  when crossing memory domains on the CXL.mem path between the Host
+  and the Device.
+
+* The kernel (OSPM) determines Endpoint SPA range and interleaving
+  configuration using platform specific address translation methods.
+
+Benefits of the Change
+----------------------
+
+Without the change, the OSPM may not determine the memory region and
+Root Decoder of an Endpoint and its corresponding HDM decoder. Region
+creation would fail. Platforms with a different interconnect
+architecture would fail to setup and use CXL.
+
+References
+----------
+
+.. [#cxl-spec-3.2] Compute Express Link Specification, Revision 3.2, Version 1.0,
+   https://www.computeexpresslink.org/
+
+.. [#amd-ppr-58088] AMD Family 1Ah Models 00h–0Fh and Models 10h–1Fh,
+   ACPI v6.5 Porting Guide, Publication # 58088,
+   https://www.amd.com/en/search/documentation/hub.html
+
+Detailed Description of the Change
+----------------------------------
+
+Add the following paragraph in 8.2.4.20 CXL HDM Decoder Capability
+Structure of the specification [#cxl-spec-3.2]_ to the end:
+
+"A device may use a different HPA space that is not common to other
+components of the host domain. The platform is responsible for address
+translation when crossing HPA spaces. The OSPM must determine the
+interleaving configuration and perform address translation to HPA
+ranges of the HDM decoders as needed. The translation mechanism is
+host-specific and implementation dependent."
-- 
2.47.3

Re: [PATCH v1 1/2] cxl, doc: Moving conventions in separate files

[Jonathan Corbet]

Robert Richter <rrichter@amd.com> writes:

A nit, but it's something that bugs me...

[...]

> new file mode 100644
> index 000000000000..3049c1f58f7d
> --- /dev/null
> +++ b/Documentation/driver-api/cxl/conventions/cxl-lmh.rst
> @@ -0,0 +1,136 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +.. include:: <isonum.txt>

This include has been cargo-culted around the docs...the only real use
of it is to write |copy| rather than ©, but these docs don't even do
that.  It can be taken out.

Thanks,

jon

Re: [PATCH v1 2/2] Documentation/driver-api/cxl: ACPI PRM Address Translation Support and AMD Zen5 enablement

[Gregory Price]

On Fri, Nov 21, 2025 at 01:16:51PM +0100, Robert Richter wrote:
> This adds a convetion document for the following patch series:
> 
>  [v7,00/11] cxl: ACPI PRM Address Translation Support and AMD Zen5 enablement
>  https://patchwork.kernel.org/project/cxl/cover/20251114213931.30754-1-rrichter@amd.com/
> 
> Signed-off-by: Robert Richter <rrichter@amd.com>

With a once-through I don't see any obvious nits.  lgtm

Reviewed-by: Gregory Price <gourry@gourry.net>

Re: [PATCH v1 2/2] Documentation/driver-api/cxl: ACPI PRM Address Translation Support and AMD Zen5 enablement

[Randy Dunlap]

On 11/21/25 4:16 AM, Robert Richter wrote:
> This adds a convetion document for the following patch series:
> 

              convention

>  [v7,00/11] cxl: ACPI PRM Address Translation Support and AMD Zen5 enablement
>  https://patchwork.kernel.org/project/cxl/cover/20251114213931.30754-1-rrichter@amd.com/
> 
> Signed-off-by: Robert Richter <rrichter@amd.com>
> ---
>  Documentation/driver-api/cxl/conventions.rst  |   1 +
>  .../driver-api/cxl/conventions/cxl-atl.rst    | 175 ++++++++++++++++++
>  2 files changed, 176 insertions(+)
>  create mode 100644 Documentation/driver-api/cxl/conventions/cxl-atl.rst
> 


> diff --git a/Documentation/driver-api/cxl/conventions/cxl-atl.rst b/Documentation/driver-api/cxl/conventions/cxl-atl.rst
> new file mode 100644
> index 000000000000..9c14856e1782
> --- /dev/null
> +++ b/Documentation/driver-api/cxl/conventions/cxl-atl.rst
> @@ -0,0 +1,175 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +.. include:: <isonum.txt>
> +
> +ACPI PRM CXL Address Translation
> +================================
> +
> +Document
> +--------
> +
> +CXL Revision 3.2, Version 1.0
> +
> +License
> +-------
> +
> +SPDX-License Identifier: CC-BY-4.0
> +
> +Creator/Contributors
> +--------------------
> +
> +- Robert Richter, AMD
> +
> +Summary of the Change
> +---------------------
> +
> +The CXL Fixed Memory Window Structure (CFMWS) describes zero or more
> +Host Physical Address (HPA) windows that are associated with each CXL
> +Host Bridge. The HPA ranges of an CFMWS may include addresses that are

                               of a

> +currently assigned to CXL.mem devices, or an OS may assign ranges from
> +an address window to a device.
> +
> +Host-managed Device Memory is Device-attached memory that is mapped to
> +system coherent address space and accessible to the Host using
> +standard write-back semantics. The managed address range is configured
> +in the CXL HDM Decoder registers of the device. An HDM Decoder in a
> +device is responsible for converting HPA into DPA by stripping off
> +specific address bits.
> +
> +CXL devices and CXL bridges use the same HPA space. It is common
> +across all components that belong to the same host domain. The view of
> +the address region must be consistent on the CXL.mem path between the
> +Host and the Device.
> +
> +This is described in the current CXL specification (Table 1-1, 3.3.1,
> +8.2.4.20, 9.13.1, 9.18.1.3). [#cxl-spec-3.2]_
> +
> +Depending on the interconnect architecture of the platform, components
> +attached to a host may not share the same host physical address space.
> +Those platforms need address translation to convert an HPA between the
> +host and the attached component, such as a CXL device. The translation
> +mechanism is host-specific and implementation dependent.
> +
> +E.g., x86 AMD platforms use a Data Fabric that manages access to
> +physical memory. Devices have an own memory space and can be

                            have their own
?

> +configured to use 'Normalized addresses' different to System Physical
> +Addresses (SPA). Address translation is needed then. Details are
> +described also under x86 AMD
> +Documentation/admin-guide/RAS/address-translation.rst.
> +
> +Those AMD platforms provide PRM handlers in firmware to perform
> +various types of address translation, including for CXL endpoints.
> +AMD Zen5 systems implement the ACPI PRM CXL Address Translation
> +firmware call. The ACPI PRM handler has a specific GUID to uniquely
> +identify platforms with support of Normalized addressing. This is
> +documented in the ACPI v6.5 Porting Guide, Address Translation - CXL
> +DPA to System Physical Address.  [#amd-ppr-58088]_
> +
> +When in Normalized address mode, HDM decoder address ranges must be
> +configured and handled differently. Hardware addresses used in the HDM
> +decoder configurations of an endpoint are not SPA and need to be
> +translated from the endpoint's to its CXL host bridge's address range.
> +This is esp. important to find an endpoint's associated CXL Host

           especially

> +Bridge and HPA window described in the CFMWS. Also, the interleave
> +decoding is done by the Data Fabric and the endpoint does not perform
> +decoding when converting HPA to DPA. Instead, interleaving is switched
> +off for the endpoint (1 way). Finally, address translation might also
> +be needed to inspect the Endpoint's hardware addresses, such as during
> +profiling, tracing or error handling.
> +
> +For example, with Normalized addressing the HDM decoders could look as
> +following:
> +
> +.. code-block:: none
> +
> + /sys/bus/cxl/devices/endpoint5/decoder5.0/interleave_granularity:256
> + /sys/bus/cxl/devices/endpoint5/decoder5.0/interleave_ways:1
> + /sys/bus/cxl/devices/endpoint5/decoder5.0/size:0x2000000000
> + /sys/bus/cxl/devices/endpoint5/decoder5.0/start:0x0
> + /sys/bus/cxl/devices/endpoint8/decoder8.0/interleave_granularity:256
> + /sys/bus/cxl/devices/endpoint8/decoder8.0/interleave_ways:1
> + /sys/bus/cxl/devices/endpoint8/decoder8.0/size:0x2000000000
> + /sys/bus/cxl/devices/endpoint8/decoder8.0/start:0x0
> + /sys/bus/cxl/devices/endpoint11/decoder11.0/interleave_granularity:256
> + /sys/bus/cxl/devices/endpoint11/decoder11.0/interleave_ways:1
> + /sys/bus/cxl/devices/endpoint11/decoder11.0/size:0x2000000000
> + /sys/bus/cxl/devices/endpoint11/decoder11.0/start:0x0
> + /sys/bus/cxl/devices/endpoint13/decoder13.0/interleave_granularity:256
> + /sys/bus/cxl/devices/endpoint13/decoder13.0/interleave_ways:1
> + /sys/bus/cxl/devices/endpoint13/decoder13.0/size:0x2000000000
> + /sys/bus/cxl/devices/endpoint13/decoder13.0/start:0x0
> +
> +Note the endpoint interleaving configurations with a direct mapping
> +(1-way).
> +
> +With PRM calls, the kernel can determine the following mappings:
> +
> +.. code-block:: none
> +
> + cxl decoder5.0: address mapping found for 0000:e2:00.0 (hpa -> spa):
> +   0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
> + cxl decoder8.0: address mapping found for 0000:e3:00.0 (hpa -> spa):
> +   0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
> + cxl decoder11.0: address mapping found for 0000:e4:00.0 (hpa -> spa):
> +   0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
> + cxl decoder13.0: address mapping found for 0000:e1:00.0 (hpa -> spa):
> +   0x0+0x2000000000 -> 0x850000000+0x8000000000 ways:4 granularity:256
> +
> +The corresponding CXL host bridge (HDM) decoders and root decoder
> +(CFMWS) show and match with the calculated endpoint mappings:
> +
> +.. code-block:: none
> +
> + /sys/bus/cxl/devices/port1/decoder1.0/interleave_granularity:256
> + /sys/bus/cxl/devices/port1/decoder1.0/interleave_ways:4
> + /sys/bus/cxl/devices/port1/decoder1.0/size:0x8000000000
> + /sys/bus/cxl/devices/port1/decoder1.0/start:0x850000000
> + /sys/bus/cxl/devices/port1/decoder1.0/target_list:0,1,2,3
> + /sys/bus/cxl/devices/port1/decoder1.0/target_type:expander
> + /sys/bus/cxl/devices/root0/decoder0.0/interleave_granularity:256
> + /sys/bus/cxl/devices/root0/decoder0.0/interleave_ways:1
> + /sys/bus/cxl/devices/root0/decoder0.0/size:0x8000000000
> + /sys/bus/cxl/devices/root0/decoder0.0/start:0x850000000
> + /sys/bus/cxl/devices/root0/decoder0.0/target_list:7
> +
> +The following changes of the specification are needed:
> +
> +* Allow a CXL device to be in a different HPA space other than the
> +  host's space.
> +
> +* The platform can use implementation-specific address translation
> +  when crossing memory domains on the CXL.mem path between the Host
> +  and the Device.
> +
> +* The kernel (OSPM) determines Endpoint SPA range and interleaving
> +  configuration using platform specific address translation methods.

                         platform-specific

> +
> +Benefits of the Change
> +----------------------
> +
> +Without the change, the OSPM may not determine the memory region and
> +Root Decoder of an Endpoint and its corresponding HDM decoder. Region
> +creation would fail. Platforms with a different interconnect
> +architecture would fail to setup and use CXL.
> +
> +References
> +----------
> +
> +.. [#cxl-spec-3.2] Compute Express Link Specification, Revision 3.2, Version 1.0,
> +   https://www.computeexpresslink.org/
> +
> +.. [#amd-ppr-58088] AMD Family 1Ah Models 00h–0Fh and Models 10h–1Fh,
> +   ACPI v6.5 Porting Guide, Publication # 58088,
> +   https://www.amd.com/en/search/documentation/hub.html
> +
> +Detailed Description of the Change
> +----------------------------------
> +
> +Add the following paragraph in 8.2.4.20 CXL HDM Decoder Capability
> +Structure of the specification [#cxl-spec-3.2]_ to the end:
> +
> +"A device may use a different HPA space that is not common to other
> +components of the host domain. The platform is responsible for address
> +translation when crossing HPA spaces. The OSPM must determine the
> +interleaving configuration and perform address translation to HPA
> +ranges of the HDM decoders as needed. The translation mechanism is
> +host-specific and implementation dependent."

-- 
~Randy