[PATCH v11 00/12] PCI/CXL: Add CXL reset support for Type 2 devices

[Srirangan Madhavan <smadhavan@nvidia.com>]

Hi folks!

This series adds CXL Reset support for CXL Type 2 devices through the
existing PCI reset_method ABI. The reset sequence follows the CXL 4.0
specification [1], including CXL.cache disable, optional cache
writeback, CXL Reset initiation, ResetComplete polling, and ResetError
reporting.

The userspace ABI is the existing PCI reset interface:

    /sys/bus/pci/devices/.../reset_method
    /sys/bus/pci/devices/.../reset

Userspace can select "cxl_reset" in reset_method and then trigger reset
through the existing reset attribute.

CXL Reset is exposed conservatively. The target must be a
single-function CXL Type 2 device with CXL.cache and CXL.mem enabled,
and the kernel must have cached HDM state sufficient to validate affected
ranges and restore decoder programming. The reset preserves device
memory; it does not request CXL Reset Memory Clear.

Following Dan's v6 feedback, this uses the existing PCI reset_method
interface instead of adding a memdev-specific sysfs ABI.

v11 changes from v10 [2]:
- Rebase onto v7.3-rc1, which includes the CXL Type 2 enabling work, and
  use the merged public CXL PCI header.
- Rename the built-in support symbol to CONFIG_CXL_RESET and fix
  built-in/module linkage for CONFIG_CXL_BUS=m.
- Fix tools/testing/cxl linkage without compiling production helper
  objects twice in the mock cxl_core module.
- Simplify decoder snapshot handling and preserve endpoint DPA skip
  ownership under the existing DPA lock.
- Keep policy out of shared decoder parsing by accepting spec-legal
  zero-size decoder state, while skipping zero-size ranges during reset
  validation.
- Tighten early HDM cache setup, capability checks, BAR bounds, MSE
  enable/restore handling, and cache publication locking.
- Keep the conservative multifunction-device rejection requested in
  earlier review.
- Harden post-reset failure handling so MSE and BME remain disabled when
  HDM state cannot be safely restored.
- Serialize CXL bus-reset HDM restore against region updates and flush
  posted HDM writes before PCI_COMMAND can restore bus mastering.
- Address the remaining reviewability and naming feedback from v10.

Motivation:
-----------
- Type 2 devices need a CXL-specific reset mechanism beyond existing PCI
  reset methods.

- FLR does not reset CXL.cache or CXL.mem protocol state. CXL Reset is
  the architectural reset mechanism for those protocols.

- The PCI reset_method ABI lets userspace select CXL Reset before
  falling back to broader bus reset methods.

Change Description:
-------------------

Patch 1: cxl: Move HDM decoder programming helpers
- Introduce a decoder settings snapshot collected under the DPA lock.
- Move low-level settings-based HDM programming into resource.c.

Patch 2: cxl: Make HDM commit helpers available to reset code
- Build generic HDM resource helpers under CONFIG_CXL_RESET.
- Export the lock and commit helpers needed when cxl_core is modular.

Patch 3: cxl: Share HDM decoder decode logic
- Share HDM decoder decode and validation between normal CXL core
  enumeration and early PCI HDM cache setup.
- Decode zero-size committed decoder state without imposing caller policy.

Patch 4: cxl: Cache decoder settings on PCI devices
- Cache CXL core HDM decoder settings in pci_dev->hdm.
- Refresh the cache as decoders are enumerated, committed, or reset.

Patch 5: cxl: Cache endpoint decoder settings during PCI enumeration
- Snapshot endpoint HDM state during PCI enumeration.
- Cache the HDM register locator as BAR-relative metadata and preserve raw
  decoder state for reset restore.

Patch 6: cxl: Add CXL Device Reset helper
- Add the internal DVSEC reset sequence.
- Disable CXL.cache, perform cache writeback where supported, initiate
  CXL Reset, and wait for completion.

Patch 7: cxl: Validate HDM ranges before CXL reset
- Collect enabled cached HDM ranges, reject busy ranges, and perform CPU
  cache invalidation when a runtime backend is available.

Patch 8: PCI/CXL: Reject CXL Reset on multifunction devices
- Conservatively reject CXL Reset where function scope cannot be assumed.

Patch 9: cxl: Restore CXL HDM state after PCI reset
- Restore cached global and per-decoder HDM state after reset while the
  target IOMMU reset block remains active.

Patch 10: PCI/CXL: Expose CXL Reset as a PCI reset method
- Add "cxl_reset" to the PCI reset-method table ahead of FLR.

Patch 11: Documentation/ABI: Document CXL Reset PCI reset method
- Document the new reset_method value and reset behavior.

Patch 12: PCI/CXL: Restore HDM state after CXL bus reset
- Restore cached CXL HDM decoder state after cxl_bus reset.

The CPU cache invalidation step depends on
cpu_cache_invalidate_memregion() support for the affected address ranges.
If no runtime provider is available, the kernel emits a warning and
continues after the affected HDM ranges have been reserved.

Example:

    echo cxl_reset > /sys/bus/pci/devices/0000:bb:dd.f/reset_method
    echo 1 > /sys/bus/pci/devices/0000:bb:dd.f/reset

Testing:
- Built every intermediate patch, plus the v7.3-rc1 baseline, with
  CONFIG_CXL_BUS=m. All 12 patches passed a full kernel build and
  `make M=tools/testing/cxl/` independently.
- Built and booted the series on an arm64 NVIDIA NVL72 system and
  successfully exercised CXL Reset through the PCI reset sysfs ABI.
- Earlier revision testing ran 100 iterations of cxl_reset and exercised
  cxl_bus separately with HDM state restoration.

References:
[1] https://computeexpresslink.org/wp-content/uploads/2026/02/CXL-Specification_rev4p0_ver1p0_2026February26_clean_evalcopy_v2.pdf
[2] https://lore.kernel.org/linux-cxl/20260804192958.1823952-1-smadhavan@nvidia.com/
[3] https://lore.kernel.org/linux-cxl/20260709010304.680422-1-smadhavan@nvidia.com/
[4] https://lore.kernel.org/linux-cxl/20260703220508.546528-1-smadhavan@nvidia.com/
[5] https://lore.kernel.org/linux-cxl/20260623032453.3404772-1-smadhavan@nvidia.com/

Srirangan Madhavan (12):
  cxl: Move HDM decoder programming helpers
  cxl: Make HDM commit helpers available to reset code
  cxl: Share HDM decoder decode logic
  cxl: Cache decoder settings on PCI devices
  cxl: Cache endpoint decoder settings during PCI enumeration
  cxl: Add CXL Device Reset helper
  cxl: Validate HDM ranges before CXL reset
  PCI/CXL: Reject CXL Reset on multifunction devices
  cxl: Restore CXL HDM state after PCI reset
  PCI/CXL: Expose CXL Reset as a PCI reset method
  Documentation/ABI: Document CXL Reset PCI reset method
  PCI/CXL: Restore HDM state after CXL bus reset

 Documentation/ABI/testing/sysfs-bus-pci |   15 +
 drivers/cxl/Kconfig                     |    4 +
 drivers/cxl/core/Makefile               |    2 +-
 drivers/cxl/core/core.h                 |    8 +
 drivers/cxl/core/hdm.c                  |  324 +++---
 drivers/cxl/core/pci.c                  |    7 +-
 drivers/cxl/core/port.c                 |   19 +
 drivers/cxl/core/regs.c                 |    4 +
 drivers/cxl/core/resource.c             | 1351 +++++++++++++++++++++++
 drivers/cxl/cxl.h                       |   41 -
 drivers/pci/bus.c                       |    2 +
 drivers/pci/pci.c                       |    8 +
 drivers/pci/probe.c                     |    2 +
 include/cxl/cxl.h                       |  116 +-
 include/linux/pci.h                     |    8 +-
 include/uapi/linux/pci_regs.h           |   14 +
 tools/testing/cxl/Kbuild                |    1 -
 17 files changed, 1715 insertions(+), 211 deletions(-)
 create mode 100644 drivers/cxl/core/resource.c


base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
-- 
2.43.0