[PATCH v9 00/11] PCI/CXL: Add CXL reset support for Type 2 devices

[Srirangan Madhavan <smadhavan@nvidia.com>]

Hi folks!

This series adds CXL Reset support for CXL Type 2 devices through the
existing PCI reset_method ABI. The reset sequence follows the CXL 4.0
specification [1], including CXL.cache disable, optional cache
writeback, CXL Reset initiation, ResetComplete polling, and ResetError
reporting.

The userspace ABI is the existing PCI reset interface:

    /sys/bus/pci/devices/.../reset_method
    /sys/bus/pci/devices/.../reset

Userspace can select "cxl_reset" in reset_method and then trigger reset
through the existing reset attribute.

Following Dan's v6 feedback, this replaces the proposed memdev sysfs ABI
with the existing PCI reset_method interface.

v9 changes from v8 [2]:
- Rebase the series onto v7.2-rc2 so it applies cleanly on a Linus rc
  tag.
- Include the cxl_bus HDM restore add-on as patch 11/11 so b4 orders it
  after the main CXL reset series.
- Address only patch application issues in v9 to gather feedback on the
  current direction. Other review comments, including Dave's code
  structure/readability feedback, are deferred to v10.

v8 changes from v7 [3]:
- Drop the PCI helper export patch.
- Drop the multi-function sibling coordination patch. cxl_reset is only
  exposed as a function-scoped reset method when the CXL reset scope is
  limited to the target function.
- Keep the reset-scope discovery needed to reject non-function-scoped
  CXL resets.
- Cache HDM location as BAR-relative metadata instead of keeping an
  enum-time MMIO mapping.
- Restore HDM through a temporary mapping based on the current BAR
  assignment after PCI config state is restored.
- Cache raw HDM decoder register state so uncommitted decoders can be
  restored as uncommitted, while committed decoders are recommitted
  through the normal HDM commit flow.
- Share HDM decoder decode and validation between normal CXL core
  enumeration and early PCI HDM caching.
- Put cxl_reset ahead of FLR in reset_method priority because FLR does
  not reset CXL.cache or CXL.mem protocol state.

Motivation:
-----------
- Type 2 devices need a CXL-specific reset mechanism beyond existing PCI
  reset methods.

- FLR does not reset CXL.cache or CXL.mem protocol state. CXL Reset is
  the architectural reset mechanism for those protocols.

- The PCI reset_method ABI lets userspace select this narrower CXL reset
  before falling back to broader bus reset methods.

Change Description:
-------------------

Patch 1: cxl: Split decoder programming into a reusable helper
- Move shared decoder settings to include/cxl/cxl.h.
- Factor low-level HDM register programming into cxl_commit().

Patch 2: cxl: Cache decoder settings on PCI devices
- Cache CXL core HDM decoder settings in pci_dev->hdm.
- Refresh the cache as decoders are enumerated, committed, or reset.

Patch 3: cxl: Share HDM decoder decode logic
- Share HDM decoder decode and validation between normal CXL core
  enumeration and early PCI HDM cache setup.

Patch 4: cxl: Cache endpoint decoder settings during PCI enumeration
- Snapshot endpoint HDM state during PCI capability initialization.
- Cache the HDM register locator as BAR-relative metadata.
- Cache raw decoder register state in addition to committed decoder
  settings.

Patch 5: cxl: Add CXL Device Reset helper
- Add the internal DVSEC reset sequence.
- Disable CXL.cache, perform cache writeback where supported, initiate
  CXL Reset, and wait for completion.

Patch 6: cxl: Validate HDM ranges before CXL reset
- Collect enabled cached HDM ranges.
- Reject reset if affected ranges are busy.
- Invalidate CPU caches when runtime cache-invalidation support is
  available, otherwise continue after warning.

Patch 7: PCI/CXL: Discover the CXL reset scope
- Discover whether CXL Reset is function-scoped using the Non-CXL
  Function Map and CXL cache/mem capability bits.

Patch 8: cxl: Restore CXL HDM state after PCI reset
- Restore cached global and per-decoder HDM state after reset.
- Re-map HDM registers from the current BAR assignment during restore.
- Replay raw decoder state for uncommitted decoders and recommit
  decoders that were committed before reset.

Patch 9: PCI/CXL: Expose CXL Reset as a PCI reset method
- Add "cxl_reset" to the PCI reset-method table for Type 2 reset-capable
  CXL devices.
- Prioritize cxl_reset ahead of FLR.

Patch 10: Documentation/ABI: Document CXL Reset PCI reset method
- Document the new reset_method value and reset behavior.

Patch 11: PCI/CXL: Restore HDM state after CXL bus reset
- Restore cached CXL HDM decoder state after cxl_bus reset.

The CPU cache invalidation step depends on
cpu_cache_invalidate_memregion() support for the affected address ranges.
If no runtime provider is available, the kernel emits a warning and
continues after the affected HDM ranges have been reserved.

Example:

    echo cxl_reset > /sys/bus/pci/devices/0000:bb:dd.f/reset_method
    echo 1 > /sys/bus/pci/devices/0000:bb:dd.f/reset

Testing:
- Ran 100 iterations of cxl_reset through the PCI reset sysfs ABI on a
  CXL Type 2 device. All iterations completed successfully and
  ResetComplete was observed.
- Exercised cxl_bus reset separately with the HDM restore add-on patch.

References:
[1] https://computeexpresslink.org/wp-content/uploads/2026/02/CXL-Specification_rev4p0_ver1p0_2026February26_clean_evalcopy_v2.pdf
[2] https://lore.kernel.org/linux-cxl/20260703220508.546528-1-smadhavan@nvidia.com/
[3] https://lore.kernel.org/linux-cxl/20260623032453.3404772-1-smadhavan@nvidia.com/

Srirangan Madhavan (11):
  cxl: Split decoder programming into a reusable helper
  cxl: Cache decoder settings on PCI devices
  cxl: Share HDM decoder decode logic
  cxl: Cache endpoint decoder settings during PCI enumeration
  cxl: Add CXL Device Reset helper
  cxl: Validate HDM ranges before CXL reset
  PCI/CXL: Discover the CXL reset scope
  cxl: Restore CXL HDM state after PCI reset
  PCI/CXL: Expose CXL Reset as a PCI reset method
  Documentation/ABI: Document CXL Reset PCI reset method
  PCI/CXL: Restore HDM state after CXL bus reset

 Documentation/ABI/testing/sysfs-bus-pci |   14 +
 drivers/cxl/Kconfig                     |    4 +
 drivers/cxl/core/Makefile               |    2 +-
 drivers/cxl/core/hdm.c                  |  257 ++---
 drivers/cxl/core/region.c               |    6 +-
 drivers/cxl/core/regs.c                 |    4 +
 drivers/cxl/core/reset.c                | 1359 +++++++++++++++++++++++
 drivers/cxl/cxl.h                       |   43 -
 drivers/pci/pci.c                       |    5 +
 drivers/pci/probe.c                     |    3 +
 include/cxl/cxl.h                       |  108 +-
 include/linux/pci.h                     |    8 +-
 include/uapi/linux/pci_regs.h           |   14 +
 tools/testing/cxl/test/cxl.c            |   10 +-
 14 files changed, 1628 insertions(+), 209 deletions(-)
 create mode 100644 drivers/cxl/core/reset.c


base-commit: 8cdeaa50eae8dad34885515f62559ee83e7e8dda
-- 
2.43.0