[PATCH v7 00/11] PCI/CXL: Add CXL reset support for Type 2 devices

[Srirangan Madhavan <smadhavan@nvidia.com>]

Hi folks!

This series adds CXL Reset support for CXL Type 2 devices through the
existing PCI reset_method ABI. The reset sequence follows the CXL 4.0
specification [1], including CXL.cache disable, optional cache
writeback, CXL Reset initiation, ResetComplete polling, and ResetError
reporting.

The userspace ABI is the existing PCI reset interface:

    /sys/bus/pci/devices/.../reset_method
    /sys/bus/pci/devices/.../reset

Userspace can select "cxl_reset" in reset_method and then trigger reset
through the existing reset attribute.

Following Dan's v6 feedback, this replaces the proposed memdev sysfs ABI
with the existing PCI reset_method interface.

v7 changes from v6 [2]:
- Move the ABI from a CXL memdev attribute to PCI reset_method.
- Drop the memdev dependency from reset entry; advertise cxl_reset for
  Type 2 functions that report CXL Reset support in the CXL Device DVSEC.
- Incorporate Dan's HDM reset refactor: shared decoder settings,
  pci_dev->hdm cached state, and built-in CONFIG_CXL_HDM helpers.
- Cache endpoint HDM settings during PCI enumeration when MMIO decoding
  is already enabled, and let CXL core refresh the same cache later.
- Reduce the earlier PCI/CXL save/restore series [3] to the HDM state
  cache and restore infrastructure needed by this reset flow.
- Use cached HDM ranges to reject reset while affected ranges are busy
  and to invalidate CPU caches before reset.
- Discover the CXL reset scope with the Non-CXL Function Map and CXL
  cache/mem capability bits.
- Quiesce affected sibling functions with PCI save/disable and IOMMU
  reset prepare/done before executing reset.
- Restore cached HDM decoder state after reset before completing PCI
  reset recovery.
- Keep CXL Reset Memory Clear disabled.

Motivation:
-----------
- Type 2 devices need a CXL-specific reset mechanism beyond existing PCI
  reset methods.

- FLR does not reset CXL.cache or CXL.mem protocol state. CXL Reset is
  the architectural reset mechanism for those protocols.

- The PCI reset_method ABI lets userspace select this narrower CXL reset
  before falling back to broader bus reset methods.

Change Description:
-------------------

Patch 1: cxl/hdm: Split decoder programming into a reusable helper
- Move shared decoder settings to include/cxl/cxl.h.
- Factor low-level HDM register programming into cxl_commit().

Patch 2: cxl/hdm: Cache decoder settings on PCI devices
- Cache CXL core HDM decoder settings in pci_dev->hdm.
- Refresh the cache as decoders are enumerated, committed, or reset.

Patch 3: cxl/hdm: Cache endpoint decoder settings during PCI enumeration
- Snapshot endpoint HDM state during PCI capability initialization when
  memory decoding is already enabled.
- Reuse the same cache when CXL core later enumerates the device.

Patch 4: PCI: Export pci_dev_save_and_disable() and pci_dev_restore()
- Export PCI reset lifecycle helpers for CXL reset orchestration.

Patch 5: PCI/CXL: Add CXL Device Reset helper
- Add the internal DVSEC reset sequence.
- Disable CXL.cache, perform cache writeback where supported, initiate
  CXL Reset, and wait for completion.

Patch 6: PCI/CXL: Validate HDM ranges before CXL reset
- Collect enabled cached HDM ranges.
- Reject reset if affected ranges are busy and invalidate CPU caches.

Patch 7: PCI/CXL: Discover the CXL reset scope
- Discover same-scope CXL functions with the Non-CXL Function Map and
  CXL cache/mem capability bits.

Patch 8: PCI/CXL: Coordinate sibling functions for CXL reset
- Lock, save, disable, and IOMMU-block affected sibling functions.
- Include mem-capable siblings in HDM range validation and cache flush.

Patch 9: cxl/pci: Restore CXL HDM state after PCI reset
- Restore cached global and per-decoder HDM state after reset.
- Keep IOMMU reset blocks active until HDM restore completes.

Patch 10: PCI/CXL: Expose CXL Reset as a PCI reset method
- Add "cxl_reset" to the PCI reset_method table for Type 2 reset-capable
  CXL devices.

Patch 11: Documentation/ABI: Document CXL Reset PCI reset method
- Document the new reset_method value and reset behavior.

The CPU cache invalidation step depends on
cpu_cache_invalidate_memregion() support for the affected address ranges.
If no provider is available, reset fails before hardware reset is
requested.

Example:

    echo cxl_reset > /sys/bus/pci/devices/0000:bb:dd.f/reset_method
    echo 1 > /sys/bus/pci/devices/0000:bb:dd.f/reset

Basic CXL DVSEC reset testing was done on a CXL Type 2 device. The reset
sequence completed successfully and ResetComplete was observed.

References:
[1] https://computeexpresslink.org/wp-content/uploads/2026/02/CXL-Specification_rev4p0_ver1p0_2026February26_clean_evalcopy_v2.pdf
[2] https://lore.kernel.org/linux-cxl/20260528083154.137979-1-smadhavan@nvidia.com/
[3] https://lore.kernel.org/linux-cxl/20260306080026.116789-1-smadhavan@nvidia.com/

Srirangan Madhavan (11):
  cxl/hdm: Split decoder programming into a reusable helper
  cxl/hdm: Cache decoder settings on PCI devices
  cxl/hdm: Cache endpoint decoder settings during PCI enumeration
  PCI: Export pci_dev_save_and_disable() and pci_dev_restore()
  PCI/CXL: Add CXL Device Reset helper
  PCI/CXL: Validate HDM ranges before CXL reset
  PCI/CXL: Discover the CXL reset scope
  PCI/CXL: Coordinate sibling functions for CXL reset
  cxl/pci: Restore CXL HDM state after PCI reset
  PCI/CXL: Expose CXL Reset as a PCI reset method
  Documentation/ABI: Document CXL Reset PCI reset method

 Documentation/ABI/testing/sysfs-bus-pci |   14 +
 drivers/cxl/Kconfig                     |    4 +
 drivers/cxl/core/Makefile               |    2 +-
 drivers/cxl/core/hdm.c                  |  234 ++---
 drivers/cxl/core/region.c               |    6 +-
 drivers/cxl/core/reset.c                | 1276 +++++++++++++++++++++++
 drivers/cxl/cxl.h                       |   43 -
 drivers/pci/pci.c                       |   25 +-
 drivers/pci/probe.c                     |    2 +
 include/cxl/cxl.h                       |   85 +-
 include/linux/pci.h                     |   10 +-
 include/uapi/linux/pci_regs.h           |   15 +
 tools/testing/cxl/test/cxl.c            |   10 +-
 13 files changed, 1554 insertions(+), 172 deletions(-)
 create mode 100644 drivers/cxl/core/reset.c

base-commit: 72afdd8181219f459142e571999b3b44ef7b85fb
-- 
2.43.0