[PATCH v10 00/12] PCI/CXL: Add CXL reset support for Type 2 devices

[Srirangan Madhavan <smadhavan@nvidia.com>]

Hi folks!

This series adds CXL Reset support for CXL Type 2 devices through the
existing PCI reset_method ABI. The reset sequence follows the CXL 4.0
specification [1], including CXL.cache disable, optional cache
writeback, CXL Reset initiation, ResetComplete polling, and ResetError
reporting.

The userspace ABI is the existing PCI reset interface:

    /sys/bus/pci/devices/.../reset_method
    /sys/bus/pci/devices/.../reset

Userspace can select "cxl_reset" in reset_method and then trigger reset
through the existing reset attribute.

Following Dan's v6 feedback, this replaces the proposed memdev sysfs ABI
with the existing PCI reset_method interface.

v10 changes from v9 [2]:
- Address review comments deferred from v9.
- Split HDM decoder helper refactoring into smaller reviewable patches.
- Rename the built-in CXL helper file to resource.c and keep reset-specific
  logic separate from the decoder programming refactor.
- Add a helper to resolve a CXL port's upstream PCI device.
- Fix tools/testing/cxl Kbuild coverage for resource.o.
- Keep early HDM decoder parsing shared with normal CXL HDM enumeration.
- Gate CXL Reset execution on cached HDM state and current CXL.cache/CXL.mem
  enablement.
- Reject CXL Reset on multifunction devices to avoid reset scope ambiguity.
- Order CXL cache disable/writeback before PCI reset quiesce and IOMMU reset
  preparation.
- Restore cached HDM state without replaying transient commit/error/lock
  control bits.
- Update ABI documentation to match the implemented reset ordering and CPU
  cache invalidation behavior.

v9 changes from v8 [3]:
- Rebase the series onto v7.2-rc2 so it applies cleanly on a Linus rc
  tag.
- Include the cxl_bus HDM restore add-on as patch 11/11 so b4 orders it
  after the main CXL reset series.
- Address only patch application issues in v9 to gather feedback on the
  current direction. Other review comments, including Dave's code
  structure/readability feedback, are deferred to v10.

v8 changes from v7 [4]:
- Drop the PCI helper export patch.
- Drop the multi-function sibling coordination patch. cxl_reset is only
  exposed as a function-scoped reset method when the CXL reset scope is
  limited to the target function.
- Keep the reset-scope discovery needed to reject non-function-scoped
  CXL resets.
- Cache HDM location as BAR-relative metadata instead of keeping an
  enum-time MMIO mapping.
- Restore HDM through a temporary mapping based on the current BAR
  assignment after PCI config state is restored.
- Cache raw HDM decoder register state so uncommitted decoders can be
  restored as uncommitted, while committed decoders are recommitted
  through the normal HDM commit flow.
- Share HDM decoder decode and validation between normal CXL core
  enumeration and early PCI HDM caching.
- Put cxl_reset ahead of FLR in reset_method priority because FLR does
  not reset CXL.cache or CXL.mem protocol state.

Motivation:
-----------
- Type 2 devices need a CXL-specific reset mechanism beyond existing PCI
  reset methods.

- FLR does not reset CXL.cache or CXL.mem protocol state. CXL Reset is
  the architectural reset mechanism for those protocols.

- The PCI reset_method ABI lets userspace select this narrower CXL reset
  before falling back to broader bus reset methods.

Change Description:
-------------------

Patch 1: cxl: Move HDM decoder programming helpers
- Move low-level HDM decoder programming helpers into resource.c.
- Keep the movement mechanical so later patches can change the interface
  separately.

Patch 2: cxl: Pass decoder settings to HDM commit helpers
- Introduce a decoder settings snapshot for HDM commit programming.
- Route decoder commit through helpers that can be reused by reset restore.

Patch 3: cxl: Share HDM decoder decode logic
- Add a small helper to resolve a CXL port's upstream PCI device.
- Share HDM decoder decode and validation between normal CXL core
  enumeration and early PCI HDM cache setup.

Patch 4: cxl: Cache decoder settings on PCI devices
- Cache CXL core HDM decoder settings in pci_dev->hdm.
- Refresh the cache as decoders are enumerated, committed, or reset.

Patch 5: cxl: Cache endpoint decoder settings during PCI enumeration
- Snapshot endpoint HDM state during PCI enumeration.
- Cache the HDM register locator as BAR-relative metadata.
- Cache raw decoder register state in addition to committed decoder
  settings.

Patch 6: cxl: Add CXL Device Reset helper
- Add the internal DVSEC reset sequence.
- Disable CXL.cache, perform cache writeback where supported, initiate
  CXL Reset, and wait for completion.

Patch 7: cxl: Validate HDM ranges before CXL reset
- Collect enabled cached HDM ranges.
- Reject reset if affected ranges are busy.
- Invalidate CPU caches when runtime cache-invalidation support is
  available, otherwise continue after warning.

Patch 8: PCI/CXL: Reject CXL Reset on multifunction devices
- Reject CXL Reset for multifunction devices to avoid reset scope ambiguity.

Patch 9: cxl: Restore CXL HDM state after PCI reset
- Restore cached global and per-decoder HDM state after reset.
- Re-map HDM registers from the current BAR assignment during restore.
- Replay raw decoder state for uncommitted decoders and recommit
  decoders that were committed before reset.

Patch 10: PCI/CXL: Expose CXL Reset as a PCI reset method
- Add "cxl_reset" to the PCI reset-method table for Type 2 reset-capable
  CXL devices.
- Prioritize cxl_reset ahead of FLR.

Patch 11: Documentation/ABI: Document CXL Reset PCI reset method
- Document the new reset_method value and reset behavior.

Patch 12: PCI/CXL: Restore HDM state after CXL bus reset
- Restore cached CXL HDM decoder state after cxl_bus reset.

The CPU cache invalidation step depends on
cpu_cache_invalidate_memregion() support for the affected address ranges.
If no runtime provider is available, the kernel emits a warning and
continues after the affected HDM ranges have been reserved.

Example:

    echo cxl_reset > /sys/bus/pci/devices/0000:bb:dd.f/reset_method
    echo 1 > /sys/bus/pci/devices/0000:bb:dd.f/reset

Testing:
- Ran 100 iterations of cxl_reset through the PCI reset sysfs ABI on a
  CXL Type 2 device. All iterations completed successfully and
  ResetComplete was observed.
- Exercised cxl_bus reset separately with the HDM restore add-on patch.
- Built drivers/cxl/core/resource.o and drivers/pci/pci.o during v10
  preparation.

References:
[1] https://computeexpresslink.org/wp-content/uploads/2026/02/CXL-Specification_rev4p0_ver1p0_2026February26_clean_evalcopy_v2.pdf
[2] https://lore.kernel.org/linux-cxl/20260709010304.680422-1-smadhavan@nvidia.com/
[3] https://lore.kernel.org/linux-cxl/20260703220508.546528-1-smadhavan@nvidia.com/
[4] https://lore.kernel.org/linux-cxl/20260623032453.3404772-1-smadhavan@nvidia.com/

Srirangan Madhavan (12):
  cxl: Move HDM decoder programming helpers
  cxl: Pass decoder settings to HDM commit helpers
  cxl: Share HDM decoder decode logic
  cxl: Cache decoder settings on PCI devices
  cxl: Cache endpoint decoder settings during PCI enumeration
  cxl: Add CXL Device Reset helper
  cxl: Validate HDM ranges before CXL reset
  PCI/CXL: Reject CXL Reset on multifunction devices
  cxl: Restore CXL HDM state after PCI reset
  PCI/CXL: Expose CXL Reset as a PCI reset method
  Documentation/ABI: Document CXL Reset PCI reset method
  PCI/CXL: Restore HDM state after CXL bus reset

 Documentation/ABI/testing/sysfs-bus-pci |   16 +
 drivers/cxl/Kconfig                     |    4 +
 drivers/cxl/core/Makefile               |    2 +-
 drivers/cxl/core/core.h                 |    7 +
 drivers/cxl/core/hdm.c                  |  351 +++---
 drivers/cxl/core/port.c                 |   19 +
 drivers/cxl/core/regs.c                 |    4 +
 drivers/cxl/core/resource.c             | 1341 +++++++++++++++++++++++
 drivers/cxl/cxl.h                       |   41 -
 drivers/pci/bus.c                       |    2 +
 drivers/pci/pci.c                       |    8 +
 drivers/pci/probe.c                     |    2 +
 include/cxl/cxl.h                       |  116 +-
 include/linux/pci.h                     |    8 +-
 include/uapi/linux/pci_regs.h           |   14 +
 tools/testing/cxl/Kbuild                |    2 +-
 16 files changed, 1727 insertions(+), 210 deletions(-)
 create mode 100644 drivers/cxl/core/resource.c


base-commit: 8cdeaa50eae8dad34885515f62559ee83e7e8dda
-- 
2.43.0