Re: [PATCH v8 00/10] PCI/CXL: Add CXL reset support for Type 2 devices

[Dave Jiang <dave.jiang@intel.com>]

On 7/6/26 4:13 PM, Dave Jiang wrote:
> 
> 
> On 7/3/26 3:04 PM, Srirangan Madhavan wrote:
>> Hi folks!
>>
>> This series adds CXL Reset support for CXL Type 2 devices through the
>> existing PCI reset_method ABI. The reset sequence follows the CXL 4.0
>> specification [1], including CXL.cache disable, optional cache
>> writeback, CXL Reset initiation, ResetComplete polling, and ResetError
>> reporting.
>>
>> The userspace ABI is the existing PCI reset interface:
>>
>>     /sys/bus/pci/devices/.../reset_method
>>     /sys/bus/pci/devices/.../reset
>>
>> Userspace can select "cxl_reset" in reset_method and then trigger reset
>> through the existing reset attribute.
>>
>> Following Dan's v6 feedback, this replaces the proposed memdev sysfs ABI
>> with the existing PCI reset_method interface.
>>
>> v8 changes from v7 [2]:
>> - Drop the PCI helper export patch.
>> - Drop the multi-function sibling coordination patch. cxl_reset is only
>>   exposed as a function-scoped reset method when the CXL reset scope is
>>   limited to the target function.
>> - Keep the reset-scope discovery needed to reject non-function-scoped
>>   CXL resets.
>> - Cache HDM location as BAR-relative metadata instead of keeping an
>>   enum-time MMIO mapping.
>> - Restore HDM through a temporary mapping based on the current BAR
>>   assignment after PCI config state is restored.
>> - Cache raw HDM decoder register state so uncommitted decoders can be
>>   restored as uncommitted, while committed decoders are recommitted
>>   through the normal HDM commit flow.
>> - Share HDM decoder decode and validation between normal CXL core
>>   enumeration and early PCI HDM caching.
>> - Put cxl_reset ahead of FLR in reset_method priority because FLR does
>>   not reset CXL.cache or CXL.mem protocol state.
>>
>> Motivation:
>> -----------
>> - Type 2 devices need a CXL-specific reset mechanism beyond existing PCI
>>   reset methods.
>>
>> - FLR does not reset CXL.cache or CXL.mem protocol state. CXL Reset is
>>   the architectural reset mechanism for those protocols.
>>
>> - The PCI reset_method ABI lets userspace select this narrower CXL reset
>>   before falling back to broader bus reset methods.
>>
>> Change Description:
>> -------------------
>>
>> Patch 1: cxl/hdm: Split decoder programming into a reusable helper
>> - Move shared decoder settings to include/cxl/cxl.h.
>> - Factor low-level HDM register programming into cxl_commit().
>>
>> Patch 2: cxl/hdm: Cache decoder settings on PCI devices
>> - Cache CXL core HDM decoder settings in pci_dev->hdm.
>> - Refresh the cache as decoders are enumerated, committed, or reset.
>>
>> Patch 3: cxl/hdm: Share HDM decoder decode logic
>> - Share HDM decoder decode and validation between normal CXL core
>>   enumeration and early PCI HDM cache setup.
>>
>> Patch 4: cxl/hdm: Cache endpoint decoder settings during PCI enumeration
>> - Snapshot endpoint HDM state during PCI capability initialization.
>> - Cache the HDM register locator as BAR-relative metadata.
>> - Cache raw decoder register state in addition to committed decoder
>>   settings.
>>
>> Patch 5: PCI/CXL: Add CXL Device Reset helper
>> - Add the internal DVSEC reset sequence.
>> - Disable CXL.cache, perform cache writeback where supported, initiate
>>   CXL Reset, and wait for completion.
>>
>> Patch 6: PCI/CXL: Validate HDM ranges before CXL reset
>> - Collect enabled cached HDM ranges.
>> - Reject reset if affected ranges are busy.
>> - Invalidate CPU caches when runtime cache-invalidation support is
>>   available, otherwise continue after warning.
>>
>> Patch 7: PCI/CXL: Discover the CXL reset scope
>> - Discover whether CXL Reset is function-scoped using the Non-CXL
>>   Function Map and CXL cache/mem capability bits.
>>
>> Patch 8: cxl/pci: Restore CXL HDM state after PCI reset
>> - Restore cached global and per-decoder HDM state after reset.
>> - Re-map HDM registers from the current BAR assignment during restore.
>> - Replay raw decoder state for uncommitted decoders and recommit
>>   decoders that were committed before reset.
>>
>> Patch 9: PCI/CXL: Expose CXL Reset as a PCI reset method
>> - Add "cxl_reset" to the PCI reset_method table for Type 2 reset-capable
>>   CXL devices.
>> - Prioritize cxl_reset ahead of FLR.
>>
>> Patch 10: Documentation/ABI: Document CXL Reset PCI reset method
>> - Document the new reset_method value and reset behavior.
>>
>> The CPU cache invalidation step depends on
>> cpu_cache_invalidate_memregion() support for the affected address ranges.
>> If no runtime provider is available, the kernel emits a warning and
>> continues after the affected HDM ranges have been reserved.
>>
>> Example:
>>
>>     echo cxl_reset > /sys/bus/pci/devices/0000:bb:dd.f/reset_method
>>     echo 1 > /sys/bus/pci/devices/0000:bb:dd.f/reset
>>
>> Testing:
>> - Ran 100 iterations of cxl_reset through the PCI reset sysfs ABI on a
>>   CXL Type 2 device. All iterations completed successfully and
>>   ResetComplete was observed.
>> - Exercised cxl_bus reset separately with an add-on HDM restore patch.
>>
>> References:
>> [1] https://computeexpresslink.org/wp-content/uploads/2026/02/CXL-Specification_rev4p0_ver1p0_2026February26_clean_evalcopy_v2.pdf
>> [2] https://lore.kernel.org/linux-cxl/20260623032453.3404772-1-smadhavan@nvidia.com/
>> [3] https://lore.kernel.org/linux-cxl/20260306080026.116789-1-smadhavan@nvidia.com/
>>
>> Srirangan Madhavan (10):
>>   cxl/hdm: Split decoder programming into a reusable helper
>>   cxl/hdm: Cache decoder settings on PCI devices
>>   cxl/hdm: Share HDM decoder decode logic
>>   cxl/hdm: Cache endpoint decoder settings during PCI enumeration
>>   PCI/CXL: Add CXL Device Reset helper
>>   PCI/CXL: Validate HDM ranges before CXL reset
>>   PCI/CXL: Discover the CXL reset scope
>>   cxl/pci: Restore CXL HDM state after PCI reset
>>   PCI/CXL: Expose CXL Reset as a PCI reset method
>>   Documentation/ABI: Document CXL Reset PCI reset method
>>
>>  Documentation/ABI/testing/sysfs-bus-pci |   14 +
>>  drivers/cxl/Kconfig                     |    4 +
>>  drivers/cxl/core/Makefile               |    2 +-
>>  drivers/cxl/core/hdm.c                  |  257 ++---
>>  drivers/cxl/core/region.c               |    6 +-
>>  drivers/cxl/core/regs.c                 |    4 +
>>  drivers/cxl/core/reset.c                | 1354 +++++++++++++++++++++++
>>  drivers/cxl/cxl.h                       |   43 -
>>  drivers/pci/pci.c                       |    2 +
>>  drivers/pci/probe.c                     |    3 +
>>  include/cxl/cxl.h                       |  102 +-
>>  include/linux/pci.h                     |    8 +-
>>  include/uapi/linux/pci_regs.h           |   15 +
>>  tools/testing/cxl/test/cxl.c            |   10 +-
>>  14 files changed, 1615 insertions(+), 209 deletions(-)
>>  create mode 100644 drivers/cxl/core/reset.c
>>
>> base-commit: 90cf2e0d702c8a132ccbe72e7687f33c04c14658
> 
> 
> Hi Srirangan,
> I am not able to apply the patch series using b4 with the above base commit.
> b4 picks up the last RFC patch as the first patch and fails to apply. However, skipping it b4 is still having trouble. Can you push a public git branch somewhere please? Also in the future the last RFC patch should be labeled 11/11. Thanks!
> 

BTW, can this be applied cleanly against a Linus rc tag? If so please do that next rev. I understand you need the type2 stuff for testing. But preferably when posting, post it against a Linus tag if there are no symbol dependencies. I'd really like sashiko run a review through it and currently it is not able to. Thank you.