Re: [PATCH v3 1/2] PCI: Add D3cold as general reset method

[Alex Williamson <alex@shazbot.org>]

On Thu, 14 May 2026 14:23:15 +0200
Jose Ignacio Tornos Martinez <jtornosm@redhat.com> wrote:

> Hello Alex,
> 
> Thank you again for your feedbak.
> 
> > This fall through is invalid, if D3cold can't be reached the reset should
> > be handled by pci_pm_reset() where NoSoftRst is honored.  pci_pr3_preset()
> > should be tested in all cases, not just probe.  I think we also need to
> > test device flags that would prevent a D3cold transition,
> > like pci_pm_reset().  Thanks,  
> Ok, I've implemented it as a strict method (only when _PR3 is present) as
> you suggested for the next version. I think it can be very helpful.
> 
> However, I'm facing a challenge with a part of our deployment scenario and
> if possible I would appreciate your guidance (spoiler: d3hot possibility
> would be very helpful).
> 
> Let me comment on about this setup in our lab:
> - Qualcomm WiFi cards (ath11k 17cb:1103, ath12k 17cb:1107) and modems
>   (SDX62/SDX65 17cb:0308)
> - M.2 format devices using passive PCIe adapters (not native M.2 slots)
> Problem:
> - No FLR capability on devices
> - NoSoftRst+ set (blocks PM reset)
> - SBR broken through passive adapters (PERST# signal not properly wired)
> - No _PR3 ACPI resources for standard PCIe slots (only for native M.2/
>   Thunderbolt)
> - d3cold_allowed=1 but pci_pr3_present() returns false
> Result:
> Default kernel has no working reset method (reset_methods shows only "bus"
> which doesn't work).
> 
> I've tested the D3hot transition behavior and this is what I see before/
> after:
> - Command register: Cleared to 0x0000 (device disabled)
> - BARs: Preserved (0x84200004 unchanged)
> - Device successfully reinitializes and works in VFIO passthrough, so we
>   can use it.

Are you testing with the patch that marks these devices
PCI_DEV_FLAGS_NO_BUS_RESET?

I want to be sure that bouncing the device through D3hot is actually an
improvement versus no reset at all, and we're not just performing a
no-op reset to avoid the bus reset.

That said, if you're testing before BAR restore and those registers
aren't being cleared, this seems at best a device specific reset, but
evidence is a bit sketchy.  In the general case, a transition through
D3hot for a device that does not claim NoSoftRst- cannot be consider to
satisfy a reset request.
 
> I don't dare to generalize D3hot in pci_pm_reset() with something like:
> if (csr & PCI_PM_CTRL_NO_SOFT_RESET) {
>   pci_read_config_word(dev, dev->pm_cap + PCI_PM_PMC, &pmc);
> 
>   /* If device supports D3hot, try reset despite NoSoftRst */
>   if (!pmc & PCI_PM_CAP_D3HOT_MASK)
>     return -ENOTTY;
> }
> because some devices could truly advertise NoSoftRst+ and d3hot could not be
> usable as a reset.

This should be the default, no assumed reset.  The device is
specifically able to indicate soft reset via D3hot.

> I don't know if some device-specific quirk could be acceptable to bypass
> NoSoftRst check for these Qualcomm devices (similar to
> PCI_DEV_FLAGS_FORCE_PM_RESET from v2), although d3hot is usable in these
> cases.
> 
> Or maybe the generalization could be to create a user parameter to bypass
> PCI_PM_CTRL_NO_SOFT_RESET for the configured device. This would be manual but
> at least it would allow the device to work on VMs.

vfio-pci will try to put unused device into a low power state when
they're unused.  If the kernel exposes no function level reset
mechanism, wouldn't the device experience D3hot after some idle timeout
and then be usable again?  Could you otherwise set the device power
state via sysfs?  It's certainly possible to create a userspace vfio
stub driver that uses the low-power ioctl to trigger D3hot.

Unfortunately not all devices are sufficiently spec compliant to work
cleanly with vfio, especially consumer grade devices without vendor
involvement.

> And finally, what I would choose: move pm to the end of the resets methods
> ignoring NoSoftRst, prioritizing hardware resets first (FLR, bus, d3cold)
> and using d3hot as the last resource if none of the others work. But this is
> an important change and there could be some reasons for pm positioning that
> I ignore, so if you don't mind I would like to consult you first.

We're getting into policy there, where the purpose of exposing
reset_methods through sysfs is to allow userspace to define policy.
There's obviously some policy in the default ordering, but we generally
take the approach of flagging broken resets rather than enforcing an
order.  In general I agree with your suspicion of pm reset, there's no
hard definition of a soft reset in the spec afaik.  In fact, QEMU tries
to infer whether the vfio reset ioctl likely implements pm reset by
looking at the device features and will prefer a bus reset when the
alternative is likely a pm reset.

If you can block the known broken resets and manage to trigger a D3hot
transition externally, that may provide a reasonably useful
environment.  We've also been known to implement the sketchier resets
that can't fully be justified as a kernel quirk in QEMU itself.  Thanks,

Alex