[PATCH v3 1/2] PCI: Add D3cold as general reset method

[PATCH v3 1/2] PCI: Add D3cold as general reset method

[Jose Ignacio Tornos Martinez]

Add D3cold power cycle as a general PCI reset method, available for
single-function devices. This provides a more robust reset mechanism
than D3hot for devices where a full power cycle is beneficial.

The implementation uses pci_set_power_state(dev, PCI_D3cold), which
automatically handles platform differences:
- Platforms WITH _PR3 ACPI power resources: true D3cold (power cycle)
- Platforms WITHOUT _PR3: automatic fallback to D3hot transition

D3cold reset is placed at the end of the reset hierarchy as a last
resort before giving up, since it provides a strong reset when other
methods are unavailable or broken.

Reset hierarchy with this change:
1. device_specific
2. acpi
3. flr
4. af_flr
5. pm (D3hot via config space)
6. bus (SBR)
7. cxl_bus
8. d3cold (NEW - power cycle with D3hot fallback)

This benefits devices that:
- Have broken or missing FLR
- Advertise NoSoftRst+ incorrectly (blocking D3hot PM reset)
- Have broken bus reset implementations
- Need power cycle for reliable reset
- Are used in VFIO passthrough scenarios

Signed-off-by: Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
---
v3: Implement d3cold as a general PCI core reset method instead of
  device-specific quirk approach from v2 (Alex Williamson suggestion)
v2: https://lore.kernel.org/all/20260508145153.717641-2-jtornosm@redhat.com/

 drivers/pci/pci.c   | 41 +++++++++++++++++++++++++++++++++++++++++
 include/linux/pci.h |  2 +-
 2 files changed, 42 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 8f7cfcc00090..6da8feff7ccc 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -4491,6 +4491,46 @@ static int pci_pm_reset(struct pci_dev *dev, bool probe)
 	return ret;
 }
 
+/**
+ * pci_d3cold_reset - Put device into D3cold (or D3hot) and back to D0 for reset
+ * @dev: PCI device to reset
+ * @probe: if true, check if D3cold reset is supported; if false, perform reset
+ *
+ * Attempt to reset the device by transitioning through D3cold and back to D0.
+ * On platforms with ACPI _PR3 power resources, this performs a true D3cold
+ * power cycle (actual power removal). On platforms without _PR3 support,
+ * pci_set_power_state() automatically falls back to D3hot, providing a
+ * D3hot->D0 reset transition.
+ *
+ * Only available for single-function devices to avoid affecting other
+ * functions in multi-function devices.
+ *
+ * Returns 0 if device can be/was reset this way, -ENOTTY if not supported,
+ * or other negative error code on failure.
+ */
+static int pci_d3cold_reset(struct pci_dev *dev, bool probe)
+{
+	int ret;
+
+	if (dev->multifunction)
+		return -ENOTTY;
+
+	if (probe) {
+		if (!pci_pr3_present(dev))
+			pci_dbg(dev, "d3cold reset: no _PR3 support, will use D3hot fallback\n");
+		return 0;
+	}
+
+	if (dev->current_state != PCI_D0)
+		return -EINVAL;
+
+	ret = pci_set_power_state(dev, PCI_D3cold);
+	if (ret)
+		return ret;
+
+	return pci_set_power_state(dev, PCI_D0);
+}
+
 /**
  * pcie_wait_for_link_status - Wait for link status change
  * @pdev: Device whose link to wait for.
@@ -5065,6 +5105,7 @@ const struct pci_reset_fn_method pci_reset_fn_methods[] = {
 	{ pci_pm_reset, .name = "pm" },
 	{ pci_reset_bus_function, .name = "bus" },
 	{ cxl_reset_bus_function, .name = "cxl_bus" },
+	{ pci_d3cold_reset, .name = "d3cold" },
 };
 
 /**
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 2c4454583c11..1ca7b880ead7 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -51,7 +51,7 @@
 			       PCI_STATUS_PARITY)
 
 /* Number of reset methods used in pci_reset_fn_methods array in pci.c */
-#define PCI_NUM_RESET_METHODS 8
+#define PCI_NUM_RESET_METHODS 9
 
 #define PCI_RESET_PROBE		true
 #define PCI_RESET_DO_RESET	false
-- 
2.53.0

[PATCH v3 2/2] PCI: Disable broken bus reset on Qualcomm devices

[Jose Ignacio Tornos Martinez]

Some Qualcomm PCIe devices do not properly support Secondary Bus Reset
(SBR). These devices have no FLR capability and advertise NoSoftRst+
(blocking PM reset), leaving bus reset as the only available method.
However, bus reset does not work reliably for these devices.

The problem manifests in VFIO passthrough scenarios with these affected
devices:

- ath11k WiFi (17cb:1103): Normal VM operation works fine, including
  clean shutdown/reboot. However, when the VM terminates uncleanly
  (crash, force-off), VFIO attempts to reset the device before it can
  be assigned to another VM. Without a working reset method, the device
  remains in an undefined state, preventing reuse.

- ath12k WiFi (17cb:1107): Same behavior as ath11k.

- SDX62/SDX65 5G modems (17cb:0308): Never successfully initialize even
  on first VM assignment without proper reset capability.

Disable bus reset for these devices (following the pattern of other
Atheros/Qualcomm devices) so the d3cold reset method (added in previous
patch) is used instead, which provides working reset via D3cold power
cycle or D3hot fallback.

Signed-off-by: Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
---
v3: Try to address Alex Williamson comments:
    - Change from PCI_DEV_FLAGS_FORCE_PM_RESET quirk (v2) to quirk_no_bus_reset
    - Rely on d3cold general reset method (patch 1) instead of forcing PM reset
v2: https://lore.kernel.org/all/20260508145153.717641-2-jtornosm@redhat.com/

 drivers/pci/quirks.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 000000000000..111111111111 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -3789,6 +3789,9 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATHEROS, 0x0030, quirk_no_bus_reset);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATHEROS, 0x0032, quirk_no_bus_reset);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATHEROS, 0x003c, quirk_no_bus_reset);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATHEROS, 0x0033, quirk_no_bus_reset);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATHEROS, 0x0034, quirk_no_bus_reset);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATHEROS, 0x003e, quirk_no_bus_reset);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_QCOM, 0x1103, quirk_no_bus_reset); /* ath11k */
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_QCOM, 0x1107, quirk_no_bus_reset); /* ath12k */
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_QCOM, 0x0308, quirk_no_bus_reset); /* SDX62/SDX65 */

 /*
  * Root port on some Cavium CN8xxx chips do not successfully complete a bus
--
2.53.0

Re: [PATCH v3 1/2] PCI: Add D3cold as general reset method

[Alex Williamson]

On Wed, May 13, 2026, at 6:23 AM, Jose Ignacio Tornos Martinez wrote:
> Add D3cold power cycle as a general PCI reset method, available for
> single-function devices. This provides a more robust reset mechanism
> than D3hot for devices where a full power cycle is beneficial.
>
> The implementation uses pci_set_power_state(dev, PCI_D3cold), which
> automatically handles platform differences:
> - Platforms WITH _PR3 ACPI power resources: true D3cold (power cycle)
> - Platforms WITHOUT _PR3: automatic fallback to D3hot transition
>
> D3cold reset is placed at the end of the reset hierarchy as a last
> resort before giving up, since it provides a strong reset when other
> methods are unavailable or broken.
>
> Reset hierarchy with this change:
> 1. device_specific
> 2. acpi
> 3. flr
> 4. af_flr
> 5. pm (D3hot via config space)
> 6. bus (SBR)
> 7. cxl_bus
> 8. d3cold (NEW - power cycle with D3hot fallback)
>
> This benefits devices that:
> - Have broken or missing FLR
> - Advertise NoSoftRst+ incorrectly (blocking D3hot PM reset)
> - Have broken bus reset implementations
> - Need power cycle for reliable reset
> - Are used in VFIO passthrough scenarios
>
> Signed-off-by: Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
> ---
> v3: Implement d3cold as a general PCI core reset method instead of
>   device-specific quirk approach from v2 (Alex Williamson suggestion)
> v2: https://lore.kernel.org/all/20260508145153.717641-2-jtornosm@redhat.com/
>
>  drivers/pci/pci.c   | 41 +++++++++++++++++++++++++++++++++++++++++
>  include/linux/pci.h |  2 +-
>  2 files changed, 42 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 8f7cfcc00090..6da8feff7ccc 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -4491,6 +4491,46 @@ static int pci_pm_reset(struct pci_dev *dev, bool probe)
>  	return ret;
>  }
> 
> +/**
> + * pci_d3cold_reset - Put device into D3cold (or D3hot) and back to D0 
> for reset
> + * @dev: PCI device to reset
> + * @probe: if true, check if D3cold reset is supported; if false, 
> perform reset
> + *
> + * Attempt to reset the device by transitioning through D3cold and 
> back to D0.
> + * On platforms with ACPI _PR3 power resources, this performs a true 
> D3cold
> + * power cycle (actual power removal). On platforms without _PR3 
> support,
> + * pci_set_power_state() automatically falls back to D3hot, providing a
> + * D3hot->D0 reset transition.
> + *
> + * Only available for single-function devices to avoid affecting other
> + * functions in multi-function devices.
> + *
> + * Returns 0 if device can be/was reset this way, -ENOTTY if not 
> supported,
> + * or other negative error code on failure.
> + */
> +static int pci_d3cold_reset(struct pci_dev *dev, bool probe)
> +{
> +	int ret;
> +
> +	if (dev->multifunction)
> +		return -ENOTTY;
> +
> +	if (probe) {
> +		if (!pci_pr3_present(dev))
> +			pci_dbg(dev, "d3cold reset: no _PR3 support, will use D3hot 
> fallback\n");

This fall through is invalid, if D3cold can't be reached the reset should be handled by pci_pm_reset() where NoSoftRst is honored.  pci_pr3_preset() should be tested in all cases, not just probe.  I think we also need to test device flags that would prevent a D3cold transition, like pci_pm_reset().  Thanks,

Alex

> +		return 0;
> +	}
> +
> +	if (dev->current_state != PCI_D0)
> +		return -EINVAL;
> +
> +	ret = pci_set_power_state(dev, PCI_D3cold);
> +	if (ret)
> +		return ret;
> +
> +	return pci_set_power_state(dev, PCI_D0);
> +}
> +
>  /**
>   * pcie_wait_for_link_status - Wait for link status change
>   * @pdev: Device whose link to wait for.
> @@ -5065,6 +5105,7 @@ const struct pci_reset_fn_method 
> pci_reset_fn_methods[] = {
>  	{ pci_pm_reset, .name = "pm" },
>  	{ pci_reset_bus_function, .name = "bus" },
>  	{ cxl_reset_bus_function, .name = "cxl_bus" },
> +	{ pci_d3cold_reset, .name = "d3cold" },
>  };
> 
>  /**
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 2c4454583c11..1ca7b880ead7 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -51,7 +51,7 @@
>  			       PCI_STATUS_PARITY)
> 
>  /* Number of reset methods used in pci_reset_fn_methods array in pci.c */
> -#define PCI_NUM_RESET_METHODS 8
> +#define PCI_NUM_RESET_METHODS 9
> 
>  #define PCI_RESET_PROBE		true
>  #define PCI_RESET_DO_RESET	false
> -- 
> 2.53.0

Re: [PATCH v3 1/2] PCI: Add D3cold as general reset method

[Lukas Wunner]

On Wed, May 13, 2026 at 02:23:48PM +0200, Jose Ignacio Tornos Martinez wrote:
> Add D3cold power cycle as a general PCI reset method, available for
> single-function devices. This provides a more robust reset mechanism
> than D3hot for devices where a full power cycle is beneficial.
> 
> The implementation uses pci_set_power_state(dev, PCI_D3cold), which
> automatically handles platform differences:
> - Platforms WITH _PR3 ACPI power resources: true D3cold (power cycle)
> - Platforms WITHOUT _PR3: automatic fallback to D3hot transition

Why fall back to D3hot, even though the name of the reset method
suggests it uses D3cold?  If D3cold isn't supported, I'd just not
expose the new reset method.

> Reset hierarchy with this change:
> 1. device_specific
> 2. acpi
> 3. flr
> 4. af_flr
> 5. pm (D3hot via config space)
> 6. bus (SBR)
> 7. cxl_bus
> 8. d3cold (NEW - power cycle with D3hot fallback)

I'd call it "powercycle" for clarity but that's just my preferred
bikeshed color.

> +static int pci_d3cold_reset(struct pci_dev *dev, bool probe)
> +{
> +	int ret;
> +
> +	if (dev->multifunction)
> +		return -ENOTTY;
> +
> +	if (probe) {
> +		if (!pci_pr3_present(dev))
> +			pci_dbg(dev, "d3cold reset: no _PR3 support, will use D3hot fallback\n");
> +		return 0;
> +	}

So I'd just return -ENOTTY here in the !pci_pr3_present() case
and I'd do that for probe == 0 as well.

Thanks,

Lukas

Re: [PATCH v3 1/2] PCI: Add D3cold as general reset method

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
- [Critical] Silent reset failure and state leak via NoSoftRst+ bypass
- [High] Missing IOMMU coordination during reset sequence
- [High] Missing device readiness wait (CRS polling) after power transition
- [High] Catastrophic power loss for downstream hierarchy on single-function bridges
- [Medium] False advertisement of reset support in probe mode
--

commit 372c424ac498f1ceef866b966b4539db619980ea
Author: Jose Ignacio Tornos Martinez <jtornosm@redhat.com>

PCI: Add D3cold as general reset method

This commit adds a D3cold power cycle as a general PCI reset method for
single-function devices. It implements pci_d3cold_reset() which attempts to
transition the device through D3cold and back to D0, falling back to D3hot if
ACPI _PR3 is unavailable.

> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 8f7cfcc000901..6da8feff7ccc3 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c

[ ... ]

> @@ -4491,6 +4491,46 @@ static int pci_pm_reset(struct pci_dev *dev, bool probe)
>  	return ret;
>  }
>  
> +/**
> + * pci_d3cold_reset - Put device into D3cold (or D3hot) and back to D0 for reset
> + * @dev: PCI device to reset
> + * @probe: if true, check if D3cold reset is supported; if false, perform reset
> + *
> + * Attempt to reset the device by transitioning through D3cold and back to D0.
> + * On platforms with ACPI _PR3 power resources, this performs a true D3cold
> + * power cycle (actual power removal). On platforms without _PR3 support,
> + * pci_set_power_state() automatically falls back to D3hot, providing a
> + * D3hot->D0 reset transition.
> + *
> + * Only available for single-function devices to avoid affecting other
> + * functions in multi-function devices.
> + *
> + * Returns 0 if device can be/was reset this way, -ENOTTY if not supported,
> + * or other negative error code on failure.
> + */
> +static int pci_d3cold_reset(struct pci_dev *dev, bool probe)
> +{
> +	int ret;
> +
> +	if (dev->multifunction)
> +		return -ENOTTY;

Many PCIe bridges (like Root Ports and Downstream Switch Ports) are
single-function devices.

If pci_d3cold_reset() is triggered on a single-function bridge,
pci_set_power_state() will physically shut off power to the entire downstream
hierarchy. Because pci_reset_function() is intended to reset only the target
function, does this unexpectedly drop power to subordinate devices without
unbinding their drivers or saving their state?

Should there be a check for dev->subordinate to prevent execution on bridges?

> +
> +	if (probe) {
> +		if (!pci_pr3_present(dev))
> +			pci_dbg(dev, "d3cold reset: no _PR3 support, will use D3hot fallback\n");
> +		return 0;

When called with probe = true, this unconditionally returns 0 if the device
is single-function, even if it lacks both ACPI _PR3 and native PM
capabilities (dev->pm_cap is not checked).

Could this incorrectly register d3cold as a supported reset method for hardware
that physically cannot change power states?

> +	}
> +
> +	if (dev->current_state != PCI_D0)
> +		return -EINVAL;
> +
> +	ret = pci_set_power_state(dev, PCI_D3cold);

Other PCI reset methods in the kernel bracket their operations with IOMMU
coordination. For example:

static int pci_pm_reset(struct pci_dev *dev, bool probe)
{
        ...
        ret = pci_dev_reset_iommu_prepare(dev);
        ...
        pci_dev_reset_iommu_done(dev);
        return ret;
}

By omitting these calls, does pci_d3cold_reset() allow power to be removed
while the device might still have in-flight DMA or active IOMMU mappings,
potentially triggering spurious DMA faults or fatal IOMMU errors?

> +	if (ret)
> +		return ret;
> +
> +	return pci_set_power_state(dev, PCI_D0);

When true D3cold is unavailable, pci_set_power_state() transparently falls
back to putting the device into D3hot. If the device advertises the
PCI_PM_CTRL_NO_SOFT_RESET bit, a D3hot->D0 transition explicitly does not
reset the device's internal state.

Other reset methods like pci_pm_reset() explicitly check this:

static int pci_pm_reset(struct pci_dev *dev, bool probe)
{
        ...
        pci_read_config_word(dev, dev->pm_cap + PCI_PM_CTRL, &csr);
        if (csr & PCI_PM_CTRL_NO_SOFT_RESET)
                return -ENOTTY;
        ...
}

Since pci_d3cold_reset() does not check this bit and returns 0 on success,
does this falsely report a successful reset to subsystems like VFIO that rely
on this to wipe guest state, potentially leaking state across VM boundaries?

Additionally, returning a device from D3cold or D3hot to D0 constitutes a
Conventional Reset, where devices may require up to 1 second to reinitialize
and return Configuration Request Retry Status (CRS).

Existing methods ensure readiness before returning:

static int pci_pm_reset(struct pci_dev *dev, bool probe)
{
        ...
        ret = pci_dev_wait(dev, "PM D3hot->D0", PCIE_RESET_READY_POLL_MS);
        ...
        return ret;
}

Does returning immediately from pci_set_power_state() mean the caller might
instantly invoke pci_dev_restore() while the hardware is still unresponsive?

> +}

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260513122349.268753-1-jtornosm@redhat.com?part=1

Re: [PATCH v3 1/2] PCI: Add D3cold as general reset method

[Jose Ignacio Tornos Martinez]

Hello Alex,

Thank you again for your feedbak.

> This fall through is invalid, if D3cold can't be reached the reset should
> be handled by pci_pm_reset() where NoSoftRst is honored.  pci_pr3_preset()
> should be tested in all cases, not just probe.  I think we also need to
> test device flags that would prevent a D3cold transition,
> like pci_pm_reset().  Thanks,
Ok, I've implemented it as a strict method (only when _PR3 is present) as
you suggested for the next version. I think it can be very helpful.

However, I'm facing a challenge with a part of our deployment scenario and
if possible I would appreciate your guidance (spoiler: d3hot possibility
would be very helpful).

Let me comment on about this setup in our lab:
- Qualcomm WiFi cards (ath11k 17cb:1103, ath12k 17cb:1107) and modems
  (SDX62/SDX65 17cb:0308)
- M.2 format devices using passive PCIe adapters (not native M.2 slots)
Problem:
- No FLR capability on devices
- NoSoftRst+ set (blocks PM reset)
- SBR broken through passive adapters (PERST# signal not properly wired)
- No _PR3 ACPI resources for standard PCIe slots (only for native M.2/
  Thunderbolt)
- d3cold_allowed=1 but pci_pr3_present() returns false
Result:
Default kernel has no working reset method (reset_methods shows only "bus"
which doesn't work).

I've tested the D3hot transition behavior and this is what I see before/
after:
- Command register: Cleared to 0x0000 (device disabled)
- BARs: Preserved (0x84200004 unchanged)
- Device successfully reinitializes and works in VFIO passthrough, so we
  can use it.

I don't dare to generalize D3hot in pci_pm_reset() with something like:
if (csr & PCI_PM_CTRL_NO_SOFT_RESET) {
  pci_read_config_word(dev, dev->pm_cap + PCI_PM_PMC, &pmc);

  /* If device supports D3hot, try reset despite NoSoftRst */
  if (!pmc & PCI_PM_CAP_D3HOT_MASK)
    return -ENOTTY;
}
because some devices could truly advertise NoSoftRst+ and d3hot could not be
usable as a reset.

I don't know if some device-specific quirk could be acceptable to bypass
NoSoftRst check for these Qualcomm devices (similar to
PCI_DEV_FLAGS_FORCE_PM_RESET from v2), although d3hot is usable in these
cases.

Or maybe the generalization could be to create a user parameter to bypass
PCI_PM_CTRL_NO_SOFT_RESET for the configured device. This would be manual but
at least it would allow the device to work on VMs.

And finally, what I would choose: move pm to the end of the resets methods
ignoring NoSoftRst, prioritizing hardware resets first (FLR, bus, d3cold)
and using d3hot as the last resource if none of the others work. But this is
an important change and there could be some reasons for pm positioning that
I ignore, so if you don't mind I would like to consult you first.

What do you think? Could you help me?
I would like to address these devices on VMs in some way. 

Thanks

Best regards
José Ignacio

Re: [PATCH v3 1/2] PCI: Add D3cold as general reset method

[Jose Ignacio Tornos Martinez]

Hello Lukas,

> Why fall back to D3hot, even though the name of the reset method
> suggests it uses D3cold?  If D3cold isn't supported, I'd just not
> expose the new reset method.
Ok, the next version will include D3cold only if supported.

> I'd call it "powercycle" for clarity but that's just my preferred
> bikeshed color.
I don't have any preference, although we can understand clearer what
it is. If there are consensus, I am ok with powercycle.

> So I'd just return -ENOTTY here in the !pci_pr3_present() case
> and I'd do that for probe == 0 as well.
Ok, I am going to wait for Alex Williamson opinion for the other cases
but I will do this as you say in the next version.

Thanks

Best regards
José Ignacio

Re: [PATCH v3 1/2] PCI: Add D3cold as general reset method

[Alex Williamson]

On Thu, 14 May 2026 14:30:58 +0200
Jose Ignacio Tornos Martinez <jtornosm@redhat.com> wrote:
> 
> > I'd call it "powercycle" for clarity but that's just my preferred
> > bikeshed color.  
> I don't have any preference, although we can understand clearer what
> it is. If there are consensus, I am ok with powercycle.

I think "powercycle" significantly expands the scope vs what is being
proposed here.  This is specifically only testing one method available
for slot power, the one used for D3cold.  If targeted more generically
as "powercycle", that would logically imply support for other slot
power control mechanisms, such as for hotplug slots.  Thanks,

Alex