Linux PCI subsystem development
 help / color / mirror / Atom feed
* [PATCH 0/1] PCI: Fix lspci hang when a reset waits for a vfio-held device
@ 2026-11-19 12:35 Ionut Nechita (Wind River)
  2026-11-19 12:35 ` [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock Ionut Nechita (Wind River)
  0 siblings, 1 reply; 3+ messages in thread
From: Ionut Nechita (Wind River) @ 2026-11-19 12:35 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: helgaas, linux-pci, linux-kernel, stable, Dave Jiang,
	Alex Williamson, lukas, sebott, schnelle, bblock, linux,
	matthew.brost, michal.wajdeczko, piotr.piorkowski, dtatulea, mani,
	kbusch, lkml, alifm, julianr, ionut_n2001, sunlightlinux,
	Ionut Nechita

From: Ionut Nechita <ionut.nechita@windriver.com>

Hi Bjorn,

When a function reset has to wait for the device lock of the function
being reset, pci_reset_function() keeps config access to the upstream
bridge blocked for the whole wait. If the device lock is held for a long
time, every config read of that bridge sleeps in pci_wait_cfg(), so
lspci, and any tool or agent that runs it, hangs in D state.

We hit this with vfio-pci: a userspace driver keeps a VF open, and a
driver unbind of that VF, a write to sriov_numvfs on the PF and a write
to the PF's "reset" attribute queue up behind each other on the device
locks. The reset is the one that leaves config access to the bridge
blocked. The patch takes both device locks before blocking config
access, so config access is only blocked while the reset itself runs.
The lock order is unchanged.

Testing:
- Reproduced on 6.18.59 (PREEMPT_RT) with an Intel FPGA PAC N3000, PF
  and VFs on vfio-pci, one VF held open by a minimal vfio user (opens
  the group and gets the device fd). After the unbind, sriov_numvfs=0
  and reset writes, a read of the bridge's config file and lspci hang
  in pci_wait_cfg() without the patch and complete with it. Killing
  the vfio user releases the blocked writes as before.
- Build-tested on pci/next and next-20261118, with and without
  PREEMPT_RT (W=1).

Not addressed here:
- __pci_bus_lock() (bus and slot reset) has the same pattern: it blocks
  config access to the bridge before taking the device locks below it.
  I kept this patch minimal; I can send a follow-up if the approach
  looks right.
- The sriov_numvfs write in the scenario above holds
  pci_rescan_remove_lock while it waits, so a PCI rescan or another
  sriov_numvfs write also waits until the VF is released. That is a
  separate issue and not changed by this patch.

Thanks,
Ionut

Ionut Nechita (1):
  PCI: Don't block bridge config access while waiting for the device
    lock

 drivers/pci/pci.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)

-- 
2.55.0


^ permalink raw reply	[flat|nested] 3+ messages in thread

* [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock
  2026-11-19 12:35 [PATCH 0/1] PCI: Fix lspci hang when a reset waits for a vfio-held device Ionut Nechita (Wind River)
@ 2026-11-19 12:35 ` Ionut Nechita (Wind River)
  2026-11-19 12:43   ` sashiko-bot
  0 siblings, 1 reply; 3+ messages in thread
From: Ionut Nechita (Wind River) @ 2026-11-19 12:35 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: helgaas, linux-pci, linux-kernel, stable, Dave Jiang,
	Alex Williamson, lukas, sebott, schnelle, bblock, linux,
	matthew.brost, michal.wajdeczko, piotr.piorkowski, dtatulea, mani,
	kbusch, lkml, alifm, julianr, ionut_n2001, sunlightlinux,
	Ionut Nechita

From: Ionut Nechita <ionut.nechita@windriver.com>

pci_reset_function() locks the upstream bridge with pci_dev_lock(),
which takes the bridge's device lock and then blocks config access to
the bridge, and only then takes the device lock of the function being
reset. If another task holds that device lock, the reset sleeps with
config access to the bridge blocked, and every user config read of the
bridge sleeps in pci_wait_cfg() for as long.

This happens with vfio-pci when a userspace driver keeps a VF open:

  1. A driver unbind of the VF waits in vfio_unregister_group_dev()
     for the user to release the device, holding the VF device lock.
  2. Writing 0 to sriov_numvfs on the PF takes the PF device lock and
     waits in sriov_disable() for the VF device lock.
  3. Writing 1 to the PF "reset" attribute calls pci_reset_function(),
     which blocks config access to the PF's upstream bridge and waits
     for the PF device lock.

From then on lspci, which reads the config space of every device,
hangs in D state, and so does any tool or agent that runs it, until
the userspace driver releases the VF:

  lspci           D
    pci_wait_cfg+0x60/0xc0
    pci_user_read_config_dword+0xcc/0xf0
    pci_read_config+0xf5/0x290
    kernfs_fop_read_iter+0xb6/0x1b0
    vfs_read+0x21f/0x340

This was hit with an Intel FPGA PAC N3000 FEC device bound to vfio-pci
(PF and VFs), where the SR-IOV FEC operator reconfigures the PF while a
DPDK application holds one of its VFs.

Take the device locks first, bridge before device as before, and block
config access to the bridge and the device only once both are held.
The order between the two device locks is unchanged. Config access is
now blocked only while the reset runs, not while waiting for a lock.

The reset itself still waits for the device lock, so the tasks in
steps 1-3 stay blocked until the user releases the VF. Only config
reads of unrelated callers stop being blocked.

Fixes: 7e89efc6e9e4 ("PCI: Lock upstream bridge for pci_reset_function()")
Cc: stable@vger.kernel.org
Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>
---
 drivers/pci/pci.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index d79f73b28..9c8d5dda9 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -5345,19 +5345,35 @@ int pci_reset_function(struct pci_dev *dev)
 	 * no upstream bridge configuration to hold consistent.
 	 */
 	bridge = pci_upstream_bridge(dev);
+
+	/*
+	 * Take the device locks, bridge before device, before blocking
+	 * config access to either of them.  device_lock() can sleep for as
+	 * long as another task holds it, e.g. a driver unbind waiting for
+	 * a vfio user to release the device.  If config access to the
+	 * bridge were already blocked at that point, every config read of
+	 * the bridge (lspci, sysfs "config") would sleep in pci_wait_cfg()
+	 * for the same time.
+	 */
 	if (bridge)
-		pci_dev_lock(bridge);
+		device_lock(&bridge->dev);
+	device_lock(&dev->dev);
+	if (bridge)
+		pci_cfg_access_lock(bridge);
+	pci_cfg_access_lock(dev);
 
-	pci_dev_lock(dev);
 	pci_dev_save_and_disable(dev);
 
 	rc = __pci_reset_function_locked(dev);
 
 	pci_dev_restore(dev);
-	pci_dev_unlock(dev);
 
+	pci_cfg_access_unlock(dev);
 	if (bridge)
-		pci_dev_unlock(bridge);
+		pci_cfg_access_unlock(bridge);
+	device_unlock(&dev->dev);
+	if (bridge)
+		device_unlock(&bridge->dev);
 
 	return rc;
 }
-- 
2.55.0


^ permalink raw reply related	[flat|nested] 3+ messages in thread

* Re: [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock
  2026-11-19 12:35 ` [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock Ionut Nechita (Wind River)
@ 2026-11-19 12:43   ` sashiko-bot
  0 siblings, 0 replies; 3+ messages in thread
From: sashiko-bot @ 2026-11-19 12:43 UTC (permalink / raw)
  To: Ionut Nechita (Wind River); +Cc: linux-pci

> From: Ionut Nechita <ionut.nechita@windriver.com>
> 
> pci_reset_function() locks the upstream bridge with pci_dev_lock(),
> which takes the bridge's device lock and then blocks config access to
> the bridge, and only then takes the device lock of the function being
> reset. If another task holds that device lock, the reset sleeps with
> config access to the bridge blocked, and every user config read of the
> bridge sleeps in pci_wait_cfg() for as long.
> 
> This happens with vfio-pci when a userspace driver keeps a VF open:
> 
>   1. A driver unbind of the VF waits in vfio_unregister_group_dev()
>      for the user to release the device, holding the VF device lock.
>   2. Writing 0 to sriov_numvfs on the PF takes the PF device lock and
>      waits in sriov_disable() for the VF device lock.
> [ ... ]
> reads of unrelated callers stop being blocked.
> 
> Fixes: 7e89efc6e9e4 ("PCI: Lock upstream bridge for pci_reset_function()")
> Cc: stable@vger.kernel.org
> Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119123504.25906-1-ionut.nechita@windriver.com?part=1


^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2026-11-19 12:43 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-19 12:35 [PATCH 0/1] PCI: Fix lspci hang when a reset waits for a vfio-held device Ionut Nechita (Wind River)
2026-11-19 12:35 ` [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock Ionut Nechita (Wind River)
2026-11-19 12:43   ` sashiko-bot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox