* [PATCH 0/1] PCI: Fix lspci hang when a reset waits for a vfio-held device
@ 2026-11-19 12:35 Ionut Nechita (Wind River)
2026-11-19 12:35 ` [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock Ionut Nechita (Wind River)
0 siblings, 1 reply; 3+ messages in thread
From: Ionut Nechita (Wind River) @ 2026-11-19 12:35 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: helgaas, linux-pci, linux-kernel, stable, Dave Jiang,
Alex Williamson, lukas, sebott, schnelle, bblock, linux,
matthew.brost, michal.wajdeczko, piotr.piorkowski, dtatulea, mani,
kbusch, lkml, alifm, julianr, ionut_n2001, sunlightlinux,
Ionut Nechita
From: Ionut Nechita <ionut.nechita@windriver.com>
Hi Bjorn,
When a function reset has to wait for the device lock of the function
being reset, pci_reset_function() keeps config access to the upstream
bridge blocked for the whole wait. If the device lock is held for a long
time, every config read of that bridge sleeps in pci_wait_cfg(), so
lspci, and any tool or agent that runs it, hangs in D state.
We hit this with vfio-pci: a userspace driver keeps a VF open, and a
driver unbind of that VF, a write to sriov_numvfs on the PF and a write
to the PF's "reset" attribute queue up behind each other on the device
locks. The reset is the one that leaves config access to the bridge
blocked. The patch takes both device locks before blocking config
access, so config access is only blocked while the reset itself runs.
The lock order is unchanged.
Testing:
- Reproduced on 6.18.59 (PREEMPT_RT) with an Intel FPGA PAC N3000, PF
and VFs on vfio-pci, one VF held open by a minimal vfio user (opens
the group and gets the device fd). After the unbind, sriov_numvfs=0
and reset writes, a read of the bridge's config file and lspci hang
in pci_wait_cfg() without the patch and complete with it. Killing
the vfio user releases the blocked writes as before.
- Build-tested on pci/next and next-20261118, with and without
PREEMPT_RT (W=1).
Not addressed here:
- __pci_bus_lock() (bus and slot reset) has the same pattern: it blocks
config access to the bridge before taking the device locks below it.
I kept this patch minimal; I can send a follow-up if the approach
looks right.
- The sriov_numvfs write in the scenario above holds
pci_rescan_remove_lock while it waits, so a PCI rescan or another
sriov_numvfs write also waits until the VF is released. That is a
separate issue and not changed by this patch.
Thanks,
Ionut
Ionut Nechita (1):
PCI: Don't block bridge config access while waiting for the device
lock
drivers/pci/pci.c | 24 ++++++++++++++++++++----
1 file changed, 20 insertions(+), 4 deletions(-)
--
2.55.0
^ permalink raw reply [flat|nested] 3+ messages in thread* [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock
2026-11-19 12:35 [PATCH 0/1] PCI: Fix lspci hang when a reset waits for a vfio-held device Ionut Nechita (Wind River)
@ 2026-11-19 12:35 ` Ionut Nechita (Wind River)
2026-11-19 22:32 ` Alex Williamson
0 siblings, 1 reply; 3+ messages in thread
From: Ionut Nechita (Wind River) @ 2026-11-19 12:35 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: helgaas, linux-pci, linux-kernel, stable, Dave Jiang,
Alex Williamson, lukas, sebott, schnelle, bblock, linux,
matthew.brost, michal.wajdeczko, piotr.piorkowski, dtatulea, mani,
kbusch, lkml, alifm, julianr, ionut_n2001, sunlightlinux,
Ionut Nechita
From: Ionut Nechita <ionut.nechita@windriver.com>
pci_reset_function() locks the upstream bridge with pci_dev_lock(),
which takes the bridge's device lock and then blocks config access to
the bridge, and only then takes the device lock of the function being
reset. If another task holds that device lock, the reset sleeps with
config access to the bridge blocked, and every user config read of the
bridge sleeps in pci_wait_cfg() for as long.
This happens with vfio-pci when a userspace driver keeps a VF open:
1. A driver unbind of the VF waits in vfio_unregister_group_dev()
for the user to release the device, holding the VF device lock.
2. Writing 0 to sriov_numvfs on the PF takes the PF device lock and
waits in sriov_disable() for the VF device lock.
3. Writing 1 to the PF "reset" attribute calls pci_reset_function(),
which blocks config access to the PF's upstream bridge and waits
for the PF device lock.
From then on lspci, which reads the config space of every device,
hangs in D state, and so does any tool or agent that runs it, until
the userspace driver releases the VF:
lspci D
pci_wait_cfg+0x60/0xc0
pci_user_read_config_dword+0xcc/0xf0
pci_read_config+0xf5/0x290
kernfs_fop_read_iter+0xb6/0x1b0
vfs_read+0x21f/0x340
This was hit with an Intel FPGA PAC N3000 FEC device bound to vfio-pci
(PF and VFs), where the SR-IOV FEC operator reconfigures the PF while a
DPDK application holds one of its VFs.
Take the device locks first, bridge before device as before, and block
config access to the bridge and the device only once both are held.
The order between the two device locks is unchanged. Config access is
now blocked only while the reset runs, not while waiting for a lock.
The reset itself still waits for the device lock, so the tasks in
steps 1-3 stay blocked until the user releases the VF. Only config
reads of unrelated callers stop being blocked.
Fixes: 7e89efc6e9e4 ("PCI: Lock upstream bridge for pci_reset_function()")
Cc: stable@vger.kernel.org
Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>
---
drivers/pci/pci.c | 24 ++++++++++++++++++++----
1 file changed, 20 insertions(+), 4 deletions(-)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index d79f73b28..9c8d5dda9 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -5345,19 +5345,35 @@ int pci_reset_function(struct pci_dev *dev)
* no upstream bridge configuration to hold consistent.
*/
bridge = pci_upstream_bridge(dev);
+
+ /*
+ * Take the device locks, bridge before device, before blocking
+ * config access to either of them. device_lock() can sleep for as
+ * long as another task holds it, e.g. a driver unbind waiting for
+ * a vfio user to release the device. If config access to the
+ * bridge were already blocked at that point, every config read of
+ * the bridge (lspci, sysfs "config") would sleep in pci_wait_cfg()
+ * for the same time.
+ */
if (bridge)
- pci_dev_lock(bridge);
+ device_lock(&bridge->dev);
+ device_lock(&dev->dev);
+ if (bridge)
+ pci_cfg_access_lock(bridge);
+ pci_cfg_access_lock(dev);
- pci_dev_lock(dev);
pci_dev_save_and_disable(dev);
rc = __pci_reset_function_locked(dev);
pci_dev_restore(dev);
- pci_dev_unlock(dev);
+ pci_cfg_access_unlock(dev);
if (bridge)
- pci_dev_unlock(bridge);
+ pci_cfg_access_unlock(bridge);
+ device_unlock(&dev->dev);
+ if (bridge)
+ device_unlock(&bridge->dev);
return rc;
}
--
2.55.0
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock
2026-11-19 12:35 ` [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock Ionut Nechita (Wind River)
@ 2026-11-19 22:32 ` Alex Williamson
0 siblings, 0 replies; 3+ messages in thread
From: Alex Williamson @ 2026-11-19 22:32 UTC (permalink / raw)
To: Ionut Nechita (Wind River)
Cc: Bjorn Helgaas, helgaas, linux-pci, linux-kernel, stable,
Dave Jiang, Alex Williamson, lukas, sebott, schnelle, bblock,
linux, matthew.brost, michal.wajdeczko, piotr.piorkowski,
dtatulea, mani, kbusch, lkml, alifm, julianr, ionut_n2001,
sunlightlinux
On Thu, 19 Nov 2026 14:35:04 +0200
"Ionut Nechita (Wind River)" <ionut.nechita@windriver.com> wrote:
> From: Ionut Nechita <ionut.nechita@windriver.com>
>
> pci_reset_function() locks the upstream bridge with pci_dev_lock(),
> which takes the bridge's device lock and then blocks config access to
> the bridge, and only then takes the device lock of the function being
> reset. If another task holds that device lock, the reset sleeps with
> config access to the bridge blocked, and every user config read of the
> bridge sleeps in pci_wait_cfg() for as long.
>
> This happens with vfio-pci when a userspace driver keeps a VF open:
>
> 1. A driver unbind of the VF waits in vfio_unregister_group_dev()
> for the user to release the device, holding the VF device lock.
> 2. Writing 0 to sriov_numvfs on the PF takes the PF device lock and
> waits in sriov_disable() for the VF device lock.
> 3. Writing 1 to the PF "reset" attribute calls pci_reset_function(),
> which blocks config access to the PF's upstream bridge and waits
> for the PF device lock.
>
> From then on lspci, which reads the config space of every device,
> hangs in D state, and so does any tool or agent that runs it, until
> the userspace driver releases the VF:
>
> lspci D
> pci_wait_cfg+0x60/0xc0
> pci_user_read_config_dword+0xcc/0xf0
> pci_read_config+0xf5/0x290
> kernfs_fop_read_iter+0xb6/0x1b0
> vfs_read+0x21f/0x340
>
> This was hit with an Intel FPGA PAC N3000 FEC device bound to vfio-pci
> (PF and VFs), where the SR-IOV FEC operator reconfigures the PF while a
> DPDK application holds one of its VFs.
>
> Take the device locks first, bridge before device as before, and block
> config access to the bridge and the device only once both are held.
> The order between the two device locks is unchanged. Config access is
> now blocked only while the reset runs, not while waiting for a lock.
>
> The reset itself still waits for the device lock, so the tasks in
> steps 1-3 stay blocked until the user releases the VF. Only config
> reads of unrelated callers stop being blocked.
>
> Fixes: 7e89efc6e9e4 ("PCI: Lock upstream bridge for pci_reset_function()")
> Cc: stable@vger.kernel.org
> Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>
> ---
> drivers/pci/pci.c | 24 ++++++++++++++++++++----
> 1 file changed, 20 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index d79f73b28..9c8d5dda9 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -5345,19 +5345,35 @@ int pci_reset_function(struct pci_dev *dev)
> * no upstream bridge configuration to hold consistent.
> */
> bridge = pci_upstream_bridge(dev);
> +
> + /*
> + * Take the device locks, bridge before device, before blocking
> + * config access to either of them. device_lock() can sleep for as
> + * long as another task holds it, e.g. a driver unbind waiting for
> + * a vfio user to release the device. If config access to the
> + * bridge were already blocked at that point, every config read of
> + * the bridge (lspci, sysfs "config") would sleep in pci_wait_cfg()
> + * for the same time.
> + */
> if (bridge)
> - pci_dev_lock(bridge);
> + device_lock(&bridge->dev);
> + device_lock(&dev->dev);
> + if (bridge)
> + pci_cfg_access_lock(bridge);
> + pci_cfg_access_lock(dev);
>
> - pci_dev_lock(dev);
> pci_dev_save_and_disable(dev);
>
> rc = __pci_reset_function_locked(dev);
>
> pci_dev_restore(dev);
> - pci_dev_unlock(dev);
>
> + pci_cfg_access_unlock(dev);
> if (bridge)
> - pci_dev_unlock(bridge);
> + pci_cfg_access_unlock(bridge);
> + device_unlock(&dev->dev);
> + if (bridge)
> + device_unlock(&bridge->dev);
>
> return rc;
> }
Reviewed-by: Alex Williamson <alex.williamson@nvidia.com>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2026-11-19 22:32 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-19 12:35 [PATCH 0/1] PCI: Fix lspci hang when a reset waits for a vfio-held device Ionut Nechita (Wind River)
2026-11-19 12:35 ` [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock Ionut Nechita (Wind River)
2026-11-19 22:32 ` Alex Williamson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox