From: Alex Williamson <alex.williamson@nvidia.com>
To: "Ionut Nechita (Wind River)" <ionut.nechita@windriver.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>,
helgaas@kernel.org, linux-pci@vger.kernel.org,
linux-kernel@vger.kernel.org, stable@vger.kernel.org,
Dave Jiang <dave.jiang@intel.com>,
Alex Williamson <alex.williamson@redhat.com>,
lukas@wunner.de, sebott@linux.ibm.com, schnelle@linux.ibm.com,
bblock@linux.ibm.com, linux@roeck-us.net,
matthew.brost@intel.com, michal.wajdeczko@intel.com,
piotr.piorkowski@intel.com, dtatulea@nvidia.com, mani@kernel.org,
kbusch@kernel.org, lkml@mageta.org, alifm@linux.ibm.com,
julianr@linux.ibm.com, ionut_n2001@yahoo.com,
sunlightlinux@gmail.com
Subject: Re: [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock
Date: Thu, 19 Nov 2026 14:32:27 -0800 [thread overview]
Message-ID: <20261119143227.7c00a09e@nvidia.com> (raw)
In-Reply-To: <20261119123504.25906-2-ionut.nechita@windriver.com>
On Thu, 19 Nov 2026 14:35:04 +0200
"Ionut Nechita (Wind River)" <ionut.nechita@windriver.com> wrote:
> From: Ionut Nechita <ionut.nechita@windriver.com>
>
> pci_reset_function() locks the upstream bridge with pci_dev_lock(),
> which takes the bridge's device lock and then blocks config access to
> the bridge, and only then takes the device lock of the function being
> reset. If another task holds that device lock, the reset sleeps with
> config access to the bridge blocked, and every user config read of the
> bridge sleeps in pci_wait_cfg() for as long.
>
> This happens with vfio-pci when a userspace driver keeps a VF open:
>
> 1. A driver unbind of the VF waits in vfio_unregister_group_dev()
> for the user to release the device, holding the VF device lock.
> 2. Writing 0 to sriov_numvfs on the PF takes the PF device lock and
> waits in sriov_disable() for the VF device lock.
> 3. Writing 1 to the PF "reset" attribute calls pci_reset_function(),
> which blocks config access to the PF's upstream bridge and waits
> for the PF device lock.
>
> From then on lspci, which reads the config space of every device,
> hangs in D state, and so does any tool or agent that runs it, until
> the userspace driver releases the VF:
>
> lspci D
> pci_wait_cfg+0x60/0xc0
> pci_user_read_config_dword+0xcc/0xf0
> pci_read_config+0xf5/0x290
> kernfs_fop_read_iter+0xb6/0x1b0
> vfs_read+0x21f/0x340
>
> This was hit with an Intel FPGA PAC N3000 FEC device bound to vfio-pci
> (PF and VFs), where the SR-IOV FEC operator reconfigures the PF while a
> DPDK application holds one of its VFs.
>
> Take the device locks first, bridge before device as before, and block
> config access to the bridge and the device only once both are held.
> The order between the two device locks is unchanged. Config access is
> now blocked only while the reset runs, not while waiting for a lock.
>
> The reset itself still waits for the device lock, so the tasks in
> steps 1-3 stay blocked until the user releases the VF. Only config
> reads of unrelated callers stop being blocked.
>
> Fixes: 7e89efc6e9e4 ("PCI: Lock upstream bridge for pci_reset_function()")
> Cc: stable@vger.kernel.org
> Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>
> ---
> drivers/pci/pci.c | 24 ++++++++++++++++++++----
> 1 file changed, 20 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index d79f73b28..9c8d5dda9 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -5345,19 +5345,35 @@ int pci_reset_function(struct pci_dev *dev)
> * no upstream bridge configuration to hold consistent.
> */
> bridge = pci_upstream_bridge(dev);
> +
> + /*
> + * Take the device locks, bridge before device, before blocking
> + * config access to either of them. device_lock() can sleep for as
> + * long as another task holds it, e.g. a driver unbind waiting for
> + * a vfio user to release the device. If config access to the
> + * bridge were already blocked at that point, every config read of
> + * the bridge (lspci, sysfs "config") would sleep in pci_wait_cfg()
> + * for the same time.
> + */
> if (bridge)
> - pci_dev_lock(bridge);
> + device_lock(&bridge->dev);
> + device_lock(&dev->dev);
> + if (bridge)
> + pci_cfg_access_lock(bridge);
> + pci_cfg_access_lock(dev);
>
> - pci_dev_lock(dev);
> pci_dev_save_and_disable(dev);
>
> rc = __pci_reset_function_locked(dev);
>
> pci_dev_restore(dev);
> - pci_dev_unlock(dev);
>
> + pci_cfg_access_unlock(dev);
> if (bridge)
> - pci_dev_unlock(bridge);
> + pci_cfg_access_unlock(bridge);
> + device_unlock(&dev->dev);
> + if (bridge)
> + device_unlock(&bridge->dev);
>
> return rc;
> }
Reviewed-by: Alex Williamson <alex.williamson@nvidia.com>
prev parent reply other threads:[~2026-11-19 22:32 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 12:35 [PATCH 0/1] PCI: Fix lspci hang when a reset waits for a vfio-held device Ionut Nechita (Wind River)
2026-11-19 12:35 ` [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock Ionut Nechita (Wind River)
2026-11-19 22:32 ` Alex Williamson [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119143227.7c00a09e@nvidia.com \
--to=alex.williamson@nvidia.com \
--cc=alex.williamson@redhat.com \
--cc=alifm@linux.ibm.com \
--cc=bblock@linux.ibm.com \
--cc=bhelgaas@google.com \
--cc=dave.jiang@intel.com \
--cc=dtatulea@nvidia.com \
--cc=helgaas@kernel.org \
--cc=ionut.nechita@windriver.com \
--cc=ionut_n2001@yahoo.com \
--cc=julianr@linux.ibm.com \
--cc=kbusch@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux@roeck-us.net \
--cc=lkml@mageta.org \
--cc=lukas@wunner.de \
--cc=mani@kernel.org \
--cc=matthew.brost@intel.com \
--cc=michal.wajdeczko@intel.com \
--cc=piotr.piorkowski@intel.com \
--cc=schnelle@linux.ibm.com \
--cc=sebott@linux.ibm.com \
--cc=stable@vger.kernel.org \
--cc=sunlightlinux@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox