All of lore.kernel.org
 help / color / mirror / Atom feed
From: Bjorn Helgaas <helgaas@kernel.org>
To: "Ionut Nechita (Wind River)" <ionut.nechita@windriver.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>,
	linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
	stable@vger.kernel.org, Dave Jiang <dave.jiang@intel.com>,
	Alex Williamson <alex.williamson@redhat.com>,
	lukas@wunner.de, sebott@linux.ibm.com, schnelle@linux.ibm.com,
	bblock@linux.ibm.com, linux@roeck-us.net,
	matthew.brost@intel.com, michal.wajdeczko@intel.com,
	piotr.piorkowski@intel.com, dtatulea@nvidia.com, mani@kernel.org,
	kbusch@kernel.org, lkml@mageta.org, alifm@linux.ibm.com,
	julianr@linux.ibm.com, ionut_n2001@yahoo.com,
	sunlightlinux@gmail.com
Subject: Re: [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock
Date: Fri, 20 Nov 2026 10:48:47 -0600	[thread overview]
Message-ID: <20261120164847.GA887103@bhelgaas> (raw)
In-Reply-To: <20261119123504.25906-2-ionut.nechita@windriver.com>

On Thu, Nov 19, 2026 at 02:35:04PM +0200, Ionut Nechita (Wind River) wrote:
> From: Ionut Nechita <ionut.nechita@windriver.com>
> 
> pci_reset_function() locks the upstream bridge with pci_dev_lock(),
> which takes the bridge's device lock and then blocks config access to
> the bridge, and only then takes the device lock of the function being
> reset. If another task holds that device lock, the reset sleeps with
> config access to the bridge blocked, and every user config read of the
> bridge sleeps in pci_wait_cfg() for as long.
> 
> This happens with vfio-pci when a userspace driver keeps a VF open:
> 
>   1. A driver unbind of the VF waits in vfio_unregister_group_dev()
>      for the user to release the device, holding the VF device lock.
>   2. Writing 0 to sriov_numvfs on the PF takes the PF device lock and
>      waits in sriov_disable() for the VF device lock.
>   3. Writing 1 to the PF "reset" attribute calls pci_reset_function(),
>      which blocks config access to the PF's upstream bridge and waits
>      for the PF device lock.
> 
> From then on lspci, which reads the config space of every device,
> hangs in D state, and so does any tool or agent that runs it, until
> the userspace driver releases the VF:
> 
>   lspci           D
>     pci_wait_cfg+0x60/0xc0
>     pci_user_read_config_dword+0xcc/0xf0
>     pci_read_config+0xf5/0x290
>     kernfs_fop_read_iter+0xb6/0x1b0
>     vfs_read+0x21f/0x340
> 
> This was hit with an Intel FPGA PAC N3000 FEC device bound to vfio-pci
> (PF and VFs), where the SR-IOV FEC operator reconfigures the PF while a
> DPDK application holds one of its VFs.
> 
> Take the device locks first, bridge before device as before, and block
> config access to the bridge and the device only once both are held.
> The order between the two device locks is unchanged. Config access is
> now blocked only while the reset runs, not while waiting for a lock.
> 
> The reset itself still waits for the device lock, so the tasks in
> steps 1-3 stay blocked until the user releases the VF. Only config
> reads of unrelated callers stop being blocked.
> 
> Fixes: 7e89efc6e9e4 ("PCI: Lock upstream bridge for pci_reset_function()")
> Cc: stable@vger.kernel.org
> Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>

Applied to pci/reset for v7.5, thank you!

> ---
>  drivers/pci/pci.c | 24 ++++++++++++++++++++----
>  1 file changed, 20 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index d79f73b28..9c8d5dda9 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -5345,19 +5345,35 @@ int pci_reset_function(struct pci_dev *dev)
>  	 * no upstream bridge configuration to hold consistent.
>  	 */
>  	bridge = pci_upstream_bridge(dev);
> +
> +	/*
> +	 * Take the device locks, bridge before device, before blocking
> +	 * config access to either of them.  device_lock() can sleep for as
> +	 * long as another task holds it, e.g. a driver unbind waiting for
> +	 * a vfio user to release the device.  If config access to the
> +	 * bridge were already blocked at that point, every config read of
> +	 * the bridge (lspci, sysfs "config") would sleep in pci_wait_cfg()
> +	 * for the same time.
> +	 */
>  	if (bridge)
> -		pci_dev_lock(bridge);
> +		device_lock(&bridge->dev);
> +	device_lock(&dev->dev);
> +	if (bridge)
> +		pci_cfg_access_lock(bridge);
> +	pci_cfg_access_lock(dev);
>  
> -	pci_dev_lock(dev);
>  	pci_dev_save_and_disable(dev);
>  
>  	rc = __pci_reset_function_locked(dev);
>  
>  	pci_dev_restore(dev);
> -	pci_dev_unlock(dev);
>  
> +	pci_cfg_access_unlock(dev);
>  	if (bridge)
> -		pci_dev_unlock(bridge);
> +		pci_cfg_access_unlock(bridge);
> +	device_unlock(&dev->dev);
> +	if (bridge)
> +		device_unlock(&bridge->dev);
>  
>  	return rc;
>  }
> -- 
> 2.55.0
> 

      parent reply	other threads:[~2026-11-20 16:48 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 12:35 [PATCH 0/1] PCI: Fix lspci hang when a reset waits for a vfio-held device Ionut Nechita (Wind River)
2026-11-19 12:35 ` [PATCH 1/1] PCI: Don't block bridge config access while waiting for the device lock Ionut Nechita (Wind River)
2026-11-19 12:43   ` sashiko-bot
2026-11-19 22:32   ` Alex Williamson
2026-11-20 16:48   ` Bjorn Helgaas [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261120164847.GA887103@bhelgaas \
    --to=helgaas@kernel.org \
    --cc=alex.williamson@redhat.com \
    --cc=alifm@linux.ibm.com \
    --cc=bblock@linux.ibm.com \
    --cc=bhelgaas@google.com \
    --cc=dave.jiang@intel.com \
    --cc=dtatulea@nvidia.com \
    --cc=ionut.nechita@windriver.com \
    --cc=ionut_n2001@yahoo.com \
    --cc=julianr@linux.ibm.com \
    --cc=kbusch@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=linux@roeck-us.net \
    --cc=lkml@mageta.org \
    --cc=lukas@wunner.de \
    --cc=mani@kernel.org \
    --cc=matthew.brost@intel.com \
    --cc=michal.wajdeczko@intel.com \
    --cc=piotr.piorkowski@intel.com \
    --cc=schnelle@linux.ibm.com \
    --cc=sebott@linux.ibm.com \
    --cc=stable@vger.kernel.org \
    --cc=sunlightlinux@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.