All of lore.kernel.org
 help / color / mirror / Atom feed
From: Samiullah Khawaja <skhawaja@google.com>
To: Alex Williamson <alex@shazbot.org>
Cc: Bjorn Helgaas <bhelgaas@google.com>,
	Kevin Tian <kevin.tian@intel.com>,
	 Jason Gunthorpe <jgg@ziepe.ca>,
	Anthony Pighin <anthony.pighin@nokia.com>,
	 Dave Jiang <dave.jiang@intel.com>,
	Dan Williams <dan.j.williams@intel.com>,
	kvm@vger.kernel.org,  linux-pci@vger.kernel.org,
	linux-kernel@vger.kernel.org, stable@vger.kernel.org
Subject: Re: [PATCH] vfio/pci: Only lock the upstream bridge if the reset may need it
Date: Fri, 20 Nov 2026 22:21:16 +0000	[thread overview]
Message-ID: <awC5Ldqmue6DiO44@google.com> (raw)
In-Reply-To: <20261119143237.4bf54796@shazbot.org>

On Thu, Nov 19, 2026 at 02:32:37PM -0800, Alex Williamson wrote:
>On Wed, 18 Nov 2026 16:37:06 -0800
>Alex Williamson <alex@shazbot.org> wrote:
>
>> On Thu, 12 Nov 2026 23:57:00 +0000
>> Samiullah Khawaja <skhawaja@google.com> wrote:
>>
>> > Commit 962ae6892d8b ("vfio/pci: Lock upstream bridge for
>> > vfio_pci_core_disable()") makes vfio_pci_core_disable() trylock the
>> > upstream bridge before resetting the device, and skips the reset if the
>> > trylock fails.
>> >
>> > All VFs of a PF have the same upstream bridge. Take a device with two
>> > VFs, both used by one process. When the process is killed, both VFs can
>> > be released at the same time. The first one takes the bridge lock and
>> > does its FLR, the second one fails the trylock and is never reset.
>> >
>> > The bridge lock is only needed for a bus reset, which is done through
>> > the bridge. A VF is reset with FLR, and pci_parent_bus_reset() never
>> > resets the bus for it because the PF and the other VFs are on the same
>> > bus.
>> >
>> > Add pci_reset_needs_bridge_lock(), which returns true if any reset
>> > method enabled for the device is a bus reset, and only lock the bridge
>> > in vfio_pci_core_disable() when it does.
>> >
>> > Fixes: 962ae6892d8b ("vfio/pci: Lock upstream bridge for vfio_pci_core_disable()")
>> > Cc: stable@vger.kernel.org
>> > Signed-off-by: Samiullah Khawaja <skhawaja@google.com>
>> > ---
>> > Tested in QEMU with an igb PF behind a PCIe root port, bound to vfio-pci
>> > with two VFs. Both VFs are opened through VFIO and then closed from two
>> > threads at the same time, counting FLRs with a kprobe on pcie_flr().
>> > Without this patch one VF missed its FLR in 50 of 50 rounds, with it
>> > both VFs were reset in every round. I will send a VFIO selftest for this
>> > separately.
>> > ---
>> >  drivers/pci/pci.c                | 28 ++++++++++++++++++++++++++++
>> >  drivers/vfio/pci/vfio_pci_core.c |  3 ++-
>> >  include/linux/pci.h              |  1 +
>> >  3 files changed, 31 insertions(+), 1 deletion(-)
>> >

[snip]

>> > +
>> > +		if (pci_reset_fn_methods[m].reset_fn == pci_reset_bus_function ||
>> > +		    pci_reset_fn_methods[m].reset_fn == cxl_reset_bus_function)
>>
>> This looks like a maintenance nightmare.  What if instead we did
>> something like:
>>
>> bool pci_parent_bus_reset_possible(struct pci_dev *dev)
>> {
>> 	struct pci_dev *pdev;
>>
>> 	if (pci_is_root_bus(dev->bus) || dev->subordinate ||
>> 	    !dev->bus->self || dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET)
>> 		return false;
>>
>> 	list_for_each_entry(pdev, &dev->bus->devices, bus_list)
>> 		if (pdev != dev)
>> 			return false;
>>
>> 	return true;
>> }
>> EXPORT_SYMBOL_GPL(pci_parent_bus_reset_possible);
>>
>> static int pci_parent_bus_reset(struct pci_dev *dev, bool probe)
>> {
>> 	if (!pci_parent_bus_reset_possible(dev))
>> 		return -ENOTTY;
>>
>> 	if (probe)
>> 		return 0;
>>
>> 	return pci_bridge_secondary_bus_reset(dev->bus->self);
>> }
>>
>> Where we'd then condition acquiring the bridge lock on whether it's
>> even possible for a function scoped reset to perform a bus reset.
>>
>> It's a bit broader of a test than whether we actually have the necessary
>> methods in our reset_fn list, but since we're not locking the reset_fn
>> list between test and function-reset, it's also safer.  Thanks,
>
>After exploring Ionut's patch[1], what I propose above is probably
>wrong.  We don't have any guarantees that the sibling topology is
>static and this bus-reset-possible function presents it as immutable.
>The existing bus-reset function has the same race, but the gap is
>smaller, let's not make it bigger.
>
>There are however immutable properties we can test that cover a fair
>number of cases.  For example:
>
>bool pci_reset_function_needs_bridge_lock(struct pci_dev *dev)
>{
>	return pci_upstream_bridge(dev) &&
>		!(dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET) && !dev->is_virtfn;
>}
>EXPORT_SYMBOL_GPL(pci_reset_function_needs_bridge_lock);
>
>We should be able to use this as a condition on whether we acquire the
>bridge device lock, in pci_reset_function() and in vfio-pci code.
>Thanks,

This sounds good and it will also fix the bug that I am seeing.

The only caveat is that pci_dev_wait() might fail during VF reset and
then it might try to do a bus reset if the VF is behind a pciehp or CXL
port. I would prefer to do it only for vfio-pci code as that is where
the reset skip issue is. pci_reset_function() takes the lock and
serializes the VF reset. Are you ok with doing it just for vfio-pci?

Thanks,
Sami

      parent reply	other threads:[~2026-11-20 22:21 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-12 23:57 [PATCH] vfio/pci: Only lock the upstream bridge if the reset may need it Samiullah Khawaja
2026-11-13  0:26 ` sashiko-bot
2026-11-19  0:37 ` Alex Williamson
2026-11-19 22:32   ` Alex Williamson
2026-11-20  8:51     ` Ionut Nechita (Wind River)
2026-11-20 22:21     ` Samiullah Khawaja [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=awC5Ldqmue6DiO44@google.com \
    --to=skhawaja@google.com \
    --cc=alex@shazbot.org \
    --cc=anthony.pighin@nokia.com \
    --cc=bhelgaas@google.com \
    --cc=dan.j.williams@intel.com \
    --cc=dave.jiang@intel.com \
    --cc=jgg@ziepe.ca \
    --cc=kevin.tian@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.