Re: [PATCH v3] PCI: account for sysfs-disabled reset in pci_{slot,bus}_resettable()

[Alex Williamson <alex.williamson@redhat.com>]

On Tue, 4 Mar 2025 17:40:50 -0600
Bjorn Helgaas <helgaas@kernel.org> wrote:

> On Fri, Feb 07, 2025 at 02:56:00PM -0600, Nishanth Aravamudan wrote:
> > Commit d88f521da3ef ("PCI: Allow userspace to query and set
> > device reset mechanism") added support for userspace to disable reset of
> > specific PCI devices (by echo'ing "" into reset_method) and
> > pci_{slot,bus}_resettable() methods do not check pci_reset_supported()
> > to see if userspace has disabled reset.

The reset_method attribute selects which reset mechanism is available
to pci_reset_function().  It is not intended to interact with the
slot and bus scope reset mechanisms.

> > 
> > __pci_reset_bus()  
> > 	-> pci_bus_reset(..., PCI_RESET_PROBE)
> > 		-> pci_bus_resettable()  
> > 
> > __pci_reset_slot()  
> > 	-> pci_slot_reset(..., PCI_RESET_PROBE)
> > 		-> pci_slot_resettable()  
> > 
> > pci_reset_bus()  
> > 	-> pci_probe_reset_slot()
> > 		-> pci_slot_reset(..., PCI_RESET_PROBE)
> > 			-> pci_bus_resettable()  
> > 	if true:
> > 		__pci_reset_slot()
> > 	else:
> > 		__pci_reset_bus()
> > 
> > I was able to reproduce this issue with a vfio device passed to a qemu
> > guest, where I had disabled PCI reset via sysfs. Both
> > vfio_pci_ioctl_get_pci_hot_reset_info() and
> > vfio_pci_ioctl_pci_hot_reset() check if either the vdev's slot or bus is
> > not resettable by calling pci_probe_reset_slot(). Before my change, this
> > ends up ignoring the sysfs file contents and vfio-pci happily ends up
> > issuing a reset to that device.

And that's exactly how it's supposed to work, bus and slot resets are
different mechanisms.  This change has broken vfio-pci's ability to
perform bus/slot reset for the vast majority of use cases, where the
reset_methods array is empty because we have devices that have no reset
mechanism other than bus reset.  This is seen for example in the
trivial case of a multi-function GPU and audio device.  We can no
longer perform a bus reset here because audio function has no reset
mechanism[1].

The irony here is that the bus/slot reset is meant to provide resets
when the device does not support function level reset, but now we need
function level reset support in order to perform a bus/slot reset.

This change has been pushed to stable trees and is already beginning to
cause problems:

https://lore.kernel.org/all/808e1111-27b7-f35b-6d5c-5b275e73677b@absolutedigital.net/

Please revert this change with a cc to stable.  Thanks,

Alex

[1] Both the GPU and the audio device should report no available reset,
but I suspect we have an ordering problem that function 1 hadn't been
discovered before we probed bus reset on function 0.

> > 
> > Add an explicit check of pci_reset_supported() in both
> > pci_slot_resettable() and pci_bus_resettable() to ensure both the reset
> > status and reset execution are both bypassed if an administrator
> > disables it for a device.
> > 
> > Fixes: d88f521da3ef ("PCI: Allow userspace to query and set device reset mechanism")
> > Signed-off-by: Nishanth Aravamudan <naravamudan@nvidia.com>
> > Cc: Bjorn Helgaas <bhelgaas@google.com>
> > Cc: Alex Williamson <alex.williamson@redhat.com>
> > Cc: Raphael Norwitz <raphael.norwitz@nutanix.com>
> > Cc: Amey Narkhede <ameynarkhede03@gmail.com>
> > Cc: linux-pci@vger.kernel.org
> > Cc: linux-kernel@vger.kernel.org
> > Cc: Jason Gunthorpe <jgg@nvidia.com>
> > Cc: Yishai Hadas <yishaih@nvidia.com>
> > Cc: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > Cc: Kevin Tian <kevin.tian@intel.com>
> > Cc: kvm@vger.kernel.org  
> 
> Applied to pci/reset for v6.15, thanks!
> 
> > ---
> > 
> > Changes since v2:
> >  - update commit message to include more details
> > 
> > Changes since v1:
> >  - fix capitalization and ()s
> >  - clarify same checks are done in reset path
> > 
> >  drivers/pci/pci.c | 4 ++++
> >  1 file changed, 4 insertions(+)
> > 
> > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > index 869d204a70a3..738d29375ad3 100644
> > --- a/drivers/pci/pci.c
> > +++ b/drivers/pci/pci.c
> > @@ -5405,6 +5405,8 @@ static bool pci_bus_resettable(struct pci_bus *bus)
> >  		return false;
> >  
> >  	list_for_each_entry(dev, &bus->devices, bus_list) {
> > +		if (!pci_reset_supported(dev))
> > +			return false;
> >  		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
> >  		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
> >  			return false;
> > @@ -5481,6 +5483,8 @@ static bool pci_slot_resettable(struct pci_slot *slot)
> >  	list_for_each_entry(dev, &slot->bus->devices, bus_list) {
> >  		if (!dev->slot || dev->slot != slot)
> >  			continue;
> > +		if (!pci_reset_supported(dev))
> > +			return false;
> >  		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
> >  		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
> >  			return false;
> > -- 
> > 2.34.1
> >   
>