[PATCH] pci: account for sysfs-disabled reset in pci_{slot,bus}_resettable

[PATCH] pci: account for sysfs-disabled reset in pci_{slot,bus}_resettable

[Nishanth Aravamudan]

vfio_pci_ioctl_get_pci_hot_reset_info checks if either the vdev's slot
or bus is not resettable by calling pci_probe_reset_{slot,bus}. Those
functions in turn call pci_{slot,bus}_resettable() to see if the PCI
device supports reset.

However, commit d88f521da3ef ("PCI: Allow userspace to query and set
device reset mechanism") added support for userspace to disable reset of
specific PCI devices (by echo'ing "" into reset_method) and
pci_{slot,bus}_resettable methods do not check pci_reset_supported() to
see if userspace has disabled reset. Therefore, if an administrator
disables PCI reset of a specific device, but then uses vfio-pci with
that device (e.g. with qemu), vfio-pci will happily end up issuing a
reset to that device.

Add an explicit check of pci_reset_supported() in both paths.

Fixes: d88f521da3ef ("PCI: Allow userspace to query and set device reset mechanism")
Signed-off-by: Nishanth Aravamudan <naravamudan@nvidia.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Raphael Norwitz <raphael.norwitz@nutanix.com>
Cc: Amey Narkhede <ameynarkhede03@gmail.com>
Cc: linux-pci@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: Yishai Hadas <yishaih@nvidia.com>
Cc: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Cc: Kevin Tian <kevin.tian@intel.com>
Cc: kvm@vger.kernel.org
---
 drivers/pci/pci.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 661f98c6c63a..809936e1c3b7 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -5536,6 +5536,8 @@ static bool pci_bus_resettable(struct pci_bus *bus)
 		return false;
 
 	list_for_each_entry(dev, &bus->devices, bus_list) {
+		if (!pci_reset_supported(dev))
+			return false;
 		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
 		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
 			return false;
@@ -5612,6 +5614,8 @@ static bool pci_slot_resettable(struct pci_slot *slot)
 	list_for_each_entry(dev, &slot->bus->devices, bus_list) {
 		if (!dev->slot || dev->slot != slot)
 			continue;
+		if (!pci_reset_supported(dev))
+			return false;
 		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
 		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
 			return false;

Re: [PATCH] pci: account for sysfs-disabled reset in pci_{slot,bus}_resettable

[Jason Gunthorpe]

On Mon, Jan 06, 2025 at 03:52:31PM -0600, Nishanth Aravamudan wrote:
> vfio_pci_ioctl_get_pci_hot_reset_info checks if either the vdev's slot
> or bus is not resettable by calling pci_probe_reset_{slot,bus}. Those
> functions in turn call pci_{slot,bus}_resettable() to see if the PCI
> device supports reset.

This change makes sense to me, but..

> However, commit d88f521da3ef ("PCI: Allow userspace to query and set
> device reset mechanism") added support for userspace to disable reset of
> specific PCI devices (by echo'ing "" into reset_method) and
> pci_{slot,bus}_resettable methods do not check pci_reset_supported() to
> see if userspace has disabled reset. Therefore, if an administrator
> disables PCI reset of a specific device, but then uses vfio-pci with
> that device (e.g. with qemu), vfio-pci will happily end up issuing a
> reset to that device.

How does vfio-pci endup issuing a reset? It looked like all the paths
are blocked in the pci core with pci_reset_supported()? Is there also
a path that vfio is calling that is missing a pci_reset_supported()
check? If yes that should probably be fixed in another patch.

Or do you mean that VFIO tries to do a reset but it fails and nothing
happens, the real issue is that hot_reset_info is reporting incorrect
information to userspace?

Jason

Re: [PATCH] pci: account for sysfs-disabled reset in pci_{slot,bus}_resettable

[Bjorn Helgaas]

Please update subject line to match historical capitalization
convention.

On Mon, Jan 06, 2025 at 03:52:31PM -0600, Nishanth Aravamudan wrote:
> vfio_pci_ioctl_get_pci_hot_reset_info checks if either the vdev's slot
> or bus is not resettable by calling pci_probe_reset_{slot,bus}. Those
> functions in turn call pci_{slot,bus}_resettable() to see if the PCI
> device supports reset.
> 
> However, commit d88f521da3ef ("PCI: Allow userspace to query and set
> device reset mechanism") added support for userspace to disable reset of
> specific PCI devices (by echo'ing "" into reset_method) and
> pci_{slot,bus}_resettable methods do not check pci_reset_supported() to
> see if userspace has disabled reset. Therefore, if an administrator
> disables PCI reset of a specific device, but then uses vfio-pci with
> that device (e.g. with qemu), vfio-pci will happily end up issuing a
> reset to that device.

Please consistently add "()" after function names.

> Add an explicit check of pci_reset_supported() in both paths.

Bjorn

Re: [PATCH] pci: account for sysfs-disabled reset in pci_{slot,bus}_resettable

[Nishanth Aravamudan]

On Mon, Jan 13, 2025 at 04:42:00PM -0400, Jason Gunthorpe wrote:
> On Mon, Jan 06, 2025 at 03:52:31PM -0600, Nishanth Aravamudan wrote:
> > vfio_pci_ioctl_get_pci_hot_reset_info checks if either the vdev's slot
> > or bus is not resettable by calling pci_probe_reset_{slot,bus}. Those
> > functions in turn call pci_{slot,bus}_resettable() to see if the PCI
> > device supports reset.
> 
> This change makes sense to me, but..
> 
> > However, commit d88f521da3ef ("PCI: Allow userspace to query and set
> > device reset mechanism") added support for userspace to disable reset of
> > specific PCI devices (by echo'ing "" into reset_method) and
> > pci_{slot,bus}_resettable methods do not check pci_reset_supported() to
> > see if userspace has disabled reset. Therefore, if an administrator
> > disables PCI reset of a specific device, but then uses vfio-pci with
> > that device (e.g. with qemu), vfio-pci will happily end up issuing a
> > reset to that device.
> 
> How does vfio-pci endup issuing a reset? It looked like all the paths
> are blocked in the pci core with pci_reset_supported()? Is there also
> a path that vfio is calling that is missing a pci_reset_supported()
> check? If yes that should probably be fixed in another patch.

This is the path I observed:

drivers/vfio/vfio_pci_core::vfio_pci_ioctl_get_pci_hot_reset_info()
	indicates a reset is possible if either
	-> drivers/pci/pci.c::pci_probe_reset_slot() ||
	-> drivers/pci/pci.c::pci_probe_reset_bus()
	returns 0

drivers/pci/pci.c::pci_probe_reset_slot()
	-> pci_slot_reset(..., PCI_RESET_PROBE)
		-> pci_slot_resettable()
drivers/pci/pci.c::pci_probe_reset_bus()
	-> pci_bus_reset(..., PCI_RESET_PROBE)
		-> pci_bus_resettable()

Both pci_{slot,bus}_resettable() before my change returned true even if the
sysfs files indicated resets were disabled.

Separate from this path, e.g., a poorly-behaving userspace that ignores
or does not execute the VFIO_DEVICE_GET_PCI_HOT_RESET_INFO ioctl before
issuing a VFIO_DEVICE_PCI_HOT_RESET ioctl, actual resets check the same
return values:

drivers/vfio/vfio_pci_core::vfio_pci_ioctl_pci_hot_reset()
	indicates a reset is possible if either
	-> drivers/pci/pci.c::pci_probe_reset_slot() ||
	-> drivers/pci/pci.c::pci_probe_reset_bus()
	returns 0

	and will then issue a reset to the device via either
	-> vfio_pci_ioctl_pci_hot_reset_groups() ||
	-> vfio_pci_dev_set_hot_reset()

Thanks,
Nish

Re: [PATCH] pci: account for sysfs-disabled reset in pci_{slot,bus}_resettable

[Nishanth Aravamudan]

On Mon, Jan 13, 2025 at 05:25:40PM -0600, Bjorn Helgaas wrote:
> Please update subject line to match historical capitalization
> convention.

Thank you, I will fix in v2.

> 
> On Mon, Jan 06, 2025 at 03:52:31PM -0600, Nishanth Aravamudan wrote:
> > vfio_pci_ioctl_get_pci_hot_reset_info checks if either the vdev's slot
> > or bus is not resettable by calling pci_probe_reset_{slot,bus}. Those
> > functions in turn call pci_{slot,bus}_resettable() to see if the PCI
> > device supports reset.
> >
> > However, commit d88f521da3ef ("PCI: Allow userspace to query and set
> > device reset mechanism") added support for userspace to disable reset of
> > specific PCI devices (by echo'ing "" into reset_method) and
> > pci_{slot,bus}_resettable methods do not check pci_reset_supported() to
> > see if userspace has disabled reset. Therefore, if an administrator
> > disables PCI reset of a specific device, but then uses vfio-pci with
> > that device (e.g. with qemu), vfio-pci will happily end up issuing a
> > reset to that device.
> 
> Please consistently add "()" after function names.

Will fix in v2, as well.

-Nish

[PATCH v2] PCI: account for sysfs-disabled reset in pci_{slot,bus}_resettable()

[Nishanth Aravamudan]

Both vfio_pci_ioctl_get_pci_hot_reset_info() and
vfio_pci_ioctl_pci_hot_reset() check if either the vdev's slot or bus is
not resettable by calling pci_probe_reset_{slot,bus}(). Those functions
in turn call pci_{slot,bus}_resettable() to see if the PCI device
supports reset.

However, commit d88f521da3ef ("PCI: Allow userspace to query and set
device reset mechanism") added support for userspace to disable reset of
specific PCI devices (by echo'ing "" into reset_method) and
pci_{slot,bus}_resettable() methods do not check pci_reset_supported()
to see if userspace has disabled reset. Therefore, if an administrator
disables PCI reset of a specific device, but then uses vfio-pci with
that device (e.g. with qemu), vfio-pci will happily end up issuing a
reset to that device.

Add an explicit check of pci_reset_supported() in both
pci_slot_resettable() and pci_bus_resettable() to ensure both the hot
reset status and hot reset execution are both bypassed if an
administrator disables it for a vfio-pci managed device.

Fixes: d88f521da3ef ("PCI: Allow userspace to query and set device reset mechanism")
Signed-off-by: Nishanth Aravamudan <naravamudan@nvidia.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Raphael Norwitz <raphael.norwitz@nutanix.com>
Cc: Amey Narkhede <ameynarkhede03@gmail.com>
Cc: linux-pci@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: Yishai Hadas <yishaih@nvidia.com>
Cc: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Cc: Kevin Tian <kevin.tian@intel.com>
Cc: kvm@vger.kernel.org

---

Changes since v1:
 - fix capitalization and ()s
 - clarify same checks are done in reset path
---
 drivers/pci/pci.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 661f98c6c63a..809936e1c3b7 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -5536,6 +5536,8 @@ static bool pci_bus_resettable(struct pci_bus *bus)
 		return false;
 
 	list_for_each_entry(dev, &bus->devices, bus_list) {
+		if (!pci_reset_supported(dev))
+			return false;
 		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
 		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
 			return false;
@@ -5612,6 +5614,8 @@ static bool pci_slot_resettable(struct pci_slot *slot)
 	list_for_each_entry(dev, &slot->bus->devices, bus_list) {
 		if (!dev->slot || dev->slot != slot)
 			continue;
+		if (!pci_reset_supported(dev))
+			return false;
 		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
 		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
 			return false;
-- 
2.34.1

Re: [PATCH] pci: account for sysfs-disabled reset in pci_{slot,bus}_resettable

[Jason Gunthorpe]

On Wed, Jan 22, 2025 at 12:14:02PM -0600, Nishanth Aravamudan wrote:
> On Mon, Jan 13, 2025 at 04:42:00PM -0400, Jason Gunthorpe wrote:
> > On Mon, Jan 06, 2025 at 03:52:31PM -0600, Nishanth Aravamudan wrote:
> > > vfio_pci_ioctl_get_pci_hot_reset_info checks if either the vdev's slot
> > > or bus is not resettable by calling pci_probe_reset_{slot,bus}. Those
> > > functions in turn call pci_{slot,bus}_resettable() to see if the PCI
> > > device supports reset.
> > 
> > This change makes sense to me, but..
> > 
> > > However, commit d88f521da3ef ("PCI: Allow userspace to query and set
> > > device reset mechanism") added support for userspace to disable reset of
> > > specific PCI devices (by echo'ing "" into reset_method) and
> > > pci_{slot,bus}_resettable methods do not check pci_reset_supported() to
> > > see if userspace has disabled reset. Therefore, if an administrator
> > > disables PCI reset of a specific device, but then uses vfio-pci with
> > > that device (e.g. with qemu), vfio-pci will happily end up issuing a
> > > reset to that device.
> > 
> > How does vfio-pci endup issuing a reset? It looked like all the paths
> > are blocked in the pci core with pci_reset_supported()? Is there also
> > a path that vfio is calling that is missing a pci_reset_supported()
> > check? If yes that should probably be fixed in another patch.
> 
> This is the path I observed:

You didn't answer the question, I didn't ask about pci_probe_*() I
asked why doesn't pci_reset_supported() directly block the actual
reset?

Should we be adding:

@@ -5919,6 +5919,9 @@ int __pci_reset_bus(struct pci_bus *bus)
  */
 int pci_reset_bus(struct pci_dev *pdev)
 {
+       if (!pci_reset_supported(pdev))
+               return -EOPNOTSUPP;
+
        return (!pci_probe_reset_slot(pdev->slot)) ?
            __pci_reset_slot(pdev->slot) : __pci_reset_bus(pdev->bus);

And maybe more?

Jason

Re: [PATCH] pci: account for sysfs-disabled reset in pci_{slot,bus}_resettable

[Nishanth Aravamudan]

On Thu, Jan 23, 2025 at 09:33:12AM -0400, Jason Gunthorpe wrote:
> On Wed, Jan 22, 2025 at 12:14:02PM -0600, Nishanth Aravamudan wrote:
> > On Mon, Jan 13, 2025 at 04:42:00PM -0400, Jason Gunthorpe wrote:
> > > On Mon, Jan 06, 2025 at 03:52:31PM -0600, Nishanth Aravamudan wrote:
> > > > vfio_pci_ioctl_get_pci_hot_reset_info checks if either the vdev's slot
> > > > or bus is not resettable by calling pci_probe_reset_{slot,bus}. Those
> > > > functions in turn call pci_{slot,bus}_resettable() to see if the PCI
> > > > device supports reset.
> > > 
> > > This change makes sense to me, but..
> > > 
> > > > However, commit d88f521da3ef ("PCI: Allow userspace to query and set
> > > > device reset mechanism") added support for userspace to disable reset of
> > > > specific PCI devices (by echo'ing "" into reset_method) and
> > > > pci_{slot,bus}_resettable methods do not check pci_reset_supported() to
> > > > see if userspace has disabled reset. Therefore, if an administrator
> > > > disables PCI reset of a specific device, but then uses vfio-pci with
> > > > that device (e.g. with qemu), vfio-pci will happily end up issuing a
> > > > reset to that device.
> > > 
> > > How does vfio-pci endup issuing a reset? It looked like all the paths
> > > are blocked in the pci core with pci_reset_supported()? Is there also
> > > a path that vfio is calling that is missing a pci_reset_supported()
> > > check? If yes that should probably be fixed in another patch.
> > 
> > This is the path I observed:
> 
> You didn't answer the question, I didn't ask about pci_probe_*() I
> asked why doesn't pci_reset_supported() directly block the actual
> reset?

Sorry, I misunderstood your question.

__pci_reset_bus()
	-> pci_bus_reset(..., PCI_RESET_PROBE)
		-> pci_bus_resettable()

__pci_reset_slot()
	-> pci_slot_reset(..., PCI_RESET_PROBE)
		-> pci_slot_resettable()

pci_reset_bus()
	-> pci_probe_reset_slot()
		-> pci_slot_reset(..., PCI_RESET_PROBE)
			-> pci_bus_resettable()
	if true:
		__pci_reset_slot()
	else:
		__pci_reset_bus()

Before my change, both call paths would end up calling
pci_slot_resettable() and not checking the sysfs file-contents.

Please let me know if that addresses your concern, I think my changes
fixes the paths you are talking about as well. If I need to clarify this
in the commit message, I can.

-Nish

[PATCH v3] PCI: account for sysfs-disabled reset in pci_{slot,bus}_resettable()

[Nishanth Aravamudan]

Commit d88f521da3ef ("PCI: Allow userspace to query and set
device reset mechanism") added support for userspace to disable reset of
specific PCI devices (by echo'ing "" into reset_method) and
pci_{slot,bus}_resettable() methods do not check pci_reset_supported()
to see if userspace has disabled reset.

__pci_reset_bus()
	-> pci_bus_reset(..., PCI_RESET_PROBE)
		-> pci_bus_resettable()

__pci_reset_slot()
	-> pci_slot_reset(..., PCI_RESET_PROBE)
		-> pci_slot_resettable()

pci_reset_bus()
	-> pci_probe_reset_slot()
		-> pci_slot_reset(..., PCI_RESET_PROBE)
			-> pci_bus_resettable()
	if true:
		__pci_reset_slot()
	else:
		__pci_reset_bus()

I was able to reproduce this issue with a vfio device passed to a qemu
guest, where I had disabled PCI reset via sysfs. Both
vfio_pci_ioctl_get_pci_hot_reset_info() and
vfio_pci_ioctl_pci_hot_reset() check if either the vdev's slot or bus is
not resettable by calling pci_probe_reset_slot(). Before my change, this
ends up ignoring the sysfs file contents and vfio-pci happily ends up
issuing a reset to that device.

Add an explicit check of pci_reset_supported() in both
pci_slot_resettable() and pci_bus_resettable() to ensure both the reset
status and reset execution are both bypassed if an administrator
disables it for a device.

Fixes: d88f521da3ef ("PCI: Allow userspace to query and set device reset mechanism")
Signed-off-by: Nishanth Aravamudan <naravamudan@nvidia.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Raphael Norwitz <raphael.norwitz@nutanix.com>
Cc: Amey Narkhede <ameynarkhede03@gmail.com>
Cc: linux-pci@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: Yishai Hadas <yishaih@nvidia.com>
Cc: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Cc: Kevin Tian <kevin.tian@intel.com>
Cc: kvm@vger.kernel.org

---

Changes since v2:
 - update commit message to include more details

Changes since v1:
 - fix capitalization and ()s
 - clarify same checks are done in reset path

 drivers/pci/pci.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 869d204a70a3..738d29375ad3 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -5405,6 +5405,8 @@ static bool pci_bus_resettable(struct pci_bus *bus)
 		return false;
 
 	list_for_each_entry(dev, &bus->devices, bus_list) {
+		if (!pci_reset_supported(dev))
+			return false;
 		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
 		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
 			return false;
@@ -5481,6 +5483,8 @@ static bool pci_slot_resettable(struct pci_slot *slot)
 	list_for_each_entry(dev, &slot->bus->devices, bus_list) {
 		if (!dev->slot || dev->slot != slot)
 			continue;
+		if (!pci_reset_supported(dev))
+			return false;
 		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
 		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
 			return false;
-- 
2.34.1

Re: [PATCH v3] PCI: account for sysfs-disabled reset in pci_{slot,bus}_resettable()

[Bjorn Helgaas]

On Fri, Feb 07, 2025 at 02:56:00PM -0600, Nishanth Aravamudan wrote:
> Commit d88f521da3ef ("PCI: Allow userspace to query and set
> device reset mechanism") added support for userspace to disable reset of
> specific PCI devices (by echo'ing "" into reset_method) and
> pci_{slot,bus}_resettable() methods do not check pci_reset_supported()
> to see if userspace has disabled reset.
> 
> __pci_reset_bus()
> 	-> pci_bus_reset(..., PCI_RESET_PROBE)
> 		-> pci_bus_resettable()
> 
> __pci_reset_slot()
> 	-> pci_slot_reset(..., PCI_RESET_PROBE)
> 		-> pci_slot_resettable()
> 
> pci_reset_bus()
> 	-> pci_probe_reset_slot()
> 		-> pci_slot_reset(..., PCI_RESET_PROBE)
> 			-> pci_bus_resettable()
> 	if true:
> 		__pci_reset_slot()
> 	else:
> 		__pci_reset_bus()
> 
> I was able to reproduce this issue with a vfio device passed to a qemu
> guest, where I had disabled PCI reset via sysfs. Both
> vfio_pci_ioctl_get_pci_hot_reset_info() and
> vfio_pci_ioctl_pci_hot_reset() check if either the vdev's slot or bus is
> not resettable by calling pci_probe_reset_slot(). Before my change, this
> ends up ignoring the sysfs file contents and vfio-pci happily ends up
> issuing a reset to that device.
> 
> Add an explicit check of pci_reset_supported() in both
> pci_slot_resettable() and pci_bus_resettable() to ensure both the reset
> status and reset execution are both bypassed if an administrator
> disables it for a device.
> 
> Fixes: d88f521da3ef ("PCI: Allow userspace to query and set device reset mechanism")
> Signed-off-by: Nishanth Aravamudan <naravamudan@nvidia.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Alex Williamson <alex.williamson@redhat.com>
> Cc: Raphael Norwitz <raphael.norwitz@nutanix.com>
> Cc: Amey Narkhede <ameynarkhede03@gmail.com>
> Cc: linux-pci@vger.kernel.org
> Cc: linux-kernel@vger.kernel.org
> Cc: Jason Gunthorpe <jgg@nvidia.com>
> Cc: Yishai Hadas <yishaih@nvidia.com>
> Cc: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> Cc: Kevin Tian <kevin.tian@intel.com>
> Cc: kvm@vger.kernel.org

Applied to pci/reset for v6.15, thanks!

> ---
> 
> Changes since v2:
>  - update commit message to include more details
> 
> Changes since v1:
>  - fix capitalization and ()s
>  - clarify same checks are done in reset path
> 
>  drivers/pci/pci.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 869d204a70a3..738d29375ad3 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -5405,6 +5405,8 @@ static bool pci_bus_resettable(struct pci_bus *bus)
>  		return false;
>  
>  	list_for_each_entry(dev, &bus->devices, bus_list) {
> +		if (!pci_reset_supported(dev))
> +			return false;
>  		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
>  		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
>  			return false;
> @@ -5481,6 +5483,8 @@ static bool pci_slot_resettable(struct pci_slot *slot)
>  	list_for_each_entry(dev, &slot->bus->devices, bus_list) {
>  		if (!dev->slot || dev->slot != slot)
>  			continue;
> +		if (!pci_reset_supported(dev))
> +			return false;
>  		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
>  		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
>  			return false;
> -- 
> 2.34.1
> 

Re: [PATCH v3] PCI: account for sysfs-disabled reset in pci_{slot,bus}_resettable()

[Alex Williamson]

On Tue, 4 Mar 2025 17:40:50 -0600
Bjorn Helgaas <helgaas@kernel.org> wrote:

> On Fri, Feb 07, 2025 at 02:56:00PM -0600, Nishanth Aravamudan wrote:
> > Commit d88f521da3ef ("PCI: Allow userspace to query and set
> > device reset mechanism") added support for userspace to disable reset of
> > specific PCI devices (by echo'ing "" into reset_method) and
> > pci_{slot,bus}_resettable() methods do not check pci_reset_supported()
> > to see if userspace has disabled reset.

The reset_method attribute selects which reset mechanism is available
to pci_reset_function().  It is not intended to interact with the
slot and bus scope reset mechanisms.

> > 
> > __pci_reset_bus()  
> > 	-> pci_bus_reset(..., PCI_RESET_PROBE)
> > 		-> pci_bus_resettable()  
> > 
> > __pci_reset_slot()  
> > 	-> pci_slot_reset(..., PCI_RESET_PROBE)
> > 		-> pci_slot_resettable()  
> > 
> > pci_reset_bus()  
> > 	-> pci_probe_reset_slot()
> > 		-> pci_slot_reset(..., PCI_RESET_PROBE)
> > 			-> pci_bus_resettable()  
> > 	if true:
> > 		__pci_reset_slot()
> > 	else:
> > 		__pci_reset_bus()
> > 
> > I was able to reproduce this issue with a vfio device passed to a qemu
> > guest, where I had disabled PCI reset via sysfs. Both
> > vfio_pci_ioctl_get_pci_hot_reset_info() and
> > vfio_pci_ioctl_pci_hot_reset() check if either the vdev's slot or bus is
> > not resettable by calling pci_probe_reset_slot(). Before my change, this
> > ends up ignoring the sysfs file contents and vfio-pci happily ends up
> > issuing a reset to that device.

And that's exactly how it's supposed to work, bus and slot resets are
different mechanisms.  This change has broken vfio-pci's ability to
perform bus/slot reset for the vast majority of use cases, where the
reset_methods array is empty because we have devices that have no reset
mechanism other than bus reset.  This is seen for example in the
trivial case of a multi-function GPU and audio device.  We can no
longer perform a bus reset here because audio function has no reset
mechanism[1].

The irony here is that the bus/slot reset is meant to provide resets
when the device does not support function level reset, but now we need
function level reset support in order to perform a bus/slot reset.

This change has been pushed to stable trees and is already beginning to
cause problems:

https://lore.kernel.org/all/808e1111-27b7-f35b-6d5c-5b275e73677b@absolutedigital.net/

Please revert this change with a cc to stable.  Thanks,

Alex

[1] Both the GPU and the audio device should report no available reset,
but I suspect we have an ordering problem that function 1 hadn't been
discovered before we probed bus reset on function 0.

> > 
> > Add an explicit check of pci_reset_supported() in both
> > pci_slot_resettable() and pci_bus_resettable() to ensure both the reset
> > status and reset execution are both bypassed if an administrator
> > disables it for a device.
> > 
> > Fixes: d88f521da3ef ("PCI: Allow userspace to query and set device reset mechanism")
> > Signed-off-by: Nishanth Aravamudan <naravamudan@nvidia.com>
> > Cc: Bjorn Helgaas <bhelgaas@google.com>
> > Cc: Alex Williamson <alex.williamson@redhat.com>
> > Cc: Raphael Norwitz <raphael.norwitz@nutanix.com>
> > Cc: Amey Narkhede <ameynarkhede03@gmail.com>
> > Cc: linux-pci@vger.kernel.org
> > Cc: linux-kernel@vger.kernel.org
> > Cc: Jason Gunthorpe <jgg@nvidia.com>
> > Cc: Yishai Hadas <yishaih@nvidia.com>
> > Cc: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > Cc: Kevin Tian <kevin.tian@intel.com>
> > Cc: kvm@vger.kernel.org  
> 
> Applied to pci/reset for v6.15, thanks!
> 
> > ---
> > 
> > Changes since v2:
> >  - update commit message to include more details
> > 
> > Changes since v1:
> >  - fix capitalization and ()s
> >  - clarify same checks are done in reset path
> > 
> >  drivers/pci/pci.c | 4 ++++
> >  1 file changed, 4 insertions(+)
> > 
> > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > index 869d204a70a3..738d29375ad3 100644
> > --- a/drivers/pci/pci.c
> > +++ b/drivers/pci/pci.c
> > @@ -5405,6 +5405,8 @@ static bool pci_bus_resettable(struct pci_bus *bus)
> >  		return false;
> >  
> >  	list_for_each_entry(dev, &bus->devices, bus_list) {
> > +		if (!pci_reset_supported(dev))
> > +			return false;
> >  		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
> >  		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
> >  			return false;
> > @@ -5481,6 +5483,8 @@ static bool pci_slot_resettable(struct pci_slot *slot)
> >  	list_for_each_entry(dev, &slot->bus->devices, bus_list) {
> >  		if (!dev->slot || dev->slot != slot)
> >  			continue;
> > +		if (!pci_reset_supported(dev))
> > +			return false;
> >  		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
> >  		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
> >  			return false;
> > -- 
> > 2.34.1
> >   
> 

[PATCH] Revert "PCI: Avoid reset when disabled via sysfs"

[Alex Williamson]

This reverts commit 479380efe1625e251008d24b2810283db60d6fcd.

The reset_method attribute on a PCI device is only intended to manage
the availability of function scoped resets for a device.  It was never
intended to restrict resets targeting the bus or slot.

In introducing a restriction that each device must support function
level reset by testing pci_reset_supported(), we essentially create a
catch-22, that a device must have a function scope reset in order to
support bus/slot reset, when we use bus/slot reset to effect a reset
of a device that does not support a function scoped reset, especially
multi-function devices.

This breaks the majority of uses cases where vfio-pci uses bus/slot
resets to manage multifunction devices that do not support function
scoped resets.

Fixes: 479380efe162 ("PCI: Avoid reset when disabled via sysfs")
Reported-by: Cal Peake <cp@absolutedigital.net>
Link: https://lore.kernel.org/all/808e1111-27b7-f35b-6d5c-5b275e73677b@absolutedigital.net
Cc: stable@vger.kernel.org
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 drivers/pci/pci.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 4d7c9f64ea24..e77d5b53c0ce 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -5429,8 +5429,6 @@ static bool pci_bus_resettable(struct pci_bus *bus)
 		return false;
 
 	list_for_each_entry(dev, &bus->devices, bus_list) {
-		if (!pci_reset_supported(dev))
-			return false;
 		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
 		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
 			return false;
@@ -5507,8 +5505,6 @@ static bool pci_slot_resettable(struct pci_slot *slot)
 	list_for_each_entry(dev, &slot->bus->devices, bus_list) {
 		if (!dev->slot || dev->slot != slot)
 			continue;
-		if (!pci_reset_supported(dev))
-			return false;
 		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
 		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
 			return false;
-- 
2.48.1

RE: [PATCH] Revert "PCI: Avoid reset when disabled via sysfs"

[Tian, Kevin]

> From: Alex Williamson <alex.williamson@redhat.com>
> Sent: Tuesday, April 15, 2025 5:18 AM
> 
> This reverts commit 479380efe1625e251008d24b2810283db60d6fcd.
> 
> The reset_method attribute on a PCI device is only intended to manage
> the availability of function scoped resets for a device.  It was never
> intended to restrict resets targeting the bus or slot.
> 
> In introducing a restriction that each device must support function
> level reset by testing pci_reset_supported(), we essentially create a
> catch-22, that a device must have a function scope reset in order to
> support bus/slot reset, when we use bus/slot reset to effect a reset
> of a device that does not support a function scoped reset, especially
> multi-function devices.
> 
> This breaks the majority of uses cases where vfio-pci uses bus/slot
> resets to manage multifunction devices that do not support function
> scoped resets.
> 
> Fixes: 479380efe162 ("PCI: Avoid reset when disabled via sysfs")
> Reported-by: Cal Peake <cp@absolutedigital.net>
> Link: https://lore.kernel.org/all/808e1111-27b7-f35b-6d5c-
> 5b275e73677b@absolutedigital.net
> Cc: stable@vger.kernel.org
> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

Re: [PATCH] Revert "PCI: Avoid reset when disabled via sysfs"

[Bjorn Helgaas]

On Mon, Apr 14, 2025 at 03:18:23PM -0600, Alex Williamson wrote:
> This reverts commit 479380efe1625e251008d24b2810283db60d6fcd.
> 
> The reset_method attribute on a PCI device is only intended to manage
> the availability of function scoped resets for a device.  It was never
> intended to restrict resets targeting the bus or slot.
> 
> In introducing a restriction that each device must support function
> level reset by testing pci_reset_supported(), we essentially create a
> catch-22, that a device must have a function scope reset in order to
> support bus/slot reset, when we use bus/slot reset to effect a reset
> of a device that does not support a function scoped reset, especially
> multi-function devices.
> 
> This breaks the majority of uses cases where vfio-pci uses bus/slot
> resets to manage multifunction devices that do not support function
> scoped resets.
> 
> Fixes: 479380efe162 ("PCI: Avoid reset when disabled via sysfs")
> Reported-by: Cal Peake <cp@absolutedigital.net>
> Link: https://lore.kernel.org/all/808e1111-27b7-f35b-6d5c-5b275e73677b@absolutedigital.net
> Cc: stable@vger.kernel.org
> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

Applied with Kevin's reviewed-by to pci/for-linus for v6.15, thanks,
and sorry for the breakage.

> ---
>  drivers/pci/pci.c | 4 ----
>  1 file changed, 4 deletions(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 4d7c9f64ea24..e77d5b53c0ce 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -5429,8 +5429,6 @@ static bool pci_bus_resettable(struct pci_bus *bus)
>  		return false;
>  
>  	list_for_each_entry(dev, &bus->devices, bus_list) {
> -		if (!pci_reset_supported(dev))
> -			return false;
>  		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
>  		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
>  			return false;
> @@ -5507,8 +5505,6 @@ static bool pci_slot_resettable(struct pci_slot *slot)
>  	list_for_each_entry(dev, &slot->bus->devices, bus_list) {
>  		if (!dev->slot || dev->slot != slot)
>  			continue;
> -		if (!pci_reset_supported(dev))
> -			return false;
>  		if (dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
>  		    (dev->subordinate && !pci_bus_resettable(dev->subordinate)))
>  			return false;
> -- 
> 2.48.1
>