Linux PCI subsystem development
 help / color / mirror / Atom feed
* [PATCH V1 0/4] PCI: Don't list reset methods that can't reset the device
@ 2026-11-19 10:06 Vidya Sagar
  2026-11-19 10:06 ` [PATCH V1 1/4] PCI/CXL: Don't offer 'cxl_bus' reset where a bus reset can't work Vidya Sagar
                   ` (3 more replies)
  0 siblings, 4 replies; 9+ messages in thread
From: Vidya Sagar @ 2026-11-19 10:06 UTC (permalink / raw)
  To: Bjorn Helgaas, Dan Williams, Dave Jiang, Jonathan Cameron,
	Joerg Roedel, Jason Gunthorpe, Nicolin Chen, Kevin Tian
  Cc: vsethi, kthota, mmaddireddy, kumarahul, sagar.tv, linux-pci,
	linux-kernel, Vidya Sagar

On a 4-function ConnectX-8 below a CXL-capable Root Port that has no
hotplug slot, "reset_method" lists methods that can never reset the
device:

- Every function lists "cxl_bus".  It can be selected, but every reset
  then fails with -ENOTTY, because a Secondary Bus Reset of the Root
  Port would also reset the other functions.

- Function 0 also lists "bus", which the kernel then refuses to select
  ("Unsupported reset method").  Its reset methods are probed while it
  is still alone on its bus, before the other functions are scanned.

Separately, probing "bus" or a device-specific reset runs the IOMMU
reset hooks even though no reset follows.  Reset methods are probed
again on every write to "reset_method", so on a bound, ATS-capable
device behind an IOMMU each such write blocks the device's DMA for a
moment and, on arm-smmu-v3, sends ATC invalidations.

Patch 1 offers "cxl_bus" only where the slot or parent bus reset that it
relies on can reach the device, i.e., under the same conditions as
"bus" except for the CXL SBR mask.  The probe uses the same slot and
parent-bus checks that the "bus" probe has always made, from the same
callers.

Patch 2 skips the IOMMU reset hooks when only probing, as pcie_flr(),
pci_af_flr(), pci_pm_reset() and pci_dev_acpi_reset() already do.

Patch 3 takes the device lock when "reset_method" is written.  A reset,
which already holds that lock, could otherwise walk the list while a
write is replacing it.

Patch 4 re-checks function 0's reset methods at the end of
pci_scan_slot(), once the other functions are on the bus, and drops the
ones that no longer work.  By then device_add() has already created
function 0's "reset_method" file and sent the "add" uevent, so the
re-check only removes methods and keeps the order of the rest, rather
than overwriting a list that userspace may have set.  It holds the
device lock, so a write can't land between reading the list and storing
the result.  It needs the first three patches: without patch 1, function
0 still lists "cxl_bus"; without patch 2, the re-check, which runs after
device_add(), would cycle function 0's IOMMU domains during enumeration;
and without patch 3, writes to "reset_method" don't take the lock.

Patches 1 and 2 have Fixes tags (v6.10 and v7.0).  Patch 2 uses the
helper that patch 1 adds, so a backport of patch 2 needs patch 1 too.
Patches 3 and 4 address long-standing behaviour and have no Fixes tags.

Not covered: "reset_method" of a device that is already added is not
updated when functions appear on its bus later, e.g., VFs or functions
added by a rescan.  Writes to "reset_method" and the reset itself still
check again and refuse a bus reset in that case.

Testing: all four patches were tested on the ConnectX-8 above, applied
incrementally (patch 1, then patches 1-2, then patches 1-4):
- Patch 1: after "echo default", functions 0 and 1 list only "flr".
  Function 0 still lists "flr bus cxl_bus" at boot and refuses both
  "bus" and "cxl_bus", which is what patch 4 addresses.
- Patches 1-2: writing "reset_method" on a function with its driver
  bound no longer runs the IOMMU reset hooks.
- Patches 1-4: function 0 lists only "flr" at boot.  A single-function
  ConnectX-8 below the same kind of Root Port still lists "bus" and
  "cxl_bus".
- Patches 1-4: 100 remove/rescan cycles of the 4-function ConnectX-8,
  with a udev rule writing "flr bus" to function 0's "reset_method" on
  "add", a loop writing the same while function 0 was alone on the bus
  during each rescan, and a loop reading "reset_method" of all
  functions.  Writes landed before the re-check in all 100 cycles,
  function 0 ended every cycle with "flr", and there were no warnings.
  Writing "bus" instead left function 0 with an empty list in all 100
  cycles, i.e., the re-check kept the written list minus "bus".
- Patches 1-4, plus a temporary 200 ms delay in the re-check between
  reading the list and storing the result: 100 remove/rescan cycles,
  each writing "" to function 0's "reset_method" as soon as the last
  function appeared.  Every write waited for the re-check (203-206 ms),
  and function 0 ended every cycle with an empty list.

The analysis, the fixes and the changelogs were drafted with an AI
coding assistant (Claude Opus 5.5), working from logs of these resets
and a reading of drivers/pci.  I reviewed all of it and did the testing
described above.

Vidya Sagar (4):
  PCI/CXL: Don't offer 'cxl_bus' reset where a bus reset can't work
  PCI: Don't run IOMMU reset hooks when only probing reset methods
  PCI/sysfs: Take the device lock when setting reset methods
  PCI: Re-check function 0 reset methods after scanning its slot

 drivers/pci/pci-sysfs.c |  3 ++
 drivers/pci/pci.c       | 65 +++++++++++++++++++++++++++++++++++++----
 drivers/pci/pci.h       |  1 +
 drivers/pci/probe.c     | 16 +++++++++-
 drivers/pci/quirks.c    |  3 ++
 5 files changed, 81 insertions(+), 7 deletions(-)

-- 
2.43.0


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH V1 1/4] PCI/CXL: Don't offer 'cxl_bus' reset where a bus reset can't work
  2026-11-19 10:06 [PATCH V1 0/4] PCI: Don't list reset methods that can't reset the device Vidya Sagar
@ 2026-11-19 10:06 ` Vidya Sagar
  2026-11-19 10:20   ` sashiko-bot
  2026-11-19 10:06 ` [PATCH V1 2/4] PCI: Don't run IOMMU reset hooks when only probing reset methods Vidya Sagar
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 9+ messages in thread
From: Vidya Sagar @ 2026-11-19 10:06 UTC (permalink / raw)
  To: Bjorn Helgaas, Dan Williams, Dave Jiang, Jonathan Cameron,
	Joerg Roedel, Jason Gunthorpe, Nicolin Chen, Kevin Tian
  Cc: vsethi, kthota, mmaddireddy, kumarahul, sagar.tv, linux-pci,
	linux-kernel, Vidya Sagar

cxl_reset_bus_function() reports the "cxl_bus" reset method as supported
whenever the device's upstream bridge has a CXL Port DVSEC.  The reset
itself goes through pci_reset_bus_function(), which can only reset the
device through a hotplug slot or a Secondary Bus Reset of the parent
bridge.  It returns -ENOTTY when neither is possible, e.g., when other
functions share the device's bus, when the device is a bridge, or when it
has PCI_DEV_FLAGS_NO_BUS_RESET.

"cxl_bus" is then listed in "reset_method" but always fails.  For example,
on function 1 of a 4-function ConnectX-8 below a CXL-capable Root Port
without a hotplug slot, "bus" is correctly not offered, but "cxl_bus" is:

  # cat /sys/bus/pci/devices/0001:01:00.1/reset_method
  flr cxl_bus
  # echo cxl_bus > /sys/bus/pci/devices/0001:01:00.1/reset_method
  # echo 1 > /sys/bus/pci/devices/0001:01:00.1/reset
  bash: echo: write error: Inappropriate ioctl for device

Each attempt still temporarily sets "Unmask SBR" in the Root Port and runs
the IOMMU reset prepare/done hooks, although no reset happens.

Move the slot and parent bus reset from pci_reset_bus_function() into a
new __pci_reset_bus_function() and use it for the "cxl_bus" probe.  The
probe now makes the same slot and parent-bus checks as the "bus" probe.
"cxl_bus" is then offered under the same conditions as "bus", except for
the CXL SBR mask that "cxl_bus" exists to clear.

Fixes: 53c49b6e6dd2 ("PCI/CXL: Add 'cxl_bus' reset method for devices below CXL Ports")
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
---
 drivers/pci/pci.c | 27 +++++++++++++++++++++------
 1 file changed, 21 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 7c6f2aba8144..921d076c4c1d 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -5030,6 +5030,21 @@ static bool cxl_sbr_masked(struct pci_dev *dev)
 	return true;
 }
 
+/*
+ * Reset "dev" through its hotplug slot or, failing that, a Secondary Bus
+ * Reset of its parent bridge.  Callers deal with CXL SBR masking.
+ */
+static int __pci_reset_bus_function(struct pci_dev *dev, bool probe)
+{
+	int rc;
+
+	rc = pci_dev_reset_slot_function(dev, probe);
+	if (rc != -ENOTTY)
+		return rc;
+
+	return pci_parent_bus_reset(dev, probe);
+}
+
 static int pci_reset_bus_function(struct pci_dev *dev, bool probe)
 {
 	struct pci_dev *bridge = pci_upstream_bridge(dev);
@@ -5048,12 +5063,8 @@ static int pci_reset_bus_function(struct pci_dev *dev, bool probe)
 		return rc;
 	}
 
-	rc = pci_dev_reset_slot_function(dev, probe);
-	if (rc != -ENOTTY)
-		goto done;
+	rc = __pci_reset_bus_function(dev, probe);
 
-	rc = pci_parent_bus_reset(dev, probe);
-done:
 	pci_dev_reset_iommu_done(dev);
 	return rc;
 }
@@ -5072,8 +5083,12 @@ static int cxl_reset_bus_function(struct pci_dev *dev, bool probe)
 	if (!dvsec)
 		return -ENOTTY;
 
+	/*
+	 * Unmasking SBR only helps if a slot or bus reset can reach "dev" at
+	 * all, e.g., not when other functions share its bus.
+	 */
 	if (probe)
-		return 0;
+		return __pci_reset_bus_function(dev, probe);
 
 	rc = pci_read_config_word(bridge, dvsec + PCI_DVSEC_CXL_PORT_CTL, &reg);
 	if (rc)
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH V1 2/4] PCI: Don't run IOMMU reset hooks when only probing reset methods
  2026-11-19 10:06 [PATCH V1 0/4] PCI: Don't list reset methods that can't reset the device Vidya Sagar
  2026-11-19 10:06 ` [PATCH V1 1/4] PCI/CXL: Don't offer 'cxl_bus' reset where a bus reset can't work Vidya Sagar
@ 2026-11-19 10:06 ` Vidya Sagar
  2026-11-19 10:14   ` sashiko-bot
  2026-11-19 10:06 ` [PATCH V1 3/4] PCI/sysfs: Take the device lock when setting " Vidya Sagar
  2026-11-19 10:06 ` [PATCH V1 4/4] PCI: Re-check function 0 reset methods after scanning its slot Vidya Sagar
  3 siblings, 1 reply; 9+ messages in thread
From: Vidya Sagar @ 2026-11-19 10:06 UTC (permalink / raw)
  To: Bjorn Helgaas, Dan Williams, Dave Jiang, Jonathan Cameron,
	Joerg Roedel, Jason Gunthorpe, Nicolin Chen, Kevin Tian
  Cc: vsethi, kthota, mmaddireddy, kumarahul, sagar.tv, linux-pci,
	linux-kernel, Vidya Sagar

pci_reset_bus_function() and __pci_dev_specific_reset() call
pci_dev_reset_iommu_prepare() and pci_dev_reset_iommu_done() even when
"probe" is set and no reset follows.  The other reset methods that use
these hooks, pcie_flr(), pci_af_flr(), pci_pm_reset() and
pci_dev_acpi_reset(), only call them when actually resetting.

Reset methods are probed at enumeration, before the IOMMU is set up, but
also whenever "reset_method" is written in sysfs.  For an ATS-capable
device behind an IOMMU, each such probe of "bus" or of a device-specific
reset attaches the blocking domain and then re-attaches the device's
domains.  That blocks any DMA in the window and, on arm-smmu-v3, sends
ATC invalidations, even while a driver is bound and using the device.

Return before the IOMMU hooks when probing, as the other reset methods
do.

Fixes: f5b16b802174 ("PCI: Suspend iommu function prior to resetting a device")
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
---
 drivers/pci/pci.c    | 3 +++
 drivers/pci/quirks.c | 3 +++
 2 files changed, 6 insertions(+)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 921d076c4c1d..918a4d95f914 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -5057,6 +5057,9 @@ static int pci_reset_bus_function(struct pci_dev *dev, bool probe)
 	if (bridge && pcie_is_cxl(bridge) && cxl_sbr_masked(bridge))
 		return -ENOTTY;
 
+	if (probe)
+		return __pci_reset_bus_function(dev, probe);
+
 	rc = pci_dev_reset_iommu_prepare(dev);
 	if (rc) {
 		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", rc);
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 7f1f47ea2ee4..79c326b0e80f 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -4406,6 +4406,9 @@ static int __pci_dev_specific_reset(struct pci_dev *dev, bool probe,
 {
 	int ret;
 
+	if (probe)
+		return i->reset(dev, probe);
+
 	ret = pci_dev_reset_iommu_prepare(dev);
 	if (ret) {
 		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", ret);
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH V1 3/4] PCI/sysfs: Take the device lock when setting reset methods
  2026-11-19 10:06 [PATCH V1 0/4] PCI: Don't list reset methods that can't reset the device Vidya Sagar
  2026-11-19 10:06 ` [PATCH V1 1/4] PCI/CXL: Don't offer 'cxl_bus' reset where a bus reset can't work Vidya Sagar
  2026-11-19 10:06 ` [PATCH V1 2/4] PCI: Don't run IOMMU reset hooks when only probing reset methods Vidya Sagar
@ 2026-11-19 10:06 ` Vidya Sagar
  2026-11-19 10:20   ` sashiko-bot
  2026-11-19 10:06 ` [PATCH V1 4/4] PCI: Re-check function 0 reset methods after scanning its slot Vidya Sagar
  3 siblings, 1 reply; 9+ messages in thread
From: Vidya Sagar @ 2026-11-19 10:06 UTC (permalink / raw)
  To: Bjorn Helgaas, Dan Williams, Dave Jiang, Jonathan Cameron,
	Joerg Roedel, Jason Gunthorpe, Nicolin Chen, Kevin Tian
  Cc: vsethi, kthota, mmaddireddy, kumarahul, sagar.tv, linux-pci,
	linux-kernel, Vidya Sagar

reset_method_store() updates dev->reset_methods without holding any
lock.  A reset can run at the same time: __pci_reset_function_locked()
walks the list one entry at a time while the write copies the new list
over it, so the reset may use a mix of the old and the new list.

Take the device lock in reset_method_store(), as
__pci_reset_function_locked() already requires.  This also serializes
concurrent writes.  The reset methods already run under this lock during
a reset, and probing a method does less than running it, so probing them
with the lock held is safe.  A write now waits while a reset of the
device is in progress.

Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
---
 drivers/pci/pci-sysfs.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c
index 6669e9ad5bd3..dff3c2d9408c 100644
--- a/drivers/pci/pci-sysfs.c
+++ b/drivers/pci/pci-sysfs.c
@@ -1643,6 +1643,9 @@ static ssize_t reset_method_store(struct device *dev,
 	int m, n;
 	u8 reset_methods[PCI_NUM_RESET_METHODS] = {};
 
+	/* Serialize against resets and other updates of the list */
+	guard(device)(dev);
+
 	if (sysfs_streq(buf, "")) {
 		pdev->reset_methods[0] = 0;
 		pci_warn(pdev, "All device reset methods disabled by user");
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH V1 4/4] PCI: Re-check function 0 reset methods after scanning its slot
  2026-11-19 10:06 [PATCH V1 0/4] PCI: Don't list reset methods that can't reset the device Vidya Sagar
                   ` (2 preceding siblings ...)
  2026-11-19 10:06 ` [PATCH V1 3/4] PCI/sysfs: Take the device lock when setting " Vidya Sagar
@ 2026-11-19 10:06 ` Vidya Sagar
  2026-11-19 10:32   ` sashiko-bot
  3 siblings, 1 reply; 9+ messages in thread
From: Vidya Sagar @ 2026-11-19 10:06 UTC (permalink / raw)
  To: Bjorn Helgaas, Dan Williams, Dave Jiang, Jonathan Cameron,
	Joerg Roedel, Jason Gunthorpe, Nicolin Chen, Kevin Tian
  Cc: vsethi, kthota, mmaddireddy, kumarahul, sagar.tv, linux-pci,
	linux-kernel, Vidya Sagar

pci_device_add() probes a device's reset methods before it adds the
device to bus->devices.  pci_scan_slot() adds function 0 first, so when
function 0 is probed it is still alone on its bus and the "bus" probe,
which refuses a bus reset if other devices share the bus, succeeds.  Once
the other functions are added, the same check refuses the reset, so
function 0 of a multi-function device lists "bus" but can't use it.  For
example, on function 0 of a 4-function ConnectX-8:

  # cat /sys/bus/pci/devices/0002:01:00.0/reset_method
  flr bus cxl_bus
  # echo bus > /sys/bus/pci/devices/0002:01:00.0/reset_method
  bash: echo: write error: Invalid argument
  # echo default > /sys/bus/pci/devices/0002:01:00.0/reset_method
  # cat /sys/bus/pci/devices/0002:01:00.0/reset_method
  flr

The other functions are probed after function 0 is on the bus, so they
are not affected.

At the end of pci_scan_slot(), if the scan added function 0 and other
functions, probe each of function 0's reset methods again and drop the
ones that now fail.  By then device_add() has already created function
0's "reset_method" file and sent the "add" uevent, so userspace may have
changed the list already.  Add pci_prune_reset_methods() for this: unlike
pci_init_reset_methods(), it only removes methods and keeps the order of
the rest.  Hold the device lock while doing so.  reset_method_store()
takes the same lock, so a write can't land between reading the list and
storing the result.  pci_bus_add_device() takes this lock later in the
same scan, for device_initial_probe().

No driver can bind to function 0, and its runtime PM is not enabled,
until pci_bus_add_device(), which runs after the slot scan.

Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
---
 drivers/pci/pci.c   | 35 +++++++++++++++++++++++++++++++++++
 drivers/pci/pci.h   |  1 +
 drivers/pci/probe.c | 16 +++++++++++++++-
 3 files changed, 51 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 918a4d95f914..d9b26208a76c 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -5306,6 +5306,41 @@ void pci_init_reset_methods(struct pci_dev *dev)
 	dev->reset_methods[i] = 0;
 }
 
+/**
+ * pci_prune_reset_methods - drop reset methods that no longer work
+ * @dev: PCI device to check
+ *
+ * Probe each of @dev's current reset methods again and remove the ones
+ * that fail, keeping the order of the rest.  Unlike pci_init_reset_methods(),
+ * this never adds a method, so a list set through sysfs is kept, except for
+ * methods that can no longer work.
+ *
+ * Context: The caller must hold the device lock of @dev, which serializes
+ * this against resets and writes to the "reset_method" sysfs file.
+ */
+void pci_prune_reset_methods(struct pci_dev *dev)
+{
+	u8 methods[PCI_NUM_RESET_METHODS] = {};
+	int m, i, n = 0, rc;
+
+	might_sleep();
+	device_lock_assert(&dev->dev);
+
+	for (i = 0; i < PCI_NUM_RESET_METHODS; i++) {
+		m = dev->reset_methods[i];
+		if (!m)
+			break;
+
+		rc = pci_reset_fn_methods[m].reset_fn(dev, PCI_RESET_PROBE);
+		if (!rc)
+			methods[n++] = m;
+		else if (rc != -ENOTTY)
+			break;
+	}
+
+	memcpy(dev->reset_methods, methods, sizeof(dev->reset_methods));
+}
+
 /**
  * pci_reset_function - quiesce and reset a PCI device function
  * @dev: PCI device to reset
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 77efa4c5c960..6e82c82acdb6 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -242,6 +242,7 @@ int pci_mmap_fits(struct pci_dev *pdev, int resno, struct vm_area_struct *vmai,
 
 bool pci_reset_supported(struct pci_dev *dev);
 void pci_init_reset_methods(struct pci_dev *dev);
+void pci_prune_reset_methods(struct pci_dev *dev);
 int pci_bridge_secondary_bus_reset(struct pci_dev *dev);
 int pci_bus_error_reset(struct pci_dev *dev);
 int pci_try_reset_bridge(struct pci_dev *bridge);
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 9a178e0e5f7f..a36bf679c393 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2874,7 +2874,7 @@ static int only_one_child(struct pci_bus *bus)
  */
 int pci_scan_slot(struct pci_bus *bus, int devfn)
 {
-	struct pci_dev *dev;
+	struct pci_dev *dev, *dev0 = NULL;
 	int fn = 0, nr = 0;
 
 	if (only_one_child(bus) && (devfn > 0))
@@ -2887,6 +2887,8 @@ int pci_scan_slot(struct pci_bus *bus, int devfn)
 				nr++;
 			if (fn > 0)
 				dev->multifunction = 1;
+			else
+				dev0 = dev;
 		} else if (fn == 0) {
 			/*
 			 * Function 0 is required unless we are running on
@@ -2899,6 +2901,18 @@ int pci_scan_slot(struct pci_bus *bus, int devfn)
 		fn = next_fn(bus, dev, fn);
 	} while (fn >= 0);
 
+	/*
+	 * Function 0's reset methods were probed before the other functions
+	 * were on bus->devices, so a bus reset looked possible then.  Its
+	 * "reset_method" file already exists and userspace may have changed
+	 * the list, so only drop the methods that can't work now.  The device
+	 * lock keeps a write to that file from landing in the middle.
+	 */
+	if (dev0 && !pci_dev_is_added(dev0) && nr > 1) {
+		guard(device)(&dev0->dev);
+		pci_prune_reset_methods(dev0);
+	}
+
 	/* Only one slot has PCIe device */
 	if (bus->self && nr)
 		pcie_aspm_init_link_state(bus->self);
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [PATCH V1 2/4] PCI: Don't run IOMMU reset hooks when only probing reset methods
  2026-11-19 10:06 ` [PATCH V1 2/4] PCI: Don't run IOMMU reset hooks when only probing reset methods Vidya Sagar
@ 2026-11-19 10:14   ` sashiko-bot
  0 siblings, 0 replies; 9+ messages in thread
From: sashiko-bot @ 2026-11-19 10:14 UTC (permalink / raw)
  To: Vidya Sagar; +Cc: linux-pci

> pci_reset_bus_function() and __pci_dev_specific_reset() call
> pci_dev_reset_iommu_prepare() and pci_dev_reset_iommu_done() even when
> "probe" is set and no reset follows.  The other reset methods that use
> these hooks, pcie_flr(), pci_af_flr(), pci_pm_reset() and
> pci_dev_acpi_reset(), only call them when actually resetting.
> 
> Reset methods are probed at enumeration, before the IOMMU is set up, but
> also whenever "reset_method" is written in sysfs.  For an ATS-capable
> device behind an IOMMU, each such probe of "bus" or of a device-specific
> reset attaches the blocking domain and then re-attaches the device's
> domains.  That blocks any DMA in the window and, on arm-smmu-v3, sends
> ATC invalidations, even while a driver is bound and using the device.
> 
> Return before the IOMMU hooks when probing, as the other reset methods
> do.
> 
> Fixes: f5b16b802174 ("PCI: Suspend iommu function prior to resetting a device")
> Signed-off-by: Vidya Sagar <vidyas@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119100628.1948990-2-vidyas@nvidia.com?part=2


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH V1 1/4] PCI/CXL: Don't offer 'cxl_bus' reset where a bus reset can't work
  2026-11-19 10:06 ` [PATCH V1 1/4] PCI/CXL: Don't offer 'cxl_bus' reset where a bus reset can't work Vidya Sagar
@ 2026-11-19 10:20   ` sashiko-bot
  0 siblings, 0 replies; 9+ messages in thread
From: sashiko-bot @ 2026-11-19 10:20 UTC (permalink / raw)
  To: Vidya Sagar; +Cc: linux-pci

> cxl_reset_bus_function() reports the "cxl_bus" reset method as supported
> whenever the device's upstream bridge has a CXL Port DVSEC.  The reset
> itself goes through pci_reset_bus_function(), which can only reset the
> device through a hotplug slot or a Secondary Bus Reset of the parent
> bridge.  It returns -ENOTTY when neither is possible, e.g., when other
> functions share the device's bus, when the device is a bridge, or when it
> has PCI_DEV_FLAGS_NO_BUS_RESET.
> 
> "cxl_bus" is then listed in "reset_method" but always fails.  For example,
> on function 1 of a 4-function ConnectX-8 below a CXL-capable Root Port
> without a hotplug slot, "bus" is correctly not offered, but "cxl_bus" is:
> 
>   # cat /sys/bus/pci/devices/0001:01:00.1/reset_method
>   flr cxl_bus
>   # echo cxl_bus > /sys/bus/pci/devices/0001:01:00.1/reset_method
>   # echo 1 > /sys/bus/pci/devices/0001:01:00.1/reset
>   bash: echo: write error: Inappropriate ioctl for device
> 
> Each attempt still temporarily sets "Unmask SBR" in the Root Port and runs
> the IOMMU reset prepare/done hooks, although no reset happens.
> 
> Move the slot and parent bus reset from pci_reset_bus_function() into a
> new __pci_reset_bus_function() and use it for the "cxl_bus" probe.  The
> probe now makes the same slot and parent-bus checks as the "bus" probe.
> "cxl_bus" is then offered under the same conditions as "bus", except for
> the CXL SBR mask that "cxl_bus" exists to clear.
> 
> Fixes: 53c49b6e6dd2 ("PCI/CXL: Add 'cxl_bus' reset method for devices below CXL Ports")
> Signed-off-by: Vidya Sagar <vidyas@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119100628.1948990-2-vidyas@nvidia.com?part=1


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH V1 3/4] PCI/sysfs: Take the device lock when setting reset methods
  2026-11-19 10:06 ` [PATCH V1 3/4] PCI/sysfs: Take the device lock when setting " Vidya Sagar
@ 2026-11-19 10:20   ` sashiko-bot
  0 siblings, 0 replies; 9+ messages in thread
From: sashiko-bot @ 2026-11-19 10:20 UTC (permalink / raw)
  To: Vidya Sagar; +Cc: linux-pci

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] PCI/sysfs: missing Fixes trailer for reset_method_store() lock
  fix
--

commit f6a788ffd2891c157884c924ff811b7d6b5ba6aa
Author: Vidya Sagar <vidyas@nvidia.com>

> PCI/sysfs: Take the device lock when setting reset methods
>
> reset_method_store() updates dev->reset_methods without holding any
> lock.  A reset can run at the same time: __pci_reset_function_locked()
> walks the list one entry at a time while the write copies the new list
> over it, so the reset may use a mix of the old and the new list.
>
> Take the device lock in reset_method_store(), as
> __pci_reset_function_locked() already requires.  This also serializes
> concurrent writes.  The reset methods already run under this lock during
> a reset, and probing a method does less than running it, so probing them
> with the lock held is safe.  A write now waits while a reset of the
> device is in progress.
>
> Signed-off-by: Vidya Sagar <vidyas@nvidia.com>

[Severity: Low]
This fixes a race condition in reset_method_store().  Should it include:

  Fixes: d88f521da3ef ("PCI: Allow userspace to query and set device reset mechanism")

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119100628.1948990-2-vidyas@nvidia.com?part=3

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH V1 4/4] PCI: Re-check function 0 reset methods after scanning its slot
  2026-11-19 10:06 ` [PATCH V1 4/4] PCI: Re-check function 0 reset methods after scanning its slot Vidya Sagar
@ 2026-11-19 10:32   ` sashiko-bot
  0 siblings, 0 replies; 9+ messages in thread
From: sashiko-bot @ 2026-11-19 10:32 UTC (permalink / raw)
  To: Vidya Sagar; +Cc: linux-pci

Thank you for your contribution! Sashiko AI review found 4 potential
issues to consider:
- [High] PCI: AB-BA deadlock between pci_scan_slot() and pciehp
  reset_lock
- [Medium] PCI: stale reset sysfs attributes after
  pci_prune_reset_methods()
- [Medium] PCI: incomplete reset method pruning conditions in
  pci_scan_slot()
- [Medium] PCI: late or missing multifunction flag breaks hotplug slot
  reset probe
--

commit a12801a8cda3240b7730ba41ca17db3de977e4a2
Author: Vidya Sagar <vidyas@nvidia.com>

PCI: Re-check function 0 reset methods after scanning its slot

This commit adds pci_prune_reset_methods() and calls it on function 0 at
the end of pci_scan_slot() when multiple functions are discovered,
removing reset methods that no longer probe successfully once sibling
functions are on the bus.

> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 05d049f7ba75b..4175cca8e6e31 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -5489,6 +5489,41 @@ void pci_init_reset_methods(struct pci_dev *dev)
>  	dev->reset_methods[i] = 0;
>  }
>  
> +/**
> + * pci_prune_reset_methods - drop reset methods that no longer work
> + * @dev: PCI device to check
> + *
> + * Probe each of @dev's current reset methods again and remove the ones
> + * that fail, keeping the order of the rest.  Unlike pci_init_reset_methods(),
> + * this never adds a method, so a list set through sysfs is kept, except for
> + * methods that can no longer work.
> + *
> + * Context: The caller must hold the device lock of @dev, which serializes
> + * this against resets and writes to the "reset_method" sysfs file.
> + */
> +void pci_prune_reset_methods(struct pci_dev *dev)
> +{
> +	u8 methods[PCI_NUM_RESET_METHODS] = {};
> +	int m, i, n = 0, rc;
> +
> +	might_sleep();
> +	device_lock_assert(&dev->dev);
> +
> +	for (i = 0; i < PCI_NUM_RESET_METHODS; i++) {
> +		m = dev->reset_methods[i];
> +		if (!m)
> +			break;
> +
> +		rc = pci_reset_fn_methods[m].reset_fn(dev, PCI_RESET_PROBE);
> +		if (!rc)
> +			methods[n++] = m;
> +		else if (rc != -ENOTTY)
> +			break;
> +	}
> +
> +	memcpy(dev->reset_methods, methods, sizeof(dev->reset_methods));
> +}

[Severity: Medium]
What happens to the reset and reset_method sysfs attributes if pruning
removes every method from dev->reset_methods[]?

When function 0 of a multi-function device supports no function-level
reset methods, bus (and cxl_bus) initially succeed during
pci_device_add() while bus->devices is empty.  device_add() then sees
pci_reset_supported(dev0) == true and creates the reset and reset_method
files.  When pci_prune_reset_methods() later zeroes
dev0->reset_methods[], pci_reset_supported(dev0) becomes false, but the
sysfs groups are not updated, leaving an empty reset_method file and a
reset file that fails with -ENOTTY on function 0 while functions 1..N
omit both files.

> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index 222ad696a1581..0f9b4115abd8d 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -2882,7 +2882,7 @@ static int only_one_child(struct pci_bus *bus)
>   */
>  int pci_scan_slot(struct pci_bus *bus, int devfn)
>  {
> -	struct pci_dev *dev;
> +	struct pci_dev *dev, *dev0 = NULL;
>  	int fn = 0, nr = 0;
>  
>  	if (only_one_child(bus) && (devfn > 0))
> @@ -2895,6 +2895,8 @@ int pci_scan_slot(struct pci_bus *bus, int devfn)
>  				nr++;
>  			if (fn > 0)
>  				dev->multifunction = 1;
> +			else
> +				dev0 = dev;

[Severity: Medium]
Can bus and cxl_bus still be left enabled on multi-function devices in a
hotplug slot that implements reset_slot?

__pci_reset_bus_function() tries pci_dev_reset_slot_function() before
pci_parent_bus_reset(), and pci_dev_reset_slot_function() rejects
multi-function devices by checking dev->multifunction rather than
dev->bus->devices:

drivers/pci/pci.c:pci_dev_reset_slot_function() {
	if (dev->subordinate || !dev->slot ||
	    dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET ||
	    (dev->multifunction && !dev->slot->per_func_slot))
		return -ENOTTY;

	return pci_reset_hotplug_slot(dev->slot->hotplug, probe);
}

On ARI multi-function devices, PCI_HEADER_TYPE_MFD is hardwired to 0b in
config space and pci_scan_slot() never sets dev0->multifunction = 1 when
fn > 0 is found, so pci_prune_reset_methods(dev0) still succeeds in
pci_dev_reset_slot_function() and keeps bus on function 0.  In addition,
for fn > 0 whose own Header Type register does not set
PCI_HEADER_TYPE_MFD, dev->multifunction is only set to 1 here after
pci_scan_single_device() has already probed reset methods, so fn > 0
records bus during probe and is never pruned.

>  		} else if (fn == 0) {
>  			/*
>  			 * Function 0 is required unless we are running on
> @@ -2907,6 +2909,18 @@ int pci_scan_slot(struct pci_bus *bus, int devfn)
>  		fn = next_fn(bus, dev, fn);
>  	} while (fn >= 0);
>  
> +	/*
> +	 * Function 0's reset methods were probed before the other functions
> +	 * were on bus->devices, so a bus reset looked possible then.  Its
> +	 * "reset_method" file already exists and userspace may have changed
> +	 * the list, so only drop the methods that can't work now.  The device
> +	 * lock keeps a write to that file from landing in the middle.
> +	 */
> +	if (dev0 && !pci_dev_is_added(dev0) && nr > 1) {

[Severity: Medium]
Does this condition miss other cases where the first device on a bus is
probed alone and later shares bus->devices?

pci_parent_bus_reset() refuses a bus reset whenever any other device is
on dev->bus->devices, but pruning only runs when fn 0 and another
function are both newly discovered in the same pci_scan_slot() call.
The first device on a bus still keeps an unusable bus reset method when:
- hypervisor_isolated_pci_functions() scans multiple non-zero functions
  without fn 0 (dev0 remains NULL),
- a later rescan adds a newly enabled sibling function alongside an
  already-added fn 0 (pci_dev_is_added(dev0) is already true and nr can
  be 1), or
- a bus has multiple devices at different slot numbers scanned across
  separate pci_scan_slot() calls.

> +		guard(device)(&dev0->dev);
> +		pci_prune_reset_methods(dev0);
> +	}

[Severity: High]
Does taking device_lock(&dev0->dev) inside pci_scan_slot() invert the
lock order against pciehp's ctrl->reset_lock?

During hotplug enumeration, pciehp_configure_device() holds
ctrl->reset_lock across pci_scan_slot() and pci_hp_add_bridge(), and only
drops ctrl->reset_lock around pci_bus_add_devices() to avoid an ABBA
deadlock with device_lock.  Meanwhile, slot and function resets acquire
device_lock(&dev->dev) before pciehp_reset_slot() takes ctrl->reset_lock
for write:

CPU0                              CPU1
----                              ----
down_read(&ctrl->reset_lock)
                                  device_lock(&dev0->dev)
                                  down_write(&ctrl->reset_lock)
device_lock(&dev0->dev)

 *** DEADLOCK ***

CPU0: pciehp_ist() -> ... -> pciehp_configure_device() -> pci_scan_slot()
CPU1: pci_reset_function() / reset_method_store() / pci_slot_reset() ->
      ... -> pciehp_reset_slot()

Because device_add(&dev0->dev) and list_add_tail(&dev0->bus_list,
&bus->devices) have already run before functions 1..N are scanned, CPU1
can lock dev0 while CPU0 is still scanning the remaining functions in
pci_scan_slot().

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119100628.1948990-2-vidyas@nvidia.com?part=4

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2026-11-19 10:32 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-19 10:06 [PATCH V1 0/4] PCI: Don't list reset methods that can't reset the device Vidya Sagar
2026-11-19 10:06 ` [PATCH V1 1/4] PCI/CXL: Don't offer 'cxl_bus' reset where a bus reset can't work Vidya Sagar
2026-11-19 10:20   ` sashiko-bot
2026-11-19 10:06 ` [PATCH V1 2/4] PCI: Don't run IOMMU reset hooks when only probing reset methods Vidya Sagar
2026-11-19 10:14   ` sashiko-bot
2026-11-19 10:06 ` [PATCH V1 3/4] PCI/sysfs: Take the device lock when setting " Vidya Sagar
2026-11-19 10:20   ` sashiko-bot
2026-11-19 10:06 ` [PATCH V1 4/4] PCI: Re-check function 0 reset methods after scanning its slot Vidya Sagar
2026-11-19 10:32   ` sashiko-bot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox