Linux PCI subsystem development
 help / color / mirror / Atom feed
From: Danny Gonzalez <digonzal@google.com>
To: Bjorn Helgaas <bhelgaas@google.com>,
	Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>,
	 Robin Murphy <robin.murphy@arm.com>
Cc: "Rafael J . Wysocki" <rafael@kernel.org>,
	Len Brown <lenb@kernel.org>, Jason Gunthorpe <jgg@nvidia.com>,
	 Nicolin Chen <nicolinc@nvidia.com>,
	Lu Baolu <baolu.lu@linux.intel.com>,
	 Kevin Tian <kevin.tian@intel.com>,
	decot@google.com, praan@google.com,  sheenamo@google.com,
	linux-pci@vger.kernel.org, iommu@lists.linux.dev,
	 linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org,
	 Danny Gonzalez <digonzal@google.com>
Subject: [PATCH v3 1/2] iommu: Track whether pci_dev_reset_iommu_done() needs to restore state
Date: Fri, 20 Nov 2026 00:23:10 +0000	[thread overview]
Message-ID: <20261120002311.394649-2-digonzal@google.com> (raw)
In-Reply-To: <20261120002311.394649-1-digonzal@google.com>

Currently, pci_dev_reset_iommu_done() relies on the assumption that
pci_ats_supported() and dev_has_iommu() always return the same values
they returned in pci_dev_reset_iommu_prepare(). While that may be true
today, relying on both calls evaluating identically is fragile: if they
ever return different values, gdev->reset_depth, group->recovery_cnt,
group->domain, and gdev->blocked will get out of sync.

Have pci_dev_reset_iommu_prepare() track whether
pci_dev_reset_iommu_done() needs to restore IOMMU state via a @restore
output flag, and pass @restore to pci_dev_reset_iommu_done() instead of
re-evaluating pci_ats_supported() and dev_has_iommu().

Signed-off-by: Danny Gonzalez <digonzal@google.com>
---

v3:
 - New patch based on review feedback to avoid relying on
   pci_ats_supported() and dev_has_iommu() returning identical values
   across pci_dev_reset_iommu_prepare() and pci_dev_reset_iommu_done()

 drivers/iommu/iommu.c  | 14 ++++++++++----
 drivers/pci/pci-acpi.c |  5 +++--
 drivers/pci/pci.c      | 25 +++++++++++++++----------
 drivers/pci/quirks.c   |  5 +++--
 include/linux/iommu.h  |  9 +++++----
 5 files changed, 36 insertions(+), 22 deletions(-)

diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index b486b8bbd1fc..8a563eb38bba 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -4010,6 +4010,7 @@ EXPORT_SYMBOL_NS_GPL(iommu_replace_group_handle, "IOMMUFD_INTERNAL");
 /**
  * pci_dev_reset_iommu_prepare() - Block IOMMU to prepare for a PCI device reset
  * @pdev: PCI device that is going to enter a reset routine
+ * @restore: set to true if pci_dev_reset_iommu_done() needs to restore state
  *
  * The PCIe r6.0, sec 10.3.1 IMPLEMENTATION NOTE recommends to disable and block
  * ATS before initiating a reset. This means that a PCIe device during the reset
@@ -4031,7 +4032,7 @@ EXPORT_SYMBOL_NS_GPL(iommu_replace_group_handle, "IOMMUFD_INTERNAL");
  * case, callers must ensure there will be no racy iommu_release_device() call,
  * which otherwise would UAF the dev->iommu_group pointer.
  */
-int pci_dev_reset_iommu_prepare(struct pci_dev *pdev)
+int pci_dev_reset_iommu_prepare(struct pci_dev *pdev, bool *restore)
 {
 	struct iommu_group *group = pdev->dev.iommu_group;
 	struct group_device *gdev;
@@ -4039,8 +4040,12 @@ int pci_dev_reset_iommu_prepare(struct pci_dev *pdev)
 	void *entry;
 	int ret;
 
-	if (!pci_ats_supported(pdev) || !dev_has_iommu(&pdev->dev))
+	if (!pci_ats_supported(pdev) || !dev_has_iommu(&pdev->dev)) {
+		*restore = false;
 		return 0;
+	}
+
+	*restore = true;
 
 	guard(mutex)(&group->mutex);
 
@@ -4132,6 +4137,7 @@ static bool group_device_dma_alias_is_blocked(struct iommu_group *group,
 /**
  * pci_dev_reset_iommu_done() - Restore IOMMU after a PCI device reset is done
  * @pdev: PCI device that has finished a reset routine
+ * @restore: whether pci_dev_reset_iommu_prepare() blocked the IOMMU
  *
  * After a PCIe device finishes a reset routine, it wants to restore its IOMMU
  * activity, including new translation and cache invalidation, by re-attaching
@@ -4143,14 +4149,14 @@ static bool group_device_dma_alias_is_blocked(struct iommu_group *group,
  * Note that, although unlikely, there is a risk that re-attaching domains might
  * fail due to some unexpected happening like OOM.
  */
-void pci_dev_reset_iommu_done(struct pci_dev *pdev)
+void pci_dev_reset_iommu_done(struct pci_dev *pdev, bool restore)
 {
 	struct iommu_group *group = pdev->dev.iommu_group;
 	struct group_device *gdev;
 	unsigned long pasid;
 	void *entry;
 
-	if (!pci_ats_supported(pdev) || !dev_has_iommu(&pdev->dev))
+	if (!restore)
 		return;
 
 	guard(mutex)(&group->mutex);
diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
index 1150f2fbabf4..83a581796001 100644
--- a/drivers/pci/pci-acpi.c
+++ b/drivers/pci/pci-acpi.c
@@ -955,6 +955,7 @@ void pci_set_acpi_fwnode(struct pci_dev *dev)
 int pci_dev_acpi_reset(struct pci_dev *dev, bool probe)
 {
 	acpi_handle handle = ACPI_HANDLE(&dev->dev);
+	bool restore;
 	int ret;
 
 	if (!handle || !acpi_has_method(handle, "_RST"))
@@ -963,7 +964,7 @@ int pci_dev_acpi_reset(struct pci_dev *dev, bool probe)
 	if (probe)
 		return 0;
 
-	ret = pci_dev_reset_iommu_prepare(dev);
+	ret = pci_dev_reset_iommu_prepare(dev, &restore);
 	if (ret) {
 		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", ret);
 		return ret;
@@ -974,7 +975,7 @@ int pci_dev_acpi_reset(struct pci_dev *dev, bool probe)
 		ret = -ENOTTY;
 	}
 
-	pci_dev_reset_iommu_done(dev);
+	pci_dev_reset_iommu_done(dev, restore);
 	return ret;
 }
 
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 7c6f2aba8144..087594cc8095 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -4440,6 +4440,7 @@ EXPORT_SYMBOL(pci_wait_for_pending_transaction);
  */
 int pcie_flr(struct pci_dev *dev)
 {
+	bool restore;
 	int ret;
 
 	if (!pci_dev_config_accessible(dev, "FLR"))
@@ -4449,7 +4450,7 @@ int pcie_flr(struct pci_dev *dev)
 		pci_err(dev, "timed out waiting for pending transaction; performing function level reset anyway\n");
 
 	/* Have to call it after waiting for pending DMA transaction */
-	ret = pci_dev_reset_iommu_prepare(dev);
+	ret = pci_dev_reset_iommu_prepare(dev, &restore);
 	if (ret) {
 		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", ret);
 		return ret;
@@ -4469,7 +4470,7 @@ int pcie_flr(struct pci_dev *dev)
 
 	ret = pci_dev_wait(dev, "FLR", PCIE_RESET_READY_POLL_MS);
 done:
-	pci_dev_reset_iommu_done(dev);
+	pci_dev_reset_iommu_done(dev, restore);
 	return ret;
 }
 EXPORT_SYMBOL_GPL(pcie_flr);
@@ -4498,6 +4499,7 @@ EXPORT_SYMBOL_GPL(pcie_reset_flr);
 
 static int pci_af_flr(struct pci_dev *dev, bool probe)
 {
+	bool restore;
 	int ret;
 	int pos;
 	u8 cap;
@@ -4526,7 +4528,7 @@ static int pci_af_flr(struct pci_dev *dev, bool probe)
 		pci_err(dev, "timed out waiting for pending transaction; performing AF function level reset anyway\n");
 
 	/* Have to call it after waiting for pending DMA transaction */
-	ret = pci_dev_reset_iommu_prepare(dev);
+	ret = pci_dev_reset_iommu_prepare(dev, &restore);
 	if (ret) {
 		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", ret);
 		return ret;
@@ -4547,7 +4549,7 @@ static int pci_af_flr(struct pci_dev *dev, bool probe)
 
 	ret = pci_dev_wait(dev, "AF_FLR", PCIE_RESET_READY_POLL_MS);
 done:
-	pci_dev_reset_iommu_done(dev);
+	pci_dev_reset_iommu_done(dev, restore);
 	return ret;
 }
 
@@ -4568,6 +4570,7 @@ static int pci_af_flr(struct pci_dev *dev, bool probe)
  */
 static int pci_pm_reset(struct pci_dev *dev, bool probe)
 {
+	bool restore;
 	u16 csr;
 	int ret;
 
@@ -4584,7 +4587,7 @@ static int pci_pm_reset(struct pci_dev *dev, bool probe)
 	if (dev->current_state != PCI_D0)
 		return -EINVAL;
 
-	ret = pci_dev_reset_iommu_prepare(dev);
+	ret = pci_dev_reset_iommu_prepare(dev, &restore);
 	if (ret) {
 		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", ret);
 		return ret;
@@ -4601,7 +4604,7 @@ static int pci_pm_reset(struct pci_dev *dev, bool probe)
 	pci_dev_d3_sleep(dev);
 
 	ret = pci_dev_wait(dev, "PM D3hot->D0", PCIE_RESET_READY_POLL_MS);
-	pci_dev_reset_iommu_done(dev);
+	pci_dev_reset_iommu_done(dev, restore);
 	return ret;
 }
 
@@ -5033,6 +5036,7 @@ static bool cxl_sbr_masked(struct pci_dev *dev)
 static int pci_reset_bus_function(struct pci_dev *dev, bool probe)
 {
 	struct pci_dev *bridge = pci_upstream_bridge(dev);
+	bool restore;
 	int rc;
 
 	/*
@@ -5042,7 +5046,7 @@ static int pci_reset_bus_function(struct pci_dev *dev, bool probe)
 	if (bridge && pcie_is_cxl(bridge) && cxl_sbr_masked(bridge))
 		return -ENOTTY;
 
-	rc = pci_dev_reset_iommu_prepare(dev);
+	rc = pci_dev_reset_iommu_prepare(dev, &restore);
 	if (rc) {
 		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", rc);
 		return rc;
@@ -5054,7 +5058,7 @@ static int pci_reset_bus_function(struct pci_dev *dev, bool probe)
 
 	rc = pci_parent_bus_reset(dev, probe);
 done:
-	pci_dev_reset_iommu_done(dev);
+	pci_dev_reset_iommu_done(dev, restore);
 	return rc;
 }
 
@@ -5062,6 +5066,7 @@ static int cxl_reset_bus_function(struct pci_dev *dev, bool probe)
 {
 	struct pci_dev *bridge;
 	u16 dvsec, reg, val;
+	bool restore;
 	int rc;
 
 	bridge = pci_upstream_bridge(dev);
@@ -5079,7 +5084,7 @@ static int cxl_reset_bus_function(struct pci_dev *dev, bool probe)
 	if (rc)
 		return -ENOTTY;
 
-	rc = pci_dev_reset_iommu_prepare(dev);
+	rc = pci_dev_reset_iommu_prepare(dev, &restore);
 	if (rc) {
 		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", rc);
 		return rc;
@@ -5099,7 +5104,7 @@ static int cxl_reset_bus_function(struct pci_dev *dev, bool probe)
 		pci_write_config_word(bridge, dvsec + PCI_DVSEC_CXL_PORT_CTL,
 				      reg);
 
-	pci_dev_reset_iommu_done(dev);
+	pci_dev_reset_iommu_done(dev, restore);
 	return rc;
 }
 
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 7f1f47ea2ee4..f6b26467c9e0 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -4404,16 +4404,17 @@ static const struct pci_dev_reset_methods pci_dev_reset_methods[] = {
 static int __pci_dev_specific_reset(struct pci_dev *dev, bool probe,
 				    const struct pci_dev_reset_methods *i)
 {
+	bool restore;
 	int ret;
 
-	ret = pci_dev_reset_iommu_prepare(dev);
+	ret = pci_dev_reset_iommu_prepare(dev, &restore);
 	if (ret) {
 		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", ret);
 		return ret;
 	}
 
 	ret = i->reset(dev, probe);
-	pci_dev_reset_iommu_done(dev);
+	pci_dev_reset_iommu_done(dev, restore);
 	return ret;
 }
 
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index ef5043752a36..7d6e27d0d918 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -1223,8 +1223,8 @@ ioasid_t iommu_alloc_global_pasid(struct device *dev);
 void iommu_free_global_pasid(ioasid_t pasid);
 
 /* PCI device reset functions */
-int pci_dev_reset_iommu_prepare(struct pci_dev *pdev);
-void pci_dev_reset_iommu_done(struct pci_dev *pdev);
+int pci_dev_reset_iommu_prepare(struct pci_dev *pdev, bool *restore);
+void pci_dev_reset_iommu_done(struct pci_dev *pdev, bool restore);
 #else /* CONFIG_IOMMU_API */
 
 struct iommu_ops {};
@@ -1549,12 +1549,13 @@ static inline ioasid_t iommu_alloc_global_pasid(struct device *dev)
 
 static inline void iommu_free_global_pasid(ioasid_t pasid) {}
 
-static inline int pci_dev_reset_iommu_prepare(struct pci_dev *pdev)
+static inline int pci_dev_reset_iommu_prepare(struct pci_dev *pdev, bool *restore)
 {
+	*restore = false;
 	return 0;
 }
 
-static inline void pci_dev_reset_iommu_done(struct pci_dev *pdev)
+static inline void pci_dev_reset_iommu_done(struct pci_dev *pdev, bool restore)
 {
 }
 #endif /* CONFIG_IOMMU_API */
-- 
2.56.1.708.g77570a6a1d-goog


  reply	other threads:[~2026-11-20  0:23 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-20  0:23 [PATCH v3 0/2] PCI/ATS: Disable ATS when ACS TB is set upstream Danny Gonzalez
2026-11-20  0:23 ` Danny Gonzalez [this message]
2026-11-20  0:33   ` [PATCH v3 1/2] iommu: Track whether pci_dev_reset_iommu_done() needs to restore state sashiko-bot
2026-11-20  0:23 ` [PATCH v3 2/2] PCI/ATS: Disable ATS when ACS TB is set upstream Danny Gonzalez
2026-11-20  0:34   ` sashiko-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261120002311.394649-2-digonzal@google.com \
    --to=digonzal@google.com \
    --cc=baolu.lu@linux.intel.com \
    --cc=bhelgaas@google.com \
    --cc=decot@google.com \
    --cc=iommu@lists.linux.dev \
    --cc=jgg@nvidia.com \
    --cc=joro@8bytes.org \
    --cc=kevin.tian@intel.com \
    --cc=lenb@kernel.org \
    --cc=linux-acpi@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=nicolinc@nvidia.com \
    --cc=praan@google.com \
    --cc=rafael@kernel.org \
    --cc=robin.murphy@arm.com \
    --cc=sheenamo@google.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox