[PATCH v8 0/4] iommu: Standardize ATS robustness and state tracking

[PATCH v8 0/4] iommu: Standardize ATS robustness and state tracking

[Pranjal Shrivastava]

The primary motivation for this series is an ATS state mismatch observed
under heavy load (via iova_stress). A failure in pci_enable_ats() leaves
IOMMU drivers like arm-smmu-v3 with inconsistent state leading to PCI core
warnings during device detach.

While David's recent work [1] addressed a discovery race for specific
quirked devices by moving them to the HEADER phase, gaps remained
regarding how Virtual Functions (VFs) inherit state from their Physical
Functions (PFs). Specifically, pci_ats_supported() did not account for
PF-level quirked status, and pci_prepare_ats() lacked STU validation for
VFs.

Based on discussion with Jason and Baolu in v3/v5, it was decided that the
IOMMU drivers should explicitly check pci_ats_supported() before calling
pci_prepare_ats(). To enforce this, pci_prepare_ats() now noisily checks
for support via WARN_ON(). Furthermore, the device probe should fail if
pci_prepare_ats() fails. Since these early gates preclude software
configuration errors, any remaining failure during pci_enable_ats() is
treated as a kernel bug.

Following the discussion with the community, the driver-specific series
have been posted separately:

 - Intel IOMMU fixes reported by Sashiko [2]
 - Refactors for AMD IOMMU [3]

[1] https://lore.kernel.org/linux-pci/20260403222750.1215002-1-dmatlack@google.com/
[2] https://lore.kernel.org/all/20260531170254.60493-1-praan@google.com/
[3] https://lore.kernel.org/all/20260601134204.2150602-1-praan@google.com/

[v8]
 - Collected R-b tags from Kevin & Lu
 - Dropped the SMMU dev_iommu_priv_set(dev, NULL) patch.

[v7]
 - https://lore.kernel.org/all/20260601143644.2358771-1-praan@google.com/
 - Moved patch PCI/ATS: Mandate checking pci_ats_supported() before pci_prepare_ats()
    to the AMD series [3] to maintain bisectibility
 - Added a UAF fix for arm-smmu-v3 to set iommu->priv = NULL

[v6]
  - Reverted the decoupling of pci_ats_supported() from pci_prepare_ats().
  - Added a WARN_ON() to the internal support check in pci_prepare_ats().
  - Dropped the standalone Intel bugfixes (RB-tree and UAF) to be sent as a
   separate standalone series per maintainer request.
  - Kept the folded UAF fix in the AMD IOMMU patch to ensure the new error
   path is immediately safe.
  - Collected Reviewed-by tags from Lu Baolu for PCI core patches.

[v5]
  - https://lore.kernel.org/all/20260528202353.3422206-1-praan@google.com/
  - Decoupled pci_ats_supported() from pci_prepare_ats() in the PCI core.
  - Rebased SMMUv3 support on top of Nicolin Chen's "Always-On ATS" series.
  - Fixed pre-existing RB-tree corruption in VT-d probe (Baolu/Sashiko).
  - Addressed the pre-existing UAF in AMD IOMMU probe suggested by Sashiko.

[v4]
  - https://lore.kernel.org/all/20260525184347.4059549-1-praan@google.com/
  - Standardized the pattern across Intel VT-d and AMD IOMMU drivers.
  - Replaced the SMMUv3 ats_prepared gate with a fatal probe-fail logic.
  - Utilized WARN() macros for runtime enablement failures in all drivers.
  - Collected R-b tags from Jason and Sami.

Pranjal Shrivastava (4):
  PCI/ATS: Ensure pci_ats_supported() is PF-aware for VFs
  PCI/ATS: Validate STU for VFs in pci_prepare_ats()
  iommu/arm-smmu-v3: Standardize ATS enablement failure reporting
  iommu/vt-d: Fail probe on ATS configuration failure

 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  7 ++++++-
 drivers/iommu/intel/iommu.c                 | 15 ++++++++++++---
 drivers/pci/ats.c                           | 13 ++++++++++---
 3 files changed, 28 insertions(+), 7 deletions(-)


base-commit: 542627e4a4aad76ccdf66436d7a8ea0d29069796
-- 
2.54.0.1032.g2f8565e1d1-goog

[PATCH v8 1/4] PCI/ATS: Ensure pci_ats_supported() is PF-aware for VFs

[Pranjal Shrivastava]

Update pci_ats_supported() to additionally check the associated PF's
status when called on a VF. This ensures that PF-level quirks and
untrusted status are correctly propagated to VFs, providing a robust
support check that aligns with the kernel's PF-centric ATS configuration
model and is immune to the timing of VF-specific fixups.

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Pranjal Shrivastava <praan@google.com>
---
 drivers/pci/ats.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c
index 96efa00d9743..679a3c3c1d54 100644
--- a/drivers/pci/ats.c
+++ b/drivers/pci/ats.c
@@ -40,10 +40,13 @@ void pci_ats_init(struct pci_dev *dev)
  */
 bool pci_ats_supported(struct pci_dev *dev)
 {
-	if (!dev->ats_cap)
+	if (!dev->ats_cap || dev->untrusted)
 		return false;
 
-	return (dev->untrusted == 0);
+	if (dev->is_virtfn)
+		return pci_ats_supported(pci_physfn(dev));
+
+	return true;
 }
 EXPORT_SYMBOL_GPL(pci_ats_supported);
 
-- 
2.54.0.1032.g2f8565e1d1-goog

[PATCH v8 2/4] PCI/ATS: Validate STU for VFs in pci_prepare_ats()

[Pranjal Shrivastava]

While every PCI Function that implements ATS has an independent ATS
Extended Capability structure with a Read/Write Smallest Translation
Unit (STU) field, the kernel manages SR-IOV ATS by requiring the IOMMU
driver to configure the STU on the Physical Function (PF) before any
any Virtual Functions (VFs) are created.

Currently, pci_prepare_ats() bails out early for VFs, assuming that the
PF has already been correctly prepared. However, this creates a potential
mismatch if a VF is subsequently prepared with a different page shift.

Update pci_prepare_ats() to validate that the requested page shift (ps)
matches the STU already configured in the associated PF. This ensures
early detection of incompatible configurations and maintains the kernel's
policy of consistent STU sizing across all functions associated with a
given SMMU.

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Pranjal Shrivastava <praan@google.com>
---
 drivers/pci/ats.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c
index 679a3c3c1d54..9cb23780093d 100644
--- a/drivers/pci/ats.c
+++ b/drivers/pci/ats.c
@@ -73,8 +73,12 @@ int pci_prepare_ats(struct pci_dev *dev, int ps)
 	if (ps < PCI_ATS_MIN_STU)
 		return -EINVAL;
 
-	if (dev->is_virtfn)
+	if (dev->is_virtfn) {
+		if (pci_physfn(dev)->ats_stu != ps)
+			return -EINVAL;
+
 		return 0;
+	}
 
 	dev->ats_stu = ps;
 	ctrl = PCI_ATS_CTRL_STU(dev->ats_stu - PCI_ATS_MIN_STU);
-- 
2.54.0.1032.g2f8565e1d1-goog

[PATCH v8 3/4] iommu/arm-smmu-v3: Standardize ATS enablement failure reporting

[Pranjal Shrivastava]

Update arm_smmu_enable_ats() to wrap the pci_enable_ats() call in
WARN_ON(). Since probe-time checks now preclude configuration errors
any failure during hardware enablement is considered a kernel bug.

Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Pranjal Shrivastava <praan@google.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index a10affb483a4..0494c68daa19 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2956,7 +2956,12 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master)
 	 * ATC invalidation of PASID 0 causes the entire ATC to be flushed.
 	 */
 	arm_smmu_atc_inv_master(master, IOMMU_NO_PASID);
-	if (pci_enable_ats(pdev, stu))
+
+	 /*
+	  * Since pci_prepare_ats() has already verified the HW capability
+	  * and programmed the STE, pci_enable_ats() should not fail here.
+	  */
+	if (WARN_ON(pci_enable_ats(pdev, stu)))
 		dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu);
 }
 
-- 
2.54.0.1032.g2f8565e1d1-goog

[PATCH v8 4/4] iommu/vt-d: Fail probe on ATS configuration failure

[Pranjal Shrivastava]

Update the Intel VT-d driver to handle ATS configuration and enablement
more strictly. Specifically, update the device probe to fail if
pci_prepare_ats() returns an error. This ensures that any ATS-capable
master reaching the attach phase is guaranteed to have a valid config.

Additionally, update iommu_enable_pci_ats() to WARN() if pci_enable_ats
fails. Since earlier checks in the probe phase preclude config-related
failures, any failure during hardware enablement is considered a kernel
bug.

Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Pranjal Shrivastava <praan@google.com>
---
 drivers/iommu/intel/iommu.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 849d06dfe1ae..f39451323553 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -876,8 +876,14 @@ static void iommu_enable_pci_ats(struct device_domain_info *info)
 	if (!pci_ats_page_aligned(pdev))
 		return;
 
-	if (!pci_enable_ats(pdev, VTD_PAGE_SHIFT))
-		info->ats_enabled = 1;
+	/*
+	 * pci_enable_ats() should not fail here because earlier checks
+	 * have already verified support and configuration.
+	 */
+	if (WARN_ON(pci_enable_ats(pdev, VTD_PAGE_SHIFT)))
+		return;
+
+	info->ats_enabled = 1;
 }
 
 static void iommu_disable_pci_ats(struct device_domain_info *info)
@@ -3292,7 +3298,10 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
 
 	dev_iommu_priv_set(dev, info);
 	if (pdev && pci_ats_supported(pdev)) {
-		pci_prepare_ats(pdev, VTD_PAGE_SHIFT);
+		ret = pci_prepare_ats(pdev, VTD_PAGE_SHIFT);
+		if (ret)
+			goto free;
+
 		ret = device_rbtree_insert(iommu, info);
 		if (ret)
 			goto free;
-- 
2.54.0.1032.g2f8565e1d1-goog

Re: [PATCH v8 3/4] iommu/arm-smmu-v3: Standardize ATS enablement failure reporting

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] The newly introduced WARN_ON() in arm_smmu_enable_ats() can be triggered from userspace for a Virtual Function (VF) due to an ignored probe-time error, potentially crashing the kernel.
- [Low] The newly added inline comment and commit message contain factual inaccuracies regarding STE programming and probe-time validation.
--

commit e98cc7d134471b7c205436036caf90a8f69d9088
Author: Pranjal Shrivastava <praan@google.com>

iommu/arm-smmu-v3: Standardize ATS enablement failure reporting

Update arm_smmu_enable_ats() to wrap the pci_enable_ats() call in
WARN_ON(). Since probe-time checks now preclude configuration errors
any failure during hardware enablement is considered a kernel bug.

> Since probe-time checks now preclude configuration errors
> any failure during hardware enablement is considered a kernel bug.

[Severity: Low]
This isn't a bug, but the commit message states that probe-time checks
now preclude configuration errors. As noted below, the return value of
pci_prepare_ats() isn't checked during probe, so this description doesn't
seem to match the actual driver behavior.

> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> index e8d7dbe495f03..a9bec5c7cbf98 100644
> --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> @@ -3065,7 +3065,12 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master)
>  	 * ATC invalidation of PASID 0 causes the entire ATC to be flushed.
>  	 */
>  	arm_smmu_atc_inv_master(master, IOMMU_NO_PASID);
> -	if (pci_enable_ats(pdev, stu))
> +
> +	 /*
> +	  * Since pci_prepare_ats() has already verified the HW capability
> +	  * and programmed the STE, pci_enable_ats() should not fail here.
> +	  */

[Severity: Low]
This isn't a bug, but there's a typo in the comment here. pci_prepare_ats()
programs the STU (Smallest Translation Unit) in the PCI ATS capability, not
the STE (Stream Table Entry), which is SMMU-specific.

> +	if (WARN_ON(pci_enable_ats(pdev, stu)))
>  		dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu);
>  }

[Severity: High]
Is it possible for pci_enable_ats() to fail here and trigger this warning from
userspace?

Looking at arm_smmu_probe_device(), the return value of pci_prepare_ats()
appears to be ignored:

arm_smmu_probe_device() {
	...
	pci_prepare_ats(to_pci_dev(dev), stu);
	...
}

If a Virtual Function (VF) is probed but its Physical Function (PF) has a
mismatched STU, pci_prepare_ats() would fail silently during probe. Later,
when userspace triggers a domain attach (e.g., via VFIO),
arm_smmu_enable_ats() would call pci_enable_ats() on the VF, which fails and
triggers this newly added WARN_ON.

Also, if the error is ignored here and the driver continues as if ATS is
enabled, could this trigger a second WARN_ON(!dev->ats_enabled) in
pci_disable_ats() during domain detachment?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260604182116.3179005-1-praan@google.com?part=3

Re: [PATCH v8 4/4] iommu/vt-d: Fail probe on ATS configuration failure

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Unconditional rbtree removal of an unlinked node on error paths in `intel_iommu_probe_device()` empties the device tracking tree.
--

commit edcf0a64984f0847eb81ddb322f530951d47a01a
Author: Pranjal Shrivastava <praan@google.com>

iommu/vt-d: Fail probe on ATS configuration failure

Update the Intel VT-d driver to handle ATS configuration and enablement
more strictly. Specifically, update the device probe to fail if
pci_prepare_ats() returns an error. This ensures that any ATS-capable
master reaching the attach phase is guaranteed to have a valid config.

> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index c3d18cd77d2f1..9242cd9651307 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c

[ ... ]

> @@ -3288,7 +3294,10 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
>  
>  	dev_iommu_priv_set(dev, info);
>  	if (pdev && pci_ats_supported(pdev)) {
> -		pci_prepare_ats(pdev, VTD_PAGE_SHIFT);
> +		ret = pci_prepare_ats(pdev, VTD_PAGE_SHIFT);
> +		if (ret)
> +			goto free;
> +
>  		ret = device_rbtree_insert(iommu, info);
>  		if (ret)
>  			goto free;

[Severity: High]
This is a pre-existing issue, but unconditional rbtree removal of an unlinked
node on error paths in intel_iommu_probe_device() empties the device
tracking tree.

If a device does not support ATS, it skips the device_rbtree_insert() call
above. If a subsequent failure occurs, such as intel_pasid_alloc_table()
failing, the error path jumps to clear_rbtree and unconditionally calls
device_rbtree_remove():

intel_iommu_probe_device() {
    ...
    clear_rbtree:
        device_rbtree_remove(info);
}

This function directly invokes rb_erase() without verifying whether the node
is actually linked to the tree:

device_rbtree_remove() {
    ...
    rb_erase(&info->node, &iommu->device_rbtree);
}

Because the node is unlinked and completely zero-initialized, the parent
pointer evaluates to NULL. The rbtree implementation treats the node as the
tree's root and overwrites the root with NULL, completely clearing the IOMMU
device tracking tree.

Could we add a check to verify the node is actually linked before calling
rb_erase(), or only jump to clear_rbtree if the node was successfully
inserted?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260604182116.3179005-1-praan@google.com?part=4

Re: [PATCH v8 3/4] iommu/arm-smmu-v3: Standardize ATS enablement failure reporting

[Nicolin Chen]

On Thu, Jun 04, 2026 at 06:21:15PM +0000, Pranjal Shrivastava wrote:
> Update arm_smmu_enable_ats() to wrap the pci_enable_ats() call in
> WARN_ON(). Since probe-time checks now preclude configuration errors
> any failure during hardware enablement is considered a kernel bug.
> 
> Reviewed-by: Kevin Tian <kevin.tian@intel.com>
> Signed-off-by: Pranjal Shrivastava <praan@google.com>
 
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>

Re: [PATCH v8 3/4] iommu/arm-smmu-v3: Standardize ATS enablement failure reporting

[Samiullah Khawaja]

On Thu, Jun 04, 2026 at 06:21:15PM +0000, Pranjal Shrivastava wrote:
>Update arm_smmu_enable_ats() to wrap the pci_enable_ats() call in
>WARN_ON(). Since probe-time checks now preclude configuration errors
>any failure during hardware enablement is considered a kernel bug.
>
>Reviewed-by: Kevin Tian <kevin.tian@intel.com>
>Signed-off-by: Pranjal Shrivastava <praan@google.com>
>---
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
>diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>index a10affb483a4..0494c68daa19 100644
>--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>@@ -2956,7 +2956,12 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master)
> 	 * ATC invalidation of PASID 0 causes the entire ATC to be flushed.
> 	 */
> 	arm_smmu_atc_inv_master(master, IOMMU_NO_PASID);
>-	if (pci_enable_ats(pdev, stu))
>+
>+	 /*
>+	  * Since pci_prepare_ats() has already verified the HW capability
>+	  * and programmed the STE, pci_enable_ats() should not fail here.
>+	  */
>+	if (WARN_ON(pci_enable_ats(pdev, stu)))
> 		dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu);
> }
>
>-- 
>2.54.0.1032.g2f8565e1d1-goog
>

Reviewed-by: Samiullah Khawaja <skhawaja@google.com>

Re: [PATCH v8 4/4] iommu/vt-d: Fail probe on ATS configuration failure

[Samiullah Khawaja]

On Thu, Jun 04, 2026 at 06:21:16PM +0000, Pranjal Shrivastava wrote:
>Update the Intel VT-d driver to handle ATS configuration and enablement
>more strictly. Specifically, update the device probe to fail if
>pci_prepare_ats() returns an error. This ensures that any ATS-capable
>master reaching the attach phase is guaranteed to have a valid config.
>
>Additionally, update iommu_enable_pci_ats() to WARN() if pci_enable_ats
>fails. Since earlier checks in the probe phase preclude config-related
>failures, any failure during hardware enablement is considered a kernel
>bug.
>
>Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
>Reviewed-by: Kevin Tian <kevin.tian@intel.com>
>Signed-off-by: Pranjal Shrivastava <praan@google.com>
>---
> drivers/iommu/intel/iommu.c | 15 ++++++++++++---
> 1 file changed, 12 insertions(+), 3 deletions(-)
>
>diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
>index 849d06dfe1ae..f39451323553 100644
>--- a/drivers/iommu/intel/iommu.c
>+++ b/drivers/iommu/intel/iommu.c
>@@ -876,8 +876,14 @@ static void iommu_enable_pci_ats(struct device_domain_info *info)
> 	if (!pci_ats_page_aligned(pdev))
> 		return;
>
>-	if (!pci_enable_ats(pdev, VTD_PAGE_SHIFT))
>-		info->ats_enabled = 1;
>+	/*
>+	 * pci_enable_ats() should not fail here because earlier checks
>+	 * have already verified support and configuration.
>+	 */
>+	if (WARN_ON(pci_enable_ats(pdev, VTD_PAGE_SHIFT)))
>+		return;
>+
>+	info->ats_enabled = 1;
> }
>
> static void iommu_disable_pci_ats(struct device_domain_info *info)
>@@ -3292,7 +3298,10 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
>
> 	dev_iommu_priv_set(dev, info);
> 	if (pdev && pci_ats_supported(pdev)) {
>-		pci_prepare_ats(pdev, VTD_PAGE_SHIFT);
>+		ret = pci_prepare_ats(pdev, VTD_PAGE_SHIFT);
>+		if (ret)
>+			goto free;
>+
> 		ret = device_rbtree_insert(iommu, info);
> 		if (ret)
> 			goto free;
>-- 
>2.54.0.1032.g2f8565e1d1-goog
>

Reviewed-by: Samiullah Khawaja <skhawaja@google.com>