[RFC PATCH] iommu/arm-smmu-v3: Allow nested attach for PCI bridges without vDEVICE

[RFC PATCH] iommu/arm-smmu-v3: Allow nested attach for PCI bridges without vDEVICE

[Dmitry Malkin]

IOMMU groups can contain passed-through endpoints and host PCI bridges.
QEMU creates vDEVICEs for the endpoints, but not for bridges that are
not visible to the guest.

Since f0ea207ed781 ("iommu/arm-smmu-v3: Introduce struct
arm_smmu_vmaster"), group-wide translated-domain attachment therefore
fails when the first bridge is visited:

  pcieport 0009:00:00.0:
    missing vDEVICE association for translated nested domain:
    cfg=5 ret=-2

  vfio-pci 0009:03:00.0:
    VFIO_DEVICE_ATTACH_IOMMUFD_PT attach failed for pt_id 19: -ENOENT

The same configuration worked with Linux 6.14.

Allow PCI bridges without a vDEVICE to attach without a vmaster.
Endpoint devices still require a vDEVICE, preserving vSID handling for
invalidations and virtual events. Log when the bridge exception is used.

Fixes: f0ea207ed781 ("iommu/arm-smmu-v3: Introduce struct arm_smmu_vmaster")
Cc: stable@vger.kernel.org
Signed-off-by: Dmitry Malkin <dma@nebius.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
@@ -5,6 +5,8 @@
 
 #include <uapi/linux/iommufd.h>
 
+#include <linux/pci.h>
+
 #include "arm-smmu-v3.h"
 
 void *arm_smmu_hw_info(struct device *dev, u32 *length,
@@ -118,6 +120,17 @@ int arm_smmu_attach_prepare_vmaster(struct arm_smmu_attach_state *state,
 		if (cfg == STRTAB_STE_0_CFG_ABORT ||
 		    cfg == STRTAB_STE_0_CFG_BYPASS)
 			return 0;
+		/*
+		 * Group-wide domain attachment also visits host PCI bridges. Such a
+		 * bridge is not exposed to the VM and therefore has no virtual SID.
+		 */
+		if (dev_is_pci(state->master->dev) &&
+		    pci_is_bridge(to_pci_dev(state->master->dev))) {
+			dev_info_ratelimited(state->master->dev,
+					     "skipping vDEVICE requirement for translated nested domain: cfg=%u ret=%d\n",
+					     cfg, ret);
+			return 0;
+		}
 		return ret;
 	}
 
-- 
2.43.0

Re: [RFC PATCH] iommu/arm-smmu-v3: Allow nested attach for PCI bridges without vDEVICE

[Pranjal Shrivastava]

On Fri, Aug 14, 2026 at 10:07:09AM +0000, Dmitry Malkin wrote:

Hi Dmitry,

[...]

> ---
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
> --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
> @@ -5,6 +5,8 @@
>  
>  #include <uapi/linux/iommufd.h>
>  
> +#include <linux/pci.h>
> +
>  #include "arm-smmu-v3.h"
>  
>  void *arm_smmu_hw_info(struct device *dev, u32 *length,
> @@ -118,6 +120,17 @@ int arm_smmu_attach_prepare_vmaster(struct arm_smmu_attach_state *state,
>  		if (cfg == STRTAB_STE_0_CFG_ABORT ||
>  		    cfg == STRTAB_STE_0_CFG_BYPASS)
>  			return 0;
> +		/*
> +		 * Group-wide domain attachment also visits host PCI bridges. Such a
> +		 * bridge is not exposed to the VM and therefore has no virtual SID.
> +		 */
> +		if (dev_is_pci(state->master->dev) &&
> +		    pci_is_bridge(to_pci_dev(state->master->dev))) {
> +			dev_info_ratelimited(state->master->dev,
> +					     "skipping vDEVICE requirement for translated nested domain: cfg=%u ret=%d\n",
> +					     cfg, ret);
> +			return 0;
> +		}
>  		return ret;
>  	}


I agree that this is needed for old bridges. However, should this check
actually live higher up in IOMMUFD? Instead of the SMMU driver deciding
to bypass the error, what if iommufd itself intercepts the vDEVICE 
mapping failure during the group iteration? If IOMMUFD sees that the 
device lacking a vDEVICE is an IOMMU group alias/bridge, it could 
explicitly tell the underlying driver to proceed with a NULL vmaster?

Also, regarding RID aliasing: while it's true the host bridge doesn't 
need a vDEVICE for its own host-consumed DMAs (like AER/PME), are we
confident this won't break guest-injected events if the bridge aliases
the downstream endpoint's traffic? For e.g. if the "real" endpoint's
traffic is aliased to the bridge's RID and the bridge has no vmaster,
won't we lose the ability to inject IO Page faults into the guest?

Thanks,
Praan

Re: [RFC PATCH] iommu/arm-smmu-v3: Allow nested attach for PCI bridges without vDEVICE

[Dmitry Malkin]

Hi Praan,

> I agree that this is needed for old bridges. However, should this check
> actually live higher up in IOMMUFD? Instead of the SMMU driver deciding
> to bypass the error, what if iommufd itself intercepts the vDEVICE
> mapping failure during the group iteration? If IOMMUFD sees that the
> device lacking a vDEVICE is an IOMMU group alias/bridge, it could
> explicitly tell the underlying driver to proceed with a NULL vmaster?
>
> Also, regarding RID aliasing: while it's true the host bridge doesn't
> need a vDEVICE for its own host-consumed DMAs (like AER/PME), are we
> confident this won't break guest-injected events if the bridge aliases
> the downstream endpoint's traffic? For e.g. if the "real" endpoint's
> traffic is aliased to the bridge's RID and the bridge has no vmaster,
> won't we lose the ability to inject IO Page faults into the guest?

Thanks, both points make sense.

IOMMUFD currently cannot handle this alone. It calls
iommu_replace_group_handle(), while the generic IOMMU core performs the
per-device iteration. IOMMUFD receives only the final error and cannot
tell attach_dev() to proceed with a NULL driver-private vmaster for a
specific device without a generic API change.

For the bridge case, I suggest limiting the exception to PCIe port types
that pci_for_each_dma_alias() explicitly skips, avoiding bridges that may
alias a downstream RID:

static bool arm_smmu_is_nonaliasing_pcie_port(struct device *dev)
{
	struct pci_dev *pdev;

	if (!dev_is_pci(dev))
		return false;

	pdev = to_pci_dev(dev);
	if (!pci_is_bridge(pdev) || !pci_is_pcie(pdev))
		return false;

	switch (pci_pcie_type(pdev)) {
	case PCI_EXP_TYPE_ROOT_PORT:
	case PCI_EXP_TYPE_UPSTREAM:
	case PCI_EXP_TYPE_DOWNSTREAM:
		return true;
	default:
		return false;
	}
}

Thanks,
Dmitry

Re: [RFC PATCH] iommu/arm-smmu-v3: Allow nested attach for PCI bridges without vDEVICE

[Jason Gunthorpe]

On Fri, Aug 14, 2026 at 10:07:09AM +0000, Dmitry Malkin wrote:
> IOMMU groups can contain passed-through endpoints and host PCI bridges.
> QEMU creates vDEVICEs for the endpoints, but not for bridges that are
> not visible to the guest.

But why exactly do you have pci bridges in your groups? That isn't
entirely normal?

> Since f0ea207ed781 ("iommu/arm-smmu-v3: Introduce struct
> arm_smmu_vmaster"), group-wide translated-domain attachment therefore
> fails when the first bridge is visited:
> 
>   pcieport 0009:00:00.0:
>     missing vDEVICE association for translated nested domain:
>     cfg=5 ret=-2
> 
>   vfio-pci 0009:03:00.0:
>     VFIO_DEVICE_ATTACH_IOMMUFD_PT attach failed for pt_id 19: -ENOENT
> 
> The same configuration worked with Linux 6.14.

Hmm, but without a viommu we shouldn't have this vmaster stuff
activating in the first place, I thought? 

+               if (new_domain->type == IOMMU_DOMAIN_NESTED) {
+                       ret = arm_smmu_attach_prepare_vmaster(
+                               state, to_smmu_nested_domain(new_domain));

That should only be happening if the VMM is setting up vIOMMU.

And vIOMMU wasn't supported in 6.14, so if it "worked" the VMM was
doing something quite different - not seeing this as a regression

If viommu should support multi-device groups, I'm not sure. I want to
know why you even have multi-device groups in the first place because
that is not normal at all.

Jason

Re: [RFC PATCH] iommu/arm-smmu-v3: Allow nested attach for PCI bridges without vDEVICE

[Jason Gunthorpe]

On Fri, Aug 14, 2026 at 02:15:45PM +0000, Pranjal Shrivastava wrote:

> Also, regarding RID aliasing: while it's true the host bridge doesn't
> need a vDEVICE for its own host-consumed DMAs (like AER/PME), are we

RID aliasing shouldn't exist in any modern system. Unfortunately Linux
gets some things wrong in its ACS processing :\

Jason

Re: [RFC PATCH] iommu/arm-smmu-v3: Allow nested attach for PCI bridges without vDEVICE

[Dmitry Malkin]

Hi Jason,

> If viommu should support multi-device groups, I'm not sure. I want to
> know why you even have multi-device groups in the first place because
> that is not normal at all.

That is because of the 'pci=config_acs' kernel command-line parameter. This causes the kernel to create a single IOMMU group containing multiple PCIe devices, including the bridge (PCIe Root Port).

Dmitry