Re: [PATCH v4 13/16] iommu/amd: Track host Domain ID mapping for each guest Domain ID

[Jason Gunthorpe <jgg@nvidia.com>]

On Tue, Oct 21, 2025 at 01:43:21AM +0000, Suravee Suthikulpanit wrote:
> Each nested domain is assigned guest domain ID (gDomID), which guest OS
> programs into guest Device Table Entry (gDTE). For each gDomID, the driver
> assigns a corresponding host domain ID (hDomID), which will be programmed
> into the host Device Table Entry (hDTE).
> 
> The gDTE to hDTE 1:1 mapping is stored in the nest parent domain using
> an xarray (struct protection_domain.gdomid_array). When invalidate the
> nest parent domain, the INVALIDATE_IOMMU_PAGES must be issued for each
> hDomID in the gdomid_array.

I think this should be stored in the viommu..

It is a small unrealistic detail but very pedantically the API allows
creating two VIOMMU's from the same NEST PARENT domain and if someone
did this then each of the VIOMMU should have its own private gDomID
number space and own separated xarray.

Allowing two VIOMMUs to share the same hDomID could be problematic
because we don't know the PASID layout is consistent.

> +static int iommu_flush_hdom_ids(struct amd_iommu *iommu,
> +				u64 address, size_t size,
> +				struct protection_domain *parent)
> +{
> +	int ret = 0;
> +	unsigned long i;
> +	struct iommu_cmd cmd;
> +	struct nested_domain *ndom;
> +
> +	xa_for_each(&parent->gdomid_array, i, ndom) {

This doesn't seem right.. There could be many nested_domains sharing
the same gDomID..

I expect this xarray to have a struct like

struct gdomid {
   refcount_t users;
   u32 hdomid;
};

And each nested_domain will go into the viommu and either allocate a
new gdomid or ++users for the existing one. Inverse when destroying a
nested_domain.

> @@ -92,6 +92,49 @@ amd_iommu_alloc_domain_nested(struct iommufd_viommu *viommu, u32 flags,
>  	ndom->domain.type = IOMMU_DOMAIN_NESTED;
>  	ndom->viommu = aviommu;
>  
> +	/*
> +	 * Normally, when a guest has multiple pass-through devices,
> +	 * the IOMMU driver setup DTEs with the same stage-2 table and
> +	 * use the same host domain ID (hDomId). In case of nested translation,
> +	 * if the guest setup different stage-1 tables with same PASID,
> +	 * IOMMU would use the same TLB tag. This will results in TLB
> +	 * aliasing issue.
> +	 *
> +	 * The guest is assigning gDomIDs based on its own algorithm for managing
> +	 * cache tags of (DomID, PASID). Within a single viommu, the nest parent domain
> +	 * (w/ S2 table) is used by all DTEs. But we need to consistently map the gDomID
> +	 * to a single hDomID. This is done using an xarray in the nest parent domain to
> +	 * keep track of the gDomID mapping. When the S2 is changed, the INVALIDATE_IOMMU_PAGES
> +	 * command must be issued for each hDomID in the xarray.
> +	 *
> +	 * Since there is no invalidation support and no viommu yet, just always use a
> +	 * unique hDomID for now.

It is not "for now" anymore, this is the correct algorithm..

Jason