Re: [PATCH v3 14/15] iommu/amd: Refactor logic to program the host page table in DTE

[Jason Gunthorpe <jgg@nvidia.com>]

On Thu, Oct 09, 2025 at 11:57:54PM +0000, Suravee Suthikulpanit wrote:
> Introduce the set_dte_v1() helper function to configure IOMMU host (v1)
> page table into DTE.
> 
> There is no functional change.
> 
> Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> ---
>  drivers/iommu/amd/iommu.c | 54 +++++++++++++++++++++------------------
>  1 file changed, 29 insertions(+), 25 deletions(-)
> 
> diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
> index ffb1adfd75c0..2a536d02aeab 100644
> --- a/drivers/iommu/amd/iommu.c
> +++ b/drivers/iommu/amd/iommu.c
> @@ -2044,6 +2044,32 @@ static void set_dte_gcr3_table(struct amd_iommu *iommu,
>  		target->data[2] |= FIELD_PREP(DTE_GPT_LEVEL_MASK, GUEST_PGTABLE_4_LEVEL);
>  }
>  
> +static void set_dte_v1(struct iommu_dev_data *dev_data,
> +		       struct protection_domain *domain, u16 domid,
> +		       struct dev_table_entry *new)
> +{
> +	/*
> +	 * When SNP is enabled, we can only support TV=1 with non-zero domain ID.
> +	 * This is prevented by the SNP-enable and IOMMU_DOMAIN_IDENTITY check in
> +	 * do_iommu_domain_alloc().
> +	 */
> +	WARN_ON(amd_iommu_snp_en && (domid == 0));
> +
> +	if (domain->iop.mode != PAGE_MODE_NONE)
> +		new->data[0] |= iommu_virt_to_phys(domain->iop.root);

Use a FIELD_PREP here too

> +	new->data[0] |= FIELD_PREP(DTE_MODE_MASK, domain->iop.mode);
> +	new->data[0] |= DTE_FLAG_IR | DTE_FLAG_IW | DTE_FLAG_TV;
> +
> +	if (domain->dirty_tracking)
> +		new->data[0] |= DTE_FLAG_HAD;
> +
> +	if (dev_data->ats_enabled)
> +		new->data[1] |= DTE_FLAG_IOTLB;
> +
> +	new->data[1] |= domid;

  new->data[1] |= FIELD_PREP(DTE_DOMID_MASK, domid);

> @@ -2061,36 +2087,14 @@ static void set_dte_entry(struct amd_iommu *iommu,
>  
>  	amd_iommu_make_clear_dte(dev_data, &new);
>  
> -	if (domain->iop.mode != PAGE_MODE_NONE)
> -		new.data[0] |= iommu_virt_to_phys(domain->iop.root);
> -
> -	new.data[0] |= (domain->iop.mode & DEV_ENTRY_MODE_MASK)
> -		    << DEV_ENTRY_MODE_SHIFT;
> -
> -	new.data[0] |= DTE_FLAG_IR | DTE_FLAG_IW;
> +	old_domid = READ_ONCE(dte->data[1]) & DTE_DOMID_MASK;
>  
> -	/*
> -	 * When SNP is enabled, we can only support TV=1 with non-zero domain ID.
> -	 * This is prevented by the SNP-enable and IOMMU_DOMAIN_IDENTITY check in
> -	 * do_iommu_domain_alloc().
> -	 */
> -	WARN_ON(amd_iommu_snp_en && (domid == 0));
> -	new.data[0] |= DTE_FLAG_TV;
> +	set_dte_v1(dev_data, domain, domid, &new);
> +	set_dte_gcr3_table(iommu, dev_data, &new);

This seems weird, I would expect this to be written:

if (gcr3_info && gcr3_info->gcr3_tbl)
	set_dte_gcr3_table(iommu, dev_data, &new);
else
	set_dte_v1(dev_data, domain, domid, &new);

It is nonsense to call both gcr3 and v1 in this function that does not
setup two stages.

So, I'd just put this code in both the v1 and gcr3 functions:

 +	new->data[0] |= DTE_FLAG_IR | DTE_FLAG_IW | DTE_FLAG_TV;
 +	if (dev_data->ats_enabled)
 +		new->data[1] |= DTE_FLAG_IOTLB;

(does IR/IW apply to GCR3?)

And then WARN_ON(domain->iop.mode != PAGE_MODE_NONE) as it should be
illegal to call set_dte_v1() on a domain that is not a v1 domain.

But this is overall the right idea and direction.

Jason