Re: [PATCH] iommu/amd: Add Secure ATS support

[Jason Gunthorpe <jgg@ziepe.ca>]

On Tue, Feb 25, 2025 at 01:18:43PM +0000, Robin Murphy wrote:

> > My thought is to add a flag in iommufd_hw_capabilities to report the
> > capability, and add a flag in iommufd_hwpt_alloc_flags to let iommu driver
> > know if HPT is needed when allocating S2 domain. Userspace attaches the
> > un-trusted devices to the domains with HPT. While trusted devices can be
> > attached to 'normal' S2 domains.  Will to send it out soon.:)
> > 
> > > With Secure ATS, for ATS requests IOMMU return GPA to device instead
> > > of SPA.
> > 
> > I suppose AMD Secure ATS only works under nested translation configuration.
> > is it? I guess admin may want some flexibility to opt-in it. :)
> 
> Heh, and in SMMUv3 we have both options, with split-stage ATS for nested
> translation, and our Device Permission Table for full ATS, so it seems like
> there should be room for some kind of generalised capability.

Yes, lets not have another driver-only command line parameter but
somthing global managed by the core code.

What are the options here?
 1) Translated Address (TA) is a full physical address and IOMMU does
    do anything (today)

 2) IOMMU converts a S1 IOVA into a TA as a S2 IOVA address and the
    IOMMU runs it through the S2 to validate it. (ARM calls this
    split-stage). Requires nesting

 3) TA is an IOVA and the IOMMU runs it through the full translation
    to validate it. ATS is just used to signal non-present

 4) TA is an full physical address and the IOMMU validates the full
    physical using some kind of permission structure (ARM calls this
    Device Permission Table)

What are the three HWs doing? 

I see #2 and #4 clearly in the SMM spec. Is #3 a special case of #2 (a
STE with a S2 and S1DSS bypass)?

I think AMD is doing #3 from the docs.

What is Intel doing?

A domain alloc flag to put the domain into #3 mode seems like a good
start to me. The core code can decide to activate the flag for the
default domain. Suggest starting from pci->untrusted.

Also, #3 requires PCI topology support, the ACS flags need to be set
to route all TA's to the host. The core code should check and validate
this before turning it on.

I think #2 can be requested through the vSTE on ARM?

Not sure how to setup #4? The core code also needs to detect coherent
systems (ie CXL/etc) and refuse to do anything other than #1/#4..

Jason