Re: [RFC PATCH 3/3] iommufd/tsm: Add tsm_bind/unbind iommufd ioctls

[Xu Yilun <yilun.xu@linux.intel.com>]

On Wed, Jun 04, 2025 at 09:36:37AM -0300, Jason Gunthorpe wrote:
> On Wed, Jun 04, 2025 at 01:58:55PM +0800, Xu Yilun wrote:
> 
> > But the p2p case may impact AMD, AMD have legacy IOMMUPT on its secure
> > DMA path. And if you invalidate MMIO (in turn unmaps IOMMUPT) when
> > bound, may trigger HW protection mechanism against DMA silent drop.
> 
> As I understand AMD it sort of has a single translation and relies on
> its RMP for security. So I think the MMIO remains mapped always in
> the iommufd IOAS on AMD?

Depends on how IOMMUFD/IOMMU driver reacts to VFIO's MMIO invalidation
when bound. Legacy IOMMUPT is not controlled by firmware, but it's part
of AMD's secure DMA path, so we have to do something on VMM side.

In legacy case I assume IOMMUFD should unmap MMIO range in response to
MMIO invalidation (via DMABUF move notify), is it? Will do something
different when bound?

Anyway, seems we all agreed VFIO should ensure device unbound first,
or no MMIO invalidation. This blocks the issue from happening.

> 
> > SEV-TIO Firmware Interface SPEC, Section 2.11
> > 
> > "If a bound TDI sends a request to the root complex, and the IOMMU detects a fault caused by host
> > configuration, the root complex fences the ASID from all further I/O to or from that guest. A host
> > fault is either a host page table fault or an RMP check violation. ASID fencing means that the
> > IOMMU blocks all further I/O from the root complex to the guest that the TDI was bound, and the
> > root complex blocks all MMIO accesses by the guest. When a guest writes to MMIO, the write is
> > silently dropped. When a guest reads from MMIO, the guest reads 1s."
> 
> Sounds to me like the guest has to do things properly or the guest
> gets itself killed. I wonder how feasible this really is..

I think both guest & host. If host unmaps some legacy IOMMUPT entry
delibrately or accidently, the issue happens.

> 
> > BTW: What is ARM's secure DMA path, does it goes through independent
> > Secure IOPT? So for p2p when VFIO invalidates MMIO, how the Secure IOPT
> > react? How to avoid DMA slient drop?
> 
> On ARM T=1/0 traffic goes to two different iommu instances.
> 
> As I understand it the T=1 traffic will go through an TSM controlled
> IOMMU that uses the ARM equivalent of the S-EPT for translation. Ie
> the CPU and IOMMU translation are enforced to be identical.

We are on the same boat... I need to check how ARM operates on this
S-EPT equivalent, also in KVM?

Based on this I doubt ARM also has the immediate DMA silent drop issue.

Anyway, unbind the device first.

> 
> T=0 traffic will go through an iommufd controlled iommu and it will
> use the IOAS for translation.
> 
> I've also understood this is quite similar to Intel.
> 
> (IMHO this design is a mistake, but oh well)
> 
> Jason