Re: [RFC PATCH v2 14/22] iommufd: Add TIO calls

[Xu Yilun <yilun.xu@linux.intel.com>]

On Wed, Mar 05, 2025 at 03:28:42PM -0400, Jason Gunthorpe wrote:
> On Mon, Mar 03, 2025 at 01:32:47PM +0800, Xu Yilun wrote:
> > All these settings cannot really take function until guest verifies them
> > and does TDISP start. Guest verification does not (should not) need host
> > awareness.
> > 
> > Our solution is, separate the secure DMA setting and secure device setting
> > in different components, iommufd & vfio.
> > 
> > Guest require bind:
> >   - ioctl(iommufd, IOMMU_VIOMMU_ALLOC, {.type = IOMMU_VIOMMU_TYPE_KVM_VALID,
> > 					.kvm_fd = kvm_fd,
> > 					.out_viommu_id = &viommu_id});
> >   - ioctl(iommufd, IOMMU_HWPT_ALLOC, {.flag = IOMMU_HWPT_ALLOC_TRUSTED,
> > 				      .pt_id = viommu_id,
> > 				      .out_hwpt_id = &hwpt_id});
> >   - ioctl(vfio_fd, VFIO_DEVICE_ATTACH_IOMMUFD_PT, {.pt_id = hwpt_id})
> >     - do secure DMA setting in Intel iommu driver.
> > 
> >   - ioctl(vfio_fd, VFIO_DEVICE_TSM_BIND, ...)
> >     - do bind in Intel TSM driver.
> 
> Except what do command do you issue to the secure world for TSM_BIND
> and what are it's argument? Again you can't include the vBDF or vIOMMU
> ID here.

Bind for TDX doesn't require vBDF or vIOMMU ID. The seamcall is like:

u64 tdh_devif_create(u64 stream_id,     // IDE stream ID, PF0 stuff
                     u64 devif_id,      // TDI ID, it is the host BDF
                     u64 tdr_pa,        // TDX VM core metadate page, TDX Connect uses it as CoCo-VM ID
                     u64 devifcs_pa)    // metadate page provide to firmware

While for AMD:
        ...
        b.guest_device_id = guest_rid;  //TDI ID, it is the vBDF
        b.gctx_paddr = gctx_paddr;      //AMDs CoCo-VM ID

        ret = sev_tio_do_cmd(SEV_CMD_TIO_TDI_BIND, &b, ...


Neither of them use vIOMMU ID or any IOMMU info, so the only concern is
vBDF.

Basically from host POV the two interfaces does the same thing, connect
the CoCo-VM ID with the TDI ID, for which Intel uses host BDF while AMD
uses vBDF. But AMD firmware cannot know anything meaningful about the
vBDF, it is just a magic number to index TDI metadata.

So I don't think we have to introduce vBDF concept in kernel. AMD uses
QEMU created vBDF as TDI ID, that's fine, QEMU should ensure the
validity of the vBDF.

> 
> vfio also can't validate that the hwpt is in the right state when it
> executes this function.

Not sure if VFIO has to validate, or is there a requirement that
secure DMA should be in right state before bind. TDX doesn't require
this, and I didn't see the requirement in SEV-TIO spec. I.e. the
bind firmware calls don't check DMA state.

In my opinion, TDI bind means put device in LOCKED state and related
metadate management in firmware. After bind the DMA cannot work. It
is the guest's resposibility to validate everything (including DMA)
is in the right state, then issues RUN, then DMA works. I.e. guest tsm
calls check DMA state.  That's why I think Secure DMA configuration
on host could be in a separated flow from bind.

> 
> You could also issue the TSM bind against the idev on the iommufd
> side..

But I cannot figure out how idev could ensure no mmap on VFIO, and how
idev could call dma_buf_move_notify.

Thanks,
Yilun

> 
> Part of my problem here is I don't see anyone who seems to have read
> all three specs and is trying to mush them together. Everyone is
> focused on their own spec. I know there are subtle differences :\
> 
> Jason