Re: [RFC PATCH v2 14/22] iommufd: Add TIO calls

[Xu Yilun <yilun.xu@linux.intel.com>]

On Fri, Feb 28, 2025 at 08:37:11PM -0400, Jason Gunthorpe wrote:
> On Thu, Feb 27, 2025 at 11:59:18AM +0800, Xu Yilun wrote:
> > On Wed, Feb 26, 2025 at 09:12:02AM -0400, Jason Gunthorpe wrote:
> > > On Wed, Feb 26, 2025 at 06:49:18PM +0800, Xu Yilun wrote:
> > > 
> > > > E.g. I don't think VFIO driver would expect its MMIO access suddenly
> > > > failed without knowing what happened.
> > > 
> > > What do people expect to happen here anyhow? Do you still intend to
> > > mmap any of the MMIO into the hypervisor? No, right? It is all locked
> > 
> > Not expecting mmap the MMIO, but I switched to another way. VFIO doesn't
> > disallow mmap until bind, and if there is mmap on bind, bind failed.
> > That's my understanding of your comments.
> 
> That seems reasonable
> 
> > Another concern is about dma-buf importer (e.g. KVM) mapping the MMIO.
> > Recall we are working on the VFIO dma-buf solution, on bind/unbind the
> > MMIO accessibility is being changed and importers should be notified to
> > remove their mapping beforehand, and rebuild later if possible.
> > An immediate requirement for Intel TDX is, KVM should remove secure EPT
> > mapping for MMIO before unbind.
> 
> dmabuf can do that..

Yes, dmabuf can do that via notify. dmabuf is implemented in VFIO,
so iommufd/vdevice couldn't operate on dmabuf and send the notify.

> 
> > > > The implementation is basically no difference from:
> > > > 
> > > > +       vdev = container_of(iommufd_get_object(ucmd->ictx, cmd->vdevice_id,
> > > > +                                              IOMMUFD_OBJ_VDEVICE),
> > > > 
> > > > The real concern is the device owner, VFIO, should initiate the bind.
> > > 
> > > There is a big different, the above has correct locking, the other
> > > does not :)
> > 
> > Could you elaborate more on that? Any locking problem if we implement
> > bind/unbind outside iommufd. Thanks in advance.
> 
> You will be unable to access any information iommufd has in the viommu
> and vdevice objects. So you will not be able to pass a viommu ID or
> vBDF to the secure world unless you enter through an iommufd path, and
> use iommufd_get_object() to obtain the required locks.
>  
> I don't know what the API signatures are for all three platforms to
> tell if this is a problem or not.

Seems not a problem for Intel TDX. Basically secure DMA settings for TDX
is just to build the secure IOMMUPT, only need host BDF. Also secure
device setting needs no secure DMA info.

All these settings cannot really take function until guest verifies them
and does TDISP start. Guest verification does not (should not) need host
awareness.

Our solution is, separate the secure DMA setting and secure device setting
in different components, iommufd & vfio.

Guest require bind:
  - ioctl(iommufd, IOMMU_VIOMMU_ALLOC, {.type = IOMMU_VIOMMU_TYPE_KVM_VALID,
					.kvm_fd = kvm_fd,
					.out_viommu_id = &viommu_id});
  - ioctl(iommufd, IOMMU_HWPT_ALLOC, {.flag = IOMMU_HWPT_ALLOC_TRUSTED,
				      .pt_id = viommu_id,
				      .out_hwpt_id = &hwpt_id});
  - ioctl(vfio_fd, VFIO_DEVICE_ATTACH_IOMMUFD_PT, {.pt_id = hwpt_id})
    - do secure DMA setting in Intel iommu driver.

  - ioctl(vfio_fd, VFIO_DEVICE_TSM_BIND, ...)
    - do bind in Intel TSM driver.

Thanks,
Yilun

> 
> Jason