Re: [RFC PATCH 3/3] iommufd/tsm: Add tsm_bind/unbind iommufd ioctls

[Jason Gunthorpe <jgg@nvidia.com>]

On Fri, Jul 18, 2025 at 05:15:42PM +0800, Xu Yilun wrote:
> > I would say this discussion is irrelevant to this case since we are
> > not doing any kind of in-place conversion.
> > 
> > 1) At VM start the VMM gets the shared DMABUF and maps it to IOAS and
> >    EPT
> > 2) Bind request comes in, VMM unmaps shared DMABUF from IOAS and EPT
> >    then closes it.
> > 3) Bind is done
> > 4) VMM opens a shared and private DMABUF FD and learns the valid
> >    ranges in both DMABUFs (ie what PFNS are private/shared)
> 
> Who decides the dmabuf be shared or private? I assume you mean
> userspace, VFIO doesn't know about the shareability of each dmabuf?

I imagined userspace would put a flag in the ioctl to VFIO to get the
dmabuf kind it wants. VFIO would have to be told from iommufd what the
per-pfn shared/private layout is and can use that to manage the
dmabufs.

> > 5) VMM maps the shared DMABUF fragments to the EPT and IOAS
> 
> Userspace could set a shared only memory slot.
> 
> > 6) VMM maps the private DMABUF fragments to the S-EPT
> 
> Userspace could set a private only memory slot, but maybe the concern
> is KVM can't trust userspace about the assertion of private. 

I imagined kvm can query the dmabuf and learn it is all private from
VFIO. The whole dmabuf, not per-pfn. Same for #5 it can check it is
shared memory too.

I sort of imagine using the same mechanism for p2p where we can mark
memory in the dmabuf with a 'provider' indicating these details. We
will have to see.

> If VFIO should be aware of the shareability, IOMMUFD to VFIO callback is
> needed after TSM bind.

Yes, whever this list of per-PFN shared/private must be shared between
VFIO and the TSM.

> It is simpler on kernel side. FYR, may introduce some complexity in
> userspace, this may result in 3+ dmabufs. There may be shared holes
> in the BAR, and kvm slot can't be gfn sparse. This may not be a big
> problem.

Userspace probably needs to create multiple slots for sparsity

Again this seems much better if userspace handles it and just uses
simple mostly existing KVM interfaces.

Why 3+ dmabufs? Isn't just one shared and one private per BAR?

Jason