Re: [RFC PATCH 10/30] vfio/pci: Export vfio dma-buf specific info for importers

[Jason Gunthorpe <jgg@nvidia.com>]

On Mon, Jul 13, 2026 at 08:08:14PM +0100, Fuad Tabba wrote:
> On Sun, 12 Jul 2026 at 02:02, Ackerley Tng <ackerleytng@google.com> wrote:
> >
> > Jason Gunthorpe <jgg@nvidia.com> writes:
> >
> > > On Thu, May 29, 2025 at 01:34:53PM +0800, Xu Yilun wrote:
> > >> Export vfio dma-buf specific info by attaching vfio_dma_buf_data in
> > >> struct dma_buf::priv. Provide a helper vfio_dma_buf_get_data() for
> > >> importers to fetch these data. Exporters identify VFIO dma-buf by
> > >> successfully getting these data.
> > >>
> > >> VFIO dma-buf supports disabling host access to these exported MMIO
> > >> regions when the device is converted to private. Exporters like KVM
> > >> need to identify this type of dma-buf to decide if it is good to use.
> > >> KVM only allows host unaccessible MMIO regions been mapped in private
> > >> roots.
> > >>
> > >> Export struct kvm * handler attached to the vfio device. This
> > >> allows KVM to do another sanity check. MMIO should only be assigned to
> > >> a CoCo VM if its owner device is already assigned to the same VM.
> > >
> > > This doesn't seem right, it should be encapsulated into the standard
> > > DMABUF API in some way.
> > >
> >
> > I'd like to propose an alternative. I've been working on guest_memfd and
> > new to the world of IO, please help me along! :)
> >
> > It seems like using dmabufs are used a little awkwardly here. IIUC
> > dmabufs were originally meant to expose memory of one device to another
> > device, mostly meant to share memory. Dmabufs do expose MMIO too, for
> > device to device communications. Without virtualization, userspace MMIO
> > would be done by mmap()-ing a VFIO fd and having the userspace program
> > write to the userspace addresses.
> >
> > Before CoCo, device passthrough (MMIO) is mostly handled by mmap()-ing a
> > VFIO fd and setting up the userspace address in a KVM memslot for the
> > guest.
> >
> > With CoCo, is the problem we're solving that we want KVM to know what
> > pfns to set up in stage 2 page tables, but not via userspace addresses?
> >
> > guest_memfd already does that for regular host memory, tracks the
> > private/shared-ness of the memory, tracks which struct kvm the memory
> > belongs to.
> >
> > guest_memfd functions as KVM's bridge to host memory. KVM already can
> > ask guest_memfd for the pfn to map into stage 2 page tables, and already
> > asks guest_memfd for the shared/private state of the memory. guest_memfd
> > already also blocks the host from faulting guest private memory
> > (mmap()-ing is always allowed).
> >
> >
> > Instead of using dmabuf as the intermediary between the MMIO PFNs and
> > KVM, why not use guest_memfd?
> >
> > What if we make guest_memfd accept a VFIO fd, or a dmabuf fd?
> 
> This is interesting for pKVM too, provided it covers more than MMIO.
> 
> We need guest_memfd to be backable by a dmabuf for ordinary guest memory, not
> only for device MMIO. There is mobile hardware that doesn't tolerate scattered
> private memory (DMA engines that can't gather, IOMMU page-table size
> constraints), and a CMA-backed dmabuf heap is the practical way to get
> contiguous memory at runtime. 

Why can't guestmemfd allocate directly from CMA? Allocating struct
page memory through dmabuf just to put it back in a guestmemfd sounds
very ugly to me.

> HugeTLB doesn't help, it wants boot-time
> reservation. Those pages are struct-page backed, so it's a different problem
> from the non-struct-page MMIO case, and the shared parts still need to be
> GUP-able.

Isn't dmabuf pretty allergic to mmaping refcounted struct page backed
memory since that wrecks its lifetime model?

> More important for the API shape: conversions have to work on subsets of such a
> region, at page granularity. A pKVM guest doesn't know what backs its memory, so
> it will issue share/unshare hypercalls over arbitrary ranges of whatever it was
> given. If a dmabuf-backed guest_memfd can only be converted as a whole, we can't
> use it for memory, and the guest can't be taught to care.

More reasons not to involve DMABUF since guestmemfd already does all
of this...

Jason