Re: [PATCH 10/10] vfio/pci: Add dma-buf export support for MMIO regions

[Jason Gunthorpe <jgg@nvidia.com>]

On Tue, Jul 29, 2025 at 08:44:21PM +0100, Robin Murphy wrote:

> In this case with just one single
> contiguous mapping, it is clearly objectively worse to have to bounce in and
> out of the IOMMU layer 3 separate times and store a dma_map_state,

The non-contiguous mappings are comming back, it was in earlier drafts
of this. Regardless, the point is to show how to use the general API
that we would want to bring into the DRM drivers that don't have
contiguity even though VFIO is a bit special.

> Oh yeah, and mapping MMIO with regular memory attributes (IOMMU_CACHE)
> rather than appropriate ones (IOMMU_MMIO), as this will end up doing, isn't
> guaranteed not to end badly either (e.g. if the system interconnect ends up
> merging consecutive write bursts and exceeding the target root port's MPS.)

Yes, I recently noticed this too, it should be fixed..

But so we are all on the same page, alot of the PCI P2P systems are
setup so P2P does not transit through the iommu. It either takes the
ACS path through a switch or it uses ATS and takes a different ACS
path through a switch. It only transits through the iommu in
misconfigured systems or in the rarer case of P2P between root ports.

> And again, if the IOMMU is in bypass (the idea of P2P with vfio-noiommu simply
> isn't worth entertaining) 

Not quite. DMABUF is sort of upside down.

For example if we are exporting a DMABUF from VFIO and importing it to
RDMA then RDMA will call VFIO to make an attachment and the above VFIO
code will perform the DMA map to the RDMA struct device. DMABUF
returns a dma mapped scatterlist back to the RDMA driver.

The above dma_map_phys(rdma_dev,...) can be in bypass because the rdma
device can legitimately be in bypass, or not have a iommu, or
whatever.

> AFAICS you're *depending* on this call being an effective no-op, and thus
> only demonstrating that the dma_map_phys() idea is still entirely
> unnecessary.

It should not be a full no-op, and it should be closer to
dma map resource to avoid the mmio issues.

It should be failing for cases where it is not supported (ie
swiotlb=force), it should still be calling the legacy dma_ops, and it
should be undoing any CC mangling with the address. (also the
pci_p2pdma_bus_addr_map() needs to deal with any CC issues too)

Jason