Re: [PATCH 4/4] vfio/pci: Allow MMIO regions to be exported through dma-buf

[Jason Gunthorpe <jgg@nvidia.com>]

On Mon, Aug 29, 2022 at 01:04:05PM +0800, Yan Zhao wrote:

> > +static struct sg_table *
> > +vfio_pci_dma_buf_map(struct dma_buf_attachment *attachment,
> > +		     enum dma_data_direction dir)
> > +{
> > +	size_t sgl_size = dma_get_max_seg_size(attachment->dev);
> > +	struct vfio_pci_dma_buf *priv = attachment->dmabuf->priv;
> > +	struct scatterlist *sgl;
> > +	struct sg_table *sgt;
> > +	dma_addr_t dma_addr;
> > +	unsigned int nents;
> > +	size_t offset;
> > +	int ret;
> > +
> > +	dma_resv_assert_held(priv->dmabuf->resv);
> > +
> > +	if (!attachment->peer2peer)
> > +		return ERR_PTR(-EPERM);
> > +
> > +	if (!priv->revoked)
> > +		return ERR_PTR(-ENODEV);
> > +
> > +	sgt = kzalloc(sizeof(*sgt), GFP_KERNEL);
> > +	if (!sgt)
> > +		return ERR_PTR(-ENOMEM);
> > +
> > +	nents = DIV_ROUND_UP(priv->dmabuf->size, sgl_size);
> > +	ret = sg_alloc_table(sgt, nents, GFP_KERNEL);
> > +	if (ret)
> > +		goto err_kfree_sgt;
> > +
> > +	/*
> > +	 * Since the memory being mapped is a device memory it could never be in
> > +	 * CPU caches.
> > +	 */
> > +	dma_addr = dma_map_resource(
> > +		attachment->dev,
> > +		pci_resource_start(priv->vdev->pdev, priv->index) +
> > +			priv->offset,
> > +		priv->dmabuf->size, dir, DMA_ATTR_SKIP_CPU_SYNC);
> dma_map_resource maps the phys to an IOVA in device's
> default_domain, which, however, may not be the domain that the device is
> currently attached to.

dmabuf is defined only in terms of the DMA API, so it creates a SGL
with DMA API mapped dma_addr's

> So, the importer of this sgt will convert this dma_addr back to phys?

No, it is illegal to call this API if the importer is not using the
DMA API.

I'm expecting to propose some new dmabuf API for this use-case, when
we get to it. For now all importers use the DMA API.

> BTW, I don't see the assignment of priv->index in below
> vfio_pci_core_feature_dma_buf(), is it equal to
> get_dma_buf.region_index ?

Yes, I noticed that too and have fixed it - the testing was all done
on index 0.

Jason