Re: [PATCH v2 4/4] vfio/pci: Allow MMIO regions to be exported through dma-buf

[Jason Gunthorpe <jgg@nvidia.com>]

On Wed, Sep 07, 2022 at 05:05:57AM -0700, Christoph Hellwig wrote:
> On Tue, Sep 06, 2022 at 08:48:28AM -0300, Jason Gunthorpe wrote:
> > Right, this whole thing is the "standard" that dmabuf has adopted
> > instead of the struct pages. Once the AMD GPU driver started doing
> > this some time ago other drivers followed.
> 
> But it is simple wrong.  The scatterlist requires struct page backing.
> In theory a physical address would be enough, but when Dan Williams
> sent patches for that Linus shot them down.

Yes, you said that, and I said that when the AMD driver first merged
it - but it went in anyhow and now people are using it in a bunch of
places.

I'm happy that Christian wants to start trying to fix it, and will
help him, but it doesn't really impact this. Whatever fix is cooked up
will apply equally to vfio and habana.

> That being said the scatterlist is the wrong interface here (and
> probably for most of it's uses).  We really want a lot-level struct
> with just the dma_address and length for the DMA side, and leave it
> separate from that what is used to generate it (in most cases that
> would be a bio_vec).

Oh definitely

> > Now we have struct pages, almost, but I'm not sure if their limits are
> > compatible with VFIO? This has to work for small bars as well.
> 
> Why would small BARs be problematic for the pages?  The pages are more
> a problem for gigantic BARs do the memory overhead.

How do I get a struct page * for a 4k BAR in vfio?

The docs say:

 ..hotplug api on memory block boundaries. The implementation relies on
 this lack of user-api constraint to allow sub-section sized memory
 ranges to be specified to :c:func:`arch_add_memory`, the top-half of
 memory hotplug. Sub-section support allows for 2MB as the cross-arch
 common alignment granularity for :c:func:`devm_memremap_pages`.

Jason