Re: [PATCH 4/9] vfio/pci: Convert BAR mmap() to use a DMABUF

[Matt Evans <mattev@meta.com>]

Hi Leon, Alex, Jason,

On 06/05/2026 06:35, Leon Romanovsky wrote:
> 
> On Tue, May 05, 2026 at 08:50:58AM -0600, Alex Williamson wrote:
>> On Tue, 5 May 2026 13:49:11 +0300
>> Leon Romanovsky <leon@kernel.org> wrote:
>>
>>> On Mon, May 04, 2026 at 04:40:41AM -0300, Jason Gunthorpe wrote:
>>>> On Fri, May 01, 2026 at 04:19:15PM -0600, Alex Williamson wrote:
>>>>    
>>>>> Exporting dma-bufs from vfio-pci is a feature, but mmap of MMIO BARs is
>>>>> a legacy requirement.  That legacy requirement now depends on
>>>>> PCI_P2PDMA, which depends on 64BIT and ZONE_DEVICE.
>>>>
>>>> That should be split up now, Leon missed it when he added the new
>>>> APIs that didn't require ZONE_DEVICE..
>>>
>>> Sorry, what did I miss here?
>>> VFIO_DMABUF is an optional feature and is enabled only when P2P support is
>>> available. It does not affect legacy systems where P2P cannot be enabled.
>>
>> If we look at the long term view of moving exclusively to cdev/iommufd,
>> where VFIO_DMABUF becomes the mechanism for implementing P2P DMA
>> mappings, VFIO_DMABUF may be optional, but it's highly desirable for
>> legacy compatibility.  There's an argument though that providing P2P
>> compatibility on platforms that support PCI_P2PDMA is probably
>> sufficient.
>>
>> However, in providing mmap of dmabufs as a feature, this series is
>> wiring all mmaps through dmabufs and therefore that dependency becomes
>> fundamental to the use of vfio-pci.  Thus the discussion whether the
>> noted config requirements could be lifted.  Thanks,
> 
> Right, there was no need to remove ZONE_DEVICE when I added my code, and I
> left the task of cleaning it out of is_pci_p2pdma_page() for another day.
> Without ZONE_DEVICE, all pages are treated as non‑P2P.
Just checking I'm following the deps correctly...

If we remove the PCI_P2PDMA dependency on ZONE_DEVICE then (without) 
it's present with P2P hard-disabled (as Leon says).  Already 
`is_pci_p2pdma_page() == false` because folio_is_zone_device() is always 
false in that config.

The only dependency this series adds is for pcim_p2pdma_provider() (and 
support).  We can select PCI_P2PDMA for provider purposes (VFIO!) but if 
!ZONE_DEVICE there's no P2P.

Nothing in PCI_P2PDMA depends on 64BIT and that dependency can be 
removed [1].  (ZONE_DEVICE depends on 64BIT.)

That seems a path to VFIO operating unchanged on 32-bit platforms and 
others without ZONE_DEVICE.

(PCI_P2PDMA currently also depends on NEED_SG_DMA_FLAGS, which seems to 
be unnecessary if !ZONE_DEVICE is preventing P2P pages (no 
sg_dma_mark_bus_address() would happen for example), is that right?)


Then VFIO_PCI depends on a subset of PCI_P2PDMA configuation:

"Diet" PCI_P2PDMA_CORE available even when !ZONE_DEVICE:
  - Compatible with 32-bit platforms; doesn't need to depend on 64BIT
  - Returns a provider, which allows VFIO mmap(), DMABUF mmap().
  - Doesn't allow P2P DMA (eg. via DMABUFs).
  - Doesn't need to select NEED_SG_DMA_FLAGS.

"Classic" PCI_P2PDMA depending on ZONE_DEVICE:
  - Behaves as PCI_P2PDMA does today
  - If ZONE_DEVICE then PCI_P2PDMA enabled, which enables VFIO_PCI_DMABUF

Meaning:

32-bit configs (or 64-bit without ZONE_DEVICE):
  - No P2P anyway
  - VFIO mmap() works
  - VFIO_PCI_DMABUF is disabled (no point enabling DMABUF export since it
    can't be used for its original P2P purpose)

64-bit configs that support ZONE_DEVICE:
  - As today, DMABUF P2P, mmap() of BARs via VFIO or DMABUF


Just added PCI_P2PDMA_CORE and (functionally) ~no code changes are 
needed since disabling P2P just falls out of !ZONE_DEVICE.  But most of 
p2pdma.c does nothing in that config (except for pcim_p2pdma_provider() 
and its support) so benefits from some #ifdef to reduce it.  WDYT about 
a pcip2pdma_core.c containing just the provider management?


Thanks,

Matt

[1]:  The drivers/pci/Kconfig's PCI_P2PDMA states:
   # The need for the scatterlist DMA bus address flag means PCI P2PDMA
   # requires 64bit
I think this isn't quite right as AFAICT the flags are usable on 32b 
too, and actually it's ZONE_DEVICE that requires 64bit.