Re: [PATCH v6 9/9] vfio/pci: Permanently revoke a DMABUF on request

[Alex Mastro <amastro@fb.com>]

Hello!

On Mon, Sep 21, 2026 at 10:22:08AM -0300, Jason Gunthorpe wrote:
> On Mon, Sep 21, 2026 at 02:08:47PM +0100, Matt Evans wrote:
>
> > Not quite; the priv->revoked flag tracks temporary periods of
> > inaccessibility.  An example is VFIO resetting a function; the BAR
> > mappings as seen by the CPU and DMABUFs made from the BARs are all made
> > inaccessible before the reset, and made accessible again after the
> > reset.
>
> From the importer perspective this is a permanent revoke.
...
> From an importer perspective it sees the revoke happen and then that's
> it, the dmabuf never does anything further. The importer has to unmap
> and start from scratch, get a FD and map it.

That may be true for iommufd, but from what I can tell, already-imported
vfio-pci dma-buf can be resurrected after VFIO_DEVICE_RESET by other dynamic
importers (e.g. mlx5). That seems to be the case today, without this series. I
clanked together a toy program and accompanying bpftrace script demonstrating
this [1][2].

The program
1. exports a vfio-pci dma-buf
2. imports the dma-buf with ibv_reg_dmabuf_mr()
3. VFIO_DEVICE_RESET
4. ibv_advise_mr(PREFETCH|FLUSH) on the original MR. It succeeds, triggering
	mlx5_ib_advise_mr_prefetch
	 -> pagefault_dmabuf_mr
	 -> ib_umem_dmabuf_map_pages
	 -> dma_buf_map_attachment
	 -> vfio_pci_dma_buf_map
5. re-importing the original dma-buf fd to create a new MR also succeeds.

So I empathize with Matt's contention that the _existing_ behavior that the
priv->revoked flag represents is actually "temporarily revoked": the importer
can use the same dma-buf again, later, without having to re-import it!

What are we missing?

On Tue, Sep 22, 2026 at 09:54:21AM -0300, Jason Gunthorpe wrote:
> On Tue, Sep 22, 2026 at 03:46:37PM +0300, Leon Romanovsky wrote:
> > On Tue, Sep 22, 2026 at 09:41:37AM -0300, Jason Gunthorpe wrote:
> > > On Mon, Sep 21, 2026 at 04:09:18PM +0200, Christian König wrote:
> > > > >> I would avoid that and just re-create the DMA-buf fd from
> > > > >> scratch. The extra overhead is negligible and one way state
> > > > >> transmissions are usually much easier to handle.
> > > > > 
> > > > > Yeah, maybe we should have done that. Might be too late now.
> > > > 
> > > > It's already uAPI? 
> > > 
> > > Yeah, but Matt is making some changes here so maybe new stuff can
> > > avoid this. I'm not sure.
> > 
> > Jason, the proposed semantics is not UAPI yet.
> 
> What I'm talking about is, the revoke/unrevoke flow for a single FD
> was added from the start. For example vfio_pci_ioctl_reset() does it:
> 
> +       vfio_pci_dma_buf_move(vdev, true);
>         ret = pci_try_reset_function(vdev->pdev);
> +       if (__vfio_pci_memory_enabled(vdev))
> +               vfio_pci_dma_buf_move(vdev, false);
>         up_write(&vdev->memory_lock);
> 
> The false restores the exsting dmabuf fds back to normal operation.

The toy progam sequence above shows that both of the following are the case
today before this series:
- an already-imported dma-buf fd can come back to life after a reset.
- an already-exported dma-buf fd can be re-imported after a reset.

This series doesn't intend to change the behavior of either. Is the confusion
about whether the current behavior is intentional and/or desirable? If the
answer to both is "no", then IMO this series paves the way nicely towards making
PERM_REVOKED the only supported semantic later.

[1] https://github.com/opsound/vfio-tools/blob/9744d0f17edd67ad8eb21134ff75cdc2d0678bbc/vfio_dmabuf_reset.c
[2] https://github.com/opsound/vfio-tools/blob/9744d0f17edd67ad8eb21134ff75cdc2d0678bbc/vfio_dmabuf_reset.bt

Alex