Re: [PATCH v3 6/7] vfio: Wait for dma-buf invalidation to complete

Leon Romanovsky
2026-01-21T07:59:34Z

On Tue, Jan 20, 2026 at 12:44:50PM -0800, Matthew Brost wrote:
> On Tue, Jan 20, 2026 at 04:07:06PM +0200, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro@nvidia.com>
> >
> > dma-buf invalidation is performed asynchronously by hardware, so VFIO must
> > wait until all affected objects have been fully invalidated.
> >
> > Fixes: 5d74781ebc86 ("vfio/pci: Add dma-buf export support for MMIO regions")
> > Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> > ---
> > drivers/vfio/pci/vfio_pci_dmabuf.c | 5 +++++
> > 1 file changed, 5 insertions(+)
> >
> > diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci_dmabuf.c
> > index d4d0f7d08c53..33bc6a1909dd 100644
> > --- a/drivers/vfio/pci/vfio_pci_dmabuf.c
> > +++ b/drivers/vfio/pci/vfio_pci_dmabuf.c
> > @@ -321,6 +321,9 @@ void vfio_pci_dma_buf_move(struct vfio_pci_core_device *vdev, bool revoked)
> > dma_resv_lock(priv->dmabuf->resv, NULL);
> > priv->revoked = revoked;
> > dma_buf_move_notify(priv->dmabuf);
> > + dma_resv_wait_timeout(priv->dmabuf->resv,
> > + DMA_RESV_USAGE_KERNEL, false,
> > + MAX_SCHEDULE_TIMEOUT);
>
> Should we explicitly call out in the dma_buf_move_notify() /
> invalidate_mappings kernel-doc that KERNEL slots are the mechanism
> for communicating asynchronous dma_buf_move_notify /
> invalidate_mappings events via fences?
>
> Yes, this is probably implied, but it wouldn’t hurt to state this
> explicitly as part of the cross-driver contract.
>
> Here is what we have now:
>
> * - Dynamic importers should set fences for any access that they can't
> * disable immediately from their &dma_buf_attach_ops.invalidate_mappings
> * callback.

I believe I documented this in patch 4:
https://lore.kernel.org/all/20260120-dmabuf-revoke-v3-4-b7e0b07b8214@nvidia.com/"
Is there anything else that should be added?

1275 /**
1276 * dma_buf_move_notify - notify attachments that DMA-buf is moving
1277 *
1278 * @dmabuf: [in] buffer which is moving
1279 *
1280 * Informs all attachments that they need to destroy and recreate all their
1281 * mappings. If the attachment is dynamic then the dynamic importer is expected
1282 * to invalidate any caches it has of the mapping result and perform a new
1283 * mapping request before allowing HW to do any further DMA.
1284 *
1285 * If the attachment is pinned then this informs the pinned importer that
1286 * the underlying mapping is no longer available. Pinned importers may take
1287 * this is as a permanent revocation so exporters should not trigger it
1288 * lightly.
1289 *
1290 * For legacy pinned importers that cannot support invalidation this is a NOP.
1291 * Drivers can call dma_buf_attach_revocable() to determine if the importer
1292 * supports this.
1293 *
1294 * NOTE: The invalidation triggers asynchronous HW operation and the callers
1295 * need to wait for this operation to complete by calling
1296 * to dma_resv_wait_timeout().
1297 */

Thanks

>
> Matt
>
> > dma_resv_unlock(priv->dmabuf->resv);
> > }
> > fput(priv->dmabuf->file);
> > @@ -342,6 +345,8 @@ void vfio_pci_dma_buf_cleanup(struct vfio_pci_core_device *vdev)
> > priv->vdev = NULL;
> > priv->revoked = true;
> > dma_buf_move_notify(priv->dmabuf);
> > + dma_resv_wait_timeout(priv->dmabuf->resv, DMA_RESV_USAGE_KERNEL,
> > + false, MAX_SCHEDULE_TIMEOUT);
> > dma_resv_unlock(priv->dmabuf->resv);
> > vfio_device_put_registration(&vdev->vdev);
> > fput(priv->dmabuf->file);
> >
> > --
> > 2.52.0
> >