Re: [PATCH v3 6/7] vfio: Wait for dma-buf invalidation to complete

["Christian König" <christian.koenig@amd.com>]

On 1/21/26 14:31, Jason Gunthorpe wrote:
> On Wed, Jan 21, 2026 at 10:20:51AM +0100, Christian König wrote:
>> On 1/20/26 15:07, Leon Romanovsky wrote:
>>> From: Leon Romanovsky <leonro@nvidia.com>
>>>
>>> dma-buf invalidation is performed asynchronously by hardware, so VFIO must
>>> wait until all affected objects have been fully invalidated.
>>>
>>> Fixes: 5d74781ebc86 ("vfio/pci: Add dma-buf export support for MMIO regions")
>>> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
>>
>> Reviewed-by: Christian König <christian.koenig@amd.com>
>>
>> Please also keep in mind that the while this wait for all fences for
>> correctness you also need to keep the mapping valid until
>> dma_buf_unmap_attachment() was called.
> 
> Can you elaborate on this more?
> 
> I think what we want for dma_buf_attach_revocable() is the strong
> guarentee that the importer stops doing all access to the memory once
> this sequence is completed and the exporter can rely on it. I don't
> think this works any other way.
> 
> This is already true for dynamic move capable importers, right?

Not quite, no.

> For the non-revocable importers I can see the invalidate sequence is
> more of an advisory thing and you can't know the access is gone until
> the map is undone.
> 
>> In other words you can only redirect the DMA-addresses previously
>> given out into nirvana (or a dummy memory or similar), but you still
>> need to avoid re-using them for something else.
> 
> Does any driver do this? If you unload/reload a GPU driver it is
> going to re-use the addresses handed out?

I never fully read through all the source code, but if I'm not completely mistaken that is enforced for all GPU drivers through the DMA-buf and DRM layer lifetime handling and I think even in other in kernel frameworks like V4L, alsa etc...

What roughly happens is that each DMA-buf mapping through a couple of hoops keeps a reference on the device, so even after a hotplug event the device can only fully go away after all housekeeping structures are destroyed and buffers freed.

Background is that a lot of device still make reads even after you have invalidated a mapping, but then discard the result.

So when you don't have same grace period you end up with PCI AER, warnings from IOMMU, random accesses to PCI BARs which just happen to be in the old location of something etc...

I would rather like to keep that semantics even for forcefully shootdowns since it proved to be rather reliable.

Regards,
Christian.

> 
> Jason