Re: [PATCH v3 6/7] vfio: Wait for dma-buf invalidation to complete

["Christian König" <christian.koenig@amd.com>]

On 1/23/26 15:11, Jason Gunthorpe wrote:
> On Thu, Jan 22, 2026 at 07:44:04PM -0400, Jason Gunthorpe wrote:
>> On Thu, Jan 22, 2026 at 12:32:03PM +0100, Christian König wrote:
>>>>> What roughly happens is that each DMA-buf mapping through a couple
>>>>> of hoops keeps a reference on the device, so even after a hotplug
>>>>> event the device can only fully go away after all housekeeping
>>>>> structures are destroyed and buffers freed.
>>>>
>>>> A simple reference on the device means nothing for these kinds of
>>>> questions. It does not stop unloading and reloading a driver.
>>>
>>> Well as far as I know it stops the PCIe address space from being re-used.
>>>
>>> So when you do an "echo 1 > remove" and then an re-scan on the
>>> upstream bridge that works, but you get different addresses for your
>>> MMIO BARs!
>>
>> That's pretty a niche scenario.. Most people don't rescan their PCI
>> bus. If you just do rmmod/insmod then it will be re-used, there is no
>> rescan to move the MMIO around on that case.
> 
> Ah I just remembered there is another important detail here.
> 
> It is illegal to call the DMA API after your driver is unprobed. The
> kernel can oops. So if a driver is allowing remove() to complete
> before all the dma_buf_unmaps have been called it is buggy and risks
> an oops.
> 
> https://lore.kernel.org/lkml/8067f204-1380-4d37-8ffd-007fc6f26738@kernel.org/T/#m0c7dda0fb5981240879c5ca489176987d688844c
> 
> As calling a dma_buf_unmap() -> dma_unma_sg() after remove() returns
> is not allowed..

That is not even in the hands of the driver. The DMA-buf framework itself does a module_get() on the exporter.

So as long as a DMA-buf exists you *can't* rmmod the module which provides the exporting driver (expect of course of force unloading).

Revoking the DMA mappings won't change anything on that, the importer needs to stop using the DMA-buf and drop all their references.

Christian.

> 
> Jason