Re: [PATCH v4 1/5] PCI/P2PDMA: Don't enforce ACS check for device functions of Intel GPUs

["Christian König" <christian.koenig@amd.com>]

On 25.09.25 12:51, Thomas Hellström wrote:
>>> In that case I strongly suggest to add a private DMA-buf interface
>>> for the DMA-
>>> bufs exported by vfio-pci which returns which BAR and offset the
>>> DMA-buf
>>> represents.
> 
> @Christian, Is what you're referring to here the "dma_buf private
> interconnect" we've been discussing previously, now only between vfio-
> pci and any interested importers instead of private to a known exporter
> and importer?
> 
> If so I have a POC I can post as an RFC on a way to negotiate such an
> interconnect.

I was just about to write something up as well, but feel free to go ahead if you already have something.

>> Does this private dmabuf interface already exist or does it need to
>> be created
>> from the ground up?

Every driver which supports both exporting and importing DMA-buf has code to detect when somebody tries to re-import a buffer previously exported from the same device.

Now some drivers like amdgpu and I think XE as well also detect if the buffer is from another device handled by the same driver which potentially have private interconnects (XGMI or similar).

See function amdgpu_dmabuf_is_xgmi_accessible() in amdgpu_dma_buf.c for an example.

>> If it already exists, could you please share an example/reference of
>> how you
>> have used it with amdgpu or other drivers?

Well what's new is that we need to do this between two drivers unreleated to each other.

As far as I know previously that was all inside AMD drivers for example, while in this case vfio is a common vendor agnostic driver.

So we should probably make sure to get that right and vendor agnostic etc....

>> If it doesn't exist, I was wondering if it should be based on any
>> particular best
>> practices/ideas (or design patterns) that already exist in other
>> drivers?
> 
> @Vivek, another question: Also on the guest side we're exporting dma-
> mapped adresses that are imported and somehow decoded by the guest
> virtio-gpu driver? Is something similar needed there?
> 
> Also how would the guest side VF driver know that what is assumed to be
> a PF on the same device is actually a PF on the same device and not a
> completely different device with another driver? (In which case I
> assume it would like to export a system dma-buf)?

Another question is how is lifetime handled? E.g. does the guest know that a DMA-buf exists for it's BAR area?

Regards,
Christian.

> 
> Thanks,
> Thomas
> 
> 
> 
>>
>>>
>>> Ideally using the same structure Qemu used to provide the offset to
>>> the vfio-
>>> pci driver, but not a must have.
>>>
>>> This way the driver for the GPU PF (XE) can leverage this
>>> interface, validates
>>> that the DMA-buf comes from a VF it feels responsible for and do
>>> the math to
>>> figure out in which parts of the VRAM needs to be accessed to
>>> scanout the
>>> picture.
>> Sounds good. This is definitely a viable path forward and it looks
>> like we are all
>> in agreement with this idea.
>>
>> I guess we can start exploring how to implement the private dmabuf
>> interface
>> mechanism right away.
>>
>> Thanks,
>> Vivek
>>
>>>
>>> This way this private vfio-pci interface can also be used by
>>> iommufd for
>>> example.
>>>
>>> Regards,
>>> Christian.
>>>
>>>>
>>>> Thanks,
>>>> Vivek
>>>>
>>>>>
>>>>> Regards,
>>>>> Christian.
>>>>>
>>>>>>
>>>>>>> What Simona agreed on is exactly what I proposed as well,
>>>>>>> that you
>>>>>>> get a private interface for exactly that use case.
>>>>>>
>>>>>> A "private" interface to exchange phys_addr_t between at
>>>>>> least
>>>>>> VFIO/KVM/iommufd - sure no complaint with that.
>>>>>>
>>>>>> Jason
>>>>
>>
>