Re: [PATCH v5 0/4] vfio/dma-buf: add TPH support for peer-to-peer access

["Christian König" <christian.koenig@amd.com>]

On 6/1/26 19:47, Jason Gunthorpe wrote:
> On Mon, Jun 01, 2026 at 11:59:55AM +0200, Christian König wrote:
>>>> When you have a complete open source driver stack which utilizes
>>>> VFIO passthrough as the interface to communicate with the kernel
>>>> drivers then we can eventually talk about that.
>>>
>>> That decision is not up to dmabuf
>>
>> Yes it is. This is the DMA-buf API which is added here.
> 
> It is a DMA-buf kernel API that is added, I think it is overreaching
> to try to veto a VFIO uAPI that calls it..

Well as long as that is a private interface between VFIO and mlx5 I have no objection at all.

But when it starts to affect DMA-buf I need to make sure that it works for everybody. And without even being able to test it that becomes really tricky.

>>> - what VFIO subsystem requires to
>>> accept a new UAPI is up to the VFIO maintainers.
>>
>> As far as I can see vfio-pci is used as a stub driver to avoid
>> opening up the real driver.
> 
> Yeah, that's probably true.
> 
> Frankly, I'd rather they use VFIO like this than to upstream another
> driver for proprietary custom built HW which nobody except Meta even
> has, let alone can use.

Yeah, that's a good point.

>> Upstreaming an API changes only makes sense if others can use it and
>> this here is something completely special to this particular use
>> case, without all components involved nobody else can use it.
> 
> It is not 'nobody else can use it', if it was true VFIO wouldn't be
> leaning toward the uAPI being OK.
> 
> This exposes a PCI SIG defined TPH capability in a reasonable simple
> VFIO uAPI that can be re-used by any other device that happens to
> support TPH on inbound MMIO. The uAPI has sensible general semantics
> based around the PCI spec.

That it's implementing an official PCI spec is a good argument.

But on the other hand looking at the spec it's not really specifying much since everything is architecture specific.

> Anyone can repeat the demonstration Meta outlined in their cover
> letter: Use this new VFIO uAPI, import the DMABUF to mlx5, use a PCI
> analyzer and you will see the PCI SIG defined TPH bits set the way the
> VFIO uAPI says they should be set.
> 
> There is nothing uniquely tied to Meta's device here, or unusable by
> someone else's devices. Arguably this is actually a mlx5 feature to
> allow VFIO to control its TPH generation HW.

Would it be possible to demonstrate the functionality with some FPGA implementing an PCIe endpoint?

Doesn't needs to be anything funky, just the ability to exercise this for basically everybody who can spend a few $ on the HW.

Regards,
Christian.

> For a long time the general kernel has had a philosophy that as long
> as these niche features are generally implemented and *in theory*
> usable by anyone who wants it is OK. Every knows the initial userspace
> implementations of *alot* of stuff starts locked up in proprietary
> software owned by database, and now cloud, companies. Yes, some
> subystems are stricter, that's OK too.
> 
>> So as far as I can see that here is a no-go. But at the end of the
>> day it's Linus who needs to say if that's ok or not, that's why I
>> put him on CC.
> 
> Well, based on what I heard when I argued for fwctl, and the
> discussion with sched_ext, I don't think implementing functionality a
> standards body defined in a logical way is going to raise concerns..
> 
> Jason