Re: [PATCH v4 1/5] PCI/P2PDMA: Don't enforce ACS check for device functions of Intel GPUs

[Matthew Brost <matthew.brost@intel.com>]

On Tue, Sep 23, 2025 at 03:48:59PM +0200, Christian König wrote:
> On 23.09.25 15:38, Jason Gunthorpe wrote:
> > On Tue, Sep 23, 2025 at 03:28:53PM +0200, Christian König wrote:
> >> On 23.09.25 15:12, Jason Gunthorpe wrote:
> >>>> When you want to communicate addresses in a device specific address
> >>>> space you need a device specific type for that and not abuse
> >>>> phys_addr_t.
> >>>
> >>> I'm not talking about abusing phys_addr_t, I'm talking about putting a
> >>> legitimate CPU address in there.
> >>>
> >>> You can argue it is hack in Xe to reverse engineer the VRAM offset
> >>> from a CPU physical, and I would be sympathetic, but it does allow
> >>> VFIO to be general not specialized to Xe.
> >>
> >> No, exactly that doesn't work for all use cases. That's why I'm
> >> pushing back so hard on using phys_addr_t or CPU addresses.
> >>
> >> See the CPU address is only valid temporary because the VF BAR is
> >> only a window into the device memory.
> > 
> > I know, generally yes.
> > 
> > But there should be no way that a VFIO VF driver in the hypervisor
> > knows what is currently mapped to the VF's BAR. The only way I can
> > make sense of what Xe is doing here is if the VF BAR is a static
> > aperture of the VRAM..
> > 
> > Would be nice to know the details.
> 
> Yeah, that's why i asked how VFIO gets the information which parts of the it's BAR should be part of the DMA-buf?
> 

Vivek can confirm for sure, but I believe the VF knows the size of its
VRAM space and simply wants to pass along the offset and allocation
order within that space. The PF knows where the VF's VRAM is located in
the BAR, and the combination of the VF base offset and the individual
allocation offset is what gets programmed into the PF page tables.

> That would be really interesting to know.
> 
> Regards,
> Christian.
> 
> >  
> >> What Simona agreed on is exactly what I proposed as well, that you
> >> get a private interface for exactly that use case.

Do you have a link to the conversation with Simona? I'd lean towards a
kernel-wide generic interface if possible.

Regarding phys_addr_t vs. dma_addr_t, I don't have a strong opinion. But
what about using an array of unsigned long with the order encoded
similarly to HMM PFNs? Drivers can interpret the address portion of the
data based on their individual use cases.

Also, to make this complete—do you think we'd need to teach TTM to
understand this new type of dma-buf, like we do for SG list dma-bufs? It
would seem a bit pointless if we just had to convert this new dma-buf
back into an SG list to pass it along to TTM.

The scope of this seems considerably larger than the original series. It
would be good for all stakeholders to reach an agreement so Vivek can
move forward.

Matt

> > 
> > A "private" interface to exchange phys_addr_t between at least
> > VFIO/KVM/iommufd - sure no complaint with that.
> > 
> > Jason
>