Re: [PATCH for-next] RDMA/efa: Expose device P2P DMA support via device query

[Jason Gunthorpe <jgg@nvidia.com>]

On Tue, May 05, 2026 at 08:15:14AM +0000, Yonatan Nachum wrote:
> On Mon, May 04, 2026 at 04:34:12AM -0300, Jason Gunthorpe wrote:
> > On Sun, May 03, 2026 at 03:02:46PM +0000, Yonatan Nachum wrote:
> > > Expose device P2P DMA support using the query device verbs.
> > > If the device support P2P DMA, it can DMA directly to and from a peer
> > > PCIe device
> > 
> > This doesn't seem right, this should be policed by failing to
> > established p2p mappings and to fail mapping dmabufs not with random
> > user space bits like this.
> 
> The motivation here is to avoid requiring userspace to speculatively
> attempt registering accelerator MRs just to discover whether the device
> supports P2P DMA.

This isn't how the kernel works there isn't a "this device can do
p2p", the question is always answered with pairs of devices. There is
no way for a single device to know it doesn't do p2p with any other
device in the system.

> Beyond the API awkwardness, this also has a real
> performance impact — initializing an accelerator context can take
> seconds, and by advertising this as a device capability, userspace can
> know upfront whether it's worth going down that path.

This seems like a misconfiguration. Since so much of this topologcal
information is not discoverable at run time usually systems have an
external data file explaining how to best use the system. I wouldn't
expect the runtime to be trying to probe this at run time.

If you *really* want probing then it has to be dmabuf based because
you must probe with pairs of PCI devices.

> 2. When we get a dmabuf, I can't distinguish what is the backing memory,
> for example DRAM or accelerator memory and we can't just blindly fail
> all cases.

You must rely on the p2p subsystem to make this determination and the
dmabuf exporter has to call into it. This will deal with those
problems.

Jason