Re: [PATCH for-next] RDMA/efa: Expose device P2P DMA support via device query

[Jason Gunthorpe <jgg@nvidia.com>]

On Wed, Sep 23, 2026 at 04:38:30PM +0000, Michael Margolin wrote:

> I'm reviving this conversation since it is becoming an
> increasing need to reduce the long startup times. The major
> part of this time is related to CUDA device initialization
> needed only to allocate GPU memory, export a dmabuf, and
> test accessibility via MR registration. Since obtaining a
> dmabuf fd requires CUDA initialization, which is exactly
> the cost we are trying to avoid, we are looking for a
> solution that doesn't require dmabuf.

You have to use dmabuf. Go complain to the cuda people their stuff is
too slow if that is the actual problem.

Precompute it and put it in a topology file like everyone else does :\

> I have two options in mind:
> 
> 1. Introduce a new (common or EFA-specific) verb that takes
>    a peer device BDF and queries the p2p subsystem,
>    returning something like:

No way. No BDFs in uapis.

> 2. Expose EFA_DEV_CAP(dev, PCIE_PEER_ACCESS_ENABLED)
>    directly to userspace. As you already stated, this is a
>    bit awkward as a device isn't expected to be aware of
>    system topology, but since EFA lives in the cloud it is
>    partially aware of the platform configuration and in
>    particular the ability to access peer devices over PCIe.

If you already know your VMs are always safe what is even the issue?
What are you probing for?

Lets please not hack around bad userspace with even worse kernel uAPI.

Jason