Re: P2PDMA support status for the sappire rapids+

[Bjorn Helgaas <helgaas@kernel.org>]

[+cc Logan, Michael]

On Tue, Apr 08, 2025 at 05:48:16PM +0300, Damir Chanyshev wrote:
> Hello experts!
> First thank you so much for everything.
> 
> I have a question regarding p2pdma support, I am not an expert in the
> kernel subsystems, please pardon my stupidity.
> While investigating performance opportunities, I stumbled with dma
> between cpu root complex and pcie switch root complex. And found white
> list for the Intel platforms [1]
> 
> Typical topology looks like this rdma nic<>cpu<>pcie switch<>gpu/xpu,
> for each socket.
> Questions:
> List not updated because new hardware doesn't support this feature?
> (For example abandoned for the CXL3+ )
> Or just not tested for the new platforms?
> 
> [1]
> https://lore.kernel.org/all/20220209162801.7647-1-michael.j.ruhl@intel.com/T/#m3f4e4194770274c2873a130ad684a74469038585

The pci_p2pdma_whitelist[] is only updated when somebody tests a
platform and determines that its RC can route peer-to-peer
transactions between separate Root Ports.  This routing is not
required by the PCIe spec, so we can't assume that all platforms
support it.

From PCIe r6.0, sec 1.3.1:

  The capability to route peer-to-peer transactions between hierarchy
  domains through an RC is optional and implementation dependent.

Bjorn