Re: [PATCH v13 00/19] PCI/P2PDMA: Route peer-to-peer DMA by TLP class

[Leon Romanovsky <leon@kernel.org>]

On Wed, Nov 11, 2026 at 04:41:36PM -0600, Bjorn Helgaas wrote:
> On Wed, Nov 11, 2026 at 02:16:02PM +0200, Leon Romanovsky wrote:
> > On Thu, Nov 05, 2026 at 12:04:31PM +0200, Leon Romanovsky wrote:
> > > Punch line for Linus from Jason:
> > > "Enhance detecting if the PCI fabric can support a peer to peer path
> > > by evaluating the full bi-directional fabric path over each of the
> > > three traffic type ACS controls. Expand the number of fabric
> > > configurations where the P2P algorithm makes the correct determination
> > > of support to include non-uniform ACS and situations where translated
> > > takes different routes from untranslated."
> > 
> > Bjorn,
> > 
> > Just a friendly reminder.
> 
> Thanks for the reminder.  I put this on pci/p2pdma for build testing
> (no issues expected; I always try to get build testing before merging
> into linux-next).
> 
> I'm still struggling to describe this in a way that connects to the
> code changes and is accessible to non-experts, so please correct
> things I get wrong here.

When asked to explain what I’m working on during hallway conversations, I usually
start with a top-down view:

“In order to support P2P in VMs, the virtual PCI topology needs to know whether
P2P is possible. However, the guest OS cannot see the physical topology because
it is hidden by the hypervisor. Since the VM and hypervisor run the same OS, we
need a generic way to report whether P2P is supported, rather than relying on a
CPU-supported whitelist. This is why we need ACPI HMAT [1].

However, HMAT alone is not enough: it is static and comes from firmware, the
BIOS, or the platform. We also need to account for ACS registers and different
traffic types.

This series provides the ACS, traffic piece of the puzzle.”

[1] https://lore.kernel.org/linux-pci/20260812-hmat-p2p-v1-0-75ac41380585@nvidia.com/

> 
> I had the impression that this series helps VMs by making P2PDMA work
> with passthrough devices in cases where it currently doesn't, but
> maybe that's not the case; I don't see any mention of that in the
> series.

It is a fundamental building block (see my explanation above), but it is also
useful on its own. Even without VMs or HMAT, ACS and routing are important
(see the removed patch [2]): they allow us to dynamically enable or disable
ATS for each DMA mapping, support asymmetric PCI topologies, and more broadly
enable P2P by checking every PCI route.

This is why I didn't mention VM to the cover letter.

[2] https://lore.kernel.org/all/20260920-fix-p2p-acs-v4-0-v7-19-ca0828ab697c@nvidia.com/

> 
> IIUC the P2PDMA core doesn't actually *control* any TLP routing, i.e.,
> it doesn't change any ACS controls or change any TLP attributes that
> affect how TLPs are routed, so to me, these subject lines suggest more
> than is actually happening:
> 
>   Route Relaxed Ordering Completions directly
>   Reject Translated Requests blocked by Translation Blocking
>   Route Translated Requests under Direct Translated P2P
> 
> It seems like we're just *learning* whether RO Completions will be
> routed directly, whether Translated Requests will be rejected, etc?

Right, the path and these bits are static.

> 
> I think the core basically looks at the ACS controls on the path
> between two devices and decides whether:
> 
>   (a) the devices can do P2P DMA directly (PCI_P2PDMA_MAP_BUS_ADDR),
> 
>   (b) they can do P2P DMA but only through host bridges on the
>   allowlist (PCI_P2PDMA_MAP_THRU_HOST_BRIDGE),
> 
>   (c) no P2P DMA is possible (PCI_P2PDMA_MAP_NOT_SUPPORTED),

This is equivalent to returning -EOPNOTSUPP when the functionality is
unavailable.

> 
>   (d) PCI_P2PDMA_MAP_NONE -- I don't understand this one; the comment
>   says "not a P2PDMA transfer", and pci_p2pdma_state() returns it when
>   the page is not a p2pdma_page, I guess because
>   pci_p2pdma_add_resource() has not set MEMORY_DEVICE_PCI_P2PDMA?
>   Not sure how this is used.

This means that this is regular DMA transfer which involves CPU.

> 
> I guess one benefit of this series is that a path where P2P Completion
> Redirect is set used to mean that pci_p2pdma_map_type() would return
> PCI_P2PDMA_MAP_NOT_SUPPORTED (unless the RC was allow-listed), and now
> it might return PCI_P2PDMA_MAP_BUS_ADDR because if P2P Request
> Redirect is clear, Requests can be routed directly?

I'm not sure the first part (return PCI_P2PDMA_MAP_NOT_SUPPORTED) is correct,
since callers have so far ensured that P2P is supported before calling this
function. The second part is correct.

> 
> That sounds great as long as the Request uses Relaxed Ordering, so the
> Completion won't be affected by P2P Completion Redirect.  But devices
> aren't *required* to use RO even if enabled, and I'm doubtful that a
> non-RO Completion redirected to the RC is going to work.  So I don't
> understand how p2pdma.c can guarantee safety.
> 
> Translated Requests seems similar -- are we assuming that if ACS
> Direct Translated P2P is set and ATS Enable is set, the device
> *always* generates Translated Requests?

PCI spec allows "A Function is permitted to intermix translated
and untranslated requests" and p2p doesn't decide anything except
if routing is possible or not.

Thanks

> 
> Bjorn