Re: [PATCH v13 2/5] PCI/TPH: Add requester/completer type helpers

Re: [PATCH v13 2/5] PCI/TPH: Add requester/completer type helpers

[fengchengwen]

Acked-by: Chengwen Feng <fengchengwen@huawei.com>

On 8/1/2026 5:15 AM, Zhiping Zhang wrote:
> Add pcie_tph_enabled_req_type() so drivers can query the enabled TPH
> requester mode without reaching into pci_dev internals.
> 
> Add pcie_tph_completer_type() so drivers that publish TPH metadata for a
> device acting as a completer can gate on the "TPH Completer Supported"
> field of Device Capabilities 2 (bits 13:12, PCI_EXP_DEVCAP2_TPH_COMP_MASK)
> rather than reusing requester-side state. It applies the same reserved-
> encoding fold as get_rp_completer_type(), inlined, so the reserved 0b10
> value never reaches callers.
> 
> This keeps pci_dev::tph_req_type and the completer-capability decode
> inside the PCI/TPH code and provides !CONFIG_PCIE_TPH stubs for callers.
> 
> Signed-off-by: Zhiping Zhang <zhipingz@meta.com>

Re: [PATCH v13 0/5] vfio/dma-buf: add TPH support for peer-to-peer access

[Zhiping Zhang]

 Hi Christian,

A gentle ping on this series, especially patch 3, which adds the
optional dma-buf get_pci_tph() callback. Could you please review this
when you have a chance?

https://lore.kernel.org/linux-pci/20260731211601.3033906-4-zhipingz@meta.com/

Thanks,
Zhiping


On Fri, Jul 31, 2026 at 2:22 PM Zhiping Zhang <zhipingz@meta.com> wrote:
>
> This series adds TLP Processing Hints (TPH) support to the VFIO dma-buf
> export path, allowing importing drivers (e.g. mlx5) to use the
> exporter's steering tag when performing peer-to-peer DMA into a
> VFIO-owned device.
>
> There is no separate in-tree vendor kernel driver for the target device:
> vfio-pci is the in-tree driver and the targeted device is managed
> from userspace via VFIO passthrough. That is why the ST has to flow
> through a uAPI: userspace owns the device and its ST table, so it is the
> entity that can configure a meaningful value for a given dma-buf. The
> kernel-visible participants are still in-tree: vfio-pci exports the
> dma-buf and mlx5 imports it.
>
> On the effect: the endpoint's PCIe ingress block uses the ST as
> an in-band instruction for the incoming P2P TLP -- selecting a target
> cache partition and, on writes, an in-flight operation on the data
> before it lands. The dma-buf callback keeps this opaque to the
> framework -- only the producer (userspace owner of the VFIO device)
> and the consumer (endpoint block) need to interpret the value. The
> dma-buf get_pci_tph callback itself is optional, but workloads that
> depend on the endpoint's in-flight operation need it because fallback
> does not produce the same result.
>
> The dma-buf hook is intentionally generic and discoverable rather than
> a private side channel. The exporter owns the completing address
> space for the dma-buf and decides whether it can provide a meaningful
> ST/PH tuple for that completer; the dma-buf core keeps the tuple opaque,
> and importers merely request the namespace they support and place the
> returned value on generated TLPs. Exporters that cannot derive a
> meaningful tuple simply return -EOPNOTSUPP.
>
> TPH is advisory: a steering tag that is not honored on the path (for
> example an intermediate routing element that does not forward the TPH
> prefix) is ignored and the request completes as an ordinary,
> non-TPH transaction (PCIe Base 6.4 sec 2.2.7.1). This series therefore
> targets the same-Root-Port / common-switch topology, where the ST
> reaches the completer; cross-Root-Port P2P is best-effort and is not
> gated in the uAPI, since supplying an unused ST is harmless and there
> is no discoverable "TPH routing" capability to test against.
>
> Patch 1 folds the reserved 0b10 "TPH Completer Supported" encoding into
> "not supported" in get_rp_completer_type(), so only architected values
> can reach the TPH Requester Enable field. It was previously posted
> standalone to linux-pci; per Alex Williamson's v12 review it now travels
> with the series, which removes the cross-tree ordering dependency and
> lets review tooling apply the series as posted.
> Patch 2 adds small PCI/TPH type helpers so drivers can query the enabled
> TPH requester mode and the device's TPH Completer Supported field
> without reaching into pci_dev internals (and so callers in
> CONFIG_PCIE_TPH=n builds get a clean fallback). pcie_tph_completer_type()
> applies the same reserved-encoding fold as get_rp_completer_type(),
> inlined locally so the helper is self-contained.
> Patch 3 adds the optional dma_buf_ops::get_pci_tph callback plus the
> dma_buf_get_pci_tph() importer wrapper so importers can fetch TPH
> metadata from an exporter under dmabuf->resv.
> Patch 4 implements get_pci_tph in vfio-pci and adds the new uAPI
> (VFIO_DEVICE_FEATURE_DMA_BUF_TPH) for userspace to attach the metadata.
> Patch 5 wires up the mlx5 RDMA driver as a consumer. It also enforces the
> dma_buf_get_pci_tph() steering-tag lifetime: the tag is only valid for the
> mapping it was queried against, and the mkey's TPH fields cannot be
> reprogrammed in place. mlx5 therefore records the registration-time tuple
> and re-queries after each dma-buf mapping is established under
> dmabuf->resv; unchanged tuples continue with the existing mkey, while
> changed or missing tuples fail the remap rather than continue with a
> stale hint. For vfio-pci BAR dma-bufs this is expected to be a no-op
> because invalidation is revoke/quiesce, not movement to a new backing
> placement, and the userspace-provided tuple is not changed by the
> revoke/un-revoke path.
>
> Build-tested with both CONFIG_PCIE_TPH=y and CONFIG_PCIE_TPH=n.
> Functional validation on the target topology: PCIe analyzer captures
> on the P2P TLPs confirm the ST emitted by mlx5 matches the value
> configured through VFIO_DEVICE_FEATURE_DMA_BUF_TPH, and the end-to-end
> P2P workload only produces results consistent with the endpoint's
> ST-selected in-flight operation. For example, with userspace
> configuring 8-bit ST=0xf0 and PH=2, an analyzer capture of a peer-to-
> peer MWr64 shows "STP MWr64 TC=0 OHC=2 ..." followed by "OHC-B
> ST=F0h PH=2 HV=1":
> (TLP Captures)
> 08000260 -> STP MWr64 TC=0 OHC=2 TS=0 Attr=0 L=8
> F0000004 -> RID=4h:0h.0h EP- Tag=F0h
> E0200000 -> AddrH=000020E0h
> 00080006 -> AddrL=06000800h
> 90F00000 -> OHC-B ST=F0h PH=2 HV=1 AMA=0 AV-
>
> The dma-buf get_pci_tph interface has also been exercised by a second,
> independent importer: a different vendor's NIC whose driver is not yet
> upstream, locally taught to call dma_buf_get_pci_tph(). A PCIe analyzer
> confirmed the ST it placed on outbound P2P TLPs matches the value
> configured through VFIO_DEVICE_FEATURE_DMA_BUF_TPH, the same result as
> with mlx5. Two unrelated importer drivers exercising the callback
> end-to-end shows the interface is not tied to a single consumer. That
> importer change is out-of-tree and not part of this series. For that
> second importer, with userspace configuring 8-bit ST=0xe0 and PH=0,
> an analyzer capture shows:
> (TLP Captures)
> 08200260 -> STP MWr64 TC=0 OHC=2 TS=1 Attr=0 L=8
> 4E00004C -> RID=4Ch:0h.0h EP- Tag=4Eh
> 00170000 -> AddrH=00001700h
> 00200006 -> AddrL=06002000h
> 10E00000 -> OHC-B ST=E0h PH=0 HV=1 AMA=0 AV-
>
> Changes since v12:
>   Patch 1 (PCI/TPH, new to the series): the reserved-encoding fold,
>   previously posted standalone to linux-pci [1], is now the first patch
>   here (Alex Williamson). Sashiko could not apply v12 because of that
>   external dependency; with the fold in-series and the mlx5 leak fix in
>   linux-next, v13 has none. The code is unchanged from the standalone
>   v3; the Fixes: tag is dropped, since no code path can reach the
>   reserved encoding today and the patch is hardening rather than a fix
>   for observed silicon.
>
>   Patch 2 (PCI/TPH): inline the reserved-encoding fold in
>   pcie_tph_completer_type() rather than calling the helper that earlier
>   folding revisions added; that helper was dropped in folding v3 per
>   Bjorn Helgaas and Wei Huang.
>
>   Patch 3 (dma-buf): no functional change.
>
>   Patch 4 (vfio/pci): also gate the DMA_BUF_TPH feature on
>   vdev->pci_ops->get_dmabuf_phys, matching
>   vfio_pci_core_feature_dma_buf(). Without it PROBE reported the feature
>   as supported on a device that advertises TPH Completer support but
>   cannot export a vfio dma-buf at all, so nothing could ever carry the
>   metadata (Alex Williamson, who raised this to uAPI-affecting
>   severity).
>
>   Patch 5 (mlx5): keep the !dev->st early-out in mlx5_st_alloc_index()
>   ahead of the pcie_tph_get_cpu_st() call, so splitting out
>   mlx5_st_alloc_index_by_tag() neither adds an ACPI _DSM invocation on
>   devices without ST support nor changes the errno userspace sees when
>   the _DSM lookup fails (Alex Williamson). The commit message now
>   describes this rather than presenting the split as a pure extraction.
>
> Previous link:
> v12: https://lore.kernel.org/linux-pci/20260715204008.3911275-1-zhipingz@meta.com/
> v11: https://lore.kernel.org/linux-pci/20260702181025.2694961-1-zhipingz@meta.com/
> v10: https://lore.kernel.org/linux-pci/20260630224328.3218796-1-zhipingz@meta.com/
> v9: https://lore.kernel.org/dri-devel/20260622184211.2229399-1-zhipingz@meta.com/
> v8: https://lore.kernel.org/dri-devel/20260615065912.2177918-1-zhipingz@meta.com/
> v7: https://lore.kernel.org/dri-devel/20260611161546.4075580-1-zhipingz@meta.com/
> v6: https://lore.kernel.org/dri-devel/20260608185646.4085127-1-zhipingz@meta.com/
> v5: https://lore.kernel.org/dri-devel/20260526144401.1485788-1-zhipingz@meta.com/
> v4: https://lore.kernel.org/linux-pci/20260519201401.1558410-1-zhipingz@meta.com/
> v3: https://lore.kernel.org/linux-pci/20260512184755.4137227-1-zhipingz@meta.com/
> v2: https://lore.kernel.org/linux-pci/20260430200704.352228-1-zhipingz@meta.com/
>
> Zhiping Zhang (5):
>   PCI/TPH: treat reserved 0b10 completer encoding as unsupported
>   PCI/TPH: Add requester/completer type helpers
>   dma-buf: add optional get_pci_tph() callback
>   vfio/pci: implement get_pci_tph and DMA_BUF_TPH feature
>   RDMA/mlx5: get tph for p2p access when registering dma-buf mr
>
>  drivers/dma-buf/dma-buf.c                     |  32 ++++
>  drivers/infiniband/hw/mlx5/main.c             |   1 +
>  drivers/infiniband/hw/mlx5/mlx5_ib.h          |  11 ++
>  drivers/infiniband/hw/mlx5/mr.c               | 151 +++++++++++++++++-
>  drivers/infiniband/hw/mlx5/odp.c              |   7 +
>  .../net/ethernet/mellanox/mlx5/core/lib/st.c  |  52 +++++-
>  drivers/pci/tph.c                             |  55 ++++++-
>  drivers/vfio/pci/vfio_pci_core.c              |   3 +
>  drivers/vfio/pci/vfio_pci_dmabuf.c            | 120 +++++++++++++-
>  drivers/vfio/pci/vfio_pci_priv.h              |  13 ++
>  include/linux/dma-buf.h                       |  25 +++
>  include/linux/mlx5/driver.h                   |  15 ++
>  include/linux/pci-tph.h                       |   8 +
>  include/uapi/linux/vfio.h                     |  43 +++++
>  14 files changed, 517 insertions(+), 19 deletions(-)
>
> --
> 2.53.0-Meta
>

Re: [PATCH v13 0/5] vfio/dma-buf: add TPH support for peer-to-peer access

[Zhiping Zhang]

Hi Christian,

Another ping on this series for your attention. Pls see below for more details.

Thanks,
Zhiping

On Fri, Aug 14, 2026 at 11:08 AM Zhiping Zhang <zhipingz@meta.com> wrote:
>
>  Hi Christian,
>
> A gentle ping on this series, especially patch 3, which adds the
> optional dma-buf get_pci_tph() callback. Could you please review this
> when you have a chance?
>
> https://lore.kernel.org/linux-pci/20260731211601.3033906-4-zhipingz@meta.com/
>
> Thanks,
> Zhiping
>
>
> On Fri, Jul 31, 2026 at 2:22 PM Zhiping Zhang <zhipingz@meta.com> wrote:
> >
> > This series adds TLP Processing Hints (TPH) support to the VFIO dma-buf
> > export path, allowing importing drivers (e.g. mlx5) to use the
> > exporter's steering tag when performing peer-to-peer DMA into a
> > VFIO-owned device.
> >
> > There is no separate in-tree vendor kernel driver for the target device:
> > vfio-pci is the in-tree driver and the targeted device is managed
> > from userspace via VFIO passthrough. That is why the ST has to flow
> > through a uAPI: userspace owns the device and its ST table, so it is the
> > entity that can configure a meaningful value for a given dma-buf. The
> > kernel-visible participants are still in-tree: vfio-pci exports the
> > dma-buf and mlx5 imports it.
> >
> > On the effect: the endpoint's PCIe ingress block uses the ST as
> > an in-band instruction for the incoming P2P TLP -- selecting a target
> > cache partition and, on writes, an in-flight operation on the data
> > before it lands. The dma-buf callback keeps this opaque to the
> > framework -- only the producer (userspace owner of the VFIO device)
> > and the consumer (endpoint block) need to interpret the value. The
> > dma-buf get_pci_tph callback itself is optional, but workloads that
> > depend on the endpoint's in-flight operation need it because fallback
> > does not produce the same result.
> >
> > The dma-buf hook is intentionally generic and discoverable rather than
> > a private side channel. The exporter owns the completing address
> > space for the dma-buf and decides whether it can provide a meaningful
> > ST/PH tuple for that completer; the dma-buf core keeps the tuple opaque,
> > and importers merely request the namespace they support and place the
> > returned value on generated TLPs. Exporters that cannot derive a
> > meaningful tuple simply return -EOPNOTSUPP.
> >
> > TPH is advisory: a steering tag that is not honored on the path (for
> > example an intermediate routing element that does not forward the TPH
> > prefix) is ignored and the request completes as an ordinary,
> > non-TPH transaction (PCIe Base 6.4 sec 2.2.7.1). This series therefore
> > targets the same-Root-Port / common-switch topology, where the ST
> > reaches the completer; cross-Root-Port P2P is best-effort and is not
> > gated in the uAPI, since supplying an unused ST is harmless and there
> > is no discoverable "TPH routing" capability to test against.
> >
> > Patch 1 folds the reserved 0b10 "TPH Completer Supported" encoding into
> > "not supported" in get_rp_completer_type(), so only architected values
> > can reach the TPH Requester Enable field. It was previously posted
> > standalone to linux-pci; per Alex Williamson's v12 review it now travels
> > with the series, which removes the cross-tree ordering dependency and
> > lets review tooling apply the series as posted.
> > Patch 2 adds small PCI/TPH type helpers so drivers can query the enabled
> > TPH requester mode and the device's TPH Completer Supported field
> > without reaching into pci_dev internals (and so callers in
> > CONFIG_PCIE_TPH=n builds get a clean fallback). pcie_tph_completer_type()
> > applies the same reserved-encoding fold as get_rp_completer_type(),
> > inlined locally so the helper is self-contained.
> > Patch 3 adds the optional dma_buf_ops::get_pci_tph callback plus the
> > dma_buf_get_pci_tph() importer wrapper so importers can fetch TPH
> > metadata from an exporter under dmabuf->resv.
> > Patch 4 implements get_pci_tph in vfio-pci and adds the new uAPI
> > (VFIO_DEVICE_FEATURE_DMA_BUF_TPH) for userspace to attach the metadata.
> > Patch 5 wires up the mlx5 RDMA driver as a consumer. It also enforces the
> > dma_buf_get_pci_tph() steering-tag lifetime: the tag is only valid for the
> > mapping it was queried against, and the mkey's TPH fields cannot be
> > reprogrammed in place. mlx5 therefore records the registration-time tuple
> > and re-queries after each dma-buf mapping is established under
> > dmabuf->resv; unchanged tuples continue with the existing mkey, while
> > changed or missing tuples fail the remap rather than continue with a
> > stale hint. For vfio-pci BAR dma-bufs this is expected to be a no-op
> > because invalidation is revoke/quiesce, not movement to a new backing
> > placement, and the userspace-provided tuple is not changed by the
> > revoke/un-revoke path.
> >
> > Build-tested with both CONFIG_PCIE_TPH=y and CONFIG_PCIE_TPH=n.
> > Functional validation on the target topology: PCIe analyzer captures
> > on the P2P TLPs confirm the ST emitted by mlx5 matches the value
> > configured through VFIO_DEVICE_FEATURE_DMA_BUF_TPH, and the end-to-end
> > P2P workload only produces results consistent with the endpoint's
> > ST-selected in-flight operation. For example, with userspace
> > configuring 8-bit ST=0xf0 and PH=2, an analyzer capture of a peer-to-
> > peer MWr64 shows "STP MWr64 TC=0 OHC=2 ..." followed by "OHC-B
> > ST=F0h PH=2 HV=1":
> > (TLP Captures)
> > 08000260 -> STP MWr64 TC=0 OHC=2 TS=0 Attr=0 L=8
> > F0000004 -> RID=4h:0h.0h EP- Tag=F0h
> > E0200000 -> AddrH=000020E0h
> > 00080006 -> AddrL=06000800h
> > 90F00000 -> OHC-B ST=F0h PH=2 HV=1 AMA=0 AV-
> >
> > The dma-buf get_pci_tph interface has also been exercised by a second,
> > independent importer: a different vendor's NIC whose driver is not yet
> > upstream, locally taught to call dma_buf_get_pci_tph(). A PCIe analyzer
> > confirmed the ST it placed on outbound P2P TLPs matches the value
> > configured through VFIO_DEVICE_FEATURE_DMA_BUF_TPH, the same result as
> > with mlx5. Two unrelated importer drivers exercising the callback
> > end-to-end shows the interface is not tied to a single consumer. That
> > importer change is out-of-tree and not part of this series. For that
> > second importer, with userspace configuring 8-bit ST=0xe0 and PH=0,
> > an analyzer capture shows:
> > (TLP Captures)
> > 08200260 -> STP MWr64 TC=0 OHC=2 TS=1 Attr=0 L=8
> > 4E00004C -> RID=4Ch:0h.0h EP- Tag=4Eh
> > 00170000 -> AddrH=00001700h
> > 00200006 -> AddrL=06002000h
> > 10E00000 -> OHC-B ST=E0h PH=0 HV=1 AMA=0 AV-
> >
> > Changes since v12:
> >   Patch 1 (PCI/TPH, new to the series): the reserved-encoding fold,
> >   previously posted standalone to linux-pci [1], is now the first patch
> >   here (Alex Williamson). Sashiko could not apply v12 because of that
> >   external dependency; with the fold in-series and the mlx5 leak fix in
> >   linux-next, v13 has none. The code is unchanged from the standalone
> >   v3; the Fixes: tag is dropped, since no code path can reach the
> >   reserved encoding today and the patch is hardening rather than a fix
> >   for observed silicon.
> >
> >   Patch 2 (PCI/TPH): inline the reserved-encoding fold in
> >   pcie_tph_completer_type() rather than calling the helper that earlier
> >   folding revisions added; that helper was dropped in folding v3 per
> >   Bjorn Helgaas and Wei Huang.
> >
> >   Patch 3 (dma-buf): no functional change.
> >
> >   Patch 4 (vfio/pci): also gate the DMA_BUF_TPH feature on
> >   vdev->pci_ops->get_dmabuf_phys, matching
> >   vfio_pci_core_feature_dma_buf(). Without it PROBE reported the feature
> >   as supported on a device that advertises TPH Completer support but
> >   cannot export a vfio dma-buf at all, so nothing could ever carry the
> >   metadata (Alex Williamson, who raised this to uAPI-affecting
> >   severity).
> >
> >   Patch 5 (mlx5): keep the !dev->st early-out in mlx5_st_alloc_index()
> >   ahead of the pcie_tph_get_cpu_st() call, so splitting out
> >   mlx5_st_alloc_index_by_tag() neither adds an ACPI _DSM invocation on
> >   devices without ST support nor changes the errno userspace sees when
> >   the _DSM lookup fails (Alex Williamson). The commit message now
> >   describes this rather than presenting the split as a pure extraction.
> >
> > Previous link:
> > v12: https://lore.kernel.org/linux-pci/20260715204008.3911275-1-zhipingz@meta.com/
> > v11: https://lore.kernel.org/linux-pci/20260702181025.2694961-1-zhipingz@meta.com/
> > v10: https://lore.kernel.org/linux-pci/20260630224328.3218796-1-zhipingz@meta.com/
> > v9: https://lore.kernel.org/dri-devel/20260622184211.2229399-1-zhipingz@meta.com/
> > v8: https://lore.kernel.org/dri-devel/20260615065912.2177918-1-zhipingz@meta.com/
> > v7: https://lore.kernel.org/dri-devel/20260611161546.4075580-1-zhipingz@meta.com/
> > v6: https://lore.kernel.org/dri-devel/20260608185646.4085127-1-zhipingz@meta.com/
> > v5: https://lore.kernel.org/dri-devel/20260526144401.1485788-1-zhipingz@meta.com/
> > v4: https://lore.kernel.org/linux-pci/20260519201401.1558410-1-zhipingz@meta.com/
> > v3: https://lore.kernel.org/linux-pci/20260512184755.4137227-1-zhipingz@meta.com/
> > v2: https://lore.kernel.org/linux-pci/20260430200704.352228-1-zhipingz@meta.com/
> >
> > Zhiping Zhang (5):
> >   PCI/TPH: treat reserved 0b10 completer encoding as unsupported
> >   PCI/TPH: Add requester/completer type helpers
> >   dma-buf: add optional get_pci_tph() callback
> >   vfio/pci: implement get_pci_tph and DMA_BUF_TPH feature
> >   RDMA/mlx5: get tph for p2p access when registering dma-buf mr
> >
> >  drivers/dma-buf/dma-buf.c                     |  32 ++++
> >  drivers/infiniband/hw/mlx5/main.c             |   1 +
> >  drivers/infiniband/hw/mlx5/mlx5_ib.h          |  11 ++
> >  drivers/infiniband/hw/mlx5/mr.c               | 151 +++++++++++++++++-
> >  drivers/infiniband/hw/mlx5/odp.c              |   7 +
> >  .../net/ethernet/mellanox/mlx5/core/lib/st.c  |  52 +++++-
> >  drivers/pci/tph.c                             |  55 ++++++-
> >  drivers/vfio/pci/vfio_pci_core.c              |   3 +
> >  drivers/vfio/pci/vfio_pci_dmabuf.c            | 120 +++++++++++++-
> >  drivers/vfio/pci/vfio_pci_priv.h              |  13 ++
> >  include/linux/dma-buf.h                       |  25 +++
> >  include/linux/mlx5/driver.h                   |  15 ++
> >  include/linux/pci-tph.h                       |   8 +
> >  include/uapi/linux/vfio.h                     |  43 +++++
> >  14 files changed, 517 insertions(+), 19 deletions(-)
> >
> > --
> > 2.53.0-Meta
> >

Re: [PATCH v13 0/5] vfio/dma-buf: add TPH support for peer-to-peer access - ping

[Zhiping Zhang]

Hi Christian,

Third ping on this series. The only patch still without review is 3/5,
which adds an optional get_pci_tph() callback to dma_buf_ops:

https://lore.kernel.org/kvm/20260731211601.3033906-4-zhipingz@meta.com/

It lets an exporter report the PCIe TPH steering tag for a
peer-to-peer mapping. The callback is optional -- exporters that don't
implement it behave exactly as today -- and nothing else in dma-buf
changes.

The rest of the series is settled: Chengwen acked patch 2, the
automated review is clean, and there are no open comments from v12.
Alex is holding his ack until the dma-buf side has been looked at, so
this one callback is gating the whole series.

If you have concerns about the API shape or the locking, I'll respin.
If someone else should review the dma-buf side, tell me who and I'll
take it there. Otherwise I'll check with Alex and Jason next week on
how to proceed, since 7.3 is in rc and the next window is about a
month out.

Thanks,
Zhiping

On Wed, Sep 2, 2026 at 1:53 PM Zhiping Zhang <zhipingz@meta.com> wrote:
>
> Hi Christian,
>
> Another ping on this series for your attention. Pls see below for more details.
>
> Thanks,
> Zhiping
>
> On Fri, Aug 14, 2026 at 11:08 AM Zhiping Zhang <zhipingz@meta.com> wrote:
> >
> >  Hi Christian,
> >
> > A gentle ping on this series, especially patch 3, which adds the
> > optional dma-buf get_pci_tph() callback. Could you please review this
> > when you have a chance?
> >
> > https://lore.kernel.org/linux-pci/20260731211601.3033906-4-zhipingz@meta.com/
> >
> > Thanks,
> > Zhiping
> >
> >
> > On Fri, Jul 31, 2026 at 2:22 PM Zhiping Zhang <zhipingz@meta.com> wrote:
> > >
> > > This series adds TLP Processing Hints (TPH) support to the VFIO dma-buf
> > > export path, allowing importing drivers (e.g. mlx5) to use the
> > > exporter's steering tag when performing peer-to-peer DMA into a
> > > VFIO-owned device.
> > >
> > > There is no separate in-tree vendor kernel driver for the target device:
> > > vfio-pci is the in-tree driver and the targeted device is managed
> > > from userspace via VFIO passthrough. That is why the ST has to flow
> > > through a uAPI: userspace owns the device and its ST table, so it is the
> > > entity that can configure a meaningful value for a given dma-buf. The
> > > kernel-visible participants are still in-tree: vfio-pci exports the
> > > dma-buf and mlx5 imports it.
> > >
> > > On the effect: the endpoint's PCIe ingress block uses the ST as
> > > an in-band instruction for the incoming P2P TLP -- selecting a target
> > > cache partition and, on writes, an in-flight operation on the data
> > > before it lands. The dma-buf callback keeps this opaque to the
> > > framework -- only the producer (userspace owner of the VFIO device)
> > > and the consumer (endpoint block) need to interpret the value. The
> > > dma-buf get_pci_tph callback itself is optional, but workloads that
> > > depend on the endpoint's in-flight operation need it because fallback
> > > does not produce the same result.
> > >
> > > The dma-buf hook is intentionally generic and discoverable rather than
> > > a private side channel. The exporter owns the completing address
> > > space for the dma-buf and decides whether it can provide a meaningful
> > > ST/PH tuple for that completer; the dma-buf core keeps the tuple opaque,
> > > and importers merely request the namespace they support and place the
> > > returned value on generated TLPs. Exporters that cannot derive a
> > > meaningful tuple simply return -EOPNOTSUPP.
> > >
> > > TPH is advisory: a steering tag that is not honored on the path (for
> > > example an intermediate routing element that does not forward the TPH
> > > prefix) is ignored and the request completes as an ordinary,
> > > non-TPH transaction (PCIe Base 6.4 sec 2.2.7.1). This series therefore
> > > targets the same-Root-Port / common-switch topology, where the ST
> > > reaches the completer; cross-Root-Port P2P is best-effort and is not
> > > gated in the uAPI, since supplying an unused ST is harmless and there
> > > is no discoverable "TPH routing" capability to test against.
> > >
> > > Patch 1 folds the reserved 0b10 "TPH Completer Supported" encoding into
> > > "not supported" in get_rp_completer_type(), so only architected values
> > > can reach the TPH Requester Enable field. It was previously posted
> > > standalone to linux-pci; per Alex Williamson's v12 review it now travels
> > > with the series, which removes the cross-tree ordering dependency and
> > > lets review tooling apply the series as posted.
> > > Patch 2 adds small PCI/TPH type helpers so drivers can query the enabled
> > > TPH requester mode and the device's TPH Completer Supported field
> > > without reaching into pci_dev internals (and so callers in
> > > CONFIG_PCIE_TPH=n builds get a clean fallback). pcie_tph_completer_type()
> > > applies the same reserved-encoding fold as get_rp_completer_type(),
> > > inlined locally so the helper is self-contained.
> > > Patch 3 adds the optional dma_buf_ops::get_pci_tph callback plus the
> > > dma_buf_get_pci_tph() importer wrapper so importers can fetch TPH
> > > metadata from an exporter under dmabuf->resv.
> > > Patch 4 implements get_pci_tph in vfio-pci and adds the new uAPI
> > > (VFIO_DEVICE_FEATURE_DMA_BUF_TPH) for userspace to attach the metadata.
> > > Patch 5 wires up the mlx5 RDMA driver as a consumer. It also enforces the
> > > dma_buf_get_pci_tph() steering-tag lifetime: the tag is only valid for the
> > > mapping it was queried against, and the mkey's TPH fields cannot be
> > > reprogrammed in place. mlx5 therefore records the registration-time tuple
> > > and re-queries after each dma-buf mapping is established under
> > > dmabuf->resv; unchanged tuples continue with the existing mkey, while
> > > changed or missing tuples fail the remap rather than continue with a
> > > stale hint. For vfio-pci BAR dma-bufs this is expected to be a no-op
> > > because invalidation is revoke/quiesce, not movement to a new backing
> > > placement, and the userspace-provided tuple is not changed by the
> > > revoke/un-revoke path.
> > >
> > > Build-tested with both CONFIG_PCIE_TPH=y and CONFIG_PCIE_TPH=n.
> > > Functional validation on the target topology: PCIe analyzer captures
> > > on the P2P TLPs confirm the ST emitted by mlx5 matches the value
> > > configured through VFIO_DEVICE_FEATURE_DMA_BUF_TPH, and the end-to-end
> > > P2P workload only produces results consistent with the endpoint's
> > > ST-selected in-flight operation. For example, with userspace
> > > configuring 8-bit ST=0xf0 and PH=2, an analyzer capture of a peer-to-
> > > peer MWr64 shows "STP MWr64 TC=0 OHC=2 ..." followed by "OHC-B
> > > ST=F0h PH=2 HV=1":
> > > (TLP Captures)
> > > 08000260 -> STP MWr64 TC=0 OHC=2 TS=0 Attr=0 L=8
> > > F0000004 -> RID=4h:0h.0h EP- Tag=F0h
> > > E0200000 -> AddrH=000020E0h
> > > 00080006 -> AddrL=06000800h
> > > 90F00000 -> OHC-B ST=F0h PH=2 HV=1 AMA=0 AV-
> > >
> > > The dma-buf get_pci_tph interface has also been exercised by a second,
> > > independent importer: a different vendor's NIC whose driver is not yet
> > > upstream, locally taught to call dma_buf_get_pci_tph(). A PCIe analyzer
> > > confirmed the ST it placed on outbound P2P TLPs matches the value
> > > configured through VFIO_DEVICE_FEATURE_DMA_BUF_TPH, the same result as
> > > with mlx5. Two unrelated importer drivers exercising the callback
> > > end-to-end shows the interface is not tied to a single consumer. That
> > > importer change is out-of-tree and not part of this series. For that
> > > second importer, with userspace configuring 8-bit ST=0xe0 and PH=0,
> > > an analyzer capture shows:
> > > (TLP Captures)
> > > 08200260 -> STP MWr64 TC=0 OHC=2 TS=1 Attr=0 L=8
> > > 4E00004C -> RID=4Ch:0h.0h EP- Tag=4Eh
> > > 00170000 -> AddrH=00001700h
> > > 00200006 -> AddrL=06002000h
> > > 10E00000 -> OHC-B ST=E0h PH=0 HV=1 AMA=0 AV-
> > >
> > > Changes since v12:
> > >   Patch 1 (PCI/TPH, new to the series): the reserved-encoding fold,
> > >   previously posted standalone to linux-pci [1], is now the first patch
> > >   here (Alex Williamson). Sashiko could not apply v12 because of that
> > >   external dependency; with the fold in-series and the mlx5 leak fix in
> > >   linux-next, v13 has none. The code is unchanged from the standalone
> > >   v3; the Fixes: tag is dropped, since no code path can reach the
> > >   reserved encoding today and the patch is hardening rather than a fix
> > >   for observed silicon.
> > >
> > >   Patch 2 (PCI/TPH): inline the reserved-encoding fold in
> > >   pcie_tph_completer_type() rather than calling the helper that earlier
> > >   folding revisions added; that helper was dropped in folding v3 per
> > >   Bjorn Helgaas and Wei Huang.
> > >
> > >   Patch 3 (dma-buf): no functional change.
> > >
> > >   Patch 4 (vfio/pci): also gate the DMA_BUF_TPH feature on
> > >   vdev->pci_ops->get_dmabuf_phys, matching
> > >   vfio_pci_core_feature_dma_buf(). Without it PROBE reported the feature
> > >   as supported on a device that advertises TPH Completer support but
> > >   cannot export a vfio dma-buf at all, so nothing could ever carry the
> > >   metadata (Alex Williamson, who raised this to uAPI-affecting
> > >   severity).
> > >
> > >   Patch 5 (mlx5): keep the !dev->st early-out in mlx5_st_alloc_index()
> > >   ahead of the pcie_tph_get_cpu_st() call, so splitting out
> > >   mlx5_st_alloc_index_by_tag() neither adds an ACPI _DSM invocation on
> > >   devices without ST support nor changes the errno userspace sees when
> > >   the _DSM lookup fails (Alex Williamson). The commit message now
> > >   describes this rather than presenting the split as a pure extraction.
> > >
> > > Previous link:
> > > v12: https://lore.kernel.org/linux-pci/20260715204008.3911275-1-zhipingz@meta.com/
> > > v11: https://lore.kernel.org/linux-pci/20260702181025.2694961-1-zhipingz@meta.com/
> > > v10: https://lore.kernel.org/linux-pci/20260630224328.3218796-1-zhipingz@meta.com/
> > > v9: https://lore.kernel.org/dri-devel/20260622184211.2229399-1-zhipingz@meta.com/
> > > v8: https://lore.kernel.org/dri-devel/20260615065912.2177918-1-zhipingz@meta.com/
> > > v7: https://lore.kernel.org/dri-devel/20260611161546.4075580-1-zhipingz@meta.com/
> > > v6: https://lore.kernel.org/dri-devel/20260608185646.4085127-1-zhipingz@meta.com/
> > > v5: https://lore.kernel.org/dri-devel/20260526144401.1485788-1-zhipingz@meta.com/
> > > v4: https://lore.kernel.org/linux-pci/20260519201401.1558410-1-zhipingz@meta.com/
> > > v3: https://lore.kernel.org/linux-pci/20260512184755.4137227-1-zhipingz@meta.com/
> > > v2: https://lore.kernel.org/linux-pci/20260430200704.352228-1-zhipingz@meta.com/
> > >
> > > Zhiping Zhang (5):
> > >   PCI/TPH: treat reserved 0b10 completer encoding as unsupported
> > >   PCI/TPH: Add requester/completer type helpers
> > >   dma-buf: add optional get_pci_tph() callback
> > >   vfio/pci: implement get_pci_tph and DMA_BUF_TPH feature
> > >   RDMA/mlx5: get tph for p2p access when registering dma-buf mr
> > >
> > >  drivers/dma-buf/dma-buf.c                     |  32 ++++
> > >  drivers/infiniband/hw/mlx5/main.c             |   1 +
> > >  drivers/infiniband/hw/mlx5/mlx5_ib.h          |  11 ++
> > >  drivers/infiniband/hw/mlx5/mr.c               | 151 +++++++++++++++++-
> > >  drivers/infiniband/hw/mlx5/odp.c              |   7 +
> > >  .../net/ethernet/mellanox/mlx5/core/lib/st.c  |  52 +++++-
> > >  drivers/pci/tph.c                             |  55 ++++++-
> > >  drivers/vfio/pci/vfio_pci_core.c              |   3 +
> > >  drivers/vfio/pci/vfio_pci_dmabuf.c            | 120 +++++++++++++-
> > >  drivers/vfio/pci/vfio_pci_priv.h              |  13 ++
> > >  include/linux/dma-buf.h                       |  25 +++
> > >  include/linux/mlx5/driver.h                   |  15 ++
> > >  include/linux/pci-tph.h                       |   8 +
> > >  include/uapi/linux/vfio.h                     |  43 +++++
> > >  14 files changed, 517 insertions(+), 19 deletions(-)
> > >
> > > --
> > > 2.53.0-Meta
> > >

Re: [PATCH v13 0/5] vfio/dma-buf: add TPH support for peer-to-peer access - ping

[Christian König]

Hi Zhiping,

sorry I'm completely underwater at the moment and don't have time to take another look at the full set.

But IIRC you already fixed my documentation requirements and skimming over the patch once more I can't see anything wrong of hand.

So feel free to add Acked-by: Christian König <christian.koenig@amd.com> to patch "dma-buf: add optional get_pci_tph() callback" and push it upstream through the VFIO channels.

Regards,
Christian.

On 9/10/26 23:46, Zhiping Zhang wrote:
> Hi Christian,
> 
> Third ping on this series. The only patch still without review is 3/5,
> which adds an optional get_pci_tph() callback to dma_buf_ops:
> 
> https://lore.kernel.org/kvm/20260731211601.3033906-4-zhipingz@meta.com/
> 
> It lets an exporter report the PCIe TPH steering tag for a
> peer-to-peer mapping. The callback is optional -- exporters that don't
> implement it behave exactly as today -- and nothing else in dma-buf
> changes.
> 
> The rest of the series is settled: Chengwen acked patch 2, the
> automated review is clean, and there are no open comments from v12.
> Alex is holding his ack until the dma-buf side has been looked at, so
> this one callback is gating the whole series.
> 
> If you have concerns about the API shape or the locking, I'll respin.
> If someone else should review the dma-buf side, tell me who and I'll
> take it there. Otherwise I'll check with Alex and Jason next week on
> how to proceed, since 7.3 is in rc and the next window is about a
> month out.
> 
> Thanks,
> Zhiping
> 
> On Wed, Sep 2, 2026 at 1:53 PM Zhiping Zhang <zhipingz@meta.com> wrote:
>>
>> Hi Christian,
>>
>> Another ping on this series for your attention. Pls see below for more details.
>>
>> Thanks,
>> Zhiping
>>
>> On Fri, Aug 14, 2026 at 11:08 AM Zhiping Zhang <zhipingz@meta.com> wrote:
>>>
>>>  Hi Christian,
>>>
>>> A gentle ping on this series, especially patch 3, which adds the
>>> optional dma-buf get_pci_tph() callback. Could you please review this
>>> when you have a chance?
>>>
>>> https://lore.kernel.org/linux-pci/20260731211601.3033906-4-zhipingz@meta.com/
>>>
>>> Thanks,
>>> Zhiping
>>>
>>>
>>> On Fri, Jul 31, 2026 at 2:22 PM Zhiping Zhang <zhipingz@meta.com> wrote:
>>>>
>>>> This series adds TLP Processing Hints (TPH) support to the VFIO dma-buf
>>>> export path, allowing importing drivers (e.g. mlx5) to use the
>>>> exporter's steering tag when performing peer-to-peer DMA into a
>>>> VFIO-owned device.
>>>>
>>>> There is no separate in-tree vendor kernel driver for the target device:
>>>> vfio-pci is the in-tree driver and the targeted device is managed
>>>> from userspace via VFIO passthrough. That is why the ST has to flow
>>>> through a uAPI: userspace owns the device and its ST table, so it is the
>>>> entity that can configure a meaningful value for a given dma-buf. The
>>>> kernel-visible participants are still in-tree: vfio-pci exports the
>>>> dma-buf and mlx5 imports it.
>>>>
>>>> On the effect: the endpoint's PCIe ingress block uses the ST as
>>>> an in-band instruction for the incoming P2P TLP -- selecting a target
>>>> cache partition and, on writes, an in-flight operation on the data
>>>> before it lands. The dma-buf callback keeps this opaque to the
>>>> framework -- only the producer (userspace owner of the VFIO device)
>>>> and the consumer (endpoint block) need to interpret the value. The
>>>> dma-buf get_pci_tph callback itself is optional, but workloads that
>>>> depend on the endpoint's in-flight operation need it because fallback
>>>> does not produce the same result.
>>>>
>>>> The dma-buf hook is intentionally generic and discoverable rather than
>>>> a private side channel. The exporter owns the completing address
>>>> space for the dma-buf and decides whether it can provide a meaningful
>>>> ST/PH tuple for that completer; the dma-buf core keeps the tuple opaque,
>>>> and importers merely request the namespace they support and place the
>>>> returned value on generated TLPs. Exporters that cannot derive a
>>>> meaningful tuple simply return -EOPNOTSUPP.
>>>>
>>>> TPH is advisory: a steering tag that is not honored on the path (for
>>>> example an intermediate routing element that does not forward the TPH
>>>> prefix) is ignored and the request completes as an ordinary,
>>>> non-TPH transaction (PCIe Base 6.4 sec 2.2.7.1). This series therefore
>>>> targets the same-Root-Port / common-switch topology, where the ST
>>>> reaches the completer; cross-Root-Port P2P is best-effort and is not
>>>> gated in the uAPI, since supplying an unused ST is harmless and there
>>>> is no discoverable "TPH routing" capability to test against.
>>>>
>>>> Patch 1 folds the reserved 0b10 "TPH Completer Supported" encoding into
>>>> "not supported" in get_rp_completer_type(), so only architected values
>>>> can reach the TPH Requester Enable field. It was previously posted
>>>> standalone to linux-pci; per Alex Williamson's v12 review it now travels
>>>> with the series, which removes the cross-tree ordering dependency and
>>>> lets review tooling apply the series as posted.
>>>> Patch 2 adds small PCI/TPH type helpers so drivers can query the enabled
>>>> TPH requester mode and the device's TPH Completer Supported field
>>>> without reaching into pci_dev internals (and so callers in
>>>> CONFIG_PCIE_TPH=n builds get a clean fallback). pcie_tph_completer_type()
>>>> applies the same reserved-encoding fold as get_rp_completer_type(),
>>>> inlined locally so the helper is self-contained.
>>>> Patch 3 adds the optional dma_buf_ops::get_pci_tph callback plus the
>>>> dma_buf_get_pci_tph() importer wrapper so importers can fetch TPH
>>>> metadata from an exporter under dmabuf->resv.
>>>> Patch 4 implements get_pci_tph in vfio-pci and adds the new uAPI
>>>> (VFIO_DEVICE_FEATURE_DMA_BUF_TPH) for userspace to attach the metadata.
>>>> Patch 5 wires up the mlx5 RDMA driver as a consumer. It also enforces the
>>>> dma_buf_get_pci_tph() steering-tag lifetime: the tag is only valid for the
>>>> mapping it was queried against, and the mkey's TPH fields cannot be
>>>> reprogrammed in place. mlx5 therefore records the registration-time tuple
>>>> and re-queries after each dma-buf mapping is established under
>>>> dmabuf->resv; unchanged tuples continue with the existing mkey, while
>>>> changed or missing tuples fail the remap rather than continue with a
>>>> stale hint. For vfio-pci BAR dma-bufs this is expected to be a no-op
>>>> because invalidation is revoke/quiesce, not movement to a new backing
>>>> placement, and the userspace-provided tuple is not changed by the
>>>> revoke/un-revoke path.
>>>>
>>>> Build-tested with both CONFIG_PCIE_TPH=y and CONFIG_PCIE_TPH=n.
>>>> Functional validation on the target topology: PCIe analyzer captures
>>>> on the P2P TLPs confirm the ST emitted by mlx5 matches the value
>>>> configured through VFIO_DEVICE_FEATURE_DMA_BUF_TPH, and the end-to-end
>>>> P2P workload only produces results consistent with the endpoint's
>>>> ST-selected in-flight operation. For example, with userspace
>>>> configuring 8-bit ST=0xf0 and PH=2, an analyzer capture of a peer-to-
>>>> peer MWr64 shows "STP MWr64 TC=0 OHC=2 ..." followed by "OHC-B
>>>> ST=F0h PH=2 HV=1":
>>>> (TLP Captures)
>>>> 08000260 -> STP MWr64 TC=0 OHC=2 TS=0 Attr=0 L=8
>>>> F0000004 -> RID=4h:0h.0h EP- Tag=F0h
>>>> E0200000 -> AddrH=000020E0h
>>>> 00080006 -> AddrL=06000800h
>>>> 90F00000 -> OHC-B ST=F0h PH=2 HV=1 AMA=0 AV-
>>>>
>>>> The dma-buf get_pci_tph interface has also been exercised by a second,
>>>> independent importer: a different vendor's NIC whose driver is not yet
>>>> upstream, locally taught to call dma_buf_get_pci_tph(). A PCIe analyzer
>>>> confirmed the ST it placed on outbound P2P TLPs matches the value
>>>> configured through VFIO_DEVICE_FEATURE_DMA_BUF_TPH, the same result as
>>>> with mlx5. Two unrelated importer drivers exercising the callback
>>>> end-to-end shows the interface is not tied to a single consumer. That
>>>> importer change is out-of-tree and not part of this series. For that
>>>> second importer, with userspace configuring 8-bit ST=0xe0 and PH=0,
>>>> an analyzer capture shows:
>>>> (TLP Captures)
>>>> 08200260 -> STP MWr64 TC=0 OHC=2 TS=1 Attr=0 L=8
>>>> 4E00004C -> RID=4Ch:0h.0h EP- Tag=4Eh
>>>> 00170000 -> AddrH=00001700h
>>>> 00200006 -> AddrL=06002000h
>>>> 10E00000 -> OHC-B ST=E0h PH=0 HV=1 AMA=0 AV-
>>>>
>>>> Changes since v12:
>>>>   Patch 1 (PCI/TPH, new to the series): the reserved-encoding fold,
>>>>   previously posted standalone to linux-pci [1], is now the first patch
>>>>   here (Alex Williamson). Sashiko could not apply v12 because of that
>>>>   external dependency; with the fold in-series and the mlx5 leak fix in
>>>>   linux-next, v13 has none. The code is unchanged from the standalone
>>>>   v3; the Fixes: tag is dropped, since no code path can reach the
>>>>   reserved encoding today and the patch is hardening rather than a fix
>>>>   for observed silicon.
>>>>
>>>>   Patch 2 (PCI/TPH): inline the reserved-encoding fold in
>>>>   pcie_tph_completer_type() rather than calling the helper that earlier
>>>>   folding revisions added; that helper was dropped in folding v3 per
>>>>   Bjorn Helgaas and Wei Huang.
>>>>
>>>>   Patch 3 (dma-buf): no functional change.
>>>>
>>>>   Patch 4 (vfio/pci): also gate the DMA_BUF_TPH feature on
>>>>   vdev->pci_ops->get_dmabuf_phys, matching
>>>>   vfio_pci_core_feature_dma_buf(). Without it PROBE reported the feature
>>>>   as supported on a device that advertises TPH Completer support but
>>>>   cannot export a vfio dma-buf at all, so nothing could ever carry the
>>>>   metadata (Alex Williamson, who raised this to uAPI-affecting
>>>>   severity).
>>>>
>>>>   Patch 5 (mlx5): keep the !dev->st early-out in mlx5_st_alloc_index()
>>>>   ahead of the pcie_tph_get_cpu_st() call, so splitting out
>>>>   mlx5_st_alloc_index_by_tag() neither adds an ACPI _DSM invocation on
>>>>   devices without ST support nor changes the errno userspace sees when
>>>>   the _DSM lookup fails (Alex Williamson). The commit message now
>>>>   describes this rather than presenting the split as a pure extraction.
>>>>
>>>> Previous link:
>>>> v12: https://lore.kernel.org/linux-pci/20260715204008.3911275-1-zhipingz@meta.com/
>>>> v11: https://lore.kernel.org/linux-pci/20260702181025.2694961-1-zhipingz@meta.com/
>>>> v10: https://lore.kernel.org/linux-pci/20260630224328.3218796-1-zhipingz@meta.com/
>>>> v9: https://lore.kernel.org/dri-devel/20260622184211.2229399-1-zhipingz@meta.com/
>>>> v8: https://lore.kernel.org/dri-devel/20260615065912.2177918-1-zhipingz@meta.com/
>>>> v7: https://lore.kernel.org/dri-devel/20260611161546.4075580-1-zhipingz@meta.com/
>>>> v6: https://lore.kernel.org/dri-devel/20260608185646.4085127-1-zhipingz@meta.com/
>>>> v5: https://lore.kernel.org/dri-devel/20260526144401.1485788-1-zhipingz@meta.com/
>>>> v4: https://lore.kernel.org/linux-pci/20260519201401.1558410-1-zhipingz@meta.com/
>>>> v3: https://lore.kernel.org/linux-pci/20260512184755.4137227-1-zhipingz@meta.com/
>>>> v2: https://lore.kernel.org/linux-pci/20260430200704.352228-1-zhipingz@meta.com/
>>>>
>>>> Zhiping Zhang (5):
>>>>   PCI/TPH: treat reserved 0b10 completer encoding as unsupported
>>>>   PCI/TPH: Add requester/completer type helpers
>>>>   dma-buf: add optional get_pci_tph() callback
>>>>   vfio/pci: implement get_pci_tph and DMA_BUF_TPH feature
>>>>   RDMA/mlx5: get tph for p2p access when registering dma-buf mr
>>>>
>>>>  drivers/dma-buf/dma-buf.c                     |  32 ++++
>>>>  drivers/infiniband/hw/mlx5/main.c             |   1 +
>>>>  drivers/infiniband/hw/mlx5/mlx5_ib.h          |  11 ++
>>>>  drivers/infiniband/hw/mlx5/mr.c               | 151 +++++++++++++++++-
>>>>  drivers/infiniband/hw/mlx5/odp.c              |   7 +
>>>>  .../net/ethernet/mellanox/mlx5/core/lib/st.c  |  52 +++++-
>>>>  drivers/pci/tph.c                             |  55 ++++++-
>>>>  drivers/vfio/pci/vfio_pci_core.c              |   3 +
>>>>  drivers/vfio/pci/vfio_pci_dmabuf.c            | 120 +++++++++++++-
>>>>  drivers/vfio/pci/vfio_pci_priv.h              |  13 ++
>>>>  include/linux/dma-buf.h                       |  25 +++
>>>>  include/linux/mlx5/driver.h                   |  15 ++
>>>>  include/linux/pci-tph.h                       |   8 +
>>>>  include/uapi/linux/vfio.h                     |  43 +++++
>>>>  14 files changed, 517 insertions(+), 19 deletions(-)
>>>>
>>>> --
>>>> 2.53.0-Meta
>>>>