[RFC 2/2] RMDA MLX5: get tph for p2p access when registering dmabuf mr

[RFC 2/2] RMDA MLX5: get tph for p2p access when registering dmabuf mr

[Zhiping Zhang]

The patch adds a local function to check and get tph info when available during
dmabuf mr registration. Note the DMAH workflow for CPU still takes precedence in
the process. Currently, it only works with the direct st_mode. Compatibility
with other st_modes will be added in the forma patch set.

Signed-off-by: Zhiping Zhang <zhipingz@meta.com>
---
 drivers/infiniband/hw/mlx5/mr.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 325fa04cbe8a..c3eb5b24ef29 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -46,6 +46,8 @@
 #include "data_direct.h"
 #include "dmah.h"
 
+MODULE_IMPORT_NS("DMA_BUF");
+
 enum {
 	MAX_PENDING_REG_MR = 8,
 };
@@ -1623,6 +1625,32 @@ static struct dma_buf_attach_ops mlx5_ib_dmabuf_attach_ops = {
 	.move_notify = mlx5_ib_dmabuf_invalidate_cb,
 };
 
+static void get_tph_mr_dmabuf(struct mlx5_ib_dev *dev, int fd, u16 *st_index,
+							  u8 *ph)
+{
+	int ret;
+	struct dma_buf *dmabuf;
+	struct mlx5_core_dev *mdev = dev->mdev;
+
+	dmabuf = dma_buf_get(fd);
+	if (IS_ERR(dmabuf))
+		return;
+
+	if (!dmabuf->ops->get_tph)
+		goto end_dbuf_put;
+
+	ret = dmabuf->ops->get_tph(dmabuf, st_index, ph);
+	if (ret) {
+		*st_index = MLX5_MKC_PCIE_TPH_NO_STEERING_TAG_INDEX;
+		*ph = MLX5_IB_NO_PH;
+		mlx5_ib_dbg(dev, "get_tph failed (%d)\n", ret);
+		goto end_dbuf_put;
+	}
+
+end_dbuf_put:
+	dma_buf_put(dmabuf);
+};
+
 static struct ib_mr *
 reg_user_mr_dmabuf(struct ib_pd *pd, struct device *dma_device,
 		   u64 offset, u64 length, u64 virt_addr,
@@ -1662,6 +1690,8 @@ reg_user_mr_dmabuf(struct ib_pd *pd, struct device *dma_device,
 		ph = dmah->ph;
 		if (dmah->valid_fields & BIT(IB_DMAH_CPU_ID_EXISTS))
 			st_index = mdmah->st_index;
+	} else {
+		get_tph_mr_dmabuf(dev, fd, &st_index, &ph);
 	}
 
 	mr = alloc_cacheable_mr(pd, &umem_dmabuf->umem, virt_addr,
-- 
2.47.3

Re: [RFC 1/2] Vfio: add callback to get tph info for dmabuf

[Keith Busch]

The subject prefix should be lower case "vfio" to match the subsystem
commit style.

On Tue, Feb 10, 2026 at 11:39:54AM -0800, Zhiping Zhang wrote:
> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> index ac2329f24141..bff2f5f7e38d 100644
> --- a/include/uapi/linux/vfio.h
> +++ b/include/uapi/linux/vfio.h
> @@ -1501,6 +1501,8 @@ struct vfio_region_dma_range {
>  struct vfio_device_feature_dma_buf {
>  	__u32	region_index;
>  	__u32	open_flags;
> +	__u16   steering_tag;
> +	__u8    ph;
>  	__u32   flags;
>  	__u32   nr_ranges;
>  	struct vfio_region_dma_range dma_ranges[] __counted_by(nr_ranges);

I don't think you can add fields to a uapi struct like this since it
breaks comptibility. Instead, I think you may be able to carve it out of
the "flags" field since it's currently reserved to be 0, so I guess it's
potentially available to define a new feature.

Re: [RFC 2/2] RMDA MLX5: get tph for p2p access when registering dmabuf mr

[Keith Busch]

On Tue, Feb 10, 2026 at 11:39:55AM -0800, Zhiping Zhang wrote:
> +static void get_tph_mr_dmabuf(struct mlx5_ib_dev *dev, int fd, u16 *st_index,
> +							  u8 *ph)
> +{
> +	int ret;
> +	struct dma_buf *dmabuf;
> +
> +	dmabuf = dma_buf_get(fd);
> +	if (IS_ERR(dmabuf))
> +		return;
> +
> +	if (!dif there's any implication mabuf->ops->get_tph)
> +		goto end_dbuf_put;
> +
> +	ret = dmabuf->ops->get_tph(dmabuf, st_index, ph);

You defined the "get_tph" function to take a pointer to a raw steering
tag value, but you're passing in the steering index to it's table.

But in general, since you're letting the user put whatever they want in
the vfio private area, should there be some validation that it's in the
valid range? I'm also not quite sure how user space comes to know what
steering tag to use, or what harm might happen if the wrong one is used.

Re: [RFC 2/2] RMDA MLX5: get tph for p2p access when registering dmabuf mr

[Jason Gunthorpe]

On Thu, Feb 26, 2026 at 06:21:28PM -0700, Keith Busch wrote:
> On Tue, Feb 10, 2026 at 11:39:55AM -0800, Zhiping Zhang wrote:
> > +static void get_tph_mr_dmabuf(struct mlx5_ib_dev *dev, int fd, u16 *st_index,
> > +							  u8 *ph)
> > +{
> > +	int ret;
> > +	struct dma_buf *dmabuf;
> > +
> > +	dmabuf = dma_buf_get(fd);
> > +	if (IS_ERR(dmabuf))
> > +		return;
> > +
> > +	if (!dif there's any implication mabuf->ops->get_tph)
> > +		goto end_dbuf_put;
> > +
> > +	ret = dmabuf->ops->get_tph(dmabuf, st_index, ph);
> 
> You defined the "get_tph" function to take a pointer to a raw steering
> tag value, but you're passing in the steering index to it's table.

Yeah that's weird, there should be one TPH for a DMABUF, not many.

> But in general, since you're letting the user put whatever they want in
> the vfio private area, should there be some validation that it's in the
> valid range? I'm also not quite sure how user space comes to know what
> steering tag to use, or what harm might happen if the wrong one is used.

If the device is VFIO compatible then it needs to ensure that whatever
it does with its steering tags fit the security model of VFIO. You
can't harm the device - you can't reach outside the VFIO sandbox (eg
into another VF or something) and so on.

Under these conditions the kernel doesn't care what TPH is used, just
let userspace specify the raw bits on the wire.

Jason