[PATCH rdma-next v2 0/6] RDMA: add per-attribute UMEM for SRQ create and CQ resize

[PATCH rdma-next v2 0/6] RDMA: add per-attribute UMEM for SRQ create and CQ resize

[Jiri Pirko]

From: Jiri Pirko <jiri@nvidia.com>

This series continues extending the per-attribute UMEM model to the
couple of other uverbs flows that pin user buffers, so that userspace
can back those buffers using previously introduce descriptor
infrastructure.

The underlying infrastructure - ib_umem_get_attr_or_va() and the
per-command UMEM attributes - is already in place and used by the CQ and
QP create methods. This series applies the same model to the flows that
were still VA-only:
- SRQ create: the WQE buffer and the mlx5 doorbell record.
- CQ resize: the resized CQ buffer. Resize CQ was only reachable
  through the legacy write() command (IB_USER_VERBS_CMD_RESIZE_CQ), so
  a modern ioctl method (UVERBS_METHOD_CQ_RESIZE) is added first; the
  per-attribute UMEM descriptor is then carried on top of it.

Wire this up in mlx5 driver for both cases. When the new UMEM
attribute is not supplied, the code falls back to the existing UHW.

---

base-commit: 20ff9350862468af21b46cae2c22d17d6ec637f9
see individual patches for changelog

Jiri Pirko (6):
  RDMA/uverbs: Add SRQ buffer UMEM attribute
  RDMA/mlx5: Use UMEM attribute for SRQ buffer in create_srq
  RDMA/mlx5: Use UMEM attribute for SRQ doorbell record
  RDMA/uverbs: Add ioctl method for CQ resize
  RDMA/uverbs: Add CQ resize buffer UMEM attribute
  RDMA/mlx5: Use UMEM attribute for CQ resize buffer

 drivers/infiniband/core/uverbs_std_types_cq.c | 45 ++++++++++++++++++-
 .../infiniband/core/uverbs_std_types_srq.c    |  2 +
 drivers/infiniband/hw/mlx5/cq.c               |  9 ++--
 drivers/infiniband/hw/mlx5/main.c             |  1 +
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |  1 +
 drivers/infiniband/hw/mlx5/srq.c              | 25 ++++++++++-
 include/uapi/rdma/ib_user_ioctl_cmds.h        |  9 ++++
 include/uapi/rdma/mlx5_user_ioctl_cmds.h      |  4 ++
 8 files changed, 90 insertions(+), 6 deletions(-)

-- 
2.54.0

[PATCH rdma-next v2 1/6] RDMA/uverbs: Add SRQ buffer UMEM attribute

[Jiri Pirko]

From: Jiri Pirko <jiri@nvidia.com>

Apply the per-attribute UMEM model to the SRQ create method. Add an
optional UMEM attribute that backs the SRQ WQE buffer, so userspace can
supply it as either a VA or a dma-buf through a single descriptor,
consistent with the CQ and QP create methods.

mlx5 is the only driver that pins an SRQ WQE buffer via umem; it maps a
single ucmd->buf_addr region through this attribute. No other driver
implements a user SRQ buffer, so none of them use the attribute.

Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
 drivers/infiniband/core/uverbs_std_types_srq.c | 2 ++
 include/uapi/rdma/ib_user_ioctl_cmds.h         | 1 +
 2 files changed, 3 insertions(+)

diff --git a/drivers/infiniband/core/uverbs_std_types_srq.c b/drivers/infiniband/core/uverbs_std_types_srq.c
index e5513f828bdc..0421bdd225df 100644
--- a/drivers/infiniband/core/uverbs_std_types_srq.c
+++ b/drivers/infiniband/core/uverbs_std_types_srq.c
@@ -192,6 +192,8 @@ DECLARE_UVERBS_NAMED_METHOD(
 	UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_CREATE_SRQ_RESP_SRQ_NUM,
 			   UVERBS_ATTR_TYPE(u32),
 			   UA_OPTIONAL),
+	UVERBS_ATTR_UMEM(UVERBS_ATTR_CREATE_SRQ_BUF_UMEM,
+			 UA_OPTIONAL),
 	UVERBS_ATTR_UHW());
 
 static int UVERBS_HANDLER(UVERBS_METHOD_SRQ_DESTROY)(
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 839835bd4b23..1fef1e86b302 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -189,6 +189,7 @@ enum uverbs_attrs_create_srq_cmd_attr_ids {
 	UVERBS_ATTR_CREATE_SRQ_RESP_MAX_WR,
 	UVERBS_ATTR_CREATE_SRQ_RESP_MAX_SGE,
 	UVERBS_ATTR_CREATE_SRQ_RESP_SRQ_NUM,
+	UVERBS_ATTR_CREATE_SRQ_BUF_UMEM,
 };
 
 enum uverbs_attrs_destroy_srq_cmd_attr_ids {
-- 
2.54.0

[PATCH rdma-next v2 2/6] RDMA/mlx5: Use UMEM attribute for SRQ buffer in create_srq

[Jiri Pirko]

From: Jiri Pirko <jiri@nvidia.com>

Use the per-attribute UMEM helper to pin the SRQ buffer umem on demand.
ib_umem_get_attr_or_va() resolves the new CREATE_SRQ_BUF_UMEM attribute
when present and otherwise falls back to the existing UHW ucmd->buf_addr
VA, preserving the legacy behavior.

Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
 drivers/infiniband/hw/mlx5/srq.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/srq.c b/drivers/infiniband/hw/mlx5/srq.c
index 5bc48fef3744..6fa4c5a9a0d5 100644
--- a/drivers/infiniband/hw/mlx5/srq.c
+++ b/drivers/infiniband/hw/mlx5/srq.c
@@ -48,6 +48,8 @@ static int create_srq_user(struct ib_pd *pd, struct mlx5_ib_srq *srq,
 	struct mlx5_ib_create_srq ucmd;
 	struct mlx5_ib_ucontext *ucontext = rdma_udata_to_drv_context(
 		udata, struct mlx5_ib_ucontext, ibucontext);
+	struct uverbs_attr_bundle *attrs =
+		rdma_udata_to_uverbs_attr_bundle(udata);
 	int err;
 	u32 uidx = MLX5_IB_DEFAULT_UIDX;
 
@@ -66,7 +68,9 @@ static int create_srq_user(struct ib_pd *pd, struct mlx5_ib_srq *srq,
 
 	srq->wq_sig = !!(ucmd.flags & MLX5_SRQ_FLAG_SIGNATURE);
 
-	srq->umem = ib_umem_get_va(pd->device, ucmd.buf_addr, buf_size, 0);
+	srq->umem = ib_umem_get_attr_or_va(pd->device, attrs,
+					   UVERBS_ATTR_CREATE_SRQ_BUF_UMEM,
+					   ucmd.buf_addr, buf_size, 0);
 	if (IS_ERR(srq->umem)) {
 		mlx5_ib_dbg(dev, "failed umem get, size %d\n", buf_size);
 		err = PTR_ERR(srq->umem);
-- 
2.54.0

[PATCH rdma-next v2 3/6] RDMA/mlx5: Use UMEM attribute for SRQ doorbell record

[Jiri Pirko]

From: Jiri Pirko <jiri@nvidia.com>

Add an optional mlx5 driver-namespace UMEM attribute on SRQ create so
userspace can supply the doorbell record umem explicitly, symmetric to
the CQ and QP sides. Resolve it inside mlx5_ib_db_map_user() and use it
as a private DBR page when present; otherwise take the existing UHW
share-or-pin path that preserves per-page DBR sharing across CQ/QP/SRQ
in the same process.

Add mlx5's first UVERBS_OBJECT_SRQ UAPI definition chain to attach the
new attr.

Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
 drivers/infiniband/hw/mlx5/main.c        |  1 +
 drivers/infiniband/hw/mlx5/mlx5_ib.h     |  1 +
 drivers/infiniband/hw/mlx5/srq.c         | 19 ++++++++++++++++++-
 include/uapi/rdma/mlx5_user_ioctl_cmds.h |  4 ++++
 4 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index a558ac5bb219..e07479346f1a 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -4468,6 +4468,7 @@ static const struct uapi_definition mlx5_ib_defs[] = {
 	UAPI_DEF_CHAIN(mlx5_ib_dm_defs),
 	UAPI_DEF_CHAIN(mlx5_ib_create_cq_defs),
 	UAPI_DEF_CHAIN(mlx5_ib_create_qp_defs),
+	UAPI_DEF_CHAIN(mlx5_ib_create_srq_defs),
 
 	UAPI_DEF_CHAIN_OBJ_TREE(UVERBS_OBJECT_DEVICE, &mlx5_ib_query_context),
 	UAPI_DEF_CHAIN_OBJ_TREE(UVERBS_OBJECT_MR, &mlx5_ib_reg_dmabuf_mr),
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 0550cdfacad4..1b77668eb50a 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -1516,6 +1516,7 @@ extern const struct uapi_definition mlx5_ib_qos_defs[];
 extern const struct uapi_definition mlx5_ib_std_types_defs[];
 extern const struct uapi_definition mlx5_ib_create_cq_defs[];
 extern const struct uapi_definition mlx5_ib_create_qp_defs[];
+extern const struct uapi_definition mlx5_ib_create_srq_defs[];
 
 static inline int is_qp1(enum ib_qp_type qp_type)
 {
diff --git a/drivers/infiniband/hw/mlx5/srq.c b/drivers/infiniband/hw/mlx5/srq.c
index 6fa4c5a9a0d5..a973c1b7515f 100644
--- a/drivers/infiniband/hw/mlx5/srq.c
+++ b/drivers/infiniband/hw/mlx5/srq.c
@@ -10,6 +10,9 @@
 #include "mlx5_ib.h"
 #include "srq.h"
 
+#define UVERBS_MODULE_NAME mlx5_ib
+#include <rdma/uverbs_named_ioctl.h>
+
 static void *get_wqe(struct mlx5_ib_srq *srq, int n)
 {
 	return mlx5_frag_buf_get_wqe(&srq->fbc, n);
@@ -78,7 +81,9 @@ static int create_srq_user(struct ib_pd *pd, struct mlx5_ib_srq *srq,
 	}
 	in->umem = srq->umem;
 
-	err = mlx5_ib_db_map_user(ucontext, NULL, 0, ucmd.db_addr, &srq->db);
+	err = mlx5_ib_db_map_user(ucontext, attrs,
+				  MLX5_IB_ATTR_CREATE_SRQ_DBR_BUF_UMEM,
+				  ucmd.db_addr, &srq->db);
 	if (err) {
 		mlx5_ib_dbg(dev, "map doorbell failed\n");
 		goto err_umem;
@@ -466,3 +471,15 @@ int mlx5_ib_post_srq_recv(struct ib_srq *ibsrq, const struct ib_recv_wr *wr,
 
 	return err;
 }
+
+ADD_UVERBS_ATTRIBUTES_SIMPLE(
+	mlx5_ib_srq_create,
+	UVERBS_OBJECT_SRQ,
+	UVERBS_METHOD_SRQ_CREATE,
+	UVERBS_ATTR_UMEM(MLX5_IB_ATTR_CREATE_SRQ_DBR_BUF_UMEM,
+			 UA_OPTIONAL));
+
+const struct uapi_definition mlx5_ib_create_srq_defs[] = {
+	UAPI_DEF_CHAIN_OBJ_TREE(UVERBS_OBJECT_SRQ, &mlx5_ib_srq_create),
+	{},
+};
diff --git a/include/uapi/rdma/mlx5_user_ioctl_cmds.h b/include/uapi/rdma/mlx5_user_ioctl_cmds.h
index ddb898afd813..3528743e3858 100644
--- a/include/uapi/rdma/mlx5_user_ioctl_cmds.h
+++ b/include/uapi/rdma/mlx5_user_ioctl_cmds.h
@@ -281,6 +281,10 @@ enum mlx5_ib_create_qp_attrs {
 	MLX5_IB_ATTR_CREATE_QP_DBR_BUF_UMEM = UVERBS_ID_DRIVER_NS_WITH_UHW,
 };
 
+enum mlx5_ib_create_srq_attrs {
+	MLX5_IB_ATTR_CREATE_SRQ_DBR_BUF_UMEM = UVERBS_ID_DRIVER_NS_WITH_UHW,
+};
+
 enum mlx5_ib_reg_dmabuf_mr_attrs {
 	MLX5_IB_ATTR_REG_DMABUF_MR_ACCESS_FLAGS = (1U << UVERBS_ID_NS_SHIFT),
 };
-- 
2.54.0

[PATCH rdma-next v2 4/6] RDMA/uverbs: Add ioctl method for CQ resize

[Jiri Pirko]

From: Jiri Pirko <jiri@nvidia.com>

Resize CQ is currently only reachable through the legacy write()
uverbs command (IB_USER_VERBS_CMD_RESIZE_CQ). Add an equivalent modern
ioctl method, UVERBS_METHOD_CQ_RESIZE, on the CQ object so the
operation is available through the ioctl interface and can carry
per-attribute extensions. The handler mirrors the legacy command: it
looks up the CQ, calls resize_user_cq() and returns the new cqe count.
The legacy write path is left in place for ABI compatibility.

Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
v1->v2:
- removed unneeded cq check
---
 drivers/infiniband/core/uverbs_std_types_cq.c | 43 ++++++++++++++++++-
 include/uapi/rdma/ib_user_ioctl_cmds.h        |  7 +++
 2 files changed, 49 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/core/uverbs_std_types_cq.c b/drivers/infiniband/core/uverbs_std_types_cq.c
index 148cdd180dab..4699747f4ad3 100644
--- a/drivers/infiniband/core/uverbs_std_types_cq.c
+++ b/drivers/infiniband/core/uverbs_std_types_cq.c
@@ -228,11 +228,52 @@ DECLARE_UVERBS_NAMED_METHOD(
 			    UVERBS_ATTR_TYPE(struct ib_uverbs_destroy_cq_resp),
 			    UA_MANDATORY));
 
+static int UVERBS_HANDLER(UVERBS_METHOD_CQ_RESIZE)(
+	struct uverbs_attr_bundle *attrs)
+{
+	struct ib_cq *cq =
+		uverbs_attr_get_obj(attrs, UVERBS_ATTR_RESIZE_CQ_HANDLE);
+	u32 cqe;
+	int ret;
+
+	if (!cq->device->ops.resize_user_cq)
+		return -EOPNOTSUPP;
+
+	ret = uverbs_copy_from(&cqe, attrs, UVERBS_ATTR_RESIZE_CQ_CQE);
+	if (ret)
+		return ret;
+
+	if (!cqe)
+		return -EINVAL;
+
+	ret = cq->device->ops.resize_user_cq(cq, cqe, &attrs->driver_udata);
+	if (ret)
+		return ret;
+
+	return uverbs_copy_to(attrs, UVERBS_ATTR_RESIZE_CQ_RESP_CQE,
+			      &cq->cqe, sizeof(cq->cqe));
+}
+
+DECLARE_UVERBS_NAMED_METHOD(
+	UVERBS_METHOD_CQ_RESIZE,
+	UVERBS_ATTR_IDR(UVERBS_ATTR_RESIZE_CQ_HANDLE,
+			UVERBS_OBJECT_CQ,
+			UVERBS_ACCESS_READ,
+			UA_MANDATORY),
+	UVERBS_ATTR_PTR_IN(UVERBS_ATTR_RESIZE_CQ_CQE,
+			   UVERBS_ATTR_TYPE(u32),
+			   UA_MANDATORY),
+	UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_RESIZE_CQ_RESP_CQE,
+			    UVERBS_ATTR_TYPE(u32),
+			    UA_MANDATORY),
+	UVERBS_ATTR_UHW());
+
 DECLARE_UVERBS_NAMED_OBJECT(
 	UVERBS_OBJECT_CQ,
 	UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_ucq_object), uverbs_free_cq),
 	&UVERBS_METHOD(UVERBS_METHOD_CQ_CREATE),
-	&UVERBS_METHOD(UVERBS_METHOD_CQ_DESTROY)
+	&UVERBS_METHOD(UVERBS_METHOD_CQ_DESTROY),
+	&UVERBS_METHOD(UVERBS_METHOD_CQ_RESIZE)
 );
 
 const struct uapi_definition uverbs_def_obj_cq[] = {
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 1fef1e86b302..5d2451a03a83 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -125,6 +125,12 @@ enum uverbs_attrs_destroy_cq_cmd_attr_ids {
 	UVERBS_ATTR_DESTROY_CQ_RESP,
 };
 
+enum uverbs_attrs_resize_cq_cmd_attr_ids {
+	UVERBS_ATTR_RESIZE_CQ_HANDLE,
+	UVERBS_ATTR_RESIZE_CQ_CQE,
+	UVERBS_ATTR_RESIZE_CQ_RESP_CQE,
+};
+
 enum uverbs_attrs_create_flow_action_esp {
 	UVERBS_ATTR_CREATE_FLOW_ACTION_ESP_HANDLE,
 	UVERBS_ATTR_FLOW_ACTION_ESP_ATTRS,
@@ -205,6 +211,7 @@ enum uverbs_methods_srq {
 enum uverbs_methods_cq {
 	UVERBS_METHOD_CQ_CREATE,
 	UVERBS_METHOD_CQ_DESTROY,
+	UVERBS_METHOD_CQ_RESIZE,
 };
 
 enum uverbs_attrs_create_wq_cmd_attr_ids {
-- 
2.54.0

[PATCH rdma-next v2 5/6] RDMA/uverbs: Add CQ resize buffer UMEM attribute

[Jiri Pirko]

From: Jiri Pirko <jiri@nvidia.com>

Add an optional UMEM attribute to the CQ resize method that backs the
resized CQ buffer, so userspace can supply it as either a VA or a
dma-buf through a single descriptor, consistent with the CQ and QP
create methods.

mlx5 is the only driver that pins a resized CQ buffer via umem; it maps
a single ucmd->buf_addr region through this attribute.

Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
 drivers/infiniband/core/uverbs_std_types_cq.c | 2 ++
 include/uapi/rdma/ib_user_ioctl_cmds.h        | 1 +
 2 files changed, 3 insertions(+)

diff --git a/drivers/infiniband/core/uverbs_std_types_cq.c b/drivers/infiniband/core/uverbs_std_types_cq.c
index 4699747f4ad3..7904f8862018 100644
--- a/drivers/infiniband/core/uverbs_std_types_cq.c
+++ b/drivers/infiniband/core/uverbs_std_types_cq.c
@@ -266,6 +266,8 @@ DECLARE_UVERBS_NAMED_METHOD(
 	UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_RESIZE_CQ_RESP_CQE,
 			    UVERBS_ATTR_TYPE(u32),
 			    UA_MANDATORY),
+	UVERBS_ATTR_UMEM(UVERBS_ATTR_RESIZE_CQ_BUF_UMEM,
+			 UA_OPTIONAL),
 	UVERBS_ATTR_UHW());
 
 DECLARE_UVERBS_NAMED_OBJECT(
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 5d2451a03a83..7cab5daefe63 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -129,6 +129,7 @@ enum uverbs_attrs_resize_cq_cmd_attr_ids {
 	UVERBS_ATTR_RESIZE_CQ_HANDLE,
 	UVERBS_ATTR_RESIZE_CQ_CQE,
 	UVERBS_ATTR_RESIZE_CQ_RESP_CQE,
+	UVERBS_ATTR_RESIZE_CQ_BUF_UMEM,
 };
 
 enum uverbs_attrs_create_flow_action_esp {
-- 
2.54.0

[PATCH rdma-next v2 6/6] RDMA/mlx5: Use UMEM attribute for CQ resize buffer

[Jiri Pirko]

From: Jiri Pirko <jiri@nvidia.com>

Use the per-attribute UMEM helper to pin the resized CQ buffer umem on
demand. ib_umem_get_attr_or_va() resolves the new RESIZE_CQ_BUF_UMEM
attribute when present and otherwise falls back to the existing UHW
ucmd->buf_addr VA, preserving the legacy behavior.

Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
 drivers/infiniband/hw/mlx5/cq.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 49b4bf148a4a..a2d574aaebe1 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -1245,9 +1245,12 @@ static int resize_user(struct mlx5_ib_dev *dev, struct mlx5_ib_cq *cq,
 	if (ucmd.cqe_size && SIZE_MAX / ucmd.cqe_size <= entries - 1)
 		return -EINVAL;
 
-	umem = ib_umem_get_va(&dev->ib_dev, ucmd.buf_addr,
-			      (size_t)ucmd.cqe_size * entries,
-			      IB_ACCESS_LOCAL_WRITE);
+	umem = ib_umem_get_attr_or_va(&dev->ib_dev,
+				      rdma_udata_to_uverbs_attr_bundle(udata),
+				      UVERBS_ATTR_RESIZE_CQ_BUF_UMEM,
+				      ucmd.buf_addr,
+				      (size_t)ucmd.cqe_size * entries,
+				      IB_ACCESS_LOCAL_WRITE);
 	if (IS_ERR(umem)) {
 		err = PTR_ERR(umem);
 		return err;
-- 
2.54.0

Re: [PATCH rdma-next v2 4/6] RDMA/uverbs: Add ioctl method for CQ resize

[Leon Romanovsky]

On Mon, Jun 15, 2026 at 10:50:38AM +0200, Jiri Pirko wrote:
> From: Jiri Pirko <jiri@nvidia.com>
> 
> Resize CQ is currently only reachable through the legacy write()
> uverbs command (IB_USER_VERBS_CMD_RESIZE_CQ). Add an equivalent modern
> ioctl method, UVERBS_METHOD_CQ_RESIZE, on the CQ object so the
> operation is available through the ioctl interface and can carry
> per-attribute extensions. The handler mirrors the legacy command: it
> looks up the CQ, calls resize_user_cq() and returns the new cqe count.
> The legacy write path is left in place for ABI compatibility.

I have a general question. Do we actually need CQ resizing, given that it is
rarely implemented and often incorrect in existing drivers? Maybe this is a
good time to consider deprecating that path.

Thanks