[PATCH rdma-next v2 0/8] RDMA/bnxt_re: Support QP uapi extensions

[PATCH rdma-next v2 0/8] RDMA/bnxt_re: Support QP uapi extensions

[Sriharsha Basavapatna]

Hi,

This patchset adds QP uapi extensions to the bnxt_re driver.
This is required by applications that need to manage some of the
RDMA HW resources directly and to implement the datapath in the
application.

This series supports application allocated memory for QPs.
The application takes into account SQ/RQ ring sizing constraints
(extra entries, rounding up etc) while allocating this memory.
The driver should avoid duplicating this logic while creating
these QPs.

The ring buffers provided by the application are pinned by the
uverbs layer before they are passed (as ib_umem) to the driver.
The uverbs changes are in review (see Changes below).

uAPI changes in this series:
- Patch#5: new uapi parameter 'sq_npsn' in bnxt_re_qp_req.
- Patch#7: new driver specific attribute 'DBR_HANDLE' for doorbell region.
- Patch#8: new comp_mask 'APP_ALLOCATED_QP_ENABLE' in bnxt_re_qp_req.

Patch#1 Refactor bnxt_re_init_user_qp()
Patch#2 Update rq depth for app allocated QPs
Patch#3 Update sq depth for app allocated QPs
Patch#4 Update umem for app allocated QPs
Patch#5 Update msn table size for app allocated QPs
Patch#6 Update hwq depth for app allocated QPs
Patch#7 Support doorbells for app allocated QPs
Patch#8 Enable app allocated QPs 

Thanks,
-Harsha

******

Changes:

v2:
- Rebased to umem_list uverbs patch series:
  https://patchwork.kernel.org/project/linux-rdma/cover/20260325150048.168341-1-jiri@resnulli.us/
- Deleted Patch#9; create_qp_umem devop is not supported.

v1: https://lore.kernel.org/linux-rdma/20260320135437.48716-1-sriharsha.basavapatna@broadcom.com/

******

Sriharsha Basavapatna (8):
  RDMA/bnxt_re: Refactor bnxt_re_init_user_qp()
  RDMA/bnxt_re: Update rq depth for app allocated QPs
  RDMA/bnxt_re: Update sq depth for app allocated QPs
  RDMA/bnxt_re: Update umem for app allocated QPs
  RDMA/bnxt_re: Update msn table size for app allocated QPs
  RDMA/bnxt_re: Update hwq depth for app allocated QPs
  RDMA/bnxt_re: Support doorbells for app allocated QPs
  RDMA/bnxt_re: Enable app allocated QPs

 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 292 ++++++++++++++---------
 drivers/infiniband/hw/bnxt_re/ib_verbs.h |   1 +
 drivers/infiniband/hw/bnxt_re/uapi.c     |  18 ++
 include/uapi/rdma/bnxt_re-abi.h          |   6 +
 4 files changed, 210 insertions(+), 107 deletions(-)

-- 
2.51.2.636.ga99f379adf

[PATCH rdma-next v2 1/8] RDMA/bnxt_re: Refactor bnxt_re_init_user_qp()

[Sriharsha Basavapatna]

The umem changes for CQ added a helper - bnxt_re_setup_sginfo().
Use the same helper for QP creation since we support only 4K
pages for QP ring memory too.

Add a new helper function bnxt_re_get_psn_bytes() to improve
readability as this code will be updated in subsequent patches.

Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 127 +++++++++++++----------
 1 file changed, 73 insertions(+), 54 deletions(-)

diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index e63780c78781..2c7b1cfb7b1e 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1136,34 +1136,64 @@ static int bnxt_re_setup_swqe_size(struct bnxt_re_qp *qp,
 	return 0;
 }
 
+static int bnxt_re_setup_sginfo(struct bnxt_re_dev *rdev,
+				struct ib_umem *umem,
+				struct bnxt_qplib_sg_info *sginfo)
+{
+	unsigned long page_size;
+
+	if (!umem)
+		return -EINVAL;
+
+	page_size = ib_umem_find_best_pgsz(umem, SZ_4K, 0);
+	if (!page_size || page_size != SZ_4K)
+		return -EINVAL;
+
+	sginfo->umem = umem;
+	sginfo->npages = ib_umem_num_dma_blocks(umem, page_size);
+	sginfo->pgsize = page_size;
+	sginfo->pgshft = __builtin_ctz(page_size);
+	return 0;
+}
+
+static int bnxt_re_get_psn_bytes(struct bnxt_re_dev *rdev,
+				 struct bnxt_re_ucontext *cntx,
+				 struct bnxt_qplib_qp *qplib_qp,
+				 struct bnxt_re_qp_req *ureq)
+{
+	int psn_sz, psn_nume;
+
+	psn_sz = bnxt_qplib_is_chip_gen_p5_p7(rdev->chip_ctx) ?
+				sizeof(struct sq_psn_search_ext) :
+				sizeof(struct sq_psn_search);
+	if (cntx && bnxt_re_is_var_size_supported(rdev, cntx)) {
+		psn_nume = ureq->sq_slots;
+	} else {
+		psn_nume = (qplib_qp->wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC) ?
+		qplib_qp->sq.max_wqe : ((qplib_qp->sq.max_wqe * qplib_qp->sq.wqe_size) /
+			 sizeof(struct bnxt_qplib_sge));
+	}
+	if (_is_host_msn_table(rdev->qplib_res.dattr->dev_cap_flags2))
+		psn_nume = roundup_pow_of_two(psn_nume);
+
+	return psn_nume * psn_sz;
+}
+
 static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
 				struct bnxt_re_qp *qp, struct bnxt_re_ucontext *cntx,
 				struct bnxt_re_qp_req *ureq)
 {
 	struct bnxt_qplib_qp *qplib_qp;
-	int bytes = 0, psn_sz;
 	struct ib_umem *umem;
-	int psn_nume;
+	int bytes;
+	int rc;
 
 	qplib_qp = &qp->qplib_qp;
 
 	bytes = (qplib_qp->sq.max_wqe * qplib_qp->sq.wqe_size);
 	/* Consider mapping PSN search memory only for RC QPs. */
-	if (qplib_qp->type == CMDQ_CREATE_QP_TYPE_RC) {
-		psn_sz = bnxt_qplib_is_chip_gen_p5_p7(rdev->chip_ctx) ?
-						   sizeof(struct sq_psn_search_ext) :
-						   sizeof(struct sq_psn_search);
-		if (cntx && bnxt_re_is_var_size_supported(rdev, cntx)) {
-			psn_nume = ureq->sq_slots;
-		} else {
-			psn_nume = (qplib_qp->wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC) ?
-			qplib_qp->sq.max_wqe : ((qplib_qp->sq.max_wqe * qplib_qp->sq.wqe_size) /
-				 sizeof(struct bnxt_qplib_sge));
-		}
-		if (_is_host_msn_table(rdev->qplib_res.dattr->dev_cap_flags2))
-			psn_nume = roundup_pow_of_two(psn_nume);
-		bytes += (psn_nume * psn_sz);
-	}
+	if (qplib_qp->type == CMDQ_CREATE_QP_TYPE_RC)
+		bytes += bnxt_re_get_psn_bytes(rdev, cntx, qplib_qp, ureq);
 
 	bytes = PAGE_ALIGN(bytes);
 	umem = ib_umem_get(&rdev->ibdev, ureq->qpsva, bytes,
@@ -1172,33 +1202,42 @@ static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
 		return PTR_ERR(umem);
 
 	qp->sumem = umem;
-	qplib_qp->sq.sg_info.umem = umem;
-	qplib_qp->sq.sg_info.pgsize = PAGE_SIZE;
-	qplib_qp->sq.sg_info.pgshft = PAGE_SHIFT;
-	qplib_qp->qp_handle = ureq->qp_handle;
+	rc = bnxt_re_setup_sginfo(rdev, qp->sumem, &qplib_qp->sq.sg_info);
+	if (rc)
+		goto fail;
+
+	if (qp->qplib_qp.srq)
+		goto done;
 
-	if (!qp->qplib_qp.srq) {
-		bytes = (qplib_qp->rq.max_wqe * qplib_qp->rq.wqe_size);
-		bytes = PAGE_ALIGN(bytes);
-		umem = ib_umem_get(&rdev->ibdev, ureq->qprva, bytes,
-				   IB_ACCESS_LOCAL_WRITE);
-		if (IS_ERR(umem))
-			goto rqfail;
-		qp->rumem = umem;
-		qplib_qp->rq.sg_info.umem = umem;
-		qplib_qp->rq.sg_info.pgsize = PAGE_SIZE;
-		qplib_qp->rq.sg_info.pgshft = PAGE_SHIFT;
+	bytes = (qplib_qp->rq.max_wqe * qplib_qp->rq.wqe_size);
+	bytes = PAGE_ALIGN(bytes);
+	umem = ib_umem_get(&rdev->ibdev, ureq->qprva, bytes,
+			   IB_ACCESS_LOCAL_WRITE);
+	if (IS_ERR(umem)) {
+		rc = PTR_ERR(umem);
+		goto fail;
 	}
 
+	qp->rumem = umem;
+	rc = bnxt_re_setup_sginfo(rdev, qp->rumem, &qplib_qp->rq.sg_info);
+	if (rc)
+		goto rqfail;
+
+done:
+	qplib_qp->qp_handle = ureq->qp_handle;
 	qplib_qp->dpi = &cntx->dpi;
 	qplib_qp->is_user = true;
 	return 0;
+
 rqfail:
+	ib_umem_release(qp->rumem);
+	qp->rumem = NULL;
+	memset(&qplib_qp->rq.sg_info, 0, sizeof(qplib_qp->rq.sg_info));
+fail:
 	ib_umem_release(qp->sumem);
 	qp->sumem = NULL;
 	memset(&qplib_qp->sq.sg_info, 0, sizeof(qplib_qp->sq.sg_info));
-
-	return PTR_ERR(umem);
+	return rc;
 }
 
 static struct bnxt_re_ah *bnxt_re_create_shadow_qp_ah
@@ -3345,26 +3384,6 @@ int bnxt_re_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata)
 	return ib_respond_empty_udata(udata);
 }
 
-static int bnxt_re_setup_sginfo(struct bnxt_re_dev *rdev,
-				struct ib_umem *umem,
-				struct bnxt_qplib_sg_info *sginfo)
-{
-	unsigned long page_size;
-
-	if (!umem)
-		return -EINVAL;
-
-	page_size = ib_umem_find_best_pgsz(umem, SZ_4K, 0);
-	if (!page_size || page_size != SZ_4K)
-		return -EINVAL;
-
-	sginfo->umem = umem;
-	sginfo->npages = ib_umem_num_dma_blocks(umem, page_size);
-	sginfo->pgsize = page_size;
-	sginfo->pgshft = __builtin_ctz(page_size);
-	return 0;
-}
-
 int bnxt_re_create_user_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 			   struct uverbs_attr_bundle *attrs)
 {
-- 
2.51.2.636.ga99f379adf

[PATCH rdma-next v2 2/8] RDMA/bnxt_re: Update rq depth for app allocated QPs

[Sriharsha Basavapatna]

For app allocated QPs, there's no need to add extra slots or
to round up the slot count. Use 'max_recv_wr' count provided
by the application as is.

Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 2c7b1cfb7b1e..61879222248d 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1475,7 +1475,9 @@ static struct bnxt_re_qp *bnxt_re_create_shadow_qp
 
 static int bnxt_re_init_rq_attr(struct bnxt_re_qp *qp,
 				struct ib_qp_init_attr *init_attr,
-				struct bnxt_re_ucontext *uctx)
+				struct bnxt_re_ucontext *uctx,
+				bool app_qp,
+				struct bnxt_re_qp_req *ureq)
 {
 	struct bnxt_qplib_dev_attr *dev_attr;
 	struct bnxt_qplib_qp *qplqp;
@@ -1500,12 +1502,16 @@ static int bnxt_re_init_rq_attr(struct bnxt_re_qp *qp,
 		init_attr->cap.max_recv_sge = rq->max_sge;
 		rq->wqe_size = bnxt_re_setup_rwqe_size(qplqp, rq->max_sge,
 						       dev_attr->max_qp_sges);
-		/* Allocate 1 more than what's provided so posting max doesn't
-		 * mean empty.
-		 */
-		rq->max_wqe = bnxt_re_init_depth(init_attr->cap.max_recv_wr + 1,
-						 dev_attr->max_qp_wqes + 1,
-						 uctx);
+		if (!app_qp) {
+			/* Allocate 1 more than what's provided so posting max doesn't
+			 * mean empty.
+			 */
+			rq->max_wqe = bnxt_re_init_depth(init_attr->cap.max_recv_wr + 1,
+							 dev_attr->max_qp_wqes + 1,
+							 uctx);
+		} else {
+			rq->max_wqe = init_attr->cap.max_recv_wr;
+		}
 		rq->max_sw_wqe = rq->max_wqe;
 		rq->q_full_delta = 0;
 		rq->sg_info.pgsize = PAGE_SIZE;
@@ -1678,6 +1684,7 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
 	struct bnxt_qplib_qp *qplqp;
 	struct bnxt_re_dev *rdev;
 	struct bnxt_re_cq *cq;
+	bool app_qp = false;
 	int rc = 0, qptype;
 
 	rdev = qp->rdev;
@@ -1724,7 +1731,7 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
 	}
 
 	/* Setup RQ/SRQ */
-	rc = bnxt_re_init_rq_attr(qp, init_attr, uctx);
+	rc = bnxt_re_init_rq_attr(qp, init_attr, uctx, app_qp, ureq);
 	if (rc)
 		return rc;
 	if (init_attr->qp_type == IB_QPT_GSI)
-- 
2.51.2.636.ga99f379adf

[PATCH rdma-next v2 3/8] RDMA/bnxt_re: Update sq depth for app allocated QPs

[Sriharsha Basavapatna]

For app allocated QPs, there's no need to reserve extra slots.
The application accounts for this while allocating the SQ.

Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 61879222248d..9a101f862c32 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1542,7 +1542,8 @@ static void bnxt_re_adjust_gsi_rq_attr(struct bnxt_re_qp *qp)
 static int bnxt_re_init_sq_attr(struct bnxt_re_qp *qp,
 				struct ib_qp_init_attr *init_attr,
 				struct bnxt_re_ucontext *uctx,
-				struct bnxt_re_qp_req *ureq)
+				struct bnxt_re_qp_req *ureq,
+				bool app_qp)
 {
 	struct bnxt_qplib_dev_attr *dev_attr;
 	struct bnxt_qplib_qp *qplqp;
@@ -1583,13 +1584,18 @@ static int bnxt_re_init_sq_attr(struct bnxt_re_qp *qp,
 			sq->max_sw_wqe = sq->max_wqe;
 
 	}
-	sq->q_full_delta = diff + 1;
-	/*
-	 * Reserving one slot for Phantom WQE. Application can
-	 * post one extra entry in this case. But allowing this to avoid
-	 * unexpected Queue full condition
-	 */
-	qplqp->sq.q_full_delta -= 1;
+	if (!app_qp) {
+		sq->q_full_delta = diff + 1;
+		/*
+		 * Reserving one slot for Phantom WQE. Application can
+		 * post one extra entry in this case. But allowing this to avoid
+		 * unexpected Queue full condition
+		 */
+		qplqp->sq.q_full_delta -= 1;
+	} else {
+		sq->q_full_delta = 0;
+	}
+
 	qplqp->sq.sg_info.pgsize = PAGE_SIZE;
 	qplqp->sq.sg_info.pgshft = PAGE_SHIFT;
 
@@ -1738,7 +1744,7 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
 		bnxt_re_adjust_gsi_rq_attr(qp);
 
 	/* Setup SQ */
-	rc = bnxt_re_init_sq_attr(qp, init_attr, uctx, ureq);
+	rc = bnxt_re_init_sq_attr(qp, init_attr, uctx, ureq, app_qp);
 	if (rc)
 		return rc;
 	if (init_attr->qp_type == IB_QPT_GSI)
-- 
2.51.2.636.ga99f379adf

[PATCH rdma-next v2 4/8] RDMA/bnxt_re: Update umem for app allocated QPs

[Sriharsha Basavapatna]

For app allocated QPs, use sq_umem and rq_umem provided by the
application via uverbs layer. The uverbs layer maps these app
provided buffers before calling create_qp(). Use the new API
ib_umem_list_load_or_get(). The uverbs layer frees these buffers
if there's any error during QP creation or when the QP gets
destroyed. The driver must not free these umems, so delete the
driver function bnxt_re_qp_free_umem().

Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 45 ++++++++----------------
 1 file changed, 15 insertions(+), 30 deletions(-)

diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 9a101f862c32..b249bc2d2583 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -995,12 +995,6 @@ static void bnxt_re_del_unique_gid(struct bnxt_re_dev *rdev)
 		dev_err(rdev_to_dev(rdev), "Failed to delete unique GID, rc: %d\n", rc);
 }
 
-static void bnxt_re_qp_free_umem(struct bnxt_re_qp *qp)
-{
-	ib_umem_release(qp->rumem);
-	ib_umem_release(qp->sumem);
-}
-
 /* Queue Pairs */
 int bnxt_re_destroy_qp(struct ib_qp *ib_qp, struct ib_udata *udata)
 {
@@ -1047,8 +1041,6 @@ int bnxt_re_destroy_qp(struct ib_qp *ib_qp, struct ib_udata *udata)
 	if (qp->qplib_qp.type == CMDQ_CREATE_QP_TYPE_RAW_ETHERTYPE)
 		bnxt_re_del_unique_gid(rdev);
 
-	bnxt_re_qp_free_umem(qp);
-
 	/* Flush all the entries of notification queue associated with
 	 * given qp.
 	 */
@@ -1196,47 +1188,44 @@ static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
 		bytes += bnxt_re_get_psn_bytes(rdev, cntx, qplib_qp, ureq);
 
 	bytes = PAGE_ALIGN(bytes);
-	umem = ib_umem_get(&rdev->ibdev, ureq->qpsva, bytes,
-			   IB_ACCESS_LOCAL_WRITE);
+	umem = ib_umem_list_load_or_get(qp->ib_qp.umem_list,
+					UVERBS_BUF_QP_SQ_BUF,
+					&rdev->ibdev, ureq->qpsva,
+					bytes, IB_ACCESS_LOCAL_WRITE);
 	if (IS_ERR(umem))
 		return PTR_ERR(umem);
 
-	qp->sumem = umem;
-	rc = bnxt_re_setup_sginfo(rdev, qp->sumem, &qplib_qp->sq.sg_info);
+	rc = bnxt_re_setup_sginfo(rdev, umem, &qplib_qp->sq.sg_info);
 	if (rc)
-		goto fail;
+		return rc;
+	qp->sumem = umem;
 
 	if (qp->qplib_qp.srq)
 		goto done;
 
 	bytes = (qplib_qp->rq.max_wqe * qplib_qp->rq.wqe_size);
 	bytes = PAGE_ALIGN(bytes);
-	umem = ib_umem_get(&rdev->ibdev, ureq->qprva, bytes,
-			   IB_ACCESS_LOCAL_WRITE);
+	umem = ib_umem_list_load_or_get(qp->ib_qp.umem_list,
+					UVERBS_BUF_QP_RQ_BUF,
+					&rdev->ibdev, ureq->qprva,
+					bytes, IB_ACCESS_LOCAL_WRITE);
 	if (IS_ERR(umem)) {
 		rc = PTR_ERR(umem);
 		goto fail;
 	}
 
-	qp->rumem = umem;
-	rc = bnxt_re_setup_sginfo(rdev, qp->rumem, &qplib_qp->rq.sg_info);
+	rc = bnxt_re_setup_sginfo(rdev, umem, &qplib_qp->rq.sg_info);
 	if (rc)
-		goto rqfail;
+		goto fail;
+	qp->rumem = umem;
 
 done:
 	qplib_qp->qp_handle = ureq->qp_handle;
 	qplib_qp->dpi = &cntx->dpi;
 	qplib_qp->is_user = true;
 	return 0;
-
-rqfail:
-	ib_umem_release(qp->rumem);
-	qp->rumem = NULL;
-	memset(&qplib_qp->rq.sg_info, 0, sizeof(qplib_qp->rq.sg_info));
 fail:
-	ib_umem_release(qp->sumem);
 	qp->sumem = NULL;
-	memset(&qplib_qp->sq.sg_info, 0, sizeof(qplib_qp->sq.sg_info));
 	return rc;
 }
 
@@ -1760,12 +1749,9 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
 
 	rc = bnxt_re_setup_qp_hwqs(qp);
 	if (rc)
-		goto free_umem;
+		return rc;
 
 	return 0;
-free_umem:
-	bnxt_re_qp_free_umem(qp);
-	return rc;
 }
 
 static int bnxt_re_create_shadow_gsi(struct bnxt_re_qp *qp,
@@ -1985,7 +1971,6 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
 	bnxt_qplib_destroy_qp(&rdev->qplib_res, &qp->qplib_qp);
 free_hwq:
 	bnxt_qplib_free_qp_res(&rdev->qplib_res, &qp->qplib_qp);
-	bnxt_re_qp_free_umem(qp);
 fail:
 	return rc;
 }
-- 
2.51.2.636.ga99f379adf

[PATCH rdma-next v2 5/8] RDMA/bnxt_re: Update msn table size for app allocated QPs

[Sriharsha Basavapatna]

For app allocated QPs, the driver shouldn't use slots/round-up logic
to compute the msn table size. The application handles this logic
and computes 'sq_npsn' and passes it to the driver using a new uapi
parameter.

Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 61 +++++++++++++++---------
 include/uapi/rdma/bnxt_re-abi.h          |  1 +
 2 files changed, 40 insertions(+), 22 deletions(-)

diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index b249bc2d2583..b6aed318623d 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1151,29 +1151,39 @@ static int bnxt_re_setup_sginfo(struct bnxt_re_dev *rdev,
 static int bnxt_re_get_psn_bytes(struct bnxt_re_dev *rdev,
 				 struct bnxt_re_ucontext *cntx,
 				 struct bnxt_qplib_qp *qplib_qp,
-				 struct bnxt_re_qp_req *ureq)
+				 struct bnxt_re_qp_req *ureq,
+				 bool app_qp)
 {
 	int psn_sz, psn_nume;
 
-	psn_sz = bnxt_qplib_is_chip_gen_p5_p7(rdev->chip_ctx) ?
-				sizeof(struct sq_psn_search_ext) :
-				sizeof(struct sq_psn_search);
-	if (cntx && bnxt_re_is_var_size_supported(rdev, cntx)) {
-		psn_nume = ureq->sq_slots;
+	if (rdev->dev_attr &&
+	    _is_host_msn_table(rdev->dev_attr->dev_cap_flags2))
+		psn_sz = sizeof(struct sq_msn_search);
+	else
+		psn_sz = bnxt_qplib_is_chip_gen_p5_p7(rdev->chip_ctx) ?
+					sizeof(struct sq_psn_search_ext) :
+					sizeof(struct sq_psn_search);
+	if (!app_qp) {
+		if (cntx && bnxt_re_is_var_size_supported(rdev, cntx)) {
+			psn_nume = ureq->sq_slots;
+		} else {
+			psn_nume = (qplib_qp->wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC) ?
+			qplib_qp->sq.max_wqe : ((qplib_qp->sq.max_wqe * qplib_qp->sq.wqe_size) /
+				 sizeof(struct bnxt_qplib_sge));
+		}
+		if (_is_host_msn_table(rdev->qplib_res.dattr->dev_cap_flags2))
+			psn_nume = roundup_pow_of_two(psn_nume);
 	} else {
-		psn_nume = (qplib_qp->wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC) ?
-		qplib_qp->sq.max_wqe : ((qplib_qp->sq.max_wqe * qplib_qp->sq.wqe_size) /
-			 sizeof(struct bnxt_qplib_sge));
+		psn_nume = ureq->sq_npsn;
 	}
-	if (_is_host_msn_table(rdev->qplib_res.dattr->dev_cap_flags2))
-		psn_nume = roundup_pow_of_two(psn_nume);
 
 	return psn_nume * psn_sz;
 }
 
 static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
 				struct bnxt_re_qp *qp, struct bnxt_re_ucontext *cntx,
-				struct bnxt_re_qp_req *ureq)
+				struct bnxt_re_qp_req *ureq,
+				bool app_qp)
 {
 	struct bnxt_qplib_qp *qplib_qp;
 	struct ib_umem *umem;
@@ -1185,7 +1195,7 @@ static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
 	bytes = (qplib_qp->sq.max_wqe * qplib_qp->sq.wqe_size);
 	/* Consider mapping PSN search memory only for RC QPs. */
 	if (qplib_qp->type == CMDQ_CREATE_QP_TYPE_RC)
-		bytes += bnxt_re_get_psn_bytes(rdev, cntx, qplib_qp, ureq);
+		bytes += bnxt_re_get_psn_bytes(rdev, cntx, qplib_qp, ureq, app_qp);
 
 	bytes = PAGE_ALIGN(bytes);
 	umem = ib_umem_list_load_or_get(qp->ib_qp.umem_list,
@@ -1637,7 +1647,9 @@ static int bnxt_re_init_qp_type(struct bnxt_re_dev *rdev,
 	return qptype;
 }
 
-static void bnxt_re_qp_calculate_msn_psn_size(struct bnxt_re_qp *qp)
+static void bnxt_re_qp_calculate_msn_psn_size(struct bnxt_re_qp *qp,
+					      bool app_qp,
+					      struct bnxt_re_qp_req *req)
 {
 	struct bnxt_qplib_qp *qplib_qp = &qp->qplib_qp;
 	struct bnxt_qplib_q *sq = &qplib_qp->sq;
@@ -1660,12 +1672,17 @@ static void bnxt_re_qp_calculate_msn_psn_size(struct bnxt_re_qp *qp)
 
 	/* Update msn tbl size */
 	if (qplib_qp->is_host_msn_tbl && qplib_qp->psn_sz) {
-		if (wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC)
-			qplib_qp->msn_tbl_sz =
-				roundup_pow_of_two(bnxt_qplib_set_sq_size(sq, wqe_mode));
-		else
-			qplib_qp->msn_tbl_sz =
-				roundup_pow_of_two(bnxt_qplib_set_sq_size(sq, wqe_mode)) / 2;
+		if (!app_qp) {
+			if (wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC)
+				qplib_qp->msn_tbl_sz =
+					roundup_pow_of_two(bnxt_qplib_set_sq_size(sq, wqe_mode));
+			else
+				qplib_qp->msn_tbl_sz =
+					roundup_pow_of_two(bnxt_qplib_set_sq_size(sq, wqe_mode))
+						/ 2;
+		} else {
+			qplib_qp->msn_tbl_sz = req->sq_npsn / 2; /* WQE_MODE_VARIABLE */
+		}
 		qplib_qp->msn = 0;
 	}
 }
@@ -1740,12 +1757,12 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
 		bnxt_re_adjust_gsi_sq_attr(qp, init_attr, uctx);
 
 	if (uctx) { /* This will update DPI and qp_handle */
-		rc = bnxt_re_init_user_qp(rdev, pd, qp, uctx, ureq);
+		rc = bnxt_re_init_user_qp(rdev, pd, qp, uctx, ureq, app_qp);
 		if (rc)
 			return rc;
 	}
 
-	bnxt_re_qp_calculate_msn_psn_size(qp);
+	bnxt_re_qp_calculate_msn_psn_size(qp, app_qp, ureq);
 
 	rc = bnxt_re_setup_qp_hwqs(qp);
 	if (rc)
diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h
index 40955eaba32e..db8400f2ce3b 100644
--- a/include/uapi/rdma/bnxt_re-abi.h
+++ b/include/uapi/rdma/bnxt_re-abi.h
@@ -135,6 +135,7 @@ struct bnxt_re_qp_req {
 	__aligned_u64 qp_handle;
 	__aligned_u64 comp_mask;
 	__u32 sq_slots;
+	__u32 sq_npsn;
 };
 
 struct bnxt_re_qp_resp {
-- 
2.51.2.636.ga99f379adf

[PATCH rdma-next v2 6/8] RDMA/bnxt_re: Update hwq depth for app allocated QPs

[Sriharsha Basavapatna]

The hwq depth shouldn't be computed using slots/round-up logic for
app allocated QPs, use the max_wqe value saved earlier.

Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 28 +++++++++++++++++-------
 1 file changed, 20 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index b6aed318623d..0e865cba2c45 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1335,7 +1335,7 @@ static int bnxt_re_qp_alloc_init_xrrq(struct bnxt_re_qp *qp)
 	return rc;
 }
 
-static int bnxt_re_setup_qp_hwqs(struct bnxt_re_qp *qp)
+static int bnxt_re_setup_qp_hwqs(struct bnxt_re_qp *qp, bool app_qp)
 {
 	struct bnxt_qplib_res *res = &qp->rdev->qplib_res;
 	struct bnxt_qplib_qp *qplib_qp = &qp->qplib_qp;
@@ -1349,12 +1349,17 @@ static int bnxt_re_setup_qp_hwqs(struct bnxt_re_qp *qp)
 	hwq_attr.res = res;
 	hwq_attr.sginfo = &sq->sg_info;
 	hwq_attr.stride = bnxt_qplib_get_stride();
-	hwq_attr.depth = bnxt_qplib_get_depth(sq, wqe_mode, true);
 	hwq_attr.aux_stride = qplib_qp->psn_sz;
-	hwq_attr.aux_depth = (qplib_qp->psn_sz) ?
-		bnxt_qplib_set_sq_size(sq, wqe_mode) : 0;
-	if (qplib_qp->is_host_msn_tbl && qplib_qp->psn_sz)
+	if (!app_qp) {
+		hwq_attr.depth = bnxt_qplib_get_depth(sq, wqe_mode, true);
+		hwq_attr.aux_depth = (qplib_qp->psn_sz) ?
+				bnxt_qplib_set_sq_size(sq, wqe_mode) : 0;
+		if (qplib_qp->is_host_msn_tbl && qplib_qp->psn_sz)
+			hwq_attr.aux_depth = qplib_qp->msn_tbl_sz;
+	} else {
+		hwq_attr.depth = sq->max_wqe;
 		hwq_attr.aux_depth = qplib_qp->msn_tbl_sz;
+	}
 	hwq_attr.type = HWQ_TYPE_QUEUE;
 	rc = bnxt_qplib_alloc_init_hwq(&sq->hwq, &hwq_attr);
 	if (rc)
@@ -1365,10 +1370,16 @@ static int bnxt_re_setup_qp_hwqs(struct bnxt_re_qp *qp)
 		      CMDQ_CREATE_QP_SQ_LVL_SFT);
 	sq->hwq.pg_sz_lvl = pg_sz_lvl;
 
+	if (qplib_qp->srq)
+		goto done;
+
 	hwq_attr.res = res;
 	hwq_attr.sginfo = &rq->sg_info;
 	hwq_attr.stride = bnxt_qplib_get_stride();
-	hwq_attr.depth = bnxt_qplib_get_depth(rq, qplib_qp->wqe_mode, false);
+	if (!app_qp)
+		hwq_attr.depth = bnxt_qplib_get_depth(rq, qplib_qp->wqe_mode, false);
+	else
+		hwq_attr.depth = (rq->max_wqe * rq->wqe_size) / hwq_attr.stride;
 	hwq_attr.aux_stride = 0;
 	hwq_attr.aux_depth = 0;
 	hwq_attr.type = HWQ_TYPE_QUEUE;
@@ -1381,6 +1392,7 @@ static int bnxt_re_setup_qp_hwqs(struct bnxt_re_qp *qp)
 		      CMDQ_CREATE_QP_RQ_LVL_SFT);
 	rq->hwq.pg_sz_lvl = pg_sz_lvl;
 
+done:
 	if (qplib_qp->psn_sz) {
 		rc = bnxt_re_qp_alloc_init_xrrq(qp);
 		if (rc)
@@ -1449,7 +1461,7 @@ static struct bnxt_re_qp *bnxt_re_create_shadow_qp
 	qp->qplib_qp.rq_hdr_buf_size = BNXT_QPLIB_MAX_GRH_HDR_SIZE_IPV6;
 	qp->qplib_qp.dpi = &rdev->dpi_privileged;
 
-	rc = bnxt_re_setup_qp_hwqs(qp);
+	rc = bnxt_re_setup_qp_hwqs(qp, false);
 	if (rc)
 		goto fail;
 
@@ -1764,7 +1776,7 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
 
 	bnxt_re_qp_calculate_msn_psn_size(qp, app_qp, ureq);
 
-	rc = bnxt_re_setup_qp_hwqs(qp);
+	rc = bnxt_re_setup_qp_hwqs(qp, app_qp);
 	if (rc)
 		return rc;
 
-- 
2.51.2.636.ga99f379adf

[PATCH rdma-next v2 7/8] RDMA/bnxt_re: Support doorbells for app allocated QPs

[Sriharsha Basavapatna]

App allocated QPs can use a separate doorbell for each QP.
This doorbell region can be passed through a new driver specific
DBR_HANDLE attribute, during QP creation. When this attribute
is set, associate the QP with the given doorbell region.

While the QP holds a reference to the dbr, the dbr itself
cannot be destroyed and is rejected with EBUSY error.

Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 35 ++++++++++++++++++++----
 drivers/infiniband/hw/bnxt_re/ib_verbs.h |  1 +
 drivers/infiniband/hw/bnxt_re/uapi.c     | 18 ++++++++++++
 include/uapi/rdma/bnxt_re-abi.h          |  4 +++
 4 files changed, 53 insertions(+), 5 deletions(-)

diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 0e865cba2c45..7084f49ec28f 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1010,6 +1010,9 @@ int bnxt_re_destroy_qp(struct ib_qp *ib_qp, struct ib_udata *udata)
 	if (rc)
 		return rc;
 
+	if (qp->dbr_obj)
+		atomic_dec(&qp->dbr_obj->usecnt);
+
 	bnxt_re_debug_rem_qpinfo(rdev, qp);
 
 	bnxt_qplib_flush_cqn_wq(&qp->qplib_qp);
@@ -1183,7 +1186,8 @@ static int bnxt_re_get_psn_bytes(struct bnxt_re_dev *rdev,
 static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
 				struct bnxt_re_qp *qp, struct bnxt_re_ucontext *cntx,
 				struct bnxt_re_qp_req *ureq,
-				bool app_qp)
+				bool app_qp,
+				struct bnxt_re_dbr_obj *dbr_obj)
 {
 	struct bnxt_qplib_qp *qplib_qp;
 	struct ib_umem *umem;
@@ -1230,8 +1234,11 @@ static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
 	qp->rumem = umem;
 
 done:
+	if (dbr_obj)
+		qplib_qp->dpi = &dbr_obj->dpi;
+	else
+		qplib_qp->dpi = &cntx->dpi;
 	qplib_qp->qp_handle = ureq->qp_handle;
-	qplib_qp->dpi = &cntx->dpi;
 	qplib_qp->is_user = true;
 	return 0;
 fail:
@@ -1702,7 +1709,8 @@ static void bnxt_re_qp_calculate_msn_psn_size(struct bnxt_re_qp *qp,
 static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
 				struct ib_qp_init_attr *init_attr,
 				struct bnxt_re_ucontext *uctx,
-				struct bnxt_re_qp_req *ureq)
+				struct bnxt_re_qp_req *ureq,
+				struct bnxt_re_dbr_obj *dbr_obj)
 {
 	struct bnxt_qplib_dev_attr *dev_attr;
 	struct bnxt_qplib_qp *qplqp;
@@ -1769,7 +1777,8 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
 		bnxt_re_adjust_gsi_sq_attr(qp, init_attr, uctx);
 
 	if (uctx) { /* This will update DPI and qp_handle */
-		rc = bnxt_re_init_user_qp(rdev, pd, qp, uctx, ureq, app_qp);
+		rc = bnxt_re_init_user_qp(rdev, pd, qp, uctx, ureq, app_qp,
+					  dbr_obj);
 		if (rc)
 			return rc;
 	}
@@ -1902,7 +1911,9 @@ static int bnxt_re_add_unique_gid(struct bnxt_re_dev *rdev)
 int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
 		      struct ib_udata *udata)
 {
+	struct bnxt_re_dbr_obj *dbr_obj = NULL;
 	struct bnxt_qplib_dev_attr *dev_attr;
+	struct uverbs_attr_bundle *attrs;
 	struct bnxt_re_ucontext *uctx;
 	struct bnxt_re_qp_req ureq;
 	struct bnxt_re_dev *rdev;
@@ -1923,6 +1934,17 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
 		rc = ib_copy_validate_udata_in_cm(udata, ureq, qp_handle, 0);
 		if (rc)
 			return rc;
+
+		attrs = rdma_udata_to_uverbs_attr_bundle(udata);
+		if (uverbs_attr_is_valid(attrs,
+					 BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE)) {
+			dbr_obj = uverbs_attr_get_obj(attrs,
+						      BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE);
+			if (IS_ERR(dbr_obj))
+				return PTR_ERR(dbr_obj);
+			atomic_inc(&dbr_obj->usecnt);
+			qp->dbr_obj = dbr_obj;
+		}
 	}
 
 	rc = bnxt_re_test_qp_limits(rdev, qp_init_attr, dev_attr);
@@ -1932,7 +1954,8 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
 	}
 
 	qp->rdev = rdev;
-	rc = bnxt_re_init_qp_attr(qp, pd, qp_init_attr, uctx, &ureq);
+	rc = bnxt_re_init_qp_attr(qp, pd, qp_init_attr, uctx, &ureq,
+				  dbr_obj);
 	if (rc)
 		goto fail;
 
@@ -2001,6 +2024,8 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
 free_hwq:
 	bnxt_qplib_free_qp_res(&rdev->qplib_res, &qp->qplib_qp);
 fail:
+	if (dbr_obj)
+		atomic_dec(&dbr_obj->usecnt);
 	return rc;
 }
 
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
index 08f71a94d55d..fdf3dd4432c1 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
@@ -96,6 +96,7 @@ struct bnxt_re_qp {
 	struct bnxt_re_cq	*scq;
 	struct bnxt_re_cq	*rcq;
 	struct dentry		*dentry;
+	struct bnxt_re_dbr_obj *dbr_obj; /* doorbell region */
 };
 
 struct bnxt_re_cq {
diff --git a/drivers/infiniband/hw/bnxt_re/uapi.c b/drivers/infiniband/hw/bnxt_re/uapi.c
index 3eaee7101615..83e5ddf6ecf6 100644
--- a/drivers/infiniband/hw/bnxt_re/uapi.c
+++ b/drivers/infiniband/hw/bnxt_re/uapi.c
@@ -398,6 +398,9 @@ static int bnxt_re_dbr_cleanup(struct ib_uobject *uobject,
 	struct bnxt_re_dbr_obj *obj = uobject->object;
 	struct bnxt_re_dev *rdev = obj->rdev;
 
+	if (atomic_read(&obj->usecnt))
+		return -EBUSY;
+
 	rdma_user_mmap_entry_remove(&obj->entry->rdma_entry);
 	bnxt_qplib_free_uc_dpi(&rdev->qplib_res, &obj->dpi);
 	return 0;
@@ -459,11 +462,26 @@ DECLARE_UVERBS_NAMED_METHOD(BNXT_RE_METHOD_GET_DEFAULT_DBR,
 DECLARE_UVERBS_GLOBAL_METHODS(BNXT_RE_OBJECT_DEFAULT_DBR,
 			      &UVERBS_METHOD(BNXT_RE_METHOD_GET_DEFAULT_DBR));
 
+ADD_UVERBS_ATTRIBUTES_SIMPLE(
+	bnxt_re_qp_create,
+	UVERBS_OBJECT_QP,
+	UVERBS_METHOD_QP_CREATE,
+	UVERBS_ATTR_IDR(BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE,
+			BNXT_RE_OBJECT_DBR,
+			UVERBS_ACCESS_READ,
+			UA_OPTIONAL));
+
+const struct uapi_definition bnxt_re_create_qp_defs[] = {
+	UAPI_DEF_CHAIN_OBJ_TREE(UVERBS_OBJECT_QP, &bnxt_re_qp_create),
+	{},
+};
+
 const struct uapi_definition bnxt_re_uapi_defs[] = {
 	UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_ALLOC_PAGE),
 	UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_NOTIFY_DRV),
 	UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_GET_TOGGLE_MEM),
 	UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_DBR),
 	UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_DEFAULT_DBR),
+	UAPI_DEF_CHAIN(bnxt_re_create_qp_defs),
 	{}
 };
diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h
index db8400f2ce3b..4da8cda337dc 100644
--- a/include/uapi/rdma/bnxt_re-abi.h
+++ b/include/uapi/rdma/bnxt_re-abi.h
@@ -138,6 +138,10 @@ struct bnxt_re_qp_req {
 	__u32 sq_npsn;
 };
 
+enum bnxt_re_create_qp_attrs {
+	BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE = UVERBS_ID_DRIVER_NS_WITH_UHW,
+};
+
 struct bnxt_re_qp_resp {
 	__u32 qpid;
 	__u32 rsvd;
-- 
2.51.2.636.ga99f379adf

[PATCH rdma-next v2 8/8] RDMA/bnxt_re: Enable app allocated QPs

[Sriharsha Basavapatna]

The driver supports a new comp_mask: APP_ALLOCATED_QP_ENABLE.
The application sets this comp_mask bit in the CREATE_QP ureq
to indicate direct control of the QP. The driver goes through
the required processing for app allocated QPs. Only variable
WQE mode is supported for these QPs.

Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 33 ++++++++++++++----------
 include/uapi/rdma/bnxt_re-abi.h          |  1 +
 2 files changed, 21 insertions(+), 13 deletions(-)

diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 7084f49ec28f..35e19157b061 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1710,13 +1710,13 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
 				struct ib_qp_init_attr *init_attr,
 				struct bnxt_re_ucontext *uctx,
 				struct bnxt_re_qp_req *ureq,
-				struct bnxt_re_dbr_obj *dbr_obj)
+				struct bnxt_re_dbr_obj *dbr_obj,
+				bool app_qp)
 {
 	struct bnxt_qplib_dev_attr *dev_attr;
 	struct bnxt_qplib_qp *qplqp;
 	struct bnxt_re_dev *rdev;
 	struct bnxt_re_cq *cq;
-	bool app_qp = false;
 	int rc = 0, qptype;
 
 	rdev = qp->rdev;
@@ -1734,6 +1734,8 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
 		return qptype;
 	qplqp->type = (u8)qptype;
 	qplqp->wqe_mode = bnxt_re_is_var_size_supported(rdev, uctx);
+	if (app_qp && qplqp->wqe_mode != BNXT_QPLIB_WQE_MODE_VARIABLE)
+		return -EOPNOTSUPP;
 	qplqp->dev_cap_flags = dev_attr->dev_cap_flags;
 	qplqp->cctx = rdev->chip_ctx;
 	if (init_attr->qp_type == IB_QPT_RC) {
@@ -1920,6 +1922,7 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
 	struct bnxt_re_pd *pd;
 	struct bnxt_re_qp *qp;
 	struct ib_pd *ib_pd;
+	bool app_qp = false;
 	u32 active_qps;
 	int rc;
 
@@ -1931,19 +1934,23 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
 
 	uctx = rdma_udata_to_drv_context(udata, struct bnxt_re_ucontext, ib_uctx);
 	if (udata) {
-		rc = ib_copy_validate_udata_in_cm(udata, ureq, qp_handle, 0);
+		rc = ib_copy_validate_udata_in_cm(udata, ureq, qp_handle,
+						  BNXT_RE_QP_REQ_MASK_APP_ALLOCATED_QP_ENABLE);
 		if (rc)
 			return rc;
 
-		attrs = rdma_udata_to_uverbs_attr_bundle(udata);
-		if (uverbs_attr_is_valid(attrs,
-					 BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE)) {
-			dbr_obj = uverbs_attr_get_obj(attrs,
-						      BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE);
-			if (IS_ERR(dbr_obj))
-				return PTR_ERR(dbr_obj);
-			atomic_inc(&dbr_obj->usecnt);
-			qp->dbr_obj = dbr_obj;
+		if (ureq.comp_mask & BNXT_RE_QP_REQ_MASK_APP_ALLOCATED_QP_ENABLE) {
+			attrs = rdma_udata_to_uverbs_attr_bundle(udata);
+			if (uverbs_attr_is_valid(attrs,
+						 BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE)) {
+				dbr_obj = uverbs_attr_get_obj(attrs,
+							      BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE);
+				if (IS_ERR(dbr_obj))
+					return PTR_ERR(dbr_obj);
+				atomic_inc(&dbr_obj->usecnt);
+				qp->dbr_obj = dbr_obj;
+			}
+			app_qp = true;
 		}
 	}
 
@@ -1955,7 +1962,7 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
 
 	qp->rdev = rdev;
 	rc = bnxt_re_init_qp_attr(qp, pd, qp_init_attr, uctx, &ureq,
-				  dbr_obj);
+				  dbr_obj, app_qp);
 	if (rc)
 		goto fail;
 
diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h
index 4da8cda337dc..edb0329b700e 100644
--- a/include/uapi/rdma/bnxt_re-abi.h
+++ b/include/uapi/rdma/bnxt_re-abi.h
@@ -127,6 +127,7 @@ struct bnxt_re_resize_cq_req {
 
 enum bnxt_re_qp_mask {
 	BNXT_RE_QP_REQ_MASK_VAR_WQE_SQ_SLOTS = 0x1,
+	BNXT_RE_QP_REQ_MASK_APP_ALLOCATED_QP_ENABLE = 0x2,
 };
 
 struct bnxt_re_qp_req {
-- 
2.51.2.636.ga99f379adf

Re: [PATCH rdma-next v2 0/8] RDMA/bnxt_re: Support QP uapi extensions

[Jason Gunthorpe]

On Fri, Mar 27, 2026 at 02:47:47PM +0530, Sriharsha Basavapatna wrote:
> Sriharsha Basavapatna (8):
>   RDMA/bnxt_re: Refactor bnxt_re_init_user_qp()
>   RDMA/bnxt_re: Update rq depth for app allocated QPs
>   RDMA/bnxt_re: Update sq depth for app allocated QPs
>   RDMA/bnxt_re: Update umem for app allocated QPs
>   RDMA/bnxt_re: Update msn table size for app allocated QPs
>   RDMA/bnxt_re: Update hwq depth for app allocated QPs
>   RDMA/bnxt_re: Support doorbells for app allocated QPs
>   RDMA/bnxt_re: Enable app allocated QPs

Can you split this into two halfs, one just QP stuff and the other
allowing QPs to use different umems? I think we can merge the first
part more quickly

Jason

Re: [PATCH rdma-next v2 8/8] RDMA/bnxt_re: Enable app allocated QPs

[Jason Gunthorpe]

On Fri, Mar 27, 2026 at 02:47:55PM +0530, Sriharsha Basavapatna wrote:
> The driver supports a new comp_mask: APP_ALLOCATED_QP_ENABLE.
> The application sets this comp_mask bit in the CREATE_QP ureq
> to indicate direct control of the QP. The driver goes through
> the required processing for app allocated QPs. Only variable
> WQE mode is supported for these QPs.

I thought we talked about this, no weird names like this.

> @@ -1734,6 +1734,8 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
>  		return qptype;
>  	qplqp->type = (u8)qptype;
>  	qplqp->wqe_mode = bnxt_re_is_var_size_supported(rdev, uctx);
> +	if (app_qp && qplqp->wqe_mode != BNXT_QPLIB_WQE_MODE_VARIABLE)
> +		return -EOPNOTSUPP;

Give a sensible name for whatever this is and use it only for this.

I kind of thinking you can just fully drop it? What is the point in
checking userspace set VARIABLE? It isn't signalling HW and it isn't
protecting anything.

> -		attrs = rdma_udata_to_uverbs_attr_bundle(udata);
> -		if (uverbs_attr_is_valid(attrs,
> -					 BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE)) {
> -			dbr_obj = uverbs_attr_get_obj(attrs,
> -						      BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE);
> -			if (IS_ERR(dbr_obj))
> -				return PTR_ERR(dbr_obj);
> -			atomic_inc(&dbr_obj->usecnt);
> -			qp->dbr_obj = dbr_obj;
> +		if (ureq.comp_mask & BNXT_RE_QP_REQ_MASK_APP_ALLOCATED_QP_ENABLE) {
> +			attrs = rdma_udata_to_uverbs_attr_bundle(udata);

And don't do this, BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE is perfectly fine
to be signaled by attribute and it looks like it is fully orthogonal
to the other features.

Jason

Re: [PATCH rdma-next v2 0/8] RDMA/bnxt_re: Support QP uapi extensions

[Sriharsha Basavapatna]

[-- Attachment #1: Type: text/plain, Size: 1003 bytes --]

On Fri, Apr 10, 2026 at 8:55 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Fri, Mar 27, 2026 at 02:47:47PM +0530, Sriharsha Basavapatna wrote:
> > Sriharsha Basavapatna (8):
> >   RDMA/bnxt_re: Refactor bnxt_re_init_user_qp()
> >   RDMA/bnxt_re: Update rq depth for app allocated QPs
> >   RDMA/bnxt_re: Update sq depth for app allocated QPs
> >   RDMA/bnxt_re: Update umem for app allocated QPs
> >   RDMA/bnxt_re: Update msn table size for app allocated QPs
> >   RDMA/bnxt_re: Update hwq depth for app allocated QPs
> >   RDMA/bnxt_re: Support doorbells for app allocated QPs
> >   RDMA/bnxt_re: Enable app allocated QPs
>
> Can you split this into two halfs, one just QP stuff and the other
> allowing QPs to use different umems? I think we can merge the first
> part more quickly
>
> Jason
I have split this into a series with just QP changes (dropped the
'Update umem' patch above) and I can post the umem patch when the
kernel uverbs changes are ready.
Thanks,
-Harsha

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5505 bytes --]

Re: [PATCH rdma-next v2 8/8] RDMA/bnxt_re: Enable app allocated QPs

[Sriharsha Basavapatna]

[-- Attachment #1: Type: text/plain, Size: 2727 bytes --]

On Fri, Apr 10, 2026 at 8:57 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Fri, Mar 27, 2026 at 02:47:55PM +0530, Sriharsha Basavapatna wrote:
> > The driver supports a new comp_mask: APP_ALLOCATED_QP_ENABLE.
> > The application sets this comp_mask bit in the CREATE_QP ureq
> > to indicate direct control of the QP. The driver goes through
> > the required processing for app allocated QPs. Only variable
> > WQE mode is supported for these QPs.
>
> I thought we talked about this, no weird names like this.
Are you talking about the comp_mask name or the wqe_mode (VARIABLE)
name? We used the comp_mask name (APP_ALLOCATED_QP) because the
application allocates (and owns/manages) this QP memory instead of the
library.
If it is the comp_mask, how about one of these alternatives?
- USER_MEMORY_QP
- USER_MANAGED_QP
- USER_CONFIGURABLE_QP
- EXTERNAL_MEMORY_QP
- DIRECT_ACCESS_QP

For the WQE variable mode, please see my response below.

>
> > @@ -1734,6 +1734,8 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
> >               return qptype;
> >       qplqp->type = (u8)qptype;
> >       qplqp->wqe_mode = bnxt_re_is_var_size_supported(rdev, uctx);
> > +     if (app_qp && qplqp->wqe_mode != BNXT_QPLIB_WQE_MODE_VARIABLE)
> > +             return -EOPNOTSUPP;
>
> Give a sensible name for whatever this is and use it only for this.
>
> I kind of thinking you can just fully drop it? What is the point in
> checking userspace set VARIABLE? It isn't signalling HW and it isn't
> protecting anything.
VARIABLE wqe_mode is not something that we introduced in this series.
It already exists and is understood by the FW/HW. In
bnxt_qplib_create_qp(), hardware is signaled through flags.

>
> > -             attrs = rdma_udata_to_uverbs_attr_bundle(udata);
> > -             if (uverbs_attr_is_valid(attrs,
> > -                                      BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE)) {
> > -                     dbr_obj = uverbs_attr_get_obj(attrs,
> > -                                                   BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE);
> > -                     if (IS_ERR(dbr_obj))
> > -                             return PTR_ERR(dbr_obj);
> > -                     atomic_inc(&dbr_obj->usecnt);
> > -                     qp->dbr_obj = dbr_obj;
> > +             if (ureq.comp_mask & BNXT_RE_QP_REQ_MASK_APP_ALLOCATED_QP_ENABLE) {
> > +                     attrs = rdma_udata_to_uverbs_attr_bundle(udata);
>
> And don't do this, BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE is perfectly fine
> to be signaled by attribute and it looks like it is fully orthogonal
> to the other features.
I'll update this.

Thanks,
-Harsha
>
> Jason

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5505 bytes --]

Re: [PATCH rdma-next v2 8/8] RDMA/bnxt_re: Enable app allocated QPs

[Jason Gunthorpe]

On Tue, Apr 14, 2026 at 11:43:51AM +0530, Sriharsha Basavapatna wrote:
> On Fri, Apr 10, 2026 at 8:57 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> >
> > On Fri, Mar 27, 2026 at 02:47:55PM +0530, Sriharsha Basavapatna wrote:
> > > The driver supports a new comp_mask: APP_ALLOCATED_QP_ENABLE.
> > > The application sets this comp_mask bit in the CREATE_QP ureq
> > > to indicate direct control of the QP. The driver goes through
> > > the required processing for app allocated QPs. Only variable
> > > WQE mode is supported for these QPs.
> >
> > I thought we talked about this, no weird names like this.
> Are you talking about the comp_mask name or the wqe_mode (VARIABLE)
> name? We used the comp_mask name (APP_ALLOCATED_QP) because the
> application allocates (and owns/manages) this QP memory instead of the
> library.
> If it is the comp_mask, how about one of these alternatives?
> - USER_MEMORY_QP
> - USER_MANAGED_QP
> - USER_CONFIGURABLE_QP
> - EXTERNAL_MEMORY_QP
> - DIRECT_ACCESS_QP
> 
> For the WQE variable mode, please see my response below.

Still no, make the flags reflect micro functionality relative to the
kernel operation. None of the above in any way explain what the flag
is doing to the ioctl.

The only thing the flag does is this:

> > > @@ -1734,6 +1734,8 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
> > >               return qptype;
> > >       qplqp->type = (u8)qptype;
> > >       qplqp->wqe_mode = bnxt_re_is_var_size_supported(rdev, uctx);
> > > +     if (app_qp && qplqp->wqe_mode != BNXT_QPLIB_WQE_MODE_VARIABLE)
> > > +             return -EOPNOTSUPP;
> >
> > Give a sensible name for whatever this is and use it only for
> > this.

So what is this even? FORCE_WQE_MODE_TO_VARIABLE?

> > I kind of thinking you can just fully drop it? What is the point in
> > checking userspace set VARIABLE? It isn't signalling HW and it isn't
> > protecting anything.
> VARIABLE wqe_mode is not something that we introduced in this series.
> It already exists and is understood by the FW/HW. In
> bnxt_qplib_create_qp(), hardware is signaled through flags.

Yes, and it's fine, you added app_qp and the only thing it does is
check that userspace set VARIABLE. Why?

Jason

Re: [PATCH rdma-next v2 8/8] RDMA/bnxt_re: Enable app allocated QPs

[Sriharsha Basavapatna]

[-- Attachment #1: Type: text/plain, Size: 2880 bytes --]

On Tue, Apr 14, 2026 at 6:04 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Tue, Apr 14, 2026 at 11:43:51AM +0530, Sriharsha Basavapatna wrote:
> > On Fri, Apr 10, 2026 at 8:57 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> > >
> > > On Fri, Mar 27, 2026 at 02:47:55PM +0530, Sriharsha Basavapatna wrote:
> > > > The driver supports a new comp_mask: APP_ALLOCATED_QP_ENABLE.
> > > > The application sets this comp_mask bit in the CREATE_QP ureq
> > > > to indicate direct control of the QP. The driver goes through
> > > > the required processing for app allocated QPs. Only variable
> > > > WQE mode is supported for these QPs.
> > >
> > > I thought we talked about this, no weird names like this.
> > Are you talking about the comp_mask name or the wqe_mode (VARIABLE)
> > name? We used the comp_mask name (APP_ALLOCATED_QP) because the
> > application allocates (and owns/manages) this QP memory instead of the
> > library.
> > If it is the comp_mask, how about one of these alternatives?
> > - USER_MEMORY_QP
> > - USER_MANAGED_QP
> > - USER_CONFIGURABLE_QP
> > - EXTERNAL_MEMORY_QP
> > - DIRECT_ACCESS_QP
> >
> > For the WQE variable mode, please see my response below.
>
> Still no, make the flags reflect micro functionality relative to the
> kernel operation. None of the above in any way explain what the flag
> is doing to the ioctl.
>
> The only thing the flag does is this:
>
> > > > @@ -1734,6 +1734,8 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
> > > >               return qptype;
> > > >       qplqp->type = (u8)qptype;
> > > >       qplqp->wqe_mode = bnxt_re_is_var_size_supported(rdev, uctx);
> > > > +     if (app_qp && qplqp->wqe_mode != BNXT_QPLIB_WQE_MODE_VARIABLE)
> > > > +             return -EOPNOTSUPP;
> > >
> > > Give a sensible name for whatever this is and use it only for
> > > this.
>
> So what is this even? FORCE_WQE_MODE_TO_VARIABLE?
It ensures the user/app can only operate in VARIABLE wqe_mode when
app_qp is configured. Static WQE mode is unsupported for app_qps;
therefore, we reject it here. Logic in other parts of the code
subsequently uses wqe_mode.
>
> > > I kind of thinking you can just fully drop it? What is the point in
> > > checking userspace set VARIABLE? It isn't signalling HW and it isn't
> > > protecting anything.
> > VARIABLE wqe_mode is not something that we introduced in this series.
> > It already exists and is understood by the FW/HW. In
> > bnxt_qplib_create_qp(), hardware is signaled through flags.
>
> Yes, and it's fine, you added app_qp and the only thing it does is
> check that userspace set VARIABLE. Why?
No, app_qp (boolean) is used to make other decisions too. It is passed
down to other routines and all that logic is in earlier patches (2-7).
This patch just enables it.

Thanks,
-Harsha
>
> Jason

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5505 bytes --]

Re: [PATCH rdma-next v2 8/8] RDMA/bnxt_re: Enable app allocated QPs

[Jason Gunthorpe]

On Tue, Apr 14, 2026 at 07:10:41PM +0530, Sriharsha Basavapatna wrote:

> > Yes, and it's fine, you added app_qp and the only thing it does is
> > check that userspace set VARIABLE. Why?
> No, app_qp (boolean) is used to make other decisions too. It is passed
> down to other routines and all that logic is in earlier patches (2-7).
> This patch just enables it.

So list what it actually does?

Jason

Re: [PATCH rdma-next v2 8/8] RDMA/bnxt_re: Enable app allocated QPs

[Sriharsha Basavapatna]

[-- Attachment #1: Type: text/plain, Size: 688 bytes --]

On Tue, Apr 14, 2026 at 7:24 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Tue, Apr 14, 2026 at 07:10:41PM +0530, Sriharsha Basavapatna wrote:
>
> > > Yes, and it's fine, you added app_qp and the only thing it does is
> > > check that userspace set VARIABLE. Why?
> > No, app_qp (boolean) is used to make other decisions too. It is passed
> > down to other routines and all that logic is in earlier patches (2-7).
> > This patch just enables it.
>
> So list what it actually does?
That is described in the commit messages of prior patches. To summarize:
- update rq depth
- update sq depth
- update msn table size
- update hwq depth

Thanks,
-Harsha
>
> Jason
>

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5505 bytes --]

Re: [PATCH rdma-next v2 8/8] RDMA/bnxt_re: Enable app allocated QPs

[Jason Gunthorpe]

On Tue, Apr 14, 2026 at 07:36:48PM +0530, Sriharsha Basavapatna wrote:
> On Tue, Apr 14, 2026 at 7:24 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> >
> > On Tue, Apr 14, 2026 at 07:10:41PM +0530, Sriharsha Basavapatna wrote:
> >
> > > > Yes, and it's fine, you added app_qp and the only thing it does is
> > > > check that userspace set VARIABLE. Why?
> > > No, app_qp (boolean) is used to make other decisions too. It is passed
> > > down to other routines and all that logic is in earlier patches (2-7).
> > > This patch just enables it.
> >
> > So list what it actually does?
> That is described in the commit messages of prior patches. To summarize:
> - update rq depth
> - update sq depth
> - update msn table size
> - update hwq depth

What does "update" mean? All these parameters already exist in the input.

Jason

Re: [PATCH rdma-next v2 8/8] RDMA/bnxt_re: Enable app allocated QPs

[Sriharsha Basavapatna]

[-- Attachment #1: Type: text/plain, Size: 1273 bytes --]

On Tue, Apr 14, 2026 at 7:49 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Tue, Apr 14, 2026 at 07:36:48PM +0530, Sriharsha Basavapatna wrote:
> > On Tue, Apr 14, 2026 at 7:24 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> > >
> > > On Tue, Apr 14, 2026 at 07:10:41PM +0530, Sriharsha Basavapatna wrote:
> > >
> > > > > Yes, and it's fine, you added app_qp and the only thing it does is
> > > > > check that userspace set VARIABLE. Why?
> > > > No, app_qp (boolean) is used to make other decisions too. It is passed
> > > > down to other routines and all that logic is in earlier patches (2-7).
> > > > This patch just enables it.
> > >
> > > So list what it actually does?
> > That is described in the commit messages of prior patches. To summarize:
> > - update rq depth
> > - update sq depth
> > - update msn table size
> > - update hwq depth
>
> What does "update" mean? All these parameters already exist in the input.
Yes, they are existing variables, but they are set differently if
app_qp is enabled. We don't need to compute their values (such as
adding extra slots or rounding up, etc). The application handles that,
and the driver just uses those values. And so we "update them
differently for app_qps".
Thanks,
-Harsha
>
> Jason

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5505 bytes --]

Re: [PATCH rdma-next v2 8/8] RDMA/bnxt_re: Enable app allocated QPs

[Jason Gunthorpe]

On Tue, Apr 14, 2026 at 08:04:22PM +0530, Sriharsha Basavapatna wrote:
> On Tue, Apr 14, 2026 at 7:49 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> >
> > On Tue, Apr 14, 2026 at 07:36:48PM +0530, Sriharsha Basavapatna wrote:
> > > On Tue, Apr 14, 2026 at 7:24 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> > > >
> > > > On Tue, Apr 14, 2026 at 07:10:41PM +0530, Sriharsha Basavapatna wrote:
> > > >
> > > > > > Yes, and it's fine, you added app_qp and the only thing it does is
> > > > > > check that userspace set VARIABLE. Why?
> > > > > No, app_qp (boolean) is used to make other decisions too. It is passed
> > > > > down to other routines and all that logic is in earlier patches (2-7).
> > > > > This patch just enables it.
> > > >
> > > > So list what it actually does?
> > > That is described in the commit messages of prior patches. To summarize:
> > > - update rq depth
> > > - update sq depth
> > > - update msn table size
> > > - update hwq depth
> >
> > What does "update" mean? All these parameters already exist in the input.
> Yes, they are existing variables, but they are set differently if
> app_qp is enabled. We don't need to compute their values (such as
> adding extra slots or rounding up, etc). The application handles that,
> and the driver just uses those values. And so we "update them
> differently for app_qps".

So your comp_mask setting is more like "RAW VALUES" for those existing
inputs, similar to what you did in CQ

Jasoon

Re: [PATCH rdma-next v2 8/8] RDMA/bnxt_re: Enable app allocated QPs

[Sriharsha Basavapatna]

[-- Attachment #1: Type: text/plain, Size: 1718 bytes --]

On Tue, Apr 14, 2026 at 8:39 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Tue, Apr 14, 2026 at 08:04:22PM +0530, Sriharsha Basavapatna wrote:
> > On Tue, Apr 14, 2026 at 7:49 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> > >
> > > On Tue, Apr 14, 2026 at 07:36:48PM +0530, Sriharsha Basavapatna wrote:
> > > > On Tue, Apr 14, 2026 at 7:24 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> > > > >
> > > > > On Tue, Apr 14, 2026 at 07:10:41PM +0530, Sriharsha Basavapatna wrote:
> > > > >
> > > > > > > Yes, and it's fine, you added app_qp and the only thing it does is
> > > > > > > check that userspace set VARIABLE. Why?
> > > > > > No, app_qp (boolean) is used to make other decisions too. It is passed
> > > > > > down to other routines and all that logic is in earlier patches (2-7).
> > > > > > This patch just enables it.
> > > > >
> > > > > So list what it actually does?
> > > > That is described in the commit messages of prior patches. To summarize:
> > > > - update rq depth
> > > > - update sq depth
> > > > - update msn table size
> > > > - update hwq depth
> > >
> > > What does "update" mean? All these parameters already exist in the input.
> > Yes, they are existing variables, but they are set differently if
> > app_qp is enabled. We don't need to compute their values (such as
> > adding extra slots or rounding up, etc). The application handles that,
> > and the driver just uses those values. And so we "update them
> > differently for app_qps".
>
> So your comp_mask setting is more like "RAW VALUES" for those existing
> inputs, similar to what you did in CQ
Yes, but CQ was simple and in one place, while QP involves more logic.
Thanks,
-Harsha
>
> Jasoon

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5505 bytes --]