* [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions
@ 2026-05-08 8:58 Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 1/7] RDMA/bnxt_re: Refactor bnxt_re_init_user_qp() Sriharsha Basavapatna
` (7 more replies)
0 siblings, 8 replies; 9+ messages in thread
From: Sriharsha Basavapatna @ 2026-05-08 8:58 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
Hi,
This patchset adds QP uapi extensions to the bnxt_re driver.
This is required by applications that need to manage some of the
RDMA HW resources directly and to implement the datapath in the
application.
This series supports application allocated memory for QPs.
The application takes into account SQ/RQ ring sizing constraints
(extra entries, rounding up etc) while allocating this memory.
The driver should avoid duplicating this logic while creating
these QPs.
uAPI changes in this series:
- Patch#4: new uapi parameter 'sq_npsn' in bnxt_re_qp_req.
- Patch#6: new driver specific attribute 'DBR_HANDLE' for doorbell region.
- Patch#7: new comp_mask 'FIXED_QUE_ATTR' in bnxt_re_qp_req.
Patch#1 Refactor bnxt_re_init_user_qp()
Patch#2 Update rq depth for app allocated QPs
Patch#3 Update sq depth for app allocated QPs
Patch#4 Update msn table size for app allocated QPs
Patch#5 Update hwq depth for app allocated QPs
Patch#6 Support doorbells for app allocated QPs
Patch#7 Enable app allocated QPs
Thanks,
-Harsha
******
Changes:
v4:
- Rebased to latest for-next tree (Linux 7.1-rc1, commit: 254f49634ee1).
- Renamed QP req comp_mask: APP_ALLOCATED_QP_ENABLE -> FIXED_QUE_ATTR.
v3:
- Removed umem patch from the series, that is dependent on uverbs support.
- Patch#7: Process DBR_HANDLE attr regardless of app_qp comp_mask.
v2:
- Rebased to umem_list uverbs patch series:
https://patchwork.kernel.org/project/linux-rdma/cover/20260325150048.168341-1-jiri@resnulli.us/
- Deleted Patch#9; create_qp_umem devop is not supported.
v3: https://lore.kernel.org/linux-rdma/20260415054957.36745-1-sriharsha.basavapatna@broadcom.com/
v2: https://lore.kernel.org/linux-rdma/20260327091755.47754-1-sriharsha.basavapatna@broadcom.com/
v1: https://lore.kernel.org/linux-rdma/20260320135437.48716-1-sriharsha.basavapatna@broadcom.com/
******
Sriharsha Basavapatna (7):
RDMA/bnxt_re: Refactor bnxt_re_init_user_qp()
RDMA/bnxt_re: Update rq depth for app allocated QPs
RDMA/bnxt_re: Update sq depth for app allocated QPs
RDMA/bnxt_re: Update msn table size for app allocated QPs
RDMA/bnxt_re: Update hwq depth for app allocated QPs
RDMA/bnxt_re: Support doorbells for app allocated QPs
RDMA/bnxt_re: Enable app allocated QPs
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 270 +++++++++++++++--------
drivers/infiniband/hw/bnxt_re/ib_verbs.h | 1 +
drivers/infiniband/hw/bnxt_re/uapi.c | 18 ++
include/uapi/rdma/bnxt_re-abi.h | 6 +
4 files changed, 206 insertions(+), 89 deletions(-)
--
2.51.2.636.ga99f379adf
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH rdma-next v4 1/7] RDMA/bnxt_re: Refactor bnxt_re_init_user_qp()
2026-05-08 8:58 [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions Sriharsha Basavapatna
@ 2026-05-08 8:58 ` Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 2/7] RDMA/bnxt_re: Update rq depth for app allocated QPs Sriharsha Basavapatna
` (6 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Sriharsha Basavapatna @ 2026-05-08 8:58 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
The umem changes for CQ added a helper - bnxt_re_setup_sginfo().
Use the same helper for QP creation since we support only 4K
pages for QP ring memory too.
Add a new helper function bnxt_re_get_psn_bytes() to improve
readability as this code will be updated in subsequent patches.
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 127 +++++++++++++----------
1 file changed, 73 insertions(+), 54 deletions(-)
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 7ed294516b7e..561d491f12ff 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1136,34 +1136,64 @@ static int bnxt_re_setup_swqe_size(struct bnxt_re_qp *qp,
return 0;
}
+static int bnxt_re_setup_sginfo(struct bnxt_re_dev *rdev,
+ struct ib_umem *umem,
+ struct bnxt_qplib_sg_info *sginfo)
+{
+ unsigned long page_size;
+
+ if (!umem)
+ return -EINVAL;
+
+ page_size = ib_umem_find_best_pgsz(umem, SZ_4K, 0);
+ if (!page_size || page_size != SZ_4K)
+ return -EINVAL;
+
+ sginfo->umem = umem;
+ sginfo->npages = ib_umem_num_dma_blocks(umem, page_size);
+ sginfo->pgsize = page_size;
+ sginfo->pgshft = __builtin_ctz(page_size);
+ return 0;
+}
+
+static int bnxt_re_get_psn_bytes(struct bnxt_re_dev *rdev,
+ struct bnxt_re_ucontext *cntx,
+ struct bnxt_qplib_qp *qplib_qp,
+ struct bnxt_re_qp_req *ureq)
+{
+ int psn_sz, psn_nume;
+
+ psn_sz = bnxt_qplib_is_chip_gen_p5_p7(rdev->chip_ctx) ?
+ sizeof(struct sq_psn_search_ext) :
+ sizeof(struct sq_psn_search);
+ if (cntx && bnxt_re_is_var_size_supported(rdev, cntx)) {
+ psn_nume = ureq->sq_slots;
+ } else {
+ psn_nume = (qplib_qp->wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC) ?
+ qplib_qp->sq.max_wqe : ((qplib_qp->sq.max_wqe * qplib_qp->sq.wqe_size) /
+ sizeof(struct bnxt_qplib_sge));
+ }
+ if (_is_host_msn_table(rdev->qplib_res.dattr->dev_cap_flags2))
+ psn_nume = roundup_pow_of_two(psn_nume);
+
+ return psn_nume * psn_sz;
+}
+
static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
struct bnxt_re_qp *qp, struct bnxt_re_ucontext *cntx,
struct bnxt_re_qp_req *ureq)
{
struct bnxt_qplib_qp *qplib_qp;
- int bytes = 0, psn_sz;
struct ib_umem *umem;
- int psn_nume;
+ int bytes;
+ int rc;
qplib_qp = &qp->qplib_qp;
bytes = (qplib_qp->sq.max_wqe * qplib_qp->sq.wqe_size);
/* Consider mapping PSN search memory only for RC QPs. */
- if (qplib_qp->type == CMDQ_CREATE_QP_TYPE_RC) {
- psn_sz = bnxt_qplib_is_chip_gen_p5_p7(rdev->chip_ctx) ?
- sizeof(struct sq_psn_search_ext) :
- sizeof(struct sq_psn_search);
- if (cntx && bnxt_re_is_var_size_supported(rdev, cntx)) {
- psn_nume = ureq->sq_slots;
- } else {
- psn_nume = (qplib_qp->wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC) ?
- qplib_qp->sq.max_wqe : ((qplib_qp->sq.max_wqe * qplib_qp->sq.wqe_size) /
- sizeof(struct bnxt_qplib_sge));
- }
- if (_is_host_msn_table(rdev->qplib_res.dattr->dev_cap_flags2))
- psn_nume = roundup_pow_of_two(psn_nume);
- bytes += (psn_nume * psn_sz);
- }
+ if (qplib_qp->type == CMDQ_CREATE_QP_TYPE_RC)
+ bytes += bnxt_re_get_psn_bytes(rdev, cntx, qplib_qp, ureq);
bytes = PAGE_ALIGN(bytes);
umem = ib_umem_get(&rdev->ibdev, ureq->qpsva, bytes,
@@ -1172,33 +1202,42 @@ static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
return PTR_ERR(umem);
qp->sumem = umem;
- qplib_qp->sq.sg_info.umem = umem;
- qplib_qp->sq.sg_info.pgsize = PAGE_SIZE;
- qplib_qp->sq.sg_info.pgshft = PAGE_SHIFT;
- qplib_qp->qp_handle = ureq->qp_handle;
+ rc = bnxt_re_setup_sginfo(rdev, qp->sumem, &qplib_qp->sq.sg_info);
+ if (rc)
+ goto fail;
+
+ if (qp->qplib_qp.srq)
+ goto done;
- if (!qp->qplib_qp.srq) {
- bytes = (qplib_qp->rq.max_wqe * qplib_qp->rq.wqe_size);
- bytes = PAGE_ALIGN(bytes);
- umem = ib_umem_get(&rdev->ibdev, ureq->qprva, bytes,
- IB_ACCESS_LOCAL_WRITE);
- if (IS_ERR(umem))
- goto rqfail;
- qp->rumem = umem;
- qplib_qp->rq.sg_info.umem = umem;
- qplib_qp->rq.sg_info.pgsize = PAGE_SIZE;
- qplib_qp->rq.sg_info.pgshft = PAGE_SHIFT;
+ bytes = (qplib_qp->rq.max_wqe * qplib_qp->rq.wqe_size);
+ bytes = PAGE_ALIGN(bytes);
+ umem = ib_umem_get(&rdev->ibdev, ureq->qprva, bytes,
+ IB_ACCESS_LOCAL_WRITE);
+ if (IS_ERR(umem)) {
+ rc = PTR_ERR(umem);
+ goto fail;
}
+ qp->rumem = umem;
+ rc = bnxt_re_setup_sginfo(rdev, qp->rumem, &qplib_qp->rq.sg_info);
+ if (rc)
+ goto rqfail;
+
+done:
+ qplib_qp->qp_handle = ureq->qp_handle;
qplib_qp->dpi = &cntx->dpi;
qplib_qp->is_user = true;
return 0;
+
rqfail:
+ ib_umem_release(qp->rumem);
+ qp->rumem = NULL;
+ memset(&qplib_qp->rq.sg_info, 0, sizeof(qplib_qp->rq.sg_info));
+fail:
ib_umem_release(qp->sumem);
qp->sumem = NULL;
memset(&qplib_qp->sq.sg_info, 0, sizeof(qplib_qp->sq.sg_info));
-
- return PTR_ERR(umem);
+ return rc;
}
static struct bnxt_re_ah *bnxt_re_create_shadow_qp_ah
@@ -3345,26 +3384,6 @@ int bnxt_re_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata)
return ib_respond_empty_udata(udata);
}
-static int bnxt_re_setup_sginfo(struct bnxt_re_dev *rdev,
- struct ib_umem *umem,
- struct bnxt_qplib_sg_info *sginfo)
-{
- unsigned long page_size;
-
- if (!umem)
- return -EINVAL;
-
- page_size = ib_umem_find_best_pgsz(umem, SZ_4K, 0);
- if (!page_size || page_size != SZ_4K)
- return -EINVAL;
-
- sginfo->umem = umem;
- sginfo->npages = ib_umem_num_dma_blocks(umem, page_size);
- sginfo->pgsize = page_size;
- sginfo->pgshft = __builtin_ctz(page_size);
- return 0;
-}
-
int bnxt_re_create_user_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
struct uverbs_attr_bundle *attrs)
{
--
2.51.2.636.ga99f379adf
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH rdma-next v4 2/7] RDMA/bnxt_re: Update rq depth for app allocated QPs
2026-05-08 8:58 [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 1/7] RDMA/bnxt_re: Refactor bnxt_re_init_user_qp() Sriharsha Basavapatna
@ 2026-05-08 8:58 ` Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 3/7] RDMA/bnxt_re: Update sq " Sriharsha Basavapatna
` (5 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Sriharsha Basavapatna @ 2026-05-08 8:58 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
For app allocated QPs, there's no need to add extra slots or
to round up the slot count. Use 'max_recv_wr' count provided
by the application as is.
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 23 +++++++++++++++--------
1 file changed, 15 insertions(+), 8 deletions(-)
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 561d491f12ff..0ab4ac2f4c41 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1475,7 +1475,9 @@ static struct bnxt_re_qp *bnxt_re_create_shadow_qp
static int bnxt_re_init_rq_attr(struct bnxt_re_qp *qp,
struct ib_qp_init_attr *init_attr,
- struct bnxt_re_ucontext *uctx)
+ struct bnxt_re_ucontext *uctx,
+ bool fixed_que_attr,
+ struct bnxt_re_qp_req *ureq)
{
struct bnxt_qplib_dev_attr *dev_attr;
struct bnxt_qplib_qp *qplqp;
@@ -1500,12 +1502,16 @@ static int bnxt_re_init_rq_attr(struct bnxt_re_qp *qp,
init_attr->cap.max_recv_sge = rq->max_sge;
rq->wqe_size = bnxt_re_setup_rwqe_size(qplqp, rq->max_sge,
dev_attr->max_qp_sges);
- /* Allocate 1 more than what's provided so posting max doesn't
- * mean empty.
- */
- rq->max_wqe = bnxt_re_init_depth(init_attr->cap.max_recv_wr + 1,
- dev_attr->max_qp_wqes + 1,
- uctx);
+ if (!fixed_que_attr) {
+ /* Allocate 1 more than what's provided so posting max doesn't
+ * mean empty.
+ */
+ rq->max_wqe = bnxt_re_init_depth(init_attr->cap.max_recv_wr + 1,
+ dev_attr->max_qp_wqes + 1,
+ uctx);
+ } else {
+ rq->max_wqe = init_attr->cap.max_recv_wr;
+ }
rq->max_sw_wqe = rq->max_wqe;
rq->q_full_delta = 0;
rq->sg_info.pgsize = PAGE_SIZE;
@@ -1676,6 +1682,7 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
{
struct bnxt_qplib_dev_attr *dev_attr;
struct bnxt_qplib_qp *qplqp;
+ bool fixed_que_attr = false;
struct bnxt_re_dev *rdev;
struct bnxt_re_cq *cq;
int rc = 0, qptype;
@@ -1724,7 +1731,7 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
}
/* Setup RQ/SRQ */
- rc = bnxt_re_init_rq_attr(qp, init_attr, uctx);
+ rc = bnxt_re_init_rq_attr(qp, init_attr, uctx, fixed_que_attr, ureq);
if (rc)
return rc;
if (init_attr->qp_type == IB_QPT_GSI)
--
2.51.2.636.ga99f379adf
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH rdma-next v4 3/7] RDMA/bnxt_re: Update sq depth for app allocated QPs
2026-05-08 8:58 [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 1/7] RDMA/bnxt_re: Refactor bnxt_re_init_user_qp() Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 2/7] RDMA/bnxt_re: Update rq depth for app allocated QPs Sriharsha Basavapatna
@ 2026-05-08 8:58 ` Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 4/7] RDMA/bnxt_re: Update msn table size " Sriharsha Basavapatna
` (4 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Sriharsha Basavapatna @ 2026-05-08 8:58 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
For app allocated QPs, there's no need to reserve extra slots.
The application accounts for this while allocating the SQ.
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 24 +++++++++++++++---------
1 file changed, 15 insertions(+), 9 deletions(-)
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 0ab4ac2f4c41..2954674fdd33 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1542,7 +1542,8 @@ static void bnxt_re_adjust_gsi_rq_attr(struct bnxt_re_qp *qp)
static int bnxt_re_init_sq_attr(struct bnxt_re_qp *qp,
struct ib_qp_init_attr *init_attr,
struct bnxt_re_ucontext *uctx,
- struct bnxt_re_qp_req *ureq)
+ struct bnxt_re_qp_req *ureq,
+ bool fixed_que_attr)
{
struct bnxt_qplib_dev_attr *dev_attr;
struct bnxt_qplib_qp *qplqp;
@@ -1583,13 +1584,18 @@ static int bnxt_re_init_sq_attr(struct bnxt_re_qp *qp,
sq->max_sw_wqe = sq->max_wqe;
}
- sq->q_full_delta = diff + 1;
- /*
- * Reserving one slot for Phantom WQE. Application can
- * post one extra entry in this case. But allowing this to avoid
- * unexpected Queue full condition
- */
- qplqp->sq.q_full_delta -= 1;
+ if (!fixed_que_attr) {
+ sq->q_full_delta = diff + 1;
+ /*
+ * Reserving one slot for Phantom WQE. Application can
+ * post one extra entry in this case. But allowing this to avoid
+ * unexpected Queue full condition
+ */
+ qplqp->sq.q_full_delta -= 1;
+ } else {
+ sq->q_full_delta = 0;
+ }
+
qplqp->sq.sg_info.pgsize = PAGE_SIZE;
qplqp->sq.sg_info.pgshft = PAGE_SHIFT;
@@ -1738,7 +1744,7 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
bnxt_re_adjust_gsi_rq_attr(qp);
/* Setup SQ */
- rc = bnxt_re_init_sq_attr(qp, init_attr, uctx, ureq);
+ rc = bnxt_re_init_sq_attr(qp, init_attr, uctx, ureq, fixed_que_attr);
if (rc)
return rc;
if (init_attr->qp_type == IB_QPT_GSI)
--
2.51.2.636.ga99f379adf
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH rdma-next v4 4/7] RDMA/bnxt_re: Update msn table size for app allocated QPs
2026-05-08 8:58 [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions Sriharsha Basavapatna
` (2 preceding siblings ...)
2026-05-08 8:58 ` [PATCH rdma-next v4 3/7] RDMA/bnxt_re: Update sq " Sriharsha Basavapatna
@ 2026-05-08 8:58 ` Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 5/7] RDMA/bnxt_re: Update hwq depth " Sriharsha Basavapatna
` (3 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Sriharsha Basavapatna @ 2026-05-08 8:58 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
For app allocated QPs, the driver shouldn't use slots/round-up logic
to compute the msn table size. The application handles this logic
and computes 'sq_npsn' and passes it to the driver using a new uapi
parameter.
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 61 +++++++++++++++---------
include/uapi/rdma/bnxt_re-abi.h | 1 +
2 files changed, 40 insertions(+), 22 deletions(-)
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 2954674fdd33..33c6b8985b4f 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1159,29 +1159,39 @@ static int bnxt_re_setup_sginfo(struct bnxt_re_dev *rdev,
static int bnxt_re_get_psn_bytes(struct bnxt_re_dev *rdev,
struct bnxt_re_ucontext *cntx,
struct bnxt_qplib_qp *qplib_qp,
- struct bnxt_re_qp_req *ureq)
+ struct bnxt_re_qp_req *ureq,
+ bool fixed_que_attr)
{
int psn_sz, psn_nume;
- psn_sz = bnxt_qplib_is_chip_gen_p5_p7(rdev->chip_ctx) ?
- sizeof(struct sq_psn_search_ext) :
- sizeof(struct sq_psn_search);
- if (cntx && bnxt_re_is_var_size_supported(rdev, cntx)) {
- psn_nume = ureq->sq_slots;
+ if (rdev->dev_attr &&
+ _is_host_msn_table(rdev->dev_attr->dev_cap_flags2))
+ psn_sz = sizeof(struct sq_msn_search);
+ else
+ psn_sz = bnxt_qplib_is_chip_gen_p5_p7(rdev->chip_ctx) ?
+ sizeof(struct sq_psn_search_ext) :
+ sizeof(struct sq_psn_search);
+ if (!fixed_que_attr) {
+ if (cntx && bnxt_re_is_var_size_supported(rdev, cntx)) {
+ psn_nume = ureq->sq_slots;
+ } else {
+ psn_nume = (qplib_qp->wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC) ?
+ qplib_qp->sq.max_wqe : ((qplib_qp->sq.max_wqe * qplib_qp->sq.wqe_size) /
+ sizeof(struct bnxt_qplib_sge));
+ }
+ if (_is_host_msn_table(rdev->qplib_res.dattr->dev_cap_flags2))
+ psn_nume = roundup_pow_of_two(psn_nume);
} else {
- psn_nume = (qplib_qp->wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC) ?
- qplib_qp->sq.max_wqe : ((qplib_qp->sq.max_wqe * qplib_qp->sq.wqe_size) /
- sizeof(struct bnxt_qplib_sge));
+ psn_nume = ureq->sq_npsn;
}
- if (_is_host_msn_table(rdev->qplib_res.dattr->dev_cap_flags2))
- psn_nume = roundup_pow_of_two(psn_nume);
return psn_nume * psn_sz;
}
static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
struct bnxt_re_qp *qp, struct bnxt_re_ucontext *cntx,
- struct bnxt_re_qp_req *ureq)
+ struct bnxt_re_qp_req *ureq,
+ bool fixed_que_attr)
{
struct bnxt_qplib_qp *qplib_qp;
struct ib_umem *umem;
@@ -1193,7 +1203,7 @@ static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
bytes = (qplib_qp->sq.max_wqe * qplib_qp->sq.wqe_size);
/* Consider mapping PSN search memory only for RC QPs. */
if (qplib_qp->type == CMDQ_CREATE_QP_TYPE_RC)
- bytes += bnxt_re_get_psn_bytes(rdev, cntx, qplib_qp, ureq);
+ bytes += bnxt_re_get_psn_bytes(rdev, cntx, qplib_qp, ureq, fixed_que_attr);
bytes = PAGE_ALIGN(bytes);
umem = ib_umem_get(&rdev->ibdev, ureq->qpsva, bytes,
@@ -1648,7 +1658,9 @@ static int bnxt_re_init_qp_type(struct bnxt_re_dev *rdev,
return qptype;
}
-static void bnxt_re_qp_calculate_msn_psn_size(struct bnxt_re_qp *qp)
+static void bnxt_re_qp_calculate_msn_psn_size(struct bnxt_re_qp *qp,
+ bool fixed_que_attr,
+ struct bnxt_re_qp_req *req)
{
struct bnxt_qplib_qp *qplib_qp = &qp->qplib_qp;
struct bnxt_qplib_q *sq = &qplib_qp->sq;
@@ -1671,12 +1683,17 @@ static void bnxt_re_qp_calculate_msn_psn_size(struct bnxt_re_qp *qp)
/* Update msn tbl size */
if (qplib_qp->is_host_msn_tbl && qplib_qp->psn_sz) {
- if (wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC)
- qplib_qp->msn_tbl_sz =
- roundup_pow_of_two(bnxt_qplib_set_sq_size(sq, wqe_mode));
- else
- qplib_qp->msn_tbl_sz =
- roundup_pow_of_two(bnxt_qplib_set_sq_size(sq, wqe_mode)) / 2;
+ if (!fixed_que_attr) {
+ if (wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC)
+ qplib_qp->msn_tbl_sz =
+ roundup_pow_of_two(bnxt_qplib_set_sq_size(sq, wqe_mode));
+ else
+ qplib_qp->msn_tbl_sz =
+ roundup_pow_of_two(bnxt_qplib_set_sq_size(sq, wqe_mode))
+ / 2;
+ } else {
+ qplib_qp->msn_tbl_sz = req->sq_npsn / 2; /* WQE_MODE_VARIABLE */
+ }
qplib_qp->msn = 0;
}
}
@@ -1751,12 +1768,12 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
bnxt_re_adjust_gsi_sq_attr(qp, init_attr, uctx);
if (uctx) { /* This will update DPI and qp_handle */
- rc = bnxt_re_init_user_qp(rdev, pd, qp, uctx, ureq);
+ rc = bnxt_re_init_user_qp(rdev, pd, qp, uctx, ureq, fixed_que_attr);
if (rc)
return rc;
}
- bnxt_re_qp_calculate_msn_psn_size(qp);
+ bnxt_re_qp_calculate_msn_psn_size(qp, fixed_que_attr, ureq);
rc = bnxt_re_setup_qp_hwqs(qp);
if (rc)
diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h
index 40955eaba32e..db8400f2ce3b 100644
--- a/include/uapi/rdma/bnxt_re-abi.h
+++ b/include/uapi/rdma/bnxt_re-abi.h
@@ -135,6 +135,7 @@ struct bnxt_re_qp_req {
__aligned_u64 qp_handle;
__aligned_u64 comp_mask;
__u32 sq_slots;
+ __u32 sq_npsn;
};
struct bnxt_re_qp_resp {
--
2.51.2.636.ga99f379adf
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH rdma-next v4 5/7] RDMA/bnxt_re: Update hwq depth for app allocated QPs
2026-05-08 8:58 [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions Sriharsha Basavapatna
` (3 preceding siblings ...)
2026-05-08 8:58 ` [PATCH rdma-next v4 4/7] RDMA/bnxt_re: Update msn table size " Sriharsha Basavapatna
@ 2026-05-08 8:58 ` Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 6/7] RDMA/bnxt_re: Support doorbells " Sriharsha Basavapatna
` (2 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Sriharsha Basavapatna @ 2026-05-08 8:58 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
The hwq depth shouldn't be computed using slots/round-up logic for
app allocated QPs, use the max_wqe value saved earlier.
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 28 +++++++++++++++++-------
1 file changed, 20 insertions(+), 8 deletions(-)
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 33c6b8985b4f..a04cbfb36135 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1346,7 +1346,7 @@ static int bnxt_re_qp_alloc_init_xrrq(struct bnxt_re_qp *qp)
return rc;
}
-static int bnxt_re_setup_qp_hwqs(struct bnxt_re_qp *qp)
+static int bnxt_re_setup_qp_hwqs(struct bnxt_re_qp *qp, bool fixed_que_attr)
{
struct bnxt_qplib_res *res = &qp->rdev->qplib_res;
struct bnxt_qplib_qp *qplib_qp = &qp->qplib_qp;
@@ -1360,12 +1360,17 @@ static int bnxt_re_setup_qp_hwqs(struct bnxt_re_qp *qp)
hwq_attr.res = res;
hwq_attr.sginfo = &sq->sg_info;
hwq_attr.stride = bnxt_qplib_get_stride();
- hwq_attr.depth = bnxt_qplib_get_depth(sq, wqe_mode, true);
hwq_attr.aux_stride = qplib_qp->psn_sz;
- hwq_attr.aux_depth = (qplib_qp->psn_sz) ?
- bnxt_qplib_set_sq_size(sq, wqe_mode) : 0;
- if (qplib_qp->is_host_msn_tbl && qplib_qp->psn_sz)
+ if (!fixed_que_attr) {
+ hwq_attr.depth = bnxt_qplib_get_depth(sq, wqe_mode, true);
+ hwq_attr.aux_depth = (qplib_qp->psn_sz) ?
+ bnxt_qplib_set_sq_size(sq, wqe_mode) : 0;
+ if (qplib_qp->is_host_msn_tbl && qplib_qp->psn_sz)
+ hwq_attr.aux_depth = qplib_qp->msn_tbl_sz;
+ } else {
+ hwq_attr.depth = sq->max_wqe;
hwq_attr.aux_depth = qplib_qp->msn_tbl_sz;
+ }
hwq_attr.type = HWQ_TYPE_QUEUE;
rc = bnxt_qplib_alloc_init_hwq(&sq->hwq, &hwq_attr);
if (rc)
@@ -1376,10 +1381,16 @@ static int bnxt_re_setup_qp_hwqs(struct bnxt_re_qp *qp)
CMDQ_CREATE_QP_SQ_LVL_SFT);
sq->hwq.pg_sz_lvl = pg_sz_lvl;
+ if (qplib_qp->srq)
+ goto done;
+
hwq_attr.res = res;
hwq_attr.sginfo = &rq->sg_info;
hwq_attr.stride = bnxt_qplib_get_stride();
- hwq_attr.depth = bnxt_qplib_get_depth(rq, qplib_qp->wqe_mode, false);
+ if (!fixed_que_attr)
+ hwq_attr.depth = bnxt_qplib_get_depth(rq, qplib_qp->wqe_mode, false);
+ else
+ hwq_attr.depth = (rq->max_wqe * rq->wqe_size) / hwq_attr.stride;
hwq_attr.aux_stride = 0;
hwq_attr.aux_depth = 0;
hwq_attr.type = HWQ_TYPE_QUEUE;
@@ -1392,6 +1403,7 @@ static int bnxt_re_setup_qp_hwqs(struct bnxt_re_qp *qp)
CMDQ_CREATE_QP_RQ_LVL_SFT);
rq->hwq.pg_sz_lvl = pg_sz_lvl;
+done:
if (qplib_qp->psn_sz) {
rc = bnxt_re_qp_alloc_init_xrrq(qp);
if (rc)
@@ -1460,7 +1472,7 @@ static struct bnxt_re_qp *bnxt_re_create_shadow_qp
qp->qplib_qp.rq_hdr_buf_size = BNXT_QPLIB_MAX_GRH_HDR_SIZE_IPV6;
qp->qplib_qp.dpi = &rdev->dpi_privileged;
- rc = bnxt_re_setup_qp_hwqs(qp);
+ rc = bnxt_re_setup_qp_hwqs(qp, false);
if (rc)
goto fail;
@@ -1775,7 +1787,7 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
bnxt_re_qp_calculate_msn_psn_size(qp, fixed_que_attr, ureq);
- rc = bnxt_re_setup_qp_hwqs(qp);
+ rc = bnxt_re_setup_qp_hwqs(qp, fixed_que_attr);
if (rc)
goto free_umem;
--
2.51.2.636.ga99f379adf
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH rdma-next v4 6/7] RDMA/bnxt_re: Support doorbells for app allocated QPs
2026-05-08 8:58 [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions Sriharsha Basavapatna
` (4 preceding siblings ...)
2026-05-08 8:58 ` [PATCH rdma-next v4 5/7] RDMA/bnxt_re: Update hwq depth " Sriharsha Basavapatna
@ 2026-05-08 8:58 ` Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 7/7] RDMA/bnxt_re: Enable " Sriharsha Basavapatna
2026-05-12 16:53 ` [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions Jason Gunthorpe
7 siblings, 0 replies; 9+ messages in thread
From: Sriharsha Basavapatna @ 2026-05-08 8:58 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
App allocated QPs can use a separate doorbell for each QP.
This doorbell region can be passed through a new driver specific
DBR_HANDLE attribute, during QP creation. When this attribute
is set, associate the QP with the given doorbell region.
While the QP holds a reference to the dbr, the dbr itself
cannot be destroyed and is rejected with EBUSY error.
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 35 ++++++++++++++++++++----
drivers/infiniband/hw/bnxt_re/ib_verbs.h | 1 +
drivers/infiniband/hw/bnxt_re/uapi.c | 18 ++++++++++++
include/uapi/rdma/bnxt_re-abi.h | 4 +++
4 files changed, 53 insertions(+), 5 deletions(-)
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index a04cbfb36135..5d36a2fc8a10 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1016,6 +1016,9 @@ int bnxt_re_destroy_qp(struct ib_qp *ib_qp, struct ib_udata *udata)
if (rc)
return rc;
+ if (qp->dbr_obj)
+ atomic_dec(&qp->dbr_obj->usecnt);
+
bnxt_re_debug_rem_qpinfo(rdev, qp);
bnxt_qplib_flush_cqn_wq(&qp->qplib_qp);
@@ -1191,7 +1194,8 @@ static int bnxt_re_get_psn_bytes(struct bnxt_re_dev *rdev,
static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
struct bnxt_re_qp *qp, struct bnxt_re_ucontext *cntx,
struct bnxt_re_qp_req *ureq,
- bool fixed_que_attr)
+ bool fixed_que_attr,
+ struct bnxt_re_dbr_obj *dbr_obj)
{
struct bnxt_qplib_qp *qplib_qp;
struct ib_umem *umem;
@@ -1234,8 +1238,11 @@ static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
goto rqfail;
done:
+ if (dbr_obj)
+ qplib_qp->dpi = &dbr_obj->dpi;
+ else
+ qplib_qp->dpi = &cntx->dpi;
qplib_qp->qp_handle = ureq->qp_handle;
- qplib_qp->dpi = &cntx->dpi;
qplib_qp->is_user = true;
return 0;
@@ -1713,7 +1720,8 @@ static void bnxt_re_qp_calculate_msn_psn_size(struct bnxt_re_qp *qp,
static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
struct ib_qp_init_attr *init_attr,
struct bnxt_re_ucontext *uctx,
- struct bnxt_re_qp_req *ureq)
+ struct bnxt_re_qp_req *ureq,
+ struct bnxt_re_dbr_obj *dbr_obj)
{
struct bnxt_qplib_dev_attr *dev_attr;
struct bnxt_qplib_qp *qplqp;
@@ -1780,7 +1788,8 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
bnxt_re_adjust_gsi_sq_attr(qp, init_attr, uctx);
if (uctx) { /* This will update DPI and qp_handle */
- rc = bnxt_re_init_user_qp(rdev, pd, qp, uctx, ureq, fixed_que_attr);
+ rc = bnxt_re_init_user_qp(rdev, pd, qp, uctx, ureq, fixed_que_attr,
+ dbr_obj);
if (rc)
return rc;
}
@@ -1916,7 +1925,9 @@ static int bnxt_re_add_unique_gid(struct bnxt_re_dev *rdev)
int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
struct ib_udata *udata)
{
+ struct bnxt_re_dbr_obj *dbr_obj = NULL;
struct bnxt_qplib_dev_attr *dev_attr;
+ struct uverbs_attr_bundle *attrs;
struct bnxt_re_ucontext *uctx;
struct bnxt_re_qp_req ureq;
struct bnxt_re_dev *rdev;
@@ -1937,6 +1948,17 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
rc = ib_copy_validate_udata_in_cm(udata, ureq, qp_handle, 0);
if (rc)
return rc;
+
+ attrs = rdma_udata_to_uverbs_attr_bundle(udata);
+ if (uverbs_attr_is_valid(attrs,
+ BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE)) {
+ dbr_obj = uverbs_attr_get_obj(attrs,
+ BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE);
+ if (IS_ERR(dbr_obj))
+ return PTR_ERR(dbr_obj);
+ atomic_inc(&dbr_obj->usecnt);
+ qp->dbr_obj = dbr_obj;
+ }
}
rc = bnxt_re_test_qp_limits(rdev, qp_init_attr, dev_attr);
@@ -1946,7 +1968,8 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
}
qp->rdev = rdev;
- rc = bnxt_re_init_qp_attr(qp, pd, qp_init_attr, uctx, &ureq);
+ rc = bnxt_re_init_qp_attr(qp, pd, qp_init_attr, uctx, &ureq,
+ dbr_obj);
if (rc)
goto fail;
@@ -2016,6 +2039,8 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
bnxt_qplib_free_qp_res(&rdev->qplib_res, &qp->qplib_qp);
bnxt_re_qp_free_umem(qp);
fail:
+ if (dbr_obj)
+ atomic_dec(&dbr_obj->usecnt);
return rc;
}
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
index 08f71a94d55d..fdf3dd4432c1 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
@@ -96,6 +96,7 @@ struct bnxt_re_qp {
struct bnxt_re_cq *scq;
struct bnxt_re_cq *rcq;
struct dentry *dentry;
+ struct bnxt_re_dbr_obj *dbr_obj; /* doorbell region */
};
struct bnxt_re_cq {
diff --git a/drivers/infiniband/hw/bnxt_re/uapi.c b/drivers/infiniband/hw/bnxt_re/uapi.c
index 3eaee7101615..83e5ddf6ecf6 100644
--- a/drivers/infiniband/hw/bnxt_re/uapi.c
+++ b/drivers/infiniband/hw/bnxt_re/uapi.c
@@ -398,6 +398,9 @@ static int bnxt_re_dbr_cleanup(struct ib_uobject *uobject,
struct bnxt_re_dbr_obj *obj = uobject->object;
struct bnxt_re_dev *rdev = obj->rdev;
+ if (atomic_read(&obj->usecnt))
+ return -EBUSY;
+
rdma_user_mmap_entry_remove(&obj->entry->rdma_entry);
bnxt_qplib_free_uc_dpi(&rdev->qplib_res, &obj->dpi);
return 0;
@@ -459,11 +462,26 @@ DECLARE_UVERBS_NAMED_METHOD(BNXT_RE_METHOD_GET_DEFAULT_DBR,
DECLARE_UVERBS_GLOBAL_METHODS(BNXT_RE_OBJECT_DEFAULT_DBR,
&UVERBS_METHOD(BNXT_RE_METHOD_GET_DEFAULT_DBR));
+ADD_UVERBS_ATTRIBUTES_SIMPLE(
+ bnxt_re_qp_create,
+ UVERBS_OBJECT_QP,
+ UVERBS_METHOD_QP_CREATE,
+ UVERBS_ATTR_IDR(BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE,
+ BNXT_RE_OBJECT_DBR,
+ UVERBS_ACCESS_READ,
+ UA_OPTIONAL));
+
+const struct uapi_definition bnxt_re_create_qp_defs[] = {
+ UAPI_DEF_CHAIN_OBJ_TREE(UVERBS_OBJECT_QP, &bnxt_re_qp_create),
+ {},
+};
+
const struct uapi_definition bnxt_re_uapi_defs[] = {
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_ALLOC_PAGE),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_NOTIFY_DRV),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_GET_TOGGLE_MEM),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_DBR),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_DEFAULT_DBR),
+ UAPI_DEF_CHAIN(bnxt_re_create_qp_defs),
{}
};
diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h
index db8400f2ce3b..4da8cda337dc 100644
--- a/include/uapi/rdma/bnxt_re-abi.h
+++ b/include/uapi/rdma/bnxt_re-abi.h
@@ -138,6 +138,10 @@ struct bnxt_re_qp_req {
__u32 sq_npsn;
};
+enum bnxt_re_create_qp_attrs {
+ BNXT_RE_CREATE_QP_ATTR_DBR_HANDLE = UVERBS_ID_DRIVER_NS_WITH_UHW,
+};
+
struct bnxt_re_qp_resp {
__u32 qpid;
__u32 rsvd;
--
2.51.2.636.ga99f379adf
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH rdma-next v4 7/7] RDMA/bnxt_re: Enable app allocated QPs
2026-05-08 8:58 [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions Sriharsha Basavapatna
` (5 preceding siblings ...)
2026-05-08 8:58 ` [PATCH rdma-next v4 6/7] RDMA/bnxt_re: Support doorbells " Sriharsha Basavapatna
@ 2026-05-08 8:58 ` Sriharsha Basavapatna
2026-05-12 16:53 ` [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions Jason Gunthorpe
7 siblings, 0 replies; 9+ messages in thread
From: Sriharsha Basavapatna @ 2026-05-08 8:58 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
The driver supports a new comp_mask: REQ_MASK_FIXED_QUE_ATTR.
The application sets this comp_mask bit in the CREATE_QP ureq
to indicate direct control of the QP. The driver goes through
the required processing for app allocated QPs (previous patches).
Only variable WQE mode is supported for these QPs.
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 14 ++++++++++----
include/uapi/rdma/bnxt_re-abi.h | 1 +
2 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 5d36a2fc8a10..8ba157abc051 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -1721,11 +1721,11 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
struct ib_qp_init_attr *init_attr,
struct bnxt_re_ucontext *uctx,
struct bnxt_re_qp_req *ureq,
- struct bnxt_re_dbr_obj *dbr_obj)
+ struct bnxt_re_dbr_obj *dbr_obj,
+ bool fixed_que_attr)
{
struct bnxt_qplib_dev_attr *dev_attr;
struct bnxt_qplib_qp *qplqp;
- bool fixed_que_attr = false;
struct bnxt_re_dev *rdev;
struct bnxt_re_cq *cq;
int rc = 0, qptype;
@@ -1745,6 +1745,8 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
return qptype;
qplqp->type = (u8)qptype;
qplqp->wqe_mode = bnxt_re_is_var_size_supported(rdev, uctx);
+ if (fixed_que_attr && qplqp->wqe_mode != BNXT_QPLIB_WQE_MODE_VARIABLE)
+ return -EOPNOTSUPP;
qplqp->dev_cap_flags = dev_attr->dev_cap_flags;
qplqp->cctx = rdev->chip_ctx;
if (init_attr->qp_type == IB_QPT_RC) {
@@ -1929,6 +1931,7 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
struct bnxt_qplib_dev_attr *dev_attr;
struct uverbs_attr_bundle *attrs;
struct bnxt_re_ucontext *uctx;
+ bool fixed_que_attr = false;
struct bnxt_re_qp_req ureq;
struct bnxt_re_dev *rdev;
struct bnxt_re_pd *pd;
@@ -1945,7 +1948,8 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
uctx = rdma_udata_to_drv_context(udata, struct bnxt_re_ucontext, ib_uctx);
if (udata) {
- rc = ib_copy_validate_udata_in_cm(udata, ureq, qp_handle, 0);
+ rc = ib_copy_validate_udata_in_cm(udata, ureq, qp_handle,
+ BNXT_RE_QP_REQ_MASK_FIXED_QUE_ATTR);
if (rc)
return rc;
@@ -1959,6 +1963,8 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
atomic_inc(&dbr_obj->usecnt);
qp->dbr_obj = dbr_obj;
}
+ if (ureq.comp_mask & BNXT_RE_QP_REQ_MASK_FIXED_QUE_ATTR)
+ fixed_que_attr = true;
}
rc = bnxt_re_test_qp_limits(rdev, qp_init_attr, dev_attr);
@@ -1969,7 +1975,7 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
qp->rdev = rdev;
rc = bnxt_re_init_qp_attr(qp, pd, qp_init_attr, uctx, &ureq,
- dbr_obj);
+ dbr_obj, fixed_que_attr);
if (rc)
goto fail;
diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h
index 4da8cda337dc..dd6ebe3e2fd3 100644
--- a/include/uapi/rdma/bnxt_re-abi.h
+++ b/include/uapi/rdma/bnxt_re-abi.h
@@ -127,6 +127,7 @@ struct bnxt_re_resize_cq_req {
enum bnxt_re_qp_mask {
BNXT_RE_QP_REQ_MASK_VAR_WQE_SQ_SLOTS = 0x1,
+ BNXT_RE_QP_REQ_MASK_FIXED_QUE_ATTR = 0x2,
};
struct bnxt_re_qp_req {
--
2.51.2.636.ga99f379adf
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions
2026-05-08 8:58 [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions Sriharsha Basavapatna
` (6 preceding siblings ...)
2026-05-08 8:58 ` [PATCH rdma-next v4 7/7] RDMA/bnxt_re: Enable " Sriharsha Basavapatna
@ 2026-05-12 16:53 ` Jason Gunthorpe
7 siblings, 0 replies; 9+ messages in thread
From: Jason Gunthorpe @ 2026-05-12 16:53 UTC (permalink / raw)
To: Sriharsha Basavapatna
Cc: leon, linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil
On Fri, May 08, 2026 at 02:28:51PM +0530, Sriharsha Basavapatna wrote:
> Sriharsha Basavapatna (7):
> RDMA/bnxt_re: Refactor bnxt_re_init_user_qp()
> RDMA/bnxt_re: Update rq depth for app allocated QPs
> RDMA/bnxt_re: Update sq depth for app allocated QPs
> RDMA/bnxt_re: Update msn table size for app allocated QPs
> RDMA/bnxt_re: Update hwq depth for app allocated QPs
> RDMA/bnxt_re: Support doorbells for app allocated QPs
> RDMA/bnxt_re: Enable app allocated QPs
There are alot of scary sashiko warnings, I stopped at this one:
https://sashiko.dev/#/patchset/20260508085858.21060-1-sriharsha.basavapatna%40broadcom.com
If userspace passes 0 or 1 for req->sq_npsn, qplib_qp->msn_tbl_sz will
evaluate to 0.
When this QP is used, userspace could trigger the kernel's send path via
ib_uverbs_post_send(), which calls bnxt_re_post_send() and ultimately
bnxt_qplib_post_send().
In bnxt_qplib_fill_msn_search(), there is a modulo calculation:
qp->msn %= qp->msn_tbl_sz;
Could this unvalidated req->sq_npsn lead to a divide-by-zero kernel panic?
Please go through them all. Fix the things newly added in this series,
especially the uapi issues like the above
It points out a bunch of pre-existing warnings too, please prepare a
seperate series to fix them, it can go after this one.
In future please be proactive to check sahsiko.dev don't wait for us
to look on it.
Jason
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2026-05-12 16:53 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-05-08 8:58 [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 1/7] RDMA/bnxt_re: Refactor bnxt_re_init_user_qp() Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 2/7] RDMA/bnxt_re: Update rq depth for app allocated QPs Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 3/7] RDMA/bnxt_re: Update sq " Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 4/7] RDMA/bnxt_re: Update msn table size " Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 5/7] RDMA/bnxt_re: Update hwq depth " Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 6/7] RDMA/bnxt_re: Support doorbells " Sriharsha Basavapatna
2026-05-08 8:58 ` [PATCH rdma-next v4 7/7] RDMA/bnxt_re: Enable " Sriharsha Basavapatna
2026-05-12 16:53 ` [PATCH rdma-next v4 0/7] RDMA/bnxt_re: Support QP uapi extensions Jason Gunthorpe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox