* [PATCH rdma-next v12 0/6] RDMA/bnxt_re: Support uapi extensions
@ 2026-02-11 12:49 Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 1/6] RDMA/bnxt_re: Move the UAPI methods to a dedicated file Sriharsha Basavapatna
` (5 more replies)
0 siblings, 6 replies; 15+ messages in thread
From: Sriharsha Basavapatna @ 2026-02-11 12:49 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
Hi,
This patchset provides uapi extensions in the bnxt_re driver.
This is required by applications that need to manage some of the
RDMA HW resources directly and to implement the datapath in the
application.
To support this, the library and the driver are being enhanced to
provide uverb extensions using which the application can allocate and
manage the HW resources (Doorbell, Queues etc).
This patch series is based on the udata helper series here:
https://lore.kernel.org/linux-rdma/0-v1-89ea7d615ba4+636-bnxt_re_uapi_jgg@nvidia.com/
Patch#1 Move uapi methods to a separate file
Patch#2 Refactor existing bnxt_qplib_create_qp() function
Patch#3 Support doorbell extensions
Patch#4 Refactor bnxt_re_create_cq()
Patch#5 Support dmabuf for CQ rings
Patch#6 Support application specific CQs
Thanks,
-Harsha
******
Changes:
v12:
- Patch#6
- Added a separate comp_mask enum for req_cq.
- Renamed cqe comp_flag (BNXT_RE_CQ_FIXED_NUM_CQE_ENABLE).
- Pass comp_mask flags directly to the validation function.
- Deleted a couple of debug prints.
v11:
- Patch#1
- Rename dv.c->uapi.c.
- Patch#3 (DBR)
- Renamed abi enums/vars.
- Updated commit message.
- CQ patch
- Split into 3 separate patches (#4,5,6).
- Refactored functionality into standard CQ functions.
- Renamed abi enums/vars; deleted dv functions.
- Updated commit messages.
- Dropped QP patch from the series.
v10:
- Fixed comp_mask issues:
- Driver returns cmask capability in ucntx for CQ/QP.
- Driver checks req comp_mask for supported bitmasks.
- Driver returns -EOPNOTSUPP if invalid comp_mask.
- Fixed zero initialization of req/resp structures.
- Use PD object from QP; deleted pd_id in ureq.
- Deleted debug counters.
- Split CQ/QP changes into two separate patches (#5,#6).
v9:
- Added a new uverbs patch (#1) in RDMA core.
- Supports user/app allocated memory for QP.
- Updated Patch#5 (cq/qp) to utilize umem dev op.
- Updated driver ABI file (deleted dmabuf_fd/len fields).
v8:
- Patch#3:
- Removed dpi_hash table (and lock/rcu).
- Renamed bnxt_re_alloc_dbr_obj->bnxt_re_dbr_obj.
- Added an atomic usecnt in dbr_obj.
- Patch#4:
- Registered a driver specific attribute for dbr_handle.
- Process dbr_handle during QP creation.
- Added refcnt logic to avoid dbr deletion with active QPs.
- Reverted dpi hash table lookup and related code.
- Removed dpi from req_qp ABI.
- Added ib_umem_find_best_pgsz() in umem processing.
- Added a wrapper function for dv_cq deletion.
v7:
- Patch#3:
- DBR_OFFSET attribute changed to PTR_OUT.
- Added a reserved field in struct bnxt_re_dv_db_region.
- Reordered sequence in DBR_ALLOC (hash_add -> uverbs_finalize).
- Synchronized access to dpi hash table.
- Patch#4:
- Changed dmabuf_fd type (u32->s32) in ABI.
- Changed num_dma_blocks() arg from PAGE_SIZE to SZ_4K.
- Fixed atomic read/inc race window in bnxt_re_dv_create_qplib_cq().
- Deleted bnxt_re_dv_init_ib_cq().
v6:
- Minor updates in Patch#3:
- Removed unused variables.
- Renamed & updated a uverbs method to a global.
- Minor updates in Patch#4:
- Removed unused variables, stray hunks.
v5:
- Design changes to address previous round of comments:
- Reverted changes in rdma-core (removed V4-Patch#1).
- Removed driver support for umem-reg/dereg DVs (Patch#3).
- Enhanced driver specific udata to avoid new CQ/QP ioctls (Patch#4).
- Removed additional driver functions in modify/query QP (Patch#4).
- Utilized queue-va in udata for deferred pinning (Patch#4).
v4:
- Added a new (rdma core) patch.
- Addressed code review comments in patch 5.
v3:
- Addressed code review comments in patches 1, 2 and 4.
v2:
- Fixed build warnings reported by test robot in patches 3 and 4.
v11: https://lore.kernel.org/linux-rdma/20260210165939.41625-1-sriharsha.basavapatna@broadcom.com/
v10: https://lore.kernel.org/linux-rdma/20260203050049.171026-1-sriharsha.basavapatna@broadcom.com/
v9: https://lore.kernel.org/linux-rdma/20260127103109.32163-1-sriharsha.basavapatna@broadcom.com/
v8: https://lore.kernel.org/linux-rdma/20260117080052.43279-1-sriharsha.basavapatna@broadcom.com/
v7: https://lore.kernel.org/linux-rdma/20260113170956.103779-1-sriharsha.basavapatna@broadcom.com/
v6: https://lore.kernel.org/linux-rdma/20251224042602.56255-1-sriharsha.basavapatna@broadcom.com/
v5: https://lore.kernel.org/linux-rdma/20251129165441.75274-1-sriharsha.basavapatna@broadcom.com/
v4: https://lore.kernel.org/linux-rdma/20251117061741.15752-1-sriharsha.basavapatna@broadcom.com/
v3: https://lore.kernel.org/linux-rdma/20251110145628.290296-1-sriharsha.basavapatna@broadcom.com/
v2: https://lore.kernel.org/linux-rdma/20251104072320.210596-1-sriharsha.basavapatna@broadcom.com/
v1: https://lore.kernel.org/linux-rdma/20251103105033.205586-1-sriharsha.basavapatna@broadcom.com/
******
Kalesh AP (3):
RDMA/bnxt_re: Move the UAPI methods to a dedicated file
RDMA/bnxt_re: Refactor bnxt_qplib_create_qp() function
RDMA/bnxt_re: Support doorbell extensions
Sriharsha Basavapatna (3):
RDMA/bnxt_re: Refactor bnxt_re_create_cq()
RDMA/bnxt_re: Support dmabuf for CQ rings
RDMA/bnxt_re: Support application specific CQs
drivers/infiniband/hw/bnxt_re/Makefile | 2 +-
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 574 +++++++++-------------
drivers/infiniband/hw/bnxt_re/ib_verbs.h | 12 +
drivers/infiniband/hw/bnxt_re/main.c | 1 +
drivers/infiniband/hw/bnxt_re/qplib_fp.c | 305 ++++--------
drivers/infiniband/hw/bnxt_re/qplib_fp.h | 8 +
drivers/infiniband/hw/bnxt_re/qplib_res.c | 43 ++
drivers/infiniband/hw/bnxt_re/qplib_res.h | 10 +
drivers/infiniband/hw/bnxt_re/uapi.c | 469 ++++++++++++++++++
include/uapi/rdma/bnxt_re-abi.h | 36 +-
10 files changed, 914 insertions(+), 546 deletions(-)
create mode 100644 drivers/infiniband/hw/bnxt_re/uapi.c
--
2.51.2.636.ga99f379adf
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH rdma-next v12 1/6] RDMA/bnxt_re: Move the UAPI methods to a dedicated file
2026-02-11 12:49 [PATCH rdma-next v12 0/6] RDMA/bnxt_re: Support uapi extensions Sriharsha Basavapatna
@ 2026-02-11 12:49 ` Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 2/6] RDMA/bnxt_re: Refactor bnxt_qplib_create_qp() function Sriharsha Basavapatna
` (4 subsequent siblings)
5 siblings, 0 replies; 15+ messages in thread
From: Sriharsha Basavapatna @ 2026-02-11 12:49 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
This is in preparation for upcoming patches in the series.
Driver has to support additional UAPIs for some applications.
Moving current UAPI implementation to a new file, uapi.c.
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
---
drivers/infiniband/hw/bnxt_re/Makefile | 2 +-
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 305 +-------------------
drivers/infiniband/hw/bnxt_re/ib_verbs.h | 3 +
drivers/infiniband/hw/bnxt_re/uapi.c | 339 +++++++++++++++++++++++
4 files changed, 344 insertions(+), 305 deletions(-)
create mode 100644 drivers/infiniband/hw/bnxt_re/uapi.c
diff --git a/drivers/infiniband/hw/bnxt_re/Makefile b/drivers/infiniband/hw/bnxt_re/Makefile
index f63417d2ccc6..1533c079c9da 100644
--- a/drivers/infiniband/hw/bnxt_re/Makefile
+++ b/drivers/infiniband/hw/bnxt_re/Makefile
@@ -5,4 +5,4 @@ obj-$(CONFIG_INFINIBAND_BNXT_RE) += bnxt_re.o
bnxt_re-y := main.o ib_verbs.o \
qplib_res.o qplib_rcfw.o \
qplib_sp.o qplib_fp.o hw_counters.o \
- debugfs.o
+ debugfs.o uapi.o
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 3c39469587f3..76f198390467 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -647,7 +647,7 @@ static int bnxt_re_create_fence_mr(struct bnxt_re_pd *pd)
return rc;
}
-static struct bnxt_re_user_mmap_entry*
+struct bnxt_re_user_mmap_entry*
bnxt_re_mmap_entry_insert(struct bnxt_re_ucontext *uctx, u64 mem_offset,
enum bnxt_re_mmap_flag mmap_flag, u64 *offset)
{
@@ -4607,32 +4607,6 @@ int bnxt_re_destroy_flow(struct ib_flow *flow_id)
return rc;
}
-static struct bnxt_re_cq *bnxt_re_search_for_cq(struct bnxt_re_dev *rdev, u32 cq_id)
-{
- struct bnxt_re_cq *cq = NULL, *tmp_cq;
-
- hash_for_each_possible(rdev->cq_hash, tmp_cq, hash_entry, cq_id) {
- if (tmp_cq->qplib_cq.id == cq_id) {
- cq = tmp_cq;
- break;
- }
- }
- return cq;
-}
-
-static struct bnxt_re_srq *bnxt_re_search_for_srq(struct bnxt_re_dev *rdev, u32 srq_id)
-{
- struct bnxt_re_srq *srq = NULL, *tmp_srq;
-
- hash_for_each_possible(rdev->srq_hash, tmp_srq, hash_entry, srq_id) {
- if (tmp_srq->qplib_srq.id == srq_id) {
- srq = tmp_srq;
- break;
- }
- }
- return srq;
-}
-
/* Helper function to mmap the virtual memory from user app */
int bnxt_re_mmap(struct ib_ucontext *ib_uctx, struct vm_area_struct *vma)
{
@@ -4735,280 +4709,3 @@ int bnxt_re_process_mad(struct ib_device *ibdev, int mad_flags,
ret |= IB_MAD_RESULT_REPLY;
return ret;
}
-
-static int UVERBS_HANDLER(BNXT_RE_METHOD_NOTIFY_DRV)(struct uverbs_attr_bundle *attrs)
-{
- struct bnxt_re_ucontext *uctx;
-
- uctx = container_of(ib_uverbs_get_ucontext(attrs), struct bnxt_re_ucontext, ib_uctx);
- bnxt_re_pacing_alert(uctx->rdev);
- return 0;
-}
-
-static int UVERBS_HANDLER(BNXT_RE_METHOD_ALLOC_PAGE)(struct uverbs_attr_bundle *attrs)
-{
- struct ib_uobject *uobj = uverbs_attr_get_uobject(attrs, BNXT_RE_ALLOC_PAGE_HANDLE);
- enum bnxt_re_alloc_page_type alloc_type;
- struct bnxt_re_user_mmap_entry *entry;
- enum bnxt_re_mmap_flag mmap_flag;
- struct bnxt_qplib_chip_ctx *cctx;
- struct bnxt_re_ucontext *uctx;
- struct bnxt_re_dev *rdev;
- u64 mmap_offset;
- u32 length;
- u32 dpi;
- u64 addr;
- int err;
-
- uctx = container_of(ib_uverbs_get_ucontext(attrs), struct bnxt_re_ucontext, ib_uctx);
- if (IS_ERR(uctx))
- return PTR_ERR(uctx);
-
- err = uverbs_get_const(&alloc_type, attrs, BNXT_RE_ALLOC_PAGE_TYPE);
- if (err)
- return err;
-
- rdev = uctx->rdev;
- cctx = rdev->chip_ctx;
-
- switch (alloc_type) {
- case BNXT_RE_ALLOC_WC_PAGE:
- if (cctx->modes.db_push) {
- if (bnxt_qplib_alloc_dpi(&rdev->qplib_res, &uctx->wcdpi,
- uctx, BNXT_QPLIB_DPI_TYPE_WC))
- return -ENOMEM;
- length = PAGE_SIZE;
- dpi = uctx->wcdpi.dpi;
- addr = (u64)uctx->wcdpi.umdbr;
- mmap_flag = BNXT_RE_MMAP_WC_DB;
- } else {
- return -EINVAL;
- }
-
- break;
- case BNXT_RE_ALLOC_DBR_BAR_PAGE:
- length = PAGE_SIZE;
- addr = (u64)rdev->pacing.dbr_bar_addr;
- mmap_flag = BNXT_RE_MMAP_DBR_BAR;
- break;
-
- case BNXT_RE_ALLOC_DBR_PAGE:
- length = PAGE_SIZE;
- addr = (u64)rdev->pacing.dbr_page;
- mmap_flag = BNXT_RE_MMAP_DBR_PAGE;
- break;
-
- default:
- return -EOPNOTSUPP;
- }
-
- entry = bnxt_re_mmap_entry_insert(uctx, addr, mmap_flag, &mmap_offset);
- if (!entry)
- return -ENOMEM;
-
- uobj->object = entry;
- uverbs_finalize_uobj_create(attrs, BNXT_RE_ALLOC_PAGE_HANDLE);
- err = uverbs_copy_to(attrs, BNXT_RE_ALLOC_PAGE_MMAP_OFFSET,
- &mmap_offset, sizeof(mmap_offset));
- if (err)
- return err;
-
- err = uverbs_copy_to(attrs, BNXT_RE_ALLOC_PAGE_MMAP_LENGTH,
- &length, sizeof(length));
- if (err)
- return err;
-
- err = uverbs_copy_to(attrs, BNXT_RE_ALLOC_PAGE_DPI,
- &dpi, sizeof(dpi));
- if (err)
- return err;
-
- return 0;
-}
-
-static int alloc_page_obj_cleanup(struct ib_uobject *uobject,
- enum rdma_remove_reason why,
- struct uverbs_attr_bundle *attrs)
-{
- struct bnxt_re_user_mmap_entry *entry = uobject->object;
- struct bnxt_re_ucontext *uctx = entry->uctx;
-
- switch (entry->mmap_flag) {
- case BNXT_RE_MMAP_WC_DB:
- if (uctx && uctx->wcdpi.dbr) {
- struct bnxt_re_dev *rdev = uctx->rdev;
-
- bnxt_qplib_dealloc_dpi(&rdev->qplib_res, &uctx->wcdpi);
- uctx->wcdpi.dbr = NULL;
- }
- break;
- case BNXT_RE_MMAP_DBR_BAR:
- case BNXT_RE_MMAP_DBR_PAGE:
- break;
- default:
- goto exit;
- }
- rdma_user_mmap_entry_remove(&entry->rdma_entry);
-exit:
- return 0;
-}
-
-DECLARE_UVERBS_NAMED_METHOD(BNXT_RE_METHOD_ALLOC_PAGE,
- UVERBS_ATTR_IDR(BNXT_RE_ALLOC_PAGE_HANDLE,
- BNXT_RE_OBJECT_ALLOC_PAGE,
- UVERBS_ACCESS_NEW,
- UA_MANDATORY),
- UVERBS_ATTR_CONST_IN(BNXT_RE_ALLOC_PAGE_TYPE,
- enum bnxt_re_alloc_page_type,
- UA_MANDATORY),
- UVERBS_ATTR_PTR_OUT(BNXT_RE_ALLOC_PAGE_MMAP_OFFSET,
- UVERBS_ATTR_TYPE(u64),
- UA_MANDATORY),
- UVERBS_ATTR_PTR_OUT(BNXT_RE_ALLOC_PAGE_MMAP_LENGTH,
- UVERBS_ATTR_TYPE(u32),
- UA_MANDATORY),
- UVERBS_ATTR_PTR_OUT(BNXT_RE_ALLOC_PAGE_DPI,
- UVERBS_ATTR_TYPE(u32),
- UA_MANDATORY));
-
-DECLARE_UVERBS_NAMED_METHOD_DESTROY(BNXT_RE_METHOD_DESTROY_PAGE,
- UVERBS_ATTR_IDR(BNXT_RE_DESTROY_PAGE_HANDLE,
- BNXT_RE_OBJECT_ALLOC_PAGE,
- UVERBS_ACCESS_DESTROY,
- UA_MANDATORY));
-
-DECLARE_UVERBS_NAMED_OBJECT(BNXT_RE_OBJECT_ALLOC_PAGE,
- UVERBS_TYPE_ALLOC_IDR(alloc_page_obj_cleanup),
- &UVERBS_METHOD(BNXT_RE_METHOD_ALLOC_PAGE),
- &UVERBS_METHOD(BNXT_RE_METHOD_DESTROY_PAGE));
-
-DECLARE_UVERBS_NAMED_METHOD(BNXT_RE_METHOD_NOTIFY_DRV);
-
-DECLARE_UVERBS_GLOBAL_METHODS(BNXT_RE_OBJECT_NOTIFY_DRV,
- &UVERBS_METHOD(BNXT_RE_METHOD_NOTIFY_DRV));
-
-/* Toggle MEM */
-static int UVERBS_HANDLER(BNXT_RE_METHOD_GET_TOGGLE_MEM)(struct uverbs_attr_bundle *attrs)
-{
- struct ib_uobject *uobj = uverbs_attr_get_uobject(attrs, BNXT_RE_TOGGLE_MEM_HANDLE);
- enum bnxt_re_mmap_flag mmap_flag = BNXT_RE_MMAP_TOGGLE_PAGE;
- enum bnxt_re_get_toggle_mem_type res_type;
- struct bnxt_re_user_mmap_entry *entry;
- struct bnxt_re_ucontext *uctx;
- struct ib_ucontext *ib_uctx;
- struct bnxt_re_dev *rdev;
- struct bnxt_re_srq *srq;
- u32 length = PAGE_SIZE;
- struct bnxt_re_cq *cq;
- u64 mem_offset;
- u32 offset = 0;
- u64 addr = 0;
- u32 res_id;
- int err;
-
- ib_uctx = ib_uverbs_get_ucontext(attrs);
- if (IS_ERR(ib_uctx))
- return PTR_ERR(ib_uctx);
-
- err = uverbs_get_const(&res_type, attrs, BNXT_RE_TOGGLE_MEM_TYPE);
- if (err)
- return err;
-
- uctx = container_of(ib_uctx, struct bnxt_re_ucontext, ib_uctx);
- rdev = uctx->rdev;
- err = uverbs_copy_from(&res_id, attrs, BNXT_RE_TOGGLE_MEM_RES_ID);
- if (err)
- return err;
-
- switch (res_type) {
- case BNXT_RE_CQ_TOGGLE_MEM:
- cq = bnxt_re_search_for_cq(rdev, res_id);
- if (!cq)
- return -EINVAL;
-
- addr = (u64)cq->uctx_cq_page;
- break;
- case BNXT_RE_SRQ_TOGGLE_MEM:
- srq = bnxt_re_search_for_srq(rdev, res_id);
- if (!srq)
- return -EINVAL;
-
- addr = (u64)srq->uctx_srq_page;
- break;
-
- default:
- return -EOPNOTSUPP;
- }
-
- entry = bnxt_re_mmap_entry_insert(uctx, addr, mmap_flag, &mem_offset);
- if (!entry)
- return -ENOMEM;
-
- uobj->object = entry;
- uverbs_finalize_uobj_create(attrs, BNXT_RE_TOGGLE_MEM_HANDLE);
- err = uverbs_copy_to(attrs, BNXT_RE_TOGGLE_MEM_MMAP_PAGE,
- &mem_offset, sizeof(mem_offset));
- if (err)
- return err;
-
- err = uverbs_copy_to(attrs, BNXT_RE_TOGGLE_MEM_MMAP_LENGTH,
- &length, sizeof(length));
- if (err)
- return err;
-
- err = uverbs_copy_to(attrs, BNXT_RE_TOGGLE_MEM_MMAP_OFFSET,
- &offset, sizeof(offset));
- if (err)
- return err;
-
- return 0;
-}
-
-static int get_toggle_mem_obj_cleanup(struct ib_uobject *uobject,
- enum rdma_remove_reason why,
- struct uverbs_attr_bundle *attrs)
-{
- struct bnxt_re_user_mmap_entry *entry = uobject->object;
-
- rdma_user_mmap_entry_remove(&entry->rdma_entry);
- return 0;
-}
-
-DECLARE_UVERBS_NAMED_METHOD(BNXT_RE_METHOD_GET_TOGGLE_MEM,
- UVERBS_ATTR_IDR(BNXT_RE_TOGGLE_MEM_HANDLE,
- BNXT_RE_OBJECT_GET_TOGGLE_MEM,
- UVERBS_ACCESS_NEW,
- UA_MANDATORY),
- UVERBS_ATTR_CONST_IN(BNXT_RE_TOGGLE_MEM_TYPE,
- enum bnxt_re_get_toggle_mem_type,
- UA_MANDATORY),
- UVERBS_ATTR_PTR_IN(BNXT_RE_TOGGLE_MEM_RES_ID,
- UVERBS_ATTR_TYPE(u32),
- UA_MANDATORY),
- UVERBS_ATTR_PTR_OUT(BNXT_RE_TOGGLE_MEM_MMAP_PAGE,
- UVERBS_ATTR_TYPE(u64),
- UA_MANDATORY),
- UVERBS_ATTR_PTR_OUT(BNXT_RE_TOGGLE_MEM_MMAP_OFFSET,
- UVERBS_ATTR_TYPE(u32),
- UA_MANDATORY),
- UVERBS_ATTR_PTR_OUT(BNXT_RE_TOGGLE_MEM_MMAP_LENGTH,
- UVERBS_ATTR_TYPE(u32),
- UA_MANDATORY));
-
-DECLARE_UVERBS_NAMED_METHOD_DESTROY(BNXT_RE_METHOD_RELEASE_TOGGLE_MEM,
- UVERBS_ATTR_IDR(BNXT_RE_RELEASE_TOGGLE_MEM_HANDLE,
- BNXT_RE_OBJECT_GET_TOGGLE_MEM,
- UVERBS_ACCESS_DESTROY,
- UA_MANDATORY));
-
-DECLARE_UVERBS_NAMED_OBJECT(BNXT_RE_OBJECT_GET_TOGGLE_MEM,
- UVERBS_TYPE_ALLOC_IDR(get_toggle_mem_obj_cleanup),
- &UVERBS_METHOD(BNXT_RE_METHOD_GET_TOGGLE_MEM),
- &UVERBS_METHOD(BNXT_RE_METHOD_RELEASE_TOGGLE_MEM));
-
-const struct uapi_definition bnxt_re_uapi_defs[] = {
- UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_ALLOC_PAGE),
- UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_NOTIFY_DRV),
- UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_GET_TOGGLE_MEM),
- {}
-};
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
index 76ba9ab04d5c..a11f56730a31 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
@@ -293,4 +293,7 @@ static inline u32 __to_ib_port_num(u16 port_id)
unsigned long bnxt_re_lock_cqs(struct bnxt_re_qp *qp);
void bnxt_re_unlock_cqs(struct bnxt_re_qp *qp, unsigned long flags);
+struct bnxt_re_user_mmap_entry*
+bnxt_re_mmap_entry_insert(struct bnxt_re_ucontext *uctx, u64 mem_offset,
+ enum bnxt_re_mmap_flag mmap_flag, u64 *offset);
#endif /* __BNXT_RE_IB_VERBS_H__ */
diff --git a/drivers/infiniband/hw/bnxt_re/uapi.c b/drivers/infiniband/hw/bnxt_re/uapi.c
new file mode 100644
index 000000000000..0145882e49f6
--- /dev/null
+++ b/drivers/infiniband/hw/bnxt_re/uapi.c
@@ -0,0 +1,339 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * Copyright (c) 2025, Broadcom. All rights reserved. The term
+ * Broadcom refers to Broadcom Limited and/or its subsidiaries.
+ *
+ * Description: uapi interpreter
+ */
+
+#include <rdma/ib_addr.h>
+#include <rdma/uverbs_types.h>
+#include <rdma/uverbs_std_types.h>
+#include <rdma/ib_user_ioctl_cmds.h>
+#define UVERBS_MODULE_NAME bnxt_re
+#include <rdma/uverbs_named_ioctl.h>
+#include <rdma/bnxt_re-abi.h>
+
+#include "roce_hsi.h"
+#include "qplib_res.h"
+#include "qplib_sp.h"
+#include "qplib_fp.h"
+#include "qplib_rcfw.h"
+#include "bnxt_re.h"
+#include "ib_verbs.h"
+
+static struct bnxt_re_cq *bnxt_re_search_for_cq(struct bnxt_re_dev *rdev, u32 cq_id)
+{
+ struct bnxt_re_cq *cq = NULL, *tmp_cq;
+
+ hash_for_each_possible(rdev->cq_hash, tmp_cq, hash_entry, cq_id) {
+ if (tmp_cq->qplib_cq.id == cq_id) {
+ cq = tmp_cq;
+ break;
+ }
+ }
+ return cq;
+}
+
+static struct bnxt_re_srq *bnxt_re_search_for_srq(struct bnxt_re_dev *rdev, u32 srq_id)
+{
+ struct bnxt_re_srq *srq = NULL, *tmp_srq;
+
+ hash_for_each_possible(rdev->srq_hash, tmp_srq, hash_entry, srq_id) {
+ if (tmp_srq->qplib_srq.id == srq_id) {
+ srq = tmp_srq;
+ break;
+ }
+ }
+ return srq;
+}
+
+static int UVERBS_HANDLER(BNXT_RE_METHOD_NOTIFY_DRV)(struct uverbs_attr_bundle *attrs)
+{
+ struct bnxt_re_ucontext *uctx;
+ struct ib_ucontext *ib_uctx;
+
+ ib_uctx = ib_uverbs_get_ucontext(attrs);
+ if (IS_ERR(ib_uctx))
+ return PTR_ERR(ib_uctx);
+
+ uctx = container_of(ib_uctx, struct bnxt_re_ucontext, ib_uctx);
+ if (IS_ERR(uctx))
+ return PTR_ERR(uctx);
+
+ bnxt_re_pacing_alert(uctx->rdev);
+ return 0;
+}
+
+static int UVERBS_HANDLER(BNXT_RE_METHOD_ALLOC_PAGE)(struct uverbs_attr_bundle *attrs)
+{
+ struct ib_uobject *uobj = uverbs_attr_get_uobject(attrs, BNXT_RE_ALLOC_PAGE_HANDLE);
+ enum bnxt_re_alloc_page_type alloc_type;
+ struct bnxt_re_user_mmap_entry *entry;
+ enum bnxt_re_mmap_flag mmap_flag;
+ struct bnxt_qplib_chip_ctx *cctx;
+ struct bnxt_re_ucontext *uctx;
+ struct ib_ucontext *ib_uctx;
+ struct bnxt_re_dev *rdev;
+ u64 mmap_offset;
+ u32 length;
+ u32 dpi;
+ u64 addr;
+ int err;
+
+ ib_uctx = ib_uverbs_get_ucontext(attrs);
+ if (IS_ERR(ib_uctx))
+ return PTR_ERR(ib_uctx);
+
+ uctx = container_of(ib_uctx, struct bnxt_re_ucontext, ib_uctx);
+ if (IS_ERR(uctx))
+ return PTR_ERR(uctx);
+
+ err = uverbs_get_const(&alloc_type, attrs, BNXT_RE_ALLOC_PAGE_TYPE);
+ if (err)
+ return err;
+
+ rdev = uctx->rdev;
+ cctx = rdev->chip_ctx;
+
+ switch (alloc_type) {
+ case BNXT_RE_ALLOC_WC_PAGE:
+ if (cctx->modes.db_push) {
+ if (bnxt_qplib_alloc_dpi(&rdev->qplib_res, &uctx->wcdpi,
+ uctx, BNXT_QPLIB_DPI_TYPE_WC))
+ return -ENOMEM;
+ length = PAGE_SIZE;
+ dpi = uctx->wcdpi.dpi;
+ addr = (u64)uctx->wcdpi.umdbr;
+ mmap_flag = BNXT_RE_MMAP_WC_DB;
+ } else {
+ return -EINVAL;
+ }
+
+ break;
+ case BNXT_RE_ALLOC_DBR_BAR_PAGE:
+ length = PAGE_SIZE;
+ addr = (u64)rdev->pacing.dbr_bar_addr;
+ mmap_flag = BNXT_RE_MMAP_DBR_BAR;
+ break;
+
+ case BNXT_RE_ALLOC_DBR_PAGE:
+ length = PAGE_SIZE;
+ addr = (u64)rdev->pacing.dbr_page;
+ mmap_flag = BNXT_RE_MMAP_DBR_PAGE;
+ break;
+
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ entry = bnxt_re_mmap_entry_insert(uctx, addr, mmap_flag, &mmap_offset);
+ if (!entry)
+ return -ENOMEM;
+
+ uobj->object = entry;
+ uverbs_finalize_uobj_create(attrs, BNXT_RE_ALLOC_PAGE_HANDLE);
+ err = uverbs_copy_to(attrs, BNXT_RE_ALLOC_PAGE_MMAP_OFFSET,
+ &mmap_offset, sizeof(mmap_offset));
+ if (err)
+ return err;
+
+ err = uverbs_copy_to(attrs, BNXT_RE_ALLOC_PAGE_MMAP_LENGTH,
+ &length, sizeof(length));
+ if (err)
+ return err;
+
+ err = uverbs_copy_to(attrs, BNXT_RE_ALLOC_PAGE_DPI,
+ &dpi, sizeof(dpi));
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int alloc_page_obj_cleanup(struct ib_uobject *uobject,
+ enum rdma_remove_reason why,
+ struct uverbs_attr_bundle *attrs)
+{
+ struct bnxt_re_user_mmap_entry *entry = uobject->object;
+ struct bnxt_re_ucontext *uctx = entry->uctx;
+
+ switch (entry->mmap_flag) {
+ case BNXT_RE_MMAP_WC_DB:
+ if (uctx && uctx->wcdpi.dbr) {
+ struct bnxt_re_dev *rdev = uctx->rdev;
+
+ bnxt_qplib_dealloc_dpi(&rdev->qplib_res, &uctx->wcdpi);
+ uctx->wcdpi.dbr = NULL;
+ }
+ break;
+ case BNXT_RE_MMAP_DBR_BAR:
+ case BNXT_RE_MMAP_DBR_PAGE:
+ break;
+ default:
+ goto exit;
+ }
+ rdma_user_mmap_entry_remove(&entry->rdma_entry);
+exit:
+ return 0;
+}
+
+DECLARE_UVERBS_NAMED_METHOD(BNXT_RE_METHOD_ALLOC_PAGE,
+ UVERBS_ATTR_IDR(BNXT_RE_ALLOC_PAGE_HANDLE,
+ BNXT_RE_OBJECT_ALLOC_PAGE,
+ UVERBS_ACCESS_NEW,
+ UA_MANDATORY),
+ UVERBS_ATTR_CONST_IN(BNXT_RE_ALLOC_PAGE_TYPE,
+ enum bnxt_re_alloc_page_type,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(BNXT_RE_ALLOC_PAGE_MMAP_OFFSET,
+ UVERBS_ATTR_TYPE(u64),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(BNXT_RE_ALLOC_PAGE_MMAP_LENGTH,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(BNXT_RE_ALLOC_PAGE_DPI,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_METHOD_DESTROY(BNXT_RE_METHOD_DESTROY_PAGE,
+ UVERBS_ATTR_IDR(BNXT_RE_DESTROY_PAGE_HANDLE,
+ BNXT_RE_OBJECT_ALLOC_PAGE,
+ UVERBS_ACCESS_DESTROY,
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_OBJECT(BNXT_RE_OBJECT_ALLOC_PAGE,
+ UVERBS_TYPE_ALLOC_IDR(alloc_page_obj_cleanup),
+ &UVERBS_METHOD(BNXT_RE_METHOD_ALLOC_PAGE),
+ &UVERBS_METHOD(BNXT_RE_METHOD_DESTROY_PAGE));
+
+DECLARE_UVERBS_NAMED_METHOD(BNXT_RE_METHOD_NOTIFY_DRV);
+
+DECLARE_UVERBS_GLOBAL_METHODS(BNXT_RE_OBJECT_NOTIFY_DRV,
+ &UVERBS_METHOD(BNXT_RE_METHOD_NOTIFY_DRV));
+
+/* Toggle MEM */
+static int UVERBS_HANDLER(BNXT_RE_METHOD_GET_TOGGLE_MEM)(struct uverbs_attr_bundle *attrs)
+{
+ struct ib_uobject *uobj = uverbs_attr_get_uobject(attrs, BNXT_RE_TOGGLE_MEM_HANDLE);
+ enum bnxt_re_mmap_flag mmap_flag = BNXT_RE_MMAP_TOGGLE_PAGE;
+ enum bnxt_re_get_toggle_mem_type res_type;
+ struct bnxt_re_user_mmap_entry *entry;
+ struct bnxt_re_ucontext *uctx;
+ struct ib_ucontext *ib_uctx;
+ struct bnxt_re_dev *rdev;
+ struct bnxt_re_srq *srq;
+ u32 length = PAGE_SIZE;
+ struct bnxt_re_cq *cq;
+ u64 mem_offset;
+ u32 offset = 0;
+ u64 addr = 0;
+ u32 res_id;
+ int err;
+
+ ib_uctx = ib_uverbs_get_ucontext(attrs);
+ if (IS_ERR(ib_uctx))
+ return PTR_ERR(ib_uctx);
+
+ err = uverbs_get_const(&res_type, attrs, BNXT_RE_TOGGLE_MEM_TYPE);
+ if (err)
+ return err;
+
+ uctx = container_of(ib_uctx, struct bnxt_re_ucontext, ib_uctx);
+ rdev = uctx->rdev;
+ err = uverbs_copy_from(&res_id, attrs, BNXT_RE_TOGGLE_MEM_RES_ID);
+ if (err)
+ return err;
+
+ switch (res_type) {
+ case BNXT_RE_CQ_TOGGLE_MEM:
+ cq = bnxt_re_search_for_cq(rdev, res_id);
+ if (!cq)
+ return -EINVAL;
+
+ addr = (u64)cq->uctx_cq_page;
+ break;
+ case BNXT_RE_SRQ_TOGGLE_MEM:
+ srq = bnxt_re_search_for_srq(rdev, res_id);
+ if (!srq)
+ return -EINVAL;
+
+ addr = (u64)srq->uctx_srq_page;
+ break;
+
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ entry = bnxt_re_mmap_entry_insert(uctx, addr, mmap_flag, &mem_offset);
+ if (!entry)
+ return -ENOMEM;
+
+ uobj->object = entry;
+ uverbs_finalize_uobj_create(attrs, BNXT_RE_TOGGLE_MEM_HANDLE);
+ err = uverbs_copy_to(attrs, BNXT_RE_TOGGLE_MEM_MMAP_PAGE,
+ &mem_offset, sizeof(mem_offset));
+ if (err)
+ return err;
+
+ err = uverbs_copy_to(attrs, BNXT_RE_TOGGLE_MEM_MMAP_LENGTH,
+ &length, sizeof(length));
+ if (err)
+ return err;
+
+ err = uverbs_copy_to(attrs, BNXT_RE_TOGGLE_MEM_MMAP_OFFSET,
+ &offset, sizeof(offset));
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int get_toggle_mem_obj_cleanup(struct ib_uobject *uobject,
+ enum rdma_remove_reason why,
+ struct uverbs_attr_bundle *attrs)
+{
+ struct bnxt_re_user_mmap_entry *entry = uobject->object;
+
+ rdma_user_mmap_entry_remove(&entry->rdma_entry);
+ return 0;
+}
+
+DECLARE_UVERBS_NAMED_METHOD(BNXT_RE_METHOD_GET_TOGGLE_MEM,
+ UVERBS_ATTR_IDR(BNXT_RE_TOGGLE_MEM_HANDLE,
+ BNXT_RE_OBJECT_GET_TOGGLE_MEM,
+ UVERBS_ACCESS_NEW,
+ UA_MANDATORY),
+ UVERBS_ATTR_CONST_IN(BNXT_RE_TOGGLE_MEM_TYPE,
+ enum bnxt_re_get_toggle_mem_type,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_IN(BNXT_RE_TOGGLE_MEM_RES_ID,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(BNXT_RE_TOGGLE_MEM_MMAP_PAGE,
+ UVERBS_ATTR_TYPE(u64),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(BNXT_RE_TOGGLE_MEM_MMAP_OFFSET,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(BNXT_RE_TOGGLE_MEM_MMAP_LENGTH,
+ UVERBS_ATTR_TYPE(u32),
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_METHOD_DESTROY(BNXT_RE_METHOD_RELEASE_TOGGLE_MEM,
+ UVERBS_ATTR_IDR(BNXT_RE_RELEASE_TOGGLE_MEM_HANDLE,
+ BNXT_RE_OBJECT_GET_TOGGLE_MEM,
+ UVERBS_ACCESS_DESTROY,
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_OBJECT(BNXT_RE_OBJECT_GET_TOGGLE_MEM,
+ UVERBS_TYPE_ALLOC_IDR(get_toggle_mem_obj_cleanup),
+ &UVERBS_METHOD(BNXT_RE_METHOD_GET_TOGGLE_MEM),
+ &UVERBS_METHOD(BNXT_RE_METHOD_RELEASE_TOGGLE_MEM));
+
+const struct uapi_definition bnxt_re_uapi_defs[] = {
+ UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_ALLOC_PAGE),
+ UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_NOTIFY_DRV),
+ UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_GET_TOGGLE_MEM),
+ {}
+};
--
2.51.2.636.ga99f379adf
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH rdma-next v12 2/6] RDMA/bnxt_re: Refactor bnxt_qplib_create_qp() function
2026-02-11 12:49 [PATCH rdma-next v12 0/6] RDMA/bnxt_re: Support uapi extensions Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 1/6] RDMA/bnxt_re: Move the UAPI methods to a dedicated file Sriharsha Basavapatna
@ 2026-02-11 12:49 ` Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 3/6] RDMA/bnxt_re: Support doorbell extensions Sriharsha Basavapatna
` (3 subsequent siblings)
5 siblings, 0 replies; 15+ messages in thread
From: Sriharsha Basavapatna @ 2026-02-11 12:49 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Inside bnxt_qplib_create_qp(), driver currently is doing
a lot of things like allocating HWQ memory for SQ/RQ/ORRQ/IRRQ,
initializing few of qplib_qp fields etc.
Refactored the code such that all memory allocation for HWQs
have been moved to bnxt_re_init_qp_attr() function and inside
bnxt_qplib_create_qp() function just initialize the request
structure and issue the HWRM command to firmware.
Introduced couple of new functions bnxt_re_setup_qp_hwqs() and
bnxt_re_setup_qp_swqs() moved the hwq and swq memory allocation
logic there.
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
---
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 201 ++++++++++++--
drivers/infiniband/hw/bnxt_re/qplib_fp.c | 305 +++++++---------------
drivers/infiniband/hw/bnxt_re/qplib_fp.h | 8 +
drivers/infiniband/hw/bnxt_re/qplib_res.h | 6 +
4 files changed, 295 insertions(+), 225 deletions(-)
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 76f198390467..feb364e45e14 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -996,6 +996,12 @@ static void bnxt_re_del_unique_gid(struct bnxt_re_dev *rdev)
dev_err(rdev_to_dev(rdev), "Failed to delete unique GID, rc: %d\n", rc);
}
+static void bnxt_re_qp_free_umem(struct bnxt_re_qp *qp)
+{
+ ib_umem_release(qp->rumem);
+ ib_umem_release(qp->sumem);
+}
+
/* Queue Pairs */
int bnxt_re_destroy_qp(struct ib_qp *ib_qp, struct ib_udata *udata)
{
@@ -1041,8 +1047,7 @@ int bnxt_re_destroy_qp(struct ib_qp *ib_qp, struct ib_udata *udata)
if (qp->qplib_qp.type == CMDQ_CREATE_QP_TYPE_RAW_ETHERTYPE)
bnxt_re_del_unique_gid(rdev);
- ib_umem_release(qp->rumem);
- ib_umem_release(qp->sumem);
+ bnxt_re_qp_free_umem(qp);
/* Flush all the entries of notification queue associated with
* given qp.
@@ -1186,6 +1191,7 @@ static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
}
qplib_qp->dpi = &cntx->dpi;
+ qplib_qp->is_user = true;
return 0;
rqfail:
ib_umem_release(qp->sumem);
@@ -1243,6 +1249,114 @@ static struct bnxt_re_ah *bnxt_re_create_shadow_qp_ah
return NULL;
}
+static int bnxt_re_qp_alloc_init_xrrq(struct bnxt_re_qp *qp)
+{
+ struct bnxt_qplib_res *res = &qp->rdev->qplib_res;
+ struct bnxt_qplib_qp *qplib_qp = &qp->qplib_qp;
+ struct bnxt_qplib_hwq_attr hwq_attr = {};
+ struct bnxt_qplib_sg_info sginfo = {};
+ struct bnxt_qplib_hwq *irrq, *orrq;
+ int rc, req_size;
+
+ orrq = &qplib_qp->orrq;
+ orrq->max_elements =
+ ORD_LIMIT_TO_ORRQ_SLOTS(qplib_qp->max_rd_atomic);
+ req_size = orrq->max_elements *
+ BNXT_QPLIB_MAX_ORRQE_ENTRY_SIZE + PAGE_SIZE - 1;
+ req_size &= ~(PAGE_SIZE - 1);
+ sginfo.pgsize = req_size;
+ sginfo.pgshft = PAGE_SHIFT;
+
+ hwq_attr.res = res;
+ hwq_attr.sginfo = &sginfo;
+ hwq_attr.depth = orrq->max_elements;
+ hwq_attr.stride = BNXT_QPLIB_MAX_ORRQE_ENTRY_SIZE;
+ hwq_attr.aux_stride = 0;
+ hwq_attr.aux_depth = 0;
+ hwq_attr.type = HWQ_TYPE_CTX;
+ rc = bnxt_qplib_alloc_init_hwq(orrq, &hwq_attr);
+ if (rc)
+ return rc;
+
+ irrq = &qplib_qp->irrq;
+ irrq->max_elements =
+ IRD_LIMIT_TO_IRRQ_SLOTS(qplib_qp->max_dest_rd_atomic);
+ req_size = irrq->max_elements *
+ BNXT_QPLIB_MAX_IRRQE_ENTRY_SIZE + PAGE_SIZE - 1;
+ req_size &= ~(PAGE_SIZE - 1);
+ sginfo.pgsize = req_size;
+ hwq_attr.sginfo = &sginfo;
+ hwq_attr.depth = irrq->max_elements;
+ hwq_attr.stride = BNXT_QPLIB_MAX_IRRQE_ENTRY_SIZE;
+ rc = bnxt_qplib_alloc_init_hwq(irrq, &hwq_attr);
+ if (rc)
+ goto free_orrq_hwq;
+ return 0;
+free_orrq_hwq:
+ bnxt_qplib_free_hwq(res, orrq);
+ return rc;
+}
+
+static int bnxt_re_setup_qp_hwqs(struct bnxt_re_qp *qp)
+{
+ struct bnxt_qplib_res *res = &qp->rdev->qplib_res;
+ struct bnxt_qplib_qp *qplib_qp = &qp->qplib_qp;
+ struct bnxt_qplib_hwq_attr hwq_attr = {};
+ struct bnxt_qplib_q *sq = &qplib_qp->sq;
+ struct bnxt_qplib_q *rq = &qplib_qp->rq;
+ u8 wqe_mode = qplib_qp->wqe_mode;
+ u8 pg_sz_lvl;
+ int rc;
+
+ hwq_attr.res = res;
+ hwq_attr.sginfo = &sq->sg_info;
+ hwq_attr.stride = bnxt_qplib_get_stride();
+ hwq_attr.depth = bnxt_qplib_get_depth(sq, wqe_mode, true);
+ hwq_attr.aux_stride = qplib_qp->psn_sz;
+ hwq_attr.aux_depth = (qplib_qp->psn_sz) ?
+ bnxt_qplib_set_sq_size(sq, wqe_mode) : 0;
+ if (qplib_qp->is_host_msn_tbl && qplib_qp->psn_sz)
+ hwq_attr.aux_depth = qplib_qp->msn_tbl_sz;
+ hwq_attr.type = HWQ_TYPE_QUEUE;
+ rc = bnxt_qplib_alloc_init_hwq(&sq->hwq, &hwq_attr);
+ if (rc)
+ return rc;
+
+ pg_sz_lvl = bnxt_qplib_base_pg_size(&sq->hwq) << CMDQ_CREATE_QP_SQ_PG_SIZE_SFT;
+ pg_sz_lvl |= ((sq->hwq.level & CMDQ_CREATE_QP_SQ_LVL_MASK) <<
+ CMDQ_CREATE_QP_SQ_LVL_SFT);
+ sq->hwq.pg_sz_lvl = pg_sz_lvl;
+
+ hwq_attr.res = res;
+ hwq_attr.sginfo = &rq->sg_info;
+ hwq_attr.stride = bnxt_qplib_get_stride();
+ hwq_attr.depth = bnxt_qplib_get_depth(rq, qplib_qp->wqe_mode, false);
+ hwq_attr.aux_stride = 0;
+ hwq_attr.aux_depth = 0;
+ hwq_attr.type = HWQ_TYPE_QUEUE;
+ rc = bnxt_qplib_alloc_init_hwq(&rq->hwq, &hwq_attr);
+ if (rc)
+ goto free_sq_hwq;
+ pg_sz_lvl = bnxt_qplib_base_pg_size(&rq->hwq) <<
+ CMDQ_CREATE_QP_RQ_PG_SIZE_SFT;
+ pg_sz_lvl |= ((rq->hwq.level & CMDQ_CREATE_QP_RQ_LVL_MASK) <<
+ CMDQ_CREATE_QP_RQ_LVL_SFT);
+ rq->hwq.pg_sz_lvl = pg_sz_lvl;
+
+ if (qplib_qp->psn_sz) {
+ rc = bnxt_re_qp_alloc_init_xrrq(qp);
+ if (rc)
+ goto free_rq_hwq;
+ }
+
+ return 0;
+free_rq_hwq:
+ bnxt_qplib_free_hwq(res, &rq->hwq);
+free_sq_hwq:
+ bnxt_qplib_free_hwq(res, &sq->hwq);
+ return rc;
+}
+
static struct bnxt_re_qp *bnxt_re_create_shadow_qp
(struct bnxt_re_pd *pd,
struct bnxt_qplib_res *qp1_res,
@@ -1264,6 +1378,7 @@ static struct bnxt_re_qp *bnxt_re_create_shadow_qp
qp->qplib_qp.pd = &pd->qplib_pd;
qp->qplib_qp.qp_handle = (u64)(unsigned long)(&qp->qplib_qp);
qp->qplib_qp.type = IB_QPT_UD;
+ qp->qplib_qp.cctx = rdev->chip_ctx;
qp->qplib_qp.max_inline_data = 0;
qp->qplib_qp.sig_type = true;
@@ -1296,10 +1411,14 @@ static struct bnxt_re_qp *bnxt_re_create_shadow_qp
qp->qplib_qp.rq_hdr_buf_size = BNXT_QPLIB_MAX_GRH_HDR_SIZE_IPV6;
qp->qplib_qp.dpi = &rdev->dpi_privileged;
- rc = bnxt_qplib_create_qp(qp1_res, &qp->qplib_qp);
+ rc = bnxt_re_setup_qp_hwqs(qp);
if (rc)
goto fail;
+ rc = bnxt_qplib_create_qp(qp1_res, &qp->qplib_qp);
+ if (rc)
+ goto free_hwq;
+
spin_lock_init(&qp->sq_lock);
INIT_LIST_HEAD(&qp->list);
mutex_lock(&rdev->qp_lock);
@@ -1307,6 +1426,9 @@ static struct bnxt_re_qp *bnxt_re_create_shadow_qp
atomic_inc(&rdev->stats.res.qp_count);
mutex_unlock(&rdev->qp_lock);
return qp;
+
+free_hwq:
+ bnxt_qplib_free_qp_res(&rdev->qplib_res, &qp->qplib_qp);
fail:
kfree(qp);
return NULL;
@@ -1477,6 +1599,39 @@ static int bnxt_re_init_qp_type(struct bnxt_re_dev *rdev,
return qptype;
}
+static void bnxt_re_qp_calculate_msn_psn_size(struct bnxt_re_qp *qp)
+{
+ struct bnxt_qplib_qp *qplib_qp = &qp->qplib_qp;
+ struct bnxt_qplib_q *sq = &qplib_qp->sq;
+ struct bnxt_re_dev *rdev = qp->rdev;
+ u8 wqe_mode = qplib_qp->wqe_mode;
+
+ if (rdev->dev_attr)
+ qplib_qp->is_host_msn_tbl =
+ _is_host_msn_table(rdev->dev_attr->dev_cap_flags2);
+
+ if (qplib_qp->type == CMDQ_CREATE_QP_TYPE_RC) {
+ qplib_qp->psn_sz = bnxt_qplib_is_chip_gen_p5_p7(rdev->chip_ctx) ?
+ sizeof(struct sq_psn_search_ext) :
+ sizeof(struct sq_psn_search);
+ if (qplib_qp->is_host_msn_tbl) {
+ qplib_qp->psn_sz = sizeof(struct sq_msn_search);
+ qplib_qp->msn = 0;
+ }
+ }
+
+ /* Update msn tbl size */
+ if (qplib_qp->is_host_msn_tbl && qplib_qp->psn_sz) {
+ if (wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC)
+ qplib_qp->msn_tbl_sz =
+ roundup_pow_of_two(bnxt_qplib_set_sq_size(sq, wqe_mode));
+ else
+ qplib_qp->msn_tbl_sz =
+ roundup_pow_of_two(bnxt_qplib_set_sq_size(sq, wqe_mode)) / 2;
+ qplib_qp->msn = 0;
+ }
+}
+
static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
struct ib_qp_init_attr *init_attr,
struct bnxt_re_ucontext *uctx,
@@ -1499,12 +1654,12 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
qplqp->max_inline_data = init_attr->cap.max_inline_data;
qplqp->sig_type = init_attr->sq_sig_type == IB_SIGNAL_ALL_WR;
qptype = bnxt_re_init_qp_type(rdev, init_attr);
- if (qptype < 0) {
- rc = qptype;
- goto out;
- }
+ if (qptype < 0)
+ return qptype;
qplqp->type = (u8)qptype;
qplqp->wqe_mode = bnxt_re_is_var_size_supported(rdev, uctx);
+ qplqp->dev_cap_flags = dev_attr->dev_cap_flags;
+ qplqp->cctx = rdev->chip_ctx;
if (init_attr->qp_type == IB_QPT_RC) {
qplqp->max_rd_atomic = dev_attr->max_qp_rd_atom;
qplqp->max_dest_rd_atomic = dev_attr->max_qp_init_rd_atom;
@@ -1534,20 +1689,32 @@ static int bnxt_re_init_qp_attr(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
/* Setup RQ/SRQ */
rc = bnxt_re_init_rq_attr(qp, init_attr, uctx);
if (rc)
- goto out;
+ return rc;
if (init_attr->qp_type == IB_QPT_GSI)
bnxt_re_adjust_gsi_rq_attr(qp);
/* Setup SQ */
rc = bnxt_re_init_sq_attr(qp, init_attr, uctx, ureq);
if (rc)
- goto out;
+ return rc;
if (init_attr->qp_type == IB_QPT_GSI)
bnxt_re_adjust_gsi_sq_attr(qp, init_attr, uctx);
- if (uctx) /* This will update DPI and qp_handle */
+ if (uctx) { /* This will update DPI and qp_handle */
rc = bnxt_re_init_user_qp(rdev, pd, qp, uctx, ureq);
-out:
+ if (rc)
+ return rc;
+ }
+
+ bnxt_re_qp_calculate_msn_psn_size(qp);
+
+ rc = bnxt_re_setup_qp_hwqs(qp);
+ if (rc)
+ goto free_umem;
+
+ return 0;
+free_umem:
+ bnxt_re_qp_free_umem(qp);
return rc;
}
@@ -1605,6 +1772,7 @@ static int bnxt_re_create_gsi_qp(struct bnxt_re_qp *qp, struct bnxt_re_pd *pd,
rdev = qp->rdev;
qplqp = &qp->qplib_qp;
+ qplqp->cctx = rdev->chip_ctx;
qplqp->rq_hdr_buf_size = BNXT_QPLIB_MAX_QP1_RQ_HDR_SIZE_V2;
qplqp->sq_hdr_buf_size = BNXT_QPLIB_MAX_QP1_SQ_HDR_SIZE_V2;
@@ -1710,13 +1878,14 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
if (rc == -ENODEV)
goto qp_destroy;
if (rc)
- goto fail;
+ goto free_hwq;
} else {
rc = bnxt_qplib_create_qp(&rdev->qplib_res, &qp->qplib_qp);
if (rc) {
ibdev_err(&rdev->ibdev, "Failed to create HW QP");
- goto free_umem;
+ goto free_hwq;
}
+
if (udata) {
struct bnxt_re_qp_resp resp;
@@ -1767,9 +1936,9 @@ int bnxt_re_create_qp(struct ib_qp *ib_qp, struct ib_qp_init_attr *qp_init_attr,
return 0;
qp_destroy:
bnxt_qplib_destroy_qp(&rdev->qplib_res, &qp->qplib_qp);
-free_umem:
- ib_umem_release(qp->rumem);
- ib_umem_release(qp->sumem);
+free_hwq:
+ bnxt_qplib_free_qp_res(&rdev->qplib_res, &qp->qplib_qp);
+ bnxt_re_qp_free_umem(qp);
fail:
return rc;
}
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_fp.c b/drivers/infiniband/hw/bnxt_re/qplib_fp.c
index 3e44311bf939..0ddd4f21a083 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_fp.c
+++ b/drivers/infiniband/hw/bnxt_re/qplib_fp.c
@@ -793,8 +793,6 @@ int bnxt_qplib_post_srq_recv(struct bnxt_qplib_srq *srq,
return 0;
}
-/* QP */
-
static int bnxt_qplib_alloc_init_swq(struct bnxt_qplib_q *que)
{
int indx;
@@ -813,9 +811,71 @@ static int bnxt_qplib_alloc_init_swq(struct bnxt_qplib_q *que)
return 0;
}
+static int bnxt_re_setup_qp_swqs(struct bnxt_qplib_qp *qplqp)
+{
+ struct bnxt_qplib_q *sq = &qplqp->sq;
+ struct bnxt_qplib_q *rq = &qplqp->rq;
+ int rc;
+
+ if (qplqp->is_user)
+ return 0;
+
+ rc = bnxt_qplib_alloc_init_swq(sq);
+ if (rc)
+ return rc;
+
+ if (!qplqp->srq) {
+ rc = bnxt_qplib_alloc_init_swq(rq);
+ if (rc)
+ goto free_sq_swq;
+ }
+
+ return 0;
+free_sq_swq:
+ kfree(sq->swq);
+ return rc;
+}
+
+static void bnxt_qp_init_dbinfo(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp)
+{
+ struct bnxt_qplib_q *sq = &qp->sq;
+ struct bnxt_qplib_q *rq = &qp->rq;
+
+ sq->dbinfo.hwq = &sq->hwq;
+ sq->dbinfo.xid = qp->id;
+ sq->dbinfo.db = qp->dpi->dbr;
+ sq->dbinfo.max_slot = bnxt_qplib_set_sq_max_slot(qp->wqe_mode);
+ sq->dbinfo.flags = 0;
+ if (rq->max_wqe) {
+ rq->dbinfo.hwq = &rq->hwq;
+ rq->dbinfo.xid = qp->id;
+ rq->dbinfo.db = qp->dpi->dbr;
+ rq->dbinfo.max_slot = bnxt_qplib_set_rq_max_slot(rq->wqe_size);
+ rq->dbinfo.flags = 0;
+ }
+}
+
+static void bnxt_qplib_init_psn_ptr(struct bnxt_qplib_qp *qp, int size)
+{
+ struct bnxt_qplib_hwq *sq_hwq;
+ struct bnxt_qplib_q *sq;
+ u64 fpsne, psn_pg;
+ u16 indx_pad = 0;
+
+ sq = &qp->sq;
+ sq_hwq = &sq->hwq;
+ /* First psn entry */
+ fpsne = (u64)bnxt_qplib_get_qe(sq_hwq, sq_hwq->depth, &psn_pg);
+ if (!IS_ALIGNED(fpsne, PAGE_SIZE))
+ indx_pad = (fpsne & ~PAGE_MASK) / size;
+ sq_hwq->pad_pgofft = indx_pad;
+ sq_hwq->pad_pg = (u64 *)psn_pg;
+ sq_hwq->pad_stride = size;
+}
+
+/* QP */
int bnxt_qplib_create_qp1(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp)
{
- struct bnxt_qplib_hwq_attr hwq_attr = {};
struct bnxt_qplib_rcfw *rcfw = res->rcfw;
struct creq_create_qp1_resp resp = {};
struct bnxt_qplib_cmdqmsg msg = {};
@@ -824,7 +884,6 @@ int bnxt_qplib_create_qp1(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp)
struct cmdq_create_qp1 req = {};
struct bnxt_qplib_pbl *pbl;
u32 qp_flags = 0;
- u8 pg_sz_lvl;
u32 tbl_indx;
int rc;
@@ -838,26 +897,12 @@ int bnxt_qplib_create_qp1(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp)
req.qp_handle = cpu_to_le64(qp->qp_handle);
/* SQ */
- hwq_attr.res = res;
- hwq_attr.sginfo = &sq->sg_info;
- hwq_attr.stride = sizeof(struct sq_sge);
- hwq_attr.depth = bnxt_qplib_get_depth(sq, qp->wqe_mode, false);
- hwq_attr.type = HWQ_TYPE_QUEUE;
- rc = bnxt_qplib_alloc_init_hwq(&sq->hwq, &hwq_attr);
- if (rc)
- return rc;
+ sq->max_sw_wqe = bnxt_qplib_get_depth(sq, qp->wqe_mode, true);
+ req.sq_size = cpu_to_le32(sq->max_sw_wqe);
+ req.sq_pg_size_sq_lvl = sq->hwq.pg_sz_lvl;
- rc = bnxt_qplib_alloc_init_swq(sq);
- if (rc)
- goto fail_sq;
-
- req.sq_size = cpu_to_le32(bnxt_qplib_set_sq_size(sq, qp->wqe_mode));
pbl = &sq->hwq.pbl[PBL_LVL_0];
req.sq_pbl = cpu_to_le64(pbl->pg_map_arr[0]);
- pg_sz_lvl = (bnxt_qplib_base_pg_size(&sq->hwq) <<
- CMDQ_CREATE_QP1_SQ_PG_SIZE_SFT);
- pg_sz_lvl |= (sq->hwq.level & CMDQ_CREATE_QP1_SQ_LVL_MASK);
- req.sq_pg_size_sq_lvl = pg_sz_lvl;
req.sq_fwo_sq_sge =
cpu_to_le16((sq->max_sge & CMDQ_CREATE_QP1_SQ_SGE_MASK) <<
CMDQ_CREATE_QP1_SQ_SGE_SFT);
@@ -866,24 +911,10 @@ int bnxt_qplib_create_qp1(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp)
/* RQ */
if (rq->max_wqe) {
rq->dbinfo.flags = 0;
- hwq_attr.res = res;
- hwq_attr.sginfo = &rq->sg_info;
- hwq_attr.stride = sizeof(struct sq_sge);
- hwq_attr.depth = bnxt_qplib_get_depth(rq, qp->wqe_mode, false);
- hwq_attr.type = HWQ_TYPE_QUEUE;
- rc = bnxt_qplib_alloc_init_hwq(&rq->hwq, &hwq_attr);
- if (rc)
- goto sq_swq;
- rc = bnxt_qplib_alloc_init_swq(rq);
- if (rc)
- goto fail_rq;
req.rq_size = cpu_to_le32(rq->max_wqe);
pbl = &rq->hwq.pbl[PBL_LVL_0];
req.rq_pbl = cpu_to_le64(pbl->pg_map_arr[0]);
- pg_sz_lvl = (bnxt_qplib_base_pg_size(&rq->hwq) <<
- CMDQ_CREATE_QP1_RQ_PG_SIZE_SFT);
- pg_sz_lvl |= (rq->hwq.level & CMDQ_CREATE_QP1_RQ_LVL_MASK);
- req.rq_pg_size_rq_lvl = pg_sz_lvl;
+ req.rq_pg_size_rq_lvl = rq->hwq.pg_sz_lvl;
req.rq_fwo_rq_sge =
cpu_to_le16((rq->max_sge &
CMDQ_CREATE_QP1_RQ_SGE_MASK) <<
@@ -894,7 +925,7 @@ int bnxt_qplib_create_qp1(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp)
rc = bnxt_qplib_alloc_qp_hdr_buf(res, qp);
if (rc) {
rc = -ENOMEM;
- goto rq_rwq;
+ return rc;
}
qp_flags |= CMDQ_CREATE_QP1_QP_FLAGS_RESERVED_LKEY_ENABLE;
req.qp_flags = cpu_to_le32(qp_flags);
@@ -907,73 +938,39 @@ int bnxt_qplib_create_qp1(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp)
qp->id = le32_to_cpu(resp.xid);
qp->cur_qp_state = CMDQ_MODIFY_QP_NEW_STATE_RESET;
- qp->cctx = res->cctx;
- sq->dbinfo.hwq = &sq->hwq;
- sq->dbinfo.xid = qp->id;
- sq->dbinfo.db = qp->dpi->dbr;
- sq->dbinfo.max_slot = bnxt_qplib_set_sq_max_slot(qp->wqe_mode);
- if (rq->max_wqe) {
- rq->dbinfo.hwq = &rq->hwq;
- rq->dbinfo.xid = qp->id;
- rq->dbinfo.db = qp->dpi->dbr;
- rq->dbinfo.max_slot = bnxt_qplib_set_rq_max_slot(rq->wqe_size);
- }
+
+ rc = bnxt_re_setup_qp_swqs(qp);
+ if (rc)
+ goto destroy_qp;
+ bnxt_qp_init_dbinfo(res, qp);
+
tbl_indx = map_qp_id_to_tbl_indx(qp->id, rcfw);
rcfw->qp_tbl[tbl_indx].qp_id = qp->id;
rcfw->qp_tbl[tbl_indx].qp_handle = (void *)qp;
return 0;
+destroy_qp:
+ bnxt_qplib_destroy_qp(res, qp);
fail:
bnxt_qplib_free_qp_hdr_buf(res, qp);
-rq_rwq:
- kfree(rq->swq);
-fail_rq:
- bnxt_qplib_free_hwq(res, &rq->hwq);
-sq_swq:
- kfree(sq->swq);
-fail_sq:
- bnxt_qplib_free_hwq(res, &sq->hwq);
return rc;
}
-static void bnxt_qplib_init_psn_ptr(struct bnxt_qplib_qp *qp, int size)
-{
- struct bnxt_qplib_hwq *hwq;
- struct bnxt_qplib_q *sq;
- u64 fpsne, psn_pg;
- u16 indx_pad = 0;
-
- sq = &qp->sq;
- hwq = &sq->hwq;
- /* First psn entry */
- fpsne = (u64)bnxt_qplib_get_qe(hwq, hwq->depth, &psn_pg);
- if (!IS_ALIGNED(fpsne, PAGE_SIZE))
- indx_pad = (fpsne & ~PAGE_MASK) / size;
- hwq->pad_pgofft = indx_pad;
- hwq->pad_pg = (u64 *)psn_pg;
- hwq->pad_stride = size;
-}
-
int bnxt_qplib_create_qp(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp)
{
struct bnxt_qplib_rcfw *rcfw = res->rcfw;
- struct bnxt_qplib_hwq_attr hwq_attr = {};
- struct bnxt_qplib_sg_info sginfo = {};
struct creq_create_qp_resp resp = {};
struct bnxt_qplib_cmdqmsg msg = {};
struct bnxt_qplib_q *sq = &qp->sq;
struct bnxt_qplib_q *rq = &qp->rq;
struct cmdq_create_qp req = {};
- int rc, req_size, psn_sz = 0;
- struct bnxt_qplib_hwq *xrrq;
struct bnxt_qplib_pbl *pbl;
u32 qp_flags = 0;
- u8 pg_sz_lvl;
u32 tbl_indx;
u16 nsge;
+ int rc;
- qp->is_host_msn_tbl = _is_host_msn_table(res->dattr->dev_cap_flags2);
sq->dbinfo.flags = 0;
bnxt_qplib_rcfw_cmd_prep((struct cmdq_base *)&req,
CMDQ_BASE_OPCODE_CREATE_QP,
@@ -985,56 +982,10 @@ int bnxt_qplib_create_qp(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp)
req.qp_handle = cpu_to_le64(qp->qp_handle);
/* SQ */
- if (qp->type == CMDQ_CREATE_QP_TYPE_RC) {
- psn_sz = bnxt_qplib_is_chip_gen_p5_p7(res->cctx) ?
- sizeof(struct sq_psn_search_ext) :
- sizeof(struct sq_psn_search);
-
- if (qp->is_host_msn_tbl) {
- psn_sz = sizeof(struct sq_msn_search);
- qp->msn = 0;
- }
- }
-
- hwq_attr.res = res;
- hwq_attr.sginfo = &sq->sg_info;
- hwq_attr.stride = sizeof(struct sq_sge);
- hwq_attr.depth = bnxt_qplib_get_depth(sq, qp->wqe_mode, true);
- hwq_attr.aux_stride = psn_sz;
- hwq_attr.aux_depth = psn_sz ? bnxt_qplib_set_sq_size(sq, qp->wqe_mode)
- : 0;
- /* Update msn tbl size */
- if (qp->is_host_msn_tbl && psn_sz) {
- if (qp->wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC)
- hwq_attr.aux_depth =
- roundup_pow_of_two(bnxt_qplib_set_sq_size(sq, qp->wqe_mode));
- else
- hwq_attr.aux_depth =
- roundup_pow_of_two(bnxt_qplib_set_sq_size(sq, qp->wqe_mode)) / 2;
- qp->msn_tbl_sz = hwq_attr.aux_depth;
- qp->msn = 0;
- }
-
- hwq_attr.type = HWQ_TYPE_QUEUE;
- rc = bnxt_qplib_alloc_init_hwq(&sq->hwq, &hwq_attr);
- if (rc)
- return rc;
-
- if (!sq->hwq.is_user) {
- rc = bnxt_qplib_alloc_init_swq(sq);
- if (rc)
- goto fail_sq;
-
- if (psn_sz)
- bnxt_qplib_init_psn_ptr(qp, psn_sz);
- }
- req.sq_size = cpu_to_le32(bnxt_qplib_set_sq_size(sq, qp->wqe_mode));
+ req.sq_size = cpu_to_le32(sq->max_sw_wqe);
pbl = &sq->hwq.pbl[PBL_LVL_0];
req.sq_pbl = cpu_to_le64(pbl->pg_map_arr[0]);
- pg_sz_lvl = (bnxt_qplib_base_pg_size(&sq->hwq) <<
- CMDQ_CREATE_QP_SQ_PG_SIZE_SFT);
- pg_sz_lvl |= (sq->hwq.level & CMDQ_CREATE_QP_SQ_LVL_MASK);
- req.sq_pg_size_sq_lvl = pg_sz_lvl;
+ req.sq_pg_size_sq_lvl = sq->hwq.pg_sz_lvl;
req.sq_fwo_sq_sge =
cpu_to_le16(((sq->max_sge & CMDQ_CREATE_QP_SQ_SGE_MASK) <<
CMDQ_CREATE_QP_SQ_SGE_SFT) | 0);
@@ -1043,29 +994,10 @@ int bnxt_qplib_create_qp(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp)
/* RQ */
if (!qp->srq) {
rq->dbinfo.flags = 0;
- hwq_attr.res = res;
- hwq_attr.sginfo = &rq->sg_info;
- hwq_attr.stride = sizeof(struct sq_sge);
- hwq_attr.depth = bnxt_qplib_get_depth(rq, qp->wqe_mode, false);
- hwq_attr.aux_stride = 0;
- hwq_attr.aux_depth = 0;
- hwq_attr.type = HWQ_TYPE_QUEUE;
- rc = bnxt_qplib_alloc_init_hwq(&rq->hwq, &hwq_attr);
- if (rc)
- goto sq_swq;
- if (!rq->hwq.is_user) {
- rc = bnxt_qplib_alloc_init_swq(rq);
- if (rc)
- goto fail_rq;
- }
-
req.rq_size = cpu_to_le32(rq->max_wqe);
pbl = &rq->hwq.pbl[PBL_LVL_0];
req.rq_pbl = cpu_to_le64(pbl->pg_map_arr[0]);
- pg_sz_lvl = (bnxt_qplib_base_pg_size(&rq->hwq) <<
- CMDQ_CREATE_QP_RQ_PG_SIZE_SFT);
- pg_sz_lvl |= (rq->hwq.level & CMDQ_CREATE_QP_RQ_LVL_MASK);
- req.rq_pg_size_rq_lvl = pg_sz_lvl;
+ req.rq_pg_size_rq_lvl = rq->hwq.pg_sz_lvl;
nsge = (qp->wqe_mode == BNXT_QPLIB_WQE_MODE_STATIC) ?
6 : rq->max_sge;
req.rq_fwo_rq_sge =
@@ -1091,44 +1023,9 @@ int bnxt_qplib_create_qp(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp)
req.qp_flags = cpu_to_le32(qp_flags);
/* ORRQ and IRRQ */
- if (psn_sz) {
- xrrq = &qp->orrq;
- xrrq->max_elements =
- ORD_LIMIT_TO_ORRQ_SLOTS(qp->max_rd_atomic);
- req_size = xrrq->max_elements *
- BNXT_QPLIB_MAX_ORRQE_ENTRY_SIZE + PAGE_SIZE - 1;
- req_size &= ~(PAGE_SIZE - 1);
- sginfo.pgsize = req_size;
- sginfo.pgshft = PAGE_SHIFT;
-
- hwq_attr.res = res;
- hwq_attr.sginfo = &sginfo;
- hwq_attr.depth = xrrq->max_elements;
- hwq_attr.stride = BNXT_QPLIB_MAX_ORRQE_ENTRY_SIZE;
- hwq_attr.aux_stride = 0;
- hwq_attr.aux_depth = 0;
- hwq_attr.type = HWQ_TYPE_CTX;
- rc = bnxt_qplib_alloc_init_hwq(xrrq, &hwq_attr);
- if (rc)
- goto rq_swq;
- pbl = &xrrq->pbl[PBL_LVL_0];
- req.orrq_addr = cpu_to_le64(pbl->pg_map_arr[0]);
-
- xrrq = &qp->irrq;
- xrrq->max_elements = IRD_LIMIT_TO_IRRQ_SLOTS(
- qp->max_dest_rd_atomic);
- req_size = xrrq->max_elements *
- BNXT_QPLIB_MAX_IRRQE_ENTRY_SIZE + PAGE_SIZE - 1;
- req_size &= ~(PAGE_SIZE - 1);
- sginfo.pgsize = req_size;
- hwq_attr.depth = xrrq->max_elements;
- hwq_attr.stride = BNXT_QPLIB_MAX_IRRQE_ENTRY_SIZE;
- rc = bnxt_qplib_alloc_init_hwq(xrrq, &hwq_attr);
- if (rc)
- goto fail_orrq;
-
- pbl = &xrrq->pbl[PBL_LVL_0];
- req.irrq_addr = cpu_to_le64(pbl->pg_map_arr[0]);
+ if (qp->psn_sz) {
+ req.orrq_addr = cpu_to_le64(bnxt_qplib_get_base_addr(&qp->orrq));
+ req.irrq_addr = cpu_to_le64(bnxt_qplib_get_base_addr(&qp->irrq));
}
req.pd_id = cpu_to_le32(qp->pd->id);
@@ -1136,23 +1033,23 @@ int bnxt_qplib_create_qp(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp)
sizeof(resp), 0);
rc = bnxt_qplib_rcfw_send_message(rcfw, &msg);
if (rc)
- goto fail;
+ return rc;
qp->id = le32_to_cpu(resp.xid);
+
+ if (!qp->is_user) {
+ rc = bnxt_re_setup_qp_swqs(qp);
+ if (rc)
+ goto destroy_qp;
+ }
+ bnxt_qp_init_dbinfo(res, qp);
+ if (qp->psn_sz)
+ bnxt_qplib_init_psn_ptr(qp, qp->psn_sz);
+
qp->cur_qp_state = CMDQ_MODIFY_QP_NEW_STATE_RESET;
INIT_LIST_HEAD(&qp->sq_flush);
INIT_LIST_HEAD(&qp->rq_flush);
qp->cctx = res->cctx;
- sq->dbinfo.hwq = &sq->hwq;
- sq->dbinfo.xid = qp->id;
- sq->dbinfo.db = qp->dpi->dbr;
- sq->dbinfo.max_slot = bnxt_qplib_set_sq_max_slot(qp->wqe_mode);
- if (rq->max_wqe) {
- rq->dbinfo.hwq = &rq->hwq;
- rq->dbinfo.xid = qp->id;
- rq->dbinfo.db = qp->dpi->dbr;
- rq->dbinfo.max_slot = bnxt_qplib_set_rq_max_slot(rq->wqe_size);
- }
spin_lock_bh(&rcfw->tbl_lock);
tbl_indx = map_qp_id_to_tbl_indx(qp->id, rcfw);
rcfw->qp_tbl[tbl_indx].qp_id = qp->id;
@@ -1160,18 +1057,8 @@ int bnxt_qplib_create_qp(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp)
spin_unlock_bh(&rcfw->tbl_lock);
return 0;
-fail:
- bnxt_qplib_free_hwq(res, &qp->irrq);
-fail_orrq:
- bnxt_qplib_free_hwq(res, &qp->orrq);
-rq_swq:
- kfree(rq->swq);
-fail_rq:
- bnxt_qplib_free_hwq(res, &rq->hwq);
-sq_swq:
- kfree(sq->swq);
-fail_sq:
- bnxt_qplib_free_hwq(res, &sq->hwq);
+destroy_qp:
+ bnxt_qplib_destroy_qp(res, qp);
return rc;
}
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_fp.h b/drivers/infiniband/hw/bnxt_re/qplib_fp.h
index 30c3f99be07b..5f671cc59100 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_fp.h
+++ b/drivers/infiniband/hw/bnxt_re/qplib_fp.h
@@ -279,6 +279,7 @@ struct bnxt_qplib_qp {
u8 wqe_mode;
u8 state;
u8 cur_qp_state;
+ u8 is_user;
u64 modify_flags;
u32 ext_modify_flags;
u32 max_inline_data;
@@ -344,9 +345,11 @@ struct bnxt_qplib_qp {
struct list_head rq_flush;
u32 msn;
u32 msn_tbl_sz;
+ u32 psn_sz;
bool is_host_msn_tbl;
u8 tos_dscp;
u32 ugid_index;
+ u16 dev_cap_flags;
u32 rate_limit;
u8 shaper_allocation_status;
};
@@ -617,6 +620,11 @@ static inline void bnxt_qplib_swq_mod_start(struct bnxt_qplib_q *que, u32 idx)
que->swq_start = que->swq[idx].next_idx;
}
+static inline u32 bnxt_qplib_get_stride(void)
+{
+ return sizeof(struct sq_sge);
+}
+
static inline u32 bnxt_qplib_get_depth(struct bnxt_qplib_q *que, u8 wqe_mode, bool is_sq)
{
u32 slots;
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_res.h b/drivers/infiniband/hw/bnxt_re/qplib_res.h
index 9a5dcf97b6f4..f01c1bb1fcb4 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_res.h
+++ b/drivers/infiniband/hw/bnxt_re/qplib_res.h
@@ -198,6 +198,7 @@ struct bnxt_qplib_hwq {
u32 cons; /* raw */
u8 cp_bit;
u8 is_user;
+ u8 pg_sz_lvl;
u64 *pad_pg;
u32 pad_stride;
u32 pad_pgofft;
@@ -358,6 +359,11 @@ static inline u8 bnxt_qplib_get_ring_type(struct bnxt_qplib_chip_ctx *cctx)
RING_ALLOC_REQ_RING_TYPE_ROCE_CMPL;
}
+static inline u64 bnxt_qplib_get_base_addr(struct bnxt_qplib_hwq *hwq)
+{
+ return hwq->pbl[PBL_LVL_0].pg_map_arr[0];
+}
+
static inline u8 bnxt_qplib_base_pg_size(struct bnxt_qplib_hwq *hwq)
{
u8 pg_size = BNXT_QPLIB_HWRM_PG_SIZE_4K;
--
2.51.2.636.ga99f379adf
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH rdma-next v12 3/6] RDMA/bnxt_re: Support doorbell extensions
2026-02-11 12:49 [PATCH rdma-next v12 0/6] RDMA/bnxt_re: Support uapi extensions Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 1/6] RDMA/bnxt_re: Move the UAPI methods to a dedicated file Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 2/6] RDMA/bnxt_re: Refactor bnxt_qplib_create_qp() function Sriharsha Basavapatna
@ 2026-02-11 12:49 ` Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 4/6] RDMA/bnxt_re: Refactor bnxt_re_create_cq() Sriharsha Basavapatna
` (2 subsequent siblings)
5 siblings, 0 replies; 15+ messages in thread
From: Sriharsha Basavapatna @ 2026-02-11 12:49 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
From: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Some applications may need multiple doorbells to support parallel
processing of threads that each operate on a group of resources.
The following uapi methods have been implemented in this patch.
- BNXT_RE_METHOD_DBR_ALLOC:
This will allow the appliation to create extra doorbell regions
and use the associated doorbell page index in CREATE_QP and
use the associated DB address while ringing the doorbell.
- BNXT_RE_METHOD_DBR_FREE:
Free the allocated doorbell region.
- BNXT_RE_METHOD_GET_DEFAULT_DBR:
Return the default doorbell page index and doorbell page address
associated with the ucontext.
Co-developed-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
drivers/infiniband/hw/bnxt_re/ib_verbs.h | 7 ++
drivers/infiniband/hw/bnxt_re/qplib_res.c | 43 +++++++
drivers/infiniband/hw/bnxt_re/qplib_res.h | 4 +
drivers/infiniband/hw/bnxt_re/uapi.c | 130 ++++++++++++++++++++++
include/uapi/rdma/bnxt_re-abi.h | 29 +++++
5 files changed, 213 insertions(+)
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
index a11f56730a31..33e0f66b39eb 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
@@ -164,6 +164,13 @@ struct bnxt_re_user_mmap_entry {
u8 mmap_flag;
};
+struct bnxt_re_dbr_obj {
+ struct bnxt_re_dev *rdev;
+ struct bnxt_qplib_dpi dpi;
+ struct bnxt_re_user_mmap_entry *entry;
+ atomic_t usecnt; /* QPs using this dbr */
+};
+
struct bnxt_re_flow {
struct ib_flow ib_flow;
struct bnxt_re_dev *rdev;
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_res.c b/drivers/infiniband/hw/bnxt_re/qplib_res.c
index 875d7b52c06a..30cc2d64a9ae 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_res.c
+++ b/drivers/infiniband/hw/bnxt_re/qplib_res.c
@@ -685,6 +685,49 @@ static int bnxt_qplib_alloc_pd_tbl(struct bnxt_qplib_res *res,
}
/* DPIs */
+int bnxt_qplib_alloc_uc_dpi(struct bnxt_qplib_res *res, struct bnxt_qplib_dpi *dpi)
+{
+ struct bnxt_qplib_dpi_tbl *dpit = &res->dpi_tbl;
+ struct bnxt_qplib_reg_desc *reg;
+ u32 bit_num;
+ int rc = 0;
+
+ reg = &dpit->wcreg;
+ mutex_lock(&res->dpi_tbl_lock);
+ bit_num = find_first_bit(dpit->tbl, dpit->max);
+ if (bit_num >= dpit->max) {
+ rc = -ENOMEM;
+ goto unlock;
+ }
+ /* Found unused DPI */
+ clear_bit(bit_num, dpit->tbl);
+ dpi->bit = bit_num;
+ dpi->dpi = bit_num + (reg->offset - dpit->ucreg.offset) / PAGE_SIZE;
+ dpi->umdbr = reg->bar_base + reg->offset + bit_num * PAGE_SIZE;
+unlock:
+ mutex_unlock(&res->dpi_tbl_lock);
+ return rc;
+}
+
+int bnxt_qplib_free_uc_dpi(struct bnxt_qplib_res *res, struct bnxt_qplib_dpi *dpi)
+{
+ struct bnxt_qplib_dpi_tbl *dpit = &res->dpi_tbl;
+ int rc = 0;
+
+ mutex_lock(&res->dpi_tbl_lock);
+ if (dpi->bit >= dpit->max) {
+ rc = -EINVAL;
+ goto unlock;
+ }
+
+ if (test_and_set_bit(dpi->bit, dpit->tbl))
+ rc = -EINVAL;
+ memset(dpi, 0, sizeof(*dpi));
+unlock:
+ mutex_unlock(&res->dpi_tbl_lock);
+ return rc;
+}
+
int bnxt_qplib_alloc_dpi(struct bnxt_qplib_res *res,
struct bnxt_qplib_dpi *dpi,
void *app, u8 type)
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_res.h b/drivers/infiniband/hw/bnxt_re/qplib_res.h
index f01c1bb1fcb4..ffe31c952d50 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_res.h
+++ b/drivers/infiniband/hw/bnxt_re/qplib_res.h
@@ -436,6 +436,10 @@ int bnxt_qplib_alloc_dpi(struct bnxt_qplib_res *res,
void *app, u8 type);
int bnxt_qplib_dealloc_dpi(struct bnxt_qplib_res *res,
struct bnxt_qplib_dpi *dpi);
+int bnxt_qplib_alloc_uc_dpi(struct bnxt_qplib_res *res,
+ struct bnxt_qplib_dpi *dpi);
+int bnxt_qplib_free_uc_dpi(struct bnxt_qplib_res *res,
+ struct bnxt_qplib_dpi *dpi);
void bnxt_qplib_cleanup_res(struct bnxt_qplib_res *res);
int bnxt_qplib_init_res(struct bnxt_qplib_res *res);
void bnxt_qplib_free_res(struct bnxt_qplib_res *res);
diff --git a/drivers/infiniband/hw/bnxt_re/uapi.c b/drivers/infiniband/hw/bnxt_re/uapi.c
index 0145882e49f6..05444c3bb2f3 100644
--- a/drivers/infiniband/hw/bnxt_re/uapi.c
+++ b/drivers/infiniband/hw/bnxt_re/uapi.c
@@ -331,9 +331,139 @@ DECLARE_UVERBS_NAMED_OBJECT(BNXT_RE_OBJECT_GET_TOGGLE_MEM,
&UVERBS_METHOD(BNXT_RE_METHOD_GET_TOGGLE_MEM),
&UVERBS_METHOD(BNXT_RE_METHOD_RELEASE_TOGGLE_MEM));
+static int UVERBS_HANDLER(BNXT_RE_METHOD_DBR_ALLOC)(struct uverbs_attr_bundle *attrs)
+{
+ struct bnxt_re_db_region dbr = {};
+ struct bnxt_re_ucontext *uctx;
+ struct bnxt_re_dbr_obj *obj;
+ struct ib_ucontext *ib_uctx;
+ struct bnxt_qplib_dpi *dpi;
+ struct bnxt_re_dev *rdev;
+ struct ib_uobject *uobj;
+ u64 mmap_offset;
+ int ret;
+
+ ib_uctx = ib_uverbs_get_ucontext(attrs);
+ if (IS_ERR(ib_uctx))
+ return PTR_ERR(ib_uctx);
+
+ uctx = container_of(ib_uctx, struct bnxt_re_ucontext, ib_uctx);
+ rdev = uctx->rdev;
+ uobj = uverbs_attr_get_uobject(attrs, BNXT_RE_ALLOC_DBR_HANDLE);
+
+ obj = kzalloc(sizeof(*obj), GFP_KERNEL);
+ if (!obj)
+ return -ENOMEM;
+
+ dpi = &obj->dpi;
+ ret = bnxt_qplib_alloc_uc_dpi(&rdev->qplib_res, dpi);
+ if (ret)
+ goto free_mem;
+
+ obj->entry = bnxt_re_mmap_entry_insert(uctx, dpi->umdbr,
+ BNXT_RE_MMAP_UC_DB,
+ &mmap_offset);
+ if (!obj->entry) {
+ ret = -ENOMEM;
+ goto free_dpi;
+ }
+
+ obj->rdev = rdev;
+ uobj->object = obj;
+ uverbs_finalize_uobj_create(attrs, BNXT_RE_ALLOC_DBR_HANDLE);
+
+ dbr.umdbr = dpi->umdbr;
+ dbr.dpi = dpi->dpi;
+ ret = uverbs_copy_to_struct_or_zero(attrs, BNXT_RE_ALLOC_DBR_ATTR,
+ &dbr, sizeof(dbr));
+ if (ret)
+ return ret;
+
+ ret = uverbs_copy_to(attrs, BNXT_RE_ALLOC_DBR_OFFSET,
+ &mmap_offset, sizeof(mmap_offset));
+ if (ret)
+ return ret;
+ return 0;
+free_dpi:
+ bnxt_qplib_free_uc_dpi(&rdev->qplib_res, dpi);
+free_mem:
+ kfree(obj);
+ return ret;
+}
+
+static int bnxt_re_dbr_cleanup(struct ib_uobject *uobject,
+ enum rdma_remove_reason why,
+ struct uverbs_attr_bundle *attrs)
+{
+ struct bnxt_re_dbr_obj *obj = uobject->object;
+ struct bnxt_re_dev *rdev = obj->rdev;
+
+ rdma_user_mmap_entry_remove(&obj->entry->rdma_entry);
+ bnxt_qplib_free_uc_dpi(&rdev->qplib_res, &obj->dpi);
+ return 0;
+}
+
+static int UVERBS_HANDLER(BNXT_RE_METHOD_GET_DEFAULT_DBR)(struct uverbs_attr_bundle *attrs)
+{
+ struct bnxt_re_db_region dpi = {};
+ struct bnxt_re_ucontext *uctx;
+ struct ib_ucontext *ib_uctx;
+ int ret;
+
+ ib_uctx = ib_uverbs_get_ucontext(attrs);
+ if (IS_ERR(ib_uctx))
+ return PTR_ERR(ib_uctx);
+
+ uctx = container_of(ib_uctx, struct bnxt_re_ucontext, ib_uctx);
+ dpi.umdbr = uctx->dpi.umdbr;
+ dpi.dpi = uctx->dpi.dpi;
+
+ ret = uverbs_copy_to_struct_or_zero(attrs, BNXT_RE_DEFAULT_DBR_ATTR,
+ &dpi, sizeof(dpi));
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+DECLARE_UVERBS_NAMED_METHOD(BNXT_RE_METHOD_DBR_ALLOC,
+ UVERBS_ATTR_IDR(BNXT_RE_ALLOC_DBR_HANDLE,
+ BNXT_RE_OBJECT_DBR,
+ UVERBS_ACCESS_NEW,
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(BNXT_RE_ALLOC_DBR_ATTR,
+ UVERBS_ATTR_STRUCT(struct bnxt_re_db_region,
+ umdbr),
+ UA_MANDATORY),
+ UVERBS_ATTR_PTR_OUT(BNXT_RE_ALLOC_DBR_OFFSET,
+ UVERBS_ATTR_TYPE(u64),
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_METHOD_DESTROY(BNXT_RE_METHOD_DBR_FREE,
+ UVERBS_ATTR_IDR(BNXT_RE_FREE_DBR_HANDLE,
+ BNXT_RE_OBJECT_DBR,
+ UVERBS_ACCESS_DESTROY,
+ UA_MANDATORY));
+
+DECLARE_UVERBS_NAMED_OBJECT(BNXT_RE_OBJECT_DBR,
+ UVERBS_TYPE_ALLOC_IDR(bnxt_re_dbr_cleanup),
+ &UVERBS_METHOD(BNXT_RE_METHOD_DBR_ALLOC),
+ &UVERBS_METHOD(BNXT_RE_METHOD_DBR_FREE));
+
+DECLARE_UVERBS_NAMED_METHOD(BNXT_RE_METHOD_GET_DEFAULT_DBR,
+ UVERBS_ATTR_PTR_OUT(BNXT_RE_DEFAULT_DBR_ATTR,
+ UVERBS_ATTR_STRUCT(struct bnxt_re_db_region,
+ umdbr),
+ UA_MANDATORY));
+
+DECLARE_UVERBS_GLOBAL_METHODS(BNXT_RE_OBJECT_DEFAULT_DBR,
+ &UVERBS_METHOD(BNXT_RE_METHOD_GET_DEFAULT_DBR));
+
const struct uapi_definition bnxt_re_uapi_defs[] = {
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_ALLOC_PAGE),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_NOTIFY_DRV),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_GET_TOGGLE_MEM),
+ UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_DBR),
+ UAPI_DEF_CHAIN_OBJ_TREE_NAMED(BNXT_RE_OBJECT_DEFAULT_DBR),
{}
};
diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h
index d15afa58963b..1f7685665db1 100644
--- a/include/uapi/rdma/bnxt_re-abi.h
+++ b/include/uapi/rdma/bnxt_re-abi.h
@@ -164,6 +164,8 @@ enum bnxt_re_objects {
BNXT_RE_OBJECT_ALLOC_PAGE = (1U << UVERBS_ID_NS_SHIFT),
BNXT_RE_OBJECT_NOTIFY_DRV,
BNXT_RE_OBJECT_GET_TOGGLE_MEM,
+ BNXT_RE_OBJECT_DBR,
+ BNXT_RE_OBJECT_DEFAULT_DBR,
};
enum bnxt_re_alloc_page_type {
@@ -232,4 +234,31 @@ struct bnxt_re_packet_pacing_caps {
struct bnxt_re_query_device_ex_resp {
struct bnxt_re_packet_pacing_caps packet_pacing_caps;
};
+
+struct bnxt_re_db_region {
+ __u32 dpi;
+ __u32 reserved;
+ __aligned_u64 umdbr;
+};
+
+enum bnxt_re_obj_dbr_alloc_attrs {
+ BNXT_RE_ALLOC_DBR_HANDLE = (1U << UVERBS_ID_NS_SHIFT),
+ BNXT_RE_ALLOC_DBR_ATTR,
+ BNXT_RE_ALLOC_DBR_OFFSET,
+};
+
+enum bnxt_re_obj_dbr_free_attrs {
+ BNXT_RE_FREE_DBR_HANDLE = (1U << UVERBS_ID_NS_SHIFT),
+};
+
+enum bnxt_re_obj_default_dbr_attrs {
+ BNXT_RE_DEFAULT_DBR_ATTR = (1U << UVERBS_ID_NS_SHIFT),
+};
+
+enum bnxt_re_obj_dpi_methods {
+ BNXT_RE_METHOD_DBR_ALLOC = (1U << UVERBS_ID_NS_SHIFT),
+ BNXT_RE_METHOD_DBR_FREE,
+ BNXT_RE_METHOD_GET_DEFAULT_DBR,
+};
+
#endif /* __BNXT_RE_UVERBS_ABI_H__*/
--
2.51.2.636.ga99f379adf
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH rdma-next v12 4/6] RDMA/bnxt_re: Refactor bnxt_re_create_cq()
2026-02-11 12:49 [PATCH rdma-next v12 0/6] RDMA/bnxt_re: Support uapi extensions Sriharsha Basavapatna
` (2 preceding siblings ...)
2026-02-11 12:49 ` [PATCH rdma-next v12 3/6] RDMA/bnxt_re: Support doorbell extensions Sriharsha Basavapatna
@ 2026-02-11 12:49 ` Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 5/6] RDMA/bnxt_re: Support dmabuf for CQ rings Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 6/6] RDMA/bnxt_re: Support application specific CQs Sriharsha Basavapatna
5 siblings, 0 replies; 15+ messages in thread
From: Sriharsha Basavapatna @ 2026-02-11 12:49 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
Some applications may allocate dmabuf based memory for CQs. To support
this, update the existing code to use SZ_4K to specify supported HW
page size for CQs, as we support only 4K pages for now.
Call ib_umem_find_best_pgsz() to ensure umem supports this requested
page size. A helper function includes these changes.
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 29 +++++++++++++++++++++---
1 file changed, 26 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index feb364e45e14..9fa89f330c5a 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -3348,6 +3348,26 @@ int bnxt_re_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata)
return 0;
}
+static int bnxt_re_setup_sginfo(struct bnxt_re_dev *rdev,
+ struct ib_umem *umem,
+ struct bnxt_qplib_sg_info *sginfo)
+{
+ unsigned long page_size;
+
+ if (!umem)
+ return -EINVAL;
+
+ page_size = ib_umem_find_best_pgsz(umem, SZ_4K, 0);
+ if (!page_size || page_size != SZ_4K)
+ return -EINVAL;
+
+ sginfo->umem = umem;
+ sginfo->npages = ib_umem_num_dma_blocks(umem, page_size);
+ sginfo->pgsize = page_size;
+ sginfo->pgshft = __builtin_ctz(page_size);
+ return 0;
+}
+
int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
struct uverbs_attr_bundle *attrs)
{
@@ -3379,8 +3399,6 @@ int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
if (entries > dev_attr->max_cq_wqes + 1)
entries = dev_attr->max_cq_wqes + 1;
- cq->qplib_cq.sg_info.pgsize = PAGE_SIZE;
- cq->qplib_cq.sg_info.pgshft = PAGE_SHIFT;
if (udata) {
struct bnxt_re_cq_req req;
@@ -3395,7 +3413,10 @@ int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
rc = PTR_ERR(cq->umem);
goto fail;
}
- cq->qplib_cq.sg_info.umem = cq->umem;
+ rc = bnxt_re_setup_sginfo(rdev, cq->umem, &cq->qplib_cq.sg_info);
+ if (rc)
+ goto fail;
+
cq->qplib_cq.dpi = &uctx->dpi;
} else {
cq->max_cql = min_t(u32, entries, MAX_CQL_PER_POLL);
@@ -3406,6 +3427,8 @@ int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
goto fail;
}
+ cq->qplib_cq.sg_info.pgsize = SZ_4K;
+ cq->qplib_cq.sg_info.pgshft = __builtin_ctz(SZ_4K);
cq->qplib_cq.dpi = &rdev->dpi_privileged;
}
cq->qplib_cq.max_wqe = entries;
--
2.51.2.636.ga99f379adf
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH rdma-next v12 5/6] RDMA/bnxt_re: Support dmabuf for CQ rings
2026-02-11 12:49 [PATCH rdma-next v12 0/6] RDMA/bnxt_re: Support uapi extensions Sriharsha Basavapatna
` (3 preceding siblings ...)
2026-02-11 12:49 ` [PATCH rdma-next v12 4/6] RDMA/bnxt_re: Refactor bnxt_re_create_cq() Sriharsha Basavapatna
@ 2026-02-11 12:49 ` Sriharsha Basavapatna
2026-02-13 11:12 ` Leon Romanovsky
2026-02-11 12:49 ` [PATCH rdma-next v12 6/6] RDMA/bnxt_re: Support application specific CQs Sriharsha Basavapatna
5 siblings, 1 reply; 15+ messages in thread
From: Sriharsha Basavapatna @ 2026-02-11 12:49 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
For CQs, kernel already supports pinning dmabuf based application
memory, specified through provider attributes. Register a new devop
for create_cq_umem() and process the umem argument. Refactor the
existing create_cq() handler so that code is shared across both
handlers.
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 30 +++++++++++++++++-------
drivers/infiniband/hw/bnxt_re/ib_verbs.h | 2 ++
drivers/infiniband/hw/bnxt_re/main.c | 1 +
3 files changed, 24 insertions(+), 9 deletions(-)
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 9fa89f330c5a..30aefbd0112e 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -3368,8 +3368,8 @@ static int bnxt_re_setup_sginfo(struct bnxt_re_dev *rdev,
return 0;
}
-int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
- struct uverbs_attr_bundle *attrs)
+int bnxt_re_create_cq_umem(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
+ struct ib_umem *umem, struct uverbs_attr_bundle *attrs)
{
struct bnxt_re_cq *cq = container_of(ibcq, struct bnxt_re_cq, ib_cq);
struct bnxt_re_dev *rdev = to_bnxt_re_dev(ibcq->device, ibdev);
@@ -3406,13 +3406,18 @@ int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
if (rc)
goto fail;
- cq->umem = ib_umem_get(&rdev->ibdev, req.cq_va,
- entries * sizeof(struct cq_base),
- IB_ACCESS_LOCAL_WRITE);
- if (IS_ERR(cq->umem)) {
- rc = PTR_ERR(cq->umem);
- goto fail;
+ if (umem) {
+ cq->umem = umem;
+ } else {
+ cq->umem = ib_umem_get(&rdev->ibdev, req.cq_va,
+ entries * sizeof(struct cq_base),
+ IB_ACCESS_LOCAL_WRITE);
+ if (IS_ERR(cq->umem)) {
+ rc = PTR_ERR(cq->umem);
+ goto fail;
+ }
}
+
rc = bnxt_re_setup_sginfo(rdev, cq->umem, &cq->qplib_cq.sg_info);
if (rc)
goto fail;
@@ -3480,12 +3485,19 @@ int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
free_mem:
free_page((unsigned long)cq->uctx_cq_page);
c2fail:
- ib_umem_release(cq->umem);
+ if (!umem)
+ ib_umem_release(cq->umem);
fail:
kfree(cq->cql);
return rc;
}
+int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
+ struct uverbs_attr_bundle *attrs)
+{
+ return bnxt_re_create_cq_umem(ibcq, attr, NULL, attrs);
+}
+
static void bnxt_re_resize_cq_complete(struct bnxt_re_cq *cq)
{
struct bnxt_re_dev *rdev = cq->rdev;
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
index 33e0f66b39eb..27cbe9a1c7e1 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
@@ -254,6 +254,8 @@ int bnxt_re_post_recv(struct ib_qp *qp, const struct ib_recv_wr *recv_wr,
const struct ib_recv_wr **bad_recv_wr);
int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
struct uverbs_attr_bundle *attrs);
+int bnxt_re_create_cq_umem(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
+ struct ib_umem *umem, struct uverbs_attr_bundle *attrs);
int bnxt_re_resize_cq(struct ib_cq *ibcq, int cqe, struct ib_udata *udata);
int bnxt_re_destroy_cq(struct ib_cq *cq, struct ib_udata *udata);
int bnxt_re_poll_cq(struct ib_cq *cq, int num_entries, struct ib_wc *wc);
diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c
index 73003ad25ee8..401a481afecc 100644
--- a/drivers/infiniband/hw/bnxt_re/main.c
+++ b/drivers/infiniband/hw/bnxt_re/main.c
@@ -1334,6 +1334,7 @@ static const struct ib_device_ops bnxt_re_dev_ops = {
.alloc_ucontext = bnxt_re_alloc_ucontext,
.create_ah = bnxt_re_create_ah,
.create_cq = bnxt_re_create_cq,
+ .create_cq_umem = bnxt_re_create_cq_umem,
.create_qp = bnxt_re_create_qp,
.create_srq = bnxt_re_create_srq,
.create_user_ah = bnxt_re_create_ah,
--
2.51.2.636.ga99f379adf
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH rdma-next v12 6/6] RDMA/bnxt_re: Support application specific CQs
2026-02-11 12:49 [PATCH rdma-next v12 0/6] RDMA/bnxt_re: Support uapi extensions Sriharsha Basavapatna
` (4 preceding siblings ...)
2026-02-11 12:49 ` [PATCH rdma-next v12 5/6] RDMA/bnxt_re: Support dmabuf for CQ rings Sriharsha Basavapatna
@ 2026-02-11 12:49 ` Sriharsha Basavapatna
5 siblings, 0 replies; 15+ messages in thread
From: Sriharsha Basavapatna @ 2026-02-11 12:49 UTC (permalink / raw)
To: leon, jgg
Cc: linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
This patch supports application allocated memory for CQs.
The application allocates and manages the CQs directly. To support
this, the driver exports a new comp_mask to indicate direct control
of the CQ. When this comp_mask bit is set in the ureq, the driver
maps this application allocated CQ memory into hardware. As the
application manages this memory, the CQ depth ('cqe') passed by it
must be used as is and the driver shouldn't update it.
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 9 ++++++---
include/uapi/rdma/bnxt_re-abi.h | 7 ++++++-
2 files changed, 12 insertions(+), 4 deletions(-)
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 30aefbd0112e..46850e92bbf8 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -3378,6 +3378,7 @@ int bnxt_re_create_cq_umem(struct ib_cq *ibcq, const struct ib_cq_init_attr *att
rdma_udata_to_drv_context(udata, struct bnxt_re_ucontext, ib_uctx);
struct bnxt_qplib_dev_attr *dev_attr = rdev->dev_attr;
struct bnxt_qplib_chip_ctx *cctx;
+ struct bnxt_re_cq_req req;
int cqe = attr->cqe;
int rc, entries;
u32 active_cqs;
@@ -3400,12 +3401,14 @@ int bnxt_re_create_cq_umem(struct ib_cq *ibcq, const struct ib_cq_init_attr *att
entries = dev_attr->max_cq_wqes + 1;
if (udata) {
- struct bnxt_re_cq_req req;
-
- rc = ib_copy_validate_udata_in(udata, req, cq_handle);
+ rc = ib_copy_validate_udata_in_cm(udata, req, cq_handle,
+ BNXT_RE_CQ_FIXED_NUM_CQE_ENABLE);
if (rc)
goto fail;
+ if (req.comp_mask & BNXT_RE_CQ_FIXED_NUM_CQE_ENABLE)
+ entries = cqe;
+
if (umem) {
cq->umem = umem;
} else {
diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h
index 1f7685665db1..0f631c65c63d 100644
--- a/include/uapi/rdma/bnxt_re-abi.h
+++ b/include/uapi/rdma/bnxt_re-abi.h
@@ -103,12 +103,17 @@ struct bnxt_re_pd_resp {
struct bnxt_re_cq_req {
__aligned_u64 cq_va;
__aligned_u64 cq_handle;
+ __aligned_u64 comp_mask;
};
-enum bnxt_re_cq_mask {
+enum bnxt_re_resp_cq_mask {
BNXT_RE_CQ_TOGGLE_PAGE_SUPPORT = 0x1,
};
+enum bnxt_re_req_cq_mask {
+ BNXT_RE_CQ_FIXED_NUM_CQE_ENABLE = 0x1,
+};
+
struct bnxt_re_cq_resp {
__u32 cqid;
__u32 tail;
--
2.51.2.636.ga99f379adf
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH rdma-next v12 5/6] RDMA/bnxt_re: Support dmabuf for CQ rings
2026-02-11 12:49 ` [PATCH rdma-next v12 5/6] RDMA/bnxt_re: Support dmabuf for CQ rings Sriharsha Basavapatna
@ 2026-02-13 11:12 ` Leon Romanovsky
2026-02-13 14:54 ` Jason Gunthorpe
0 siblings, 1 reply; 15+ messages in thread
From: Leon Romanovsky @ 2026-02-13 11:12 UTC (permalink / raw)
To: Sriharsha Basavapatna
Cc: jgg, linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil
On Wed, Feb 11, 2026 at 06:19:26PM +0530, Sriharsha Basavapatna wrote:
> For CQs, kernel already supports pinning dmabuf based application
> memory, specified through provider attributes. Register a new devop
> for create_cq_umem() and process the umem argument. Refactor the
> existing create_cq() handler so that code is shared across both
> handlers.
>
> Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
> Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
> ---
> drivers/infiniband/hw/bnxt_re/ib_verbs.c | 30 +++++++++++++++++-------
> drivers/infiniband/hw/bnxt_re/ib_verbs.h | 2 ++
> drivers/infiniband/hw/bnxt_re/main.c | 1 +
> 3 files changed, 24 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
> index 9fa89f330c5a..30aefbd0112e 100644
> --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
> +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
> @@ -3368,8 +3368,8 @@ static int bnxt_re_setup_sginfo(struct bnxt_re_dev *rdev,
> return 0;
> }
>
> -int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
> - struct uverbs_attr_bundle *attrs)
> +int bnxt_re_create_cq_umem(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
> + struct ib_umem *umem, struct uverbs_attr_bundle *attrs)
> {
> struct bnxt_re_cq *cq = container_of(ibcq, struct bnxt_re_cq, ib_cq);
> struct bnxt_re_dev *rdev = to_bnxt_re_dev(ibcq->device, ibdev);
> @@ -3406,13 +3406,18 @@ int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
> if (rc)
> goto fail;
>
> - cq->umem = ib_umem_get(&rdev->ibdev, req.cq_va,
> - entries * sizeof(struct cq_base),
> - IB_ACCESS_LOCAL_WRITE);
> - if (IS_ERR(cq->umem)) {
> - rc = PTR_ERR(cq->umem);
> - goto fail;
> + if (umem) {
> + cq->umem = umem;
> + } else {
> + cq->umem = ib_umem_get(&rdev->ibdev, req.cq_va,
> + entries * sizeof(struct cq_base),
> + IB_ACCESS_LOCAL_WRITE);
> + if (IS_ERR(cq->umem)) {
> + rc = PTR_ERR(cq->umem);
> + goto fail;
> + }
> }
> +
> rc = bnxt_re_setup_sginfo(rdev, cq->umem, &cq->qplib_cq.sg_info);
> if (rc)
> goto fail;
> @@ -3480,12 +3485,19 @@ int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
> free_mem:
> free_page((unsigned long)cq->uctx_cq_page);
> c2fail:
> - ib_umem_release(cq->umem);
> + if (!umem)
> + ib_umem_release(cq->umem);
> fail:
> kfree(cq->cql);
> return rc;
> }
>
> +int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
> + struct uverbs_attr_bundle *attrs)
> +{
> + return bnxt_re_create_cq_umem(ibcq, attr, NULL, attrs);
> +}
Please don't mix create_cq and create_cq_umem.
https://lore.kernel.org/linux-rdma/20260213-refactor-umem-v1-15-f3be85847922@nvidia.com/T/#u
Thanks
> +
> static void bnxt_re_resize_cq_complete(struct bnxt_re_cq *cq)
> {
> struct bnxt_re_dev *rdev = cq->rdev;
> diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
> index 33e0f66b39eb..27cbe9a1c7e1 100644
> --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h
> +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
> @@ -254,6 +254,8 @@ int bnxt_re_post_recv(struct ib_qp *qp, const struct ib_recv_wr *recv_wr,
> const struct ib_recv_wr **bad_recv_wr);
> int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
> struct uverbs_attr_bundle *attrs);
> +int bnxt_re_create_cq_umem(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
> + struct ib_umem *umem, struct uverbs_attr_bundle *attrs);
> int bnxt_re_resize_cq(struct ib_cq *ibcq, int cqe, struct ib_udata *udata);
> int bnxt_re_destroy_cq(struct ib_cq *cq, struct ib_udata *udata);
> int bnxt_re_poll_cq(struct ib_cq *cq, int num_entries, struct ib_wc *wc);
> diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c
> index 73003ad25ee8..401a481afecc 100644
> --- a/drivers/infiniband/hw/bnxt_re/main.c
> +++ b/drivers/infiniband/hw/bnxt_re/main.c
> @@ -1334,6 +1334,7 @@ static const struct ib_device_ops bnxt_re_dev_ops = {
> .alloc_ucontext = bnxt_re_alloc_ucontext,
> .create_ah = bnxt_re_create_ah,
> .create_cq = bnxt_re_create_cq,
> + .create_cq_umem = bnxt_re_create_cq_umem,
> .create_qp = bnxt_re_create_qp,
> .create_srq = bnxt_re_create_srq,
> .create_user_ah = bnxt_re_create_ah,
> --
> 2.51.2.636.ga99f379adf
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH rdma-next v12 5/6] RDMA/bnxt_re: Support dmabuf for CQ rings
2026-02-13 11:12 ` Leon Romanovsky
@ 2026-02-13 14:54 ` Jason Gunthorpe
2026-02-13 15:03 ` Sriharsha Basavapatna
` (2 more replies)
0 siblings, 3 replies; 15+ messages in thread
From: Jason Gunthorpe @ 2026-02-13 14:54 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Sriharsha Basavapatna, linux-rdma, andrew.gospodarek,
selvin.xavier, kalesh-anakkur.purayil
On Fri, Feb 13, 2026 at 01:12:56PM +0200, Leon Romanovsky wrote:
> > +int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
> > + struct uverbs_attr_bundle *attrs)
> > +{
> > + return bnxt_re_create_cq_umem(ibcq, attr, NULL, attrs);
> > +}
>
> Please don't mix create_cq and create_cq_umem.
> https://lore.kernel.org/linux-rdma/20260213-refactor-umem-v1-15-f3be85847922@nvidia.com/T/#u
Either we drop this one patch and put those 50 ahead of it, or we just
take this one and rebase the above.. The above has the advantage that
it enables all drivers to support cq dmabuf in one giant shot.
However, frankly I'm getting tired of looking at this bnxt_re stuff so
I'd like to just see it done.
Jason
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH rdma-next v12 5/6] RDMA/bnxt_re: Support dmabuf for CQ rings
2026-02-13 14:54 ` Jason Gunthorpe
@ 2026-02-13 15:03 ` Sriharsha Basavapatna
2026-02-15 7:07 ` Leon Romanovsky
2026-02-13 16:18 ` Leon Romanovsky
2026-02-13 16:22 ` Leon Romanovsky
2 siblings, 1 reply; 15+ messages in thread
From: Sriharsha Basavapatna @ 2026-02-13 15:03 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Leon Romanovsky, linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
[-- Attachment #1: Type: text/plain, Size: 1124 bytes --]
On Fri, Feb 13, 2026 at 8:24 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Fri, Feb 13, 2026 at 01:12:56PM +0200, Leon Romanovsky wrote:
> > > +int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
> > > + struct uverbs_attr_bundle *attrs)
> > > +{
> > > + return bnxt_re_create_cq_umem(ibcq, attr, NULL, attrs);
> > > +}
> >
> > Please don't mix create_cq and create_cq_umem.
> > https://lore.kernel.org/linux-rdma/20260213-refactor-umem-v1-15-f3be85847922@nvidia.com/T/#u
>
> Either we drop this one patch and put those 50 ahead of it, or we just
> take this one and rebase the above.. The above has the advantage that
> it enables all drivers to support cq dmabuf in one giant shot.
>
> However, frankly I'm getting tired of looking at this bnxt_re stuff so
> I'd like to just see it done.
>
> Jason
We would really appreciate it if this bnxt_re series can be merged first.
We have already been reviewing the changes in Leon's series and we
have noticed at least one issue with the bnxt_re changes. We will
reply on that thread.
Thanks,
-Harsha
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5505 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH rdma-next v12 5/6] RDMA/bnxt_re: Support dmabuf for CQ rings
2026-02-13 14:54 ` Jason Gunthorpe
2026-02-13 15:03 ` Sriharsha Basavapatna
@ 2026-02-13 16:18 ` Leon Romanovsky
2026-02-13 16:22 ` Leon Romanovsky
2 siblings, 0 replies; 15+ messages in thread
From: Leon Romanovsky @ 2026-02-13 16:18 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Sriharsha Basavapatna, linux-rdma, andrew.gospodarek,
selvin.xavier, kalesh-anakkur.purayil
On Fri, Feb 13, 2026 at 10:54:25AM -0400, Jason Gunthorpe wrote:
> On Fri, Feb 13, 2026 at 01:12:56PM +0200, Leon Romanovsky wrote:
> > > +int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
> > > + struct uverbs_attr_bundle *attrs)
> > > +{
> > > + return bnxt_re_create_cq_umem(ibcq, attr, NULL, attrs);
> > > +}
> >
> > Please don't mix create_cq and create_cq_umem.
> > https://lore.kernel.org/linux-rdma/20260213-refactor-umem-v1-15-f3be85847922@nvidia.com/T/#u
>
> Either we drop this one patch and put those 50 ahead of it, or we just
> take this one and rebase the above.. The above has the advantage that
> it enables all drivers to support cq dmabuf in one giant shot.
We don't need to take ALL patches now, it will be enough to take first 8
with core changes + bnge_re patch.
Thanks
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH rdma-next v12 5/6] RDMA/bnxt_re: Support dmabuf for CQ rings
2026-02-13 14:54 ` Jason Gunthorpe
2026-02-13 15:03 ` Sriharsha Basavapatna
2026-02-13 16:18 ` Leon Romanovsky
@ 2026-02-13 16:22 ` Leon Romanovsky
2026-02-16 15:00 ` Sriharsha Basavapatna
2 siblings, 1 reply; 15+ messages in thread
From: Leon Romanovsky @ 2026-02-13 16:22 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Sriharsha Basavapatna, linux-rdma, andrew.gospodarek,
selvin.xavier, kalesh-anakkur.purayil
On Fri, Feb 13, 2026 at 10:54:25AM -0400, Jason Gunthorpe wrote:
> On Fri, Feb 13, 2026 at 01:12:56PM +0200, Leon Romanovsky wrote:
> > > +int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
> > > + struct uverbs_attr_bundle *attrs)
> > > +{
> > > + return bnxt_re_create_cq_umem(ibcq, attr, NULL, attrs);
> > > +}
> >
> > Please don't mix create_cq and create_cq_umem.
> > https://lore.kernel.org/linux-rdma/20260213-refactor-umem-v1-15-f3be85847922@nvidia.com/T/#u
>
> Either we drop this one patch and put those 50 ahead of it, or we just
> take this one and rebase the above.. The above has the advantage that
> it enables all drivers to support cq dmabuf in one giant shot.
>
> However, frankly I'm getting tired of looking at this bnxt_re stuff so
> I'd like to just see it done.
In addition, push them to create 2 separate functions.
One is .create_cq_umem() for uverbs flow and another .create_cq()
variant for kernel flow.
bnxt_re_create_cq()
{
if (udata)
return bnxt_re_create_cq_umem()
.... <kernel CQ>
}
It will allow me to rebase my series more easily.
Thanks
>
> Jason
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH rdma-next v12 5/6] RDMA/bnxt_re: Support dmabuf for CQ rings
2026-02-13 15:03 ` Sriharsha Basavapatna
@ 2026-02-15 7:07 ` Leon Romanovsky
0 siblings, 0 replies; 15+ messages in thread
From: Leon Romanovsky @ 2026-02-15 7:07 UTC (permalink / raw)
To: Sriharsha Basavapatna
Cc: Jason Gunthorpe, linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil
On Fri, Feb 13, 2026 at 08:33:06PM +0530, Sriharsha Basavapatna wrote:
> On Fri, Feb 13, 2026 at 8:24 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
<...>
> We have already been reviewing the changes in Leon's series and we
> have noticed at least one issue with the bnxt_re changes. We will
> reply on that thread.
Did I overlook that reply?
Thanks
>
> Thanks,
> -Harsha
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH rdma-next v12 5/6] RDMA/bnxt_re: Support dmabuf for CQ rings
2026-02-13 16:22 ` Leon Romanovsky
@ 2026-02-16 15:00 ` Sriharsha Basavapatna
2026-02-16 19:40 ` Leon Romanovsky
0 siblings, 1 reply; 15+ messages in thread
From: Sriharsha Basavapatna @ 2026-02-16 15:00 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Jason Gunthorpe, linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil, Sriharsha Basavapatna
[-- Attachment #1: Type: text/plain, Size: 1601 bytes --]
On Fri, Feb 13, 2026 at 9:52 PM Leon Romanovsky <leon@kernel.org> wrote:
>
> On Fri, Feb 13, 2026 at 10:54:25AM -0400, Jason Gunthorpe wrote:
> > On Fri, Feb 13, 2026 at 01:12:56PM +0200, Leon Romanovsky wrote:
> > > > +int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
> > > > + struct uverbs_attr_bundle *attrs)
> > > > +{
> > > > + return bnxt_re_create_cq_umem(ibcq, attr, NULL, attrs);
> > > > +}
> > >
> > > Please don't mix create_cq and create_cq_umem.
> > > https://lore.kernel.org/linux-rdma/20260213-refactor-umem-v1-15-f3be85847922@nvidia.com/T/#u
> >
> > Either we drop this one patch and put those 50 ahead of it, or we just
> > take this one and rebase the above.. The above has the advantage that
> > it enables all drivers to support cq dmabuf in one giant shot.
> >
> > However, frankly I'm getting tired of looking at this bnxt_re stuff so
> > I'd like to just see it done.
>
> In addition, push them to create 2 separate functions.
> One is .create_cq_umem() for uverbs flow and another .create_cq()
> variant for kernel flow.
>
> bnxt_re_create_cq()
> {
> if (udata)
> return bnxt_re_create_cq_umem()
>
> .... <kernel CQ>
> }
>
> It will allow me to rebase my series more easily.
>
> Thanks
The above change can be done as a separate patch once the current
bnxt_re patch series gets merged. I can push that before your patchset
is ready to be merged. If you still want this change in the current
series itself, I can do that too. Please confirm.
Thanks,
-Harsha
>
> >
> > Jason
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5505 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH rdma-next v12 5/6] RDMA/bnxt_re: Support dmabuf for CQ rings
2026-02-16 15:00 ` Sriharsha Basavapatna
@ 2026-02-16 19:40 ` Leon Romanovsky
0 siblings, 0 replies; 15+ messages in thread
From: Leon Romanovsky @ 2026-02-16 19:40 UTC (permalink / raw)
To: Sriharsha Basavapatna
Cc: Jason Gunthorpe, linux-rdma, andrew.gospodarek, selvin.xavier,
kalesh-anakkur.purayil
On Mon, Feb 16, 2026 at 08:30:35PM +0530, Sriharsha Basavapatna wrote:
> On Fri, Feb 13, 2026 at 9:52 PM Leon Romanovsky <leon@kernel.org> wrote:
> >
> > On Fri, Feb 13, 2026 at 10:54:25AM -0400, Jason Gunthorpe wrote:
> > > On Fri, Feb 13, 2026 at 01:12:56PM +0200, Leon Romanovsky wrote:
> > > > > +int bnxt_re_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
> > > > > + struct uverbs_attr_bundle *attrs)
> > > > > +{
> > > > > + return bnxt_re_create_cq_umem(ibcq, attr, NULL, attrs);
> > > > > +}
> > > >
> > > > Please don't mix create_cq and create_cq_umem.
> > > > https://lore.kernel.org/linux-rdma/20260213-refactor-umem-v1-15-f3be85847922@nvidia.com/T/#u
> > >
> > > Either we drop this one patch and put those 50 ahead of it, or we just
> > > take this one and rebase the above.. The above has the advantage that
> > > it enables all drivers to support cq dmabuf in one giant shot.
> > >
> > > However, frankly I'm getting tired of looking at this bnxt_re stuff so
> > > I'd like to just see it done.
> >
> > In addition, push them to create 2 separate functions.
> > One is .create_cq_umem() for uverbs flow and another .create_cq()
> > variant for kernel flow.
> >
> > bnxt_re_create_cq()
> > {
> > if (udata)
> > return bnxt_re_create_cq_umem()
> >
> > .... <kernel CQ>
> > }
> >
> > It will allow me to rebase my series more easily.
> >
> > Thanks
> The above change can be done as a separate patch once the current
> bnxt_re patch series gets merged. > I can push that before your patchset
> is ready to be merged. If you still want this change in the current
> series itself, I can do that too. Please confirm.
No concerns were raised about the first eight patches, so they are ready
to merge. This series, however, still requires further work.
Thanks
>
> Thanks,
> -Harsha
> >
> > >
> > > Jason
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2026-02-16 19:40 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-02-11 12:49 [PATCH rdma-next v12 0/6] RDMA/bnxt_re: Support uapi extensions Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 1/6] RDMA/bnxt_re: Move the UAPI methods to a dedicated file Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 2/6] RDMA/bnxt_re: Refactor bnxt_qplib_create_qp() function Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 3/6] RDMA/bnxt_re: Support doorbell extensions Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 4/6] RDMA/bnxt_re: Refactor bnxt_re_create_cq() Sriharsha Basavapatna
2026-02-11 12:49 ` [PATCH rdma-next v12 5/6] RDMA/bnxt_re: Support dmabuf for CQ rings Sriharsha Basavapatna
2026-02-13 11:12 ` Leon Romanovsky
2026-02-13 14:54 ` Jason Gunthorpe
2026-02-13 15:03 ` Sriharsha Basavapatna
2026-02-15 7:07 ` Leon Romanovsky
2026-02-13 16:18 ` Leon Romanovsky
2026-02-13 16:22 ` Leon Romanovsky
2026-02-16 15:00 ` Sriharsha Basavapatna
2026-02-16 19:40 ` Leon Romanovsky
2026-02-11 12:49 ` [PATCH rdma-next v12 6/6] RDMA/bnxt_re: Support application specific CQs Sriharsha Basavapatna
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox