[PATCH for-next v2 0/2] RDMA/bnxt_re: Update the toggle page handling of CQ and SRQ

[PATCH for-next v2 0/2] RDMA/bnxt_re: Update the toggle page handling of CQ and SRQ

[Selvin Xavier]

Based on the suggestion from Jason (
https://patchwork.kernel.org/project/linux-rdma/patch/20260615224751.232802-5-selvin.xavier@broadcom.com/)
, adding the uverb object to retrieve the CQ an SRQ structures while getting the
toggle mem. To work with older rdma-core, retain the existing code with
modification.

The rdma-core pull request is here: https://github.com/linux-rdma/rdma-core/pull/1761

Please review and apply the series.

Thanks,
Selvin Xavier

v1->v2 :
    - Fix the error cleanup for SRQ and CQ create paths
    - Fix a synchronization issue for the legacy path which can cause a
      UAF

Selvin Xavier (2):
  RDMA/bnxt_re: Replace per-device hash tables with per-context XArray
  RDMA/bnxt_re: Add uverbs object handle path for CQ/SRQ toggle page

 drivers/infiniband/hw/bnxt_re/bnxt_re.h  |   6 --
 drivers/infiniband/hw/bnxt_re/ib_verbs.c |  91 +++++++++++++----
 drivers/infiniband/hw/bnxt_re/ib_verbs.h |   5 +-
 drivers/infiniband/hw/bnxt_re/main.c     |   5 -
 drivers/infiniband/hw/bnxt_re/uapi.c     | 124 +++++++++++------------
 include/uapi/rdma/bnxt_re-abi.h          |   4 +
 6 files changed, 139 insertions(+), 96 deletions(-)

-- 
2.39.3

[PATCH for-next v2 1/2] RDMA/bnxt_re: Replace per-device hash tables with per-context XArray

[Selvin Xavier]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=a, Size: 12613 bytes --]

The CQ and SRQ hash tables (cq_hash, srq_hash) on struct bnxt_re_dev
were used exclusively to look up a toggle-page pointer from a
user-space-supplied hardware queue ID in the GET_TOGGLE_MEM
ioctl handler. This approach has couple of problems. First,
because the tables are per-device, any user can look up another
user's CQ or SRQ by guessing the hardware queue ID. Second,
concurrent add and remove operations on the hash table are not
protected by any lock, leaving a race window.

The correct fix is to retrieve the CQ and SRQ objects via the uverbs
object handle, which gives built-in ownership verification and reference
pinning for the duration of the ioctl. That is added in the next patch of
this series.

To maintain backward compatibility with older rdma-core versions that
do not send a uverbs object handle, the driver must continue to support
the existing TYPE + RES_ID lookup path. This patch replaces the per-device
hash tables with per-ucontext XArrays (cq_xa and srq_xa on struct
bnxt_re_ucontext), which narrows the lookup scope to the calling context,
eliminating the cross-user visibility. Also adds Xarray locking mechanism
for synchronization.

The GET_TOGGLE_MEM ioctl handler is updated to call xa_load()
in place of the now-removed bnxt_re_search_for_cq()/
bnxt_re_search_for_srq() helpers. No ABI changes are required.

Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/bnxt_re.h  |  6 --
 drivers/infiniband/hw/bnxt_re/ib_verbs.c | 84 ++++++++++++++++++------
 drivers/infiniband/hw/bnxt_re/ib_verbs.h |  4 +-
 drivers/infiniband/hw/bnxt_re/main.c     |  5 --
 drivers/infiniband/hw/bnxt_re/uapi.c     | 55 ++++------------
 5 files changed, 80 insertions(+), 74 deletions(-)

diff --git a/drivers/infiniband/hw/bnxt_re/bnxt_re.h b/drivers/infiniband/hw/bnxt_re/bnxt_re.h
index 3a7ce4729fcf..a43e678151d3 100644
--- a/drivers/infiniband/hw/bnxt_re/bnxt_re.h
+++ b/drivers/infiniband/hw/bnxt_re/bnxt_re.h
@@ -41,7 +41,6 @@
 #define __BNXT_RE_H__
 #include <rdma/uverbs_ioctl.h>
 #include "hw_counters.h"
-#include <linux/hashtable.h>
 #define ROCE_DRV_MODULE_NAME		"bnxt_re"
 
 #define BNXT_RE_DESC	"Broadcom NetXtreme-C/E RoCE Driver"
@@ -158,9 +157,6 @@ struct bnxt_re_nq_record {
 	struct mutex		load_lock;
 };
 
-#define MAX_CQ_HASH_BITS		(16)
-#define MAX_SRQ_HASH_BITS		(16)
-
 static inline bool bnxt_re_chip_gen_p7(u16 chip_num)
 {
 	return (chip_num == CHIP_NUM_58818 ||
@@ -215,8 +211,6 @@ struct bnxt_re_dev {
 	struct bnxt_re_pacing pacing;
 	struct work_struct dbq_fifo_check_work;
 	struct delayed_work dbq_pacing_work;
-	DECLARE_HASHTABLE(cq_hash, MAX_CQ_HASH_BITS);
-	DECLARE_HASHTABLE(srq_hash, MAX_SRQ_HASH_BITS);
 	struct dentry			*dbg_root;
 	struct dentry			*qp_debugfs;
 	unsigned long			event_bitmap;
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 565762529007..d1eebd7b56f4 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -38,6 +38,7 @@
 
 #include <linux/interrupt.h>
 #include <linux/types.h>
+#include <linux/xarray.h>
 #include <linux/pci.h>
 #include <linux/netdevice.h>
 #include <linux/if_ether.h>
@@ -51,8 +52,6 @@
 #include <rdma/ib_cache.h>
 #include <rdma/ib_pma.h>
 #include <rdma/uverbs_ioctl.h>
-#include <linux/hashtable.h>
-
 #include "roce_hsi.h"
 #include "qplib_res.h"
 #include "qplib_sp.h"
@@ -2152,11 +2151,19 @@ int bnxt_re_destroy_srq(struct ib_srq *ib_srq, struct ib_udata *udata)
 	if (ret)
 		return ret;
 
-	if (rdev->chip_ctx->modes.toggle_bits & BNXT_QPLIB_SRQ_TOGGLE_BIT)
-		hash_del(&srq->hash_entry);
 	bnxt_qplib_destroy_srq(&rdev->qplib_res, qplib_srq);
-	if (rdev->chip_ctx->modes.toggle_bits & BNXT_QPLIB_SRQ_TOGGLE_BIT)
-		free_page((unsigned long)srq->uctx_srq_page);
+	if (rdev->chip_ctx->modes.toggle_bits & BNXT_QPLIB_SRQ_TOGGLE_BIT) {
+		struct bnxt_re_ucontext *uctx =
+			rdma_udata_to_drv_context(udata, struct bnxt_re_ucontext, ib_uctx);
+
+		if (uctx) {
+			/* similar to cq, use __xa_erase() with the lock already held */
+			xa_lock(&uctx->srq_xa);
+			__xa_erase(&uctx->srq_xa, srq->qplib_srq.id);
+			xa_unlock(&uctx->srq_xa);
+			free_page((unsigned long)srq->uctx_srq_page);
+		}
+	}
 	ib_umem_release(srq->umem);
 	atomic_dec(&rdev->stats.res.srq_count);
 	return ib_respond_empty_udata(udata);
@@ -2263,20 +2270,21 @@ int bnxt_re_create_srq(struct ib_srq *ib_srq,
 
 		resp.srqid = srq->qplib_srq.id;
 		if (rdev->chip_ctx->modes.toggle_bits & BNXT_QPLIB_SRQ_TOGGLE_BIT) {
-			hash_add(rdev->srq_hash, &srq->hash_entry, srq->qplib_srq.id);
 			srq->uctx_srq_page = (void *)get_zeroed_page(GFP_KERNEL);
 			if (!srq->uctx_srq_page) {
 				rc = -ENOMEM;
-				goto fail;
+				goto fail_destroy_srq;
+			}
+			if (xa_is_err(xa_store(&uctx->srq_xa, srq->qplib_srq.id,
+					       srq, GFP_KERNEL))) {
+				rc = -ENOMEM;
+				goto fail_free_toggle;
 			}
 			resp.comp_mask |= BNXT_RE_SRQ_TOGGLE_PAGE_SUPPORT;
 		}
 		rc = ib_respond_udata(udata, resp);
-		if (rc) {
-			bnxt_qplib_destroy_srq(&rdev->qplib_res,
-					       &srq->qplib_srq);
-			goto fail;
-		}
+		if (rc)
+			goto fail_respond;
 	}
 	active_srqs = atomic_inc_return(&rdev->stats.res.srq_count);
 	if (active_srqs > rdev->stats.res.srq_watermark)
@@ -2285,6 +2293,16 @@ int bnxt_re_create_srq(struct ib_srq *ib_srq,
 
 	return 0;
 
+fail_respond:
+	if (rdev->chip_ctx->modes.toggle_bits & BNXT_QPLIB_SRQ_TOGGLE_BIT) {
+		xa_lock(&uctx->srq_xa);
+		__xa_erase(&uctx->srq_xa, srq->qplib_srq.id);
+		xa_unlock(&uctx->srq_xa);
+	}
+fail_free_toggle:
+	free_page((unsigned long)srq->uctx_srq_page);
+fail_destroy_srq:
+	bnxt_qplib_destroy_srq(&rdev->qplib_res, &srq->qplib_srq);
 fail:
 	ib_umem_release(srq->umem);
 exit:
@@ -3475,11 +3493,24 @@ int bnxt_re_destroy_cq(struct ib_cq *ib_cq, struct ib_udata *udata)
 	if (ret)
 		return ret;
 
-	if (cctx->modes.toggle_bits & BNXT_QPLIB_CQ_TOGGLE_BIT)
-		hash_del(&cq->hash_entry);
 	bnxt_qplib_destroy_cq(&rdev->qplib_res, &cq->qplib_cq);
-	if (cctx->modes.toggle_bits & BNXT_QPLIB_CQ_TOGGLE_BIT)
-		free_page((unsigned long)cq->uctx_cq_page);
+	if (cctx->modes.toggle_bits & BNXT_QPLIB_CQ_TOGGLE_BIT) {
+		struct bnxt_re_ucontext *uctx =
+			rdma_udata_to_drv_context(udata, struct bnxt_re_ucontext, ib_uctx);
+
+		if (uctx) {
+		/*
+		 * Hold xa_lock across the erase so that GET_TOGGLE_MEM's
+		 * xa_lock + xa_load + dereference region is atomic with respect
+		 * to removal. xa_erase() would re-acquire the same lock and
+		 * deadlock; use __xa_erase() with the lock already held.
+		 */
+			xa_lock(&uctx->cq_xa);
+			__xa_erase(&uctx->cq_xa, cq->qplib_cq.id);
+			xa_unlock(&uctx->cq_xa);
+			free_page((unsigned long)cq->uctx_cq_page);
+		}
+	}
 
 	bnxt_re_put_nq(rdev, nq);
 
@@ -3554,14 +3585,15 @@ int bnxt_re_create_user_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *att
 	spin_lock_init(&cq->cq_lock);
 
 	if (cctx->modes.toggle_bits & BNXT_QPLIB_CQ_TOGGLE_BIT) {
-		hash_add(rdev->cq_hash, &cq->hash_entry, cq->qplib_cq.id);
-		/* Allocate a page */
 		cq->uctx_cq_page = (void *)get_zeroed_page(GFP_KERNEL);
 		if (!cq->uctx_cq_page) {
 			rc = -ENOMEM;
 			goto destroy_cq;
 		}
-
+		if (xa_is_err(xa_store(&uctx->cq_xa, cq->qplib_cq.id, cq, GFP_KERNEL))) {
+			rc = -ENOMEM;
+			goto free_toggle_page;
+		}
 		resp.comp_mask |= BNXT_RE_CQ_TOGGLE_PAGE_SUPPORT;
 	}
 	resp.cqid = cq->qplib_cq.id;
@@ -3574,6 +3606,12 @@ int bnxt_re_create_user_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *att
 	return 0;
 
 free_mem:
+	if (cctx->modes.toggle_bits & BNXT_QPLIB_CQ_TOGGLE_BIT) {
+		xa_lock(&uctx->cq_xa);
+		__xa_erase(&uctx->cq_xa, cq->qplib_cq.id);
+		xa_unlock(&uctx->cq_xa);
+	}
+free_toggle_page:
 	free_page((unsigned long)cq->uctx_cq_page);
 destroy_cq:
 	bnxt_qplib_destroy_cq(&rdev->qplib_res, &cq->qplib_cq);
@@ -4823,6 +4861,9 @@ int bnxt_re_alloc_ucontext(struct ib_ucontext *ctx, struct ib_udata *udata)
 		}
 	}
 
+	xa_init(&uctx->cq_xa);
+	xa_init(&uctx->srq_xa);
+
 	rc = ib_respond_udata(udata, resp);
 	if (rc)
 		goto cfail;
@@ -4848,6 +4889,9 @@ void bnxt_re_dealloc_ucontext(struct ib_ucontext *ib_uctx)
 	if (uctx->shpg)
 		free_page((unsigned long)uctx->shpg);
 
+	xa_destroy(&uctx->cq_xa);
+	xa_destroy(&uctx->srq_xa);
+
 	if (uctx->dpi.dbr) {
 		/* Free DPI only if this is the first PD allocated by the
 		 * application and mark the context dpi as NULL
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
index 22bf81668cfb..76f407cd3435 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
@@ -78,7 +78,6 @@ struct bnxt_re_srq {
 	struct ib_umem		*umem;
 	spinlock_t		lock;		/* protect srq */
 	void			*uctx_srq_page;
-	struct hlist_node       hash_entry;
 };
 
 struct bnxt_re_qp {
@@ -113,7 +112,6 @@ struct bnxt_re_cq {
 	struct ib_umem		*resize_umem;
 	int			resize_cqe;
 	void			*uctx_cq_page;
-	struct hlist_node	hash_entry;
 };
 
 struct bnxt_re_mr {
@@ -147,6 +145,8 @@ struct bnxt_re_ucontext {
 	void			*shpg;
 	spinlock_t		sh_lock;	/* protect shpg */
 	struct rdma_user_mmap_entry *shpage_mmap;
+	struct xarray		cq_xa;		/* cqid → bnxt_re_cq, for toggle page lookup */
+	struct xarray		srq_xa;		/* srqid → bnxt_re_srq, for toggle page lookup */
 	u64 cmask;
 };
 
diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c
index d25fdc458120..637f023b18ac 100644
--- a/drivers/infiniband/hw/bnxt_re/main.c
+++ b/drivers/infiniband/hw/bnxt_re/main.c
@@ -54,7 +54,6 @@
 #include <rdma/ib_user_verbs.h>
 #include <rdma/ib_umem.h>
 #include <rdma/ib_addr.h>
-#include <linux/hashtable.h>
 #include <linux/bnxt/ulp.h>
 
 #include "roce_hsi.h"
@@ -2337,10 +2336,6 @@ static int bnxt_re_dev_init(struct bnxt_re_dev *rdev, u8 op_type)
 		if (!(rdev->qplib_res.en_dev->flags & BNXT_EN_FLAG_ROCE_VF_RES_MGMT))
 			bnxt_re_vf_res_config(rdev);
 	}
-	hash_init(rdev->cq_hash);
-	if (rdev->chip_ctx->modes.toggle_bits & BNXT_QPLIB_SRQ_TOGGLE_BIT)
-		hash_init(rdev->srq_hash);
-
 	bnxt_re_debugfs_add_pdev(rdev);
 
 	bnxt_re_init_dcb_wq(rdev);
diff --git a/drivers/infiniband/hw/bnxt_re/uapi.c b/drivers/infiniband/hw/bnxt_re/uapi.c
index 263238a6e4cd..7e2acd0933f7 100644
--- a/drivers/infiniband/hw/bnxt_re/uapi.c
+++ b/drivers/infiniband/hw/bnxt_re/uapi.c
@@ -22,32 +22,6 @@
 #include "bnxt_re.h"
 #include "ib_verbs.h"
 
-static struct bnxt_re_cq *bnxt_re_search_for_cq(struct bnxt_re_dev *rdev, u32 cq_id)
-{
-	struct bnxt_re_cq *cq = NULL, *tmp_cq;
-
-	hash_for_each_possible(rdev->cq_hash, tmp_cq, hash_entry, cq_id) {
-		if (tmp_cq->qplib_cq.id == cq_id) {
-			cq = tmp_cq;
-			break;
-		}
-	}
-	return cq;
-}
-
-static struct bnxt_re_srq *bnxt_re_search_for_srq(struct bnxt_re_dev *rdev, u32 srq_id)
-{
-	struct bnxt_re_srq *srq = NULL, *tmp_srq;
-
-	hash_for_each_possible(rdev->srq_hash, tmp_srq, hash_entry, srq_id) {
-		if (tmp_srq->qplib_srq.id == srq_id) {
-			srq = tmp_srq;
-			break;
-		}
-	}
-	return srq;
-}
-
 static int UVERBS_HANDLER(BNXT_RE_METHOD_NOTIFY_DRV)(struct uverbs_attr_bundle *attrs)
 {
 	struct bnxt_re_ucontext *uctx;
@@ -246,10 +220,7 @@ static int UVERBS_HANDLER(BNXT_RE_METHOD_GET_TOGGLE_MEM)(struct uverbs_attr_bund
 	struct bnxt_re_user_mmap_entry *entry;
 	struct bnxt_re_ucontext *uctx;
 	struct ib_ucontext *ib_uctx;
-	struct bnxt_re_dev *rdev;
-	struct bnxt_re_srq *srq;
 	u32 length = PAGE_SIZE;
-	struct bnxt_re_cq *cq;
 	u64 mem_offset;
 	u32 offset = 0;
 	u64 addr = 0;
@@ -265,31 +236,33 @@ static int UVERBS_HANDLER(BNXT_RE_METHOD_GET_TOGGLE_MEM)(struct uverbs_attr_bund
 		return err;
 
 	uctx = container_of(ib_uctx, struct bnxt_re_ucontext, ib_uctx);
-	rdev = uctx->rdev;
 	err = uverbs_copy_from(&res_id, attrs, BNXT_RE_TOGGLE_MEM_RES_ID);
 	if (err)
 		return err;
 
 	switch (res_type) {
 	case BNXT_RE_CQ_TOGGLE_MEM:
-		cq = bnxt_re_search_for_cq(rdev, res_id);
-		if (!cq)
-			return -EINVAL;
+		struct bnxt_re_cq *cq;
 
-		addr = (u64)cq->uctx_cq_page;
+		xa_lock(&uctx->cq_xa);
+		cq = xa_load(&uctx->cq_xa, res_id);
+		if (cq)
+			addr = (u64)cq->uctx_cq_page;
+		xa_unlock(&uctx->cq_xa);
 		if (!addr)
-			return -EOPNOTSUPP;
+			return -EINVAL;
 		break;
 	case BNXT_RE_SRQ_TOGGLE_MEM:
-		srq = bnxt_re_search_for_srq(rdev, res_id);
-		if (!srq)
-			return -EINVAL;
+		struct bnxt_re_srq *srq;
 
-		addr = (u64)srq->uctx_srq_page;
+		xa_lock(&uctx->srq_xa);
+		srq = xa_load(&uctx->srq_xa, res_id);
+		if (srq)
+			addr = (u64)srq->uctx_srq_page;
+		xa_unlock(&uctx->srq_xa);
 		if (!addr)
-			return -EOPNOTSUPP;
+			return -EINVAL;
 		break;
-
 	default:
 		return -EOPNOTSUPP;
 	}
-- 
2.39.3

[PATCH for-next v2 2/2] RDMA/bnxt_re: Add uverbs object handle path for CQ/SRQ toggle page

[Selvin Xavier]

The current GET_TOGGLE_MEM ioctl requires the caller to supply
a type enum and a raw hardware queue ID (RES_ID). The kernel
looks up the CQ or SRQ by that ID without verifying that the
caller owns the resource.

Add a new, preferred code path that accepts standard uverbs
object handles (BNXT_RE_TOGGLE_MEM_CQ_HANDLE /
BNXT_RE_TOGGLE_MEM_SRQ_HANDLE) instead.

Only newer rdma-core versions support this path. Capability is
negotiated during context creation using the req mask
(BNXT_RE_COMP_MASK_REQ_UCNTX_TOGGLE_MEM_UOBJ_SUPPORT) and resp
mask (BNXT_RE_UCNTX_CMASK_TOGGLE_MEM_UOBJ_SUPPORT).
The existing TYPE + RES_ID path is retained for backward
compatibility with older rdma-core.

Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
---
 drivers/infiniband/hw/bnxt_re/ib_verbs.c |  7 +-
 drivers/infiniband/hw/bnxt_re/ib_verbs.h |  1 +
 drivers/infiniband/hw/bnxt_re/uapi.c     | 99 +++++++++++++++---------
 include/uapi/rdma/bnxt_re-abi.h          |  4 +
 4 files changed, 74 insertions(+), 37 deletions(-)

diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index d1eebd7b56f4..423c8f3184bb 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -4846,7 +4846,8 @@ int bnxt_re_alloc_ucontext(struct ib_ucontext *ctx, struct ib_udata *udata)
 		rc = ib_copy_validate_udata_in_cm(
 			udata, ureq, comp_mask,
 			BNXT_RE_COMP_MASK_REQ_UCNTX_POW2_SUPPORT |
-				BNXT_RE_COMP_MASK_REQ_UCNTX_VAR_WQE_SUPPORT);
+				BNXT_RE_COMP_MASK_REQ_UCNTX_VAR_WQE_SUPPORT |
+				BNXT_RE_COMP_MASK_REQ_UCNTX_TOGGLE_MEM_UOBJ_SUPPORT);
 		if (rc)
 			goto cfail;
 		if (ureq.comp_mask & BNXT_RE_COMP_MASK_REQ_UCNTX_POW2_SUPPORT) {
@@ -4859,6 +4860,10 @@ int bnxt_re_alloc_ucontext(struct ib_ucontext *ctx, struct ib_udata *udata)
 			if (resp.mode == BNXT_QPLIB_WQE_MODE_VARIABLE)
 				uctx->cmask |= BNXT_RE_UCNTX_CAP_VAR_WQE_ENABLED;
 		}
+		if (ureq.comp_mask & BNXT_RE_COMP_MASK_REQ_UCNTX_TOGGLE_MEM_UOBJ_SUPPORT) {
+			uctx->cmask |= BNXT_RE_UCNTX_CAP_TOGGLE_MEM_UOBJ;
+			resp.comp_mask |= BNXT_RE_UCNTX_CMASK_TOGGLE_MEM_UOBJ_SUPPORT;
+		}
 	}
 
 	xa_init(&uctx->cq_xa);
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
index 76f407cd3435..85e594a25448 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h
@@ -193,6 +193,7 @@ static inline u16 bnxt_re_get_rwqe_size(int nsge)
 enum {
 	BNXT_RE_UCNTX_CAP_POW2_DISABLED = 0x1ULL,
 	BNXT_RE_UCNTX_CAP_VAR_WQE_ENABLED = 0x2ULL,
+	BNXT_RE_UCNTX_CAP_TOGGLE_MEM_UOBJ = 0x4ULL,
 };
 
 static inline u32 bnxt_re_init_depth(u32 ent, u32 max,
diff --git a/drivers/infiniband/hw/bnxt_re/uapi.c b/drivers/infiniband/hw/bnxt_re/uapi.c
index 7e2acd0933f7..45dcaa49d6a8 100644
--- a/drivers/infiniband/hw/bnxt_re/uapi.c
+++ b/drivers/infiniband/hw/bnxt_re/uapi.c
@@ -216,57 +216,76 @@ static int UVERBS_HANDLER(BNXT_RE_METHOD_GET_TOGGLE_MEM)(struct uverbs_attr_bund
 {
 	struct ib_uobject *uobj = uverbs_attr_get_uobject(attrs, BNXT_RE_TOGGLE_MEM_HANDLE);
 	enum bnxt_re_mmap_flag mmap_flag = BNXT_RE_MMAP_TOGGLE_PAGE;
-	enum bnxt_re_get_toggle_mem_type res_type;
 	struct bnxt_re_user_mmap_entry *entry;
 	struct bnxt_re_ucontext *uctx;
 	struct ib_ucontext *ib_uctx;
+	struct ib_uobject *res_uobj;
 	u32 length = PAGE_SIZE;
 	u64 mem_offset;
 	u32 offset = 0;
 	u64 addr = 0;
-	u32 res_id;
 	int err;
 
 	ib_uctx = ib_uverbs_get_ucontext(attrs);
 	if (IS_ERR(ib_uctx))
 		return PTR_ERR(ib_uctx);
 
-	err = uverbs_get_const(&res_type, attrs, BNXT_RE_TOGGLE_MEM_TYPE);
-	if (err)
-		return err;
-
 	uctx = container_of(ib_uctx, struct bnxt_re_ucontext, ib_uctx);
-	err = uverbs_copy_from(&res_id, attrs, BNXT_RE_TOGGLE_MEM_RES_ID);
-	if (err)
-		return err;
-
-	switch (res_type) {
-	case BNXT_RE_CQ_TOGGLE_MEM:
-		struct bnxt_re_cq *cq;
+	res_uobj = uverbs_attr_get_uobject(attrs, BNXT_RE_TOGGLE_MEM_CQ_HANDLE);
+	if (!IS_ERR(res_uobj)) {
+		struct bnxt_re_cq *cq =
+			container_of((struct ib_cq *)res_uobj->object,
+				     struct bnxt_re_cq, ib_cq);
+
+		addr = (u64)cq->uctx_cq_page;
+	} else {
+		res_uobj = uverbs_attr_get_uobject(attrs, BNXT_RE_TOGGLE_MEM_SRQ_HANDLE);
+		if (!IS_ERR(res_uobj)) {
+			struct bnxt_re_srq *srq =
+				container_of((struct ib_srq *)res_uobj->object,
+					     struct bnxt_re_srq, ib_srq);
 
-		xa_lock(&uctx->cq_xa);
-		cq = xa_load(&uctx->cq_xa, res_id);
-		if (cq)
-			addr = (u64)cq->uctx_cq_page;
-		xa_unlock(&uctx->cq_xa);
-		if (!addr)
-			return -EINVAL;
-		break;
-	case BNXT_RE_SRQ_TOGGLE_MEM:
-		struct bnxt_re_srq *srq;
-
-		xa_lock(&uctx->srq_xa);
-		srq = xa_load(&uctx->srq_xa, res_id);
-		if (srq)
 			addr = (u64)srq->uctx_srq_page;
-		xa_unlock(&uctx->srq_xa);
-		if (!addr)
-			return -EINVAL;
-		break;
-	default:
-		return -EOPNOTSUPP;
+		} else {
+			/*
+			 * Legacy path: old libbnxt_re sends TYPE + RES_ID.
+			 * Look up the CQ or SRQ in the per-context XArray
+			 */
+			enum bnxt_re_get_toggle_mem_type res_type;
+			u32 res_id;
+
+			err = uverbs_get_const(&res_type, attrs,
+					       BNXT_RE_TOGGLE_MEM_TYPE);
+			if (err)
+				return err;
+			err = uverbs_copy_from(&res_id, attrs,
+					       BNXT_RE_TOGGLE_MEM_RES_ID);
+			if (err)
+				return err;
+
+			if (res_type == BNXT_RE_CQ_TOGGLE_MEM) {
+				struct bnxt_re_cq *cq;
+
+				xa_lock(&uctx->cq_xa);
+				cq = xa_load(&uctx->cq_xa, res_id);
+				if (cq)
+					addr = (u64)cq->uctx_cq_page;
+				xa_unlock(&uctx->cq_xa);
+			} else if (res_type == BNXT_RE_SRQ_TOGGLE_MEM) {
+				struct bnxt_re_srq *srq;
+
+				xa_lock(&uctx->srq_xa);
+				srq = xa_load(&uctx->srq_xa, res_id);
+				if (srq)
+					addr = (u64)srq->uctx_srq_page;
+				xa_unlock(&uctx->srq_xa);
+			}
+		}
 	}
 
+	if (!addr)
+		return -EOPNOTSUPP;
+
 	entry = bnxt_re_mmap_entry_insert(uctx, addr, mmap_flag, &mem_offset);
 	if (!entry)
 		return -ENOMEM;
@@ -308,10 +327,10 @@ DECLARE_UVERBS_NAMED_METHOD(BNXT_RE_METHOD_GET_TOGGLE_MEM,
 					    UA_MANDATORY),
 			    UVERBS_ATTR_CONST_IN(BNXT_RE_TOGGLE_MEM_TYPE,
 						 enum bnxt_re_get_toggle_mem_type,
-						 UA_MANDATORY),
+						 UA_OPTIONAL),
 			    UVERBS_ATTR_PTR_IN(BNXT_RE_TOGGLE_MEM_RES_ID,
 					       UVERBS_ATTR_TYPE(u32),
-					       UA_MANDATORY),
+					       UA_OPTIONAL),
 			    UVERBS_ATTR_PTR_OUT(BNXT_RE_TOGGLE_MEM_MMAP_PAGE,
 						UVERBS_ATTR_TYPE(u64),
 						UA_MANDATORY),
@@ -320,7 +339,15 @@ DECLARE_UVERBS_NAMED_METHOD(BNXT_RE_METHOD_GET_TOGGLE_MEM,
 						UA_MANDATORY),
 			    UVERBS_ATTR_PTR_OUT(BNXT_RE_TOGGLE_MEM_MMAP_LENGTH,
 						UVERBS_ATTR_TYPE(u32),
-						UA_MANDATORY));
+						UA_MANDATORY),
+			    UVERBS_ATTR_IDR(BNXT_RE_TOGGLE_MEM_CQ_HANDLE,
+					    UVERBS_OBJECT_CQ,
+					    UVERBS_ACCESS_READ,
+					    UA_OPTIONAL),
+			    UVERBS_ATTR_IDR(BNXT_RE_TOGGLE_MEM_SRQ_HANDLE,
+					    UVERBS_OBJECT_SRQ,
+					    UVERBS_ACCESS_READ,
+					    UA_OPTIONAL));
 
 DECLARE_UVERBS_NAMED_METHOD_DESTROY(BNXT_RE_METHOD_RELEASE_TOGGLE_MEM,
 				    UVERBS_ATTR_IDR(BNXT_RE_RELEASE_TOGGLE_MEM_HANDLE,
diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h
index a4599d7b736a..a6cfd68ed8f5 100644
--- a/include/uapi/rdma/bnxt_re-abi.h
+++ b/include/uapi/rdma/bnxt_re-abi.h
@@ -57,6 +57,7 @@ enum {
 	BNXT_RE_UCNTX_CMASK_POW2_DISABLED = 0x10ULL,
 	BNXT_RE_UCNTX_CMASK_MSN_TABLE_ENABLED = 0x40,
 	BNXT_RE_UCNTX_CMASK_QP_RATE_LIMIT_ENABLED = 0x80ULL,
+	BNXT_RE_UCNTX_CMASK_TOGGLE_MEM_UOBJ_SUPPORT = 0x400000ULL,
 };
 
 enum bnxt_re_wqe_mode {
@@ -68,6 +69,7 @@ enum bnxt_re_wqe_mode {
 enum {
 	BNXT_RE_COMP_MASK_REQ_UCNTX_POW2_SUPPORT = 0x01,
 	BNXT_RE_COMP_MASK_REQ_UCNTX_VAR_WQE_SUPPORT = 0x02,
+	BNXT_RE_COMP_MASK_REQ_UCNTX_TOGGLE_MEM_UOBJ_SUPPORT = 0x20,
 };
 
 struct bnxt_re_uctx_req {
@@ -218,6 +220,8 @@ enum bnxt_re_var_toggle_mem_attrs {
 	BNXT_RE_TOGGLE_MEM_MMAP_PAGE,
 	BNXT_RE_TOGGLE_MEM_MMAP_OFFSET,
 	BNXT_RE_TOGGLE_MEM_MMAP_LENGTH,
+	BNXT_RE_TOGGLE_MEM_CQ_HANDLE,
+	BNXT_RE_TOGGLE_MEM_SRQ_HANDLE,
 };
 
 enum bnxt_re_toggle_mem_attrs {
-- 
2.39.3

Re: [PATCH for-next v2 2/2] RDMA/bnxt_re: Add uverbs object handle path for CQ/SRQ toggle page

[Jason Gunthorpe]

On Wed, Jun 24, 2026 at 03:39:27PM -0700, Selvin Xavier wrote:
> +		} else {
> +			/*
> +			 * Legacy path: old libbnxt_re sends TYPE + RES_ID.
> +			 * Look up the CQ or SRQ in the per-context XArray
> +			 */
> +			enum bnxt_re_get_toggle_mem_type res_type;
> +			u32 res_id;
> +
> +			err = uverbs_get_const(&res_type, attrs,
> +					       BNXT_RE_TOGGLE_MEM_TYPE);
> +			if (err)
> +				return err;
> +			err = uverbs_copy_from(&res_id, attrs,
> +					       BNXT_RE_TOGGLE_MEM_RES_ID);
> +			if (err)
> +				return err;
> +
> +			if (res_type == BNXT_RE_CQ_TOGGLE_MEM) {
> +				struct bnxt_re_cq *cq;
> +
> +				xa_lock(&uctx->cq_xa);
> +				cq = xa_load(&uctx->cq_xa, res_id);
> +				if (cq)
> +					addr = (u64)cq->uctx_cq_page;
> +				xa_unlock(&uctx->cq_xa);

Does this have the same basic UAF issue on addr?

Can you change this around so it tracks the uobject in the xarray an
uses proper uobject locking consistently in both legs?

If I understand this right it should also be holding onto a refcount
of the uobject while the toggle mem exists to ensure it doesn't free
in the wrong order?

Jason

Re: [PATCH for-next v2 2/2] RDMA/bnxt_re: Add uverbs object handle path for CQ/SRQ toggle page

[Selvin Xavier]

[-- Attachment #1: Type: text/plain, Size: 1987 bytes --]

On Mon, Jun 29, 2026 at 6:45 PM Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> On Wed, Jun 24, 2026 at 03:39:27PM -0700, Selvin Xavier wrote:
> > +             } else {
> > +                     /*
> > +                      * Legacy path: old libbnxt_re sends TYPE + RES_ID.
> > +                      * Look up the CQ or SRQ in the per-context XArray
> > +                      */
> > +                     enum bnxt_re_get_toggle_mem_type res_type;
> > +                     u32 res_id;
> > +
> > +                     err = uverbs_get_const(&res_type, attrs,
> > +                                            BNXT_RE_TOGGLE_MEM_TYPE);
> > +                     if (err)
> > +                             return err;
> > +                     err = uverbs_copy_from(&res_id, attrs,
> > +                                            BNXT_RE_TOGGLE_MEM_RES_ID);
> > +                     if (err)
> > +                             return err;
> > +
> > +                     if (res_type == BNXT_RE_CQ_TOGGLE_MEM) {
> > +                             struct bnxt_re_cq *cq;
> > +
> > +                             xa_lock(&uctx->cq_xa);
> > +                             cq = xa_load(&uctx->cq_xa, res_id);
> > +                             if (cq)
> > +                                     addr = (u64)cq->uctx_cq_page;
> > +                             xa_unlock(&uctx->cq_xa);
>
> Does this have the same basic UAF issue on addr?
Yes. It is possible. Will rework.
>
> Can you change this around so it tracks the uobject in the xarray an
> uses proper uobject locking consistently in both legs?
yes.  I will make those changes.
>
> If I understand this right it should also be holding onto a refcount
> of the uobject while the toggle mem exists to ensure it doesn't free
> in the wrong order?
Ok. you are right. We have to provide a way to synchronize the mmap
with the resource's destruction.  Will include in v3.

Thanks,
Selvin
>
> Jason

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5473 bytes --]