From: Leon Romanovsky <leon@kernel.org>
To: Jason Gunthorpe <jgg@ziepe.ca>,
Selvin Xavier <selvin.xavier@broadcom.com>,
Kalesh AP <kalesh-anakkur.purayil@broadcom.com>,
Potnuri Bharat Teja <bharat@chelsio.com>,
Michael Margolin <mrgolin@amazon.com>,
Gal Pressman <gal.pressman@linux.dev>,
Yossi Leybovich <sleybo@amazon.com>,
Cheng Xu <chengyou@linux.alibaba.com>,
Kai Shen <kaishen@linux.alibaba.com>,
Chengchang Tang <tangchengchang@huawei.com>,
Junxian Huang <huangjunxian6@hisilicon.com>,
Abhijit Gangurde <abhijit.gangurde@amd.com>,
Allen Hubbe <allen.hubbe@amd.com>,
Krzysztof Czurylo <krzysztof.czurylo@intel.com>,
Tatyana Nikolova <tatyana.e.nikolova@intel.com>,
Long Li <longli@microsoft.com>,
Konstantin Taranov <kotaranov@microsoft.com>,
Yishai Hadas <yishaih@nvidia.com>,
Michal Kalderon <mkalderon@marvell.com>,
Bryan Tan <bryan-bt.tan@broadcom.com>,
Vishnu Dasa <vishnu.dasa@broadcom.com>,
Broadcom internal kernel review list
<bcm-kernel-feedback-list@broadcom.com>,
Christian Benvenuti <benve@cisco.com>,
Nelson Escobar <neescoba@cisco.com>,
Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>,
Bernard Metzler <bernard.metzler@linux.dev>,
Zhu Yanjun <zyjzyj2000@gmail.com>
Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
linux-hyperv@vger.kernel.org
Subject: Re: [PATCH rdma-next 00/50] RDMA: Ensure CQ UMEMs are managed by ib_core
Date: Wed, 25 Feb 2026 15:53:20 +0200 [thread overview]
Message-ID: <20260225135320.GF9541@unreal> (raw)
In-Reply-To: <20260213-refactor-umem-v1-0-f3be85847922@nvidia.com>
On Fri, Feb 13, 2026 at 12:57:36PM +0200, Leon Romanovsky wrote:
> Unify CQ UMEM creation, resize and release in ib_core to avoid the need
> for complex driver-side handling. This lets us rely on the internal
> reference counters of the relevant ib_XXX objects to manage UMEM
> lifetime safely and consistently.
>
> The resize cleanup made it clear that most drivers never handled this
> path correctly, and there's a good chance the functionality was never
> actually used. The most common issue was relying on the cq->resize_umem
> pointer to detect races with other CQ commands, without clearing it on
> errors and while ignoring proper locking for other CQ operations.
>
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> ---
> Leon Romanovsky (50):
> RDMA: Move DMA block iterator logic into dedicated files
> RDMA/umem: Allow including ib_umem header from any location
> RDMA/umem: Remove unnecessary includes and defines from ib_umem header
> RDMA/core: Promote UMEM to a core component
> RDMA/core: Manage CQ umem in core code
> RDMA/efa: Rely on CPU address in create‑QP
> RDMA/core: Prepare create CQ path for API unification
> RDMA/core: Reject zero CQE count
> RDMA/efa: Remove check for zero CQE count
> RDMA/mlx5: Save 4 bytes in CQ structure
> RDMA/mlx5: Provide a modern CQ creation interface
> RDMA/mlx4: Inline mlx4_ib_get_cq_umem into callers
> RDMA/mlx4: Introduce a modern CQ creation interface
> RDMA/mlx4: Remove unused create_flags field from CQ structure
I took 14 patches above, rest will need to be resubmitted.
Thanks
> RDMA/bnxt_re: Convert to modern CQ interface
> RDMA/cxgb4: Separate kernel and user CQ creation paths
> RDMA/mthca: Split user and kernel CQ creation paths
> RDMA/erdma: Separate user and kernel CQ creation paths
> RDMA/ionic: Split user and kernel CQ creation paths
> RDMA/qedr: Convert to modern CQ interface
> RDMA/vmw_pvrdma: Provide a modern CQ creation interface
> RDMA/ocrdma: Split user and kernel CQ creation paths
> RDMA/irdma: Split user and kernel CQ creation paths
> RDMA/usnic: Provide a modern CQ creation interface
> RDMA/mana: Provide a modern CQ creation interface
> RDMA/erdma: Separate user and kernel CQ creation paths
> RDMA/rdmavt: Split user and kernel CQ creation paths
> RDMA/siw: Split user and kernel CQ creation paths
> RDMA/rxe: Split user and kernel CQ creation paths
> RDMA/core: Remove legacy CQ creation fallback path
> RDMA/core: Remove unused ib_resize_cq() implementation
> RDMA: Clarify that CQ resize is a user‑space verb
> RDMA/bnxt_re: Drop support for resizing kernel CQs
> RDMA/irdma: Remove resize support for kernel CQs
> RDMA/mlx4: Remove support for kernel CQ resize
> RDMA/mlx5: Remove support for resizing kernel CQs
> RDMA/mthca: Remove resize support for kernel CQs
> RDMA/rdmavt: Remove resize support for kernel CQs
> RDMA/rxe: Remove unused kernel‑side CQ resize support
> RDMA: Properly propagate the number of CQEs as unsigned int
> RDMA/core: Generalize CQ resize locking
> RDMA/bnxt_re: Complete CQ resize in a single step
> RDMA/bnxt_re: Rely on common resize‑CQ locking
> RDMA/bnxt_re: Reduce CQ memory footprint
> RDMA/mlx4: Use generic resize-CQ lock
> RDMA/mlx4: Use on‑stack variables instead of storing them in the CQ object
> RDMA/mlx5: Use generic resize-CQ lock
> RDMA/mlx5: Select resize‑CQ callback based on device capabilities
> RDMA/mlx5: Reduce CQ memory footprint
> RDMA/mthca: Use generic resize-CQ lock
>
> drivers/infiniband/core/Makefile | 6 +-
> drivers/infiniband/core/cq.c | 3 +
> drivers/infiniband/core/device.c | 4 +-
> drivers/infiniband/core/iter.c | 43 +++
> drivers/infiniband/core/umem.c | 2 +-
> drivers/infiniband/core/uverbs_cmd.c | 18 +-
> drivers/infiniband/core/uverbs_std_types_cq.c | 35 ++-
> drivers/infiniband/core/verbs.c | 61 +---
> drivers/infiniband/hw/bnxt_re/ib_verbs.c | 246 ++++++++-------
> drivers/infiniband/hw/bnxt_re/ib_verbs.h | 9 +-
> drivers/infiniband/hw/bnxt_re/main.c | 3 +-
> drivers/infiniband/hw/bnxt_re/qplib_res.c | 2 +-
> drivers/infiniband/hw/cxgb4/cq.c | 218 +++++++++----
> drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 2 +
> drivers/infiniband/hw/cxgb4/mem.c | 2 +-
> drivers/infiniband/hw/cxgb4/provider.c | 1 +
> drivers/infiniband/hw/efa/efa.h | 6 +-
> drivers/infiniband/hw/efa/efa_main.c | 3 +-
> drivers/infiniband/hw/efa/efa_verbs.c | 44 ++-
> drivers/infiniband/hw/erdma/erdma_main.c | 1 +
> drivers/infiniband/hw/erdma/erdma_verbs.c | 99 ++++--
> drivers/infiniband/hw/erdma/erdma_verbs.h | 2 +
> drivers/infiniband/hw/hns/hns_roce_alloc.c | 2 +-
> drivers/infiniband/hw/hns/hns_roce_cq.c | 103 ++++--
> drivers/infiniband/hw/hns/hns_roce_debugfs.c | 1 -
> drivers/infiniband/hw/hns/hns_roce_device.h | 3 +-
> drivers/infiniband/hw/hns/hns_roce_main.c | 1 +
> drivers/infiniband/hw/ionic/ionic_controlpath.c | 88 ++++--
> drivers/infiniband/hw/ionic/ionic_ibdev.c | 1 +
> drivers/infiniband/hw/ionic/ionic_ibdev.h | 4 +-
> drivers/infiniband/hw/irdma/main.h | 2 +-
> drivers/infiniband/hw/irdma/verbs.c | 402 +++++++++++++-----------
> drivers/infiniband/hw/mana/cq.c | 128 +++++---
> drivers/infiniband/hw/mana/device.c | 1 +
> drivers/infiniband/hw/mana/main.c | 25 +-
> drivers/infiniband/hw/mana/mana_ib.h | 6 +-
> drivers/infiniband/hw/mana/qp.c | 42 ++-
> drivers/infiniband/hw/mana/wq.c | 14 +-
> drivers/infiniband/hw/mlx4/cq.c | 401 ++++++++---------------
> drivers/infiniband/hw/mlx4/main.c | 3 +-
> drivers/infiniband/hw/mlx4/mlx4_ib.h | 10 +-
> drivers/infiniband/hw/mlx4/mr.c | 1 +
> drivers/infiniband/hw/mlx5/cq.c | 383 ++++++++--------------
> drivers/infiniband/hw/mlx5/main.c | 9 +-
> drivers/infiniband/hw/mlx5/mem.c | 1 +
> drivers/infiniband/hw/mlx5/mlx5_ib.h | 12 +-
> drivers/infiniband/hw/mlx5/qp.c | 2 +-
> drivers/infiniband/hw/mlx5/umr.c | 1 +
> drivers/infiniband/hw/mthca/mthca_cq.c | 1 -
> drivers/infiniband/hw/mthca/mthca_provider.c | 193 ++++--------
> drivers/infiniband/hw/mthca/mthca_provider.h | 1 -
> drivers/infiniband/hw/ocrdma/ocrdma_main.c | 3 +-
> drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 70 +++--
> drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 6 +-
> drivers/infiniband/hw/qedr/main.c | 1 +
> drivers/infiniband/hw/qedr/verbs.c | 325 +++++++++++--------
> drivers/infiniband/hw/qedr/verbs.h | 2 +
> drivers/infiniband/hw/usnic/usnic_ib_main.c | 2 +-
> drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 6 +-
> drivers/infiniband/hw/usnic/usnic_ib_verbs.h | 4 +-
> drivers/infiniband/hw/vmw_pvrdma/pvrdma.h | 2 +-
> drivers/infiniband/hw/vmw_pvrdma/pvrdma_cq.c | 171 ++++++----
> drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c | 1 +
> drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h | 3 +
> drivers/infiniband/sw/rdmavt/cq.c | 224 +++++++------
> drivers/infiniband/sw/rdmavt/cq.h | 4 +-
> drivers/infiniband/sw/rdmavt/vt.c | 3 +-
> drivers/infiniband/sw/rxe/rxe_cq.c | 31 --
> drivers/infiniband/sw/rxe/rxe_loc.h | 3 -
> drivers/infiniband/sw/rxe/rxe_verbs.c | 115 +++----
> drivers/infiniband/sw/siw/siw_main.c | 1 +
> drivers/infiniband/sw/siw/siw_verbs.c | 111 +++++--
> drivers/infiniband/sw/siw/siw_verbs.h | 2 +
> include/rdma/ib_umem.h | 36 +--
> include/rdma/ib_verbs.h | 67 +---
> include/rdma/iter.h | 88 ++++++
> 76 files changed, 2085 insertions(+), 1847 deletions(-)
> ---
> base-commit: 42e3aac65c1c9eb36cdee0d8312a326196e0822f
> change-id: 20260203-refactor-umem-e5b4277e41b4
>
> Best regards,
> --
> Leon Romanovsky <leonro@nvidia.com>
>
prev parent reply other threads:[~2026-02-25 13:53 UTC|newest]
Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-02-13 10:57 [PATCH rdma-next 00/50] RDMA: Ensure CQ UMEMs are managed by ib_core Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 01/50] RDMA: Move DMA block iterator logic into dedicated files Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 02/50] RDMA/umem: Allow including ib_umem header from any location Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 03/50] RDMA/umem: Remove unnecessary includes and defines from ib_umem header Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 04/50] RDMA/core: Promote UMEM to a core component Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 05/50] RDMA/core: Manage CQ umem in core code Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 06/50] RDMA/efa: Rely on CPU address in create‑QP Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 07/50] RDMA/core: Prepare create CQ path for API unification Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 08/50] RDMA/core: Reject zero CQE count Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 09/50] RDMA/efa: Remove check for " Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 10/50] RDMA/mlx5: Save 4 bytes in CQ structure Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 11/50] RDMA/mlx5: Provide a modern CQ creation interface Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 12/50] RDMA/mlx4: Inline mlx4_ib_get_cq_umem into callers Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 13/50] RDMA/mlx4: Introduce a modern CQ creation interface Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 14/50] RDMA/mlx4: Remove unused create_flags field from CQ structure Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 15/50] RDMA/bnxt_re: Convert to modern CQ interface Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 16/50] RDMA/cxgb4: Separate kernel and user CQ creation paths Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 17/50] RDMA/mthca: Split user and kernel " Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 18/50] RDMA/erdma: Separate " Leon Romanovsky
2026-02-24 5:51 ` Cheng Xu
2026-02-24 10:57 ` Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 19/50] RDMA/ionic: Split " Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 20/50] RDMA/qedr: Convert to modern CQ interface Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 21/50] RDMA/vmw_pvrdma: Provide a modern CQ creation interface Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 22/50] RDMA/ocrdma: Split user and kernel CQ creation paths Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 23/50] RDMA/irdma: " Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 24/50] RDMA/usnic: Provide a modern CQ creation interface Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 25/50] RDMA/mana: " Leon Romanovsky
2026-02-24 22:30 ` [EXTERNAL] " Long Li
2026-02-25 8:24 ` Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 26/50] RDMA/erdma: Separate user and kernel CQ creation paths Leon Romanovsky
2026-02-24 2:20 ` Cheng Xu
2026-02-24 10:46 ` Leon Romanovsky
2026-02-26 6:17 ` Junxian Huang
2026-02-26 6:54 ` Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 27/50] RDMA/rdmavt: Split " Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 28/50] RDMA/siw: " Leon Romanovsky
2026-02-13 16:56 ` Bernard Metzler
2026-02-13 21:17 ` Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 29/50] RDMA/rxe: " Leon Romanovsky
2026-02-13 23:22 ` yanjun.zhu
2026-02-15 7:06 ` Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 30/50] RDMA/core: Remove legacy CQ creation fallback path Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 31/50] RDMA/core: Remove unused ib_resize_cq() implementation Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 32/50] RDMA: Clarify that CQ resize is a user‑space verb Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 33/50] RDMA/bnxt_re: Drop support for resizing kernel CQs Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 34/50] RDMA/irdma: Remove resize support for " Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 35/50] RDMA/mlx4: Remove support for kernel CQ resize Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 36/50] RDMA/mlx5: Remove support for resizing kernel CQs Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 37/50] RDMA/mthca: Remove resize support for " Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 38/50] RDMA/rdmavt: " Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 39/50] RDMA/rxe: Remove unused kernel‑side CQ resize support Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 40/50] RDMA: Properly propagate the number of CQEs as unsigned int Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 41/50] RDMA/core: Generalize CQ resize locking Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 42/50] RDMA/bnxt_re: Complete CQ resize in a single step Leon Romanovsky
2026-02-16 3:59 ` Selvin Xavier
2026-02-16 8:07 ` Leon Romanovsky
2026-02-17 5:02 ` Selvin Xavier
2026-02-17 7:56 ` Leon Romanovsky
2026-02-17 10:52 ` Selvin Xavier
2026-02-19 8:02 ` Selvin Xavier
2026-02-24 8:15 ` Selvin Xavier
2026-02-24 10:59 ` Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 43/50] RDMA/bnxt_re: Rely on common resize‑CQ locking Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 44/50] RDMA/bnxt_re: Reduce CQ memory footprint Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 45/50] RDMA/mlx4: Use generic resize-CQ lock Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 46/50] RDMA/mlx4: Use on‑stack variables instead of storing them in the CQ object Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 47/50] RDMA/mlx5: Use generic resize-CQ lock Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 48/50] RDMA/mlx5: Select resize‑CQ callback based on device capabilities Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 49/50] RDMA/mlx5: Reduce CQ memory footprint Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 50/50] RDMA/mthca: Use generic resize-CQ lock Leon Romanovsky
2026-02-25 13:51 ` (subset) [PATCH rdma-next 00/50] RDMA: Ensure CQ UMEMs are managed by ib_core Leon Romanovsky
2026-02-25 13:53 ` Leon Romanovsky [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260225135320.GF9541@unreal \
--to=leon@kernel.org \
--cc=abhijit.gangurde@amd.com \
--cc=allen.hubbe@amd.com \
--cc=bcm-kernel-feedback-list@broadcom.com \
--cc=benve@cisco.com \
--cc=bernard.metzler@linux.dev \
--cc=bharat@chelsio.com \
--cc=bryan-bt.tan@broadcom.com \
--cc=chengyou@linux.alibaba.com \
--cc=dennis.dalessandro@cornelisnetworks.com \
--cc=gal.pressman@linux.dev \
--cc=huangjunxian6@hisilicon.com \
--cc=jgg@ziepe.ca \
--cc=kaishen@linux.alibaba.com \
--cc=kalesh-anakkur.purayil@broadcom.com \
--cc=kotaranov@microsoft.com \
--cc=krzysztof.czurylo@intel.com \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=longli@microsoft.com \
--cc=mkalderon@marvell.com \
--cc=mrgolin@amazon.com \
--cc=neescoba@cisco.com \
--cc=selvin.xavier@broadcom.com \
--cc=sleybo@amazon.com \
--cc=tangchengchang@huawei.com \
--cc=tatyana.e.nikolova@intel.com \
--cc=vishnu.dasa@broadcom.com \
--cc=yishaih@nvidia.com \
--cc=zyjzyj2000@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox