From: Leon Romanovsky <leon@kernel.org>
To: Jason Gunthorpe <jgg@ziepe.ca>,
Selvin Xavier <selvin.xavier@broadcom.com>,
Kalesh AP <kalesh-anakkur.purayil@broadcom.com>,
Potnuri Bharat Teja <bharat@chelsio.com>,
Michael Margolin <mrgolin@amazon.com>,
Gal Pressman <gal.pressman@linux.dev>,
Yossi Leybovich <sleybo@amazon.com>,
Cheng Xu <chengyou@linux.alibaba.com>,
Kai Shen <kaishen@linux.alibaba.com>,
Chengchang Tang <tangchengchang@huawei.com>,
Junxian Huang <huangjunxian6@hisilicon.com>,
Abhijit Gangurde <abhijit.gangurde@amd.com>,
Allen Hubbe <allen.hubbe@amd.com>,
Krzysztof Czurylo <krzysztof.czurylo@intel.com>,
Tatyana Nikolova <tatyana.e.nikolova@intel.com>,
Long Li <longli@microsoft.com>,
Konstantin Taranov <kotaranov@microsoft.com>,
Yishai Hadas <yishaih@nvidia.com>,
Michal Kalderon <mkalderon@marvell.com>,
Bryan Tan <bryan-bt.tan@broadcom.com>,
Vishnu Dasa <vishnu.dasa@broadcom.com>,
Broadcom internal kernel review list
<bcm-kernel-feedback-list@broadcom.com>,
Christian Benvenuti <benve@cisco.com>,
Nelson Escobar <neescoba@cisco.com>,
Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>,
Bernard Metzler <bernard.metzler@linux.dev>,
Zhu Yanjun <zyjzyj2000@gmail.com>
Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
linux-hyperv@vger.kernel.org
Subject: Re: [PATCH rdma-next 00/50] RDMA: Ensure CQ UMEMs are managed by ib_core
Date: Wed, 25 Feb 2026 15:53:20 +0200 [thread overview]
Message-ID: <20260225135320.GF9541@unreal> (raw)
In-Reply-To: <20260213-refactor-umem-v1-0-f3be85847922@nvidia.com>
On Fri, Feb 13, 2026 at 12:57:36PM +0200, Leon Romanovsky wrote:
> Unify CQ UMEM creation, resize and release in ib_core to avoid the need
> for complex driver-side handling. This lets us rely on the internal
> reference counters of the relevant ib_XXX objects to manage UMEM
> lifetime safely and consistently.
>
> The resize cleanup made it clear that most drivers never handled this
> path correctly, and there's a good chance the functionality was never
> actually used. The most common issue was relying on the cq->resize_umem
> pointer to detect races with other CQ commands, without clearing it on
> errors and while ignoring proper locking for other CQ operations.
>
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> ---
> Leon Romanovsky (50):
> RDMA: Move DMA block iterator logic into dedicated files
> RDMA/umem: Allow including ib_umem header from any location
> RDMA/umem: Remove unnecessary includes and defines from ib_umem header
> RDMA/core: Promote UMEM to a core component
> RDMA/core: Manage CQ umem in core code
> RDMA/efa: Rely on CPU address in create‑QP
> RDMA/core: Prepare create CQ path for API unification
> RDMA/core: Reject zero CQE count
> RDMA/efa: Remove check for zero CQE count
> RDMA/mlx5: Save 4 bytes in CQ structure
> RDMA/mlx5: Provide a modern CQ creation interface
> RDMA/mlx4: Inline mlx4_ib_get_cq_umem into callers
> RDMA/mlx4: Introduce a modern CQ creation interface
> RDMA/mlx4: Remove unused create_flags field from CQ structure
I took 14 patches above, rest will need to be resubmitted.
Thanks
> RDMA/bnxt_re: Convert to modern CQ interface
> RDMA/cxgb4: Separate kernel and user CQ creation paths
> RDMA/mthca: Split user and kernel CQ creation paths
> RDMA/erdma: Separate user and kernel CQ creation paths
> RDMA/ionic: Split user and kernel CQ creation paths
> RDMA/qedr: Convert to modern CQ interface
> RDMA/vmw_pvrdma: Provide a modern CQ creation interface
> RDMA/ocrdma: Split user and kernel CQ creation paths
> RDMA/irdma: Split user and kernel CQ creation paths
> RDMA/usnic: Provide a modern CQ creation interface
> RDMA/mana: Provide a modern CQ creation interface
> RDMA/erdma: Separate user and kernel CQ creation paths
> RDMA/rdmavt: Split user and kernel CQ creation paths
> RDMA/siw: Split user and kernel CQ creation paths
> RDMA/rxe: Split user and kernel CQ creation paths
> RDMA/core: Remove legacy CQ creation fallback path
> RDMA/core: Remove unused ib_resize_cq() implementation
> RDMA: Clarify that CQ resize is a user‑space verb
> RDMA/bnxt_re: Drop support for resizing kernel CQs
> RDMA/irdma: Remove resize support for kernel CQs
> RDMA/mlx4: Remove support for kernel CQ resize
> RDMA/mlx5: Remove support for resizing kernel CQs
> RDMA/mthca: Remove resize support for kernel CQs
> RDMA/rdmavt: Remove resize support for kernel CQs
> RDMA/rxe: Remove unused kernel‑side CQ resize support
> RDMA: Properly propagate the number of CQEs as unsigned int
> RDMA/core: Generalize CQ resize locking
> RDMA/bnxt_re: Complete CQ resize in a single step
> RDMA/bnxt_re: Rely on common resize‑CQ locking
> RDMA/bnxt_re: Reduce CQ memory footprint
> RDMA/mlx4: Use generic resize-CQ lock
> RDMA/mlx4: Use on‑stack variables instead of storing them in the CQ object
> RDMA/mlx5: Use generic resize-CQ lock
> RDMA/mlx5: Select resize‑CQ callback based on device capabilities
> RDMA/mlx5: Reduce CQ memory footprint
> RDMA/mthca: Use generic resize-CQ lock
>
> drivers/infiniband/core/Makefile | 6 +-
> drivers/infiniband/core/cq.c | 3 +
> drivers/infiniband/core/device.c | 4 +-
> drivers/infiniband/core/iter.c | 43 +++
> drivers/infiniband/core/umem.c | 2 +-
> drivers/infiniband/core/uverbs_cmd.c | 18 +-
> drivers/infiniband/core/uverbs_std_types_cq.c | 35 ++-
> drivers/infiniband/core/verbs.c | 61 +---
> drivers/infiniband/hw/bnxt_re/ib_verbs.c | 246 ++++++++-------
> drivers/infiniband/hw/bnxt_re/ib_verbs.h | 9 +-
> drivers/infiniband/hw/bnxt_re/main.c | 3 +-
> drivers/infiniband/hw/bnxt_re/qplib_res.c | 2 +-
> drivers/infiniband/hw/cxgb4/cq.c | 218 +++++++++----
> drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 2 +
> drivers/infiniband/hw/cxgb4/mem.c | 2 +-
> drivers/infiniband/hw/cxgb4/provider.c | 1 +
> drivers/infiniband/hw/efa/efa.h | 6 +-
> drivers/infiniband/hw/efa/efa_main.c | 3 +-
> drivers/infiniband/hw/efa/efa_verbs.c | 44 ++-
> drivers/infiniband/hw/erdma/erdma_main.c | 1 +
> drivers/infiniband/hw/erdma/erdma_verbs.c | 99 ++++--
> drivers/infiniband/hw/erdma/erdma_verbs.h | 2 +
> drivers/infiniband/hw/hns/hns_roce_alloc.c | 2 +-
> drivers/infiniband/hw/hns/hns_roce_cq.c | 103 ++++--
> drivers/infiniband/hw/hns/hns_roce_debugfs.c | 1 -
> drivers/infiniband/hw/hns/hns_roce_device.h | 3 +-
> drivers/infiniband/hw/hns/hns_roce_main.c | 1 +
> drivers/infiniband/hw/ionic/ionic_controlpath.c | 88 ++++--
> drivers/infiniband/hw/ionic/ionic_ibdev.c | 1 +
> drivers/infiniband/hw/ionic/ionic_ibdev.h | 4 +-
> drivers/infiniband/hw/irdma/main.h | 2 +-
> drivers/infiniband/hw/irdma/verbs.c | 402 +++++++++++++-----------
> drivers/infiniband/hw/mana/cq.c | 128 +++++---
> drivers/infiniband/hw/mana/device.c | 1 +
> drivers/infiniband/hw/mana/main.c | 25 +-
> drivers/infiniband/hw/mana/mana_ib.h | 6 +-
> drivers/infiniband/hw/mana/qp.c | 42 ++-
> drivers/infiniband/hw/mana/wq.c | 14 +-
> drivers/infiniband/hw/mlx4/cq.c | 401 ++++++++---------------
> drivers/infiniband/hw/mlx4/main.c | 3 +-
> drivers/infiniband/hw/mlx4/mlx4_ib.h | 10 +-
> drivers/infiniband/hw/mlx4/mr.c | 1 +
> drivers/infiniband/hw/mlx5/cq.c | 383 ++++++++--------------
> drivers/infiniband/hw/mlx5/main.c | 9 +-
> drivers/infiniband/hw/mlx5/mem.c | 1 +
> drivers/infiniband/hw/mlx5/mlx5_ib.h | 12 +-
> drivers/infiniband/hw/mlx5/qp.c | 2 +-
> drivers/infiniband/hw/mlx5/umr.c | 1 +
> drivers/infiniband/hw/mthca/mthca_cq.c | 1 -
> drivers/infiniband/hw/mthca/mthca_provider.c | 193 ++++--------
> drivers/infiniband/hw/mthca/mthca_provider.h | 1 -
> drivers/infiniband/hw/ocrdma/ocrdma_main.c | 3 +-
> drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 70 +++--
> drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 6 +-
> drivers/infiniband/hw/qedr/main.c | 1 +
> drivers/infiniband/hw/qedr/verbs.c | 325 +++++++++++--------
> drivers/infiniband/hw/qedr/verbs.h | 2 +
> drivers/infiniband/hw/usnic/usnic_ib_main.c | 2 +-
> drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 6 +-
> drivers/infiniband/hw/usnic/usnic_ib_verbs.h | 4 +-
> drivers/infiniband/hw/vmw_pvrdma/pvrdma.h | 2 +-
> drivers/infiniband/hw/vmw_pvrdma/pvrdma_cq.c | 171 ++++++----
> drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c | 1 +
> drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h | 3 +
> drivers/infiniband/sw/rdmavt/cq.c | 224 +++++++------
> drivers/infiniband/sw/rdmavt/cq.h | 4 +-
> drivers/infiniband/sw/rdmavt/vt.c | 3 +-
> drivers/infiniband/sw/rxe/rxe_cq.c | 31 --
> drivers/infiniband/sw/rxe/rxe_loc.h | 3 -
> drivers/infiniband/sw/rxe/rxe_verbs.c | 115 +++----
> drivers/infiniband/sw/siw/siw_main.c | 1 +
> drivers/infiniband/sw/siw/siw_verbs.c | 111 +++++--
> drivers/infiniband/sw/siw/siw_verbs.h | 2 +
> include/rdma/ib_umem.h | 36 +--
> include/rdma/ib_verbs.h | 67 +---
> include/rdma/iter.h | 88 ++++++
> 76 files changed, 2085 insertions(+), 1847 deletions(-)
> ---
> base-commit: 42e3aac65c1c9eb36cdee0d8312a326196e0822f
> change-id: 20260203-refactor-umem-e5b4277e41b4
>
> Best regards,
> --
> Leon Romanovsky <leonro@nvidia.com>
>
prev parent reply other threads:[~2026-02-25 13:53 UTC|newest]
Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-02-13 10:57 [PATCH rdma-next 00/50] RDMA: Ensure CQ UMEMs are managed by ib_core Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 01/50] RDMA: Move DMA block iterator logic into dedicated files Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 02/50] RDMA/umem: Allow including ib_umem header from any location Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 03/50] RDMA/umem: Remove unnecessary includes and defines from ib_umem header Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 04/50] RDMA/core: Promote UMEM to a core component Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 05/50] RDMA/core: Manage CQ umem in core code Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 06/50] RDMA/efa: Rely on CPU address in create‑QP Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 07/50] RDMA/core: Prepare create CQ path for API unification Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 08/50] RDMA/core: Reject zero CQE count Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 09/50] RDMA/efa: Remove check for " Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 10/50] RDMA/mlx5: Save 4 bytes in CQ structure Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 11/50] RDMA/mlx5: Provide a modern CQ creation interface Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 12/50] RDMA/mlx4: Inline mlx4_ib_get_cq_umem into callers Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 13/50] RDMA/mlx4: Introduce a modern CQ creation interface Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 14/50] RDMA/mlx4: Remove unused create_flags field from CQ structure Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 15/50] RDMA/bnxt_re: Convert to modern CQ interface Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 16/50] RDMA/cxgb4: Separate kernel and user CQ creation paths Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 17/50] RDMA/mthca: Split user and kernel " Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 18/50] RDMA/erdma: Separate " Leon Romanovsky
2026-02-24 5:51 ` Cheng Xu
2026-02-24 10:57 ` Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 19/50] RDMA/ionic: Split " Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 20/50] RDMA/qedr: Convert to modern CQ interface Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 21/50] RDMA/vmw_pvrdma: Provide a modern CQ creation interface Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 22/50] RDMA/ocrdma: Split user and kernel CQ creation paths Leon Romanovsky
2026-02-13 10:57 ` [PATCH rdma-next 23/50] RDMA/irdma: " Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 24/50] RDMA/usnic: Provide a modern CQ creation interface Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 25/50] RDMA/mana: " Leon Romanovsky
2026-02-24 22:30 ` [EXTERNAL] " Long Li
2026-02-25 8:24 ` Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 26/50] RDMA/erdma: Separate user and kernel CQ creation paths Leon Romanovsky
2026-02-24 2:20 ` Cheng Xu
2026-02-24 10:46 ` Leon Romanovsky
2026-02-26 6:17 ` Junxian Huang
2026-02-26 6:54 ` Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 27/50] RDMA/rdmavt: Split " Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 28/50] RDMA/siw: " Leon Romanovsky
2026-02-13 16:56 ` Bernard Metzler
2026-02-13 21:17 ` Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 29/50] RDMA/rxe: " Leon Romanovsky
2026-02-13 23:22 ` yanjun.zhu
2026-02-15 7:06 ` Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 30/50] RDMA/core: Remove legacy CQ creation fallback path Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 31/50] RDMA/core: Remove unused ib_resize_cq() implementation Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 32/50] RDMA: Clarify that CQ resize is a user‑space verb Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 33/50] RDMA/bnxt_re: Drop support for resizing kernel CQs Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 34/50] RDMA/irdma: Remove resize support for " Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 35/50] RDMA/mlx4: Remove support for kernel CQ resize Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 36/50] RDMA/mlx5: Remove support for resizing kernel CQs Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 37/50] RDMA/mthca: Remove resize support for " Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 38/50] RDMA/rdmavt: " Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 39/50] RDMA/rxe: Remove unused kernel‑side CQ resize support Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 40/50] RDMA: Properly propagate the number of CQEs as unsigned int Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 41/50] RDMA/core: Generalize CQ resize locking Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 42/50] RDMA/bnxt_re: Complete CQ resize in a single step Leon Romanovsky
2026-02-16 3:59 ` Selvin Xavier
2026-02-16 8:07 ` Leon Romanovsky
2026-02-17 5:02 ` Selvin Xavier
2026-02-17 7:56 ` Leon Romanovsky
2026-02-17 10:52 ` Selvin Xavier
2026-02-19 8:02 ` Selvin Xavier
2026-02-24 8:15 ` Selvin Xavier
2026-02-24 10:59 ` Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 43/50] RDMA/bnxt_re: Rely on common resize‑CQ locking Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 44/50] RDMA/bnxt_re: Reduce CQ memory footprint Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 45/50] RDMA/mlx4: Use generic resize-CQ lock Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 46/50] RDMA/mlx4: Use on‑stack variables instead of storing them in the CQ object Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 47/50] RDMA/mlx5: Use generic resize-CQ lock Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 48/50] RDMA/mlx5: Select resize‑CQ callback based on device capabilities Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 49/50] RDMA/mlx5: Reduce CQ memory footprint Leon Romanovsky
2026-02-13 10:58 ` [PATCH rdma-next 50/50] RDMA/mthca: Use generic resize-CQ lock Leon Romanovsky
2026-02-25 13:51 ` (subset) [PATCH rdma-next 00/50] RDMA: Ensure CQ UMEMs are managed by ib_core Leon Romanovsky
2026-02-25 13:53 ` Leon Romanovsky [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260225135320.GF9541@unreal \
--to=leon@kernel.org \
--cc=abhijit.gangurde@amd.com \
--cc=allen.hubbe@amd.com \
--cc=bcm-kernel-feedback-list@broadcom.com \
--cc=benve@cisco.com \
--cc=bernard.metzler@linux.dev \
--cc=bharat@chelsio.com \
--cc=bryan-bt.tan@broadcom.com \
--cc=chengyou@linux.alibaba.com \
--cc=dennis.dalessandro@cornelisnetworks.com \
--cc=gal.pressman@linux.dev \
--cc=huangjunxian6@hisilicon.com \
--cc=jgg@ziepe.ca \
--cc=kaishen@linux.alibaba.com \
--cc=kalesh-anakkur.purayil@broadcom.com \
--cc=kotaranov@microsoft.com \
--cc=krzysztof.czurylo@intel.com \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=longli@microsoft.com \
--cc=mkalderon@marvell.com \
--cc=mrgolin@amazon.com \
--cc=neescoba@cisco.com \
--cc=selvin.xavier@broadcom.com \
--cc=sleybo@amazon.com \
--cc=tangchengchang@huawei.com \
--cc=tatyana.e.nikolova@intel.com \
--cc=vishnu.dasa@broadcom.com \
--cc=yishaih@nvidia.com \
--cc=zyjzyj2000@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.