public inbox for linux-rdma@vger.kernel.org
 help / color / mirror / Atom feed
From: "Yanjun.Zhu" <yanjun.zhu@linux.dev>
To: Jiri Pirko <jiri@resnulli.us>, linux-rdma@vger.kernel.org
Cc: jgg@ziepe.ca, leon@kernel.org, mrgolin@amazon.com,
	gal.pressman@linux.dev, sleybo@amazon.com, parav@nvidia.com,
	mbloch@nvidia.com, wangliang74@huawei.com,
	marco.crivellari@suse.com, roman.gushchin@linux.dev,
	phaddad@nvidia.com, lirongqing@baidu.com, ynachum@amazon.com,
	huangjunxian6@hisilicon.com, kalesh-anakkur.purayil@broadcom.com,
	ohartoov@nvidia.com, michaelgur@nvidia.com, shayd@nvidia.com,
	edwards@nvidia.com, sriharsha.basavapatna@broadcom.com,
	andrew.gospodarek@broadcom.com, selvin.xavier@broadcom.com
Subject: Re: [PATCH rdma-next 00/15] RDMA: Introduce generic buffer descriptor infrastructure for umem
Date: Fri, 27 Mar 2026 16:03:47 -0700	[thread overview]
Message-ID: <4280105c-13aa-4a02-8fd5-ea68b8936b67@linux.dev> (raw)
In-Reply-To: <20260325150048.168341-1-jiri@resnulli.us>


On 3/25/26 8:00 AM, Jiri Pirko wrote:
> From: Jiri Pirko <jiri@nvidia.com>
>
> This patchset introduces a generic buffer descriptor infrastructure
> for passing memory buffers (dma-buf or user VA) to uverbs commands,
> and wires it up for CQ and QP creation in the uverbs core, efa, mlx5,
> bnxt_re and mlx4 drivers.

While the current patch set successfully introduces a generic buffer 
descriptor

infrastructure for CQ and QP creation, it raises the question of why

Memory Region (MR) allocation and registration have not been integrated 
into

this new ib_umem_list architecture.

Given that MRs often require complex memory backing—such as discrete 
dma-buf segments

from GPUs or fragmented user-space virtual addresses—extending the 
UVERBS_ATTR_BUFFERS

array model to MRs would seem like a natural evolution. This would 
provide a unified

UAPI for handling heterogeneous memory sources and eliminate the need 
for per-command

attributes when registering composite memory regions. Are there specific 
architectural

constraints or synchronization concerns that necessitated keeping MR 
registration on its legacy

path for now?

In short, I am wondering **whether this architecture can include MR 
(memory region) or not**.

As such, CQ/QP/MR can use the same architecture.

Zhu Yanjun

> Instead of adding per-command UAPI attributes for each new buffer
> type, a single UVERBS_ATTR_BUFFERS array attribute carries all buffer
> descriptors. Each descriptor specifies a buffer type and is indexed by
> per-command slot enums. A consumption check ensures userspace and
> driver agree on which buffers are used.
> The patchset:
> 1. Introduces the core ib_umem_list infrastructure and UAPI.
> 2. Factors out CQ buffer umem processing into a helper.
> 3. Integrates umem_list into CQ creation, with fallback to existing
>     per-attribute path.
> 4-7. Converts efa, mlx5, bnxt_re and mlx4 to use umem_list for CQ
>     buffer.
> 8. Removes the legacy umem field from struct ib_cq, now that all
>     drivers use umem_list for CQ buffer management.
> 9. Adds a consumption check verifying all umem_list buffers were
>     consumed by the driver after CQ creation.
> 10. Integrates umem_list into QP creation.
> 11. Converts mlx5 QP creation to use umem_list.
> 12-15. Extends CQ and QP with doorbell record buffer slots and
>     converts mlx5 to use them.
>
> Note this re-works the original patchset trying to handle this:
> https://lore.kernel.org/all/20260203085003.71184-1-jiri@resnulli.us/
> The code is so much different I'm sending this is a new patchset.
>
> Jiri Pirko (15):
>    RDMA/core: Introduce generic buffer descriptor infrastructure for umem
>    RDMA/uverbs: Push out CQ buffer umem processing into a helper
>    RDMA/uverbs: Integrate umem_list into CQ creation
>    RDMA/efa: Use umem_list for user CQ buffer
>    RDMA/mlx5: Use umem_list for user CQ buffer
>    RDMA/bnxt_re: Use umem_list for user CQ buffer
>    RDMA/mlx4: Use umem_list for user CQ buffer
>    RDMA/uverbs: Remove legacy umem field from struct ib_cq
>    RDMA/uverbs: Verify all umem_list buffers are consumed after CQ
>      creation
>    RDMA/uverbs: Integrate umem_list into QP creation
>    RDMA/mlx5: Use umem_list for QP buffers in create_qp
>    RDMA/uverbs: Add doorbell record buffer slot to CQ umem_list
>    RDMA/mlx5: Use umem_list for CQ doorbell record
>    RDMA/uverbs: Add doorbell record buffer slot to QP umem_list
>    RDMA/mlx5: Use umem_list for QP doorbell record
>
>   drivers/infiniband/core/core_priv.h           |   1 +
>   drivers/infiniband/core/umem.c                | 248 ++++++++++++++++++
>   drivers/infiniband/core/uverbs_cmd.c          |  18 +-
>   drivers/infiniband/core/uverbs_std_types_cq.c | 158 ++++++-----
>   drivers/infiniband/core/uverbs_std_types_qp.c |  26 +-
>   drivers/infiniband/core/verbs.c               |  26 +-
>   drivers/infiniband/hw/bnxt_re/ib_verbs.c      |  23 +-
>   drivers/infiniband/hw/efa/efa_verbs.c         |  17 +-
>   drivers/infiniband/hw/mlx4/cq.c               |  21 +-
>   drivers/infiniband/hw/mlx5/cq.c               |  40 ++-
>   drivers/infiniband/hw/mlx5/doorbell.c         |  41 ++-
>   drivers/infiniband/hw/mlx5/mlx5_ib.h          |   3 +-
>   drivers/infiniband/hw/mlx5/qp.c               |  76 ++++--
>   drivers/infiniband/hw/mlx5/srq.c              |   2 +-
>   include/rdma/ib_umem.h                        |  54 ++++
>   include/rdma/ib_verbs.h                       |   5 +-
>   include/rdma/uverbs_ioctl.h                   |  14 +
>   include/uapi/rdma/ib_user_ioctl_cmds.h        |  17 ++
>   include/uapi/rdma/ib_user_ioctl_verbs.h       |  27 ++
>   19 files changed, 651 insertions(+), 166 deletions(-)
>

  parent reply	other threads:[~2026-03-27 23:03 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-03-25 15:00 [PATCH rdma-next 00/15] RDMA: Introduce generic buffer descriptor infrastructure for umem Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 01/15] RDMA/core: " Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 02/15] RDMA/uverbs: Push out CQ buffer umem processing into a helper Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 03/15] RDMA/uverbs: Integrate umem_list into CQ creation Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 04/15] RDMA/efa: Use umem_list for user CQ buffer Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 05/15] RDMA/mlx5: " Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 06/15] RDMA/bnxt_re: " Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 07/15] RDMA/mlx4: " Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 08/15] RDMA/uverbs: Remove legacy umem field from struct ib_cq Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 09/15] RDMA/uverbs: Verify all umem_list buffers are consumed after CQ creation Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 10/15] RDMA/uverbs: Integrate umem_list into QP creation Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 11/15] RDMA/mlx5: Use umem_list for QP buffers in create_qp Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 12/15] RDMA/uverbs: Add doorbell record buffer slot to CQ umem_list Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 13/15] RDMA/mlx5: Use umem_list for CQ doorbell record Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 14/15] RDMA/uverbs: Add doorbell record buffer slot to QP umem_list Jiri Pirko
2026-03-25 15:00 ` [PATCH rdma-next 15/15] RDMA/mlx5: Use umem_list for QP doorbell record Jiri Pirko
2026-03-27 23:03 ` Yanjun.Zhu [this message]
2026-03-29 11:26   ` [PATCH rdma-next 00/15] RDMA: Introduce generic buffer descriptor infrastructure for umem Jiri Pirko
2026-03-29 19:08     ` Zhu Yanjun

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4280105c-13aa-4a02-8fd5-ea68b8936b67@linux.dev \
    --to=yanjun.zhu@linux.dev \
    --cc=andrew.gospodarek@broadcom.com \
    --cc=edwards@nvidia.com \
    --cc=gal.pressman@linux.dev \
    --cc=huangjunxian6@hisilicon.com \
    --cc=jgg@ziepe.ca \
    --cc=jiri@resnulli.us \
    --cc=kalesh-anakkur.purayil@broadcom.com \
    --cc=leon@kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=lirongqing@baidu.com \
    --cc=marco.crivellari@suse.com \
    --cc=mbloch@nvidia.com \
    --cc=michaelgur@nvidia.com \
    --cc=mrgolin@amazon.com \
    --cc=ohartoov@nvidia.com \
    --cc=parav@nvidia.com \
    --cc=phaddad@nvidia.com \
    --cc=roman.gushchin@linux.dev \
    --cc=selvin.xavier@broadcom.com \
    --cc=shayd@nvidia.com \
    --cc=sleybo@amazon.com \
    --cc=sriharsha.basavapatna@broadcom.com \
    --cc=wangliang74@huawei.com \
    --cc=ynachum@amazon.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox