From: Jason Gunthorpe <jgg@ziepe.ca>
To: Leon Romanovsky <leon@kernel.org>
Cc: Jiri Pirko <jiri@resnulli.us>,
linux-rdma@vger.kernel.org, mrgolin@amazon.com,
gal.pressman@linux.dev, sleybo@amazon.com, parav@nvidia.com,
mbloch@nvidia.com, yanjun.zhu@linux.dev,
marco.crivellari@suse.com, roman.gushchin@linux.dev,
phaddad@nvidia.com, lirongqing@baidu.com, ynachum@amazon.com,
huangjunxian6@hisilicon.com, kalesh-anakkur.purayil@broadcom.com,
ohartoov@nvidia.com, michaelgur@nvidia.com, shayd@nvidia.com,
edwards@nvidia.com, sriharsha.basavapatna@broadcom.com,
andrew.gospodarek@broadcom.com, selvin.xavier@broadcom.com
Subject: Re: [PATCH rdma-next v2 01/15] RDMA/core: Introduce generic buffer descriptor infrastructure for umem
Date: Sun, 26 Apr 2026 19:50:34 -0300 [thread overview]
Message-ID: <20260426225034.GA3540346@ziepe.ca> (raw)
In-Reply-To: <20260426135340.GH440345@unreal>
On Sun, Apr 26, 2026 at 04:53:40PM +0300, Leon Romanovsky wrote:
> > Well, brainstorming idea. I'd like to hear from Leon too
> >
> > But if we set the general goals as:
> >
> > 1) All umem creations should have a struct ib_uverbs_buffer_desc at
> > the UAPI boundary
> > 2) ib_uverbs_buffer_desc should pass directly to umem code without any
> > driver touching it. ib_uverbs_buffer_desc should be the only way to
> > create a umem from a driver.
> > 3) Existing UWH umem descriptions must continue to work if the desc is
> > not provided, by reforming them into a desc
> > 3) Cleanup and lifecycle should be centralized
>
> I have mixed feelings about this. My CQ conversion showed that even a simple
> task like creating a CQ umem (numb_of_entries * size_of_entries) ends up full
> of creative hacks in various drivers. Because of that, I see real value in
> pushing as much logic as possible into the core code instead of duplicating it
> across drivers. However, my later attempt to change the QP path made it clear
> that creating umems in the core is not a viable goal in the general case.
>
> Another outcome of that work was realizing that CQ resize (and probably MR
> rereg as well) becomes messy when we keep the "old" umem around. Splitting
> creation and cleanup into different layers probably will going to hurt us
> at some point of time.
>
> To summarize:
> 1. The most practical fix is likely to provide a driver callback to create
> the umem when needed, as you suggested.
> 2. We should reduce the use of UWH as much as possible in favor of a
> well-defined schema. In the long run, we want to add more umem types,
> and many drivers should work out of the box under that model.
> 3. Explicit behavior is preferable. If a driver creates something, the
> driver should also clean it up.
>
> I'm not saying no to your proposal, just expressing my thoughts.
So, I think making small steps that upgrade all drivers will be
helpful here.
If we can get all drivers calling the same attrs function and giving
their uhw parameters that is a good step closer to being able to put
that in a function if that is how things need to go down the road.
And it does #2..
Not sure about #3, we already moved toward core destroying umems it
may not be a good idea to try to undo that now.. But maybe we just
keep that for CQ and leave QP as driver managed?
Jason
next prev parent reply other threads:[~2026-04-26 22:50 UTC|newest]
Thread overview: 82+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-04-11 14:49 [PATCH rdma-next v2 00/15] RDMA: Introduce generic buffer descriptor infrastructure for umem Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 01/15] RDMA/core: " Jiri Pirko
2026-04-12 12:33 ` Michael Margolin
2026-04-13 8:32 ` Jiri Pirko
2026-04-13 16:02 ` Michael Margolin
2026-04-13 18:22 ` Jiri Pirko
2026-04-16 12:10 ` Michael Margolin
2026-04-16 13:34 ` Jiri Pirko
2026-04-21 12:50 ` Jason Gunthorpe
2026-04-21 12:52 ` Jason Gunthorpe
2026-04-22 10:32 ` Jiri Pirko
2026-04-22 16:30 ` Jason Gunthorpe
2026-04-21 13:46 ` Jason Gunthorpe
2026-04-22 11:33 ` Jiri Pirko
2026-04-22 14:06 ` Jiri Pirko
2026-04-22 16:51 ` Jason Gunthorpe
2026-04-23 13:08 ` Jiri Pirko
2026-04-23 15:08 ` Jason Gunthorpe
[not found] ` <20260426135340.GH440345@unreal>
2026-04-26 22:50 ` Jason Gunthorpe [this message]
2026-04-11 14:49 ` [PATCH rdma-next v2 02/15] RDMA/uverbs: Push out CQ buffer umem processing into a helper Jiri Pirko
2026-04-21 13:25 ` Jason Gunthorpe
2026-04-22 10:56 ` Jiri Pirko
2026-04-22 16:32 ` Jason Gunthorpe
2026-04-11 14:49 ` [PATCH rdma-next v2 03/15] RDMA/uverbs: Integrate umem_list into CQ creation Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 04/15] RDMA/efa: Use umem_list for user CQ buffer Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 05/15] RDMA/mlx5: " Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 06/15] RDMA/bnxt_re: " Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 07/15] RDMA/mlx4: " Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 08/15] RDMA/uverbs: Remove legacy umem field from struct ib_cq Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 09/15] RDMA/uverbs: Verify all umem_list buffers are consumed after CQ creation Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 10/15] RDMA/uverbs: Integrate umem_list into QP creation Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 11/15] RDMA/mlx5: Use umem_list for QP buffers in create_qp Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 12/15] RDMA/uverbs: Add doorbell record buffer slot to CQ umem_list Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 13/15] RDMA/mlx5: Use umem_list for CQ doorbell record Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 14/15] RDMA/uverbs: Add doorbell record buffer slot to QP umem_list Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 15/15] RDMA/mlx5: Use umem_list for QP doorbell record Jiri Pirko
-- strict thread matches above, loose matches on Subject: below --
2026-04-25 6:04 [PATCH v2 0/2] RDMA/rxe: Fix per-netns UDP tunnel issues Kuniyuki Iwashima
2026-04-25 6:04 ` [PATCH v2 1/2] RDMA/rxe: Fix null-ptr-deref in kernel_sock_shutdown() Kuniyuki Iwashima
2026-04-25 15:47 ` David Ahern
2026-04-25 20:55 ` Kuniyuki Iwashima
2026-04-26 16:40 ` David Ahern
2026-04-25 21:25 ` Zhu Yanjun
2026-04-26 16:42 ` David Ahern
2026-04-25 6:04 ` [PATCH v2 2/2] RDMA/rxe: Fix up RCU usage for rxe_ns_pernet_sk6() Kuniyuki Iwashima
2026-04-25 21:26 ` Zhu Yanjun
2026-03-31 5:56 [PATCH v2 0/4] Firmware LSM hook Leon Romanovsky
2026-03-31 5:56 ` [PATCH v2 1/4] bpf: add firmware command validation hook Leon Romanovsky
2026-04-16 8:43 ` Matt Bobrowski
2026-03-31 5:56 ` [PATCH v2 2/4] selftests/bpf: add test cases for fw_validate_cmd hook Leon Romanovsky
2026-03-31 5:56 ` [PATCH v2 3/4] RDMA/mlx5: Externally validate FW commands supplied in DEVX interface Leon Romanovsky
2026-03-31 5:56 ` [PATCH v2 4/4] fwctl/mlx5: Externally validate FW commands supplied in fwctl Leon Romanovsky
2026-04-09 12:12 ` [PATCH v2 0/4] Firmware LSM hook Leon Romanovsky
2026-04-09 12:27 ` Roberto Sassu
2026-04-09 12:45 ` Leon Romanovsky
2026-04-09 21:04 ` Paul Moore
2026-04-12 9:00 ` Leon Romanovsky
2026-04-13 1:38 ` Paul Moore
2026-04-13 15:53 ` Leon Romanovsky
2026-04-13 16:42 ` Jason Gunthorpe
2026-04-13 17:36 ` Casey Schaufler
2026-04-13 19:09 ` Casey Schaufler
2026-04-13 22:36 ` Paul Moore
2026-04-13 23:19 ` Jason Gunthorpe
2026-04-14 17:05 ` Casey Schaufler
2026-04-14 19:09 ` Paul Moore
2026-04-14 20:09 ` Casey Schaufler
2026-04-14 20:44 ` Paul Moore
2026-04-14 22:42 ` Casey Schaufler
2026-04-15 21:03 ` Paul Moore
2026-04-15 21:21 ` Casey Schaufler
2026-04-14 20:27 ` Paul Moore
2026-04-15 13:47 ` Jason Gunthorpe
2026-04-15 21:40 ` Paul Moore
2026-04-17 19:17 ` Jason Gunthorpe
2026-04-21 0:58 ` Paul Moore
2026-04-24 14:36 ` Jason Gunthorpe
2026-04-24 20:59 ` Paul Moore
2026-04-24 22:13 ` Jason Gunthorpe
2026-04-23 14:09 ` Leon Romanovsky
2026-04-24 14:19 ` Jason Gunthorpe
2026-04-26 10:39 ` Leon Romanovsky
2026-04-23 13:05 ` Leon Romanovsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260426225034.GA3540346@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=andrew.gospodarek@broadcom.com \
--cc=edwards@nvidia.com \
--cc=gal.pressman@linux.dev \
--cc=huangjunxian6@hisilicon.com \
--cc=jiri@resnulli.us \
--cc=kalesh-anakkur.purayil@broadcom.com \
--cc=leon@kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=lirongqing@baidu.com \
--cc=marco.crivellari@suse.com \
--cc=mbloch@nvidia.com \
--cc=michaelgur@nvidia.com \
--cc=mrgolin@amazon.com \
--cc=ohartoov@nvidia.com \
--cc=parav@nvidia.com \
--cc=phaddad@nvidia.com \
--cc=roman.gushchin@linux.dev \
--cc=selvin.xavier@broadcom.com \
--cc=shayd@nvidia.com \
--cc=sleybo@amazon.com \
--cc=sriharsha.basavapatna@broadcom.com \
--cc=yanjun.zhu@linux.dev \
--cc=ynachum@amazon.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox