public inbox for linux-rdma@vger.kernel.org
 help / color / mirror / Atom feed
From: Jason Gunthorpe <jgg@ziepe.ca>
To: Leon Romanovsky <leon@kernel.org>
Cc: Jiri Pirko <jiri@resnulli.us>,
	linux-rdma@vger.kernel.org, mrgolin@amazon.com,
	gal.pressman@linux.dev, sleybo@amazon.com, parav@nvidia.com,
	mbloch@nvidia.com, yanjun.zhu@linux.dev,
	marco.crivellari@suse.com, roman.gushchin@linux.dev,
	phaddad@nvidia.com, lirongqing@baidu.com, ynachum@amazon.com,
	huangjunxian6@hisilicon.com, kalesh-anakkur.purayil@broadcom.com,
	ohartoov@nvidia.com, michaelgur@nvidia.com, shayd@nvidia.com,
	edwards@nvidia.com, sriharsha.basavapatna@broadcom.com,
	andrew.gospodarek@broadcom.com, selvin.xavier@broadcom.com
Subject: Re: [PATCH rdma-next v2 01/15] RDMA/core: Introduce generic buffer descriptor infrastructure for umem
Date: Sun, 26 Apr 2026 19:50:34 -0300	[thread overview]
Message-ID: <20260426225034.GA3540346@ziepe.ca> (raw)
In-Reply-To: <20260426135340.GH440345@unreal>

On Sun, Apr 26, 2026 at 04:53:40PM +0300, Leon Romanovsky wrote:
> > Well, brainstorming idea. I'd like to hear from Leon too
> > 
> > But if we set the general goals as:
> > 
> > 1) All umem creations should have a struct ib_uverbs_buffer_desc at
> >    the UAPI boundary
> > 2) ib_uverbs_buffer_desc should pass directly to umem code without any
> >    driver touching it. ib_uverbs_buffer_desc should be the only way to
> >    create a umem from a driver.
> > 3) Existing UWH umem descriptions must continue to work if the desc is
> >    not provided, by reforming them into a desc
> > 3) Cleanup and lifecycle should be centralized
> 
> I have mixed feelings about this. My CQ conversion showed that even a simple
> task like creating a CQ umem (numb_of_entries * size_of_entries) ends up full
> of creative hacks in various drivers. Because of that, I see real value in
> pushing as much logic as possible into the core code instead of duplicating it
> across drivers. However, my later attempt to change the QP path made it clear
> that creating umems in the core is not a viable goal in the general case.
> 
> Another outcome of that work was realizing that CQ resize (and probably MR
> rereg as well) becomes messy when we keep the "old" umem around. Splitting
> creation and cleanup into different layers probably will going to hurt us
> at some point of time.
> 
> To summarize:
> 1. The most practical fix is likely to provide a driver callback to create
>    the umem when needed, as you suggested.
> 2. We should reduce the use of UWH as much as possible in favor of a
>    well-defined schema. In the long run, we want to add more umem types,
>    and many drivers should work out of the box under that model.
> 3. Explicit behavior is preferable. If a driver creates something, the
>    driver should also clean it up.
> 
> I'm not saying no to your proposal, just expressing my thoughts.

So, I think making small steps that upgrade all drivers will be
helpful here.

If we can get all drivers calling the same attrs function and giving
their uhw parameters that is a good step closer to being able to put
that in a function if that is how things need to go down the road.

And it does #2..

Not sure about #3, we already moved toward core destroying umems it
may not be a good idea to try to undo that now.. But maybe we just
keep that for CQ and leave QP as driver managed?

Jason

  parent reply	other threads:[~2026-04-26 22:50 UTC|newest]

Thread overview: 82+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-04-11 14:49 [PATCH rdma-next v2 00/15] RDMA: Introduce generic buffer descriptor infrastructure for umem Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 01/15] RDMA/core: " Jiri Pirko
2026-04-12 12:33   ` Michael Margolin
2026-04-13  8:32     ` Jiri Pirko
2026-04-13 16:02       ` Michael Margolin
2026-04-13 18:22         ` Jiri Pirko
2026-04-16 12:10           ` Michael Margolin
2026-04-16 13:34             ` Jiri Pirko
2026-04-21 12:50               ` Jason Gunthorpe
2026-04-21 12:52             ` Jason Gunthorpe
2026-04-22 10:32               ` Jiri Pirko
2026-04-22 16:30                 ` Jason Gunthorpe
2026-04-21 13:46   ` Jason Gunthorpe
2026-04-22 11:33     ` Jiri Pirko
2026-04-22 14:06       ` Jiri Pirko
2026-04-22 16:51         ` Jason Gunthorpe
2026-04-23 13:08           ` Jiri Pirko
2026-04-23 15:08             ` Jason Gunthorpe
     [not found]           ` <20260426135340.GH440345@unreal>
2026-04-26 22:50             ` Jason Gunthorpe [this message]
2026-04-11 14:49 ` [PATCH rdma-next v2 02/15] RDMA/uverbs: Push out CQ buffer umem processing into a helper Jiri Pirko
2026-04-21 13:25   ` Jason Gunthorpe
2026-04-22 10:56     ` Jiri Pirko
2026-04-22 16:32       ` Jason Gunthorpe
2026-04-11 14:49 ` [PATCH rdma-next v2 03/15] RDMA/uverbs: Integrate umem_list into CQ creation Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 04/15] RDMA/efa: Use umem_list for user CQ buffer Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 05/15] RDMA/mlx5: " Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 06/15] RDMA/bnxt_re: " Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 07/15] RDMA/mlx4: " Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 08/15] RDMA/uverbs: Remove legacy umem field from struct ib_cq Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 09/15] RDMA/uverbs: Verify all umem_list buffers are consumed after CQ creation Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 10/15] RDMA/uverbs: Integrate umem_list into QP creation Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 11/15] RDMA/mlx5: Use umem_list for QP buffers in create_qp Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 12/15] RDMA/uverbs: Add doorbell record buffer slot to CQ umem_list Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 13/15] RDMA/mlx5: Use umem_list for CQ doorbell record Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 14/15] RDMA/uverbs: Add doorbell record buffer slot to QP umem_list Jiri Pirko
2026-04-11 14:49 ` [PATCH rdma-next v2 15/15] RDMA/mlx5: Use umem_list for QP doorbell record Jiri Pirko
  -- strict thread matches above, loose matches on Subject: below --
2026-04-25  6:04 [PATCH v2 0/2] RDMA/rxe: Fix per-netns UDP tunnel issues Kuniyuki Iwashima
2026-04-25  6:04 ` [PATCH v2 1/2] RDMA/rxe: Fix null-ptr-deref in kernel_sock_shutdown() Kuniyuki Iwashima
2026-04-25 15:47   ` David Ahern
2026-04-25 20:55     ` Kuniyuki Iwashima
2026-04-26 16:40       ` David Ahern
2026-04-25 21:25   ` Zhu Yanjun
2026-04-26 16:42     ` David Ahern
2026-04-25  6:04 ` [PATCH v2 2/2] RDMA/rxe: Fix up RCU usage for rxe_ns_pernet_sk6() Kuniyuki Iwashima
2026-04-25 21:26   ` Zhu Yanjun
2026-03-31  5:56 [PATCH v2 0/4] Firmware LSM hook Leon Romanovsky
2026-03-31  5:56 ` [PATCH v2 1/4] bpf: add firmware command validation hook Leon Romanovsky
2026-04-16  8:43   ` Matt Bobrowski
2026-03-31  5:56 ` [PATCH v2 2/4] selftests/bpf: add test cases for fw_validate_cmd hook Leon Romanovsky
2026-03-31  5:56 ` [PATCH v2 3/4] RDMA/mlx5: Externally validate FW commands supplied in DEVX interface Leon Romanovsky
2026-03-31  5:56 ` [PATCH v2 4/4] fwctl/mlx5: Externally validate FW commands supplied in fwctl Leon Romanovsky
2026-04-09 12:12 ` [PATCH v2 0/4] Firmware LSM hook Leon Romanovsky
2026-04-09 12:27   ` Roberto Sassu
2026-04-09 12:45     ` Leon Romanovsky
2026-04-09 21:04       ` Paul Moore
2026-04-12  9:00         ` Leon Romanovsky
2026-04-13  1:38           ` Paul Moore
2026-04-13 15:53             ` Leon Romanovsky
2026-04-13 16:42             ` Jason Gunthorpe
2026-04-13 17:36               ` Casey Schaufler
2026-04-13 19:09                 ` Casey Schaufler
2026-04-13 22:36               ` Paul Moore
2026-04-13 23:19                 ` Jason Gunthorpe
2026-04-14 17:05                   ` Casey Schaufler
2026-04-14 19:09                     ` Paul Moore
2026-04-14 20:09                       ` Casey Schaufler
2026-04-14 20:44                         ` Paul Moore
2026-04-14 22:42                           ` Casey Schaufler
2026-04-15 21:03                             ` Paul Moore
2026-04-15 21:21                               ` Casey Schaufler
2026-04-14 20:27                   ` Paul Moore
2026-04-15 13:47                     ` Jason Gunthorpe
2026-04-15 21:40                       ` Paul Moore
2026-04-17 19:17                         ` Jason Gunthorpe
2026-04-21  0:58                           ` Paul Moore
2026-04-24 14:36                             ` Jason Gunthorpe
2026-04-24 20:59                               ` Paul Moore
2026-04-24 22:13                                 ` Jason Gunthorpe
2026-04-23 14:09                           ` Leon Romanovsky
2026-04-24 14:19                             ` Jason Gunthorpe
2026-04-26 10:39                               ` Leon Romanovsky
2026-04-23 13:05                         ` Leon Romanovsky

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260426225034.GA3540346@ziepe.ca \
    --to=jgg@ziepe.ca \
    --cc=andrew.gospodarek@broadcom.com \
    --cc=edwards@nvidia.com \
    --cc=gal.pressman@linux.dev \
    --cc=huangjunxian6@hisilicon.com \
    --cc=jiri@resnulli.us \
    --cc=kalesh-anakkur.purayil@broadcom.com \
    --cc=leon@kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=lirongqing@baidu.com \
    --cc=marco.crivellari@suse.com \
    --cc=mbloch@nvidia.com \
    --cc=michaelgur@nvidia.com \
    --cc=mrgolin@amazon.com \
    --cc=ohartoov@nvidia.com \
    --cc=parav@nvidia.com \
    --cc=phaddad@nvidia.com \
    --cc=roman.gushchin@linux.dev \
    --cc=selvin.xavier@broadcom.com \
    --cc=shayd@nvidia.com \
    --cc=sleybo@amazon.com \
    --cc=sriharsha.basavapatna@broadcom.com \
    --cc=yanjun.zhu@linux.dev \
    --cc=ynachum@amazon.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox