Re: [PATCH rdma-next v2 01/15] RDMA/core: Introduce generic buffer descriptor infrastructure for umem

[Leon Romanovsky <leon@kernel.org>]

On Mon, Apr 27, 2026 at 12:48:46PM +0200, Jiri Pirko wrote:
> Mon, Apr 27, 2026 at 12:50:34AM +0200, jgg@ziepe.ca wrote:
> >On Sun, Apr 26, 2026 at 04:53:40PM +0300, Leon Romanovsky wrote:
> >> > Well, brainstorming idea. I'd like to hear from Leon too
> >> > 
> >> > But if we set the general goals as:
> >> > 
> >> > 1) All umem creations should have a struct ib_uverbs_buffer_desc at
> >> >    the UAPI boundary
> >> > 2) ib_uverbs_buffer_desc should pass directly to umem code without any
> >> >    driver touching it. ib_uverbs_buffer_desc should be the only way to
> >> >    create a umem from a driver.
> >> > 3) Existing UWH umem descriptions must continue to work if the desc is
> >> >    not provided, by reforming them into a desc
> >> > 3) Cleanup and lifecycle should be centralized
> >> 
> >> I have mixed feelings about this. My CQ conversion showed that even a simple
> >> task like creating a CQ umem (numb_of_entries * size_of_entries) ends up full
> >> of creative hacks in various drivers. Because of that, I see real value in
> >> pushing as much logic as possible into the core code instead of duplicating it
> >> across drivers. However, my later attempt to change the QP path made it clear
> >> that creating umems in the core is not a viable goal in the general case.
> >> 
> >> Another outcome of that work was realizing that CQ resize (and probably MR
> >> rereg as well) becomes messy when we keep the "old" umem around. Splitting
> >> creation and cleanup into different layers probably will going to hurt us
> >> at some point of time.
> >> 
> >> To summarize:
> >> 1. The most practical fix is likely to provide a driver callback to create
> >>    the umem when needed, as you suggested.
> >> 2. We should reduce the use of UWH as much as possible in favor of a
> >>    well-defined schema. In the long run, we want to add more umem types,
> >>    and many drivers should work out of the box under that model.
> >> 3. Explicit behavior is preferable. If a driver creates something, the
> >>    driver should also clean it up.
> >> 
> >> I'm not saying no to your proposal, just expressing my thoughts.
> >
> >So, I think making small steps that upgrade all drivers will be
> >helpful here.
> >
> >If we can get all drivers calling the same attrs function and giving
> >their uhw parameters that is a good step closer to being able to put
> >that in a function if that is how things need to go down the road.
> >
> >And it does #2..
> >
> >Not sure about #3, we already moved toward core destroying umems it
> >may not be a good idea to try to undo that now.. But maybe we just
> >keep that for CQ and leave QP as driver managed?
> 
> I think that the uniform approach is better in order not to confuse
> people. Perhaps one day we will manage to untangle the create QP flow
> and separate user/kernel paths too. But:
> 
> After lots for thinking and back and forth motions, I tend to agree
> with Leon, the init/fini alloc/free should be symmetric
> in drivers. Now, that the core will never create umem and the driver is
> always responsible to create it, the driver should be the one to
> do release. Drivers still store the umem pointer locally.
> Then there is not need for ib_umem_list, makes things much easier
> and more neat. Another pro is, the code does not care if called on
> user or kernel part.

I will try to make this a bit more complicated. One of the reasons the
core code releases the umem is that umem has a special meaning: it is
memory pinned with DMA mapped to it  and exposed to user space,
and both HW and SW may read from or write to it. A driver that forgets
to release a umem can potentially cause serious problems to whole system.

So core at least needs to be aware of all exposed umems and be ready to
cleanup them if driver is removed.

I afraid that we will never find "correct" design here. We just need to
decide which approach has less drawbacks.

Thanks

> 
>