Re: [PATCH rdma-next v2 01/15] RDMA/core: Introduce generic buffer descriptor infrastructure for umem

[Jiri Pirko <jiri@resnulli.us>]

Mon, Apr 27, 2026 at 08:54:45PM +0200, leon@kernel.org wrote:
>On Mon, Apr 27, 2026 at 12:48:46PM +0200, Jiri Pirko wrote:
>> Mon, Apr 27, 2026 at 12:50:34AM +0200, jgg@ziepe.ca wrote:
>> >On Sun, Apr 26, 2026 at 04:53:40PM +0300, Leon Romanovsky wrote:
>> >> > Well, brainstorming idea. I'd like to hear from Leon too
>> >> > 
>> >> > But if we set the general goals as:
>> >> > 
>> >> > 1) All umem creations should have a struct ib_uverbs_buffer_desc at
>> >> >    the UAPI boundary
>> >> > 2) ib_uverbs_buffer_desc should pass directly to umem code without any
>> >> >    driver touching it. ib_uverbs_buffer_desc should be the only way to
>> >> >    create a umem from a driver.
>> >> > 3) Existing UWH umem descriptions must continue to work if the desc is
>> >> >    not provided, by reforming them into a desc
>> >> > 3) Cleanup and lifecycle should be centralized
>> >> 
>> >> I have mixed feelings about this. My CQ conversion showed that even a simple
>> >> task like creating a CQ umem (numb_of_entries * size_of_entries) ends up full
>> >> of creative hacks in various drivers. Because of that, I see real value in
>> >> pushing as much logic as possible into the core code instead of duplicating it
>> >> across drivers. However, my later attempt to change the QP path made it clear
>> >> that creating umems in the core is not a viable goal in the general case.
>> >> 
>> >> Another outcome of that work was realizing that CQ resize (and probably MR
>> >> rereg as well) becomes messy when we keep the "old" umem around. Splitting
>> >> creation and cleanup into different layers probably will going to hurt us
>> >> at some point of time.
>> >> 
>> >> To summarize:
>> >> 1. The most practical fix is likely to provide a driver callback to create
>> >>    the umem when needed, as you suggested.
>> >> 2. We should reduce the use of UWH as much as possible in favor of a
>> >>    well-defined schema. In the long run, we want to add more umem types,
>> >>    and many drivers should work out of the box under that model.
>> >> 3. Explicit behavior is preferable. If a driver creates something, the
>> >>    driver should also clean it up.
>> >> 
>> >> I'm not saying no to your proposal, just expressing my thoughts.
>> >
>> >So, I think making small steps that upgrade all drivers will be
>> >helpful here.
>> >
>> >If we can get all drivers calling the same attrs function and giving
>> >their uhw parameters that is a good step closer to being able to put
>> >that in a function if that is how things need to go down the road.
>> >
>> >And it does #2..
>> >
>> >Not sure about #3, we already moved toward core destroying umems it
>> >may not be a good idea to try to undo that now.. But maybe we just
>> >keep that for CQ and leave QP as driver managed?
>> 
>> I think that the uniform approach is better in order not to confuse
>> people. Perhaps one day we will manage to untangle the create QP flow
>> and separate user/kernel paths too. But:
>> 
>> After lots for thinking and back and forth motions, I tend to agree
>> with Leon, the init/fini alloc/free should be symmetric
>> in drivers. Now, that the core will never create umem and the driver is
>> always responsible to create it, the driver should be the one to
>> do release. Drivers still store the umem pointer locally.
>> Then there is not need for ib_umem_list, makes things much easier
>> and more neat. Another pro is, the code does not care if called on
>> user or kernel part.
>
>I will try to make this a bit more complicated. One of the reasons the
>core code releases the umem is that umem has a special meaning: it is
>memory pinned with DMA mapped to it  and exposed to user space,
>and both HW and SW may read from or write to it. A driver that forgets
>to release a umem can potentially cause serious problems to whole system.
>
>So core at least needs to be aware of all exposed umems and be ready to
>cleanup them if driver is removed.

Hmm, I think this safety-accounting is possible. The list would be not
per-object but per-dev. Will try to draft it.


>
>I afraid that we will never find "correct" design here. We just need to
>decide which approach has less drawbacks.

Correct.


>
>Thanks
>
>> 
>>