Requirements to merge new heaps in the kernel

Requirements to merge new heaps in the kernel

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 721 bytes --]

Hi Sumit,

I wanted to follow-up on the discussion we had at Plumbers with John and
T.J. about (among other things) adding new heaps to the kernel.

I'm still interested in merging a carve-out driver[1], since it seems to be
in every vendor BSP and got asked again last week.

I remember from our discussion that for new heap types to be merged, we
needed a kernel use-case. Looking back, I'm not entirely sure how one
can provide that given that heaps are essentially facilities for
user-space.

Am I misremembering or missing something? What are the requirements for
you to consider adding a new heap driver?

Thanks!
Maxime

1: https://lore.kernel.org/dri-devel/20240515-dma-buf-ecc-heap-v1-1-54cbbd049511@kernel.org/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 273 bytes --]

Re: Requirements to merge new heaps in the kernel

[John Stultz]

On Tue, Oct 22, 2024 at 1:38 AM Maxime Ripard <mripard@redhat.com> wrote:
>
> I wanted to follow-up on the discussion we had at Plumbers with John and
> T.J. about (among other things) adding new heaps to the kernel.
>
> I'm still interested in merging a carve-out driver[1], since it seems to be
> in every vendor BSP and got asked again last week.
>
> I remember from our discussion that for new heap types to be merged, we
> needed a kernel use-case. Looking back, I'm not entirely sure how one
> can provide that given that heaps are essentially facilities for
> user-space.
>
> Am I misremembering or missing something? What are the requirements for
> you to consider adding a new heap driver?

It's basically the same as the DRM subsystem rules.
https://docs.kernel.org/gpu/drm-uapi.html#open-source-userspace-requirements
ie: There has to be opensource user for it, and the user has to be
more significant than a "toy" implementation (which can be a bit
subjective and contentious when trying to get out of a chicken and egg
loop).

thanks
-john

Re: Requirements to merge new heaps in the kernel

[Nicolas Dufresne]

Hi,

Le mardi 22 octobre 2024 à 09:19 -0700, John Stultz a écrit :
> On Tue, Oct 22, 2024 at 1:38 AM Maxime Ripard <mripard@redhat.com> wrote:
> > 
> > I wanted to follow-up on the discussion we had at Plumbers with John and
> > T.J. about (among other things) adding new heaps to the kernel.
> > 
> > I'm still interested in merging a carve-out driver[1], since it seems to be
> > in every vendor BSP and got asked again last week.
> > 
> > I remember from our discussion that for new heap types to be merged, we
> > needed a kernel use-case. Looking back, I'm not entirely sure how one
> > can provide that given that heaps are essentially facilities for
> > user-space.
> > 
> > Am I misremembering or missing something? What are the requirements for
> > you to consider adding a new heap driver?
> 
> It's basically the same as the DRM subsystem rules.
> https://docs.kernel.org/gpu/drm-uapi.html#open-source-userspace-requirements
> ie: There has to be opensource user for it, and the user has to be
> more significant than a "toy" implementation (which can be a bit
> subjective and contentious when trying to get out of a chicken and egg
> loop).

If there is a generic logic to decide to use a carve-out when using some
specific device on specific platform, it would not be a problem to make
userspace for it. I'm happy to take DMABuf patches in GStreamer notably (which
could greatly help ensuring zero-copy path).

But so far, all the proposals was just a base allocator, no way to know when to
use it and for which device. The actual mapping of heaps and device was left to
userspace, which to be honest would only work with a userspace Linux Allocator
library, with userspace drivers, or inside mesa if the devices are GPUs/NPUs.
This is a project Laurent Pinchard have hosted a workshop about during XDC.

Nicolas

p.s. libcamera have device specific knowledge, and could of course be a shorter
term user. Note that major distro are not happy that there is no memory
accounting for dmabuf, bypassing sandboxes and limits.

Re: Requirements to merge new heaps in the kernel

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 1308 bytes --]

On Tue, Oct 22, 2024 at 09:19:05AM -0700, John Stultz wrote:
> On Tue, Oct 22, 2024 at 1:38 AM Maxime Ripard <mripard@redhat.com> wrote:
> >
> > I wanted to follow-up on the discussion we had at Plumbers with John and
> > T.J. about (among other things) adding new heaps to the kernel.
> >
> > I'm still interested in merging a carve-out driver[1], since it seems to be
> > in every vendor BSP and got asked again last week.
> >
> > I remember from our discussion that for new heap types to be merged, we
> > needed a kernel use-case. Looking back, I'm not entirely sure how one
> > can provide that given that heaps are essentially facilities for
> > user-space.
> >
> > Am I misremembering or missing something? What are the requirements for
> > you to consider adding a new heap driver?
> 
> It's basically the same as the DRM subsystem rules.
> https://docs.kernel.org/gpu/drm-uapi.html#open-source-userspace-requirements
> ie: There has to be opensource user for it, and the user has to be
> more significant than a "toy" implementation (which can be a bit
> subjective and contentious when trying to get out of a chicken and egg
> loop).

Ok, so I'm definitely misremembering things then, I thought there was
another requirement in addition to that one. Thanks :)

Maxime

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 273 bytes --]

Re: Requirements to merge new heaps in the kernel

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 2852 bytes --]

On Tue, Oct 22, 2024 at 01:58:47PM -0400, Nicolas Dufresne wrote:
> Hi,
> 
> Le mardi 22 octobre 2024 à 09:19 -0700, John Stultz a écrit :
> > On Tue, Oct 22, 2024 at 1:38 AM Maxime Ripard <mripard@redhat.com> wrote:
> > > 
> > > I wanted to follow-up on the discussion we had at Plumbers with John and
> > > T.J. about (among other things) adding new heaps to the kernel.
> > > 
> > > I'm still interested in merging a carve-out driver[1], since it seems to be
> > > in every vendor BSP and got asked again last week.
> > > 
> > > I remember from our discussion that for new heap types to be merged, we
> > > needed a kernel use-case. Looking back, I'm not entirely sure how one
> > > can provide that given that heaps are essentially facilities for
> > > user-space.
> > > 
> > > Am I misremembering or missing something? What are the requirements for
> > > you to consider adding a new heap driver?
> > 
> > It's basically the same as the DRM subsystem rules.
> > https://docs.kernel.org/gpu/drm-uapi.html#open-source-userspace-requirements
> > ie: There has to be opensource user for it, and the user has to be
> > more significant than a "toy" implementation (which can be a bit
> > subjective and contentious when trying to get out of a chicken and egg
> > loop).
> 
> If there is a generic logic to decide to use a carve-out when using some
> specific device on specific platform, it would not be a problem to make
> userspace for it. I'm happy to take DMABuf patches in GStreamer notably (which
> could greatly help ensuring zero-copy path).

Yeah, that's one of the things we discussed at Plumbers too. My
point-of-view was that userspace also had no way to tell which kind of
buffers it would get. We settled down on the heap name providing those
semantics, and it resulted in:

https://lore.kernel.org/r/20240930144057.453751-1-mripard@kernel.org

> But so far, all the proposals was just a base allocator, no way to know when to
> use it and for which device. The actual mapping of heaps and device was left to
> userspace, which to be honest would only work with a userspace Linux Allocator
> library, with userspace drivers, or inside mesa if the devices are GPUs/NPUs.
> This is a project Laurent Pinchard have hosted a workshop about during XDC.

Yeah, that's another issue that needs to be tackled at some point indeed.

> p.s. libcamera have device specific knowledge, and could of course be a shorter
> term user. Note that major distro are not happy that there is no memory
> accounting for dmabuf, bypassing sandboxes and limits.

Meh. The same argument could be said for v4l2 or DRM/KMS, and it never
bothered anyone.

Fortunately, we're tackling that issue as well:
https://lore.kernel.org/dri-devel/20241023075302.27194-1-maarten.lankhorst@linux.intel.com/

Maxime

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 273 bytes --]

Re: Requirements to merge new heaps in the kernel

[metux]

On 22.10.24 10:38, Maxime Ripard wrote:

Hi,

> I'm still interested in merging a carve-out driver[1], since it seems to be
> in every vendor BSP and got asked again last week.
>
> I remember from our discussion that for new heap types to be merged, we
> needed a kernel use-case. Looking back, I'm not entirely sure how one
> can provide that given that heaps are essentially facilities for
> user-space.

For those who didn't follow your work, could you please give a short
intro what's that all about ?

If I understand you correctly, you'd like the infrastructure of
kmalloc() et al for things / memory regions that aren't the usual heap,
right ?

What's the practical use case ? GPU memory ? Shared memory between
nodes in a multi-CPU / cluster machine ?

Is it related to NUMA ?


thx
--mtx

Re: Requirements to merge new heaps in the kernel

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 1179 bytes --]

On Wed, Oct 30, 2024 at 12:16:22PM +0100, metux wrote:
> On 22.10.24 10:38, Maxime Ripard wrote:
> > I'm still interested in merging a carve-out driver[1], since it seems to be
> > in every vendor BSP and got asked again last week.
> > 
> > I remember from our discussion that for new heap types to be merged, we
> > needed a kernel use-case. Looking back, I'm not entirely sure how one
> > can provide that given that heaps are essentially facilities for
> > user-space.
> 
> For those who didn't follow your work, could you please give a short
> intro what's that all about ?
> 
> If I understand you correctly, you'd like the infrastructure of
> kmalloc() et al for things / memory regions that aren't the usual heap,
> right ?

No, not really. The discussion is about dma-buf heaps. They allow to
allocate buffers suitable for DMA from userspace. It might or might not
from the system memory, at the heap driver discretion.

> What's the practical use case ? GPU memory ? Shared memory between
> nodes in a multi-CPU / cluster machine ?
> 
> Is it related to NUMA ?

And since it's about DMA, it doesn't have much to do with CPUs either.

Maxime

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 273 bytes --]

Re: Requirements to merge new heaps in the kernel

[Al Viro]

On Thu, Oct 31, 2024 at 05:45:23PM +0100, Maxime Ripard wrote:
> On Wed, Oct 30, 2024 at 12:16:22PM +0100, metux wrote:
> > On 22.10.24 10:38, Maxime Ripard wrote:
> > > I'm still interested in merging a carve-out driver[1], since it seems to be
> > > in every vendor BSP and got asked again last week.
> > > 
> > > I remember from our discussion that for new heap types to be merged, we
> > > needed a kernel use-case. Looking back, I'm not entirely sure how one
> > > can provide that given that heaps are essentially facilities for
> > > user-space.
> > 
> > For those who didn't follow your work, could you please give a short
> > intro what's that all about ?
> > 
> > If I understand you correctly, you'd like the infrastructure of
> > kmalloc() et al for things / memory regions that aren't the usual heap,
> > right ?
> 
> No, not really. The discussion is about dma-buf heaps. They allow to
> allocate buffers suitable for DMA from userspace. It might or might not
> from the system memory, at the heap driver discretion.

I'm afraid you've misinterpreted that - our hexapedal friend had just
	* noticed that excessive crossposting can get it banned
	* got itself a new address
	* posted a solitary ping as the first test
	* followed that by testing the ability to cross-post (posting you'd
been replying to, contents on chatGPT level)
	* proceeded to use its shiny new address for more of the chorus
whinge exercise they'd been holding with several other similar fellows
(or sock puppets, for all I know).

Just ignore the wanker - it wasn't trying to get any information other
than "will the posting get through" anyway.