DRM Accel BoF at Linux Plumbers

DRM Accel BoF at Linux Plumbers

[Tomeu Vizoso]

Hi,

I would like to use the chance at the next Plumbers to discuss the
present challenges related to ML accelerators in mainline.

I'm myself more oriented towards edge-oriented deployments, and don't
know enough about how these accelerators are being used in the cloud
(and maybe desktop?) to tell if there is enough overlap to warrant a
common BoF.

In any case, these are the topics I would like to discuss, some
probably more relevant to the edge than to the cloud or desktop:

* What is stopping vendors from mainlining their drivers?

* How could we make it easier for them?

* Userspace API: how close are we from a common API that we can ask
userspace drivers to implement? What can be done to further this goal?

* Automated testing: DRM CI can be used, but would be good to have a
common test suite to run there. This is probably dependent on a common
userspace API.

* Other shared userspace infrastructure (compiler, execution,
synchronization, virtualization, ...)

* Firmware-mediated IP: what should we do about it, if anything?

* Any standing issues in DRM infra (GEM, gpu scheduler, DMABuf, etc)
that are hurting accel drivers?

What do people think, should we have a drivers/accel-wide BoF at
Plumbers? If so, what other topics should we have in the agenda?

Cheers,

Tomeu

Re: DRM Accel BoF at Linux Plumbers

[Laurent Pinchart]

Hi Tomeu,

On Sat, May 18, 2024 at 10:46:01AM +0200, Tomeu Vizoso wrote:
> Hi,
> 
> I would like to use the chance at the next Plumbers to discuss the
> present challenges related to ML accelerators in mainline.
> 
> I'm myself more oriented towards edge-oriented deployments, and don't
> know enough about how these accelerators are being used in the cloud
> (and maybe desktop?) to tell if there is enough overlap to warrant a
> common BoF.
> 
> In any case, these are the topics I would like to discuss, some
> probably more relevant to the edge than to the cloud or desktop:
> 
> * What is stopping vendors from mainlining their drivers?
> 
> * How could we make it easier for them?
> 
> * Userspace API: how close are we from a common API that we can ask
> userspace drivers to implement? What can be done to further this goal?
> 
> * Automated testing: DRM CI can be used, but would be good to have a
> common test suite to run there. This is probably dependent on a common
> userspace API.
> 
> * Other shared userspace infrastructure (compiler, execution,
> synchronization, virtualization, ...)
> 
> * Firmware-mediated IP: what should we do about it, if anything?
> 
> * Any standing issues in DRM infra (GEM, gpu scheduler, DMABuf, etc)
> that are hurting accel drivers?
> 
> What do people think, should we have a drivers/accel-wide BoF at
> Plumbers? If so, what other topics should we have in the agenda?

I'm interested in attending, even if so far I have limited involvement
in that area. Looking forward we're considering usage of ML accelerators
in libcamera for various purposes, so an open ecosystem will be crucial
for us.

-- 
Regards,

Laurent Pinchart

Re: DRM Accel BoF at Linux Plumbers

[Daniel Vetter]

On Sat, May 18, 2024 at 10:46:01AM +0200, Tomeu Vizoso wrote:
> Hi,
> 
> I would like to use the chance at the next Plumbers to discuss the
> present challenges related to ML accelerators in mainline.
> 
> I'm myself more oriented towards edge-oriented deployments, and don't
> know enough about how these accelerators are being used in the cloud
> (and maybe desktop?) to tell if there is enough overlap to warrant a
> common BoF.
> 
> In any case, these are the topics I would like to discuss, some
> probably more relevant to the edge than to the cloud or desktop:
> 
> * What is stopping vendors from mainlining their drivers?
> 
> * How could we make it easier for them?
> 
> * Userspace API: how close are we from a common API that we can ask
> userspace drivers to implement? What can be done to further this goal?
> 
> * Automated testing: DRM CI can be used, but would be good to have a
> common test suite to run there. This is probably dependent on a common
> userspace API.
> 
> * Other shared userspace infrastructure (compiler, execution,
> synchronization, virtualization, ...)
> 
> * Firmware-mediated IP: what should we do about it, if anything?
> 
> * Any standing issues in DRM infra (GEM, gpu scheduler, DMABuf, etc)
> that are hurting accel drivers?
> 
> What do people think, should we have a drivers/accel-wide BoF at
> Plumbers? If so, what other topics should we have in the agenda?

Yeah sounds good, and I'll try to at least attend lpc this year since it's
rather close ... Might be good to explicitly ping teams of merged and
in-flight drivers we have in accel already.

I think the topic list is at least a good starting point.

Cheers, Sima
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: DRM Accel BoF at Linux Plumbers

[Tomeu Vizoso]

On Tue, May 21, 2024 at 2:12 PM Daniel Vetter <daniel@ffwll.ch> wrote:
>
> On Sat, May 18, 2024 at 10:46:01AM +0200, Tomeu Vizoso wrote:
> > Hi,
> >
> > I would like to use the chance at the next Plumbers to discuss the
> > present challenges related to ML accelerators in mainline.
> >
> > I'm myself more oriented towards edge-oriented deployments, and don't
> > know enough about how these accelerators are being used in the cloud
> > (and maybe desktop?) to tell if there is enough overlap to warrant a
> > common BoF.
> >
> > In any case, these are the topics I would like to discuss, some
> > probably more relevant to the edge than to the cloud or desktop:
> >
> > * What is stopping vendors from mainlining their drivers?
> >
> > * How could we make it easier for them?
> >
> > * Userspace API: how close are we from a common API that we can ask
> > userspace drivers to implement? What can be done to further this goal?
> >
> > * Automated testing: DRM CI can be used, but would be good to have a
> > common test suite to run there. This is probably dependent on a common
> > userspace API.
> >
> > * Other shared userspace infrastructure (compiler, execution,
> > synchronization, virtualization, ...)
> >
> > * Firmware-mediated IP: what should we do about it, if anything?
> >
> > * Any standing issues in DRM infra (GEM, gpu scheduler, DMABuf, etc)
> > that are hurting accel drivers?
> >
> > What do people think, should we have a drivers/accel-wide BoF at
> > Plumbers? If so, what other topics should we have in the agenda?
>
> Yeah sounds good, and I'll try to at least attend lpc this year since it's
> rather close ... Might be good to explicitly ping teams of merged and
> in-flight drivers we have in accel already.

Sounds like a good idea to me. Will check if the people that sent the
previous aborted attempts are still interested in this.

Cheers,

Tomeu

> I think the topic list is at least a good starting point.
>
> Cheers, Sima
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch

Re: DRM Accel BoF at Linux Plumbers

[Jeffrey Hugo]

On 5/21/2024 8:41 AM, Tomeu Vizoso wrote:
> On Tue, May 21, 2024 at 2:12 PM Daniel Vetter <daniel@ffwll.ch> wrote:
>>
>> On Sat, May 18, 2024 at 10:46:01AM +0200, Tomeu Vizoso wrote:
>>> Hi,
>>>
>>> I would like to use the chance at the next Plumbers to discuss the
>>> present challenges related to ML accelerators in mainline.
>>>
>>> I'm myself more oriented towards edge-oriented deployments, and don't
>>> know enough about how these accelerators are being used in the cloud
>>> (and maybe desktop?) to tell if there is enough overlap to warrant a
>>> common BoF.
>>>
>>> In any case, these are the topics I would like to discuss, some
>>> probably more relevant to the edge than to the cloud or desktop:
>>>
>>> * What is stopping vendors from mainlining their drivers?
>>>
>>> * How could we make it easier for them?
>>>
>>> * Userspace API: how close are we from a common API that we can ask
>>> userspace drivers to implement? What can be done to further this goal?
>>>
>>> * Automated testing: DRM CI can be used, but would be good to have a
>>> common test suite to run there. This is probably dependent on a common
>>> userspace API.
>>>
>>> * Other shared userspace infrastructure (compiler, execution,
>>> synchronization, virtualization, ...)
>>>
>>> * Firmware-mediated IP: what should we do about it, if anything?
>>>
>>> * Any standing issues in DRM infra (GEM, gpu scheduler, DMABuf, etc)
>>> that are hurting accel drivers?
>>>
>>> What do people think, should we have a drivers/accel-wide BoF at
>>> Plumbers? If so, what other topics should we have in the agenda?
>>
>> Yeah sounds good, and I'll try to at least attend lpc this year since it's
>> rather close ... Might be good to explicitly ping teams of merged and
>> in-flight drivers we have in accel already.
> 
> Sounds like a good idea to me. Will check if the people that sent the
> previous aborted attempts are still interested in this

Looks like the Intel VPU folks are missing from this thread.

I like the idea of a BoF.  I suspect I will be remote but this list of 
topics looks good to me.  Nothing obvious missing from what I can tell.

-Jeff

Re: DRM Accel BoF at Linux Plumbers

[Jacek Lawrynowicz]

Hi,

On 21.05.2024 17:10, Jeffrey Hugo wrote:
> On 5/21/2024 8:41 AM, Tomeu Vizoso wrote:
>> On Tue, May 21, 2024 at 2:12 PM Daniel Vetter <daniel@ffwll.ch> wrote:
>>>
>>> On Sat, May 18, 2024 at 10:46:01AM +0200, Tomeu Vizoso wrote:
>>>> Hi,
>>>>
>>>> I would like to use the chance at the next Plumbers to discuss the
>>>> present challenges related to ML accelerators in mainline.
>>>>
>>>> I'm myself more oriented towards edge-oriented deployments, and don't
>>>> know enough about how these accelerators are being used in the cloud
>>>> (and maybe desktop?) to tell if there is enough overlap to warrant a
>>>> common BoF.
>>>>
>>>> In any case, these are the topics I would like to discuss, some
>>>> probably more relevant to the edge than to the cloud or desktop:
>>>>
>>>> * What is stopping vendors from mainlining their drivers?
>>>>
>>>> * How could we make it easier for them?
>>>>
>>>> * Userspace API: how close are we from a common API that we can ask
>>>> userspace drivers to implement? What can be done to further this goal?
>>>>
>>>> * Automated testing: DRM CI can be used, but would be good to have a
>>>> common test suite to run there. This is probably dependent on a common
>>>> userspace API.
>>>>
>>>> * Other shared userspace infrastructure (compiler, execution,
>>>> synchronization, virtualization, ...)
>>>>
>>>> * Firmware-mediated IP: what should we do about it, if anything?
>>>>
>>>> * Any standing issues in DRM infra (GEM, gpu scheduler, DMABuf, etc)
>>>> that are hurting accel drivers?
>>>>
>>>> What do people think, should we have a drivers/accel-wide BoF at
>>>> Plumbers? If so, what other topics should we have in the agenda?
>>>
>>> Yeah sounds good, and I'll try to at least attend lpc this year since it's
>>> rather close ... Might be good to explicitly ping teams of merged and
>>> in-flight drivers we have in accel already.
>>
>> Sounds like a good idea to me. Will check if the people that sent the
>> previous aborted attempts are still interested in this
> 
> Looks like the Intel VPU folks are missing from this thread.
Hi!

> I like the idea of a BoF.  I suspect I will be remote but this list of topics looks good to me.  Nothing obvious missing from what I can tell.
I like it too and I will try to attend. I would maybe add to the list GPU/accel interoperability.

Regards,
Jacek

Re: DRM Accel BoF at Linux Plumbers

[Tomeu Vizoso]

On Thu, May 23, 2024 at 8:35 AM Jacek Lawrynowicz
<jacek.lawrynowicz@linux.intel.com> wrote:
>
> Hi,
>
> On 21.05.2024 17:10, Jeffrey Hugo wrote:
> > On 5/21/2024 8:41 AM, Tomeu Vizoso wrote:
> >> On Tue, May 21, 2024 at 2:12 PM Daniel Vetter <daniel@ffwll.ch> wrote:
> >>>
> >>> On Sat, May 18, 2024 at 10:46:01AM +0200, Tomeu Vizoso wrote:
> >>>> Hi,
> >>>>
> >>>> I would like to use the chance at the next Plumbers to discuss the
> >>>> present challenges related to ML accelerators in mainline.
> >>>>
> >>>> I'm myself more oriented towards edge-oriented deployments, and don't
> >>>> know enough about how these accelerators are being used in the cloud
> >>>> (and maybe desktop?) to tell if there is enough overlap to warrant a
> >>>> common BoF.
> >>>>
> >>>> In any case, these are the topics I would like to discuss, some
> >>>> probably more relevant to the edge than to the cloud or desktop:
> >>>>
> >>>> * What is stopping vendors from mainlining their drivers?
> >>>>
> >>>> * How could we make it easier for them?
> >>>>
> >>>> * Userspace API: how close are we from a common API that we can ask
> >>>> userspace drivers to implement? What can be done to further this goal?
> >>>>
> >>>> * Automated testing: DRM CI can be used, but would be good to have a
> >>>> common test suite to run there. This is probably dependent on a common
> >>>> userspace API.
> >>>>
> >>>> * Other shared userspace infrastructure (compiler, execution,
> >>>> synchronization, virtualization, ...)
> >>>>
> >>>> * Firmware-mediated IP: what should we do about it, if anything?
> >>>>
> >>>> * Any standing issues in DRM infra (GEM, gpu scheduler, DMABuf, etc)
> >>>> that are hurting accel drivers?
> >>>>
> >>>> What do people think, should we have a drivers/accel-wide BoF at
> >>>> Plumbers? If so, what other topics should we have in the agenda?
> >>>
> >>> Yeah sounds good, and I'll try to at least attend lpc this year since it's
> >>> rather close ... Might be good to explicitly ping teams of merged and
> >>> in-flight drivers we have in accel already.
> >>
> >> Sounds like a good idea to me. Will check if the people that sent the
> >> previous aborted attempts are still interested in this
> >
> > Looks like the Intel VPU folks are missing from this thread.
> Hi!
>
> > I like the idea of a BoF.  I suspect I will be remote but this list of topics looks good to me.  Nothing obvious missing from what I can tell.
> I like it too and I will try to attend. I would maybe add to the list GPU/accel interoperability.

Thanks, that is a really good one.

Tomeu

Re: DRM Accel BoF at Linux Plumbers

[Tomeu Vizoso]

On Tue, May 21, 2024 at 2:12 PM Daniel Vetter <daniel@ffwll.ch> wrote:
>
> On Sat, May 18, 2024 at 10:46:01AM +0200, Tomeu Vizoso wrote:
> > Hi,
> >
> > I would like to use the chance at the next Plumbers to discuss the
> > present challenges related to ML accelerators in mainline.
> >
> > I'm myself more oriented towards edge-oriented deployments, and don't
> > know enough about how these accelerators are being used in the cloud
> > (and maybe desktop?) to tell if there is enough overlap to warrant a
> > common BoF.
> >
> > In any case, these are the topics I would like to discuss, some
> > probably more relevant to the edge than to the cloud or desktop:
> >
> > * What is stopping vendors from mainlining their drivers?
> >
> > * How could we make it easier for them?
> >
> > * Userspace API: how close are we from a common API that we can ask
> > userspace drivers to implement? What can be done to further this goal?
> >
> > * Automated testing: DRM CI can be used, but would be good to have a
> > common test suite to run there. This is probably dependent on a common
> > userspace API.
> >
> > * Other shared userspace infrastructure (compiler, execution,
> > synchronization, virtualization, ...)
> >
> > * Firmware-mediated IP: what should we do about it, if anything?
> >
> > * Any standing issues in DRM infra (GEM, gpu scheduler, DMABuf, etc)
> > that are hurting accel drivers?
> >
> > What do people think, should we have a drivers/accel-wide BoF at
> > Plumbers? If so, what other topics should we have in the agenda?
>
> Yeah sounds good, and I'll try to at least attend lpc this year since it's
> rather close ... Might be good to explicitly ping teams of merged and
> in-flight drivers we have in accel already.

Good point, I'm adding a bunch of people to CC, but I will be for sure
missing at least some, so I would be glad if people can check that
those that they know weren't missed.

Thanks,

Tomeu