New Linux accelerators discussion list [was: Re: Fostering linux community collaboration on hardware accelerators]

New Linux accelerators discussion list [was: Re: Fostering linux community collaboration on hardware accelerators]

[Andrew Donnellan]

On 17/10/17 01:07, Jonathan Cameron wrote:
> <snip>
> 
>>> So as ever with a linux community focusing on a particular topic, the
>>> obvious solution is a mailing list. There are a number of options on how
>>> do this.
>>>
>>> 1) Ask one of the industry bodies to host? Who?
>>>
>>> 2) Put together a compelling argument for linux-accelerators@vger.kernel.org
>>> as probably the most generic location for such a list.
>>
>> Happy to offer linux-accelerators@lists.ozlabs.org, which I can get set
>> up immediately (and if we want patchwork, patchwork.ozlabs.org is
>> available as always, no matter where the list is hosted).
>>
> 
> That would be great! Thanks for doing this. Much easier to find out what
> such a list is useful for by the practical option of having a list and
> see what people do with it.

[+ LKML]

We now have a mailing list:

   List:      linux-accelerators@lists.ozlabs.org
   Info:      https://lists.ozlabs.org/listinfo/linux-accelerators
   Archives:  https://lists.ozlabs.org/pipermail/linux-accelerators

I haven't set up Patchwork as yet, but if people think that's a good 
idea I can get that done too.


Andrew

-- 
Andrew Donnellan              OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com  IBM Australia Limited

Re: New Linux accelerators discussion list [was: Re: Fostering linux community collaboration on hardware accelerators]

[Jonathan Cameron]

On Tue, 17 Oct 2017 11:00:40 +1100
Andrew Donnellan <andrew.donnellan@au1.ibm.com> wrote:

> On 17/10/17 01:07, Jonathan Cameron wrote:
> > <snip>
> >   
> >>> So as ever with a linux community focusing on a particular topic, the
> >>> obvious solution is a mailing list. There are a number of options on how
> >>> do this.
> >>>
> >>> 1) Ask one of the industry bodies to host? Who?
> >>>
> >>> 2) Put together a compelling argument for linux-accelerators@vger.kernel.org
> >>> as probably the most generic location for such a list.  
> >>
> >> Happy to offer linux-accelerators@lists.ozlabs.org, which I can get set
> >> up immediately (and if we want patchwork, patchwork.ozlabs.org is
> >> available as always, no matter where the list is hosted).
> >>  
> > 
> > That would be great! Thanks for doing this. Much easier to find out what
> > such a list is useful for by the practical option of having a list and
> > see what people do with it.  
> 
> [+ LKML]
> 
> We now have a mailing list:
> 
>    List:      linux-accelerators@lists.ozlabs.org
>    Info:      https://lists.ozlabs.org/listinfo/linux-accelerators
>    Archives:  https://lists.ozlabs.org/pipermail/linux-accelerators
> 
> I haven't set up Patchwork as yet, but if people think that's a good 
> idea I can get that done too.
> 
> 
> Andrew
> 

Thanks Andrew.

A quick summary of initial thoughts on scope of this list for anyone
entering the discussion at this point.
Note it will hopefully evolve in whatever direction people find helpful.
This contains some suggestions not a barrier to wider scope!

There are a number of projects / applications involving what are
termed hardware accelerators.  These include:
* Traditional offload engines
	- Crypto, compression, media transcoding and similar accelerators
	- usecases including kTLS, Storage Encryption etc.
* Dynamic FPGA type accelerators
* ODP, DPDK and similar networking data plane - particularly dual stack
  solutions where the kernel 'plays nicely' with userspace drivers.
* AI Accelerators
* Shared Virtual Memory (SVM) bus systems including Open-CAPI, CCIX etc
* Fast data flow to/from userspace applications.

A number of existing project focus on these:
* Mainline kernel drivers
	- Numerous crypto drivers etc
	- Open-CAPI
* Various networking data plane activities
* VFIO based and similar userspace drivers
Hopefully this list can provide a broader forum where more general
questions are being considered.

The discussion that lead to this list was that a number of people would
like a general open forum on which to discuss ideas with scope beyond
simply one kernel subsystem or one particular userspace framework.

Topics might include
* RFCs and early reviews of new approaches.
* Similar hardware - who is trying to solve the same problems?
* What would we ideally want from new hardware iterations?
* Hardware description - the question of how to chose a particular
  crypto engine is very dependent on the particular data flows.
  Sometimes hardware accelerators don't actually help due to overheads.
  Understanding those barriers would be very useful.
* Upstream paths - blockers and how to work with the communities to
  overcome them.
* Fostering stronger userspace communities to allow these accelerators to 
  be easily harnessed.
	- A number of projects have been highlighted in this thread
	  OpenStack (cyborg project), openMP accelerator support 
* Robustness / security of userspace frameworks.
* Virtualisation of accelerators

Anyhow, this email was just meant to draw together some thoughts.
It will be interesting to see what the list actually gets used for :)

Jonathan

Re: New Linux accelerators discussion list [was: Re: Fostering linux community collaboration on hardware accelerators]

[Jonathan Cameron]

+ linux-accelerators@lists.ozlabs.org

Seems sensible to have this email actually go to the new list so
at least it appears in the archive.

Sorry all, I should have thought of this before pressing send,

Jonathan

On Tue, 17 Oct 2017 13:48:10 +0100
Jonathan Cameron <Jonathan.Cameron@huawei.com> wrote:

> On Tue, 17 Oct 2017 11:00:40 +1100
> Andrew Donnellan <andrew.donnellan@au1.ibm.com> wrote:
> 
> > On 17/10/17 01:07, Jonathan Cameron wrote:  
> > > <snip>
> > >     
> > >>> So as ever with a linux community focusing on a particular topic, the
> > >>> obvious solution is a mailing list. There are a number of options on how
> > >>> do this.
> > >>>
> > >>> 1) Ask one of the industry bodies to host? Who?
> > >>>
> > >>> 2) Put together a compelling argument for linux-accelerators@vger.kernel.org
> > >>> as probably the most generic location for such a list.    
> > >>
> > >> Happy to offer linux-accelerators@lists.ozlabs.org, which I can get set
> > >> up immediately (and if we want patchwork, patchwork.ozlabs.org is
> > >> available as always, no matter where the list is hosted).
> > >>    
> > > 
> > > That would be great! Thanks for doing this. Much easier to find out what
> > > such a list is useful for by the practical option of having a list and
> > > see what people do with it.    
> > 
> > [+ LKML]
> > 
> > We now have a mailing list:
> > 
> >    List:      linux-accelerators@lists.ozlabs.org
> >    Info:      https://lists.ozlabs.org/listinfo/linux-accelerators
> >    Archives:  https://lists.ozlabs.org/pipermail/linux-accelerators
> > 
> > I haven't set up Patchwork as yet, but if people think that's a good 
> > idea I can get that done too.
> > 
> > 
> > Andrew
> >   
> 
> Thanks Andrew.
> 
> A quick summary of initial thoughts on scope of this list for anyone
> entering the discussion at this point.
> Note it will hopefully evolve in whatever direction people find helpful.
> This contains some suggestions not a barrier to wider scope!
> 
> There are a number of projects / applications involving what are
> termed hardware accelerators.  These include:
> * Traditional offload engines
> 	- Crypto, compression, media transcoding and similar accelerators
> 	- usecases including kTLS, Storage Encryption etc.
> * Dynamic FPGA type accelerators
> * ODP, DPDK and similar networking data plane - particularly dual stack
>   solutions where the kernel 'plays nicely' with userspace drivers.
> * AI Accelerators
> * Shared Virtual Memory (SVM) bus systems including Open-CAPI, CCIX etc
> * Fast data flow to/from userspace applications.
> 
> A number of existing project focus on these:
> * Mainline kernel drivers
> 	- Numerous crypto drivers etc
> 	- Open-CAPI
> * Various networking data plane activities
> * VFIO based and similar userspace drivers
> Hopefully this list can provide a broader forum where more general
> questions are being considered.
> 
> The discussion that lead to this list was that a number of people would
> like a general open forum on which to discuss ideas with scope beyond
> simply one kernel subsystem or one particular userspace framework.
> 
> Topics might include
> * RFCs and early reviews of new approaches.
> * Similar hardware - who is trying to solve the same problems?
> * What would we ideally want from new hardware iterations?
> * Hardware description - the question of how to chose a particular
>   crypto engine is very dependent on the particular data flows.
>   Sometimes hardware accelerators don't actually help due to overheads.
>   Understanding those barriers would be very useful.
> * Upstream paths - blockers and how to work with the communities to
>   overcome them.
> * Fostering stronger userspace communities to allow these accelerators to 
>   be easily harnessed.
> 	- A number of projects have been highlighted in this thread
> 	  OpenStack (cyborg project), openMP accelerator support 
> * Robustness / security of userspace frameworks.
> * Virtualisation of accelerators
> 
> Anyhow, this email was just meant to draw together some thoughts.
> It will be interesting to see what the list actually gets used for :)
> 
> Jonathan