Kernel CI media test - Was: Re: [ANN] Media Summit September 16th: Final Agenda (v7)

Kernel CI media test - Was: Re: [ANN] Media Summit September 16th: Final Agenda (v7)

[Mauro Carvalho Chehab]

Hi Martin,

Em Wed, 25 Sep 2024 22:53:42 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> escreveu:

> Hi Martin,
> 
> On Fri, Sep 20, 2024 at 12:16:29PM +0000, Hecht, Martin (Avnet Silica) wrote:
> > Hey Hans and Mauro,
> > 
> > I remember also on a very little point regarding hardware for testing.
> > But we didn't go in detail again during the summit.
> > 
> > How do we can go ahead here? Are there some test systems up and
> > running somewhere centralized or how it is organized right now?  
> 
> Testing on real hardware is among our goals, but will require quite som
> extra work. We will likely need to setup lava labs and integrate them
> with media-ci. We had a discussion on Friday with kernel-ci developers,
> and we will probably benefit from ongoing work on their side. I don't
> think there's a plan to address this on our side in the very short term
> (mostly due to lack of time, we're currently focusing on getting
> media-ci up and running and integrated with the maintenance workflow).

With regards to integrating Avnet Silica labs for doing CI tests on
media hardware with upstream kernels, this is something I always wanted.

Yet, as Laurent mentioned, right now we're not doing it directly 
(but I guess Collabora is doing it for some media drivers they're 
developing).

From the discussions I had during LPC and the ones I also had one year
ago at ELCE, it seems that the best way to do it is by using Kernel CI
to aggregate results from different test labs.

The main idea is to use Kernel CI for such purpose.

With such purpose, let's start a separate thread to discuss it together
with the Kernel CI community. 

So, I'm c/c Kernel CI public ML here and Gustavo Padovan that have been 
involved on several efforts related to that. I had some hallway 
discussions with him during LPC.

It I recall correctly, we need to is:

1. To define a common test set (probably a subset of what we do 
   already for the virtual drivers);
2. add hardware platforms to Kernel CI infrastructure;
3. add some logic at Kernel CI to execute the tests at the hardware
   that will be made available at the labs.

From our discussions during the Media Summit, my understanding is that
Avnet Silica can help us with that by providing some hardware platforms
that could be integrated at Kernel CI infra and test real drivers with
real hardware. If you have someone to spare, maybe you can also contribute
with (1) and (3).

Anyway, this is just an introduction e-mail with what I captured so far
to start our discussions.


Thanks,
Mauro

Re: Kernel CI media test - Was: Re: [ANN] Media Summit September 16th: Final Agenda (v7)

[Gustavo Padovan]

Hello,


---- On Thu, 26 Sep 2024 11:14:50 -0300 Mauro Carvalho Chehab  wrote ---

 > Hi Martin, 
 >  
 > Em Wed, 25 Sep 2024 22:53:42 +0300 
 > Laurent Pinchart laurent.pinchart@ideasonboard.com> escreveu: 
 >  
 > > Hi Martin, 
 > > 
 > > On Fri, Sep 20, 2024 at 12:16:29PM +0000, Hecht, Martin (Avnet Silica) wrote: 
 > > > Hey Hans and Mauro, 
 > > > 
 > > > I remember also on a very little point regarding hardware for testing. 
 > > > But we didn't go in detail again during the summit. 
 > > > 
 > > > How do we can go ahead here? Are there some test systems up and 
 > > > running somewhere centralized or how it is organized right now? 
 > > 
 > > Testing on real hardware is among our goals, but will require quite som 
 > > extra work. We will likely need to setup lava labs and integrate them 
 > > with media-ci. We had a discussion on Friday with kernel-ci developers, 
 > > and we will probably benefit from ongoing work on their side. I don't 
 > > think there's a plan to address this on our side in the very short term 
 > > (mostly due to lack of time, we're currently focusing on getting 
 > > media-ci up and running and integrated with the maintenance workflow). 
 >  
 > With regards to integrating Avnet Silica labs for doing CI tests on 
 > media hardware with upstream kernels, this is something I always wanted. 
 >  
 > Yet, as Laurent mentioned, right now we're not doing it directly 
 > (but I guess Collabora is doing it for some media drivers they're 
 > developing). 
 >  
 > From the discussions I had during LPC and the ones I also had one year 
 > ago at ELCE, it seems that the best way to do it is by using Kernel CI 
 > to aggregate results from different test labs. 
 >  
 > The main idea is to use Kernel CI for such purpose. 
 >  
 > With such purpose, let's start a separate thread to discuss it together 
 > with the Kernel CI community. 
 >  
 > So, I'm c/c Kernel CI public ML here and Gustavo Padovan that have been 
 > involved on several efforts related to that. I had some hallway 
 > discussions with him during LPC. 
 >  
 > It I recall correctly, we need to is: 
 >  
 > 1. To define a common test set (probably a subset of what we do 
 >  already for the virtual drivers); 
 > 2. add hardware platforms to Kernel CI infrastructure; 
 > 3. add some logic at Kernel CI to execute the tests at the hardware 
 >  that will be made available at the labs. 

That is correct. We can connect labs into the KernelCI infrastructure through a
lab-runtime in maestro or if you have a CI system in place you can just listen
to tests events generate by KernelCI, execute them and send the results into
KCIDB for data aggregation as explained by Mauro.

 >  
 > From our discussions during the Media Summit, my understanding is that 
 > Avnet Silica can help us with that by providing some hardware platforms 
 > that could be integrated at Kernel CI infra and test real drivers with 
 > real hardware. If you have someone to spare, maybe you can also contribute 
 > with (1) and (3). 
 >  
 > Anyway, this is just an introduction e-mail with what I captured so far 
 > to start our discussions.

I'd happy to have a video meeting to look at the details or attend one
of the Media CI syncs.

Best, 

- Gus