Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Pull network and Patch Acceptance Consistency

Tim.Bird
2019-06-17T16:48:12Z

> -----Original Message-----
> From: Mark Brown [mailto:broonie@kernel.org]
>
> On Mon, Jun 17, 2019 at 08:03:15AM -0300, Mauro Carvalho Chehab wrote:
...
> > Are there any other subsystem currently working to get funding for
> > hosting/automation?
>
> Not sure if it's specifically what you're looking at but there's stuff
> going on that's at least very adjacent to this, more from the angle of
> providing general infrastructure than subsystem specific things and
> currently mainly foucsed on getting tests run. To me that sort of
> approach seems good since it avoids duplicated efforts between
> subsystems.
>
> There's people working on things like KernelCI (people are working on
> expanding to include runtime tests, and there's active efforts on
> securing more funding) and CKI which aren't focused on specific
> subsystems but more on general infrastructure. Tim Bird (CCed) has been
> pushing on trying to get people working in this area talking to each
> other - there's a mailing list and monthly call:
>
> https://elinux.org/Automated_Testing
>
> and one of the things people are talking about is what sorts of things
> the kernel community would find useful here so it's probably useful at
> least putting ideas for things that'd be useful in the heads of people
> who are interested in working on the infrastructure and automation end
> of things.

Indeed. Although I haven't piped up, I have been paying close attention to
these discussions, and I know from talking to others who are involved with
automated testing that they are as well.

I'm about to go on vacation, so I'll be incommunicado for about a week, but
just to highlight some of the stuff I'm keen on following up on:
- if Linus like the syzbot notification mechanism, we should definitely
follow up and try to have more tools and frameworks adopt that mechanism
It's on my to-do list to investigate this further and see how it would integrate
with my particular framework (Fuego). I think Dmitry said we should avoid
introducing bots with lots of different notification mechanisms, as that will
overload developers and just turn them off.
- I recently saw a very cool system for isolating new warnings (at all levels
of W=[0-3]) introduced by a new patch. This would be a great thing to
add to the kernel build system, IMHO, and that's also on my to-do list.
- I was very interested in discussions about the mechanism to check whether
a patch modified the binary or not. It would be nice to make this part of
the build system as well (something like: "make check-for-binary-change"

Both of the latter items require the ability to set a baseline to compare
against. So the usage sequence would be:
- make save-baseline
- <do commit or pull>
- make show-new-warnings
or
- make check-for-binary-change

Just FYI, some of the stuff that the automated testing folk are working on
are:
- standards for lab equipment management (/board management)
- standards for test definition
- standards for test results format and aggregation
- setting up a multi-framework results aggregation site

Keep the ideas flowing! While we sometimes don't chime in, I know people
are listening and thinking about the ideas presented. And many of us will be
at plumbers for live discussions, and at ELC Europe. A few of us are starting a new event
called Automated Testing Summit that we're co-locating (at least this year) with
OSSUE/ELCE on October 31, in Lyon France. Check the Linux Foundation events site
for more information.
-- Tim