Re: ❌ FAIL: Test report for kernel 5.5.0-rc6-40f39e8.cki (arm-next)

[Veronika Kabatova <vkabatov@redhat.com>]

----- Original Message -----
> From: "Ard Biesheuvel" <ard.biesheuvel@linaro.org>
> To: "Will Deacon" <will@kernel.org>
> Cc: "Jianwen Ji" <jiji@redhat.com>, "Veronika Kabatova" <vkabatov@redhat.com>, "Hangbin Liu" <haliu@redhat.com>,
> "Catalin Marinas" <catalin.marinas@arm.com>, "Jianlin Shi" <jishi@redhat.com>, "CKI Project"
> <cki-project@redhat.com>, "linux-arm-kernel" <linux-arm-kernel@lists.infradead.org>
> Sent: Thursday, January 16, 2020 3:56:05 PM
> Subject: Re: ❌ FAIL: Test report for kernel 5.5.0-rc6-40f39e8.cki (arm-next)
> 
> On Thu, 16 Jan 2020 at 15:33, Will Deacon <will@kernel.org> wrote:
> >
> > Hi Veronika,
> >
> > On Thu, Jan 16, 2020 at 09:14:49AM -0500, Veronika Kabatova wrote:
> > > > On Thu, 16 Jan 2020 at 14:20, CKI Project < cki-project@redhat.com >
> > > > wrote:
> > > > > One or more kernel tests failed:
> > > >
> > >
> > > > > aarch64:
> > > >
> > > > > ❌ Networking tunnel: gre basic
> > > >
> > > > > ❌ Networking tunnel: vxlan basic
> > > >
> > > > May I kindly suggest that you stop cc'ing the mailing list on
> > > > automated,
> > > > unmoderated CI results?
> > >
> > > > It's great that you run a CI system, but you should really have the
> > > > minimal
> > > > courtesy to double check failure reports before sending them out into
> > > > the
> > > > world.
> > >
> > > we are sorry about that. The testing and emails to the list were
> > > requested
> > > by Will a while ago. If he or Catalin change their minds about them we
> > > will
> > > stop sending them, in the meanwhile you can always set up a filter.
> >
> > The results are usually very useful (thanks!), and I think it's good to
> > have them on the list so that people can see them and reply to them, if
> > necessary. I don't /think/ anybody is disagreeing with that.
> >

We're glad to hear that!

> > However, in this specific case, the results aren't very useful because the
> > infrastructure is playing up, so I think that's what Ard is getting at when
> > he says "unmoderated".
> >
> 
> Apologies for the kneejerk reaction, but there are quite a number of
> bots helpfully cc'ing the world on their reports these days, and the
> fact that these particular ones are known to be false positives
> triggered my unfiltered response.
> 
> 

No worries!

We're working with KernelCI to streamline the upstream testing efforts.
As everything is moving forward, you should see less bots on the list
(at least from people who are also working with the project).

These particular problems we ran into were harder to debug without extra
information (and not caused by our code but a project we depend on) so
we weren't able to filter them out automatically.

> 
> > > We are not paid by ARM so the testing is a "strive for the best" effort,
> > > as we have a bunch of ARM machines we're willing to lend to community for
> > > extra coverage. We always cc the test maintainers and also us to take a
> > > look at the issues if they do occur. As Rachel mentioned, we already put
> > > a workaround in place to prevent these issues from happening further so
> > > hopefully you won't get more emails because of these Beaker bugs.
> >
> > Thanks. If you get a chance, then it would be good to stop the test emails
> > going out to the mailing list while the infrastructure is known to be
> > broken, otherwise it can distract people into investigating failures that
> > aren't actually caused by changes to the kernel.
> >

I agree.

We do have a few things in mind to reduce the false positives (which would
also filter out these ones) and pipeline stabilization in general. This
includes a switch to stop the automatic reports when we detect a reoccurring
infra problem and can't easily work around it.


Thanks,
Veronika

> > Cheers,
> >
> > Will
> 
> 
> 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel