Re: [yocto] QA notification for completed autobuilder build (yocto-4.3.rc1)

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Thu, 2023-10-19 at 10:36 +0200, Alexis Lothoré wrote:
> Hello,
> 
> On 10/18/23 08:16, Pokybuild User wrote:
> > 
> >     A build flagged for QA (yocto-4.3.rc1) was completed on the autobuilder and is available at:
> > 
> > 
> >         https://autobuilder.yocto.io/pub/releases/yocto-4.3.rc1
> 
> The regression report looks worryingly empty. It has been generated with
> yocto-4.2 as comparison base. When taking a look at yocto-testresults content
> for it, I only find a few test results (11, while it should be something around
> 300). I am not sure yet if we did some mistakes while re-uploading manually
> missing tests results during the 4.2 cycle, or if tests results are indeed
> suffering some issues for release builds (this is repeatable on different 4.2.x
> releases).

I'd noticed that too, not sure what happened but it wasn't what I'd
expected. It would be good to understand what went wrong.

> So in the mean time, here is a manual regression report between 4.3_M3 and
> yocto-4.3.rc1:
> 
> https://pastebin.com/6QbfGstR
> 
> The report has a "rate limit" applied for noisy regression (patch incoming)

I was thinking about ideas like printing the regressions first, then
the matches. I think Ross has some good feedback about how we can
improve things.

Cheers,

Richard