M3 build status

M3 build status

[Richard Purdie]

Just to quickly update everyone, a number of patches we probably wanted
in 3.1 have had issues but have had the issues resolved and the patches
have now merged. Thanks to all who dived in and helped with those!

The patches/issues I'm aware of that are left:

* ltp upgrade (musl issue)
* util-linux upgrade (breaks wic test, probably simple fix)
* coreutils ptest addition (blocked on libmodule-build-per 
  reproducibility issue)
* psplash systemd race (smurray may have a fix)
* autobuilder new branch handling for initial run test comparisons
  (open high bug with RP)
* tinfoil race (open high bug, struggling for owner)
* LockedSigs intermittent test failure (RP has open high bug, no clue 
  how to reproduce or why, total mystery)
* Intermittent selftest failure with SystemExit (High open bug, Kai 
  owns)

As ever, help with any of these welcome. We'll build M3 when a majority
of the above are ready.

Cheers,

Richard

Re: M3 build status

[akuster808]

On 2/29/20 5:13 AM, Richard Purdie wrote:
> Just to quickly update everyone, a number of patches we probably wanted
> in 3.1 have had issues but have had the issues resolved and the patches
> have now merged. Thanks to all who dived in and helped with those!
So no Bind update?

>
> The patches/issues I'm aware of that are left:
>
> * ltp upgrade (musl issue)
> * util-linux upgrade (breaks wic test, probably simple fix)
I can look at the two above but can't start until Sunday so if someone
else grabs them today, have at it.
> * coreutils ptest addition (blocked on libmodule-build-per 
>   reproducibility issue)
> * psplash systemd race (smurray may have a fix)
> * autobuilder new branch handling for initial run test comparisons
>   (open high bug with RP)
> * tinfoil race (open high bug, struggling for owner)
> * LockedSigs intermittent test failure (RP has open high bug, no clue 
>   how to reproduce or why, total mystery)
> * Intermittent selftest failure with SystemExit (High open bug, Kai 
>   owns)
>
> As ever, help with any of these welcome. We'll build M3 when a majority
> of the above are ready.

So where are patches going doing this time that will be in after the
release?

- armin
> Cheers,
>
> Richard
>
>
>
>
>

Re: M3 build status

[Richard Purdie]

On Sat, 2020-02-29 at 07:38 -0800, akuster808 wrote:
> 
> On 2/29/20 5:13 AM, Richard Purdie wrote:
> > Just to quickly update everyone, a number of patches we probably
> > wanted
> > in 3.1 have had issues but have had the issues resolved and the
> > patches
> > have now merged. Thanks to all who dived in and helped with those!
> So no Bind update?

There is still time, please send an updated patch!

> > The patches/issues I'm aware of that are left:
> > 
> > * ltp upgrade (musl issue)
> > * util-linux upgrade (breaks wic test, probably simple fix)
> I can look at the two above but can't start until Sunday so if
> someone else grabs them today, have at it.

These are now done thankfully.

> > * coreutils ptest addition (blocked on libmodule-build-per 
> >   reproducibility issue)

Still not done.

> > * psplash systemd race (smurray may have a fix)

Still not done.

> > * autobuilder new branch handling for initial run test comparisons
> >   (open high bug with RP)
> > * tinfoil race (open high bug, struggling for owner)

Both done.

> > * LockedSigs intermittent test failure (RP has open high bug, no
> > clue 
> >   how to reproduce or why, total mystery)

Spent hours on this, limited success :(

> > * Intermittent selftest failure with SystemExit (High open bug,
> > Kai owns)

Still pending.

> > As ever, help with any of these welcome. We'll build M3 when a
> > majority of the above are ready.

Also, I've realised:
* there is a high priority pseudo issue assigned to me
* gcc buildtools tarball issue

+ the bind issue you mention

> So where are patches going doing this time that will be in after the
> release?

I try not to encourage people to send upgrade patches during
stabilisation as its a distraction for everyone and means fewer people
caring about the release. I have had to queue them in the past since
people ignore this and I may have to again depending on what people do
:(.

Cheers,

Richard

Re: M3 build status

[akuster808]

On 3/2/20 8:47 AM, Richard Purdie wrote:
> On Sat, 2020-02-29 at 07:38 -0800, akuster808 wrote:
>> On 2/29/20 5:13 AM, Richard Purdie wrote:
>>> Just to quickly update everyone, a number of patches we probably
>>> wanted
>>> in 3.1 have had issues but have had the issues resolved and the
>>> patches
>>> have now merged. Thanks to all who dived in and helped with those!
>> So no Bind update?
> There is still time, please send an updated patch!
Its unclear to me what change is outstanding.

- armin
>
>>> The patches/issues I'm aware of that are left:
>>>
>>> * ltp upgrade (musl issue)
>>> * util-linux upgrade (breaks wic test, probably simple fix)
>> I can look at the two above but can't start until Sunday so if
>> someone else grabs them today, have at it.
> These are now done thankfully.
>
>>> * coreutils ptest addition (blocked on libmodule-build-per 
>>>   reproducibility issue)
> Still not done.
>
>>> * psplash systemd race (smurray may have a fix)
> Still not done.
>
>>> * autobuilder new branch handling for initial run test comparisons
>>>   (open high bug with RP)
>>> * tinfoil race (open high bug, struggling for owner)
> Both done.
>
>>> * LockedSigs intermittent test failure (RP has open high bug, no
>>> clue 
>>>   how to reproduce or why, total mystery)
> Spent hours on this, limited success :(
>
>>> * Intermittent selftest failure with SystemExit (High open bug,
>>> Kai owns)
> Still pending.
>
>>> As ever, help with any of these welcome. We'll build M3 when a
>>> majority of the above are ready.
> Also, I've realised:
> * there is a high priority pseudo issue assigned to me
> * gcc buildtools tarball issue
>
> + the bind issue you mention
>
>> So where are patches going doing this time that will be in after the
>> release?
> I try not to encourage people to send upgrade patches during
> stabilisation as its a distraction for everyone and means fewer people
> caring about the release. I have had to queue them in the past since
> people ignore this and I may have to again depending on what people do
> :(.
>
> Cheers,
>
> Richard
>
>
>
>
>

Re: M3 build status

[Richard Purdie]

On Mon, 2020-03-02 at 20:17 -0800, akuster808 wrote:
> 
> On 3/2/20 8:47 AM, Richard Purdie wrote:
> > On Sat, 2020-02-29 at 07:38 -0800, akuster808 wrote:
> > > On 2/29/20 5:13 AM, Richard Purdie wrote:
> > > > Just to quickly update everyone, a number of patches we
> > > > probably
> > > > wanted
> > > > in 3.1 have had issues but have had the issues resolved and the
> > > > patches
> > > > have now merged. Thanks to all who dived in and helped with
> > > > those!
> > > So no Bind update?
> > There is still time, please send an updated patch!
> Its unclear to me what change is outstanding.

libuv maintainer entry?

If I merge what is there if will break builds :(

Cheers,

Richard

Re: M3 build status

[Richard Purdie]

On Mon, 2020-03-02 at 16:47 +0000, Richard Purdie wrote:
> On Sat, 2020-02-29 at 07:38 -0800, akuster808 wrote:
> 
> > > * coreutils ptest addition (blocked on libmodule-build-per 
> > >   reproducibility issue)
> 
> Still not done.

This is now closer. Alex fixed the libmodule-build-perl and gettext
reproducibility issues (thanks!), I tracked down a glibc issue and
procps one I saw locally. On a fresh autobuilder run we saw:

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20200307-mk0v6ljq/packages/diff-html/
and
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20200307-5pvgdd3a/packages/diff-html/

which I believe is the remaining blocking issue on coreutils-ptest.

> > > * psplash systemd race (smurray may have a fix)
> 
> Still not done.

Now done.

> > > * LockedSigs intermittent test failure (RP has open high bug, no
> > > clue 
> > >   how to reproduce or why, total mystery)
> 
> Spent hours on this, limited success :(

Turned out to be a logging issue, hopefully fixed.

> 
> > > * Intermittent selftest failure with SystemExit (High open bug,
> > > Kai owns)
> 
> Still pending.

Still pending. There are also other intermittent Medium+ selftest bugs
we're seeing now the source of major failures has been addressed.

> > > As ever, help with any of these welcome. We'll build M3 when a
> > > majority of the above are ready.
> 
> Also, I've realised:
> * there is a high priority pseudo issue assigned to me

Not done.

> * gcc buildtools tarball issue

Moved forwards. The tarball breaks eSDK testing and my fix didn't work.

There is an open question of whether we want to automate use of this or
not and whether we support Centos7 for the LTS with or without the
tarball.

> + the bind issue you mention

We've established we can't take that.

This puts us closer to the M3 build but probably not quite there yet.

Cheers,

Richard

Re: M3 build status

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 922 bytes --]

On Sun, 8 Mar 2020 at 09:15, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> This is now closer. Alex fixed the libmodule-build-perl and gettext
> reproducibility issues (thanks!), I tracked down a glibc issue and
> procps one I saw locally. On a fresh autobuilder run we saw:
>
>
> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20200307-mk0v6ljq/packages/diff-html/
> and
>
> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20200307-5pvgdd3a/packages/diff-html/
>
> which I believe is the remaining blocking issue on coreutils-ptest.
>

Not quite. The coreutils-ptest itself seems to persistently fail here:

AssertionError: Failed ptests:
{'coreutils': ['tests/tail-2/assert.sh']}

so I'd like to have that fixed before it goes in. We have a 100% ptest pass
rate now (except for random timing fluctuations), which I don't want to see
regressed :)

Alex

[-- Attachment #2: Type: text/html, Size: 1619 bytes --]

Re: M3 build status

[Richard Purdie]

On Sun, 2020-03-08 at 14:08 +0100, Alexander Kanavin wrote:
> On Sun, 8 Mar 2020 at 09:15, Richard Purdie <
> richard.purdie@linuxfoundation.org> wrote:
> > This is now closer. Alex fixed the libmodule-build-perl and gettext
> > reproducibility issues (thanks!), I tracked down a glibc issue and
> > procps one I saw locally. On a fresh autobuilder run we saw:
> > 
> > https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20200307-mk0v6ljq/packages/diff-html/
> > and
> > https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20200307-5pvgdd3a/packages/diff-html/
> > 
> > which I believe is the remaining blocking issue on coreutils-ptest.
> 
> Not quite. The coreutils-ptest itself seems to persistently fail
> here:
> 
> AssertionError: Failed ptests:
> {'coreutils': ['tests/tail-2/assert.sh']}
> 
> so I'd like to have that fixed before it goes in. We have a 100%
> ptest pass rate now (except for random timing fluctuations), which I
> don't want to see regressed :)

I agree that would be nice and something we need to do before release,
I may let us sort that one test in M4 to enable the M3 build, I have to
take a pragmatic approach to this.

That said, there is a more pressing issue:

https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/1686

which is somehow seemingly related to the coreutils change, I just
can't see how as yet :/

(as well as the gcc-plugin reproducibility issues)

Cheers,

Richard

Re: M3 build status

[Richard Purdie]

On Sun, 2020-03-08 at 13:18 +0000, Richard Purdie wrote:
> On Sun, 2020-03-08 at 14:08 +0100, Alexander Kanavin wrote:
> > On Sun, 8 Mar 2020 at 09:15, Richard Purdie <
> > richard.purdie@linuxfoundation.org> wrote:
> > > This is now closer. Alex fixed the libmodule-build-perl and
> > > gettext
> > > reproducibility issues (thanks!), I tracked down a glibc issue
> > > and
> > > procps one I saw locally. On a fresh autobuilder run we saw:
> > > 
> > > https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20200307-mk0v6ljq/packages/diff-html/
> > > and
> > > https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20200307-5pvgdd3a/packages/diff-html/
> > > 
> > > which I believe is the remaining blocking issue on coreutils-
> > > ptest.
> > 
> > Not quite. The coreutils-ptest itself seems to persistently fail
> > here:
> > 
> > AssertionError: Failed ptests:
> > {'coreutils': ['tests/tail-2/assert.sh']}
> > 
> > so I'd like to have that fixed before it goes in. We have a 100%
> > ptest pass rate now (except for random timing fluctuations), which
> > I
> > don't want to see regressed :)
> 
> I agree that would be nice and something we need to do before
> release,
> I may let us sort that one test in M4 to enable the M3 build, I have
> to
> take a pragmatic approach to this.
> 
> That said, there is a more pressing issue:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/1686
> 
> which is somehow seemingly related to the coreutils change, I just
> can't see how as yet :/
> 
> (as well as the gcc-plugin reproducibility issues)

Caused by the new libmodule-build-perl which depends upon build-
essential and hence gcc which isn't target multilib 'safe'.

If we could stop the -ptest package dependencies leaking to -dev we'd
avoid this.

Cheers,

Richard

Re: M3 build status

[Trevor Gamblin]

On 3/8/20 10:34 AM, Richard Purdie wrote:
> On Sun, 2020-03-08 at 13:18 +0000, Richard Purdie wrote:
>> On Sun, 2020-03-08 at 14:08 +0100, Alexander Kanavin wrote:
>>> On Sun, 8 Mar 2020 at 09:15, Richard Purdie <
>>> richard.purdie@linuxfoundation.org> wrote:
>>>> This is now closer. Alex fixed the libmodule-build-perl and
>>>> gettext
>>>> reproducibility issues (thanks!), I tracked down a glibc issue
>>>> and
>>>> procps one I saw locally. On a fresh autobuilder run we saw:
>>>>
>>>> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20200307-mk0v6ljq/packages/diff-html/
>>>> and
>>>> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20200307-5pvgdd3a/packages/diff-html/
>>>>
>>>> which I believe is the remaining blocking issue on coreutils-
>>>> ptest.
>>> Not quite. The coreutils-ptest itself seems to persistently fail
>>> here:
>>>
>>> AssertionError: Failed ptests:
>>> {'coreutils': ['tests/tail-2/assert.sh']}
>>>
>>> so I'd like to have that fixed before it goes in. We have a 100%
>>> ptest pass rate now (except for random timing fluctuations), which
>>> I
>>> don't want to see regressed :)
>> I agree that would be nice and something we need to do before
>> release,
>> I may let us sort that one test in M4 to enable the M3 build, I have
>> to
>> take a pragmatic approach to this.
>>
>> That said, there is a more pressing issue:
>>
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/1686
>>
>> which is somehow seemingly related to the coreutils change, I just
>> can't see how as yet :/
>>
>> (as well as the gcc-plugin reproducibility issues)
> Caused by the new libmodule-build-perl which depends upon build-
> essential and hence gcc which isn't target multilib 'safe'.

coreutils is fighting hard to avoid getting a ptest!

I'll have a look at these issues first thing tomorrow.

>
> If we could stop the -ptest package dependencies leaking to -dev we'd
> avoid this.
>
> Cheers,
>
> Richard
>
>