Yocto Project Status WW51’17

Yocto Project Status WW51’17

[Jolley, Stephen K]

[-- Attachment #1: Type: text/plain, Size: 2648 bytes --]

Current Dev Position: YP 2.5 M2 development

Next Deadline: YP 2.5 M2 cut off of 1/15/18


SWAT team rotation: Stephano -> Maxin on Dec. 15, 2017.

SWAT team rotation: Maxin -> Cal on Dec. 22, 2017.

https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team


Key Status/Updates:

·        The QA of YP 2.5 M1 rc1 is just starting.  See: https://wiki.yoctoproject.org/wiki/2.5_QA_Status

·        The QA of YP 2.4.1 rc1 is about 80% complete.  See: https://wiki.yoctoproject.org/wiki/2.4_QA_Status. We are having some issues with the eclipse plugin testing.

·        The QA of YP 2.3.3 rc1 is about 3% complete. See: https://wiki.yoctoproject.org/wiki/2.3_QA_Status

·        YP 2.2.3 rc1 has been built and is queued to begin QA when the QA team has bandwidth.

·        We have finally tracked down the qemux86-64 kvm apic hang to an issue in the host kernel. We have a mitigation plan in mind but that is blocked on finding another separate bug causing selftest failures when upgrading the guest kernel.

·        We have identified further intermittent bugs in various releases which we’re continuing to look into (including gpg memory allocation issues on pyro, sstate writing races in morty, further occurrences of the inode count issues with our NAS on the autobuilder).


Planned upcoming dot releases:

YP 2.4.1 (Rocko) rc1 is in QA.  See https://wiki.yoctoproject.org/wiki/2.4_QA_Status.

YP 2.2.3 (Morty) rc1 is in QA.  See https://wiki.yoctoproject.org/wiki/2.3_QA_Status.

YP 2.3.3 (Pyro) is built and pending QA.  When work begins see https://wiki.yoctoproject.org/wiki/2.2_QA_Status


Key YP 2.5 Dates are:

YP 2.5 M1 release of 12/15/17 (Will be later this month.)

YP 2.5 M2 cut off of 1/15/18

YP 2.5 M2 release of 1/26/18

YP 2.5 M3 cut off of 2/19/18

YP 2.5 M3 release of 3/2/18

YP 2.5 M4 cut off of 4/2/18

YP 2.5 M4 release of 4/27/18


Tracking Metrics:

            WDD 2719 (last week 2690)

(https://wiki.yoctoproject.org/charts/combo.html)


Key Status Links for YP:

https://wiki.yoctoproject.org/wiki/Yocto_Project_v2.5_Status

https://wiki.yoctoproject.org/wiki/Yocto_2.5_Schedule

https://wiki.yoctoproject.org/wiki/Yocto_2.5_Features

[If anyone has suggestions for other information you’d like to see on this weekly status update, let us know!]

Thanks,

Stephen K. Jolley
Yocto Project Program Manager
INTEL, MS JF1-255, 2111 N.E. 25th Avenue, Hillsboro, OR 97124
•   Work Telephone:        (503) 712-0534
•    Cell:                              (208) 244-4460
• Email:                            stephen.k.jolley@intel.com


[-- Attachment #2: Type: text/html, Size: 20615 bytes --]

Re: Yocto Project Status WW51’17

[Randy MacLeod]

On 2017-12-18 10:52 AM, Jolley, Stephen K wrote:
> Current Dev Position: YP 2.5 M2 development
> 
> Next Deadline: YP 2.5 M2 cut off of 1/15/18
> 
> SWAT team rotation: Stephano -> Maxin on Dec. 15, 2017.
> 
> SWAT team rotation: Maxin -> Cal on Dec. 22, 2017.
> 
> https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team
> 
> Key Status/Updates:
> 
> ·The QA of YP 2.5 M1 rc1 is just starting.  See: 
> https://wiki.yoctoproject.org/wiki/2.5_QA_Status
> 
> ·The QA of YP 2.4.1 rc1 is about 80% complete.  See: 
> https://wiki.yoctoproject.org/wiki/2.4_QA_Status. We are having some 
> issues with the eclipse plugin testing.
> 
> ·The QA of YP 2.3.3 rc1 is about 3% complete. See: 
> https://wiki.yoctoproject.org/wiki/2.3_QA_Status
> 
> ·YP 2.2.3 rc1 has been built and is queued to begin QA when the QA team 
> has bandwidth.
> 
> ·We have finally tracked down the qemux86-64 kvm apic hang to an issue 
> in the host kernel. We have a mitigation plan in mind but that is 
> blocked on finding another separate bug causing selftest failures when 
> upgrading the guest kernel.
> 
> ·We have identified further intermittent bugs in various releases which 
> we’re continuing to look into (including gpg memory allocation issues on 
> pyro, sstate writing races in morty, further occurrences of the inode 
> count issues with our NAS on the autobuilder).

If any of the new problems prove to be difficult to debug, please
reply with the defect ID so that people can read the background
and perhaps help out in some way.

WR used to use a single large NAS for our build cluster 3+ years ago.
While it worked to first order, there was a background of mysterious
errors. We switch to local disks and pushing logs/status to
a centralized git repo and we've been happy with that choice.
-- 
# Randy MacLeod.  WR Linux
# Wind River an Intel Company

Re: Yocto Project Status WW51’17

[Richard Purdie]

On Fri, 2017-12-22 at 13:23 -0500, Randy MacLeod wrote:
> On 2017-12-18 10:52 AM, Jolley, Stephen K wrote:
> > ·We have identified further intermittent bugs in various releases
> > which 
> > we’re continuing to look into (including gpg memory allocation
> > issues on 
> > pyro, sstate writing races in morty, further occurrences of the
> > inode 
> > count issues with our NAS on the autobuilder).
> If any of the new problems prove to be difficult to debug, please
> reply with the defect ID so that people can read the background
> and perhaps help out in some way.

Thanks, I'm catching up after the holidays. The morty sstate writing
issue was resolved with a backport:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=morty&id=e3ff03599e55fd2abcaa33e29779c388cdbed94e

The gpg issue is:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=12022

I've only just realised that I appear to be the one assigned to fix it
:/. So that one remains open and in need of help. We put a single
threaded workaround into master/rocko I think but its slowing down
selftest and really needs a better fix.

> WR used to use a single large NAS for our build cluster 3+ years ago.
> While it worked to first order, there was a background of mysterious
> errors. We switch to local disks and pushing logs/status to
> a centralized git repo and we've been happy with that choice.

In this case we have it working quite well, except for one oddity where
the NFS client free inode counts go a bit crazy. We've decided to stop
checking the free inode count on the NAS for now whilst we continue to
see if we can figure it out which means builds should be working again
and not hitting this. The bug is:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=12267

So the gpg one is the one I could use help with.

Cheers,

Richard