Recipe Updating Status and call to action

Recipe Updating Status and call to action

[Saul Wold]

[-- Attachment #1: Type: text/plain, Size: 1307 bytes --]


Folks,

As we get ready for M3 opening up shortly, I wanted to get folks aware 
of where we are with M2 and the recipe updating process.  Based on my 
rough reckoning, we had about 266 recipes that needed updating at the 
start of M2, we updated about 110 (possibly more).  There are now about 
160 recipes in the update list that need tackling.

Also, the distro tracking data for Version information is getting stale, 
there are about 135 RECIPE_LATEST_VERSION metadata tags that need to be 
updated.

I have attached a spreadsheet that shows the update status of both the 
upstream (RMatch Column) and tracking data (TMatch Column).

I would like to see the team and community pitch in and complete these 
160 recipe updates for the 1.0 Release. Please be sure the ownership / 
RECIPE_MAINTAINER information is correct.

IMPORTANT: Please use distro/stage in poky-contrib to update the 
distro_tracking_fields.inc file, I have many updates in that already, I 
will do my best not to rebase that until we get it updated.

The attached spread sheet contains 3 Sheets, please review the 2nd and 
3rd sheets (Labeled Tracking Updates and Recipe Updates respectively).

Thanks.

-- 
     Sau!

Saul Wold
Yocto Component Wrangler @ Intel
Yocto Project / Poky Build System


[-- Attachment #2: m3_recpie_update.ods --]
[-- Type: application/vnd.oasis.opendocument.spreadsheet, Size: 39541 bytes --]

Re: Recipe Updating Status and call to action

[Tian, Kevin]

>From: Saul Wold
>Sent: Thursday, December 16, 2010 8:58 AM
>
>Folks,
>
>As we get ready for M3 opening up shortly, I wanted to get folks aware
>of where we are with M2 and the recipe updating process.  Based on my
>rough reckoning, we had about 266 recipes that needed updating at the
>start of M2, we updated about 110 (possibly more).  There are now about
>160 recipes in the update list that need tackling.
>
>Also, the distro tracking data for Version information is getting stale,
>there are about 135 RECIPE_LATEST_VERSION metadata tags that need to be
>updated.
>
>I have attached a spreadsheet that shows the update status of both the
>upstream (RMatch Column) and tracking data (TMatch Column).
>
>I would like to see the team and community pitch in and complete these
>160 recipe updates for the 1.0 Release. Please be sure the ownership /
>RECIPE_MAINTAINER information is correct.
>

I'm kind of surprised when seeing this data, as I thought we have upgraded more
than 110 recipes. Then distro team gathered just now to understand whether there's
anything mismatch. The major point causing confusion is that the code generating
above sheet is stateless, which doesn't track historical data while there do have
many packages releasing new versions just in M2 window. This then moves some
packages we do upgrade into the '160' group. Basically we think those items with small 
version difference fall into this category. So it's not that bad as I originally thought. :-)

Ke is collecting some data and will reply later to clarify in 160 recieps:
	- what have been upgraded in M2 window already, and
	- what have not been touched at all

With that data we'll see how much work remains in M3 for this task.

On the other hand, along with this I realize that there's one area we need further
discuss. How often should we upgrade packages in a given release cycle? MeeGo 
only does once. For Yocto we want to keep our recipes in cutting-edge which is 
why we schedule two upgrade windows in M2 and M3 this time. The policy for each 
recipe however could be a bit different:

	- M2 is our 1st upgrade window, within which we try to upgrade as many 
recipes as possible to avoid rush push in last release point. The order is done by:
		* first upgrade small version changes in a batch which normally implicates 
those recipes have been touched in previous release cycle and thus small risk
		* then upgrade the rest recipes which may need more debug time 
because they may not been touched for some time or at all

	- M3 (now) is our 2nd upgrade window, within which we try to ensure our 
final release in cutting-edge. Then I suggest a reverse order:
		* first upgrade recipes which haven't been covered in M2, which implicates 
more work required and higher risk
		* then upgrade small version changes at end of the window. This way 
can ensure catching latest version or else we may need to do it multiple times if doing
it too early. This has small risk

So in generally I propose to have a longer window for recipes which have been 
upgraded and got a new version now. For now we can have distro team focusing on
those recipes which haven't been touched yet.

Does it sound OK?

----

BTW, I think our package report system (in experimental phase now) can help
the upgrade decision clearly. A suggested flow will be:

The server checks upstream version every day or every several days. When there's
a new version newly released, a mail will be sent to the recipe owner and CC this
list.

The owner then need react to the notification:
	- verify whether this is a valid message
	- decide whether a upgrade is required. The reason for non-upgrade could be:
		* pure SCM commit (for SCM check we may need more thinking)
		* an unstable release
		* known critical bugs/security holes
		* not worthy of upgrade effort since no new features are introduced
		* ...
	- mark decision on the package report system
		* to-be-upgraded
		* not upgrade (if so, reason). If we still use RECIPE_NO_UPDATE_REASON,
then a patch may be required to update that field
		
The package report system will record all the historical versions and decisions, to
avoid trigger unnecessary messages again.

When a upgrade window starts and ends, we can then use above stateful data to
draw overall picture and track our progress accurately.

Comments? 

Thanks
Kevin

Re: [poky] Recipe Updating Status and call to action

[Tian, Kevin]

>From: Saul Wold
>Sent: Thursday, December 16, 2010 8:58 AM
>
>Folks,
>
>As we get ready for M3 opening up shortly, I wanted to get folks aware
>of where we are with M2 and the recipe updating process.  Based on my
>rough reckoning, we had about 266 recipes that needed updating at the
>start of M2, we updated about 110 (possibly more).  There are now about
>160 recipes in the update list that need tackling.
>
>Also, the distro tracking data for Version information is getting stale,
>there are about 135 RECIPE_LATEST_VERSION metadata tags that need to be
>updated.
>
>I have attached a spreadsheet that shows the update status of both the
>upstream (RMatch Column) and tracking data (TMatch Column).
>
>I would like to see the team and community pitch in and complete these
>160 recipe updates for the 1.0 Release. Please be sure the ownership /
>RECIPE_MAINTAINER information is correct.
>

I'm kind of surprised when seeing this data, as I thought we have upgraded more
than 110 recipes. Then distro team gathered just now to understand whether there's
anything mismatch. The major point causing confusion is that the code generating
above sheet is stateless, which doesn't track historical data while there do have
many packages releasing new versions just in M2 window. This then moves some
packages we do upgrade into the '160' group. Basically we think those items with small 
version difference fall into this category. So it's not that bad as I originally thought. :-)

Ke is collecting some data and will reply later to clarify in 160 recieps:
	- what have been upgraded in M2 window already, and
	- what have not been touched at all

With that data we'll see how much work remains in M3 for this task.

On the other hand, along with this I realize that there's one area we need further
discuss. How often should we upgrade packages in a given release cycle? MeeGo 
only does once. For Yocto we want to keep our recipes in cutting-edge which is 
why we schedule two upgrade windows in M2 and M3 this time. The policy for each 
recipe however could be a bit different:

	- M2 is our 1st upgrade window, within which we try to upgrade as many 
recipes as possible to avoid rush push in last release point. The order is done by:
		* first upgrade small version changes in a batch which normally implicates 
those recipes have been touched in previous release cycle and thus small risk
		* then upgrade the rest recipes which may need more debug time 
because they may not been touched for some time or at all

	- M3 (now) is our 2nd upgrade window, within which we try to ensure our 
final release in cutting-edge. Then I suggest a reverse order:
		* first upgrade recipes which haven't been covered in M2, which implicates 
more work required and higher risk
		* then upgrade small version changes at end of the window. This way 
can ensure catching latest version or else we may need to do it multiple times if doing
it too early. This has small risk

So in generally I propose to have a longer window for recipes which have been 
upgraded and got a new version now. For now we can have distro team focusing on
those recipes which haven't been touched yet.

Does it sound OK?

----

BTW, I think our package report system (in experimental phase now) can help
the upgrade decision clearly. A suggested flow will be:

The server checks upstream version every day or every several days. When there's
a new version newly released, a mail will be sent to the recipe owner and CC this
list.

The owner then need react to the notification:
	- verify whether this is a valid message
	- decide whether a upgrade is required. The reason for non-upgrade could be:
		* pure SCM commit (for SCM check we may need more thinking)
		* an unstable release
		* known critical bugs/security holes
		* not worthy of upgrade effort since no new features are introduced
		* ...
	- mark decision on the package report system
		* to-be-upgraded
		* not upgrade (if so, reason). If we still use RECIPE_NO_UPDATE_REASON,
then a patch may be required to update that field
		
The package report system will record all the historical versions and decisions, to
avoid trigger unnecessary messages again.

When a upgrade window starts and ends, we can then use above stateful data to
draw overall picture and track our progress accurately.

Comments? 

Thanks
Kevin

Re: [poky] Recipe Updating Status and call to action

[Yu, Ke]

Thanks Saul for working out this list. 

I am a bit surprised on the large number of to-be-upgraded list: 160, because according to our plan, we actually upgrade most of the recipes in M2, and only a small number of recipes are supposed to be left for M3. To find out the reason why, PRC team collect some data to analysis, and here is the result:

Among the 160, PRC team (Dexuan, Dongxiao, Edwin, Ke, Qing) owns 105 recipes. the 105 recipes can be grouped into 3 cases:
Case A: the recipe is not planned to upgrade in M2, mostly due to that it is up-to-update in M2 planning time, and later has new version available.
Case B: the recipe is planned to upgrade in M2, but not upgrade and leave it to M3
Case C: the recipe is planned to upgrade in M2, and it is upgraded in M2, but later has new version available.

According to our data collection, Case A has 31 recipes, Case B has 21 recipes, Case C has 53 recipes. In other word, in M2 time, we leave 21 recipes (Case B) for M3, but due to new version available after the planning, there are extra 84 recipes (case A + case C) added to be upgraded. Even only count Case C, we can see almost 50% ofM3  to-be-upgraded recipes are already upgraded in M2. This is probably one area we can improved in the future.

Regards
Ke

> -----Original Message-----
> From: poky-bounces@yoctoproject.org [mailto:poky-
> bounces@yoctoproject.org] On Behalf Of Saul Wold
> Sent: Thursday, December 16, 2010 8:58 AM
> To: Yocto Project Discussion; poky@yoctoproject.org
> Subject: [poky] Recipe Updating Status and call to action
> 
> 
> Folks,
> 
> As we get ready for M3 opening up shortly, I wanted to get folks aware of where
> we are with M2 and the recipe updating process.  Based on my rough reckoning,
> we had about 266 recipes that needed updating at the start of M2, we updated
> about 110 (possibly more).  There are now about 160 recipes in the update list
> that need tackling.
> 
> Also, the distro tracking data for Version information is getting stale, there are
> about 135 RECIPE_LATEST_VERSION metadata tags that need to be updated.
> 
> I have attached a spreadsheet that shows the update status of both the
> upstream (RMatch Column) and tracking data (TMatch Column).
> 
> I would like to see the team and community pitch in and complete these 160
> recipe updates for the 1.0 Release. Please be sure the ownership /
> RECIPE_MAINTAINER information is correct.
> 
> IMPORTANT: Please use distro/stage in poky-contrib to update the
> distro_tracking_fields.inc file, I have many updates in that already, I will do my
> best not to rebase that until we get it updated.
> 
> The attached spread sheet contains 3 Sheets, please review the 2nd and 3rd
> sheets (Labeled Tracking Updates and Recipe Updates respectively).
> 
> Thanks.
> 
> --
>      Sau!
> 
> Saul Wold
> Yocto Component Wrangler @ Intel
> Yocto Project / Poky Build System

Re: Recipe Updating Status and call to action

[Scott Garman]

On 12/15/2010 06:40 PM, Tian, Kevin wrote:
> On the other hand, along with this I realize that there's one area we need further
> discuss. How often should we upgrade packages in a given release cycle? MeeGo
> only does once. For Yocto we want to keep our recipes in cutting-edge which is
> why we schedule two upgrade windows in M2 and M3 this time.

I'd like to question this. Is the goal for Poky/Yocto to track the 
bleeding-edge releases of software, or is the goal to be a well-tested 
and stable foundation for embedded software applications?

Upgrading a recipe within a couple of weeks of its release may end up 
using more of our resources if/when we encounter new bugs that were 
introduced in the new release. Or worse, if we don't encounter them 
during distro builds and then our users take our release and discover 
them for themselves.

I'm not saying we need to be as conservative as long-term-support 
enterprise Linux distros, but IMHO I think racing to always upgrade our 
recipes to versions released a handful of weeks ago can be 
counterproductive in many situations.

A policy I might put forward for consideration is this: recipe upgrades 
are done once per release cycle, and upstream versions that have come 
out within the last 30 days should not be upgraded unless we have a 
really good reason for doing so.

Scott

-- 
Scott Garman
Embedded Linux Distro Engineer - Yocto Project

Re: [poky] Recipe Updating Status and call to action

[Scott Garman]

On 12/15/2010 06:40 PM, Tian, Kevin wrote:
> On the other hand, along with this I realize that there's one area we need further
> discuss. How often should we upgrade packages in a given release cycle? MeeGo
> only does once. For Yocto we want to keep our recipes in cutting-edge which is
> why we schedule two upgrade windows in M2 and M3 this time.

I'd like to question this. Is the goal for Poky/Yocto to track the 
bleeding-edge releases of software, or is the goal to be a well-tested 
and stable foundation for embedded software applications?

Upgrading a recipe within a couple of weeks of its release may end up 
using more of our resources if/when we encounter new bugs that were 
introduced in the new release. Or worse, if we don't encounter them 
during distro builds and then our users take our release and discover 
them for themselves.

I'm not saying we need to be as conservative as long-term-support 
enterprise Linux distros, but IMHO I think racing to always upgrade our 
recipes to versions released a handful of weeks ago can be 
counterproductive in many situations.

A policy I might put forward for consideration is this: recipe upgrades 
are done once per release cycle, and upstream versions that have come 
out within the last 30 days should not be upgraded unless we have a 
really good reason for doing so.

Scott

-- 
Scott Garman
Embedded Linux Distro Engineer - Yocto Project

Re: [poky] Recipe Updating Status and call to action

[Saul Wold]

On 12/16/2010 06:17 AM, Yu, Ke wrote:
> Thanks Saul for working out this list.
>
> I am a bit surprised on the large number of to-be-upgraded list: 160, because according to our plan, we actually upgrade most of the recipes in M2, and only a small number of recipes are supposed to be left for M3. To find out the reason why, PRC team collect some data to analysis, and here is the result:
>
I am sorry I gave everyone a scare with my numbers, as Kevin points out, 
currently this information is stateless, I am going my best to gather 
the information that I have at hand

> Among the 160, PRC team (Dexuan, Dongxiao, Edwin, Ke, Qing) owns 105 recipes. the 105 recipes can be grouped into 3 cases:
> Case A: the recipe is not planned to upgrade in M2, mostly due to that it is up-to-update in M2 planning time, and later has new version available.
> Case B: the recipe is planned to upgrade in M2, but not upgrade and leave it to M3
> Case C: the recipe is planned to upgrade in M2, and it is upgraded in M2, but later has new version available.
Can you send me a list of the Case C recipes, I would like to review if 
we need to update all of them.  As I said in the past some of these may 
be the Xserver and libraries which we do want to update since we went to 
an RC release first.

Sau!


>
> According to our data collection, Case A has 31 recipes, Case B has 21 recipes, Case C has 53 recipes. In other word, in M2 time, we leave 21 recipes (Case B) for M3, but due to new version available after the planning, there are extra 84 recipes (case A + case C) added to be upgraded. Even only count Case C, we can see almost 50% ofM3  to-be-upgraded recipes are already upgraded in M2. This is probably one area we can improved in the future.
>
Thanks

> Regards
> Ke
>
>> -----Original Message-----
>> From: poky-bounces@yoctoproject.org [mailto:poky-
>> bounces@yoctoproject.org] On Behalf Of Saul Wold
>> Sent: Thursday, December 16, 2010 8:58 AM
>> To: Yocto Project Discussion; poky@yoctoproject.org
>> Subject: [poky] Recipe Updating Status and call to action
>>
>>
>> Folks,
>>
>> As we get ready for M3 opening up shortly, I wanted to get folks aware of where
>> we are with M2 and the recipe updating process.  Based on my rough reckoning,
>> we had about 266 recipes that needed updating at the start of M2, we updated
>> about 110 (possibly more).  There are now about 160 recipes in the update list
>> that need tackling.
>>
>> Also, the distro tracking data for Version information is getting stale, there are
>> about 135 RECIPE_LATEST_VERSION metadata tags that need to be updated.
>>
>> I have attached a spreadsheet that shows the update status of both the
>> upstream (RMatch Column) and tracking data (TMatch Column).
>>
>> I would like to see the team and community pitch in and complete these 160
>> recipe updates for the 1.0 Release. Please be sure the ownership /
>> RECIPE_MAINTAINER information is correct.
>>
>> IMPORTANT: Please use distro/stage in poky-contrib to update the
>> distro_tracking_fields.inc file, I have many updates in that already, I will do my
>> best not to rebase that until we get it updated.
>>
>> The attached spread sheet contains 3 Sheets, please review the 2nd and 3rd
>> sheets (Labeled Tracking Updates and Recipe Updates respectively).
>>
>> Thanks.
>>
>> --
>>       Sau!
>>
>> Saul Wold
>> Yocto Component Wrangler @ Intel
>> Yocto Project / Poky Build System
>
>

Re: Recipe Updating Status and call to action

[Tian, Kevin]

>From: Garman, Scott A
>Sent: Friday, December 17, 2010 5:57 AM
>
>On 12/15/2010 06:40 PM, Tian, Kevin wrote:
>> On the other hand, along with this I realize that there's one area we need further
>> discuss. How often should we upgrade packages in a given release cycle? MeeGo
>> only does once. For Yocto we want to keep our recipes in cutting-edge which is
>> why we schedule two upgrade windows in M2 and M3 this time.
>
>I'd like to question this. Is the goal for Poky/Yocto to track the
>bleeding-edge releases of software, or is the goal to be a well-tested
>and stable foundation for embedded software applications?

I think the both. :-)

>
>Upgrading a recipe within a couple of weeks of its release may end up
>using more of our resources if/when we encounter new bugs that were
>introduced in the new release. Or worse, if we don't encounter them
>during distro builds and then our users take our release and discover
>them for themselves.

This is always being a tradeoff in all distributions. There's no simple guideline
whether there's severe risk to upgrade to a latest release, and I think the
one general rule is to pick up a newer release with enough stability. How to
judge that? This finally goes to the owner of each recipe.

>
>I'm not saying we need to be as conservative as long-term-support
>enterprise Linux distros, but IMHO I think racing to always upgrade our
>recipes to versions released a handful of weeks ago can be
>counterproductive in many situations.
>
>A policy I might put forward for consideration is this: recipe upgrades
>are done once per release cycle, and upstream versions that have come
>out within the last 30 days should not be upgraded unless we have a
>really good reason for doing so.
>

I think there's no simple guideline meaningful in general context like this. All the
decisions should come to the actual recipe maintainer, based on his knowledge
in this specific area. Once we involve with upstream more closely, we should be
able to tell whether a new release is worthy to go or not in most cases.

Besides that, what in general we can do is about the process.

That's why we come up a two phase upgrade windows in 1.0. With the first window
as the major upgrade, and the 2nd window to catch up minor version changes which
should be with little risk. If there's important release coming out in 2nd window, we
(again mainly the recipe owner) again need to make decision instead of a simple
global rule "to go" or "not to go". As I raised in early thread, I suggest to those 
minor version upgrade in later of 2nd window.

That's also why we're currently developing the package reporting system, which is
expected to check upstream release periodically and once a new release coming
out the maintainer gets notified to make decision.

So in a nutshell:
	- the general goal is to being cutting-edge
	- we work out a process which give recipe maintainers enough opportunities
to check new releases of packages they own, toward our 'cutting-edge' goal
	- Saul generates a general candidate list from those info in start of each 
upgrade window
	- then it's always recipe maintainers to make decision whether a new release 
should go or not, based on its stability, new features and risks given time to Yocto
release point. If there's a decision not to upgrade, the maintainer should document
it well

Having said that so far our process is not exactly as expected yet, and our expertise
on each area still need time. But I think that's the what we'll need to be. :-)

Thanks
Kevin

Re: [poky] Recipe Updating Status and call to action

[Tian, Kevin]

>From: Garman, Scott A
>Sent: Friday, December 17, 2010 5:57 AM
>
>On 12/15/2010 06:40 PM, Tian, Kevin wrote:
>> On the other hand, along with this I realize that there's one area we need further
>> discuss. How often should we upgrade packages in a given release cycle? MeeGo
>> only does once. For Yocto we want to keep our recipes in cutting-edge which is
>> why we schedule two upgrade windows in M2 and M3 this time.
>
>I'd like to question this. Is the goal for Poky/Yocto to track the
>bleeding-edge releases of software, or is the goal to be a well-tested
>and stable foundation for embedded software applications?

I think the both. :-)

>
>Upgrading a recipe within a couple of weeks of its release may end up
>using more of our resources if/when we encounter new bugs that were
>introduced in the new release. Or worse, if we don't encounter them
>during distro builds and then our users take our release and discover
>them for themselves.

This is always being a tradeoff in all distributions. There's no simple guideline
whether there's severe risk to upgrade to a latest release, and I think the
one general rule is to pick up a newer release with enough stability. How to
judge that? This finally goes to the owner of each recipe.

>
>I'm not saying we need to be as conservative as long-term-support
>enterprise Linux distros, but IMHO I think racing to always upgrade our
>recipes to versions released a handful of weeks ago can be
>counterproductive in many situations.
>
>A policy I might put forward for consideration is this: recipe upgrades
>are done once per release cycle, and upstream versions that have come
>out within the last 30 days should not be upgraded unless we have a
>really good reason for doing so.
>

I think there's no simple guideline meaningful in general context like this. All the
decisions should come to the actual recipe maintainer, based on his knowledge
in this specific area. Once we involve with upstream more closely, we should be
able to tell whether a new release is worthy to go or not in most cases.

Besides that, what in general we can do is about the process.

That's why we come up a two phase upgrade windows in 1.0. With the first window
as the major upgrade, and the 2nd window to catch up minor version changes which
should be with little risk. If there's important release coming out in 2nd window, we
(again mainly the recipe owner) again need to make decision instead of a simple
global rule "to go" or "not to go". As I raised in early thread, I suggest to those 
minor version upgrade in later of 2nd window.

That's also why we're currently developing the package reporting system, which is
expected to check upstream release periodically and once a new release coming
out the maintainer gets notified to make decision.

So in a nutshell:
	- the general goal is to being cutting-edge
	- we work out a process which give recipe maintainers enough opportunities
to check new releases of packages they own, toward our 'cutting-edge' goal
	- Saul generates a general candidate list from those info in start of each 
upgrade window
	- then it's always recipe maintainers to make decision whether a new release 
should go or not, based on its stability, new features and risks given time to Yocto
release point. If there's a decision not to upgrade, the maintainer should document
it well

Having said that so far our process is not exactly as expected yet, and our expertise
on each area still need time. But I think that's the what we'll need to be. :-)

Thanks
Kevin

Re: [poky] Recipe Updating Status and call to action

[Yu, Ke]

[-- Attachment #1: Type: text/plain, Size: 578 bytes --]

> -----Original Message-----
> From: Saul Wold [mailto:sgw@linux.intel.com]
> Sent: Friday, December 17, 2010 7:19 AM
> To: Yu, Ke
> Cc: Yocto Project Discussion
> Subject: Re: [poky] Recipe Updating Status and call to action
> 
> Can you send me a list of the Case C recipes, I would like to review if
> we need to update all of them.  As I said in the past some of these may
> be the Xserver and libraries which we do want to update since we went to
> an RC release first.
> 
> Sau!
> 

Sure. Please find the attached file for the list. Thanks.

Regards
Ke

[-- Attachment #2: m3.txt --]
[-- Type: text/plain, Size: 634 bytes --]

netbase
mtd-utils
pulseaudio
gstreamer
gst-plugins-base
libogg
ofono
connman
gst-plugins-good
bluez4
glib-2.0
telepathy-idle
telepathy-mission-control
telepathy-glib
libgsmd
Tremor
xproto
lttng-viewer
liburcu
ltt-control
lttng-ust
scrnsaverproto
liberation-fonts
pixman
librsvg
libgpg-error 
puzzles
libsoup-2.4
gnome-desktop
gtk-engines
gnome-keyring
gtk+ 
apr 
apr-util 
webkit-gtk 
eds-dbus
shared-mime-info
Libtasn1
Libxml2
Boost
Util-linux
libzypp
zypper
sat-sover
update-rc.d
yaffs2-utils
xf86-video-omapfb
xf86-video-intel
xf86-input-evdev
xf86-input-vmmouse
xwininfo
qemugl