OE-Core release process

OE-Core release process

[Richard Purdie]

Its been mentioned to me that we perhaps haven't made it really clear
what is happening with OE-Core at the moment. When we started this off,
we agreed to try and get on a roughly six month release cadence so
people had some idea of what to expect from the project and when. We
decided to put that in sync with what Yocto was doing since it was felt
OE-Core would equally benefit from the QA and bug fixing Yocto does as
part of its release process.

I did mention we be doing slow down of features, a calming down of
changes, then branching and releasing off the branch. Judging by
people's patch submissions, that seemed to be happening fairly
effectively.

Unfortunately we hit some infrastructure issues to the issues with the
kernel.org servers which in turn affected the Linux Foundation (who my
email goes through) and the Yocto project (who were previously on the
same router). This meant we didn't get test results when I'd have
ideally liked them or been able to branch when I wanted to. Its also
hampered some of the communication which otherwise might have been
clearer.

I think the tree is stable enough to branch now and the position is what
it is, we're just going to have to make the best of it. I'd really like
to release OE-Core and Yocto based on the same revisions. Yocto now has
a -rc2 build with timeslots planned for an -rc3 with -rc1 effectively
just having been build tested. The only way to fit a -rc4 in is to slip
release dates and due to holidays in China where the QA team is based,
that would mean a big slip.

The big problem we now face is that the more change we put into -rc3,
the higher the risk of either build failure regressions or QA
regressions which would mean the release slips.

I'm going to branch the tree today and will be carefully considering
what goes into the release branch.

There will be changes accepted on this branch going forward and there is
the possibility of subsequent point releases although we likely need to
clearly document what is acceptable change there. My quick summary would
be no feature enhancements, dot.dot point release version changes only
for bug fixes and generally bug/regression fix only along with any fixes
needed to build on newer distros. 

Also not that if for any reason this isn't good enough for anyone's
needs, there is freedom to create other "stable" branches based on this
(or otherwise) and if that proved to be more popular/useful, that would
directly influence what we'd do in OE-Core. The best branch would
win :).

Cheers,

Richard

Re: OE-Core release process

[Otavio Salvador]

On Fri, Sep 16, 2011 at 09:54, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> Its been mentioned to me that we perhaps haven't made it really clear
> what is happening with OE-Core at the moment. When we started this off,
> we agreed to try and get on a roughly six month release cadence so
> people had some idea of what to expect from the project and when. We
> decided to put that in sync with what Yocto was doing since it was felt
> OE-Core would equally benefit from the QA and bug fixing Yocto does as
> part of its release process.

I fully agree on that and I do think this will gonna be good for both
projects; what I dislike is that Yocto people keep using Poky to their
development instead of OE-Core and this it is quite boring to share
patches between both until you have  cherry-pick them.

This should be change in my point of view and people ought to use
oe-core for hacking so we all share same base tree.

My 2c.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: OE-Core release process

[Richard Purdie]

On Fri, 2011-09-16 at 10:29 -0300, Otavio Salvador wrote:
> On Fri, Sep 16, 2011 at 09:54, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > Its been mentioned to me that we perhaps haven't made it really clear
> > what is happening with OE-Core at the moment. When we started this off,
> > we agreed to try and get on a roughly six month release cadence so
> > people had some idea of what to expect from the project and when. We
> > decided to put that in sync with what Yocto was doing since it was felt
> > OE-Core would equally benefit from the QA and bug fixing Yocto does as
> > part of its release process.
> 
> I fully agree on that and I do think this will gonna be good for both
> projects; what I dislike is that Yocto people keep using Poky to their
> development instead of OE-Core and this it is quite boring to share
> patches between both until you have  cherry-pick them.
> 
> This should be change in my point of view and people ought to use
> oe-core for hacking so we all share same base tree.

The shr people keep using their shr scripts and tree and angstrom do the
same to pick two examples. No, we're not all going to use OE-Core alone,
that was never the point. So why should we single out poky either? Ok, a
lot of patches are coming from there but that's good, right? :)

I've yet to refuse a bug report on the grounds of "not OE-Core alone"
and don't plan to start :)

The patches do translate easily, if they didn't I'd be going (more)
insane!

Cheers,

Richard

Re: OE-Core release process

[Otavio Salvador]

On Fri, Sep 16, 2011 at 13:51, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Fri, 2011-09-16 at 10:29 -0300, Otavio Salvador wrote:
>> On Fri, Sep 16, 2011 at 09:54, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>> > Its been mentioned to me that we perhaps haven't made it really clear
>> > what is happening with OE-Core at the moment. When we started this off,
>> > we agreed to try and get on a roughly six month release cadence so
>> > people had some idea of what to expect from the project and when. We
>> > decided to put that in sync with what Yocto was doing since it was felt
>> > OE-Core would equally benefit from the QA and bug fixing Yocto does as
>> > part of its release process.
>>
>> I fully agree on that and I do think this will gonna be good for both
>> projects; what I dislike is that Yocto people keep using Poky to their
>> development instead of OE-Core and this it is quite boring to share
>> patches between both until you have  cherry-pick them.
>>
>> This should be change in my point of view and people ought to use
>> oe-core for hacking so we all share same base tree.
>
> The shr people keep using their shr scripts and tree and angstrom do the
> same to pick two examples. No, we're not all going to use OE-Core alone,
> that was never the point. So why should we single out poky either? Ok, a
> lot of patches are coming from there but that's good, right? :)
>
> I've yet to refuse a bug report on the grounds of "not OE-Core alone"
> and don't plan to start :)

I disagree on that.

I see shr and angstrom as not the same to Yocto mostly because it
takes, or is suppose to take, oe-core as is and merge it with bitbake,
meta-yocto and be done with it. In this case it seems logical to the
patches go directly to oe-core and Yocto merge from it avoiding a lot
of hassle for you and making contribution easy.

More then once I wanted to pick a patch that was in poky and not yet
merged into oe-core and it wasn't that easy to split it.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: OE-Core release process

[Paul Eggleton]

On Friday 16 September 2011 18:08:04 Otavio Salvador wrote:
> More then once I wanted to pick a patch that was in poky and not yet
> merged into oe-core and it wasn't that easy to split it.

Can you give a specific example of this?

Since the split, there have never been any patches intentionally merged into 
poky's OE-core parts that were not already in OE-core. (There were a few 
isolated occasions where some accidentally slipped in, these were rectified as 
soon as they were discovered.)

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: OE-Core release process

[Otavio Salvador]

On Fri, Sep 16, 2011 at 14:16, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> On Friday 16 September 2011 18:08:04 Otavio Salvador wrote:
>> More then once I wanted to pick a patch that was in poky and not yet
>> merged into oe-core and it wasn't that easy to split it.
>
> Can you give a specific example of this?
>
> Since the split, there have never been any patches intentionally merged into
> poky's OE-core parts that were not already in OE-core. (There were a few
> isolated occasions where some accidentally slipped in, these were rectified as
> soon as they were discovered.)

They all ended up in oe-core the problem is when they're not yet
merged but I want to merge them into my tree for use, test or
anything.

The combo-layer was one example. It was made available on poky contrib
tree and I had to manually export the patches instead of pulling it
into a branch for testing.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: OE-Core release process

[Paul Eggleton]

On Friday 16 September 2011 18:18:43 you wrote:
> On Fri, Sep 16, 2011 at 14:16, Paul Eggleton
> 
> <paul.eggleton@linux.intel.com> wrote:
> > On Friday 16 September 2011 18:08:04 Otavio Salvador wrote:
> >> More then once I wanted to pick a patch that was in poky and not yet
> >> merged into oe-core and it wasn't that easy to split it.
> > 
> > Can you give a specific example of this?
> > 
> > Since the split, there have never been any patches intentionally merged
> > into poky's OE-core parts that were not already in OE-core. (There were
> > a few isolated occasions where some accidentally slipped in, these were
> > rectified as soon as they were discovered.)
> 
> They all ended up in oe-core the problem is when they're not yet
> merged but I want to merge them into my tree for use, test or
> anything.

Ah, so what you mean is not that the patches were in poky, but that there 
sometimes were pull requests pointing to a branch that was on top of poky. The 
thing is, the patches are exactly the same.
 
> The combo-layer was one example. It was made available on poky contrib
> tree and I had to manually export the patches instead of pulling it
> into a branch for testing.

Git makes this almost trivially easy though, and in the combo-layer example it 
was only a few patches.

Personally, I always post my OE-core pull requests on top of an OE-core 
branch; doing that means *I* have to manually bring the patches across as files 
and use git-am to apply them onto a branch of OE-core though. I think those of 
us working on Poky know this is the accepted practice and try to do it all of 
the time, occasionally someone might forget or feel that for an RFC patch it's 
not worth the effort; but I think that's the exception rather than the rule.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: OE-Core release process

[Otavio Salvador]

On Fri, Sep 16, 2011 at 14:49, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> On Friday 16 September 2011 18:18:43 you wrote:
>> They all ended up in oe-core the problem is when they're not yet
>> merged but I want to merge them into my tree for use, test or
>> anything.
>
> Ah, so what you mean is not that the patches were in poky, but that there
> sometimes were pull requests pointing to a branch that was on top of poky. The
> thing is, the patches are exactly the same.

Yes but not pullable directly into oe-core tree.

>> The combo-layer was one example. It was made available on poky contrib
>> tree and I had to manually export the patches instead of pulling it
>> into a branch for testing.
>
> Git makes this almost trivially easy though, and in the combo-layer example it
> was only a few patches.
>
> Personally, I always post my OE-core pull requests on top of an OE-core
> branch; doing that means *I* have to manually bring the patches across as files
> and use git-am to apply them onto a branch of OE-core though. I think those of
> us working on Poky know this is the accepted practice and try to do it all of
> the time, occasionally someone might forget or feel that for an RFC patch it's
> not worth the effort; but I think that's the exception rather than the rule.

Yes I know but wouldn't be better if you could base your work on
oe-core and avoid doing this?

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: OE-Core release process

[Mark Hatle]

On 9/16/11 2:06 PM, Otavio Salvador wrote:
> On Fri, Sep 16, 2011 at 14:49, Paul Eggleton
> <paul.eggleton@linux.intel.com> wrote:
>> On Friday 16 September 2011 18:18:43 you wrote:
>>> They all ended up in oe-core the problem is when they're not yet
>>> merged but I want to merge them into my tree for use, test or
>>> anything.
>>
>> Ah, so what you mean is not that the patches were in poky, but that there
>> sometimes were pull requests pointing to a branch that was on top of poky. The
>> thing is, the patches are exactly the same.
> 
> Yes but not pullable directly into oe-core tree.

I mix and match for all of my development.  What I do is use the git remote and
include multiple remotes.. poky, github, etc..

When I find patches I want I cherry-pick them into my tree.

Your mileage may vary, works for me at least..

>>> The combo-layer was one example. It was made available on poky contrib
>>> tree and I had to manually export the patches instead of pulling it
>>> into a branch for testing.
>>
>> Git makes this almost trivially easy though, and in the combo-layer example it
>> was only a few patches.
>>
>> Personally, I always post my OE-core pull requests on top of an OE-core
>> branch; doing that means *I* have to manually bring the patches across as files
>> and use git-am to apply them onto a branch of OE-core though. I think those of
>> us working on Poky know this is the accepted practice and try to do it all of
>> the time, occasionally someone might forget or feel that for an RFC patch it's
>> not worth the effort; but I think that's the exception rather than the rule.
> 
> Yes I know but wouldn't be better if you could base your work on
> oe-core and avoid doing this?

For submitting to oe-core for pull requests, basing work off of oe-core is "best
practice".  However, forcing people who are working on a specific distribution
to do this isn't reasonable.  It can be requested of them, but IMHO thats as far
as it will go.

I'm not going to stop anyone from developing in the way that suits them best,
all I can request is the interchange to me be at least reasonable.  (git and
cherry-pick is "reasonable" to me at least.)

Note, I do about 70% development in oe-core and 30% in Poky.  Depends where the
bugs are found and what it takes to reproduce them.  I always cherry-pick/rebase
as appropriate to send pull requests to the appropriate place, but my
work-in-progress tree is whatever works for me at the time.  I suspect others
are like that as well.

--Mark

Re: OE-Core release process

[Otavio Salvador]

On Fri, Sep 16, 2011 at 16:45, Mark Hatle <mark.hatle@windriver.com> wrote:
> On 9/16/11 2:06 PM, Otavio Salvador wrote:
>> On Fri, Sep 16, 2011 at 14:49, Paul Eggleton
>> <paul.eggleton@linux.intel.com> wrote:
>>> On Friday 16 September 2011 18:18:43 you wrote:
>>>> They all ended up in oe-core the problem is when they're not yet
>>>> merged but I want to merge them into my tree for use, test or
>>>> anything.
>>>
>>> Ah, so what you mean is not that the patches were in poky, but that there
>>> sometimes were pull requests pointing to a branch that was on top of poky. The
>>> thing is, the patches are exactly the same.
>>
>> Yes but not pullable directly into oe-core tree.
>
> I mix and match for all of my development.  What I do is use the git remote and
> include multiple remotes.. poky, github, etc..
>
> When I find patches I want I cherry-pick them into my tree.
>
> Your mileage may vary, works for me at least..

Yes, works but seems easier to track one only and have it
automatically merge into the Yocto one.

This is specifically the idea behind the combo-layer creation and
would make life much easier to all people working at oe-core.

>>>> The combo-layer was one example. It was made available on poky contrib
>>>> tree and I had to manually export the patches instead of pulling it
>>>> into a branch for testing.
>>>
>>> Git makes this almost trivially easy though, and in the combo-layer example it
>>> was only a few patches.
>>>
>>> Personally, I always post my OE-core pull requests on top of an OE-core
>>> branch; doing that means *I* have to manually bring the patches across as files
>>> and use git-am to apply them onto a branch of OE-core though. I think those of
>>> us working on Poky know this is the accepted practice and try to do it all of
>>> the time, occasionally someone might forget or feel that for an RFC patch it's
>>> not worth the effort; but I think that's the exception rather than the rule.
>>
>> Yes I know but wouldn't be better if you could base your work on
>> oe-core and avoid doing this?
>
> For submitting to oe-core for pull requests, basing work off of oe-core is "best
> practice".  However, forcing people who are working on a specific distribution
> to do this isn't reasonable.  It can be requested of them, but IMHO thats as far
> as it will go.

For sporadic work I fully agree but for people working daily and that
sends pull-request very ofthenly seems sensible that the suggested
workflow to be done this way.

> I'm not going to stop anyone from developing in the way that suits them best,
> all I can request is the interchange to me be at least reasonable.  (git and
> cherry-pick is "reasonable" to me at least.)

Sure; i am not saying to us to block anything but at least suggest the
best workflow.

> Note, I do about 70% development in oe-core and 30% in Poky.  Depends where the
> bugs are found and what it takes to reproduce them.  I always cherry-pick/rebase
> as appropriate to send pull requests to the appropriate place, but my
> work-in-progress tree is whatever works for me at the time.  I suspect others
> are like that as well.

If oe-core is kept in sync, use oe-core just makes things easier from
my point of view.

As I said the idea is not force something but advice to make it easier
to everyone.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: OE-Core release process

[Chris Larson]

On Fri, Sep 16, 2011 at 12:45 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
>> Yes I know but wouldn't be better if you could base your work on
>> oe-core and avoid doing this?
>
> For submitting to oe-core for pull requests, basing work off of oe-core is "best
> practice".  However, forcing people who are working on a specific distribution
> to do this isn't reasonable.  It can be requested of them, but IMHO thats as far
> as it will go.
>

I don't see why it isn't reasonable. If they need a change done sooner
than can be done by pushing it upstream, they can put it into a local
layer until it makes it into oe-core. Seems simple enough to me.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: OE-Core release process

[Koen Kooi]

Op 17 sep. 2011, om 04:55 heeft Chris Larson het volgende geschreven:

> On Fri, Sep 16, 2011 at 12:45 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
>>> Yes I know but wouldn't be better if you could base your work on
>>> oe-core and avoid doing this?
>> 
>> For submitting to oe-core for pull requests, basing work off of oe-core is "best
>> practice".  However, forcing people who are working on a specific distribution
>> to do this isn't reasonable.  It can be requested of them, but IMHO thats as far
>> as it will go.
>> 
> 
> I don't see why it isn't reasonable. If they need a change done sooner
> than can be done by pushing it upstream, they can put it into a local
> layer until it makes it into oe-core. Seems simple enough to me.

Furthermore, poky is the only distro using this weird combo-layer arrangement, all the others I know off (angstrom, shr, micro, aurora) consume oe-core directly.

Re: OE-Core release process

[Otavio Salvador]

On Sat, Sep 17, 2011 at 06:06, Koen Kooi <koen@dominion.thruhere.net> wrote:
>> I don't see why it isn't reasonable. If they need a change done sooner
>> than can be done by pushing it upstream, they can put it into a local
>> layer until it makes it into oe-core. Seems simple enough to me.
>
> Furthermore, poky is the only distro using this weird combo-layer arrangement, all the others I know off (angstrom, shr, micro, aurora) consume oe-core directly.

At O.S. Systems we use combo-layer but we work at oe-core directly and
keep our repository up to date. That bring us the good of both Worlds
without making difficult to us to sync both.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br