openmoko-merges

openmoko-merges

[John Lee]

Hi,

Please review john_lee/openmoko-merges.  I started from
holger/openmoko-merges then rebased it to oe.dev.

The general principle is:

1) Minimal impact: most of the big changes apply to om distro only.
My thought is we merge this first, then apply to the general.

2) Keep history: I did it the hard way by cherry-picking one by one
since the history is different after the mtn->git switch.  Most of the
commit messages are kept, but that creates a lot of commits to review.

3) PKG_TAGS, qtopia, toolchain, om-utils and opkg stuffs are left out
for later merge.

Not sure how long it's going to take.  My plan is to merge it this
Monday if it's okay...


Regards,
John

Re: openmoko-merges

[Koen Kooi]

On 18-12-08 08:19, John Lee wrote:
> Hi,
>
> Please review john_lee/openmoko-merges.  I started from
> holger/openmoko-merges then rebased it to oe.dev.

> Not sure how long it's going to take.  My plan is to merge it this
> Monday if it's okay...

No. There's just too much in there too review in a week. Also, the stuff 
I did review is just waaaaay to ugly to go in.
Saying stuff will get fixed after the merge..... I've heard that before, 
no-one actually fixes things after a merge.

Most of these commits have two problems:

* They can be done in a much better, but still OM specific way (e.g. 
FILESPATH)
* They can be done in a generic way (e.g. alsa-states)


To give some small examples:

* 
http://cgit.openembedded.net/cgit.cgi?url=openembedded/commit/&h=john_lee/openmoko-merges&id=c0041ba98c41b320a36cbc5a66d0c8762e52ac62

--- a/packages/alsa/alsa-state.bb
+++ b/packages/alsa/alsa-state.bb
@@ -20,6 +20,7 @@ inherit update-rc.d

INITSCRIPT_PARAMS = "defaults 10"
+INITSCRIPT_PARAMS_openmoko = "start 39 S ."

Distro overrides are strongly discouraged and in this case not needed, 
since it's using 'defaults', so an optimization is welcomed. In a 
similar vein, the change to the initscript can be done to the regular 
one, no need to add a special openmoko version.

* 
http://cgit.openembedded.net/cgit.cgi?url=openembedded/commit/&h=john_lee/openmoko-merges&id=64b844ec5cc0c1a6acb1a97d1ea745f15eadb38d
this it what the device table feature is for, see org.oe/dev/files/ for 
the device tables, machines can have multiple ones if needed.

* 
http://cgit.openembedded.net/cgit.cgi?url=openembedded/commit/&h=john_lee/openmoko-merges&id=db77b062d51be86b946a11efcadaf8e41fd41727

If you modify FILESPATH you can reuse the initscripts from the 
initscripts packages so you don't need to copy over unchanged ones. 
Having said that, I think this should all be merged into the initscripts 
recipe so everyone can benefit from it.

I suspect review and merging would be easier if you 'collapse' the 
patches per recipe or per directory. Reviewing all the seperate commits 
will take too much time :(

Koen

Re: openmoko-merges

[John Lee]

On Thu, Dec 18, 2008 at 10:02:53AM +0100, Koen Kooi wrote:
> On 18-12-08 08:19, John Lee wrote:
>> Hi,
>>
>> Please review john_lee/openmoko-merges.  I started from
>> holger/openmoko-merges then rebased it to oe.dev.
>
>> Not sure how long it's going to take.  My plan is to merge it this
>> Monday if it's okay...
>
> No. There's just too much in there too review in a week. Also, the stuff  
> I did review is just waaaaay to ugly to go in.
> Saying stuff will get fixed after the merge..... I've heard that before,  
> no-one actually fixes things after a merge.

I can put fastboot commits to another branch and remove them here if
these are controversial.

As mentioned before, what's in /etc/init.d/ , rcS and different
runlevels varies from distro to distro, machine to machine...  Would
it be better if I remove as many overrides as possible, we merge it to
oe.dev and fix problems there?  I don't think I can test all these
combinations alone.

> Most of these commits have two problems:
>
> * They can be done in a much better, but still OM specific way (e.g.  
> FILESPATH)

Okay, I think these should really be fixed.  You mean just the
initscript-openmoko commit, or there are more ?

> * They can be done in a generic way (e.g. alsa-states)
>
> To give some small examples:
>
> *  
> http://cgit.openembedded.net/cgit.cgi?url=openembedded/commit/&h=john_lee/openmoko-merges&id=c0041ba98c41b320a36cbc5a66d0c8762e52ac62
>
> --- a/packages/alsa/alsa-state.bb
> +++ b/packages/alsa/alsa-state.bb
> @@ -20,6 +20,7 @@ inherit update-rc.d
>
> INITSCRIPT_PARAMS = "defaults 10"
> +INITSCRIPT_PARAMS_openmoko = "start 39 S ."
>
> Distro overrides are strongly discouraged and in this case not needed,  
> since it's using 'defaults', so an optimization is welcomed. In a  
> similar vein, the change to the initscript can be done to the regular  
> one, no need to add a special openmoko version.
>
> *  
> http://cgit.openembedded.net/cgit.cgi?url=openembedded/commit/&h=john_lee/openmoko-merges&id=64b844ec5cc0c1a6acb1a97d1ea745f15eadb38d
> this it what the device table feature is for, see org.oe/dev/files/ for  
> the device tables, machines can have multiple ones if needed.

I agree.  Olv is already looking into that.

> *  
> http://cgit.openembedded.net/cgit.cgi?url=openembedded/commit/&h=john_lee/openmoko-merges&id=db77b062d51be86b946a11efcadaf8e41fd41727
>
> If you modify FILESPATH you can reuse the initscripts from the  
> initscripts packages so you don't need to copy over unchanged ones.  
> Having said that, I think this should all be merged into the initscripts  
> recipe so everyone can benefit from it.
>
> I suspect review and merging would be easier if you 'collapse' the  
> patches per recipe or per directory. Reviewing all the seperate commits  
> will take too much time :(

Maybe it will be easier if people just review it by rebase first then
diff the content instead of read though commits, so one can clearly
see what will be changed, while still keeping the development process
recorded in the commit messages.


Regards,
John

Re: openmoko-merges

[Koen Kooi]

On 18-12-08 11:11, John Lee wrote:
> On Thu, Dec 18, 2008 at 10:02:53AM +0100, Koen Kooi wrote:
>> On 18-12-08 08:19, John Lee wrote:

 > As mentioned before, what's in /etc/init.d/ , rcS and different
 > runlevels varies from distro to distro, machine to machine...  Would
 > it be better if I remove as many overrides as possible, we merge it to
 > oe.dev and fix problems there?  I don't think I can test all these
 > combinations alone.

My proposal would be:

* move it to seperate branch
* remove nearly all overrides (putting them back is easier than removing 
them a few months later when everybody has forgotten why it was there)
* review it

When the review comes out ok send a mail stating "this branch will get 
merged in 2 weeks unless serious bugs emerge, please test". I don't 
think anyone can rightfully complain after that :)


>> I suspect review and merging would be easier if you 'collapse' the
>> patches per recipe or per directory. Reviewing all the seperate commits
>> will take too much time :(
>
> Maybe it will be easier if people just review it by rebase first then
> diff the content instead of read though commits, so one can clearly
> see what will be changed, while still keeping the development process
> recorded in the commit messages.

It's too big to get reviewed, plain and simple. For OE we agreed (see 
the RFC sent yesterday) that changes should be in small topic branches. 
If you put the changes in topic branches (e.g. fast-boot, bug-fixes, 
openmoko-misc) we could keep the precioussssssss development process if 
the branches are small. Small branches is the key thing.

Oh, and with 'review' I mean a real review *process* where change are 
made to the proposed patches and if needed get a few iterations of the 
commit-fix-comment-fix cycle.

regards,

Koen

Re: openmoko-merges

[Otavio Salvador]

John Lee <john_lee@openmoko.org> writes:

> Hi,
>
> Please review john_lee/openmoko-merges.  I started from
> holger/openmoko-merges then rebased it to oe.dev.
>
> The general principle is:
>
> 1) Minimal impact: most of the big changes apply to om distro only.
> My thought is we merge this first, then apply to the general.
>
> 2) Keep history: I did it the hard way by cherry-picking one by one
> since the history is different after the mtn->git switch.  Most of the
> commit messages are kept, but that creates a lot of commits to review.
>
> 3) PKG_TAGS, qtopia, toolchain, om-utils and opkg stuffs are left out
> for later merge.
>
> Not sure how long it's going to take.  My plan is to merge it this
> Monday if it's okay...

Personally I think that OE should have a clear history. There's no
reason  to keep commits like:

 Do foo
 Do something
 Fix foo broken in previous commit
 Really do something
 ...

A clearer and more useful history would be:

 Do something
 Do foo
 ...

And it should not be hard to merge it back into OpenMoko tree since
the 3-way merge will detect the same changes and should reduce the
conflicts a lot.

Anyway, I personally prefer to have clearer history then keep the
whole list of commits there. Others?

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: openmoko-merges

[John Lee]

Okay, here is what I will do:

1) commit zero controversial part into oe.dev first.

2) put fastboot related stuffs into another branch for review

3) rebase -i the rest to create a clean history, force update the
openmoko-merges branch, then ask for review again.


Regards,
John




On Thu, Dec 18, 2008 at 03:19:01PM +0800, John Lee wrote:
> Hi,
> 
> Please review john_lee/openmoko-merges.  I started from
> holger/openmoko-merges then rebased it to oe.dev.
> 
> The general principle is:
> 
> 1) Minimal impact: most of the big changes apply to om distro only.
> My thought is we merge this first, then apply to the general.
> 
> 2) Keep history: I did it the hard way by cherry-picking one by one
> since the history is different after the mtn->git switch.  Most of the
> commit messages are kept, but that creates a lot of commits to review.
> 
> 3) PKG_TAGS, qtopia, toolchain, om-utils and opkg stuffs are left out
> for later merge.
> 
> Not sure how long it's going to take.  My plan is to merge it this
> Monday if it's okay...
> 
> 
> Regards,
> John
> 
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel

Re: openmoko-merges

[Koen Kooi]

On 18-12-08 16:05, John Lee wrote:
> Okay, here is what I will do:
>
> 1) commit zero controversial part into oe.dev first.

You don't know if it's controversial or not without it being reviewed. 
So no, don't commit stuff.

> 2) put fastboot related stuffs into another branch for review

Make another branch for the stuff you thing is non-controversial so we 
can review that as well.

> 3) rebase -i the rest to create a clean history, force update the
> openmoko-merges branch, then ask for review again.

Try to keep every branch below approx 30 commits, otherwise it becomes 
to cumbersome to review.

regards,

Koen

>
>
> Regards,
> John
>
>
>
>
> On Thu, Dec 18, 2008 at 03:19:01PM +0800, John Lee wrote:
>> Hi,
>>
>> Please review john_lee/openmoko-merges.  I started from
>> holger/openmoko-merges then rebased it to oe.dev.
>>
>> The general principle is:
>>
>> 1) Minimal impact: most of the big changes apply to om distro only.
>> My thought is we merge this first, then apply to the general.
>>
>> 2) Keep history: I did it the hard way by cherry-picking one by one
>> since the history is different after the mtn->git switch.  Most of the
>> commit messages are kept, but that creates a lot of commits to review.
>>
>> 3) PKG_TAGS, qtopia, toolchain, om-utils and opkg stuffs are left out
>> for later merge.
>>
>> Not sure how long it's going to take.  My plan is to merge it this
>> Monday if it's okay...
>>
>>
>> Regards,
>> John
>>
>> _______________________________________________
>> Openembedded-devel mailing list
>> Openembedded-devel@lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel

Re: openmoko-merges

[John Lee]

On Thu, Dec 18, 2008 at 04:24:04PM +0100, Koen Kooi wrote:
> On 18-12-08 16:05, John Lee wrote:
>> Okay, here is what I will do:
>>
>> 1) commit zero controversial part into oe.dev first.
>
> You don't know if it's controversial or not without it being reviewed.  
> So no, don't commit stuff.

Although I haven't committed directly upstream for quite a while, I
think I can be the judge of that.  Feel free to revert and explain if
anything wrong.

>> 2) put fastboot related stuffs into another branch for review
>
> Make another branch for the stuff you thing is non-controversial so we  
> can review that as well.
>
>> 3) rebase -i the rest to create a clean history, force update the
>> openmoko-merges branch, then ask for review again.
>
> Try to keep every branch below approx 30 commits, otherwise it becomes  
> to cumbersome to review.

Yes sir, will do.


- John

>
> regards,
>
> Koen
>
>>
>>
>> Regards,
>> John
>>
>>
>>
>>
>> On Thu, Dec 18, 2008 at 03:19:01PM +0800, John Lee wrote:
>>> Hi,
>>>
>>> Please review john_lee/openmoko-merges.  I started from
>>> holger/openmoko-merges then rebased it to oe.dev.
>>>
>>> The general principle is:
>>>
>>> 1) Minimal impact: most of the big changes apply to om distro only.
>>> My thought is we merge this first, then apply to the general.
>>>
>>> 2) Keep history: I did it the hard way by cherry-picking one by one
>>> since the history is different after the mtn->git switch.  Most of the
>>> commit messages are kept, but that creates a lot of commits to review.
>>>
>>> 3) PKG_TAGS, qtopia, toolchain, om-utils and opkg stuffs are left out
>>> for later merge.
>>>
>>> Not sure how long it's going to take.  My plan is to merge it this
>>> Monday if it's okay...
>>>
>>>
>>> Regards,
>>> John
>>>
>>> _______________________________________________
>>> Openembedded-devel mailing list
>>> Openembedded-devel@lists.openembedded.org
>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>
>
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel

Re: openmoko-merges

[Phil Blundell]

On Thu, 2008-12-18 at 16:24 +0100, Koen Kooi wrote:
> On 18-12-08 16:05, John Lee wrote:
> > Okay, here is what I will do:
> >
> > 1) commit zero controversial part into oe.dev first.
> 
> You don't know if it's controversial or not without it being reviewed. 
> So no, don't commit stuff.

Is there now a policy of requiring review and pre-approval for all
checkins to the tree?  If so, when was this instigated?  I haven't
noticed any discussion of that and I would not be in favour of any such
change.

p.

Re: openmoko-merges

[Koen Kooi]

On 18-12-08 17:54, Phil Blundell wrote:
> On Thu, 2008-12-18 at 16:24 +0100, Koen Kooi wrote:
>> On 18-12-08 16:05, John Lee wrote:
>>> Okay, here is what I will do:
>>>
>>> 1) commit zero controversial part into oe.dev first.
>> You don't know if it's controversial or not without it being reviewed.
>> So no, don't commit stuff.
>
> Is there now a policy of requiring review and pre-approval for all
> checkins to the tree?  If so, when was this instigated?  I haven't
> noticed any discussion of that and I would not be in favour of any such
> change.

No policy for checkins per se, but for merging commits for large 
branches. In this case, a review was requested and the response was to 
*not* commit anything.
Asking for a review and then saying "I'll be the judge of things" is a 
bit backwards.
The coreteam will make a statement about merging branches soonish.

regards,

Koen

Re: openmoko-merges

[John Lee]

On Thu, Dec 18, 2008 at 07:05:51PM +0100, Koen Kooi wrote:
> On 18-12-08 17:54, Phil Blundell wrote:
>> On Thu, 2008-12-18 at 16:24 +0100, Koen Kooi wrote:
>>> On 18-12-08 16:05, John Lee wrote:
>>>> Okay, here is what I will do:
>>>>
>>>> 1) commit zero controversial part into oe.dev first.
>>> You don't know if it's controversial or not without it being reviewed.
>>> So no, don't commit stuff.
>>
>> Is there now a policy of requiring review and pre-approval for all
>> checkins to the tree?  If so, when was this instigated?  I haven't
>> noticed any discussion of that and I would not be in favour of any such
>> change.
>
> No policy for checkins per se, but for merging commits for large  
> branches. In this case, a review was requested and the response was to  
> *not* commit anything.
> Asking for a review and then saying "I'll be the judge of things" is a  
> bit backwards.
> The coreteam will make a statement about merging branches soonish.
>


Something in the om.dev are quite normal just like other commits in
oe.dev tree, such as changing sane-srcrev, adding checksum, recipes,
fixes, etc.  I doubt these need to be reviewed along with the whole
branch. (well, I realize it's me who put them there in the first
place... :p )

I'll keep working on the fastboot branch (need the help of original
author on this one.  since this will become general changes, it's not
going to be easy), and reorganize the om-merge branch a bit.  Is this
the first time of such a big merge ?  I think Graeme and I both did
some merge back before, but this one seems to be bigger...


- John

Re: openmoko-merges

[Graeme Gregory]

John Lee wrote:
> I'll keep working on the fastboot branch (need the help of original
> author on this one.  since this will become general changes, it's not
> going to be easy), and reorganize the om-merge branch a bit.  Is this
> the first time of such a big merge ?  I think Graeme and I both did
> some merge back before, but this one seems to be bigger...
>
>   
I used to use a process similar to what koen suggested where I would
only merge back into OE the final changes to a .bb not all the
mistakes/fuckups/typos that got us to that result. Then evaluate each
change on its own to judge its level of controversy.

Graeme

Re: openmoko-merges

[Ricardo Salveti de Araujo]

On Fri, Dec 19, 2008 at 7:48 AM, Graeme Gregory <dp@xora.org.uk> wrote:
> John Lee wrote:
>> I'll keep working on the fastboot branch (need the help of original
>> author on this one.  since this will become general changes, it's not
>> going to be easy), and reorganize the om-merge branch a bit.  Is this
>> the first time of such a big merge ?  I think Graeme and I both did
>> some merge back before, but this one seems to be bigger...
>>
> I used to use a process similar to what koen suggested where I would
> only merge back into OE the final changes to a .bb not all the
> mistakes/fuckups/typos that got us to that result. Then evaluate each
> change on its own to judge its level of controversy.

Yep, this is something I was thinking about this week, where I decided
to reorganize the Mamona patches to merge it back to OE.

Merging related modifications in just one patch helps a lot to review
the code and also to merge it upstream, the bad part is that we lose
all the history behind those patches. Adding a message telling that
the patch came from Mamona helps this issue, as the user could see it
at the Mamona own repository, but still don't know what's the
preferred way.

What are your suggestions? Should I create branches and rebase Mamona
repo to do the merge with all the history or just merging related
commits in one is enough (also with branches and review)?

Thanks,
-- 
Ricardo Salveti de Araujo

Re: openmoko-merges

[Tom Rini]

On Fri, Dec 19, 2008 at 04:46:03PM -0300, Ricardo Salveti de Araujo wrote:
> On Fri, Dec 19, 2008 at 7:48 AM, Graeme Gregory <dp@xora.org.uk> wrote:
> > John Lee wrote:
> >> I'll keep working on the fastboot branch (need the help of original
> >> author on this one.  since this will become general changes, it's not
> >> going to be easy), and reorganize the om-merge branch a bit.  Is this
> >> the first time of such a big merge ?  I think Graeme and I both did
> >> some merge back before, but this one seems to be bigger...
> >>
> > I used to use a process similar to what koen suggested where I would
> > only merge back into OE the final changes to a .bb not all the
> > mistakes/fuckups/typos that got us to that result. Then evaluate each
> > change on its own to judge its level of controversy.
> 
> Yep, this is something I was thinking about this week, where I decided
> to reorganize the Mamona patches to merge it back to OE.
> 
> Merging related modifications in just one patch helps a lot to review
> the code and also to merge it upstream, the bad part is that we lose
> all the history behind those patches. Adding a message telling that
> the patch came from Mamona helps this issue, as the user could see it
> at the Mamona own repository, but still don't know what's the
> preferred way.
> 
> What are your suggestions? Should I create branches and rebase Mamona
> repo to do the merge with all the history or just merging related
> commits in one is enough (also with branches and review)?

IMHO, a lot of history is worthless sometimes.  The important things to
encapsulate from a big history are the tricky things that you should
comment on, or elaborate on in the final commit message (We had to do
... because ...).  So long as the hard to get right / complex / etc bits
are explained, you don't need 10 commits showing how it was almost
right, you just need the correct solution explained.

-- 
Tom Rini

Re: openmoko-merges

[Koen Kooi]

On 19-12-08 21:03, Tom Rini wrote:
> On Fri, Dec 19, 2008 at 04:46:03PM -0300, Ricardo Salveti de Araujo wrote:
>> On Fri, Dec 19, 2008 at 7:48 AM, Graeme Gregory<dp@xora.org.uk>  wrote:
>>> John Lee wrote:
>>>> I'll keep working on the fastboot branch (need the help of original
>>>> author on this one.  since this will become general changes, it's not
>>>> going to be easy), and reorganize the om-merge branch a bit.  Is this
>>>> the first time of such a big merge ?  I think Graeme and I both did
>>>> some merge back before, but this one seems to be bigger...
>>>>
>>> I used to use a process similar to what koen suggested where I would
>>> only merge back into OE the final changes to a .bb not all the
>>> mistakes/fuckups/typos that got us to that result. Then evaluate each
>>> change on its own to judge its level of controversy.
>> Yep, this is something I was thinking about this week, where I decided
>> to reorganize the Mamona patches to merge it back to OE.
>>
>> Merging related modifications in just one patch helps a lot to review
>> the code and also to merge it upstream, the bad part is that we lose
>> all the history behind those patches. Adding a message telling that
>> the patch came from Mamona helps this issue, as the user could see it
>> at the Mamona own repository, but still don't know what's the
>> preferred way.
>>
>> What are your suggestions? Should I create branches and rebase Mamona
>> repo to do the merge with all the history or just merging related
>> commits in one is enough (also with branches and review)?
>
> IMHO, a lot of history is worthless sometimes.  The important things to
> encapsulate from a big history are the tricky things that you should
> comment on, or elaborate on in the final commit message (We had to do
> ... because ...).  So long as the hard to get right / complex / etc bits
> are explained, you don't need 10 commits showing how it was almost
> right, you just need the correct solution explained.

Comments in the code (i.e. recipes, patches) itself would be even 
better, since we might even to monthly tarballs snapshots of the 
upcomining stablebrach where people would lack a bit of history.

regards,

Koen