Git Migration Status

Git Migration Status

[Richard Purdie]

Hi,

We've made the decision we want to switch to git so what happens now? We
now have:

* Agreed policies (pending no major objections from openembedded-devel)
* A git server setup
* Web browsing of the git server
* Commit hooks for CIA and email notification of commits

What remains:

* A final mtn -> git conversion
  - There is a known problem with the mtn -> git conversion which needs 
    to be addressed. Holger has the details of this
  - We also need to decide how to map the key names
* Collection of keys for access to the git server
* Creating a BKCVS history import and graft this onto the monotone 
  history
  - There are some conversion defects. Richard has the details and is 
    trying to look into this
  - We also need to decide on key name mappings, preferably consistent 
    with the git ones.
* Rename "gittrial" -> "git" on amethyst

Of these the BKCVS import and be done and also redone at a later date so
I'm not worried about this and it doesn't affect timescales. Renaming
gittrial -> git is pretty much trivial and won't take long. Collection
of keys is also relatively straightforward, we just need someone to
document key generation and collect them. They're basically ssh public
keys and gitosis makes them trivial to add so I can take on collection
role if existing developers with access to monotone want to start
sending me keys (ssh v2 keys please).

This leaves the mtn -> git conversion which isn't trivial. Holger is our
expert in this area and I'd really like to let him have the time needed
to sort this out to his satisfaction. I appreciate he has many time
constraints so this may not happen immediately.

There is also the issue of commit ID naming. I'm tempted to propose we
use commit IDs with a domain such as userid@developers.openembedded.org.
This would allow email redirection should we ever want it without
interfering with the main domain. The alternative is to let people use
their own domains but this means the IDs change when people change email
address and isn't really consistent for the metadata and maintaining a
single identity. I'm really open to opinions on this though. 

Also documenting all users with access on the wiki might be a good idea
(we used to do this but I think the data is stale).

Once we have the mtn -> git conversion working to satisfaction I propose
we give 1 weeks warning before monotone becomes readonly, the conversion
is made and we open up git for read/write. This email serves as notice
that people need to be ready to watch out for that time.

I hope this covers everything but any questions just ask...

Cheers,

Richard

Re: Git Migration Status

[Rolf Leggewie]

Hi Richard,

thank you for the update.

Richard Purdie wrote:
> There is also the issue of commit ID naming. I'm tempted to propose we
> use commit IDs with a domain such as userid@developers.openembedded.org.

I think something under openembedded.org is a good idea for several
reasons.  But how about userid@dev.openembedded.org?

1) I like short mail addies
2) it corresponds to the dev branch

I think the IDs should be real (forwarded) mail addies so devs can be
contacted easily.

Regards

Rolf

Re: Git Migration Status

[Graeme Gregory]

On Tue, 29 Jul 2008 16:54:23 +0200
Rolf Leggewie <no2spam@nospam.arcornews.de> wrote:

> Hi Richard,
> 
> thank you for the update.
> 
> Richard Purdie wrote:
> > There is also the issue of commit ID naming. I'm tempted to propose
> > we use commit IDs with a domain such as
> > userid@developers.openembedded.org.
> 
> I think something under openembedded.org is a good idea for several
> reasons.  But how about userid@dev.openembedded.org?
> 
> 1) I like short mail addies
> 2) it corresponds to the dev branch
> 
> I think the IDs should be real (forwarded) mail addies so devs can be
> contacted easily.
> 
But please run spam assassin before forwarding.

Graeme

Re: Git Migration Status

[Otavio Salvador]

Rolf Leggewie <no2spam@nospam.arcornews.de> writes:

> Hi Richard,
>
> thank you for the update.
>
> Richard Purdie wrote:
>> There is also the issue of commit ID naming. I'm tempted to propose we
>> use commit IDs with a domain such as userid@developers.openembedded.org.
>
> I think something under openembedded.org is a good idea for several
> reasons.  But how about userid@dev.openembedded.org?

I disagree. Using real mail address give a real view of who can be
sponsoring work and using OE.  

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Git Migration Status

[Tom Rini]

On Tue, Jul 29, 2008 at 01:40:17PM -0300, Otavio Salvador wrote:
> Rolf Leggewie <no2spam@nospam.arcornews.de> writes:
> 
> > Hi Richard,
> >
> > thank you for the update.
> >
> > Richard Purdie wrote:
> >> There is also the issue of commit ID naming. I'm tempted to propose we
> >> use commit IDs with a domain such as userid@developers.openembedded.org.
> >
> > I think something under openembedded.org is a good idea for several
> > reasons.  But how about userid@dev.openembedded.org?
> 
> I disagree. Using real mail address give a real view of who can be
> sponsoring work and using OE.  

One of the big problems there is what happens when developer X stops
doing stuff for company Y?  Or does something on their own? Submission /
Signed-off/Ack'ed by, that's fine.  But for commit work I think it's
best that it's always the same and a can made functional email.

-- 
Tom Rini

Re: Git Migration Status

[Rich Pixley]

Otavio Salvador wrote:
> Rolf Leggewie <no2spam@nospam.arcornews.de> writes
>> Richard Purdie wrote:
>>     
>>> There is also the issue of commit ID naming. I'm tempted to propose we
>>> use commit IDs with a domain such as userid@developers.openembedded.org.
>>>       
>> I think something under openembedded.org is a good idea for several
>> reasons.  But how about userid@dev.openembedded.org?
>>     
> I disagree. Using real mail address give a real view of who can be
> sponsoring work and using OE.
Seems to me the point is to have commit id's which encourage and support 
responsibility.  Since we know each other primarily by email id's, (and 
yes, these change periodically), it seems completely reasonable to me to 
request that these be done using functional email addresses.  They may 
or may not represent a, (potentially temporary), work assignment or 
employment.  For someone who frequently changes work assignments or 
employment, a more generic address, either a personal one, or a cut-out 
or email forwarder like gmail intended specifically for that purpose 
might be appropriate.

I read the suggestion as meaning that it should, in general, be possible 
for me as a developer and/or consumer to contact the author of a 
particular commit, which sounds completely appropriate to me, at least 
superficially.  If the commit id's aren't email addresses, then we'd 
need to use some other mapping to figure out how to contact commit 
authors.  Or are there other ways to handle this already?

--rich

Re: Git Migration Status

[Otavio Salvador]

Tom Rini <trini@kernel.crashing.org> writes:

> On Tue, Jul 29, 2008 at 01:40:17PM -0300, Otavio Salvador wrote:
>> Rolf Leggewie <no2spam@nospam.arcornews.de> writes:
>> 
>> > Hi Richard,
>> >
>> > thank you for the update.
>> >
>> > Richard Purdie wrote:
>> >> There is also the issue of commit ID naming. I'm tempted to propose we
>> >> use commit IDs with a domain such as userid@developers.openembedded.org.
>> >
>> > I think something under openembedded.org is a good idea for several
>> > reasons.  But how about userid@dev.openembedded.org?
>> 
>> I disagree. Using real mail address give a real view of who can be
>> sponsoring work and using OE.  
>
> One of the big problems there is what happens when developer X stops
> doing stuff for company Y?  Or does something on their own? Submission /
> Signed-off/Ack'ed by, that's fine.  But for commit work I think it's
> best that it's always the same and a can made functional email.

For maintainers, I think we could have a MAINTAINERS file as is
available on Linux sources. This allows for a cannonical place for
look at.

When someone stops to work for company Y the credit should still go to
the sponsoring of the company since this doesn't changes who paid the
work.

Besides that, forcing all commits to be done from a "cannonical mail
domain" will enforce a difficult to follow policy and turn problematic
the merging of ordinary trees with fixes.

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Git Migration Status

[Tom Rini]

On Tue, Jul 29, 2008 at 02:21:19PM -0300, Otavio Salvador wrote:
> Tom Rini <trini@kernel.crashing.org> writes:
> 
> > On Tue, Jul 29, 2008 at 01:40:17PM -0300, Otavio Salvador wrote:
> >> Rolf Leggewie <no2spam@nospam.arcornews.de> writes:
> >> 
> >> > Hi Richard,
> >> >
> >> > thank you for the update.
> >> >
> >> > Richard Purdie wrote:
> >> >> There is also the issue of commit ID naming. I'm tempted to propose we
> >> >> use commit IDs with a domain such as userid@developers.openembedded.org.
> >> >
> >> > I think something under openembedded.org is a good idea for several
> >> > reasons.  But how about userid@dev.openembedded.org?
> >> 
> >> I disagree. Using real mail address give a real view of who can be
> >> sponsoring work and using OE.  
> >
> > One of the big problems there is what happens when developer X stops
> > doing stuff for company Y?  Or does something on their own? Submission /
> > Signed-off/Ack'ed by, that's fine.  But for commit work I think it's
> > best that it's always the same and a can made functional email.
> 
> For maintainers, I think we could have a MAINTAINERS file as is
> available on Linux sources. This allows for a cannonical place for
> look at. 

I will say this will have just as many problems as keeping forwards for
userid@dev.oe.org up to date, at least.

> When someone stops to work for company Y the credit should still go to
> the sponsoring of the company since this doesn't changes who paid the
> work.

Right, but why is the sponsorship in the commitid, rather than the
commit log, etc?

> Besides that, forcing all commits to be done from a "cannonical mail
> domain" will enforce a difficult to follow policy and turn problematic
> the merging of ordinary trees with fixes.

I don't follow.  You can tell git "for this tree, use me@whatever".
IIRC, these things can even be rewritten when pulled in, if someone does
a drive-by bugfixing.

-- 
Tom Rini

Re: Git Migration Status

[Otavio Salvador]

Rich Pixley <rich.pixley@palm.com> writes:

> I read the suggestion as meaning that it should, in general, be
> possible for me as a developer and/or consumer to contact the author
> of a particular commit, which sounds completely appropriate to me, at
> least superficially.  If the commit id's aren't email addresses, then
> we'd need to use some other mapping to figure out how to contact
> commit authors.  Or are there other ways to handle this already?

I fully agree with that but also some statistics[1] as being done for
Linux seems very interesting.

1. http://mirror.celinuxforum.org/gitstat/

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Git Migration Status

[Thomas Kunze]

Hi,

just a minor question:
Can we have the sender for commit messages as it is in monotone?
e.g. "mickeyl commit" instead of  "OE GIT Trial"

I find it very useful to be able to see who committed sth. with a short 
look
into my mailbox.

Regards,
Thomas

Re: Git Migration Status

[Otavio Salvador]

Thomas Kunze <thommycheck@gmx.de> writes:

> Hi,
>
> just a minor question:
> Can we have the sender for commit messages as it is in monotone?
> e.g. "mickeyl commit" instead of  "OE GIT Trial"
>
> I find it very useful to be able to see who committed sth. with a
> short look
> into my mailbox.

You'll see the merge commit and the mail script could be changed to
post the commiter id too I guess.

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Git Migration Status

[Tom Rini]

On Tue, Jul 29, 2008 at 04:31:38PM -0300, Otavio Salvador wrote:
> Thomas Kunze <thommycheck@gmx.de> writes:
> 
> > Hi,
> >
> > just a minor question:
> > Can we have the sender for commit messages as it is in monotone?
> > e.g. "mickeyl commit" instead of  "OE GIT Trial"
> >
> > I find it very useful to be able to see who committed sth. with a
> > short look
> > into my mailbox.
> 
> You'll see the merge commit and the mail script could be changed to
> post the commiter id too I guess.

The following script is something I used to use years ago on the linux
kernel git commit lists and triggered from procmail:
#!/bin/sh

TMP=`mktemp /tmp/setfrom.XXXXXX`
cat > $TMP

## Old BitKeeper logic
from=`grep '^ChangeSet' $TMP | head -1 | awk '{print $NF}'`

## New git logic
author=`sed -n '/^author /p' < $TMP | head -n 1`
if [ ! -z "$author" ]; then
  from=`echo $author | sed 's/author \(.*>\).*/\1/'`
  date=`echo $author | sed 's/author.*> //'`
fi
if test -n "$from" -a -n "$date"; then
  formail -I "From: $from" -I "Date: `date -d "$date" -R`" < $TMP
else
  if test -n "$from"; then
    formail -I "From: $from" < $TMP
  else
    cat $TMP
  fi
fi

rm $TMP

-- 
Tom Rini

Re: Git Migration Status

[Rod Whitby]

Richard Purdie wrote:
> There is also the issue of commit ID naming. I'm tempted to propose we
> use commit IDs with a domain such as userid@developers.openembedded.org.
> This would allow email redirection should we ever want it without
> interfering with the main domain. The alternative is to let people use
> their own domains but this means the IDs change when people change email
> address and isn't really consistent for the metadata and maintaining a
> single identity. I'm really open to opinions on this though. 

Just use real email addresses.  If someone doesn't want to use their 
current home or work email address, or has an address that they don't 
own (and therefore will change in the future), they can easily get a 
google mail or yahoo mail account purely for a commit email address.

The Linux kernel uses real email addresses.  Why would OE do something 
different?

Since the git commit Id is always "Real Name <email@address>" (not just 
a raw email address) then the Real Name will usually stay the same 
irrespective of the email address changing - this provides the desired 
continuity of identification.

-- Rod

Re: Git Migration Status

[Marcin Juszkiewicz]

On Wednesday 30 of July 2008 01:57:43 Rod Whitby wrote:
> Richard Purdie wrote:
> > There is also the issue of commit ID naming. I'm tempted to propose
> > we use commit IDs with a domain such as
> > userid@developers.openembedded.org. This would allow email
> > redirection should we ever want it without interfering with the
> > main domain. The alternative is to let people use their own domains
> > but this means the IDs change when people change email address and
> > isn't really consistent for the metadata and maintaining a single
> > identity. I'm really open to opinions on this though.
>
> Just use real email addresses.  If someone doesn't want to use their
> current home or work email address, or has an address that they don't
> own (and therefore will change in the future), they can easily get a
> google mail or yahoo mail account purely for a commit email address.
>
> The Linux kernel uses real email addresses.  Why would OE do
> something different?

I am also for real email addresses. If someone wants to use 
userid@dev.openembedded.org as commit one I am fine as long as this 
alias works as email. Now I do most of OE related job during my OH hours 
so I would prefer to commit as hrw@o-hand.com and as 
openembedded@haerwu.biz for my own stuff. In both situations there will 
be real name attached to email so it should not be a problem to contact 
me.

Re: Git Migration Status

[Cliff Brake]

On Wed, Jul 30, 2008 at 3:20 AM, Marcin Juszkiewicz
<openembedded@haerwu.biz> wrote:
> On Wednesday 30 of July 2008 01:57:43 Rod Whitby wrote:
>> Richard Purdie wrote:
>> > There is also the issue of commit ID naming. I'm tempted to propose
>> > we use commit IDs with a domain such as
>> > userid@developers.openembedded.org. This would allow email
>> > redirection should we ever want it without interfering with the
>> > main domain. The alternative is to let people use their own domains
>> > but this means the IDs change when people change email address and
>> > isn't really consistent for the metadata and maintaining a single
>> > identity. I'm really open to opinions on this though.
>>
>> Just use real email addresses.  If someone doesn't want to use their
>> current home or work email address, or has an address that they don't
>> own (and therefore will change in the future), they can easily get a
>> google mail or yahoo mail account purely for a commit email address.
>>
>> The Linux kernel uses real email addresses.  Why would OE do
>> something different?
>
> I am also for real email addresses. If someone wants to use
> userid@dev.openembedded.org as commit one I am fine as long as this
> alias works as email. Now I do most of OE related job during my OH hours
> so I would prefer to commit as hrw@o-hand.com and as
> openembedded@haerwu.biz for my own stuff. In both situations there will
> be real name attached to email so it should not be a problem to contact
> me.

Using real email addresses would also simplify server admin and be one
less load on the server.  When the spam starts flowing, this can
require a significant amount of network bandwidth.  Where I host my
main web site and domain, the owner claims spam to email accounts is
by far the greatest use of bandwidth, and he spends a lot of time
implementing anti-spam measures.  With limited resources, simple is
good!

Cliff

-- 
=======================
Cliff Brake
http://bec-systems.com

Re: Git Migration Status

[Otavio Salvador]

"Cliff Brake" <cliff.brake@gmail.com> writes:

> Using real email addresses would also simplify server admin and be one
> less load on the server.  When the spam starts flowing, this can
> require a significant amount of network bandwidth.  Where I host my
> main web site and domain, the owner claims spam to email accounts is
> by far the greatest use of bandwidth, and he spends a lot of time
> implementing anti-spam measures.  With limited resources, simple is
> good!

+1

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Git Migration Status

[Philip Balister]

[-- Attachment #1: Type: text/plain, Size: 667 bytes --]

Otavio Salvador wrote:
> "Cliff Brake" <cliff.brake@gmail.com> writes:
> 
>> Using real email addresses would also simplify server admin and be one
>> less load on the server.  When the spam starts flowing, this can
>> require a significant amount of network bandwidth.  Where I host my
>> main web site and domain, the owner claims spam to email accounts is
>> by far the greatest use of bandwidth, and he spends a lot of time
>> implementing anti-spam measures.  With limited resources, simple is
>> good!
> 
> +1
> 

I also agree. I would prefer to commit with my own address. I own all 
the domains, so they should last as long as I do.

Philip

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 3303 bytes --]

Re: Git Migration Status

[Richard Purdie]

On Wed, 2008-07-30 at 09:07 -0400, Philip Balister wrote:
> Otavio Salvador wrote:
> > "Cliff Brake" <cliff.brake@gmail.com> writes:
> > 
> >> Using real email addresses would also simplify server admin and be one
> >> less load on the server.  When the spam starts flowing, this can
> >> require a significant amount of network bandwidth.  Where I host my
> >> main web site and domain, the owner claims spam to email accounts is
> >> by far the greatest use of bandwidth, and he spends a lot of time
> >> implementing anti-spam measures.  With limited resources, simple is
> >> good!
> > 
> > +1
> > 
> 
> I also agree. I would prefer to commit with my own address. I own all 
> the domains, so they should last as long as I do.

Before we all start jumping on this wagon let me explain some of the
background to what I proposed. I was trying to keep that announcement
brief and it assumes some understanding of git so let me explain
further:

Git commit IDs are a totally free form fields. There is nothing stopping
me making commit A as "Richard <rp@localhost>", commit B as "rpurdie
rurdie@laptop" and then commit C as "Joe <someone@somewhere>". Commit A
was my desktop, commit B was my laptop, commit C was on some account I
borrowed.

One problem with git is that its all too easy to screw up setting the
author/commit IDs. I've done it before, I suspect I will again next time
I change machine and I'm sure others will too intentionally or
otherwise.

Going back in time, we had the same problem with bitkeeper. Anyone
looking through the bkcvs commit ids will see exactly what I mean -
there are some entries in there that are totally untraceable now, e.g.
account@1.2.3.4

Why does it matter? I'd like to be able to go to the SCM and *know* who
made a commit (knowing who authored the patch is different). I'd like to
be able to view all commits for a given "identity".

Also note that git makes a distinction between git commit IDs and git
author IDs. It makes sense to standardise the commit IDs and these are
the ones I'm suggesting match a pattern like "Foo
<id@dev.openembedded.org>". Note I'm not too bothered about the author
IDs as long as they're valid email addresses (i.e. not localhost) and
I'm also not bothered about the name used against the commit address
(Foo in the above example).  That could be "Foo (@Work)" or anything
else anyone desires as long as the id@ is valid which is what I care
about and allows us to obtain meaningful information from our metadata.

Gitweb and cgit usually show the author ID, and you have to look harder
for the commit IDs.

Also, "the kernel does this, why can't we" is a totally bogus argument:

1. If the kernel jumped off a cliff, would you too?
2. The kernel uses a pull model for development and people check IDs for
some sanity before pulling. With the push model we're going for we don't
have that luxury.

Hope that clears some of this up.

Cheers,

Richard

Re: Git Migration Status

[Rod Whitby]

Richard Purdie wrote:
> Git commit IDs are a totally free form fields. There is nothing stopping
> me making commit A as "Richard <rp@localhost>", commit B as "rpurdie
> rurdie@laptop" and then commit C as "Joe <someone@somewhere>". Commit A
> was my desktop, commit B was my laptop, commit C was on some account I
> borrowed.

However, a git commit ID is intended to be a real name and a real email 
address, and the distinction between an author id and a commit id is 
meant to have real meaning (i.e. two different people were involved in 
how this change came into being and how it got accepted into the SCM).

Note that both the author ID and commit ID are meant to be real names 
and real email addresses.  You do want to be able to contact each of 
those two people under different circumstances.

> One problem with git is that its all too easy to screw up setting the
> author/commit IDs. I've done it before, I suspect I will again next time
> I change machine and I'm sure others will too intentionally or
> otherwise.

I agree that this can be a problem.

My proposal is to keep an AUTHORS file at the top level of the OE 
repository, and make git check the commit ID against that file in the 
commit hook.

This would be *much* easier to manage than having to have pseudo email 
address aliases.  It would also mean that anyone with existing commit 
privs could update the AUTHORS file for someone else if that someone 
else needs to change their contact email address for some reason.

> Why does it matter? I'd like to be able to go to the SCM and *know* who
> made a commit (knowing who authored the patch is different). I'd like to
> be able to view all commits for a given "identity".

I'd like to do the same, and I'd like those identifiers to (a) be real 
email addresses and (b) be the same identifiers those people use on 
other git repositories so I can judge the pedigree of the person making 
the commit.

> Also, "the kernel does this, why can't we" is a totally bogus argument:
> 
> 1. If the kernel jumped off a cliff, would you too?

If it involved how to use git, yes.  Otherwise, no.

> 2. The kernel uses a pull model for development and people check IDs for
> some sanity before pulling. With the push model we're going for we don't
> have that luxury.

That is true.  However, you also need to account for a multi-level push 
model, where the second and subsequent levels may not have the same 
restrictions on IDs that you are proposing for the master OE repository. 
  In this case, the author and committer will really be two different 
people, and you want to have real contact email addresses for both.

In summary, I'm dead against identifiers that are not the usual 
identifier that OE contributors use everywhere on the internet.

-- Rod

Re: Git Migration Status

[Rod Whitby]

Richard Purdie wrote:
> We've made the decision we want to switch to git so what happens now? We
> now have:
> 
> * Agreed policies (pending no major objections from openembedded-devel)
> * A git server setup
> * Web browsing of the git server
> * Commit hooks for CIA and email notification of commits
> 
> What remains:
> 
> * A final mtn -> git conversion
>   - There is a known problem with the mtn -> git conversion which needs 
>     to be addressed. Holger has the details of this
>   - We also need to decide how to map the key names
> * Collection of keys for access to the git server
> * Creating a BKCVS history import and graft this onto the monotone 
>   history
>   - There are some conversion defects. Richard has the details and is 
>     trying to look into this
>   - We also need to decide on key name mappings, preferably consistent 
>     with the git ones.
> * Rename "gittrial" -> "git" on amethyst

BTW, since I'm complaining so much about the identifiers in other 
messages, I just want to say here that I really appreciate all the work 
that Richard and others have put into the monotone to git transition.

I'm very excited by this transition, and have already updated the 
nslu2-linux Master Makefile to use git and have begun the transition of 
nslu2-linux monotone repositories to git so that we can be ready and 
waiting for the OE flag day.

I look forward to the day when I can remove monotone from my laptop and 
from the nslu2-linux project servers ...

Thanks to all involved!

-- Rod

Re: Git Migration Status

[Richard Purdie]

I'm going to skip the rest of your argument, I don't agree the other
bits but this bit did does lead to something important: 

On Thu, 2008-07-31 at 00:42 +0930, Rod Whitby wrote:
> Richard Purdie wrote:
> > 2. The kernel uses a pull model for development and people check IDs for
> > some sanity before pulling. With the push model we're going for we don't
> > have that luxury.
> 
> That is true.  However, you also need to account for a multi-level push 
> model, where the second and subsequent levels may not have the same 
> restrictions on IDs that you are proposing for the master OE repository. 
>   In this case, the author and committer will really be two different 
> people, and you want to have real contact email addresses for both.

This isn't a problem as you describe as nowhere did I suggest making
committer == author or require any changes to author. It does bring
something else slightly related to mind though.

Say I have some tree outside OE and I commit things to is as a developer
with no access to OE directly. "We" then want to merge that into OE.
Even assuming the commit hooks allow this (there are ways and means) we
immediately lose our namespace protection. There are 101 other scenarios
that would allow this to happen and they show that namespace protection
is pointless with git.

Note that your AUTHORS file would also break with this approach unless
at merge time you grep the external tree for all email addresses and
then add them which would be tedious at best.

I think we've lost sight of the original problems so lets go back to
them:

The points I had in mind with this was to:

a) stop insane IDs entering the repository (ich@1.2.3.4)
b) gain something we could feed into the final mtn -> git and bkcvs -> 
   git conversions which would make more sense of the data. 
   "crofton@openembedded.org" really isn't a useful identifier.
c) ensure we have accurate records about our committers on the wiki
d) remove problems trying to associate related commit IDs e.g.  when 
   someone changes email address frequently and solve that problem once 
   and for all.

We can prove d) is impossible above so lets forget it. c) below partly
covers it though.

For b) I'd propose making a list of the monotone IDs we have and asking
for opinions of what to do with them. I'm happy for
"rpurdie@openembedded.org" to become "Richard Purdie <rpurdie@rpsys.net>
for example and that will increase the usefulness of the metadata (some
username mappings are much harder to guess). I already have a list of
bkcvs ids to monotone IDs so this just leaves the the monotone -> git
mappings. Is anyone willing to volunteer to collect a list?

For c), we should make updating the wiki part of the procedure for
having an ssh key added for access to the git server. Whilst some people
have protested in the past I'd like to see people's email addresses
listed there including any aliases they commit under. They're going to
appear on the web representations of git anyway. This means if an
address is dead we have an idea of what the live one might be. We can
also probably collect this information from the wiki in a script about
defunct addresses and have gitweb and cgit use it when displaying the
IDs if I remember rightly.

For a) I guess the best we can do is add some checking for insane commit
and author IDs like containing "localhost", not setting a name or
containing an IP address. I'd also like to add something to the commit
policy about only committing using sane IDs, commits with invalid IDs
are at risk of being reverted.

How does that sound?

Richard

Re: Git Migration Status

[Cliff Brake]

On Wed, Jul 30, 2008 at 10:28 AM, Richard Purdie <rpurdie@rpsys.net> wrote:
> On Wed, 2008-07-30 at 09:07 -0400, Philip Balister wrote:
>> Otavio Salvador wrote:
>> > "Cliff Brake" <cliff.brake@gmail.com> writes:
>> >
>> >> Using real email addresses would also simplify server admin and be one
>> >> less load on the server.  When the spam starts flowing, this can
>> >> require a significant amount of network bandwidth.  Where I host my
>> >> main web site and domain, the owner claims spam to email accounts is
>> >> by far the greatest use of bandwidth, and he spends a lot of time
>> >> implementing anti-spam measures.  With limited resources, simple is
>> >> good!
>> >
>> > +1
>> >
>>
>> I also agree. I would prefer to commit with my own address. I own all
>> the domains, so they should last as long as I do.
>
> Before we all start jumping on this wagon let me explain some of the
> background to what I proposed. I was trying to keep that announcement
> brief and it assumes some understanding of git so let me explain
> further:

Thanks for explaining -- this provoked me to do a little more digging ...

We have a nomenclature issue -- "commit id" is typically referred to
in the git documentation as the hash that defines a particular commit,
so perhaps "committer" is slightly more correct.  Being on the doc
team, I guess I'm supposed to be picky about stuff like this :-)

The following docs give a little more information:

http://www.kernel.org/pub/software/scm/git/docs/git-config.html
http://www.kernel.org/pub/software/scm/git/docs/git-commit-tree.html

GIT_AUTHOR_NAME
GIT_AUTHOR_EMAIL
GIT_AUTHOR_DATE
GIT_COMMITTER_NAME
GIT_COMMITTER_EMAIL
GIT_COMMITTER_DATE

Is env variables the only way to set these to be different than user.*?

The following commands:

git log --pretty=raw
git show --pretty=raw

can be used to show both committer and author id.  gitk also shows
both author and committer ID.

One thing that is useful is to be able to figure out who might have
merged a change from another branch.  For example, I could have my own
git repo where all the commit IDs are my ID (oops, I meant
committer!!!!).  Someone else could merge this repo into the master OE
repo.  In this case, the committer ID for a particular commit does not
tell you who decided to merge my branch into the master OE repo, as my
committer ID will still be listed in the commits.  You have to follow
a graph of all the merges to know for sure what happened.

There are several ways to view this:

git log --branch ... (my version of git does not have this :-(  )
gitk $commit_id..$branch_name

The second option shows a simple graphical view of how something got
into a branch and all the merge points.  The more I mess with git, the
more impressed I am that you can just try stuff like "gitk
$commit_id..$branch_name", and it just works.

Cliff

-- 
=======================
Cliff Brake
http://bec-systems.com

Re: Git Migration Status

[Cliff Brake]

On Wed, Jul 30, 2008 at 12:14 PM, Richard Purdie <rpurdie@rpsys.net> wrote:

> Say I have some tree outside OE and I commit things to is as a developer
> with no access to OE directly. "We" then want to merge that into OE.
> Even assuming the commit hooks allow this (there are ways and means) we
> immediately lose our namespace protection. There are 101 other scenarios
> that would allow this to happen and they show that namespace protection
> is pointless with git.

Agreed, I'm not sure there is any way we can force a uniform commit id
format with the possibility of merging branches.

> Note that your AUTHORS file would also break with this approach unless
> at merge time you grep the external tree for all email addresses and
> then add them which would be tedious at best.

It seems to me that whoever does the merge has to vouch for the
commits in the branch that gets merged.

> For b) I'd propose making a list of the monotone IDs we have and asking
> for opinions of what to do with them. I'm happy for
> "rpurdie@openembedded.org" to become "Richard Purdie <rpurdie@rpsys.net>
> for example and that will increase the usefulness of the metadata (some
> username mappings are much harder to guess). I already have a list of
> bkcvs ids to monotone IDs so this just leaves the the monotone -> git
> mappings. Is anyone willing to volunteer to collect a list?

Why not put a list on the wiki and ask people to edit.  I'll see what
I can find ...

> For a) I guess the best we can do is add some checking for insane commit
> and author IDs like containing "localhost", not setting a name or
> containing an IP address. I'd also like to add something to the commit
> policy about only committing using sane IDs, commits with invalid IDs
> are at risk of being reverted.

Seems like it has to come down to policy.  Anyone who wants to merge a
branch needs to make sure it has sane commit ID's first.  Its fairly
easy to track down who merged a commit, so this may require some
enforcement, but people will learn fairly quickly if its enforced.

Cliff

-- 
=======================
Cliff Brake
http://bec-systems.com

Re: Git Migration Status

[Cliff Brake]

On Wed, Jul 30, 2008 at 1:06 PM, Cliff Brake <cliff.brake@gmail.com> wrote:
> On Wed, Jul 30, 2008 at 12:14 PM, Richard Purdie <rpurdie@rpsys.net> wrote:

>> For b) I'd propose making a list of the monotone IDs we have and asking
>> for opinions of what to do with them. I'm happy for
>> "rpurdie@openembedded.org" to become "Richard Purdie <rpurdie@rpsys.net>
>> for example and that will increase the usefulness of the metadata (some
>> username mappings are much harder to guess). I already have a list of
>> bkcvs ids to monotone IDs so this just leaves the the monotone -> git
>> mappings. Is anyone willing to volunteer to collect a list?
>
> Why not put a list on the wiki and ask people to edit.  I'll see what
> I can find ...

http://wiki.openembedded.net/index.php/Monotone_to_Git_ID_mapping

Please modify as you see fit, or if you need additional information.

Are you proposing at this point that we don't require restrictions on
commiter ID (such as @dev.openembedded.org) ?

Thanks,
Cliff

-- 
=======================
Cliff Brake
http://bec-systems.com

Re: Git Migration Status

[Otavio Salvador]

Richard Purdie <rpurdie@rpsys.net> writes:

> On Wed, 2008-07-30 at 09:07 -0400, Philip Balister wrote:
>> Otavio Salvador wrote:
>> > "Cliff Brake" <cliff.brake@gmail.com> writes:
>> > 
>> >> Using real email addresses would also simplify server admin and be one
>> >> less load on the server.  When the spam starts flowing, this can
>> >> require a significant amount of network bandwidth.  Where I host my
>> >> main web site and domain, the owner claims spam to email accounts is
>> >> by far the greatest use of bandwidth, and he spends a lot of time
>> >> implementing anti-spam measures.  With limited resources, simple is
>> >> good!
>> > 
>> > +1
>> > 
>> 
>> I also agree. I would prefer to commit with my own address. I own all 
>> the domains, so they should last as long as I do.
>
> Before we all start jumping on this wagon let me explain some of the
> background to what I proposed. I was trying to keep that announcement
> brief and it assumes some understanding of git so let me explain
> further:
>
> Git commit IDs are a totally free form fields. There is nothing stopping
> me making commit A as "Richard <rp@localhost>", commit B as "rpurdie
> rurdie@laptop" and then commit C as "Joe <someone@somewhere>". Commit A
> was my desktop, commit B was my laptop, commit C was on some account I
> borrowed.
>
> One problem with git is that its all too easy to screw up setting the
> author/commit IDs. I've done it before, I suspect I will again next time
> I change machine and I'm sure others will too intentionally or
> otherwise.
>
> Going back in time, we had the same problem with bitkeeper. Anyone
> looking through the bkcvs commit ids will see exactly what I mean -
> there are some entries in there that are totally untraceable now, e.g.
> account@1.2.3.4

I fully understand and I share this POV. It will happen, indeed.

> Why does it matter? I'd like to be able to go to the SCM and *know* who
> made a commit (knowing who authored the patch is different). I'd like to
> be able to view all commits for a given "identity".

This will be a task for the commiters. Who is going to merge others
tree will need to be careful.

> Also note that git makes a distinction between git commit IDs and git
> author IDs. It makes sense to standardise the commit IDs and these are
> the ones I'm suggesting match a pattern like "Foo
> <id@dev.openembedded.org>". Note I'm not too bothered about the author
> IDs as long as they're valid email addresses (i.e. not localhost) and
> I'm also not bothered about the name used against the commit address
> (Foo in the above example).  That could be "Foo (@Work)" or anything
> else anyone desires as long as the id@ is valid which is what I care
> about and allows us to obtain meaningful information from our metadata.

I still think that standardise it will be a problem from adminstration
side and bothering for the developers. Even it is a small group that
will end up being the push rights, they can forget it and like. 

> Gitweb and cgit usually show the author ID, and you have to look harder
> for the commit IDs.
>
> Also, "the kernel does this, why can't we" is a totally bogus argument:
>
> 1. If the kernel jumped off a cliff, would you too?
> 2. The kernel uses a pull model for development and people check IDs for
> some sanity before pulling. With the push model we're going for we don't
> have that luxury.
>
> Hope that clears some of this up.

It does but the push model will not be for everyone. Those with push
rights could have their mails in a whitelist in the update hook and
those be checked. If this person pulls from someone else, we trust
this person to check for those stupid mail/settings mistakes before
pushing it.

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Git Migration Status

[Otavio Salvador]

Richard Purdie <rpurdie@rpsys.net> writes:

> Note that your AUTHORS file would also break with this approach unless
> at merge time you grep the external tree for all email addresses and
> then add them which would be tedious at best.

His proposal is for the commiters ID not the authors ID. We can assume
that someone who has push right is also an OE author.

-- 
        O T A V I O    S A L V A D O R
---------------------------------------------
 E-mail: otavio@debian.org      UIN: 5906116
 GNU/Linux User: 239058     GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
---------------------------------------------
"Microsoft sells you Windows ... Linux gives
 you the whole house."

Re: Git Migration Status

[K. Richard Pixley]

Is there a wiki page somewhere which lists the rationale for switching 
from mtn to git?

--rich

Re: Git Migration Status

[Mark Brown]

On Wed, Jul 30, 2008 at 02:35:18PM -0300, Otavio Salvador wrote:
> Richard Purdie <rpurdie@rpsys.net> writes:

> > Note that your AUTHORS file would also break with this approach unless
> > at merge time you grep the external tree for all email addresses and
> > then add them which would be tedious at best.

> His proposal is for the commiters ID not the authors ID. We can assume
> that someone who has push right is also an OE author.

Only if the person doing the push hasn't pulled commits from anywhere
else - git protocol transfers don't change the committer so if patches
enter the tree being pushed via a git pull then the committer may not be
an OE person.

-- 
"You grabbed my hand and we fell into it, like a daydream - or a fever."

Re: Git Migration Status

[Esben Haabendal]

On Wed, Jul 30, 2008 at 4:28 PM, Richard Purdie <rpurdie@rpsys.net> wrote:

> Also, "the kernel does this, why can't we" is a totally bogus argument:
>
> 1. If the kernel jumped off a cliff, would you too?
> 2. The kernel uses a pull model for development and people check IDs for
> some sanity before pulling. With the push model we're going for we don't
> have that luxury.

This raises another question I have been wondering. Does this push model
fit OE better than pull?
The pull model seems to work nicely for the Linux kernel, and as I see it, has
the advantage that the "infrastructure" adapts automagically to the needs of
the developers/community.

I am just wondering...

/Esben

Re: Git Migration Status

[Rod Whitby]

Richard Purdie wrote:
> I'm going to skip the rest of your argument, I don't agree the other
> bits but this bit did does lead to something important: 

Yeah, if we couldn't convince each other in an hour of IRC discussion, 
then we're not going to do so via email.

> Say I have some tree outside OE and I commit things to is as a developer
> with no access to OE directly. "We" then want to merge that into OE.
> Even assuming the commit hooks allow this (there are ways and means) we
> immediately lose our namespace protection. There are 101 other scenarios
> that would allow this to happen and they show that namespace protection
> is pointless with git.

Agreed.

> Note that your AUTHORS file would also break with this approach unless
> at merge time you grep the external tree for all email addresses and
> then add them which would be tedious at best.

Yes, I thought it would only check the final committer, but others have 
pointed out that it checks each one in the imported tree.  So I withdraw 
that proposal.

> I think we've lost sight of the original problems so lets go back to
> them:
> 
> The points I had in mind with this was to:
> 
> a) stop insane IDs entering the repository (ich@1.2.3.4)

Agreed.

> b) gain something we could feed into the final mtn -> git and bkcvs -> 
>    git conversions which would make more sense of the data. 
>    "crofton@openembedded.org" really isn't a useful identifier.

I agree with your proposed solution on this one.

> c) ensure we have accurate records about our committers on the wiki

I agree with your proposed solution on this one.

> d) remove problems trying to associate related commit IDs e.g.  when 
>    someone changes email address frequently and solve that problem once 
>    and for all.

People will usually retain the same Real Name part of the ID, so we can 
match quite well using that when the email address changes.

> For a) I guess the best we can do is add some checking for insane commit
> and author IDs like containing "localhost", not setting a name or
> containing an IP address. I'd also like to add something to the commit
> policy about only committing using sane IDs, commits with invalid IDs
> are at risk of being reverted.

I'm very happy with that solution, and fully agree that the policy must 
mandate sane IDs and we must forcefully reject with extreme prejudice 
any commits without a valid ID.

> How does that sound?

I'm very happy with your proposal.  Thanks for taking my views into account.

-- Rod

Re: Git Migration Status

[Rod Whitby]

Cliff Brake wrote:
> http://wiki.openembedded.net/index.php/Monotone_to_Git_ID_mapping
> 
> Please modify as you see fit, or if you need additional information.

I have added information for all the @nslu2-linux.org keys.

-- Rod

Re: Git Migration Status

[Mike (mwester)]

K. Richard Pixley wrote:
> Is there a wiki page somewhere which lists the rationale for switching
> from mtn to git?

Gaah!  No!

:D

Please see the archives of this mailing list for all the gory details,
if you really wish; it's been discussed "ad nauseum" more than once.

Mike (mwester)

Re: Git Migration Status

[K. Richard Pixley]

Mike (mwester) wrote:
> K. Richard Pixley wrote:
>   
>> Is there a wiki page somewhere which lists the rationale for switching
>> from mtn to git?
>>     
>
> Gaah!  No!
>
> :D
>
> Please see the archives of this mailing list for all the gory details,
> if you really wish; it's been discussed "ad nauseum" more than once.
I'm sure.  I was hoping for a bullet point executive summary.

I like mtn, but I'm not about to even attempt to reopen the issue.  I'm 
just curious about what the deciding factors were.  And I suspect that 
the question will recur for some time.

--rich

Re: Git Migration Status

[Cliff Brake]

On Thu, Jul 31, 2008 at 1:16 PM, K. Richard Pixley <rich.pixley@palm.com> wrote:

> I'm sure.  I was hoping for a bullet point executive summary.
>
> I like mtn, but I'm not about to even attempt to reopen the issue.  I'm just
> curious about what the deciding factors were.  And I suspect that the
> question will recur for some time.

From what I can gather from email threads, it comes down to:

- better support for branching and merging (this is where git shines)
- much faster
- much larger user base -- most people doing OE work are already using
git for kernel work (more people familiar)

I'm sure there are other reasons to switch and not to.  I'll add the
above list and links to maillist threads on:

http://wiki.openembedded.net/index.php/Git

Thanks,
Cliff

-- 
=======================
Cliff Brake
http://bec-systems.com