[HOWTO] Using git for mtd

[HOWTO] Using git for mtd

[Jörn Engel]

This is likely still full of omissions and plain errors, but there
doesn't seem to be a better document floating around.  Many thanks to
Arnd Bergmann who helped me a lot!


1. Clone Linus' tree

  $ git clone git://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git mtd

2. Change into soon-to-be mtd tree

  $ cd mtd

3. Setup mtd remotes file

  $ vi .git/remotes/mtd
  URL: git://git.infradead.org/mtd-2.6
  Pull: master:mtd

4. Fetch mtd tree

  $ git fetch mtd

5. Checkout mtd tree

  $ git checkout mtd

6. Edit some files

  Use your favorite editor.  Don't forget to tell git about new files
  with "git add".

7. Commit changes

  $ git commit -a

  FIXME: It is easy to forget a Signed-Off-By: line.  This needs to be
  automated.

8. Create private tree on pentafluge

  FIXME

9. Push to private tree on pentafluge

  $ git push ssh://pf/~/public_git/mtd-2.6.git

Jörn

-- 
There are two ways of constructing a software design: one way is to make
it so simple that there are obviously no deficiencies, and the other is
to make it so complicated that there are no obvious deficiencies.
-- C. A. R. Hoare

Re: [HOWTO] Using git for mtd

[Jörn Engel]

Next version with the FIXMEs filled in.  If anyone has comments, spots
errors, etc., please tell me.

1. Clone Linus' tree

  $ git clone git://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git mtd

2. Change into soon-to-be mtd tree

  $ cd mtd

3. Setup mtd remotes file

  $ vi .git/remotes/mtd
  URL: git://git.infradead.org/mtd-2.6
  Pull: master:mtd

4. Fetch mtd tree

  $ git fetch mtd

5. Checkout mtd tree

  $ git checkout mtd

6. Edit some files

  Use your favorite editor.  Don't forget to tell git about new files
  with "git add".

7. Commit changes

  $ git commit -a -s

  Don't forget the '-s' part or you have to add the Signed-Off-By:
  line yourself.

8. Create private tree on pentafluge

  $ ssh pentafluge.infradead.org
  $ mkdir public_git
  $ cd public_git
  $ ln -s ~/public_git
  $ git clone -l -n -s --bare /home/git/mtd-2.6.git mtd-2.6.git

9. Push to private tree on pentafluge

  $ git push ssh://pf/~/public_git/mtd-2.6.git

Jörn

-- 
Victory in war is not repetitious.
-- Sun Tzu

Re: [HOWTO] Using git for mtd

[Josh Boyer]

On 4/22/06, Jörn Engel <joern@wohnheim.fh-wedel.de> wrote:
> Next version with the FIXMEs filled in.  If anyone has comments, spots
> errors, etc., please tell me.
>
> 1. Clone Linus' tree
>
>   $ git clone git://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git mtd
>
> 2. Change into soon-to-be mtd tree
>
>   $ cd mtd
>
> 3. Setup mtd remotes file
>
>   $ vi .git/remotes/mtd
>   URL: git://git.infradead.org/mtd-2.6
>   Pull: master:mtd
>
> 4. Fetch mtd tree
>
>   $ git fetch mtd
>

Huh?

These look odd to me.  Here are the steps I use (note git 1.3.0 is required)

1. git clone git://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
linux-git

This clones the latest of Linus' tree.

2. git clone --reference linux-git git://git.infradead.org/mtd-2.6.git mtd

This clones the MTD tree, however it uses the previous tree created as
a reference to pull objects from so you don't have two copies of the
exact some git objects lying around.

The reason I do this is two fold.  One is that I want to have the
latest 2.6 tree available to me anyway.  If you don't want/need that,
then just clone the mtd-2.6 tree outright.  The second reason is that
I don't have to do wonky things like 'git fetch mtd'.  A fetch doesn't
merge the changes into the tree.  It just gets the objects.

By using --reference, you get the benefit of sharing the bulk of the
git objects and the simplicity of being able to just do 'git pull' in
both trees to update them.  You also avoid having to locally merge in
the mtd-2.6 changes, which is something David would really like to
avoid.  That way he can pull directly from your tree and doesn't get
extraneous merges.  Which also makes it easier for him to have Linus
pull from _his_ tree, because Linus hates extraneous merges.

The rest of your steps seem fine.

josh

Re: [HOWTO] Using git for mtd

[Jörn Engel]

On Sat, 22 April 2006 19:09:55 -0500, Josh Boyer wrote:
> On 4/22/06, Jörn Engel <joern@wohnheim.fh-wedel.de> wrote:
> > Next version with the FIXMEs filled in.  If anyone has comments, spots
> > errors, etc., please tell me.
> >
> > 1. Clone Linus' tree
> >
> >   $ git clone git://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git mtd
> >
> > 2. Change into soon-to-be mtd tree
> >
> >   $ cd mtd
> >
> > 3. Setup mtd remotes file
> >
> >   $ vi .git/remotes/mtd
> >   URL: git://git.infradead.org/mtd-2.6
> >   Pull: master:mtd
> >
> > 4. Fetch mtd tree
> >
> >   $ git fetch mtd
> >
> 
> Huh?
> 
> These look odd to me.  Here are the steps I use (note git 1.3.0 is required)
> 
> 1. git clone git://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
> linux-git
> 
> This clones the latest of Linus' tree.
> 
> 2. git clone --reference linux-git git://git.infradead.org/mtd-2.6.git mtd
> 
> This clones the MTD tree, however it uses the previous tree created as
> a reference to pull objects from so you don't have two copies of the
> exact some git objects lying around.
> 
> The reason I do this is two fold.  One is that I want to have the
> latest 2.6 tree available to me anyway.  If you don't want/need that,
> then just clone the mtd-2.6 tree outright.  The second reason is that
> I don't have to do wonky things like 'git fetch mtd'.  A fetch doesn't
> merge the changes into the tree.  It just gets the objects.
> 
> By using --reference, you get the benefit of sharing the bulk of the
> git objects and the simplicity of being able to just do 'git pull' in
> both trees to update them.  You also avoid having to locally merge in
> the mtd-2.6 changes, which is something David would really like to
> avoid.  That way he can pull directly from your tree and doesn't get
> extraneous merges.  Which also makes it easier for him to have Linus
> pull from _his_ tree, because Linus hates extraneous merges.

Well, we have two sets of weird and arcane command that basically do
the same.  Minor differences are:

Joern					Josh
o works well with "cp -lr"
  -linus and -mtd trees can share	-
  source files as well as git objects
o requires extra branch			+
o sequence of commands is longer	+
o works with older git versions		-

Everything else should be shared.  So people get to decide which of
the pros and cons are more important.  I happen to care a lot about
older git versions (until debian unstable has 1.3.0) and even more
about "cp -lr".  YMMV

So I guess we should have both versions in the howto.

> The rest of your steps seem fine.

Excellent.

Jörn

-- 
Fools ignore complexity.  Pragmatists suffer it.
Some can avoid it.  Geniuses remove it.
-- Perlis's Programming Proverb #58, SIGPLAN Notices, Sept.  1982

Re: [HOWTO] Using git for mtd

[David Woodhouse]

On Sat, 2006-04-22 at 19:11 +0200, Jörn Engel wrote:
> Next version with the FIXMEs filled in.  If anyone has comments, spots
> errors, etc., please tell me.
> 
> 1. Clone Linus' tree
> 
>   $ git clone git://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git mtd

Probably best to explain _why_ we're asking people to refrain from just
cloning git://git.infradead.org/mtd-2.6.git 

> 2. Change into soon-to-be mtd tree
> 
>   $ cd mtd
> 
> 3. Setup mtd remotes file
> 
>   $ vi .git/remotes/mtd
>   URL: git://git.infradead.org/mtd-2.6
>   Pull: master:mtd
> 
> 4. Fetch mtd tree
> 
>   $ git fetch mtd
> 
> 5. Checkout mtd tree
> 
>   $ git checkout mtd
> 
> 6. Edit some files
> 
>   Use your favorite editor.  Don't forget to tell git about new files
>   with "git add".
> 
> 7. Commit changes
> 
>   $ git commit -a -s
> 
>   Don't forget the '-s' part or you have to add the Signed-Off-By:
>   line yourself.
> 
> 8. Create private tree on pentafluge
> 
>   $ ssh pentafluge.infradead.org
>   $ mkdir public_git
>   $ cd public_git
>   $ ln -s ~/public_git
>   $ git clone -l -n -s --bare /home/git/mtd-2.6.git mtd-2.6.git
> 

chmod og+rx ~

The HOWTO should also make it clear that the cron job indexes users'
trees every five minutes -- it'll be that long before you can see it in
http://git.infradead.org/, although the
git://git.infradead.org/~user/mtd-2.6.git URL should work immediately.

Probably best to use the rôle address 'git.infradead.org' instead of the
machine name pentafluge.infradead.org, especially as I'm hoping to shift
git services onto a shiny new G5 if I can persuade someone to contribute
such a beast.

> 9. Push to private tree on pentafluge
> 
>   $ git push ssh://pf/~/public_git/mtd-2.6.git

s/pf/git.infradead.org/

> Jörn
> 
-- 
dwmw2

Re: [HOWTO] Using git for mtd

[Jörn Engel]

On Sun, 23 April 2006 13:08:33 +0100, David Woodhouse wrote:
> 
> Probably best to explain _why_ we're asking people to refrain from just
> cloning git://git.infradead.org/mtd-2.6.git 

Because you are a cheap bastard and count every single byte, of
course. ;)

Jörn

-- 
A quarrel is quickly settled when deserted by one party; there is
no battle unless there be two.
-- Seneca

Re: [HOWTO] Using git for mtd

[David Woodhouse]

On Sat, 2006-04-22 at 19:11 +0200, Jörn Engel wrote:
> Next version with the FIXMEs filled in.  If anyone has comments, spots
> errors, etc., please tell me.

We should also mention merges. If you're putting something into a git
tree for me to pull from, it's best if that pull will _not_ result in a
merge. That means you should base it on the current mtd-2.6 tree, not on
a tree which you've merged with Linus' tree for yourself.

Obviously, if you have a longer-lived git tree with stuff like xattr
support in it, that's expected to have merges -- I mean the above just
for the short-lived stuff.

-- 
dwmw2

Re: [HOWTO] Using git for mtd

[David Woodhouse]

On Sun, 2006-04-23 at 14:17 +0200, Jörn Engel wrote:
> On Sun, 23 April 2006 13:08:33 +0100, David Woodhouse wrote:
> > 
> > Probably best to explain _why_ we're asking people to refrain from just
> > cloning git://git.infradead.org/mtd-2.6.git 
> 
> Because you are a cheap bastard and count every single byte, of
> course. ;)

That and the fact that you IBMers managed to put pentafluge into swap
death when you were trying to clone trees from it. There's another
512MiB of RAM being delivered to the site within the next few days, and
hopefully I can get it installed during the week too. After that,
cloning directly from pentafluge ought to be a little saner. The network
still isn't great though, iirc. 

-- 
dwmw2

Re: [HOWTO] Using git for mtd (v3)

[Jörn Engel]

On Sat, 22 April 2006 19:11:33 +0200, Jörn Engel wrote:

1. Clone Linus' tree

  $ git clone git://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git mtd

  The mtd git server is behind a slow DSL line and does not have
  enough network bandwith for people to clone the full tree on a
  regular basis.

2. Change into soon-to-be mtd tree

  $ cd mtd

3. Setup mtd remotes file

  $ vi .git/remotes/mtd
  URL: git://git.infradead.org/mtd-2.6
  Pull: master:mtd

4. Fetch mtd tree

  $ git fetch mtd

5. Checkout mtd tree

  $ git checkout mtd

6. Edit some files

  Use your favorite editor.  Don't forget to tell git about new files
  with "git add".

7. Commit changes

  $ git commit -a -s

  Don't forget the '-s' part or you have to add the Signed-Off-By:
  line yourself.

8. Create private tree on pentafluge

  $ ssh git.infradead.org
  $ mkdir public_git
  $ cd public_git
  $ ln -s ~/public_git
  $ git clone -l -n -s --bare /home/git/mtd-2.6.git mtd-2.6.git
  $ chmod og+rx ~ ~/public_git

9. Push to private tree on pentafluge

  $ git push ssh://git.infradead.org/~/public_git/mtd-2.6.git

  http://git.infradead.org/ will index the tree every five minutes via
  cron.

Jörn

-- 
Sometimes, asking the right question is already the answer.
-- Unknown

Re: [HOWTO] Using git for mtd (v3)

[Josh Boyer]

On 4/23/06, Jörn Engel <joern@wohnheim.fh-wedel.de> wrote:
> On Sat, 22 April 2006 19:11:33 +0200, Jörn Engel wrote:
>
> 1. Clone Linus' tree
>
>   $ git clone git://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git mtd
>
>   The mtd git server is behind a slow DSL line and does not have
>   enough network bandwith for people to clone the full tree on a
>   regular basis.
>
> 2. Change into soon-to-be mtd tree
>
>   $ cd mtd
>
> 3. Setup mtd remotes file
>
>   $ vi .git/remotes/mtd
>   URL: git://git.infradead.org/mtd-2.6
>   Pull: master:mtd
>
> 4. Fetch mtd tree
>
>   $ git fetch mtd

Does that really do what you want though?  It will fetch the objects
from the mtd-2.6 tree, but it doesn't merge them into your local tree
from what I remember.  So to actually get the changes into the source,
you'd have to do 'git fetch mtd'; 'git merge', no?

josh

Re: [HOWTO] Using git for mtd (v3)

[David Woodhouse]

On Sun, 2006-04-23 at 14:27 +0200, Jörn Engel wrote:
>   The mtd git server is behind a slow DSL line and does not have
>   enough network bandwith for people to clone the full tree on a
>   regular basis. 

Actually it's not on a DSL line. cvs.infradead.org is under my stairs,
but git.infradead.org isn't -- it's in a data centre somewhere. Its
bandwidth isn't great, but as you say I'm a cheap bastard. It's in free
hosting which was volunteered when Red Hat so abruptly and gratuitously
pulled the plug on it a couple of years ago, and I don't feel I can
pester them to improve it.

It seems to get about 2 Mbps inbound, and 1.2Mbps outbound. To be honest
I'm not actually sure _how_ it's connected. A machine right next to it
(zeniiia.uk.linux.org) seems to have better bandwidth -- perhaps
pentafluge is being limited somehow?

-- 
dwmw2

Re: [HOWTO] Using git for mtd (v3)

[Jörn Engel]

On Sun, 23 April 2006 08:05:57 -0500, Josh Boyer wrote:
>
> > 4. Fetch mtd tree
> >
> >   $ git fetch mtd
> 
> Does that really do what you want though?  It will fetch the objects
> from the mtd-2.6 tree, but it doesn't merge them into your local tree
> from what I remember.  So to actually get the changes into the source,
> you'd have to do 'git fetch mtd'; 'git merge', no?

That doesn't do a merge, correct.  I was told that a merge is to be
avoided at all costs.  What it does is create a branch that i switch
to in step 5.

Jörn

-- 
Beware of bugs in the above code; I have only proved it correct, but
not tried it.
-- Donald Knuth

Re: [HOWTO] Using git for mtd (v3)

[David Woodhouse]

On Sun, 2006-04-23 at 14:27 +0200, Jörn Engel wrote:
> 
> 8. Create private tree on pentafluge
> 
>   $ ssh git.infradead.org
>   $ mkdir public_git
>   $ cd public_git
>   $ ln -s ~/public_git
>   $ git clone -l -n -s --bare /home/git/mtd-2.6.git mtd-2.6.git
>   $ chmod og+rx ~ ~/public_git

Edit mtd-2.6.git/description too.

You need to set up a remote branch in your local repository which
matches your new tree on pentafluge, just as you did for the 'mtd'
branch above. Except that this one is for pushing _and_ pulling, of
course:

cat > .git/remotes/mymtd << EOF
URL: git+ssh://git.infradead.org/~/public_git/mtd-2.6.git
Pull: master:mtd
Push: mtd:master
EOF

It's probably easier not to play with branches -- that's fine for people
who want to get clever, but the HOWTO should probably just tell people
to use 'git-clone --reference'; either to clone the mtd-2.6.git tree if
they want a read-only copy of that, or to clone locally on pentafluge
and then git-clone from that if they want their own tree.

> 9. Push to private tree on pentafluge
> 
>   $ git push ssh://git.infradead.org/~/public_git/mtd-2.6.git

git push mymtd

>   http://git.infradead.org/ will index the tree every five minutes via
>   cron. 
-- 
dwmw2

Re: [HOWTO] Using git for mtd (v4)

[Jörn Engel]

Next version.  This time the push should actually work.

1. Clone Linus' tree

  $ git clone git://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git mtd

  The mtd git server is behind a slow DSL line and does not have
  enough network bandwith for people to clone the full tree on a
  regular basis.

2. Change into soon-to-be mtd tree

  $ cd mtd

3. Setup mtd remotes files

  $ vi .git/remotes/mtd_in
  URL: git://git.infradead.org/mtd-2.6
  Pull: master:mtd
  $ vi .git/remotes/mtd_out
  URL: ssh://git.infradead.org/~/public_git/mtd-2.6.git
  Push: Push: mtd:master

4. Fetch mtd tree

  $ git fetch mtd_in

5. Checkout mtd tree

  $ git checkout mtd

6. Edit some files

  Use your favorite editor.  Don't forget to tell git about new files
  with "git add".

7. Commit changes

  $ git commit -a -s

  Don't forget the '-s' part or you have to add the Signed-Off-By:
  line yourself.

8. Create private tree on pentafluge

  $ ssh git.infradead.org
  $ mkdir public_git
  $ cd public_git
  $ ln -s ~/public_git
  $ git clone -l -n -s --bare /home/git/mtd-2.6.git mtd-2.6.git
  $ chmod og+rx ~ ~/public_git

9. Push to private tree on pentafluge

  $ git push mtd_out

  http://git.infradead.org/ will index the tree every five minutes via
  cron.

Jörn

-- 
The cost of changing business rules is much more expensive for software
than for a secretaty.
-- unknown

Re: [HOWTO] Using git for mtd (v4)

[David Woodhouse]

Thanks for the howto. I've mangled it somewhat and put it up at
http://www.linux-mtd.infradead.org/doc/git.html -- although Thomas'
XML->HTML conversion script seems to be eating bits of it.

-- 
dwmw2

Re: [HOWTO] Using git for mtd (v3)

[Nicolas Pitre]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 355 bytes --]

On Sun, 23 Apr 2006, Jörn Engel wrote:

> On Sat, 22 April 2006 19:11:33 +0200, Jörn Engel wrote:
> 
> 1. Clone Linus' tree
> 
>   $ git clone git://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git mtd

To be consistent you should use git.kernel.org instead of 
rsync.kernel.org.  Not that it changes anything in practice but...


Nicolas

Re: [HOWTO] Using git for mtd (v3)

[Jörn Engel]

On Sun, 23 April 2006 13:46:39 -0400, Nicolas Pitre wrote:
> On Sun, 23 Apr 2006, Jörn Engel wrote:
> 
> > On Sat, 22 April 2006 19:11:33 +0200, Jörn Engel wrote:
> > 
> > 1. Clone Linus' tree
> > 
> >   $ git clone git://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git mtd
> 
> To be consistent you should use git.kernel.org instead of 
> rsync.kernel.org.  Not that it changes anything in practice but...

David already has that:
http://www.linux-mtd.infradead.org/doc/git.html

Looks like I'll stop with my howto now.  But thanks for the review.

Jörn

-- 
When people work hard for you for a pat on the back, you've got
to give them that pat.
-- Robert Heinlein