Question on multi-level git repository heiarchy.

Question on multi-level git repository heiarchy.

[Ben Greear]

Some time back, I cloned from the official kernel site.  I have created 
several
branches and patches and such, recently branching off of 2.6.18

I want to create an intermediate level..something like:

kernel.org git tree
       |
my git master tree
 /                          \
work-station-1   work-station-2  ....


Each of the work-station repositories should push/pull from my git 
master tree, and occasionally
the git master tree will also pull from the kernel.org git tree.

To attempt this, I uploaded the .git tree from my local work 
station..the tree I cloned a while
back, to the public server.

I was then able to clone the public server git tree on my local 
work-station in a clean directory.

So far so good...

Both pub server and work station had the current branch set to 
lf_v2.6.18, and gitk showed
the head and latest patch as what I would expect.  I then did a git 
checkout -f master on the
pub server and did a pull from the upstream kernel.
This seemed to work fine.

Then, on the work-station, I did a git checkout -f master, and also did 
a pull.
In this case, it seems that it is trying to merge with changes in the 
lf_v2.6.18 branch
instead of the the main 'master' tree (see below).

So, am I going about this all wrong, or did I just run into a glitch of 
some sort?  I should
mention that the work-station below is running git 1.3, where the master 
is running git 1.4.2.4 (latest in FC4).

[greear@xeon-dt linux-2.6.dev]$ git checkout -f master
[greear@xeon-dt linux-2.6.dev]$ git pull
greearb@ns2.lanforge.com's password:
Generating pack...
Done counting 1553 objects.
Result has 1056 objects.
Deltifying 1056 objects.
 100% (1056/1056) done
Unpacking 1056 objects
Total 1056, written 1056 (delta 894), reused 0 (delta 0)
 100% (1056/1056) done
* refs/heads/master: fast forward to branch 'master' of 
greearb@ns2.lanforge.com:/pub/linux-2.6
  from 04fed361dadb7921507a470947ac23d2f26352cf to 
e80391500078b524083ba51c3df01bbaaecc94bb
* refs/heads/lf_v2.6.17.x: same as branch 'lf_v2.6.17.x' of 
greearb@ns2.lanforge.com:/pub/linux-2.6
* refs/heads/lk_v2.6.16.x.sts: same as branch 'lk_v2.6.16.x.sts' of 
greearb@ns2.lanforge.com:/pub/linux-2.6
* refs/heads/lf_v2.6.17: same as branch 'lf_v2.6.17' of 
greearb@ns2.lanforge.com:/pub/linux-2.6
* refs/heads/lf_v2.6.16.x: same as branch 'lf_v2.6.16.x' of 
greearb@ns2.lanforge.com:/pub/linux-2.6
* refs/heads/lk_v2.6.15.x.sts: same as branch 'lk_v2.6.15.x.sts' of 
greearb@ns2.lanforge.com:/pub/linux-2.6
* refs/heads/lk_v2.6.17.x: same as branch 'lk_v2.6.17.x' of 
greearb@ns2.lanforge.com:/pub/linux-2.6
* refs/heads/lf_v2.6.16: same as branch 'lf_v2.6.16' of 
greearb@ns2.lanforge.com:/pub/linux-2.6
* refs/heads/origin: same as branch 'lf_v2.6.18' of 
greearb@ns2.lanforge.com:/pub/linux-2.6
* refs/heads/lk_v2.6.16.x: same as branch 'lk_v2.6.16.x' of 
greearb@ns2.lanforge.com:/pub/linux-2.6
greearb@ns2.lanforge.com's password:
Auto-following refs/tags/v2.6.19-rc3
greearb@ns2.lanforge.com's password:
Generating pack...
Done counting 1 objects.
Deltifying 1 objects.
 100% (1/1) done
Total 1, written 1 (delta 0), reused 0 (delta 0)
Unpacking 1 objects
 100% (1/1) done
* refs/tags/v2.6.19-rc3: storing tag 'v2.6.19-rc3' of 
greearb@ns2.lanforge.com:/pub/linux-2.6
Warning: fetch updated the current branch head.
Warning: fast forwarding your working tree from
Warning: 04fed361dadb7921507a470947ac23d2f26352cf commit.
Trying really trivial in-index merge...
fatal: Merge requires file-level merging
Nope.
Merging HEAD with ea50583c9fba2e289d434bda9798a7b77ee8e3b4
Merging:
e80391500078b524083ba51c3df01bbaaecc94bb Merge 
master.kernel.org:/pub/scm/linux/kernel/git/davem/sparc-2.6
ea50583c9fba2e289d434bda9798a7b77ee8e3b4 Initial work of merging pktgen 
changes into 2.6.18.
found 1 common ancestor(s):
e478bec0ba0a83a48a0f6982934b6de079e7e6b3 Linux v2.6.18. Arrr!
Auto-merging net/core/skbuff.c
Auto-merging include/net/sock.h
Auto-merging net/core/pktgen.c
CONFLICT (content): Merge conflict in net/core/pktgen.c
Auto-merging net/decnet/dn_table.c
CONFLICT (content): Merge conflict in net/decnet/dn_table.c
Auto-merging net/ipv4/fib_frontend.c
CONFLICT (content): Merge conflict in net/ipv4/fib_frontend.c
Auto-merging arch/i386/kernel/tsc.c
Auto-merging net/ipv4/fib_rules.c
CONFLICT (content): Merge conflict in net/ipv4/fib_rules.c
Auto-merging net/core/dev.c
Auto-merging drivers/net/e1000/e1000_main.c
CONFLICT (content): Merge conflict in drivers/net/e1000/e1000_main.c
Auto-merging net/ipv4/arp.c
CONFLICT (content): Merge conflict in net/ipv4/arp.c
Auto-merging net/decnet/dn_fib.c
CONFLICT (content): Merge conflict in net/decnet/dn_fib.c
Auto-merging net/socket.c
Auto-merging include/linux/if.h
Auto-merging include/linux/compat_ioctl.h
Auto-merging include/linux/skbuff.h
Auto-merging include/linux/rtnetlink.h
CONFLICT (content): Merge conflict in include/linux/rtnetlink.h
Auto-merging fs/cifs/cifsglob.h
Auto-merging include/linux/if_vlan.h
Auto-merging net/ipv4/route.c
CONFLICT (content): Merge conflict in net/ipv4/route.c
Auto-merging net/core/ethtool.c
CONFLICT (content): Merge conflict in net/core/ethtool.c
Auto-merging net/packet/af_packet.c
Auto-merging include/linux/nfs_fs_sb.h
CONFLICT (content): Merge conflict in include/linux/nfs_fs_sb.h
Auto-merging net/8021q/vlan_dev.c
Auto-merging net/ipv6/route.c
CONFLICT (content): Merge conflict in net/ipv6/route.c
Auto-merging net/ipv4/fib_trie.c
Auto-merging include/linux/sunrpc/xprt.h
Auto-merging include/net/neighbour.h
Auto-merging include/net/ip_fib.h
CONFLICT (content): Merge conflict in include/net/ip_fib.h
Auto-merging drivers/net/e100.c
Auto-merging drivers/net/e1000/e1000.h
CONFLICT (content): Merge conflict in drivers/net/e1000/e1000.h
Auto-merging net/ipv4/fib_semantics.c
CONFLICT (content): Merge conflict in net/ipv4/fib_semantics.c
Auto-merging drivers/pci/quirks.c
Auto-merging net/ipv4/fib_lookup.h
CONFLICT (content): Merge conflict in net/ipv4/fib_lookup.h
Auto-merging net/ipv4/fib_hash.c
Auto-merging net/core/sock.c
Auto-merging net/Makefile
CONFLICT (content): Merge conflict in net/Makefile
Auto-merging net/decnet/dn_rules.c
CONFLICT (content): Merge conflict in net/decnet/dn_rules.c
Auto-merging include/linux/netdevice.h
Auto-merging net/decnet/dn_route.c
Auto-merging net/sunrpc/xprtsock.c
Auto-merging net/Kconfig
Auto-merging drivers/net/e1000/e1000_ethtool.c
CONFLICT (content): Merge conflict in drivers/net/e1000/e1000_ethtool.c
Auto-merging include/net/dn_fib.h
CONFLICT (content): Merge conflict in include/net/dn_fib.h
Auto-merging fs/nfs/super.c
CONFLICT (content): Merge conflict in fs/nfs/super.c
Auto-merging fs/cifs/connect.c
CONFLICT (content): Merge conflict in fs/cifs/connect.c

Automatic merge failed; fix conflicts and then commit the result.
[greear@xeon-dt linux-2.6.dev]$ git branch
  lf_v2.6.16
  lf_v2.6.16.x
  lf_v2.6.17
  lf_v2.6.17.x
  lf_v2.6.18
  lk_v2.6.15.x.sts
  lk_v2.6.16.x
  lk_v2.6.16.x.sts
  lk_v2.6.17.x
* master
  origin
[greear@xeon-dt linux-2.6.dev]$ gitk

-- 
Ben Greear <greearb@candelatech.com> 
Candela Technologies Inc  http://www.candelatech.com

Re: Question on multi-level git repository heiarchy.

[Shawn Pearce]

Ben Greear <greearb@candelatech.com> wrote:
> I want to create an intermediate level..something like:
> 
> kernel.org git tree
>       |
> my git master tree
> /                          \
> work-station-1   work-station-2  ....
 
> I then did a git  checkout -f master on the
> pub server and did a pull from the upstream kernel.
> This seemed to work fine.

Ah, what you really want here is to make your "my git master tree"
a bare repostiory and use fetch instead of pull.  This way you
don't need to maintain a working directory of files associated
with that repository.  So assuming you have "mygitmastertree"
as the directory do:

	mv mygitmastertree/.git mygitmastertree.git
	rm -rf mygitmastertree

and update your workstation .git/remotes/origin files such
that the URL line reads ".../mygitmastertree.git" rather than
".../mygitmastertree/.git".

Then to update "mygitmastertree" with recent changes you can use
git fetch rather than git pull:

	git --git-dir mygitmastertree.git fetch
 
> Then, on the work-station, I did a git checkout -f master, and also did 
> a pull.
> In this case, it seems that it is trying to merge with changes in the 
> lf_v2.6.18 branch
> instead of the the main 'master' tree (see below).

When you use "git pull" with no additional arguments the first
branch listed in a Pull: line of .git/remotes/origin will be the
branch merged into the current branch.  I don't know what that
branch is listed as in your workstation tree but from what you
described it sounds like it may be that lf_v2.6.18 branch, which
is why its trying to merge it.

-- 

Re: Question on multi-level git repository heiarchy.

[Ben Greear]

Shawn Pearce wrote:
> Ben Greear <greearb@candelatech.com> wrote:
>   
>> I want to create an intermediate level..something like:
>>
>> kernel.org git tree
>>       |
>> my git master tree
>> /                          \
>> work-station-1   work-station-2  ....
>>     
>  
>   
>> I then did a git  checkout -f master on the
>> pub server and did a pull from the upstream kernel.
>> This seemed to work fine.
>>     
>
> Ah, what you really want here is to make your "my git master tree"
> a bare repostiory and use fetch instead of pull.  This way you
> don't need to maintain a working directory of files associated
> with that repository.  So assuming you have "mygitmastertree"
> as the directory do:
>
> 	mv mygitmastertree/.git mygitmastertree.git
> 	rm -rf mygitmastertree
>
> and update your workstation .git/remotes/origin files such
> that the URL line reads ".../mygitmastertree.git" rather than
> ".../mygitmastertree/.git".
>
> Then to update "mygitmastertree" with recent changes you can use
> git fetch rather than git pull:
>
> 	git --git-dir mygitmastertree.git fetch
>   

Ok, I made those changes...
>  
>   
>> Then, on the work-station, I did a git checkout -f master, and also did 
>> a pull.
>> In this case, it seems that it is trying to merge with changes in the 
>> lf_v2.6.18 branch
>> instead of the the main 'master' tree (see below).
>>     
>
> When you use "git pull" with no additional arguments the first
> branch listed in a Pull: line of .git/remotes/origin will be the
> branch merged into the current branch.  I don't know what that
> branch is listed as in your workstation tree but from what you
> described it sounds like it may be that lf_v2.6.18 branch, which
> is why its trying to merge it.
>   
That is certainly not intuitive.

I want to synchronize the entire git repo, including all branches.  How 
would I go
about doing that?

Is there any way to recover my currently mangled tree on the 
workstation, or do I need
to start fresh there?  If I start fresh, do I re-clone, or is there some 
better way to get
the synchronization that I want?

Thanks,
Ben



-- 
Ben Greear <greearb@candelatech.com> 
Candela Technologies Inc  http://www.candelatech.com

Re: Question on multi-level git repository heiarchy.

[Andy Parkins]

On Friday 2006 October 27 15:05, Ben Greear wrote:

> I want to synchronize the entire git repo, including all branches.  How
> would I go
> about doing that?

That happens anyway.  "git pull" is actually a fetch + a merge.  Personally I 
recommend never doing a "git pull" without arguments.  Instead:

First, update every "Pull:" line from .git/remotes/origin
 $ git fetch
Now manually merge one of the (now locally stored) upstream trees into your 
working tree.
 $ git pull . upstream-tree

> Is there any way to recover my currently mangled tree on the
> workstation, or do I need
> to start fresh there?  If I start fresh, do I re-clone, or is there some
> better way to get

I'm relatively new to git; my experience is that there is very little that 
can't be undone.  Exactly what sort of mangling has happened to your tree?

If you've got yourself in a real state, I suggest editing .git/remotes/origin 
to pull the upstream trees to some newly named branches (maybe just 
add "new/" to the front of them).  Treat those as read-only (i.e. don't check 
them out), and start comparing branches.  "gitk --all" is incredibly useful 
for this.

Make yourself a new branch for your own work based off a new/upstream branch  
(git checkout -b fixthings new/upstream), then use git-cherry-pick to pull 
your patches out of your mangled tree to build up a branch with your changes 
on it.

Now, you can delete all the branches that you think are mangled and (if you 
really want to), rename all the new/ refs back to whatever they started as.

> the synchronization that I want?

I am going to guess that you've fallen for the same thing I did at first: 
confusing "git pull" with "git fetch"; "git pull" can get you in 
trouble, "git fetch" probably won't.


Andy
-- 
Dr Andy Parkins, M Eng (hons), MIEE

Re: Question on multi-level git repository heiarchy.

[Ben Greear]

Shawn Pearce wrote:
> Ben Greear <greearb@candelatech.com> wrote:
>> I want to create an intermediate level..something like:
>>
>> kernel.org git tree
>>       |
>> my git master tree
>> /                          \
>> work-station-1   work-station-2  ....
>  
>> I then did a git  checkout -f master on the
>> pub server and did a pull from the upstream kernel.
>> This seemed to work fine.
> 
> Ah, what you really want here is to make your "my git master tree"
> a bare repostiory and use fetch instead of pull.  This way you
> don't need to maintain a working directory of files associated
> with that repository.  So assuming you have "mygitmastertree"
> as the directory do:
> 
> 	mv mygitmastertree/.git mygitmastertree.git
> 	rm -rf mygitmastertree
> 
> and update your workstation .git/remotes/origin files such
> that the URL line reads ".../mygitmastertree.git" rather than
> ".../mygitmastertree/.git".
> 
> Then to update "mygitmastertree" with recent changes you can use
> git fetch rather than git pull:
> 
> 	git --git-dir mygitmastertree.git fetch
>  
>> Then, on the work-station, I did a git checkout -f master, and also did 
>> a pull.
>> In this case, it seems that it is trying to merge with changes in the 
>> lf_v2.6.18 branch
>> instead of the the main 'master' tree (see below).
> 
> When you use "git pull" with no additional arguments the first
> branch listed in a Pull: line of .git/remotes/origin will be the
> branch merged into the current branch.  I don't know what that
> branch is listed as in your workstation tree but from what you
> described it sounds like it may be that lf_v2.6.18 branch, which
> is why its trying to merge it.

I have been using a variation of this with some success.  (I didn't rename the .git directory
on the master tree, but the directory only contains a .git directory so I assume that
I'm using a bare tree....)

I have been able to do 'git pull origin' and 'git push origin master' to
sync and commit my lf_v2.6.18 branch from my work-station repositories (per this page's instructions:
http://www.kernel.org/pub/software/scm/git/docs/cvs-migration.html

Now, I would like to fetch down all of the changes from kernel.org to my master
tree and then update my work-station branches.  On the master,
I ran 'git fetch'.  It took a while and seemed to do the right thing.
There is still nothing other than .git in the directory, and subsequent
'git fetch' calls return quickly w/out doing anything.

However, I cannot find an incantation on the work-station machines to
pull down the changes and sync up with kernel.org.

I did a 'git fetch' on the work-station, but gitk still does not show anything
new in 'master' or 'lf_v2.6.18

I tried checking out master and running:  git pull . master
but it said Already up-to-date.

I tried checking out the master branch on the work-station and doing a 'git pull' there.
It got a lot of conflicts from the changes that I had made in the lf_v2.6.18 branch.

I want to have some local branch contain only official kernel patches, and
then fork my own branches off to track my individual changes.  My assumption
was that I could do something like:

git checkout master
git pull

and have master updated with (only) the latest tree from kernel.org and have
lf_v2.6.18 un-touched.

Any pointers are welcome.

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: Question on multi-level git repository heiarchy.

[linux]

The thing to learn is that the operation you are trying to perform
is called "fetch", not "pull".  Pull = fetch + merge.

In fact, the "fetch" part of "git pull" is optional, so it's
best to think of "git pull" as the fundamental *merge* operation.
("git merge" is a helper that you probably don't want to use directly.)

There's a valid argument that this is not good terminology (especially
because "push" is the opposite of "fetch"!), but without getting
into that discussion, there are two ways to do a merge:

- The small-steps way.
  git fetch remote
  git pull . remoteorigin

- The bold, all-at-once way
  git pull remote remoteorigin

The former acutally gives a local name to the remote branch,
then merges it in in a second step.

The latter is what folks like Linus do who aren't carefully following
remote development, but just want to merge it in.  It's better for
one-off access to remote development, because it doesn't clutter your
reporistory with a remote branch name that you're not going to use again.

Generally, I always do the former, and I recommend you do too, until
you're comfortable with making such big leaps.

Just remember:
1) "git fetch" copies from the remote repository to the local one.
   This doesn't change anything locally except for branch heads that
   it's supposed to.  And even there, it's careful.
   In particular, "git fetch" will refuse to modify the current HEAD.

2) "git pull" actually does a merge.  This *does* change the current
   HEAD.  It creates a new commit on the HEAD branch unless:
   - HEAD is already up to date, or
   - HEAD can be fast-forwarded, or
   - The merge fails.