merge two local mirrors with detached heads and similar remotes

merge two local mirrors with detached heads and similar remotes

[Zenaan Harkness]

I wish to merge two local mirrors, which each should share the same
object store, and I have been reading for many hours and so far failed
to achieve.

I did learn  git remote add, and applied my knowledge to another local
git clone/ mirror as can be seen here:
$ cd bananajour.git/
$ git remote -v
eadz    https://github.com/eadz/bananajour.git (fetch)
eadz    https://github.com/eadz/bananajour.git (push)
fresh   https://github.com/freshtonic/bananajour.git (fetch)
fresh   https://github.com/freshtonic/bananajour.git (push)
origin  https://github.com/toolmantim/bananajour.git (fetch)
origin  https://github.com/toolmantim/bananajour.git (push)
ts      https://github.com/tswicegood/bananajour.git (fetch)
ts      https://github.com/tswicegood/bananajour.git (push)
$ gb -a
* master
  remotes/eadz/master
  remotes/eadz/railscamp-hackfest
  remotes/fresh/bigbananajour
  remotes/fresh/master
  remotes/fresh/railscamp-hackfest
  remotes/origin/HEAD -> origin/master
  remotes/origin/macruby
  remotes/origin/master
  remotes/origin/railscamp-hackfest
  remotes/origin/v3
  remotes/ts/master
  remotes/ts/railscamp-hackfest


So I want to achieve the same thing (multiple remotes in one local
repo) for an earlier clone of two separate repos, which really ought
to be just one, as follows:

A)
$ cd gitjour.git/
$ cat .git/config
[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        fetch = +refs/heads/*:refs/remotes/origin/*
        url = https://github.com/chad/gitjour.git
[branch "master"]
        remote = origin
        merge = refs/heads/master
$ git branch -a
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master

B)
$ cd gitjour-new.git/
$ cat .git/config
[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        fetch = +refs/heads/*:refs/remotes/origin/*
        url = https://github.com/lachlanhardy/gitjour.git
[branch "master"]
        remote = origin
        merge = refs/heads/master
$ git branch -a
* master
  remotes/origin/8.2.0
  remotes/origin/HEAD -> origin/master
  remotes/origin/lachlanhardy
  remotes/origin/master
  remotes/origin/v8.1.0


I tried, by copying each repo A) and B) into t/ and u/ respectively, then:
$ cd t
$ git remote add lach ../u
$ git fetch latch

but I only get three resulting branches, not the expected 6:
$ gb -a
* master
  remotes/lach/master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master


I'm still learning my terminology sorry, but it seems that the origin
branches in u/ are detached heads or something. That's fine, I just
want them to appear in my t/ mirror.

I realise that I will need to update t/.git/config so that the "lach"
remote points to its canonical/ web URL, but _first_ I want to get all
the refs and objects from my local version, before hooking it up to
the web for future updates. I want to take advantage of the download
I've already done.

(Notwithstanding the fact that these repos are barely a few hundred
KiB, I also need to do this job because my 18-month old linux repos
are in a similar situation - I want to save my disk space by combining
my Ubuntu Hardy linux repo (with it's branches/ heads), and a couple
others, with my canonical Linus linux repo, into one. I did use the
--reference option back when I made these repos respectively, to share
and to minimize download as much as possible, but it seems that what I
really want is a properly shared (single) repo with multiple remotes,
so I don't have to think about what sequence of locals to download
etc.

In this vein, a chapter in the tutorial, perhaps near 'recommended
workflows', on Repository Mirroring Best Practices might be in order.

TIA
Zenaan

Re: merge two local mirrors with detached heads and similar remotes

[Zenaan Harkness]

On Sun, Nov 7, 2010 at 13:40, Zenaan Harkness <zen@freedbms.net> wrote:
> I wish to merge two local mirrors, which each should share the same
> object store
...
> So I want to achieve the same thing (multiple remotes in one local
> repo) for an earlier clone of two separate repos, which really ought
> to be just one, as follows:
>
> A)
> $ cd gitjour.git/
> $ cat .git/config

> [remote "origin"]
>        fetch = +refs/heads/*:refs/remotes/origin/*
>        url = https://github.com/chad/gitjour.git
> [branch "master"]
>        remote = origin
>        merge = refs/heads/master

> $ git branch -a
> * master
>  remotes/origin/HEAD -> origin/master
>  remotes/origin/master
>
> B)
> $ cd gitjour-new.git/
> $ cat .git/config

> [remote "origin"]
>        fetch = +refs/heads/*:refs/remotes/origin/*
>        url = https://github.com/lachlanhardy/gitjour.git
> [branch "master"]
>        remote = origin
>        merge = refs/heads/master
> $ git branch -a
> * master
>  remotes/origin/8.2.0
>  remotes/origin/HEAD -> origin/master
>  remotes/origin/lachlanhardy
>  remotes/origin/master
>  remotes/origin/v8.1.0

It looks to me like I finally solved the first half of the problem,
with clarification from the following URL:
http://www.gelato.unsw.edu.au/archives/git/0601/15567.html

In A) I modified .git/config to be as follows:
$ cat .git/config
[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        fetch = +refs/heads/*:refs/remotes/origin/*
        url = https://github.com/chad/gitjour.git
[branch "master"]
        remote = origin
        merge = refs/heads/master
[remote "lach"]
        url = ../u
#       fetch = +refs/heads/*:refs/remotes/origin/*
        fetch = +refs/remotes/origin/*:refs/remotes/lach/*
$ git fetch lach d
remote: Counting objects: 583, done.
remote: Compressing objects: 100% (303/303), done.
remote: Total 568 (delta 229), reused 566 (delta 229)
Receiving objects: 100% (568/568), 65.29 KiB, done.
Resolving deltas: 100% (229/229), completed with 3 local objects.
From ../u
 * [new branch]      origin/8.2.0 -> lach/8.2.0
 * [new branch]      origin/HEAD -> lach/HEAD
 * [new branch]      origin/lachlanhardy -> lach/lachlanhardy
 * [new branch]      origin/v8.1.0 -> lach/v8.1.0

# and finally:
$ git branch -a
* master
  remotes/lach/8.2.0
  remotes/lach/HEAD
  remotes/lach/lachlanhardy
  remotes/lach/master
  remotes/lach/v8.1.0
  remotes/origin/HEAD -> origin/master
  remotes/origin/master

---------
Looks very much like success to me. Small thing I notice is this line:
  remotes/origin/HEAD -> origin/master

has become this:
  remotes/lach/HEAD

(ie it's not a link any more - probably not particularly relevant,
perhaps git gc would automate tidying this?)
----------
So now I just update the remote URL for lach to point back to upstream
and I think I'm done. Happy days.

That tutorial email URL up top above might be old -
http://www.gelato.unsw.edu.au/archives/git/0601/15567.html is from
January 2006! - and is out of date a bit wrt the .git/config file
format (eg remotes wildcards), BUT in that email, Junio explained this
"local mirror" concept well enough that I could solve my problem here
(it appears) - implementing a fixup of my "local mirror" use case -
I'm a happy camper; thank you Junio!

May be I missed how to do this somewhere in the docs already, but I've
been reading and searching the web for two days, learning lots for
sure, but frustrated that it took me so long to find out how to do
this.

Should something like Junio's tutorial go into
Documentation/gitworkflows.xml(ARGGH!) ?
If not, where else?
Is there anything besides vim (about all I know) you can recommend for
editing xml(ARGGH!) docs?

Thanks
Zenaan

Re: merge two local mirrors with detached heads and similar remotes

[Zenaan Harkness]

> In A) I modified .git/config to be as follows:
> $ cat .git/config
> [core]
>        repositoryformatversion = 0
>        filemode = true
>        bare = false
>        logallrefupdates = true
> [remote "origin"]
>        fetch = +refs/heads/*:refs/remotes/origin/*
>        url = https://github.com/chad/gitjour.git
> [branch "master"]
>        remote = origin
>        merge = refs/heads/master
> [remote "lach"]
>        url = ../u
> #       fetch = +refs/heads/*:refs/remotes/origin/*
>        fetch = +refs/remotes/origin/*:refs/remotes/lach/*

My final change, for reference, was updating [remote "latch"] section with:
   url = https://github.com/lachlanhardy/gitjour.git
   fetch = +refs/heads/*:refs/remotes/lach/*

Appears to work with  git fetch --all
and finally followed by  git gc

Re: merge two local mirrors with detached heads and similar remotes

[Jakub Narebski]

Zenaan Harkness <zen@freedbms.net> writes:

> ---------
> Looks very much like success to me. Small thing I notice is this line:
>   remotes/origin/HEAD -> origin/master
> 
> has become this:
>   remotes/lach/HEAD
> 
> (ie it's not a link any more - probably not particularly relevant,
> perhaps git gc would automate tidying this?)

It might be detached head: you can check it by using

  $ git rev-parse --symbolic-full-name remotes/lach/HEAD

If it is 'remotes/lach/HEAD' or 'HEAD', it is detached HEAD
(unnamed branch).


> Should something like Junio's tutorial go into
> Documentation/gitworkflows.xml(ARGGH!) ?
> If not, where else?
> Is there anything besides vim (about all I know) you can recommend for
> editing xml(ARGGH!) docs?

Documentation/gitworkflows.txt is source of XML, HTML, info and
manpage versions - please edit source (in AsciiDoc), and not generated
documentation.

-- 
Jakub Narebski
Poland
ShadeHawk on #git