git fetch refs and tags

git fetch refs and tags

[Jan Weitzel]

Hello,
I have the following problem: I have 2 bare git repositories one has
several branches and tags.
If I try this in the second repository:
git fetch -f ../main.git refs/heads/master:refs/heads/master 
I'm getting also tags from other branches, if I have an old object from
one of the other branches.
I would expect to have only tags pointing to master ref. (Although the
man pages points to the behaviour regarding dangling objects). Is there
a way to avoid this? I only find --no-tags which results in having no
tags at all. Or need I git purge to remove the old objects first?
My goal is to fetch only specific branches and the related tags.

Jan

Re: git fetch refs and tags

[Johan Herland]

On Tue, Apr 23, 2013 at 12:53 PM, Jan Weitzel <J.Weitzel@phytec.de> wrote:
> Hello,
> I have the following problem: I have 2 bare git repositories one has
> several branches and tags.
> If I try this in the second repository:
> git fetch -f ../main.git refs/heads/master:refs/heads/master
> I'm getting also tags from other branches, if I have an old object from
> one of the other branches.
> I would expect to have only tags pointing to master ref. (Although the
> man pages points to the behaviour regarding dangling objects). Is there
> a way to avoid this? I only find --no-tags which results in having no
> tags at all. Or need I git purge to remove the old objects first?
> My goal is to fetch only specific branches and the related tags.

AFAIK, Git should only auto-follow tags that are reachable from the
branches you fetch (in this case master). Are you saying that you get
tags pointing to other history that is NOT reachable from the master
branch? (i.e. are you getting tags for which "git merge-base $tag
master" is not equal to "git rev-parse $tag")?

Re-reading the man page, I do see the following:

"if the repository has objects that are pointed by remote tags that it
does not yet have, then fetch those missing tags. If the other end has
tags that point at branches you are not interested in, you will not
get them."

This can be interpreted as saying that even unreachable objects in
your local repo that are pointed to by some remote tag will cause that
tag to be fetched, and in effect resuscitate the
previously-unreachable object. If this is indeed how it works, I would
be tempted to label this a bug in the auto-following behavior, as it's
probably not what most people would expect. In that case, yes, you
should be able to get your desired behavior by first purging all
unreachable objects. Something like "git gc --prune=now" should do the
job.

Hope this helps,

...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: git fetch refs and tags

[Jan Weitzel]

Am Dienstag, den 23.04.2013, 13:25 +0200 schrieb Johan Herland:
> On Tue, Apr 23, 2013 at 12:53 PM, Jan Weitzel <J.Weitzel@phytec.de> wrote:
> > Hello,
> > I have the following problem: I have 2 bare git repositories one has
> > several branches and tags.
> > If I try this in the second repository:
> > git fetch -f ../main.git refs/heads/master:refs/heads/master
> > I'm getting also tags from other branches, if I have an old object from
> > one of the other branches.
> > I would expect to have only tags pointing to master ref. (Although the
> > man pages points to the behaviour regarding dangling objects). Is there
> > a way to avoid this? I only find --no-tags which results in having no
> > tags at all. Or need I git purge to remove the old objects first?
> > My goal is to fetch only specific branches and the related tags.
> 
> AFAIK, Git should only auto-follow tags that are reachable from the
> branches you fetch (in this case master). Are you saying that you get
> tags pointing to other history that is NOT reachable from the master
> branch? (i.e. are you getting tags for which "git merge-base $tag
> master" is not equal to "git rev-parse $tag")?
> 
exactly. I reproduced it by coping a object from an other branch to the
locale repository. This results in fetching the not reachable tags.

> Re-reading the man page, I do see the following:
> 
> "if the repository has objects that are pointed by remote tags that it
> does not yet have, then fetch those missing tags. If the other end has
> tags that point at branches you are not interested in, you will not
> get them."
> 
> This can be interpreted as saying that even unreachable objects in
> your local repo that are pointed to by some remote tag will cause that
> tag to be fetched, and in effect resuscitate the
> previously-unreachable object. If this is indeed how it works, I would
> be tempted to label this a bug in the auto-following behavior, as it's
> probably not what most people would expect. In that case, yes, you
Yes my first understanding of auto-following behaviour was wrong ;)
> should be able to get your desired behavior by first purging all
> unreachable objects. Something like "git gc --prune=now" should do the
> job.
Because I run this by scripts is there a way without porcelain commands?
I saw even git prune is one.
> 
> Hope this helps,
Thanks
Jan
> 
> ...Johan
> 

Re: git fetch refs and tags

[Johan Herland]

On Tue, Apr 23, 2013 at 1:45 PM, Jan Weitzel <J.Weitzel@phytec.de> wrote:
> Am Dienstag, den 23.04.2013, 13:25 +0200 schrieb Johan Herland:
>> On Tue, Apr 23, 2013 at 12:53 PM, Jan Weitzel <J.Weitzel@phytec.de> wrote:
>> > Hello,
>> > I have the following problem: I have 2 bare git repositories one has
>> > several branches and tags.
>> > If I try this in the second repository:
>> > git fetch -f ../main.git refs/heads/master:refs/heads/master
>> > I'm getting also tags from other branches, if I have an old object from
>> > one of the other branches.
>> > I would expect to have only tags pointing to master ref. (Although the
>> > man pages points to the behaviour regarding dangling objects). Is there
>> > a way to avoid this? I only find --no-tags which results in having no
>> > tags at all. Or need I git purge to remove the old objects first?
>> > My goal is to fetch only specific branches and the related tags.
>>
>> AFAIK, Git should only auto-follow tags that are reachable from the
>> branches you fetch (in this case master). Are you saying that you get
>> tags pointing to other history that is NOT reachable from the master
>> branch? (i.e. are you getting tags for which "git merge-base $tag
>> master" is not equal to "git rev-parse $tag")?
>>
> exactly. I reproduced it by coping a object from an other branch to the
> locale repository. This results in fetching the not reachable tags.
>
>> Re-reading the man page, I do see the following:
>>
>> "if the repository has objects that are pointed by remote tags that it
>> does not yet have, then fetch those missing tags. If the other end has
>> tags that point at branches you are not interested in, you will not
>> get them."
>>
>> This can be interpreted as saying that even unreachable objects in
>> your local repo that are pointed to by some remote tag will cause that
>> tag to be fetched, and in effect resuscitate the
>> previously-unreachable object. If this is indeed how it works, I would
>> be tempted to label this a bug in the auto-following behavior, as it's
>> probably not what most people would expect. In that case, yes, you
>
> Yes my first understanding of auto-following behaviour was wrong ;)
>
>> should be able to get your desired behavior by first purging all
>> unreachable objects. Something like "git gc --prune=now" should do the
>> job.
>
> Because I run this by scripts is there a way without porcelain commands?
> I saw even git prune is one.

git prune is not sufficient, since it only removes _unpacked_ and
unreachable objects. You first need to create a new pack that does not
contain unreachable objects (git rebase -a -d), but before that you
also need to expire reflogs (git reflog expire ...) and you should
also prune packed refs (git pack-refs --prune ...).

In short there are a number of commands you need to run, and in the
right order and with the right options, but "git gc --prune=now" is
basically just a wrapper around these that Does The Right Thing(tm). I
would just use that instead.

...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: git fetch refs and tags

[Junio C Hamano]

Jan Weitzel <J.Weitzel@phytec.de> writes:

> Hello,
> I have the following problem: I have 2 bare git repositories one has
> several branches and tags.
> If I try this in the second repository:
> git fetch -f ../main.git refs/heads/master:refs/heads/master 
> I'm getting also tags from other branches, if I have an old object from
> one of the other branches.
> I would expect to have only tags pointing to master ref.

A tag that points at a commit that is reachable from 'master' will
be followed, not just pointing _at_ 'master'.  For example, when
your 'master' is a bit beyond v1.2 release, it is likely you will
also get v1.1 and v1.0 tags (if your release structure is such that
new release contains everything old releases had).

Re: git fetch refs and tags

[Johan Herland]

On Tue, Apr 23, 2013 at 4:59 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jan Weitzel <J.Weitzel@phytec.de> writes:
>
>> Hello,
>> I have the following problem: I have 2 bare git repositories one has
>> several branches and tags.
>> If I try this in the second repository:
>> git fetch -f ../main.git refs/heads/master:refs/heads/master
>> I'm getting also tags from other branches, if I have an old object from
>> one of the other branches.
>> I would expect to have only tags pointing to master ref.
>
> A tag that points at a commit that is reachable from 'master' will
> be followed, not just pointing _at_ 'master'.  For example, when
> your 'master' is a bit beyond v1.2 release, it is likely you will
> also get v1.1 and v1.0 tags (if your release structure is such that
> new release contains everything old releases had).

That is all true, and behaves as expected. However, what Jan is observing
is (I believe) equivalent to the following:

Remote repo:

  o---o---o---o <---- master, tag: spam
   \      ^---- tag: eggs
    \
     o---o---o <---- tag: sausage

Local repo:

  o---o
   \  ^---- master
    \
     o---o---o (unreachable)

Now, when Jan fetches master, he gets the new master, plus tags spam and
eggs (as expected), but he ALSO gets tag sausage, which he _shouldn't_
get, since it is not reachable from master (hence not part of the fetched
history). The following session demonstrates the problem using the above
example:

# Prepare history in repo1
$ git init repo1
Initialized empty Git repository in /home/johan/git/repo1/.git/
$ cd repo1/
$ echo foo>foo
$ git add foo
$ git commit -m a
[master (root-commit) 92f92ec] a
 1 file changed, 1 insertion(+)
 create mode 100644 foo
$ echo bar >> foo
$ git commit -am b
[master e90d835] b
 1 file changed, 1 insertion(+)
$ echo baz >> foo
$ git commit -am c
[master 09e4bb2] c
 1 file changed, 1 insertion(+)
$ echo xyzzy >> foo
$ git commit -am d
[master 13ce4fb] d
 1 file changed, 1 insertion(+)
$ git checkout -b side HEAD~3
Switched to a new branch 'side'
$ echo side >> foo
$ git commit -am side
[side 7294fc9] side
 1 file changed, 1 insertion(+)

# Copy history to repo2, and make most of it unreachable
$ cd ..
$ git clone repo1 repo2
Cloning into 'repo2'...
done.
$ cd repo2/
$ git checkout -b master origin/master~2
Switched to a new branch 'master'
$ git remote rm origin
$ git branch -D side
Deleted branch side (was 7294fc9).
$ git show-ref
e90d8356350e4f2fd5e7bfb3a0327e2fa6115c2a refs/heads/master

# Create tags in repo1
$ cd ../repo1/
$ git checkout master
Switched to branch 'master'
$ git tag spam master
$ git tag eggs master^
$ git tag sausage side
$ git branch -D side
Deleted branch side (was 7294fc9).
$ git show-ref
13ce4fbc1761562a36f755b081b0d44b657c25a9 refs/heads/master
09e4bb21112b0f0acb82ed3b0fd6af9c685c3799 refs/tags/eggs
7294fc90fc1fbd1dd941c8999518d4a4b57048b4 refs/tags/sausage
13ce4fbc1761562a36f755b081b0d44b657c25a9 refs/tags/spam

# Fetch master into repo2
$ cd ../repo2
$ git checkout -b foo
Switched to a new branch 'foo'
$ git fetch -f ../repo1 refs/heads/master:refs/heads/master
From ../repo1
   e90d835..13ce4fb  master     -> master
 * [new tag]         eggs       -> eggs
 * [new tag]         sausage    -> sausage
 * [new tag]         spam       -> spam

# See that tag sausage was fetched even though it's not reachable from master
$ git show-ref
e90d8356350e4f2fd5e7bfb3a0327e2fa6115c2a refs/heads/foo
13ce4fbc1761562a36f755b081b0d44b657c25a9 refs/heads/master
09e4bb21112b0f0acb82ed3b0fd6af9c685c3799 refs/tags/eggs
7294fc90fc1fbd1dd941c8999518d4a4b57048b4 refs/tags/sausage
13ce4fbc1761562a36f755b081b0d44b657c25a9 refs/tags/spam
$ git log --graph --decorate --all --oneline
* 7294fc9 (tag: sausage) side
| * 13ce4fb (tag: spam, master) d
| * 09e4bb2 (tag: eggs) c
| * e90d835 (HEAD, foo) b
|/
* 92f92ec a


Have fun! :)

...Johan

--
Johan Herland, <johan@herland.net>
www.herland.net

Re: git fetch refs and tags

[Jan Weitzel]

Am Dienstag, den 23.04.2013, 22:50 +0200 schrieb Johan Herland:
> On Tue, Apr 23, 2013 at 4:59 PM, Junio C Hamano <gitster@pobox.com> wrote:
> > Jan Weitzel <J.Weitzel@phytec.de> writes:
> >
> >> Hello,
> >> I have the following problem: I have 2 bare git repositories one has
> >> several branches and tags.
> >> If I try this in the second repository:
> >> git fetch -f ../main.git refs/heads/master:refs/heads/master
> >> I'm getting also tags from other branches, if I have an old object from
> >> one of the other branches.
> >> I would expect to have only tags pointing to master ref.
> >
> > A tag that points at a commit that is reachable from 'master' will
> > be followed, not just pointing _at_ 'master'.  For example, when
> > your 'master' is a bit beyond v1.2 release, it is likely you will
> > also get v1.1 and v1.0 tags (if your release structure is such that
> > new release contains everything old releases had).
> 
> That is all true, and behaves as expected. However, what Jan is observing
> is (I believe) equivalent to the following:
> 
Thank you for the explanation, you show exactly what I mean :)
Jan
> Remote repo:
> 
>   o---o---o---o <---- master, tag: spam
>    \      ^---- tag: eggs
>     \
>      o---o---o <---- tag: sausage
> 
> Local repo:
> 
>   o---o
>    \  ^---- master
>     \
>      o---o---o (unreachable)
> 
> Now, when Jan fetches master, he gets the new master, plus tags spam and
> eggs (as expected), but he ALSO gets tag sausage, which he _shouldn't_
> get, since it is not reachable from master (hence not part of the fetched
> history). The following session demonstrates the problem using the above
> example:
> 
> # Prepare history in repo1
> $ git init repo1
> Initialized empty Git repository in /home/johan/git/repo1/.git/
> $ cd repo1/
> $ echo foo>foo
> $ git add foo
> $ git commit -m a
> [master (root-commit) 92f92ec] a
>  1 file changed, 1 insertion(+)
>  create mode 100644 foo
> $ echo bar >> foo
> $ git commit -am b
> [master e90d835] b
>  1 file changed, 1 insertion(+)
> $ echo baz >> foo
> $ git commit -am c
> [master 09e4bb2] c
>  1 file changed, 1 insertion(+)
> $ echo xyzzy >> foo
> $ git commit -am d
> [master 13ce4fb] d
>  1 file changed, 1 insertion(+)
> $ git checkout -b side HEAD~3
> Switched to a new branch 'side'
> $ echo side >> foo
> $ git commit -am side
> [side 7294fc9] side
>  1 file changed, 1 insertion(+)
> 
> # Copy history to repo2, and make most of it unreachable
> $ cd ..
> $ git clone repo1 repo2
> Cloning into 'repo2'...
> done.
> $ cd repo2/
> $ git checkout -b master origin/master~2
> Switched to a new branch 'master'
> $ git remote rm origin
> $ git branch -D side
> Deleted branch side (was 7294fc9).
> $ git show-ref
> e90d8356350e4f2fd5e7bfb3a0327e2fa6115c2a refs/heads/master
> 
> # Create tags in repo1
> $ cd ../repo1/
> $ git checkout master
> Switched to branch 'master'
> $ git tag spam master
> $ git tag eggs master^
> $ git tag sausage side
> $ git branch -D side
> Deleted branch side (was 7294fc9).
> $ git show-ref
> 13ce4fbc1761562a36f755b081b0d44b657c25a9 refs/heads/master
> 09e4bb21112b0f0acb82ed3b0fd6af9c685c3799 refs/tags/eggs
> 7294fc90fc1fbd1dd941c8999518d4a4b57048b4 refs/tags/sausage
> 13ce4fbc1761562a36f755b081b0d44b657c25a9 refs/tags/spam
> 
> # Fetch master into repo2
> $ cd ../repo2
> $ git checkout -b foo
> Switched to a new branch 'foo'
> $ git fetch -f ../repo1 refs/heads/master:refs/heads/master
> From ../repo1
>    e90d835..13ce4fb  master     -> master
>  * [new tag]         eggs       -> eggs
>  * [new tag]         sausage    -> sausage
>  * [new tag]         spam       -> spam
> 
> # See that tag sausage was fetched even though it's not reachable from master
> $ git show-ref
> e90d8356350e4f2fd5e7bfb3a0327e2fa6115c2a refs/heads/foo
> 13ce4fbc1761562a36f755b081b0d44b657c25a9 refs/heads/master
> 09e4bb21112b0f0acb82ed3b0fd6af9c685c3799 refs/tags/eggs
> 7294fc90fc1fbd1dd941c8999518d4a4b57048b4 refs/tags/sausage
> 13ce4fbc1761562a36f755b081b0d44b657c25a9 refs/tags/spam
> $ git log --graph --decorate --all --oneline
> * 7294fc9 (tag: sausage) side
> | * 13ce4fb (tag: spam, master) d
> | * 09e4bb2 (tag: eggs) c
> | * e90d835 (HEAD, foo) b
> |/
> * 92f92ec a
> 
> 
> Have fun! :)
> 
> ...Johan
> 
> --
> Johan Herland, <johan@herland.net>
> www.herland.net