git fetch with refspec does not include tags?

git fetch with refspec does not include tags?

[Carlsson, Magnus]

Hi​

In the git fetch documentation it states that by default you will fetch all tags that point into the history to the branches fetched.

"By default, any tag that points into the histories being fetched is also fetched; the effect is to fetch tags that point  at branches that you are interested in. This default behavior can be changed by using the --tags or --no-tags options or by configuring remote.<name>.tagOpt. By using a refspec that fetches tags explicitly, you can fetch tags that do not point into branches  you are interested in as well."

But for me I get tags if I do "git fetch" or "git fetch origin" but if I do "git fetch origin master" I don't get tags related to the master branch.

I understand that this might be due to me specifying a refspec and then it will only get that exact refspec, but for me it's not that clear from the documentation what I should expect. I read it as when I fetch something all related tags will come along.

Using: git version 2.13.0.rc1.15.gd2bbb7c

-- Magnus

MAGNUS CARLSSON
Staff Software Engineer
ARRIS

o: +46 13 36 75 92
e: magnus.carlsson@arris.com
w: www.arris.com

ARRIS:  Legal entity: Arris Sweden AB - Registered Office: Teknikringen 2, 583 30 Linkoping, Sweden - Reg No:556518-5831 - VAT No:SE 556518-583

This electronic transmission (and any attached document) is for the sole use of the individual or entity to whom it is addressed.  It is confidential and may be attorney-client privileged.  In any event the Sender reserves, to the fullest extent, any "legal  advice privilege".  Any further distribution or copying of this message is strictly prohibited.  If you received this message in error, please notify the Sender immediately and destroy the attached message (and all attached documents).
       

Re: git fetch with refspec does not include tags?

[Jeff King]

On Thu, Aug 17, 2017 at 09:02:52AM +0000, Carlsson, Magnus wrote:

> In the git fetch documentation it states that by default you will
> fetch all tags that point into the history to the branches fetched.
> 
> "By default, any tag that points into the histories being fetched is
> also fetched; the effect is to fetch tags that point  at branches that
> you are interested in. This default behavior can be changed by using
> the --tags or --no-tags options or by configuring
> remote.<name>.tagOpt. By using a refspec that fetches tags explicitly,
> you can fetch tags that do not point into branches  you are interested
> in as well."
> 
> But for me I get tags if I do "git fetch" or "git fetch origin" but if
> I do "git fetch origin master" I don't get tags related to the master
> branch.
> 
> I understand that this might be due to me specifying a refspec and
> then it will only get that exact refspec, but for me it's not that
> clear from the documentation what I should expect. I read it as when I
> fetch something all related tags will come along.

I'll admit that our tag-autofollow behavior has often confused me. So
I'll try to untangle what's happening at least if not the reasoning. :)

I think the problem is not that you have a refspec, but that your
refspec has no destination. Looking at the fetch code, we seem to turn
on autotags only when the destination is a "real" ref and not just the
default FETCH_HEAD. Which sort-of makes sense. If you're doing a one-off
into FETCH_HEAD, you probably don't want to create tags, even if you
have the objects they point to.

But this is further complicated by the opportunistic tracking-ref
updates.  You can see some interesting behavior with a setup like this:

  git init parent
  git -C parent commit --allow-empty -m one &&
  git -C parent tag -m foo mytag

  git init child
  cd child
  git remote add origin ../parent

and then:

  # no tags, we just populate FETCH_HEAD because of the bare URL
  git fetch ../parent

  # this does fetch tags, because we're storing the result according to
  # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
  git fetch origin

  # this doesn't fetch tags, as the main command is "just" populating
  # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
  # refs/remotes/origin/master, so let's update it on the side" kicks
  # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
  # not the tags. Weird.
  git fetch origin master

  # and this one does fetch tags, because we have a real destination.
  git fetch origin master:foo

So what I'd say is:

  1. Definitely these defaults are under-documented. I couldn't find
     them anywhere in git-fetch(1).

  2. If we continue to follow the "are we storing any refs" rule for the
     default, possibly it should expand to "did we store anything,
     including opportunistic tracking-ref updates".

-Peff

Re: git fetch with refspec does not include tags?

[Carlsson, Magnus]

Thanks for your quick answer!

>  1. Definitely these defaults are under-documented. I couldn't find
>    them anywhere in git-fetch(1).

Yes, some sort of explanation would be good, especially since one of the first sentences state that you always get relevant tags.

>  2. If we continue to follow the "are we storing any refs" rule for the
>     default, possibly it should expand to "did we store anything,
>     including opportunistic tracking-ref updates".

Personally I think that I would prefer to always fetch relevant tags. If I don't want tags I can simply use the "--no-tags"

-- Magnus


MAGNUS CARLSSON
Staff Software Engineer
ARRIS

o: +46 13 36 75 92
e: magnus.carlsson@arris.com
w: www.arris.com

ARRIS:  Legal entity: Arris Sweden AB - Registered Office: Teknikringen 2, 583 30 Linkoping, Sweden - Reg No:556518-5831 - VAT No:SE 556518-583

This electronic transmission (and any attached document) is for the sole use of the individual or entity to whom it is addressed.  It is confidential and may be attorney-client privileged.  In any event the Sender reserves, to the fullest extent, any "legal advice privilege".  Any further distribution or copying of this message is strictly prohibited.  If you received this message in error, please notify the Sender immediately and destroy the attached message (and all attached documents).

________________________________________
From: Jeff King <peff@peff.net>
Sent: Thursday, August 17, 2017 11:28
To: Carlsson, Magnus
Cc: git@vger.kernel.org
Subject: Re: git fetch with refspec does not include tags?

On Thu, Aug 17, 2017 at 09:02:52AM +0000, Carlsson, Magnus wrote:

> In the git fetch documentation it states that by default you will
> fetch all tags that point into the history to the branches fetched.
>
> "By default, any tag that points into the histories being fetched is
> also fetched; the effect is to fetch tags that point  at branches that
> you are interested in. This default behavior can be changed by using
> the --tags or --no-tags options or by configuring
> remote.<name>.tagOpt. By using a refspec that fetches tags explicitly,
> you can fetch tags that do not point into branches  you are interested
> in as well."
>
> But for me I get tags if I do "git fetch" or "git fetch origin" but if
> I do "git fetch origin master" I don't get tags related to the master
> branch.
>
> I understand that this might be due to me specifying a refspec and
> then it will only get that exact refspec, but for me it's not that
> clear from the documentation what I should expect. I read it as when I
> fetch something all related tags will come along.

I'll admit that our tag-autofollow behavior has often confused me. So
I'll try to untangle what's happening at least if not the reasoning. :)

I think the problem is not that you have a refspec, but that your
refspec has no destination. Looking at the fetch code, we seem to turn
on autotags only when the destination is a "real" ref and not just the
default FETCH_HEAD. Which sort-of makes sense. If you're doing a one-off
into FETCH_HEAD, you probably don't want to create tags, even if you
have the objects they point to.

But this is further complicated by the opportunistic tracking-ref
updates.  You can see some interesting behavior with a setup like this:

  git init parent
  git -C parent commit --allow-empty -m one &&
  git -C parent tag -m foo mytag

  git init child
  cd child
  git remote add origin ../parent

and then:

  # no tags, we just populate FETCH_HEAD because of the bare URL
  git fetch ../parent

  # this does fetch tags, because we're storing the result according to
  # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
  git fetch origin

  # this doesn't fetch tags, as the main command is "just" populating
  # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
  # refs/remotes/origin/master, so let's update it on the side" kicks
  # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
  # not the tags. Weird.
  git fetch origin master

  # and this one does fetch tags, because we have a real destination.
  git fetch origin master:foo

So what I'd say is:

  1. Definitely these defaults are under-documented. I couldn't find
     them anywhere in git-fetch(1).

  2. If we continue to follow the "are we storing any refs" rule for the
     default, possibly it should expand to "did we store anything,
     including opportunistic tracking-ref updates".

-Peff

Re: git fetch with refspec does not include tags?

[Jeff King]

On Thu, Aug 17, 2017 at 11:29:48AM +0000, Carlsson, Magnus wrote:

> >  2. If we continue to follow the "are we storing any refs" rule for the
> >     default, possibly it should expand to "did we store anything,
> >     including opportunistic tracking-ref updates".
> 
> Personally I think that I would prefer to always fetch relevant tags.
> If I don't want tags I can simply use the "--no-tags"

I think it's a bit more complex because "git pull" uses "git fetch"
under the hood. In fact, your "git fetch origin master" is exactly what
gets run when you do:

  git pull origin master

That's maybe OK. But I think one-off pulls like:

  git pull https://example.com/repo.git master

probably wouldn't want it. I'd have to give it some thought.

-Peff

Re: git fetch with refspec does not include tags?

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> I think it's a bit more complex because "git pull" uses "git fetch"
> under the hood. In fact, your "git fetch origin master" is exactly what
> gets run when you do:
>
>   git pull origin master
>
> That's maybe OK. But I think one-off pulls like:
>
>   git pull https://example.com/repo.git master
>
> probably wouldn't want it. I'd have to give it some thought.

I agree with both.  If you have named remote, you presumably are
keeping copies of their branches as remote-tracking branches, and it
may be fine to follow tags.  An explicit URL used for one-off should
not grab anything but the named thing, I would think.

Re: git fetch with refspec does not include tags?

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

>   # no tags, we just populate FETCH_HEAD because of the bare URL
>   git fetch ../parent
>
>   # this does fetch tags, because we're storing the result according to
>   # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
>   git fetch origin

The above two look good.

>   # this doesn't fetch tags, as the main command is "just" populating
>   # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
>   # refs/remotes/origin/master, so let's update it on the side" kicks
>   # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
>   # not the tags. Weird.
>   git fetch origin master

Yes, it looks weird, but I suspect that it is probably more correct
not to fetch tags in this case.  I wonder if it would be a solution
not to do the "on the side" thing---after all the user didn't tell
us to update refs/remotes/origin/master with this command line.

Then not following tags will be in line with all of the above
reasoning above.

>   # and this one does fetch tags, because we have a real destination.
>   git fetch origin master:foo

Yup, that is expected.

> So what I'd say is:
>
>   1. Definitely these defaults are under-documented. I couldn't find
>      them anywhere in git-fetch(1).

Yup.

>   2. If we continue to follow the "are we storing any refs" rule for the
>      default, possibly it should expand to "did we store anything,
>      including opportunistic tracking-ref updates".

That also is a workable way to make things consistent.

Re: git fetch with refspec does not include tags?

[Kevin Daudt]

On Thu, Aug 17, 2017 at 12:38:58PM -0700, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
> 
> >   # no tags, we just populate FETCH_HEAD because of the bare URL
> >   git fetch ../parent
> >
> >   # this does fetch tags, because we're storing the result according to
> >   # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
> >   git fetch origin
> 
> The above two look good.
> 
> >   # this doesn't fetch tags, as the main command is "just" populating
> >   # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
> >   # refs/remotes/origin/master, so let's update it on the side" kicks
> >   # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
> >   # not the tags. Weird.
> >   git fetch origin master
> 
> Yes, it looks weird, but I suspect that it is probably more correct
> not to fetch tags in this case.  I wonder if it would be a solution
> not to do the "on the side" thing---after all the user didn't tell
> us to update refs/remotes/origin/master with this command line.

Isn't that how git fetch used to behave, or am I misunderstanding what
you mean? It used to be that git fetch <remote> <branch> would not
update any remote tracking branches.

From the 1.8.4 release notes:

> "git fetch origin master" unlike "git fetch origin" or "git fetch"
> did not update "refs/remotes/origin/master"; this was an early
> design decision to keep the update of remote tracking branches
> predictable, but in practice it turns out that people find it more
> convenient to opportunistically update them whenever we have a
> chance, and we have been updating them when we run "git push" which
> already breaks the original "predictability" anyway.

Re: git fetch with refspec does not include tags?

[Junio C Hamano]

Kevin Daudt <me@ikke.info> writes:

> On Thu, Aug 17, 2017 at 12:38:58PM -0700, Junio C Hamano wrote:
>> Jeff King <peff@peff.net> writes:
>> 
>> >   # no tags, we just populate FETCH_HEAD because of the bare URL
>> >   git fetch ../parent
>> >
>> >   # this does fetch tags, because we're storing the result according to
>> >   # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
>> >   git fetch origin
>> 
>> The above two look good.
>> 
>> >   # this doesn't fetch tags, as the main command is "just" populating
>> >   # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
>> >   # refs/remotes/origin/master, so let's update it on the side" kicks
>> >   # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
>> >   # not the tags. Weird.
>> >   git fetch origin master
>> 
>> Yes, it looks weird, but I suspect that it is probably more correct
>> not to fetch tags in this case.  I wonder if it would be a solution
>> not to do the "on the side" thing---after all the user didn't tell
>> us to update refs/remotes/origin/master with this command line.
>
> Isn't that how git fetch used to behave, or am I misunderstanding what
> you mean? It used to be that git fetch <remote> <branch> would not
> update any remote tracking branches.
>
> From the 1.8.4 release notes:
>
>> "git fetch origin master" unlike "git fetch origin" or "git fetch"
>> did not update "refs/remotes/origin/master"; this was an early
>> design decision to keep the update of remote tracking branches
>> predictable, but in practice it turns out that people find it more
>> convenient to opportunistically update them whenever we have a
>> chance, and we have been updating them when we run "git push" which
>> already breaks the original "predictability" anyway.

No, you are not misunderstanding anything.  The "pretend that we
immediately turned around and fetched" done by "git push" was
already breaking the predictability, but the change in 1.8.4 made it
even worse.  I am saying that going back to the old behaviour may be
one option to address the issue being discussed in this thread.

Re: git fetch with refspec does not include tags?

[Kevin Daudt]

On Thu, Aug 17, 2017 at 01:38:36PM -0700, Junio C Hamano wrote:
> Kevin Daudt <me@ikke.info> writes:
> 
> > On Thu, Aug 17, 2017 at 12:38:58PM -0700, Junio C Hamano wrote:
> >> Jeff King <peff@peff.net> writes:
> >> 
> >> >   # no tags, we just populate FETCH_HEAD because of the bare URL
> >> >   git fetch ../parent
> >> >
> >> >   # this does fetch tags, because we're storing the result according to
> >> >   # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
> >> >   git fetch origin
> >> 
> >> The above two look good.
> >> 
> >> >   # this doesn't fetch tags, as the main command is "just" populating
> >> >   # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
> >> >   # refs/remotes/origin/master, so let's update it on the side" kicks
> >> >   # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
> >> >   # not the tags. Weird.
> >> >   git fetch origin master
> >> 
> >> Yes, it looks weird, but I suspect that it is probably more correct
> >> not to fetch tags in this case.  I wonder if it would be a solution
> >> not to do the "on the side" thing---after all the user didn't tell
> >> us to update refs/remotes/origin/master with this command line.
> >
> > Isn't that how git fetch used to behave, or am I misunderstanding what
> > you mean? It used to be that git fetch <remote> <branch> would not
> > update any remote tracking branches.
> >
> > From the 1.8.4 release notes:
> >
> >> "git fetch origin master" unlike "git fetch origin" or "git fetch"
> >> did not update "refs/remotes/origin/master"; this was an early
> >> design decision to keep the update of remote tracking branches
> >> predictable, but in practice it turns out that people find it more
> >> convenient to opportunistically update them whenever we have a
> >> chance, and we have been updating them when we run "git push" which
> >> already breaks the original "predictability" anyway.
> 
> No, you are not misunderstanding anything.  The "pretend that we
> immediately turned around and fetched" done by "git push" was
> already breaking the predictability, but the change in 1.8.4 made it
> even worse.  I am saying that going back to the old behaviour may be
> one option to address the issue being discussed in this thread.
> 

Ok. The reason I'm bring this up is because we often had to tell users
in the irc channel that git fetch <remote> <branch> did not update the
remote tracking branches, which confused them, so reverting back might
reintroduce this confusion again.

Re: git fetch with refspec does not include tags?

[Jeff King]

On Thu, Aug 17, 2017 at 10:43:12PM +0200, Kevin Daudt wrote:

> > No, you are not misunderstanding anything.  The "pretend that we
> > immediately turned around and fetched" done by "git push" was
> > already breaking the predictability, but the change in 1.8.4 made it
> > even worse.  I am saying that going back to the old behaviour may be
> > one option to address the issue being discussed in this thread.
> 
> Ok. The reason I'm bring this up is because we often had to tell users
> in the irc channel that git fetch <remote> <branch> did not update the
> remote tracking branches, which confused them, so reverting back might
> reintroduce this confusion again.

Yeah, I don't think we want to go back to the original behavior. I agree
that it is partially to blame for the inconsistency that started this
thread, but I think on balance it has saved much more confusion than it
has started. And we can address that inconsistency with better tag rules
(like the "autofollow if we wrote any real refs" thing).

I don't have a patch for that yet, so if anybody feels like taking a
look, it would be much appreciated.

-Peff

Re: git fetch with refspec does not include tags?

[Jeff King]

On Sun, Aug 20, 2017 at 03:47:28AM -0400, Jeff King wrote:

> Yeah, I don't think we want to go back to the original behavior. I agree
> that it is partially to blame for the inconsistency that started this
> thread, but I think on balance it has saved much more confusion than it
> has started. And we can address that inconsistency with better tag rules
> (like the "autofollow if we wrote any real refs" thing).
> 
> I don't have a patch for that yet, so if anybody feels like taking a
> look, it would be much appreciated.

Also: I don't think we've seen a patch yet for documenting the current
auto-follow behavior.  Even if we don't make a behavior change, let's
not forget to improve that, which should be much less work. :)

-Peff

Re: git fetch with refspec does not include tags?

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Sun, Aug 20, 2017 at 03:47:28AM -0400, Jeff King wrote:
>
>> Yeah, I don't think we want to go back to the original behavior. I agree
>> that it is partially to blame for the inconsistency that started this
>> thread, but I think on balance it has saved much more confusion than it
>> has started. And we can address that inconsistency with better tag rules
>> (like the "autofollow if we wrote any real refs" thing).
>> 
>> I don't have a patch for that yet, so if anybody feels like taking a
>> look, it would be much appreciated.
>
> Also: I don't think we've seen a patch yet for documenting the current
> auto-follow behavior.  Even if we don't make a behavior change, let's
> not forget to improve that, which should be much less work. :)

Thanks for two thoughtful follow-ups.