git-fetch: default globally to --no-tags

git-fetch: default globally to --no-tags

[Brian Norris]

Hi,

I have one main concern, and I can envision a few ways that git could
solve it. I also see that there has been some previous discussion on
reltaed topics over the years, but as far as I can tell, nothing has
actually been implemented to address my concern.

--- My concern ---

When I fetch from a remote repository, the only ways to
prevent fetching tags are:

  1) git fetch --no-tags <name>

  2) git config remote.<name>.tagopt --no-tags; git fetch <name>

The former requires extra typing for a case that (arguably) should be
the default. The latter requires configuration for each repository. This
doesn't scale well, if a user desires this behavior on all repositories
they use.

I'd prefer something like this, to change the default tag-fetching
behavior globally:

  git config --global remote.tagopt --no-tags

The closest I see is that I can disable tag-fetching globally on a
per-remote name basis:

  git config --global remote.<name>.tagopt --no-tags

This brings me to a potential bug report:

--- Bug ---

When trying to use the remote.<name>.tagopt configuration option
globally, I get something like this:

  $ git config --global remote.test.tagopt --no-tags
  $ git remote update
  ...
  Fetching test
  fatal: 'test' does not appear to be a git repository
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
  error: Could not fetch test
  ...

Expected behavior: if the local repository does not have a remote named
'test', then no additional output should be printed. If the local
repository has a remote named test, then it should be fetched with the
--no-tags option.

Actual behavior: git prints warnings about the 'test' remote, just
because there is no remote named 'test.'

--- Motivations ---

This is all motivated by the fact that tag namespacing is completely
broken in git. Tags are globally namespaced, and in a true DVCS
environment, any particular developer has no control over another
developer's tag naming conventions. So this namespace can easily become
polluted, reducing the usefulness of tags as a whole [1]. This problem
seems to have been acknowledged, and proposals appeared a few years ago
[2]. But I don't see any solution for tag namespacing.

So, my current approach is to try to limit the damage done by
accidentally fetching tags. Unfortunately, I can't find any good
supported mechanisms to help me in the previous two sections.

--- TL;DR ---

My email boils down to two questions:

  (A) Has there been progress on implementing a proposal like in [2]?

  (B) Can we allow disabling (auto)tag-fetching globally? Like:

        git config --global remote.tagopt --no-tags

Thanks,
Brian

[1] I could elaborate with some horror stories, to describe how this
    becomes a day-to-day nuisance in using git for me, but I'll avoid
    the details for now. Please accept that this is a usability bug.

[2]
    http://article.gmane.org/gmane.comp.version-control.git/165799
    http://article.gmane.org/gmane.comp.version-control.git/165885

Re: git-fetch: default globally to --no-tags

[Jeff King]

On Tue, Nov 18, 2014 at 07:05:23PM -0800, Brian Norris wrote:

> When I fetch from a remote repository, the only ways to
> prevent fetching tags are:
> 
>   1) git fetch --no-tags <name>
> 
>   2) git config remote.<name>.tagopt --no-tags; git fetch <name>

Yes, there is no way to globally specify --no-tags. I don't see it as a
problem to add something like that.

> The former requires extra typing for a case that (arguably) should be
> the default.

I don't understand this assumption, though. Why don't you want to fetch
tags from the remote side? Wouldn't this break something as simple as:

  git clone git://git.kernel.org/pub/scm/git/git.git
  cd git
  git checkout v2.1.3

(Actually, no, it wouldn't, because clone side-steps the usual
tag-following procedure, but wouldn't you expect the same thing to work
after a fetch in an existing repository?)

(And now having read all the way to the bottom of your email, I at least
see why you dislike tags. But I am not convinced that turning them off
completely does not come with its own set of usability problems).

> I'd prefer something like this, to change the default tag-fetching
> behavior globally:
> 
>   git config --global remote.tagopt --no-tags

Yeah, that would make sense to me (I do not know _why_ you would want to
do it, but it would be consistent with other parts of git). If you are
adding a new config option, though, please do not follow the odd and
confusing "tagopt" config value, and make a boolean remote.autofollowtags
or something.

> --- Bug ---
> 
> When trying to use the remote.<name>.tagopt configuration option
> globally, I get something like this:
> 
>   $ git config --global remote.test.tagopt --no-tags
>   $ git remote update
>   ...
>   Fetching test
>   fatal: 'test' does not appear to be a git repository
>   fatal: Could not read from remote repository.
> 
>   Please make sure you have the correct access rights
>   and the repository exists.
>   error: Could not fetch test
>   ...
> 
> Expected behavior: if the local repository does not have a remote named
> 'test', then no additional output should be printed. If the local
> repository has a remote named test, then it should be fetched with the
> --no-tags option.
> 
> Actual behavior: git prints warnings about the 'test' remote, just
> because there is no remote named 'test.'

That is the expected behavior (sort of). Any time you define a single
key in a remote, that remote "exists", whether there is a remote.*.url
key for it or not. You can use that to set options for remotes you
specify only as a URL, though I do not know how widely-used that feature
is.

IOW, git is not complaining here that there is no remote named 'test'.
It is complaining that you have defined a remote called 'test', whose
URL defaults to 'test' (because you did not specify one in the config),
and that there is no repo called test. If you defined it as:

  git config --global remote.git://github.com/git/git.tagopt --no-tags

that would "work" in the sense that "git remote update" would download
git in every repository. ;)

I think one could argue that "git remote update" should probably skip
remotes for which no url is defined (but we _do_ still want to treat
them as remotes internally, so that when you "git fetch ...", we apply
the config correctly).

> --- Motivations ---
> 
> This is all motivated by the fact that tag namespacing is completely
> broken in git. Tags are globally namespaced, and in a true DVCS
> environment, any particular developer has no control over another
> developer's tag naming conventions. So this namespace can easily become
> polluted, reducing the usefulness of tags as a whole [1]. This problem
> seems to have been acknowledged, and proposals appeared a few years ago
> [2]. But I don't see any solution for tag namespacing.

I don't know if I would say "completely broken", as it mostly gets the
job done. But yes, there are other ways of doing it that would have
advantages.

> My email boils down to two questions:
> 
>   (A) Has there been progress on implementing a proposal like in [2]?

Not really, no. If you'd like to work on it, I'm sure people would be
happy to review patches.

>   (B) Can we allow disabling (auto)tag-fetching globally? Like:
> 
>         git config --global remote.tagopt --no-tags

That seems reasonable to me to use remote.* as defaults for all remotes.

-Peff

Re: git-fetch: default globally to --no-tags

[Junio C Hamano]

Brian Norris <computersforpeace@gmail.com> writes:

> --- TL;DR ---

You usually have TL;DR at the beginning to help people save time;
having it at the end forces the whole thing to be read and would not
help anybody. ;-)

> My email boils down to two questions:
>
>   (A) Has there been progress on implementing a proposal like in [2]?

I do not think so, and also I do not agree that "mirror everybody
else's ref hierarchy into separate namespaces" is necessarily a good
idea, especially for tags, whose reason of existence is to give
people a way to have anchoring points they agree on to have a shared
world view necessary to move things forward.

In other words, talks in [2] are attempting to solve a wrong
problem.  The problem people want to solve is to have a mechanism to
keep private anchoring points that are not necessarily shared
project wide, which tags in refs/tags hierarchy is *not*.

Like it or not, tags are meant to be used for globally shared
anchoring points and the whole machinery (e.g. "fetch" that
auto-follows tags, "clone" that gives refs/tags*:refs/tags/*
refspec) is geared towards supporting that use pattern, which will
be broken by moving tags to per-remote namespace.

I can see "git tag --local foo" that creates refs/local-tags/foo
and also adding a mechanism to propagate local-tags/ hierarchy just
like heads/ hierarchy is propagated per-remote as a solution to that
problem that does not break the "release tags" use case, though.

>   (B) Can we allow disabling (auto)tag-fetching globally? Like:
>
>         git config --global remote.tagopt --no-tags

Using remote.<variable> as a fallback for remote.<remote>.<variable>
may be a useful addition, not limited to <variable>==tagopt case.

This is a tangent, but it is an important one because we are talking
about "tagopt" specifically.  I think we should start deprecating
"*.tagopt --[no-]tags".  It started as a quick-and-dirty hack back
when "git fetch" was a shell script Porcelain, where it made it easy
to write things like this in its implementation:

	tagOpt=$(git config "remote.$name.tagopt")
        git fetch $tagOpt $name $args

which gives an impression that any command line option can go there
(e.g.  as if you could set "remote.*.tagopt = --frotz --no-tags")
and "git fetch" implementation, even after it is redone in C, must
forever parse it as if it is part of a shell command line
(e.g. splitting at $IFS, unquoting the shell quotes and interpreting
as if they came in argv[]).

This is ugly and simply unmaintainable, and we should transition
away from that, by doing something like:

 (1) Add remote.*.tags configuration, which defaults to 'follow',
     but can be set to 'true' or 'false'.  Accept '--tags' as a
     synonym to 'true' and '--no-tags' as a synonym to 'false'.

     * when set to 'follow', allow auto-following tags (the default).
     * when set to 'true', act as if --tags is given.
     * when set to 'false', act as if --no-tags is given.

 (2) Deprecate remote.*.tagopt configuration.  When it is used, give
     a warning about deprecation and encourage users to move to
     remote.*.tags setting.

 (3) Wait for several release cycles.

 (4) Remove remote.*.tags configuration support at a major version
     boundary.

Needless to say, support for remote.<variable> as a fallback for
remote.<remote>.<variable> for any <variable> can be done in
parallel to this tangent topic.

Re: git-fetch: default globally to --no-tags

[Jeff King]

On Wed, Nov 19, 2014 at 10:45:48AM -0800, Junio C Hamano wrote:

> > My email boils down to two questions:
> >
> >   (A) Has there been progress on implementing a proposal like in [2]?
> 
> I do not think so, and also I do not agree that "mirror everybody
> else's ref hierarchy into separate namespaces" is necessarily a good
> idea, especially for tags, whose reason of existence is to give
> people a way to have anchoring points they agree on to have a shared
> world view necessary to move things forward.
> 
> In other words, talks in [2] are attempting to solve a wrong
> problem.  The problem people want to solve is to have a mechanism to
> keep private anchoring points that are not necessarily shared
> project wide, which tags in refs/tags hierarchy is *not*.
> 
> Like it or not, tags are meant to be used for globally shared
> anchoring points and the whole machinery (e.g. "fetch" that
> auto-follows tags, "clone" that gives refs/tags*:refs/tags/*
> refspec) is geared towards supporting that use pattern, which will
> be broken by moving tags to per-remote namespace.
> 
> I can see "git tag --local foo" that creates refs/local-tags/foo
> and also adding a mechanism to propagate local-tags/ hierarchy just
> like heads/ hierarchy is propagated per-remote as a solution to that
> problem that does not break the "release tags" use case, though.

I am not sure I agree here that the discussions in [2] were not handling
this case. Here you are arguing for the tag-writer to distinguish
between two separate namespaces: global and local.

But I think the proposals in [2] were about pushing that logic into the
lookup phase. That is, pulling in all of the remote's tags as
"refs/remotes/origin/tags/*", and then at lookup time checking multiple
locations for tags (and preferring your local "refs/tags" to what is
pulled from a remote).

I think that system is better because it gives flexibility in resolution
to the viewer of the tags, not the writer. E.g., consider a project that
is merging two different sub-projects, project1 and project2. Each of
the sub-projects has their own global namespace with v1.0, v2.0, etc.
They would never use local-tags; these are meant to be in a per-project
global namespace.

But the superproject is pulling them both together; if it uses
refs/tags, the global namespaces will clash. Instead, it would be more
convenitn to have refs/remotes/project1/tags and so on. And then a
lookup of "v1.0" can either prefer project1 to project2, vice-versa, or
even respect neither. But the point is that the desired outcome is in
the eye of the beholder, not the writer.

> This is a tangent, but it is an important one because we are talking
> about "tagopt" specifically.  I think we should start deprecating
> "*.tagopt --[no-]tags".

Thanks for writing this out. I touched on it in the other email I sent,
but did not explain it very well. The transition you mentioned here is
exactly what I was thinking of.

-Peff

Re: git-fetch: default globally to --no-tags

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Wed, Nov 19, 2014 at 10:45:48AM -0800, Junio C Hamano wrote:
>
>> ... tags are meant to be used for globally shared
>> anchoring points and the whole machinery (e.g. "fetch" that
>> auto-follows tags, "clone" that gives refs/tags*:refs/tags/*
>> refspec) is geared towards supporting that use pattern, which will
>> be broken by moving tags to per-remote namespace.
>> 
>> I can see "git tag --local foo" that creates refs/local-tags/foo
>> and also adding a mechanism to propagate local-tags/ hierarchy just
>> like heads/ hierarchy is propagated per-remote as a solution to that
>> problem that does not break the "release tags" use case, though.
>
> I am not sure I agree here that the discussions in [2] were not handling
> this case. Here you are arguing for the tag-writer to distinguish
> between two separate namespaces: global and local.
>
> But I think the proposals in [2] were about pushing that logic into the
> lookup phase. That is, pulling in all of the remote's tags as
> "refs/remotes/origin/tags/*", and then at lookup time checking multiple
> locations for tags (and preferring your local "refs/tags" to what is
> pulled from a remote).

With a separate local-tags hierarchy, the look-up part still has to
be enhanced.  After doing "git tag v2.0" and "git tag -l snapshot00",
you would want to be able to say "git log snapshot00..v2.0" and have
these found.

If you don't allow a private local-tags hierarchy, then those who
make releases are burdened to be very careful not to contaminate
their public repository --- "git tag snapshot00" cannot be used by
them lightly just to mark their private state, if their day
typically is concluded with "git push --follow-tags", as that will
push out the "tags" that are meant to be private.

Aside from the mantra that it is a good thing to reduce the workload
for people closer to the center of the project (hence likely to be
bottleneck to slow down others) and a separate local-tags hierarchy
is a way to achieve that, a separate local-tags hierarchy also helps
people who do not make releases by letting them easily differenciate
the global tags they get from the project and the tags they and
other project participants create as private markers (their own will
be found in refs/local-tags, others' in refs/remotes/$pal/local-tags).

> ...
> But the superproject is pulling them both together; if it uses
> refs/tags, the global namespaces will clash. Instead, it would be more
> convenitn to have refs/remotes/project1/tags and so on.

Yeah, but isn't it an orthogonal issue?  refs/tags/project{1,2}/*
would be what I would recommend to use for "global" stuff whose
purpose is to give people a shared world view.

Re: git-fetch: default globally to --no-tags

[Jeff King]

On Wed, Nov 19, 2014 at 12:22:36PM -0800, Junio C Hamano wrote:

> With a separate local-tags hierarchy, the look-up part still has to
> be enhanced.  After doing "git tag v2.0" and "git tag -l snapshot00",
> you would want to be able to say "git log snapshot00..v2.0" and have
> these found.
> 
> If you don't allow a private local-tags hierarchy, then those who
> make releases are burdened to be very careful not to contaminate
> their public repository --- "git tag snapshot00" cannot be used by
> them lightly just to mark their private state, if their day
> typically is concluded with "git push --follow-tags", as that will
> push out the "tags" that are meant to be private.

It's not that I want to disallow a private local-tags hierarchy. It's
just that I consider it a completely orthogonal feature. In other words,
the problem of tags in a global namespace is not solved at all by
local-tags. It just helps people keep things out of the global namespace
that they did not want to be there in the first place[1]. It does
nothing for viewers who need to coalesce multiple global tag namespaces
(either from multiple projects, or "proposals" of global tags they have
pulled in from other non-canonical repositories).

> > But the superproject is pulling them both together; if it uses
> > refs/tags, the global namespaces will clash. Instead, it would be more
> > convenitn to have refs/remotes/project1/tags and so on.
> 
> Yeah, but isn't it an orthogonal issue?  refs/tags/project{1,2}/*
> would be what I would recommend to use for "global" stuff whose
> purpose is to give people a shared world view.

How do they get that split? They cannot use tag auto-following, which
puts the tags in the global refs/tags.

Certainly it can be done with git _today_ by setting up the appropriate
refspecs. I think this is more about making things useful out of the
box. 

-Peff

[1] I am actually not convinced that the mixing of local and global tags
    is a huge problem in practice. We do not push tags by default, so
    local tags tend to stay local. OTOH, I think the world would be a
    better place if "git push --follow-tags" were the default, which
    would entail somehow separating global from local tags.