How to create tags outside of refs/tags?

How to create tags outside of refs/tags?

[Christian Halstrick]

I would like to create a tag which full name is not starting with
refs/tags. Is that possible with 'git tag'? If not, is there any
other command to create such tags?

I would like to create such tags to mark released states of my sources
which should never be garbage collected in case no branch is pointing
them anymore. On the other hand these tags should not pollute the
namespace of normal tags, means: don't want a 'git tag' command to
list those technical tags.

Ciao
 Chris

Re: How to create tags outside of refs/tags?

[Jay Soffian]

On Mon, Feb 21, 2011 at 12:21 PM, Christian Halstrick
<christian.halstrick@gmail.com> wrote:
> I would like to create a tag which full name is not starting with
> refs/tags. Is that possible with 'git tag'?

Nope.

> If not, is there any other command to create such tags?

If it's just a light-weight tag, you can use the low-level (plumbing)
command update-ref to create an arbitrary ref:

$ git update-ref refs/blargh HEAD

If it's an annotated/signed tag you want, you'll have to first create
the tag object with 'git mktag', then use update-ref to create a ref
to that tag.

See git-tag.sh in the contrib directory of git.git.

HTH,

j.

Re: How to create tags outside of refs/tags?

[Junio C Hamano]

Jay Soffian <jaysoffian@gmail.com> writes:

> On Mon, Feb 21, 2011 at 12:21 PM, Christian Halstrick
> <christian.halstrick@gmail.com> wrote:
>> I would like to create a tag which full name is not starting with
>> refs/tags. Is that possible with 'git tag'?
>
> Nope.
>
>> If not, is there any other command to create such tags?
>
> If it's just a light-weight tag, you can use the low-level (plumbing)
> command update-ref to create an arbitrary ref:

That is a very unfriendly answer.  You shouldn't be answering everybody
who ask "How do I shoot my foot off?  git-foo command does not seem to let
me do so" with "Here is how to do that".

> $ git update-ref refs/blargh HEAD

At least, please say "that thing is no longer a tag, many git commands
would complain because refs/blargh does not have two-levels, and do not
ever think about starting that thing without 'refs/', or your 'gc' may end
up losing that object" when you give a potentially dangerous hint like
that (the last one being the most important).

Re: How to create tags outside of refs/tags?

[Christian Halstrick]

But look at my intention:

On Mon, Feb 21, 2011 at 18:21, Christian Halstrick
<christian.halstrick@gmail.com> wrote:
> I would like to create such tags to mark released states of my sources
> which should never be garbage collected in case no branch is pointing
> them anymore. On the other hand these tags should not pollute the
> namespace of normal tags, means: don't want a 'git tag' command to
> list those technical tags.

Is there any better way to achieve that certain commits are not
garbage collected than to create such tags. I love my foot and don't
want to "shoot if off" but I want to make sure the the
"release-process-machinery" in our company can be sure that certain
source states are never gc'ed. I can also live with creating normal
tags 'refs/tags/release/1.20' but would prefer a solution where 'git
tags' shows only developer created tags.

Will any ref underneath "refs/' prevent gc to garbage collect a
commit? Or only those in the well-known locations (e.g. refs/tags/*)

Re: How to create tags outside of refs/tags?

[Jeff King]

On Tue, Feb 22, 2011 at 07:17:20AM +0100, Christian Halstrick wrote:

> On Mon, Feb 21, 2011 at 18:21, Christian Halstrick
> <christian.halstrick@gmail.com> wrote:
> > I would like to create such tags to mark released states of my sources
> > which should never be garbage collected in case no branch is pointing
> > them anymore. On the other hand these tags should not pollute the
> > namespace of normal tags, means: don't want a 'git tag' command to
> > list those technical tags.
> 
> Is there any better way to achieve that certain commits are not
> garbage collected than to create such tags. I love my foot and don't
> want to "shoot if off" but I want to make sure the the
> "release-process-machinery" in our company can be sure that certain
> source states are never gc'ed. I can also live with creating normal
> tags 'refs/tags/release/1.20' but would prefer a solution where 'git
> tags' shows only developer created tags.

It sounds a lot saner to me to fix "git tag", then, to ignore certain
uninteresting bits of the tag namespace. We already do allow pattern
matching the tags shown, but:

  1. It's an fnmatch, so I don't think there is a way to do a negative
     match like "ignore everything that matches release/*".

  2. It can only be specified on the command-line, whereas obviously you
     would want this as the default for a repo when you type "git tag".

So I think we could do something like the patch below, which allows:

  # set up some boring and interesting tags
  for i in boring more-boring interesting; do
    for j in one two three; do
      git tag $i/$j
    done
  done

  # tell git to ignore boring stuff
  git config tag.ignore 'boring/*'
  git config --add tag.ignore 'more-boring/*'

  # only interesting tags are shown
  git tag

  # only interesting tags are checked for --contains
  git tag --contains HEAD

  # all tags are shown
  git tag --no-ignore

One alternative would be to allow specifying a default pattern in the
config, and then some way of marking it as a regex instead of fnmatch.
Then you could (in theory) construct a regex that negatively matches
your boring refs. Although without perl-compatibile negative look-ahead,
such regexes can get pretty hard to write.

Anyway, here's what the patch looks like. Obviously missing docs and
tests.

---
diff --git a/builtin/tag.c b/builtin/tag.c
index 46f7138..629dfa3 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -12,6 +12,7 @@
 #include "tag.h"
 #include "run-command.h"
 #include "parse-options.h"
+#include "string-list.h"
 
 static const char * const git_tag_usage[] = {
 	"git tag [-a|-s|-u <key-id>] [-f] [-m <msg>|-F <file>] <tagname> [<head>]",
@@ -27,8 +28,11 @@ struct tag_filter {
 	const char *pattern;
 	int lines;
 	struct commit_list *with_commit;
+	const struct string_list *ignore;
 };
 
+static struct string_list ignore_tags = STRING_LIST_INIT_DUP;
+
 static int show_reference(const char *refname, const unsigned char *sha1,
 			  int flag, void *cb_data)
 {
@@ -41,6 +45,13 @@ static int show_reference(const char *refname, const unsigned char *sha1,
 		char *buf, *sp, *eol;
 		size_t len;
 
+		if (filter->ignore) {
+			for (i = 0; i < filter->ignore->nr; i++)
+				if (!fnmatch(filter->ignore->items[i].string,
+					     refname, 0))
+					return 0;
+		}
+
 		if (filter->with_commit) {
 			struct commit *commit;
 
@@ -89,7 +100,8 @@ static int show_reference(const char *refname, const unsigned char *sha1,
 }
 
 static int list_tags(const char *pattern, int lines,
-			struct commit_list *with_commit)
+			struct commit_list *with_commit,
+			const struct string_list *ignore)
 {
 	struct tag_filter filter;
 
@@ -99,6 +111,7 @@ static int list_tags(const char *pattern, int lines,
 	filter.pattern = pattern;
 	filter.lines = lines;
 	filter.with_commit = with_commit;
+	filter.ignore = ignore;
 
 	for_each_tag_ref(show_reference, (void *) &filter);
 
@@ -233,6 +246,13 @@ static int git_tag_config(const char *var, const char *value, void *cb)
 		return 0;
 	}
 
+	if (!strcmp(var, "tag.ignore")) {
+		if (!value)
+			return config_error_nonbool(var);
+		string_list_append(&ignore_tags, value);
+		return 0;
+	}
+
 	return git_default_config(var, value, cb);
 }
 
@@ -365,6 +385,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	const char *msgfile = NULL, *keyid = NULL;
 	struct msg_arg msg = { 0, STRBUF_INIT };
 	struct commit_list *with_commit = NULL;
+	int ignore = 1;
 	struct option options[] = {
 		OPT_BOOLEAN('l', NULL, &list, "list tag names"),
 		{ OPTION_INTEGER, 'n', NULL, &lines, "n",
@@ -391,6 +412,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 			PARSE_OPT_LASTARG_DEFAULT,
 			parse_opt_with_commit, (intptr_t)"HEAD",
 		},
+		OPT_BOOLEAN(0, "ignore", &ignore, "respect tag.ignore config"),
 		OPT_END()
 	};
 
@@ -415,7 +437,8 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 		usage_with_options(git_tag_usage, options);
 	if (list)
 		return list_tags(argv[0], lines == -1 ? 0 : lines,
-				 with_commit);
+				 with_commit,
+				 ignore ? &ignore_tags : NULL);
 	if (lines != -1)
 		die("-n option is only allowed with -l.");
 	if (with_commit)

Re: How to create tags outside of refs/tags?

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> It sounds a lot saner to me to fix "git tag", then, to ignore certain
> uninteresting bits of the tag namespace. We already do allow pattern
> matching the tags shown, but:
>
>   1. It's an fnmatch, so I don't think there is a way to do a negative
>      match like "ignore everything that matches release/*".
>
>   2. It can only be specified on the command-line, whereas obviously you
>      would want this as the default for a repo when you type "git tag".
>
> So I think we could do something like the patch below, which allows:
>
>   # set up some boring and interesting tags
>   for i in boring more-boring interesting; do
>     for j in one two three; do
>       git tag $i/$j
>     done
>   done
>
>   # tell git to ignore boring stuff
>   git config tag.ignore 'boring/*'
>   git config --add tag.ignore 'more-boring/*'

Hmph, isn't that what "grep -v" was invented for?

Also it is unclear if the boring tags will or will not be propagated (or
should or should not be, for that matter) to outer world when you do "git
push --tags" with your "ignore".  Most likely some people do want to see
them hidden (e.g. when publishing), and some others do want to be pushed
(e.g. when backing the repository up).

Re: How to create tags outside of refs/tags?

[Jeff King]

On Tue, Feb 22, 2011 at 12:09:37AM -0800, Junio C Hamano wrote:

> >   # tell git to ignore boring stuff
> >   git config tag.ignore 'boring/*'
> >   git config --add tag.ignore 'more-boring/*'
> 
> Hmph, isn't that what "grep -v" was invented for?

Heh. Yeah, but I think the point is that you will want to do this _every
time_, so it saves typing. I guess you could make an alias. Or even more
hack-ish, a custom tag.pager that filters the results. ;)

Yet another option that would work in my example is to use
"interesting/*" as a positive pattern. But that implies that all of the
interesting ones can be grouped in that way.

I dunno. I don't really care much either way. I would probably not be a
user of this feature myself.

> Also it is unclear if the boring tags will or will not be propagated (or
> should or should not be, for that matter) to outer world when you do "git
> push --tags" with your "ignore".  Most likely some people do want to see
> them hidden (e.g. when publishing), and some others do want to be pushed
> (e.g. when backing the repository up).

I think changing the config name to tag.list-ignore would make it more
obvious what is going on. If you don't want to push certain tags, I
think refspecs are the way to do that, and this should be a purely local
display thing.

-Peff

Re: How to create tags outside of refs/tags?

[Jakub Narebski]

Jeff King <peff@peff.net> writes:

> On Tue, Feb 22, 2011 at 12:09:37AM -0800, Junio C Hamano wrote:
> 
> > >   # tell git to ignore boring stuff
> > >   git config tag.ignore 'boring/*'
> > >   git config --add tag.ignore 'more-boring/*'
> > 
> > Hmph, isn't that what "grep -v" was invented for?
> 
> Heh. Yeah, but I think the point is that you will want to do this _every
> time_, so it saves typing. I guess you could make an alias. Or even more
> hack-ish, a custom tag.pager that filters the results. ;)

[...]
> > Also it is unclear if the boring tags will or will not be propagated (or
> > should or should not be, for that matter) to outer world when you do "git
> > push --tags" with your "ignore".  Most likely some people do want to see
> > them hidden (e.g. when publishing), and some others do want to be pushed
> > (e.g. when backing the repository up).
> 
> I think changing the config name to tag.list-ignore would make it more
> obvious what is going on. If you don't want to push certain tags, I
> think refspecs are the way to do that, and this should be a purely local
> display thing.

You meant `tag.listIgnore`, weren't you?  No hyphens in config
variable names.

About refspecs: we currently do not support negative refspecs, do we?
(IIRC there was proposal to use '!' or '-' as prefix for them).  So we
can't currently exclude refs/tags/private/* while including
refs/tags/v*?

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: How to create tags outside of refs/tags?

[Jeff King]

On Tue, Feb 22, 2011 at 02:09:14AM -0800, Jakub Narebski wrote:

> > I think changing the config name to tag.list-ignore would make it more
> > obvious what is going on. If you don't want to push certain tags, I
> > think refspecs are the way to do that, and this should be a purely local
> > display thing.
> 
> You meant `tag.listIgnore`, weren't you?  No hyphens in config
> variable names.

Sure.

> About refspecs: we currently do not support negative refspecs, do we?
> (IIRC there was proposal to use '!' or '-' as prefix for them).  So we
> can't currently exclude refs/tags/private/* while including
> refs/tags/v*?

Yeah, I think you might be stuck enumerating the tags you want via
refspec. But I stand by my statement; if this feature is something we
want, it should be purely about listing tags. I don't want to get into
weird tag-specific config affecting the what we push. If somebody wants
negative refspecs, that can be a separate patch.

But like I said before, I am lukewarm on the patch in the first place.

-Peff

Re: How to create tags outside of refs/tags?

[Christian Halstrick]

On Tue, Feb 22, 2011 at 16:27, Jeff King <peff@peff.net> wrote:
> On Tue, Feb 22, 2011 at 02:09:14AM -0800, Jakub Narebski wrote:
> refspec. But I stand by my statement; if this feature is something we
> want, it should be purely about listing tags. I don't want to get into
> weird tag-specific config affecting the what we push. If somebody wants
> negative refspecs, that can be a separate patch.

I am not sure whether what is discussed now is really solving my
problem best. Maybe I rephrase again the problem I have:
I just want to make sure that certain commits are not garbage
collected. And the artefacts I create to ensure that should not
pollute the namespace of developer-created tags/branches. Is creating
a tag and then hiding it again really the best solution for that?

Here is some background: there is a tool developed in our company
which helps teams to release their git-controlled products. The
release tool gets as input just a commit ID. Whenever the release tool
has succesfully done his work on a product it wants to make sure the
this commit will never be garbage collected. Even not if month's later
the product owner resets all their branches making the released
commits beeing unreachable. And since some projects may run a lot of
attempts until they finally released some stuff we don't to have our
tags namespace to
be polluted with too many attempt-to-release tags.

Re: How to create tags outside of refs/tags?

[Junio C Hamano]

Christian Halstrick <christian.halstrick@gmail.com> writes:

> I am not sure whether what is discussed now is really solving my
> problem best. Maybe I rephrase again the problem I have:
> I just want to make sure that certain commits are not garbage
> collected. And the artefacts I create to ensure that should not
> pollute the namespace of developer-created tags/branches. Is creating
> a tag and then hiding it again really the best solution for that?
>
> Here is some background: there is a tool developed in our company which
> helps teams to release their git-controlled products. The release tool
> gets as input just a commit ID. Whenever the release tool has
> succesfully done his work on a product it wants to make sure the this
> commit will never be garbage collected. Even not if month's later the
> product owner resets all their branches making the released commits
> beeing unreachable. And since some projects may run a lot of attempts
> until they finally released some stuff we don't to have our tags
> namespace to be polluted with too many attempt-to-release tags.

How about taking advantage of the fact that you do not have to treat tag
namespace as a global thing?  For the "release tool" to "get a commit ID"
and do something useful, the "release tool" must be working on a git
repository, and hopefully that repository is something other than the ones
developers directly play with (by checking out random branches in its
working tree and compiling, etc.).

You can run "git tag" in that "release tool" repository, and if you do not
want to show that to others, never let developers fetch tags from there.

I would think that using a private namespace other than tags for your own
use (e.g. "git update-ref refs/anchor/release-20110222 $that_commit") may
be the best option that is easiest to understand, though.

Re: How to create tags outside of refs/tags?

[Christian Halstrick]

Hi,

> How about taking advantage of the fact that you do not have to treat tag
> namespace as a global thing?  For the "release tool" to "get a commit ID"
> and do something useful, the "release tool" must be working on a git
> repository, and hopefully that repository is something other than the ones
> developers directly play with (by checking out random branches in its
> working tree and compiling, etc.).

These assumptions are all correct. The "release tool" works on his clone
of a central repo. The release tool may even modify the content (e.g. increasing
release numbers in project metadata files) and create new git commits.
In the end it wants to create tags/refs locally und push these tags back
so that in the central repo the commits are not gc'ed.

Something like:

cd <releaseRepo>
git fetch origin
git checkout <commitIdToRelease>
edit pom.xml; git add pom.xml
git commit -m "release ..."
git tag <releaseName>
git push origin 'refs/tags/<releaseName>:refs/releases/<releaseName>'

should do the job, or?

Re: How to create tags outside of refs/tags?

[Michael J Gruber]

Christian Halstrick venit, vidit, dixit 22.02.2011 23:08:
> Hi,
> 
>> How about taking advantage of the fact that you do not have to treat tag
>> namespace as a global thing?  For the "release tool" to "get a commit ID"
>> and do something useful, the "release tool" must be working on a git
>> repository, and hopefully that repository is something other than the ones
>> developers directly play with (by checking out random branches in its
>> working tree and compiling, etc.).
> 
> These assumptions are all correct. The "release tool" works on his clone
> of a central repo. The release tool may even modify the content (e.g. increasing
> release numbers in project metadata files) and create new git commits.
> In the end it wants to create tags/refs locally und push these tags back
> so that in the central repo the commits are not gc'ed.
> 
> Something like:
> 
> cd <releaseRepo>
> git fetch origin
> git checkout <commitIdToRelease>
> edit pom.xml; git add pom.xml
> git commit -m "release ..."
> git tag <releaseName>
> git push origin 'refs/tags/<releaseName>:refs/releases/<releaseName>'
> 
> should do the job, or?

Yep. Also, if you do

git tag release/<releasename>

then a simple push refspec 'refs/tags/release/*:refs/releases/*' in your
config does the job. (Assuming you don't have any other tags in release/.)

Michael

Re: How to create tags outside of refs/tags?

[Christian Halstrick]

thanks all of you for your fast help. My problem is now solved.

Re: How to create tags outside of refs/tags?

[Enrico Weigelt]

* Junio C Hamano <gitster@pobox.com> wrote:

Hi folks,


> I would think that using a private namespace other than tags for your own
> use (e.g. "git update-ref refs/anchor/release-20110222 $that_commit") may
> be the best option that is easiest to understand, though.

I'm doing similar things with source crawlers in the OSS-QM
project: each time a new upstream version arrives, it gets
tag into a queue namespace, so I can pick them up an do my
usual qm works on them one by one.

git ls-remote git://pubgit.metux.de/oss-qm/glib2.git | grep queue


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: How to create tags outside of refs/tags?

[Jay Soffian]

On Tue, Feb 22, 2011 at 1:17 AM, Christian Halstrick
<christian.halstrick@gmail.com> wrote:
> Is there any better way to achieve that certain commits are not
> garbage collected than to create such tags. I love my foot and don't
> want to "shoot if off" but I want to make sure the the
> "release-process-machinery" in our company can be sure that certain
> source states are never gc'ed. I can also live with creating normal
> tags 'refs/tags/release/1.20' but would prefer a solution where 'git
> tags' shows only developer created tags.

Yes, I'm sorry I wasn't more careful/clearer in my earlier reply.

> Will any ref underneath "refs/' prevent gc to garbage collect a
> commit? Or only those in the well-known locations (e.g. refs/tags/*)

Anything under refs/ is safe, but you should create a subdirectory so
that you've got your own name-space under there. For example, the
gerrit code review system uses refs/changes/ for its refs.

Because these refs would be outside git's normal name-spaces
(refs/heads and refs/tags), they won't be fetched by the default
(non-mirror) refspec.

j.