any way to apply tag across all branches in repository?

any way to apply tag across all branches in repository?

[Chris Friesen]

Hi all,

I'm hoping you can help me out...please CC me on replies, I'm not
subscribed to the list.

We have a piece of software with a "main" branch and multiple
architecture-specific "target" branches.  At each "official" compile,
we'd like to tag the commits that went into that compile with an identifier.

Using tags normally requires that the tag be assigned to each branch
individually--is there any way to apply some sort of designator to the
head of each branch in the repository all at once rather than doing it
separately for each branch?

Thanks,

Chris

Re: any way to apply tag across all branches in repository?

[Tomas Carnecky]

On May 19, 2009, at 6:26 PM, Chris Friesen wrote:

>
> Hi all,
>
> I'm hoping you can help me out...please CC me on replies, I'm not
> subscribed to the list.
>
> We have a piece of software with a "main" branch and multiple
> architecture-specific "target" branches.  At each "official" compile,
> we'd like to tag the commits that went into that compile with an  
> identifier.
>
> Using tags normally requires that the tag be assigned to each branch
> individually--is there any way to apply some sort of designator to the
> head of each branch in the repository all at once rather than doing it
> separately for each branch?

Tags are not specific to any branch in particular. Usually you tag  
commits, and git doesn't care on which branch these commits are. That  
information is not recorded in the tag.

What you can do is create an alias that iterates over all branches and  
tags each one.

git for-each-ref refs/heads/main refs/heads/arch/ | while read sha  
type ref; do
     git tag TAGNAME $sha -m "Tagged $ref"
done

$sha is the sha where the ref points to, $type will be 'commit' and  
$ref is the full ref (refs/heads/arch/xxx for example)

tom

Re: any way to apply tag across all branches in repository?

[Chris Friesen]

Tomas Carnecky wrote:

> Tags are not specific to any branch in particular. Usually you tag  
> commits, and git doesn't care on which branch these commits are. That  
> information is not recorded in the tag.
> 
> What you can do is create an alias that iterates over all branches and  
> tags each one.
> 
> git for-each-ref refs/heads/main refs/heads/arch/ | while read sha  
> type ref; do
>      git tag TAGNAME $sha -m "Tagged $ref"
> done
> 
> $sha is the sha where the ref points to, $type will be 'commit' and  
> $ref is the full ref (refs/heads/arch/xxx for example)

I tried something like this manually but on the second branch it complained
that the tag already existed.

Chris

Re: any way to apply tag across all branches in repository?

[Brandon Casey]

Chris Friesen wrote:
> Hi all,
> 
> I'm hoping you can help me out...please CC me on replies, I'm not
> subscribed to the list.
> 
> We have a piece of software with a "main" branch and multiple
> architecture-specific "target" branches.  At each "official" compile,
> we'd like to tag the commits that went into that compile with an identifier.
> 
> Using tags normally requires that the tag be assigned to each branch
> individually--is there any way to apply some sort of designator to the
> head of each branch in the repository all at once rather than doing it
> separately for each branch?

If I understand you correctly, you are doing your primary development on
the "main" branch and then merging this into the architecture specific
branches which contain additional architecture specific changes.

All you need to do is tag the "main" branch.  Actually, you are tagging
the commit that the branch currently points at.  When this branch is
merged into the other branches, they will also contain this commit, and
'git describe' will use the tag you created when generating the version
string.

-brandon

Re: any way to apply tag across all branches in repository?

[Chris Friesen]

Brandon Casey wrote:

> If I understand you correctly, you are doing your primary development on
> the "main" branch and then merging this into the architecture specific
> branches which contain additional architecture specific changes.

Correct.

> All you need to do is tag the "main" branch.  Actually, you are tagging
> the commit that the branch currently points at.  When this branch is
> merged into the other branches, they will also contain this commit, and
> 'git describe' will use the tag you created when generating the version
> string.

I think this would work if the most recent commit is on the main branch.

However, if I make a change on the arch-specific branch, then tag the
main branch and merge it into the arch-specific branch, git tells me
the arch-specific branch is already up-to-date and the tag doesn't
get propagated.

Chris

Re: any way to apply tag across all branches in repository?

[Linus Torvalds]

On Tue, 19 May 2009, Chris Friesen wrote:
> 
> However, if I make a change on the arch-specific branch, then tag the
> main branch and merge it into the arch-specific branch, git tells me
> the arch-specific branch is already up-to-date and the tag doesn't
> get propagated.

You can always just do "git fetch --tags" to fetch any new tags without 
doing anything else.

			Linus

Re: any way to apply tag across all branches in repository?

[Chris Friesen]

Linus Torvalds wrote:
> 
> On Tue, 19 May 2009, Chris Friesen wrote:
>> However, if I make a change on the arch-specific branch, then tag the
>> main branch and merge it into the arch-specific branch, git tells me
>> the arch-specific branch is already up-to-date and the tag doesn't
>> get propagated.
> 
> You can always just do "git fetch --tags" to fetch any new tags without 
> doing anything else.

This is all in the same local repository, but with target-specific branches
containing arch-specific changes on top of a common codebase.  The
arch-specific stuff often comes from board vendors and such, and they're
never going to be merged back into the common codebase.

I'm looking for some way to conceptually tag the current head of each
branch to indicate "this commit was used to build product version FOO" so
that later on when we find a bug in our code we can tell which product
version(s) contain the bug and need to be patched in the field.

The brute-force way to do this would be to manually loop through each branch
and create a tag of the form "$branch_$version" to ensure unique tags.  But I was
hoping there was a more elegant way.

Chris

Re: any way to apply tag across all branches in repository?

[Linus Torvalds]

On Tue, 19 May 2009, Chris Friesen wrote:
> 
> This is all in the same local repository, but with target-specific branches
> containing arch-specific changes on top of a common codebase.  The
> arch-specific stuff often comes from board vendors and such, and they're
> never going to be merged back into the common codebase.
> 
> I'm looking for some way to conceptually tag the current head of each
> branch to indicate "this commit was used to build product version FOO" so
> that later on when we find a bug in our code we can tell which product
> version(s) contain the bug and need to be patched in the field.

Oh. Yes, in that case, you do need to just tag each head.

> The brute-force way to do this would be to manually loop through each branch
> and create a tag of the form "$branch_$version" to ensure unique tags.  But I was
> hoping there was a more elegant way.

Well, I would suggest that you do it fundamentally differently.

Instead of tagging each build, I would suggest just associating each build 
with the commit SHA1 of the time. That's what Linux does (if you enable 
CONFIG_LOCALVERSION_AUTO), and it's _way_ superior to lots of crazy tags.

So for example, I can do

	[torvalds@nehalem ~]$ uname -r
	2.6.30-rc6-00302-g72357d5-dirty

and it tells me exactly what kernel version I'm running (well, the "dirty"
part means that it's not exact and has some additional patches that 
weren't committed, but that's as close as you can get). It's very useful.

Now, with some other build, you might want to just encode the SHA1 
revision in your binary. For example, a simple build-time rule like a 
Makefile addition that does something like

   version.c:
	echo 'const char git_build_version = "'$(git describe)'";" > version.c

and then you just force version.o to be linked into your build.

Trust me, something like the above is _much_ better than tagging each 
branchthat you build. Partly because it means that you can do the builds 
in a distributed manner, and they'll all get the version built in, rather 
than having to rely on everybody tagging everything and then trying to 
match up the tag to some random binary.

			Linus

Re: any way to apply tag across all branches in repository?

[Chris Friesen]

Linus Torvalds wrote:
> On Tue, 19 May 2009, Chris Friesen wrote:

>> The brute-force way to do this would be to manually loop through each branch
>> and create a tag of the form "$branch_$version" to ensure unique tags.  But I was
>> hoping there was a more elegant way.
> 
> Well, I would suggest that you do it fundamentally differently.
> 
> Instead of tagging each build, I would suggest just associating each build 
> with the commit SHA1 of the time. That's what Linux does (if you enable 
> CONFIG_LOCALVERSION_AUTO), and it's _way_ superior to lots of crazy tags.
> 
> So for example, I can do
> 
> 	[torvalds@nehalem ~]$ uname -r
> 	2.6.30-rc6-00302-g72357d5-dirty
> 
> and it tells me exactly what kernel version I'm running (well, the "dirty"
> part means that it's not exact and has some additional patches that 
> weren't committed, but that's as close as you can get). It's very useful.

Agreed.  The project in question actually involves (among other things)
a linux kernel build, so we will be making use of this to work backwards
from the running kernel to the commit used to generate it.

However, we also want to be able to work in the other direction--given a
known-buggy kernel commit, which shipped versions of the product contain
the buggy code?  We do in-the-field upgrades, and different sites may be
running different versions, so it's important to be able to easily
determine which sites are currently running the buggy code so that we
can get them upgraded.  We know which sites are running which versions,
so it is useful to tag the repository branches with that version number.

> Trust me, something like the above is _much_ better than tagging each 
> branchthat you build. Partly because it means that you can do the builds 
> in a distributed manner, and they'll all get the version built in, rather 
> than having to rely on everybody tagging everything and then trying to 
> match up the tag to some random binary.

The tagging would be done only by the "official" build process (which
pulls from an "official" repository), not by each designer.  Typically
the official builds would be done weekly, more frequently if requested.

Chris

Re: any way to apply tag across all branches in repository?

[Linus Torvalds]

On Tue, 19 May 2009, Chris Friesen wrote:
> 
> The tagging would be done only by the "official" build process (which
> pulls from an "official" repository), not by each designer.  Typically
> the official builds would be done weekly, more frequently if requested.

Well, you can tag when you do that official build. Do you really do 
"official" builds from all branches? That sounds a bit insane.

Remember: you don't have to tag whatever is the "top" - tagging can happen 
later. Tagging at build-time is perfectly fine.

In fact, I'd suggest going even further. Don't tag the source branch when 
you build - tag it after it has passed whatever testing you do (I hope you 
_do_ have some extensive test-suite before release), and as you actually 
make it public (or whatever you do). Only at _that_ point, tag the tree 
with "release-$branch-$date" or something like that.

Remember: you don't have to tag the top-of branch. You can tag any commit, 
after-the-fact. So even if you've done other development since, just make 
sure to tag the commit you actually built and tested.

		Linus

Re: any way to apply tag across all branches in repository?

[Chris Friesen]

Linus Torvalds wrote:
> 
> On Tue, 19 May 2009, Chris Friesen wrote:
>> The tagging would be done only by the "official" build process (which
>> pulls from an "official" repository), not by each designer.  Typically
>> the official builds would be done weekly, more frequently if requested.
> 
> Well, you can tag when you do that official build. Do you really do 
> "official" builds from all branches? That sounds a bit insane.

We have one "official" branch for each target board...so maybe a dozen
or so branches.

Developers do private builds, but they're not tagged.

> Remember: you don't have to tag whatever is the "top" - tagging can happen 
> later. Tagging at build-time is perfectly fine.

Tagging at build-time is actually the plan.

> In fact, I'd suggest going even further. Don't tag the source branch when 
> you build - tag it after it has passed whatever testing you do (I hope you 
> _do_ have some extensive test-suite before release), and as you actually 
> make it public (or whatever you do). Only at _that_ point, tag the tree 
> with "release-$branch-$date" or something like that.

There's a fairly extensive test suite.  This might be an option.

> Remember: you don't have to tag the top-of branch. You can tag any commit, 
> after-the-fact. So even if you've done other development since, just make 
> sure to tag the commit you actually built and tested.

Good point.  I think I've got enough information to get something
working.  Thanks for all the help.

Chris

Re: any way to apply tag across all branches in repository?

[Brandon Casey]

Chris Friesen wrote:
> Brandon Casey wrote:
> 
>> If I understand you correctly, you are doing your primary development on
>> the "main" branch and then merging this into the architecture specific
>> branches which contain additional architecture specific changes.
> 
> Correct.
> 
>> All you need to do is tag the "main" branch.  Actually, you are tagging
>> the commit that the branch currently points at.  When this branch is
>> merged into the other branches, they will also contain this commit, and
>> 'git describe' will use the tag you created when generating the version
>> string.
> 
> I think this would work if the most recent commit is on the main branch.
> 
> However, if I make a change on the arch-specific branch, then tag the
> main branch and merge it into the arch-specific branch, git tells me
> the arch-specific branch is already up-to-date and the tag doesn't
> get propagated.

Tags aren't versioned.  They exist outside of the branch namespace.
So merging doesn't have any direct effect on tags.  It is the _commits_
that are merged in (which the tags point to).  If you have already
merged the main branch into the arch-specific branch, then there is
nothing else for git to do.  Your repository looks something like this
graph:

    --o--o---o---o---o---o--o   "arch"
     /                  /
   -o--o---o---o---o---o        "main"
                       |
                      my_tag

When you merge "main" into "arch", the "main" DAG* becomes a part of the
"arch" DAG.  The tag points to a specific commit, which represents a state
of the DAG, which is also now part of the "arch" DAG.

The output of 'git describe $arch_branch' will likely change after you
create the tag though.

Try these commands:

  git describe $main
  git describe $arch_branch
  git tag -m 'a test tag' my_tag $main
  git describe $main
  git describe $arch_branch

-brandon

* DAG - directed acyclic graph

Re: any way to apply tag across all branches in repository?

[Chris Friesen]

Brandon Casey wrote:

> Try these commands:
> 
>   git describe $main
>   git describe $arch_branch
>   git tag -m 'a test tag' my_tag $main
>   git describe $main
>   git describe $arch_branch

In the commands below, "main" is $main, and "arch" is $arch_branch.
I'm starting out with the arch branch checked out.

[cfriesen@localhost linux]$ git describe main
dynamic_ftrace_excluded-auto-mark-225-g7c2dc32
[cfriesen@localhost linux]$ git describe arch
dynamic_ftrace_excluded-auto-mark-225-g7c2dc32
[cfriesen@localhost linux]$ git tag -m 'a test tag' my_tag ncgl
[cfriesen@localhost linux]$ git describe arch
my_tag
[cfriesen@localhost linux]$ git describe arch
my_tag

So far so good.  Now I make a change to the arch branch, and add
another tag to the main branch.

[cfriesen@localhost linux]$ echo a > asdf
[cfriesen@localhost linux]$ git add asdf
[cfriesen@localhost linux]$ git commit
Created commit 4c8dfa7: blah
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 asdf

[cfriesen@localhost linux]$ git describe main
my_tag
[cfriesen@localhost linux]$ git describe arch
my_tag-1-g4c8dfa7


Now we add another tag to the main branch:

[cfriesen@localhost linux]$ git tag -m 'a test tag' my_tag2 main
[cfriesen@localhost linux]$ git describe main
my_tag
[cfriesen@localhost linux]$ git describe arch
my_tag-1-g4c8dfa7

I assume that since there were no code changes on the main branch,
it doesn't think that there is any difference between the two tags.

Chris

Re: any way to apply tag across all branches in repository?

[Brandon Casey]

Chris Friesen wrote:
> Brandon Casey wrote:
> 
>> Try these commands:
>>
>>   git describe $main
>>   git describe $arch_branch
>>   git tag -m 'a test tag' my_tag $main
>>   git describe $main
>>   git describe $arch_branch
> 
> In the commands below, "main" is $main, and "arch" is $arch_branch.
> I'm starting out with the arch branch checked out.
> 
> [cfriesen@localhost linux]$ git describe main
> dynamic_ftrace_excluded-auto-mark-225-g7c2dc32
> [cfriesen@localhost linux]$ git describe arch
> dynamic_ftrace_excluded-auto-mark-225-g7c2dc32
> [cfriesen@localhost linux]$ git tag -m 'a test tag' my_tag ncgl

What's "ncgl"?  Another branch at the same tip as "arch"?

> [cfriesen@localhost linux]$ git describe arch
> my_tag
> [cfriesen@localhost linux]$ git describe arch
> my_tag

Was one of those supposed to be "main"?

> So far so good.

I expected to see:

  $ git tag -m 'a test tag' my_tag main

  $ git describe main
  my_tag

  $ git describe arch
  my_tag-X-g0123456

Where 'X' is some number equal to the number of commits _not_ reachable
from "my_tag", and the digits after the 'g' are an abbreviated sha1 of
the tip commit on the arch branch.

> Now I make a change to the arch branch, and add
> another tag to the main branch.
> 
> [cfriesen@localhost linux]$ echo a > asdf
> [cfriesen@localhost linux]$ git add asdf
> [cfriesen@localhost linux]$ git commit
> Created commit 4c8dfa7: blah
>  1 files changed, 1 insertions(+), 0 deletions(-)
>  create mode 100644 asdf
> 
> [cfriesen@localhost linux]$ git describe main
> my_tag
> [cfriesen@localhost linux]$ git describe arch
> my_tag-1-g4c8dfa7

Ok, so maybe this is a test repo.  I didn't expect main and arch
to ever point to the exact same state.  This won't happen in
your real repo unless you are merging arch back into main.

Ok, you tagged main, and previously arch was at the same state,
so 'git describe' printed out 'my_tag' for both of them.  Now,
the arch branch is ahead of main by one commit, so you get an
expanded string from 'git describe' (the meaning of which I
described earlier, above).

> Now we add another tag to the main branch:
> 
> [cfriesen@localhost linux]$ git tag -m 'a test tag' my_tag2 main
> [cfriesen@localhost linux]$ git describe main
> my_tag
> [cfriesen@localhost linux]$ git describe arch
> my_tag-1-g4c8dfa7
> 
> I assume that since there were no code changes on the main branch,
> it doesn't think that there is any difference between the two tags.

Right.  There is no difference.  You created another tag pointing at
the same revision as the first tag.  Here's something else to try:

   $ git rev-parse main
   $ git rev-parse my_tag
   $ git rev-parse my_tag2

You'll see that they all print out the same sha1 string.

You can also try this:

   $ git rev-parse arch^
   # prints out same sha1 as above

   $ git rev-parse arch
   # prints out the sha1 of the commit that you just created

Off-hand, I'm not sure how 'git describe' decides which tag to use
in the describe output when there is more than one candidate.
Possibly earliest created?, possible alphabetical?  I didn't look.

Starting to make sense?

-brandon

Re: any way to apply tag across all branches in repository?

[Chris Friesen]

Brandon Casey wrote:
> Chris Friesen wrote:

>> [cfriesen@localhost linux]$ git tag -m 'a test tag' my_tag ncgl
> 
> What's "ncgl"?  Another branch at the same tip as "arch"?

Oops, missed a replace.  "ncgl" is my main branch.

> Ok, you tagged main, and previously arch was at the same state,
> so 'git describe' printed out 'my_tag' for both of them.  Now,
> the arch branch is ahead of main by one commit, so you get an
> expanded string from 'git describe' (the meaning of which I
> described earlier, above).
> 
>> Now we add another tag to the main branch:
>>
>> [cfriesen@localhost linux]$ git tag -m 'a test tag' my_tag2 main
>> [cfriesen@localhost linux]$ git describe main
>> my_tag
>> [cfriesen@localhost linux]$ git describe arch
>> my_tag-1-g4c8dfa7
>>
>> I assume that since there were no code changes on the main branch,
>> it doesn't think that there is any difference between the two tags.
> 
> Right.  There is no difference.  You created another tag pointing at
> the same revision as the first tag.  Here's something else to try:

<snip>

> Starting to make sense?

Yep.  I still need whatever identifier I use to be associated with the
head of each branch.

Based on the discussion (thanks for the explanations, by the way) I
don't see any option other than looping through each branch and using
tags with the branch name embedded in them to ensure uniqueness across
the repository.

Chris

Re: any way to apply tag across all branches in repository?

[Brandon Casey]

Chris Friesen wrote:
> Brandon Casey wrote:

> <snip>
> 
>> Starting to make sense?
> 
> Yep.  I still need whatever identifier I use to be associated with the
> head of each branch.
> 
> Based on the discussion (thanks for the explanations, by the way) I
> don't see any option other than looping through each branch and using
> tags with the branch name embedded in them to ensure uniqueness across
> the repository.

Based on this, and your other message, it sounds like you want to have a
tag named 'v1.2.3' which is treated differently for each branch.  Since
there is only a single tag namespace, this is not possible.  Tag names
must be unique.

You can also use slashes in tag names.  Perhaps you could have the illusion
of common version numbers by using tag names like $arch/$version?

-brandon

Re: any way to apply tag across all branches in repository?

[Mark Struberg]

> We have one "official" branch for each target board...so
> maybe a dozen or so branches.

Chis, to me this sounds more like you abuse SCM branches for what should be into 'modules'
In SVN this would make not a big difference (because of the silly svn:copy thingy) but for any sane SCM, tags are not directories!

If you move all your board specific code into child-modules, then you could happily tag over your whole project (including all those modules).

LieGrue,
strub

--- Chris Friesen <cfriesen@nortel.com> schrieb am Di, 19.5.2009:

> Von: Chris Friesen <cfriesen@nortel.com>
> Betreff: Re: any way to apply tag across all branches in repository?
> An: "Linus Torvalds" <torvalds@linux-foundation.org>
> CC: "Brandon Casey" <casey@nrlssc.navy.mil>, git@vger.kernel.org
> Datum: Dienstag, 19. Mai 2009, 23:31
> Linus Torvalds wrote:
> > 
> > On Tue, 19 May 2009, Chris Friesen wrote:
> >> The tagging would be done only by the "official"
> build process (which
> >> pulls from an "official" repository), not by each
> designer.  Typically
> >> the official builds would be done weekly, more
> frequently if requested.
> > 
> > Well, you can tag when you do that official build. Do
> you really do 
> > "official" builds from all branches? That sounds a bit
> insane.
> 
> We have one "official" branch for each target board...so
> maybe a dozen
> or so branches.
> 
> Developers do private builds, but they're not tagged.
> 
> > Remember: you don't have to tag whatever is the "top"
> - tagging can happen 
> > later. Tagging at build-time is perfectly fine.
> 
> Tagging at build-time is actually the plan.
> 
> > In fact, I'd suggest going even further. Don't tag the
> source branch when 
> > you build - tag it after it has passed whatever
> testing you do (I hope you 
> > _do_ have some extensive test-suite before release),
> and as you actually 
> > make it public (or whatever you do). Only at _that_
> point, tag the tree 
> > with "release-$branch-$date" or something like that.
> 
> There's a fairly extensive test suite.  This might be
> an option.
> 
> > Remember: you don't have to tag the top-of branch. You
> can tag any commit, 
> > after-the-fact. So even if you've done other
> development since, just make 
> > sure to tag the commit you actually built and tested.
> 
> Good point.  I think I've got enough information to
> get something
> working.  Thanks for all the help.
> 
> Chris
> --
> To unsubscribe from this list: send the line "unsubscribe
> git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>