[PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Brandon Casey]

Add support for creating a new tag object and retaining the tag message,
author, and date when rewriting tags. The gpg signature, if one exists,
will be stripped.

This adds nearly proper tag name filtering to filter-branch. Proper tag
name filtering would include the ability to change the tagger, tag date,
tag message, and _not_ strip a gpg signature if the tag did not change.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
---


I went back and forth over stripping the gpg signature or just leaving
it intact and creating a new tag with the invalid signature. I ended
up stripping it, and printing a message saying it was stripped along
with the original tag object's sha1.

Portability comments?

-brandon


 Documentation/git-filter-branch.txt |   14 ++++++++++----
 git-filter-branch.sh                |   14 ++++++++++++--
 t/t7003-filter-branch.sh            |   32 ++++++++++++++++++++++++++++++++
 3 files changed, 54 insertions(+), 6 deletions(-)

diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt
index 543a1cf..9364919 100644
--- a/Documentation/git-filter-branch.txt
+++ b/Documentation/git-filter-branch.txt
@@ -132,10 +132,16 @@ use "--tag-name-filter cat" to simply update the tags.  In this
 case, be very careful and make sure you have the old tags
 backed up in case the conversion has run afoul.
 +
-Note that there is currently no support for proper rewriting of
-tag objects; in layman terms, if the tag has a message or signature
-attached, the rewritten tag won't have it.  Sorry.  (It is by
-definition impossible to preserve signatures at any rate.)
+Nearly proper rewriting of tag objects is supported. If the tag has
+a message attached, a new tag object will be created with the same message,
+author, and timestamp. If the tag has a signature attached, the
+signature will be stripped. It is by definition impossible to preserve
+signatures. The reason this is "nearly" proper, is because ideally if
+the tag did not change (points to the same object, has the same name, etc.)
+it should retain any signature. That is not the case, signatures will always
+be removed, buyer beware. There is also no support for changing the
+author or timestamp (or the tag message for that matter). Tags which point
+to other tags will be rewritten to point to the underlying commit.
 
 --subdirectory-filter <directory>::
 	Only look at the history which touches the given subdirectory.
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 22b6ed4..12d5a25 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -406,8 +406,18 @@ if [ "$filter_tag_name" ]; then
 		echo "$ref -> $new_ref ($sha1 -> $new_sha1)"
 
 		if [ "$type" = "tag" ]; then
-			# Warn that we are not rewriting the tag object itself.
-			warn "unreferencing tag object $sha1t"
+			new_sha1=$(git cat-file tag "$ref" | \
+				sed -e "1c\object $new_sha1" \
+				    -e '2c\type commit'      \
+				    -e "3c\tag $new_ref"     \
+				    -e '/^-----BEGIN PGP SIGNATURE-----/Q' | \
+				git mktag) ||
+				die "Could not create new tag object for $ref"
+			if git cat-file tag "$ref" | \
+			   grep '^-----BEGIN PGP SIGNATURE-----' >/dev/null 2>&1
+			then
+				warn "gpg signature stripped from tag object $sha1t"
+			fi
 		fi
 
 		git update-ref "refs/tags/$new_ref" "$new_sha1" ||
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 6827249..1daaf54 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -203,4 +203,36 @@ test_expect_success 'Subdirectory filter with disappearing trees' '
 	test $(git rev-list master | wc -l) = 3
 '
 
+test_expect_success 'Tag name filtering retains tag message' '
+	git tag -m atag T &&
+	git cat-file tag T > expect &&
+	git filter-branch -f --tag-name-filter cat &&
+	git cat-file tag T > actual &&
+	git diff expect actual
+'
+
+faux_gpg_tag='object XXXXXX
+type commit
+tag S
+tagger T A Gger <tagger@example.com> 1206026339 -0500
+
+This is a faux gpg signed tag.
+-----BEGIN PGP SIGNATURE-----
+Version: FauxGPG v0.0.0 (FAUX/Linux)
+
+gdsfoewhxu/6l06f1kxyxhKdZkrcbaiOMtkJUA9ITAc1mlamh0ooasxkH1XwMbYQ
+acmwXaWET20H0GeAGP+7vow=
+=agpO
+-----END PGP SIGNATURE-----
+'
+test_expect_success 'Tag name filtering strips gpg signature' '
+	sha1=$(git rev-parse HEAD) &&
+	sha1t=$(echo "$faux_gpg_tag" | sed -e s/XXXXXX/$sha1/ | git mktag) &&
+	git update-ref "refs/tags/S" "$sha1t" &&
+	echo "$faux_gpg_tag" | sed -e s/XXXXXX/$sha1/ | head -n 6 > expect &&
+	git filter-branch -f --tag-name-filter cat &&
+	git cat-file tag S > actual &&
+	git diff expect actual
+'
+
 test_done
-- 
1.5.4.4.481.g5075

Re: [PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Johannes Schindelin]

Hi,

On Thu, 20 Mar 2008, Brandon Casey wrote:

> Add support for creating a new tag object and retaining the tag message, 
> author, and date when rewriting tags. The gpg signature, if one exists, 
> will be stripped.
> 
> This adds nearly proper tag name filtering to filter-branch. Proper tag 
> name filtering would include the ability to change the tagger, tag date, 
> tag message, and _not_ strip a gpg signature if the tag did not change.

I think such a change in semantics merits a new option.  Maybe 
even something like

	"--rewrite-tags=(strip-signatures|redo-signatures|no-signed)"

or some such.  But maybe not; to rewrite a tag, you have to specify it 
explicitely (or implicitely with --all).

Ciao,
Dscho

Re: [PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Brandon Casey]

Johannes Schindelin wrote:
> Hi,
> 
> On Thu, 20 Mar 2008, Brandon Casey wrote:
> 
>> Add support for creating a new tag object and retaining the tag message, 
>> author, and date when rewriting tags. The gpg signature, if one exists, 
>> will be stripped.
>>
>> This adds nearly proper tag name filtering to filter-branch. Proper tag 
>> name filtering would include the ability to change the tagger, tag date, 
>> tag message, and _not_ strip a gpg signature if the tag did not change.
> 
> I think such a change in semantics merits a new option.

I had understood the current behavior to be a shortcoming rather than
desirable behavior.

The change this patch introduces is not the stripping of the gpg signature,
it is the new ability to _retain_ the properties of a tag object. Current
behavior discards everything, including the signature.

If people expect the current behavior and would not welcome this new behavior,
then maybe a new option is needed, but I'm not convinced yet. I'm thinking
that maybe converting a tag object into a light-weight tag should be the special
operation (if that is desirable).

-brandon

Re: [PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Thu, 20 Mar 2008, Brandon Casey wrote:
>
>> Add support for creating a new tag object and retaining the tag message, 
>> author, and date when rewriting tags. The gpg signature, if one exists, 
>> will be stripped.
>> 
>> This adds nearly proper tag name filtering to filter-branch. Proper tag 
>> name filtering would include the ability to change the tagger, tag date, 
>> tag message, and _not_ strip a gpg signature if the tag did not change.
>
> I think such a change in semantics merits a new option.

It is a different story if the fix is a good one, or has room for
improvement.  For example, the sed script that knows what is on the first
and second line and relies on that knowledge to use 1c...2c... looked very
fragile to me.

However, at least to me, this looked like an attempt for a pure "fix".

If you tell filter-branch to filter a branch A and a tag T, as the command
is advertised to rewrite positive refs that were given from the command
line, isn't it natural to expect that the command would attempt its best
effort to rewrite such a tag object?

Re: [PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Johannes Schindelin]

Hi,

On Sun, 23 Mar 2008, Junio C Hamano wrote:

> If you tell filter-branch to filter a branch A and a tag T, as the 
> command is advertised to rewrite positive refs that were given from the 
> command line, isn't it natural to expect that the command would attempt 
> its best effort to rewrite such a tag object?

The thing is: signed tags cannot be rewritten.  As simple as that.  They 
could be recreated if the committer happens to be the original tagger, but 
even then, it is a rewrite of _tags_, which are _not_ supposed to be 
moving targets.

So it was very much on purpose to disallow rewriting them by default.

Ciao,
Dscho

Re: [PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi,
>
> On Sun, 23 Mar 2008, Junio C Hamano wrote:
>
>> If you tell filter-branch to filter a branch A and a tag T, as the 
>> command is advertised to rewrite positive refs that were given from the 
>> command line, isn't it natural to expect that the command would attempt 
>> its best effort to rewrite such a tag object?
>
> The thing is: signed tags cannot be rewritten.

I know that, and you know I know that if you read what you responded again
;-)

And I think stripping of gpg signature part is a reasonable best effort
for the command, _when_ the user told a signed tag to be rewritten.

Re: [PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Johannes Schindelin]

Hi,

On Mon, 24 Mar 2008, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Sun, 23 Mar 2008, Junio C Hamano wrote:
> >
> >> If you tell filter-branch to filter a branch A and a tag T, as the 
> >> command is advertised to rewrite positive refs that were given from 
> >> the command line, isn't it natural to expect that the command would 
> >> attempt its best effort to rewrite such a tag object?
> >
> > The thing is: signed tags cannot be rewritten.
> 
> I know that, and you know I know that if you read what you responded 
> again ;-)
> 
> And I think stripping of gpg signature part is a reasonable best effort 
> for the command, _when_ the user told a signed tag to be rewritten.

Yes.

BUT.

You can say "--all", and I actually expect quite a few people to do 
exactly that.  And then you cannot really say "the user explicitely asked 
to have that signed tag rewritten".

Ciao,
Dscho

Re: [PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Brandon Casey]

Johannes Schindelin wrote:
> Hi,
> 
> On Mon, 24 Mar 2008, Junio C Hamano wrote:
> 
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>>
>>> On Sun, 23 Mar 2008, Junio C Hamano wrote:
>>>
>>>> If you tell filter-branch to filter a branch A and a tag T, as the 
>>>> command is advertised to rewrite positive refs that were given from 
>>>> the command line, isn't it natural to expect that the command would 
>>>> attempt its best effort to rewrite such a tag object?
>>> The thing is: signed tags cannot be rewritten.
>> I know that, and you know I know that if you read what you responded 
>> again ;-)
>>
>> And I think stripping of gpg signature part is a reasonable best effort 
>> for the command, _when_ the user told a signed tag to be rewritten.
> 
> Yes.
> 
> BUT.
> 
> You can say "--all", and I actually expect quite a few people to do 
> exactly that.  And then you cannot really say "the user explicitely asked 
> to have that signed tag rewritten".

That sounds like saying that specifying "HEAD" as the revision (_and_
using --tag-name-filter) is not explicitly asking for tags to be rewritten.
?

Aside from that, wouldn't the signed tag currently be rewritten? With current
git, if I have a signed tag named "my_signed_release", and I can verify its
authenticity using 'git tag -v my_signed_release', and then I run filter-branch
with '--tag-name-filter cat', my previously signed tag is replaced with a light
weight tag and of course 'git tag -v my_signed_release' fails.

I don't see how the situation for signed tags changes much. The current state
is that the signature cannot be preserved, so if "new" signed tags are desired,
they must be handled specially. On the other hand, the situation for unsigned
tags becomes much easier, assuming you want to retain the tag properties. Aren't
unsigned tags more common than signed ones?

So right now, tags matching the refspec are rewritten. They are currently
rewritten with a lightweight tag. I don't think you have explained what is
wrong with defaulting to creating a new tag object which retains as much
information from the original tag object as possible. In other words, why
would a user _not_ expect that the tag filter would preserve the tag message?

Can you describe some scenario where a user would use --tag-name-filter and
would prefer the creation of new light-weight tags over tag objects? I can't
imagine one in my own situation. Maybe there is some example in your own
workflow?

-brandon

Re: [PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Brandon Casey]

Brandon Casey wrote:
> Johannes Schindelin wrote:
>> Hi,
>>
>> On Mon, 24 Mar 2008, Junio C Hamano wrote:
>>
>>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>>>
>>>> On Sun, 23 Mar 2008, Junio C Hamano wrote:
>>>>
>>>>> If you tell filter-branch to filter a branch A and a tag T, as the 
>>>>> command is advertised to rewrite positive refs that were given from 
>>>>> the command line, isn't it natural to expect that the command would 
>>>>> attempt its best effort to rewrite such a tag object?
>>>> The thing is: signed tags cannot be rewritten.
>>> I know that, and you know I know that if you read what you responded 
>>> again ;-)
>>>
>>> And I think stripping of gpg signature part is a reasonable best effort 
>>> for the command, _when_ the user told a signed tag to be rewritten.
>> Yes.
>>
>> BUT.
>>
>> You can say "--all", and I actually expect quite a few people to do 
>> exactly that.  And then you cannot really say "the user explicitely asked 
>> to have that signed tag rewritten".
> 
> That sounds like saying that specifying "HEAD" as the revision (_and_
> using --tag-name-filter) is not explicitly asking for tags to be rewritten.

Maybe that should be qualified. I think your statement is similar to my
example and so "not explicitly asking for tags" (by your definition) since
neither explicitly named the tags to be rewritten on the command line.

In both cases, tag objects would currently be replaced by lightweight tags
when --tag-name-filter is used.

-brandon

Re: [PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Johannes Schindelin]

Hi,

On Mon, 24 Mar 2008, Brandon Casey wrote:

> So right now, tags matching the refspec are rewritten. They are currently
> rewritten with a lightweight tag.

That is unintended.  My understanding of a tag was always that it is 
something immutable.  I mean, _really_ immutable.  If you released a 
certain version, then that is tagged.  You must not rewrite the tag.  
Ever.

Ciao,
Dscho

Re: [PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Brandon Casey]

Johannes Schindelin wrote:
> Hi,
> 
> On Mon, 24 Mar 2008, Brandon Casey wrote:
> 
>> So right now, tags matching the refspec are rewritten. They are currently
>> rewritten with a lightweight tag.
> 
> That is unintended.  My understanding of a tag was always that it is 
> something immutable.  I mean, _really_ immutable.  If you released a 
> certain version, then that is tagged.  You must not rewrite the tag.  
> Ever.

Then what is the intended behavior for --tag-name-filter?

-brandon

Re: [PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Johannes Schindelin]

Hi,

On Mon, 24 Mar 2008, Brandon Casey wrote:

> Johannes Schindelin wrote:
> 
> > On Mon, 24 Mar 2008, Brandon Casey wrote:
> > 
> >> So right now, tags matching the refspec are rewritten. They are 
> >> currently rewritten with a lightweight tag.
> > 
> > That is unintended.  My understanding of a tag was always that it is 
> > something immutable.  I mean, _really_ immutable.  If you released a 
> > certain version, then that is tagged.  You must not rewrite the tag.  
> > Ever.
> 
> Then what is the intended behavior for --tag-name-filter?

To be honest, I wanted to rip that part out before posting the initial 
filter-branch patches, and forgot/was too lazy.

IMO there should be clean semantics first, and a default mode that does 
_not_ rewrite tags at all first.

Maybe something as in fast-export.

Ciao,
Dscho

Re: [PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Brandon Casey]

Johannes Schindelin wrote:
> Hi,
> 
> On Mon, 24 Mar 2008, Brandon Casey wrote:
> 
>> Johannes Schindelin wrote:
>>
>>> On Mon, 24 Mar 2008, Brandon Casey wrote:
>>>
>>>> So right now, tags matching the refspec are rewritten. They are 
>>>> currently rewritten with a lightweight tag.
>>> That is unintended.  My understanding of a tag was always that it is 
>>> something immutable.  I mean, _really_ immutable.  If you released a 
>>> certain version, then that is tagged.  You must not rewrite the tag.  
>>> Ever.
>> Then what is the intended behavior for --tag-name-filter?
> 
> To be honest, I wanted to rip that part out before posting the initial 
> filter-branch patches, and forgot/was too lazy.
> 
> IMO there should be clean semantics first, and a default mode that does 
> _not_ rewrite tags at all first.

There isn't really a "default" mode for --tag-name-filter, a script must
always be supplied. The name of the option (--tag- _name_ -filter) implies
that it is intended for changing the names of tags. If the user decides
not to change the name, then I think the user would expect the old tag to
be replaced.

So the user could choose to:

          Rewrite tag ref: --tag-name-filter cat
                        or
 Give new tags a new name: --tag-name-filter 'sed -e s/$/.new/'
                        or
       Save original tags: --tag-name-filter 'sed -e s/$/.orig/'
     Then create new ones: --tag-name-filter 'sed -e s/\.orig$//' [plus additional filtering]

More precise control over tag filtering should probably have an option
with a different name like --tag-filter, at which point --tag-name-filter
could be deprecated.

-brandon

filter-branch --all?

[Eyvind Bernhardsen]

On 24. mars. 2008, at 16.10, Johannes Schindelin wrote:

[Regarding filter-branch rewriting tags]

> You can say "--all", and I actually expect quite a few people to do
> exactly that.  And then you cannot really say "the user explicitely  
> asked
> to have that signed tag rewritten".


Sorry to hijack the thread.  I've seen you mention using "--all" to  
filter-branch everything before, but I can't get it to work:


	% git filter-branch --all
	Usage: /usr/local/bin/git-filter-branch [--env-filter <command>] [-- 
tree-filter <command>] [--index-filter <command>] [--parent-filter  
<command>] [--msg-filter <command>] [--commit-filter <command>] [--tag- 
name-filter <command>] [--subdirectory-filter <directory>] [--original  
<namespace>] [-d <directory>] [-f | --force] [<rev-list options>...]


I'm trying to rewrite history to match .git/info/grafts without going  
through every graft, hence the lack of an actual filter command.   
Thinking that might be the problem, I also tried the "update-index -- 
remove <filename>" example, but it wouldn't let me use "--all" either:


	% git filter-branch --index-filter 'git update-index --remove RELEASE- 
NOTES' --all
	Usage: /usr/local/bin/git-filter-branch [--env-filter <command>] [-- 
tree-filter <command>] [--index-filter <command>] [--parent-filter  
<command>] [--msg-filter <command>] [--commit-filter <command>] [--tag- 
name-filter <command>] [--subdirectory-filter <directory>] [--original  
<namespace>] [-d <directory>] [-f | --force] [<rev-list options>...]


as opposed to:


	% git filter-branch --index-filter 'git update-index --remove RELEASE- 
NOTES' HEAD
	Rewrite 266da463b4e59eaf06287b36b6435b8218db9f40 (611/4040)
	^C


git version 1.5.5.rc0.10.g7881f.dirty (the dirtiness is in git- 
submodule.sh, and I've tried it with older, cleaner versions).

Am I just stupid and/or crazy, or does it not work?
-- 
Eyvind Bernhardsen

Re: filter-branch --all?

[Brandon Casey]

Eyvind Bernhardsen wrote:
> On 24. mars. 2008, at 16.10, Johannes Schindelin wrote:
> 
> [Regarding filter-branch rewriting tags]
> 
>> You can say "--all", and I actually expect quite a few people to do
>> exactly that.  And then you cannot really say "the user explicitely asked
>> to have that signed tag rewritten".
> 
> 
> Sorry to hijack the thread.  I've seen you mention using "--all" to
> filter-branch everything before, but I can't get it to work:
> 
> 
>     % git filter-branch --all

Try

    git filter-branch -- --all

It is actually an option to rev-parse.

-brandon

Re: filter-branch --all?

[Eyvind Bernhardsen]

On 24. mars. 2008, at 22.33, Brandon Casey wrote:
> Try
>
>    git filter-branch -- --all
>
> It is actually an option to rev-parse.

That works!  Thanks.

I managed to figure out that "--all" is a rev-parse option, but it  
never occurred to me to protect it with a "--".
-- 
Eyvind Bernhardsen

Re: [PATCH 2/2] filter-branch.sh: support nearly proper tag name filtering

[Brandon Casey]

Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
>> On Thu, 20 Mar 2008, Brandon Casey wrote:
>>
>>> Add support for creating a new tag object and retaining the tag message, 
>>> author, and date when rewriting tags. The gpg signature, if one exists, 
>>> will be stripped.
>>>
>>> This adds nearly proper tag name filtering to filter-branch. Proper tag 
>>> name filtering would include the ability to change the tagger, tag date, 
>>> tag message, and _not_ strip a gpg signature if the tag did not change.
>> I think such a change in semantics merits a new option.
> 
> It is a different story if the fix is a good one, or has room for
> improvement.  For example, the sed script that knows what is on the first
> and second line and relies on that knowledge to use 1c...2c... looked very
> fragile to me.

This tag format is described in the documentation for git-mktag as follows:

   Tag Format
   ----------
   A tag signature file has a very simple fixed format: four lines of

     object <sha1>
     type <typename>
     tag <tagname>
     tagger <tagger>

   followed by some 'optional' free-form message (some tags created
   [...]

So, as long as git-cat-file succeeds, shouldn't we get this format piped
to sed? The type is hard coded to "commit", since $new_ref will always be
a commit (the old tag could have referenced another tag), otherwise the
earlier rev-parse would have failed.

The portion that strips the signature, does rely on the fact that git-tag
always adds the signature to the end of the tag object. Rather than
stripping only between the pgp BEGIN and END markers. And the 'Q' directive
may not be portable, but I really don't know.

Suggestions?

-brandon

[PATCH] filter-branch.sh: support nearly proper tag name filtering

[Brandon Casey]

Add support for creating a new tag object and retaining the tag message,
author, and date when rewriting tags. The gpg signature, if one exists,
will be stripped.

This adds nearly proper tag name filtering to filter-branch. Proper tag
name filtering would include the ability to change the tagger, tag date,
tag message, and _not_ strip a gpg signature if the tag did not change.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
---


I learned that the 'Q' command to sed is not portable, and most versions
of sed require a newline after 'c\' like commands.

Here is an updated patch which should be portable. Nothing else has been
changed.

Here's the diff from the previous patch...

  --- a/git-filter-branch.sh
  +++ b/git-filter-branch.sh
  @@ -406,11 +406,16 @@ if [ "$filter_tag_name" ]; then
                  echo "$ref -> $new_ref ($sha1 -> $new_sha1)"
 
                  if [ "$type" = "tag" ]; then
  -                       new_sha1=$(git cat-file tag "$ref" | \
  -                               sed -e "1c\object $new_sha1" \
  -                                   -e '2c\type commit'      \
  -                                   -e "3c\tag $new_ref"     \
  -                                   -e '/^-----BEGIN PGP SIGNATURE-----/Q' | \
  +                       new_sha1=$(git cat-file tag "$ref" |
  +                               sed -n \
  +                                   -e "1c\
  +                                       object $new_sha1" \
  +                                   -e "2c\
  +                                       type commit" \
  +                                   -e "3c\
  +                                       tag $new_ref" \
  +                                   -e '/^-----BEGIN PGP SIGNATURE-----/q' \
  +                                   -e 'p' |
                                  git mktag) ||
                                  die "Could not create new tag object for $ref"
                          if git cat-file tag "$ref" | \


-brandon


 Documentation/git-filter-branch.txt |   14 ++++++++++----
 git-filter-branch.sh                |   19 +++++++++++++++++--
 t/t7003-filter-branch.sh            |   32 ++++++++++++++++++++++++++++++++
 3 files changed, 59 insertions(+), 6 deletions(-)

diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt
index 543a1cf..9364919 100644
--- a/Documentation/git-filter-branch.txt
+++ b/Documentation/git-filter-branch.txt
@@ -132,10 +132,16 @@ use "--tag-name-filter cat" to simply update the tags.  In this
 case, be very careful and make sure you have the old tags
 backed up in case the conversion has run afoul.
 +
-Note that there is currently no support for proper rewriting of
-tag objects; in layman terms, if the tag has a message or signature
-attached, the rewritten tag won't have it.  Sorry.  (It is by
-definition impossible to preserve signatures at any rate.)
+Nearly proper rewriting of tag objects is supported. If the tag has
+a message attached, a new tag object will be created with the same message,
+author, and timestamp. If the tag has a signature attached, the
+signature will be stripped. It is by definition impossible to preserve
+signatures. The reason this is "nearly" proper, is because ideally if
+the tag did not change (points to the same object, has the same name, etc.)
+it should retain any signature. That is not the case, signatures will always
+be removed, buyer beware. There is also no support for changing the
+author or timestamp (or the tag message for that matter). Tags which point
+to other tags will be rewritten to point to the underlying commit.
 
 --subdirectory-filter <directory>::
 	Only look at the history which touches the given subdirectory.
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 22b6ed4..3da3ccd 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -406,8 +406,23 @@ if [ "$filter_tag_name" ]; then
 		echo "$ref -> $new_ref ($sha1 -> $new_sha1)"
 
 		if [ "$type" = "tag" ]; then
-			# Warn that we are not rewriting the tag object itself.
-			warn "unreferencing tag object $sha1t"
+			new_sha1=$(git cat-file tag "$ref" |
+				sed -n \
+				    -e "1c\
+					object $new_sha1" \
+				    -e "2c\
+					type commit" \
+				    -e "3c\
+					tag $new_ref" \
+				    -e '/^-----BEGIN PGP SIGNATURE-----/q' \
+				    -e 'p' |
+				git mktag) ||
+				die "Could not create new tag object for $ref"
+			if git cat-file tag "$ref" | \
+			   grep '^-----BEGIN PGP SIGNATURE-----' >/dev/null 2>&1
+			then
+				warn "gpg signature stripped from tag object $sha1t"
+			fi
 		fi
 
 		git update-ref "refs/tags/$new_ref" "$new_sha1" ||
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 6827249..1daaf54 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -203,4 +203,36 @@ test_expect_success 'Subdirectory filter with disappearing trees' '
 	test $(git rev-list master | wc -l) = 3
 '
 
+test_expect_success 'Tag name filtering retains tag message' '
+	git tag -m atag T &&
+	git cat-file tag T > expect &&
+	git filter-branch -f --tag-name-filter cat &&
+	git cat-file tag T > actual &&
+	git diff expect actual
+'
+
+faux_gpg_tag='object XXXXXX
+type commit
+tag S
+tagger T A Gger <tagger@example.com> 1206026339 -0500
+
+This is a faux gpg signed tag.
+-----BEGIN PGP SIGNATURE-----
+Version: FauxGPG v0.0.0 (FAUX/Linux)
+
+gdsfoewhxu/6l06f1kxyxhKdZkrcbaiOMtkJUA9ITAc1mlamh0ooasxkH1XwMbYQ
+acmwXaWET20H0GeAGP+7vow=
+=agpO
+-----END PGP SIGNATURE-----
+'
+test_expect_success 'Tag name filtering strips gpg signature' '
+	sha1=$(git rev-parse HEAD) &&
+	sha1t=$(echo "$faux_gpg_tag" | sed -e s/XXXXXX/$sha1/ | git mktag) &&
+	git update-ref "refs/tags/S" "$sha1t" &&
+	echo "$faux_gpg_tag" | sed -e s/XXXXXX/$sha1/ | head -n 6 > expect &&
+	git filter-branch -f --tag-name-filter cat &&
+	git cat-file tag S > actual &&
+	git diff expect actual
+'
+
 test_done
-- 
1.5.4.4.481.g5075

Re: [PATCH] filter-branch.sh: support nearly proper tag name filtering

[Brandon Casey]

I probably should have called this v2. Sorry.

-brandon

Re: [PATCH] filter-branch.sh: support nearly proper tag name filtering

[Junio C Hamano]

Brandon Casey <casey@nrlssc.navy.mil> writes:

> I learned that the 'Q' command to sed is not portable, and most versions
> of sed require a newline after 'c\' like commands.

Both true.  If you want to be portable to esoteric ones (I suffered AIX
while hacking on autoconf long time ago), sed is a bitch to get right.

What I meant with my earlier comment was that it would have been more
robust if you did a sequence of:

	s/type .*/type $newtype/
        s/object .*/object $newobject/

instead of relying on the order in the existing objects, but I should have
been more explicit about c\ as well.

[PATCH v3] filter-branch.sh: support nearly proper tag name filtering

[Brandon Casey]

Add support for creating a new tag object and retaining the tag message,
author, and date when rewriting tags. The gpg signature, if one exists,
will be stripped.

This adds nearly proper tag name filtering to filter-branch. Proper tag
name filtering would include the ability to change the tagger, tag date,
tag message, and _not_ strip a gpg signature if the tag did not change.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
---


Junio C Hamano wrote:
> What I meant with my earlier comment was that it would have been more
> robust if you did a sequence of:
> 
> 	s/type .*/type $newtype/
>         s/object .*/object $newobject/
> 
> instead of relying on the order in the existing objects, but I should have
> been more explicit about c\ as well.

Yep, that is better. This version implementing your suggestions won't silently
create a valid tag from one that mktag would have barfed on.

If you think the strict ordering that mktag requires could be relaxed in the
future, then the line number addresses in sed could be changed to ranges
like:

	-e "1,4s/^object ..*/object $newobject/"

Of course, if we use ranges, we may have to worry about matching against
the tag body if the header gets any longer.

-brandon


 Documentation/git-filter-branch.txt |   14 ++++++++++----
 git-filter-branch.sh                |   16 ++++++++++++++--
 t/t7003-filter-branch.sh            |   32 ++++++++++++++++++++++++++++++++
 3 files changed, 56 insertions(+), 6 deletions(-)

diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt
index 2a78549..9d11b41 100644
--- a/Documentation/git-filter-branch.txt
+++ b/Documentation/git-filter-branch.txt
@@ -133,10 +133,16 @@ use "--tag-name-filter cat" to simply update the tags.  In this
 case, be very careful and make sure you have the old tags
 backed up in case the conversion has run afoul.
 +
-Note that there is currently no support for proper rewriting of
-tag objects; in layman terms, if the tag has a message or signature
-attached, the rewritten tag won't have it.  Sorry.  (It is by
-definition impossible to preserve signatures at any rate.)
+Nearly proper rewriting of tag objects is supported. If the tag has
+a message attached, a new tag object will be created with the same message,
+author, and timestamp. If the tag has a signature attached, the
+signature will be stripped. It is by definition impossible to preserve
+signatures. The reason this is "nearly" proper, is because ideally if
+the tag did not change (points to the same object, has the same name, etc.)
+it should retain any signature. That is not the case, signatures will always
+be removed, buyer beware. There is also no support for changing the
+author or timestamp (or the tag message for that matter). Tags which point
+to other tags will be rewritten to point to the underlying commit.
 
 --subdirectory-filter <directory>::
 	Only look at the history which touches the given subdirectory.
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 22b6ed4..05f7c7b 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -406,8 +406,20 @@ if [ "$filter_tag_name" ]; then
 		echo "$ref -> $new_ref ($sha1 -> $new_sha1)"
 
 		if [ "$type" = "tag" ]; then
-			# Warn that we are not rewriting the tag object itself.
-			warn "unreferencing tag object $sha1t"
+			new_sha1=$(git cat-file tag "$ref" |
+				sed -n \
+				    -e "1s/^object ..*/object $new_sha1/"  \
+				    -e '2s/^type ..*/type commit/'         \
+				    -e "3s/^tag ..*/tag $new_ref/"         \
+				    -e '/^-----BEGIN PGP SIGNATURE-----/q' \
+				    -e 'p' |
+				git mktag) ||
+				die "Could not create new tag object for $ref"
+			if git cat-file tag "$ref" | \
+			   grep '^-----BEGIN PGP SIGNATURE-----' >/dev/null 2>&1
+			then
+				warn "gpg signature stripped from tag object $sha1t"
+			fi
 		fi
 
 		git update-ref "refs/tags/$new_ref" "$new_sha1" ||
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 6827249..1daaf54 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -203,4 +203,36 @@ test_expect_success 'Subdirectory filter with disappearing trees' '
 	test $(git rev-list master | wc -l) = 3
 '
 
+test_expect_success 'Tag name filtering retains tag message' '
+	git tag -m atag T &&
+	git cat-file tag T > expect &&
+	git filter-branch -f --tag-name-filter cat &&
+	git cat-file tag T > actual &&
+	git diff expect actual
+'
+
+faux_gpg_tag='object XXXXXX
+type commit
+tag S
+tagger T A Gger <tagger@example.com> 1206026339 -0500
+
+This is a faux gpg signed tag.
+-----BEGIN PGP SIGNATURE-----
+Version: FauxGPG v0.0.0 (FAUX/Linux)
+
+gdsfoewhxu/6l06f1kxyxhKdZkrcbaiOMtkJUA9ITAc1mlamh0ooasxkH1XwMbYQ
+acmwXaWET20H0GeAGP+7vow=
+=agpO
+-----END PGP SIGNATURE-----
+'
+test_expect_success 'Tag name filtering strips gpg signature' '
+	sha1=$(git rev-parse HEAD) &&
+	sha1t=$(echo "$faux_gpg_tag" | sed -e s/XXXXXX/$sha1/ | git mktag) &&
+	git update-ref "refs/tags/S" "$sha1t" &&
+	echo "$faux_gpg_tag" | sed -e s/XXXXXX/$sha1/ | head -n 6 > expect &&
+	git filter-branch -f --tag-name-filter cat &&
+	git cat-file tag S > actual &&
+	git diff expect actual
+'
+
 test_done
-- 
1.5.5.rc1.1.gb5ed86.dirty

Re: [PATCH v3] filter-branch.sh: support nearly proper tag name filtering

[Junio C Hamano]

Brandon Casey <casey@nrlssc.navy.mil> writes:

> If you think the strict ordering that mktag requires could be relaxed in the
> future, then the line number addresses in sed could be changed to ranges
> like:
>
> 	-e "1,4s/^object ..*/object $newobject/"
>
> Of course, if we use ranges, we may have to worry about matching against
> the tag body if the header gets any longer.

I do not think you have to worry if you did this:

        1,/^$/{
                s/object .*/object $newobject/
                s/type .*/type $newtype/
                ...
        }

[PATCH v4] filter-branch.sh: support nearly proper tag name filtering

[Brandon Casey]

Add support for creating a new tag object and retaining the tag message,
author, and date when rewriting tags. The gpg signature, if one exists,
will be stripped.

This adds nearly proper tag name filtering to filter-branch. Proper tag
name filtering would include the ability to change the tagger, tag date,
tag message, and _not_ strip a gpg signature if the tag did not change.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
---


Junio C Hamano wrote:
> Brandon Casey <casey@nrlssc.navy.mil> writes:
>> Of course, if we use ranges, we may have to worry about matching against
>> the tag body if the header gets any longer.
> 
> I do not think you have to worry if you did this:
> 
>         1,/^$/{
>                 s/object .*/object $newobject/
>                 s/type .*/type $newtype/
>                 ...
>         }

I am a sed light-weight. Thanks for teaching.

-brandon


 Documentation/git-filter-branch.txt |   14 ++++++++++----
 git-filter-branch.sh                |   18 ++++++++++++++++--
 t/t7003-filter-branch.sh            |   32 ++++++++++++++++++++++++++++++++
 3 files changed, 58 insertions(+), 6 deletions(-)

diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt
index 2a78549..9d11b41 100644
--- a/Documentation/git-filter-branch.txt
+++ b/Documentation/git-filter-branch.txt
@@ -133,10 +133,16 @@ use "--tag-name-filter cat" to simply update the tags.  In this
 case, be very careful and make sure you have the old tags
 backed up in case the conversion has run afoul.
 +
-Note that there is currently no support for proper rewriting of
-tag objects; in layman terms, if the tag has a message or signature
-attached, the rewritten tag won't have it.  Sorry.  (It is by
-definition impossible to preserve signatures at any rate.)
+Nearly proper rewriting of tag objects is supported. If the tag has
+a message attached, a new tag object will be created with the same message,
+author, and timestamp. If the tag has a signature attached, the
+signature will be stripped. It is by definition impossible to preserve
+signatures. The reason this is "nearly" proper, is because ideally if
+the tag did not change (points to the same object, has the same name, etc.)
+it should retain any signature. That is not the case, signatures will always
+be removed, buyer beware. There is also no support for changing the
+author or timestamp (or the tag message for that matter). Tags which point
+to other tags will be rewritten to point to the underlying commit.
 
 --subdirectory-filter <directory>::
 	Only look at the history which touches the given subdirectory.
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 22b6ed4..ee90660 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -406,8 +406,22 @@ if [ "$filter_tag_name" ]; then
 		echo "$ref -> $new_ref ($sha1 -> $new_sha1)"
 
 		if [ "$type" = "tag" ]; then
-			# Warn that we are not rewriting the tag object itself.
-			warn "unreferencing tag object $sha1t"
+			new_sha1=$(git cat-file tag "$ref" |
+				sed -n \
+				    -e "1,/^$/{
+					  s/^object .*/object $new_sha1/
+					  s/^type .*/type commit/
+					  s/^tag .*/tag $new_ref/
+					}" \
+				    -e '/^-----BEGIN PGP SIGNATURE-----/q' \
+				    -e 'p' |
+				git mktag) ||
+				die "Could not create new tag object for $ref"
+			if git cat-file tag "$ref" | \
+			   grep '^-----BEGIN PGP SIGNATURE-----' >/dev/null 2>&1
+			then
+				warn "gpg signature stripped from tag object $sha1t"
+			fi
 		fi
 
 		git update-ref "refs/tags/$new_ref" "$new_sha1" ||
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 6827249..1daaf54 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -203,4 +203,36 @@ test_expect_success 'Subdirectory filter with disappearing trees' '
 	test $(git rev-list master | wc -l) = 3
 '
 
+test_expect_success 'Tag name filtering retains tag message' '
+	git tag -m atag T &&
+	git cat-file tag T > expect &&
+	git filter-branch -f --tag-name-filter cat &&
+	git cat-file tag T > actual &&
+	git diff expect actual
+'
+
+faux_gpg_tag='object XXXXXX
+type commit
+tag S
+tagger T A Gger <tagger@example.com> 1206026339 -0500
+
+This is a faux gpg signed tag.
+-----BEGIN PGP SIGNATURE-----
+Version: FauxGPG v0.0.0 (FAUX/Linux)
+
+gdsfoewhxu/6l06f1kxyxhKdZkrcbaiOMtkJUA9ITAc1mlamh0ooasxkH1XwMbYQ
+acmwXaWET20H0GeAGP+7vow=
+=agpO
+-----END PGP SIGNATURE-----
+'
+test_expect_success 'Tag name filtering strips gpg signature' '
+	sha1=$(git rev-parse HEAD) &&
+	sha1t=$(echo "$faux_gpg_tag" | sed -e s/XXXXXX/$sha1/ | git mktag) &&
+	git update-ref "refs/tags/S" "$sha1t" &&
+	echo "$faux_gpg_tag" | sed -e s/XXXXXX/$sha1/ | head -n 6 > expect &&
+	git filter-branch -f --tag-name-filter cat &&
+	git cat-file tag S > actual &&
+	git diff expect actual
+'
+
 test_done
-- 
1.5.4.4.481.g5075