[BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

[BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

[Remi Vanicat]

git filter-branch --tree-filter has a problem with filename with
accentuated letter:

$ git add foo/baré
$ git commit -m "adding a file with an accent"
Created initial commit b27ae97: adding a file with an accent
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 "foo/bar\303\251"
$ git filter-branch --tree-filter "rm -rf foo"
Rewrite b27ae977459379e4e7eee1a3d523f908903ea6ae (1/1)
WARNING: Ref 'refs/heads/master' is unchanged

there the foo/baré file still exists, but:
$ git filter-branch --tree-filter "rm -rf foo; git add -u"
will suppress the said file from history.

The culprit seem to be those line of filter-branch: (around line 279) 
		git diff-index -r $commit | cut -f 2- | tr '\012' '\000' | \
			xargs -0 git update-index --add --replace --remove
git diff-index giving the filename as "foo/bar\303\251"


-- 
Rémi Vanicat

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

[Junio C Hamano]

Remi Vanicat <vanicat@debian.org> writes:

> The culprit seem to be those line of filter-branch: (around line 279) 
> 		git diff-index -r $commit | cut -f 2- | tr '\012' '\000' | \
> 			xargs -0 git update-index --add --replace --remove
> git diff-index giving the filename as "foo/bar\303\251"

I have to wonder in what century filter-branch was written ;-)

Shouldn't those two lines be:

	git diff-index -r --name-only $commit |
        git update-index --add --replace --remove --stdin

these days, without any of the cut and cruft?

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

[Johannes Schindelin]

Hi,

On Fri, 15 Feb 2008, Junio C Hamano wrote:

> Remi Vanicat <vanicat@debian.org> writes:
> 
> > The culprit seem to be those line of filter-branch: (around line 279) 
> > 		git diff-index -r $commit | cut -f 2- | tr '\012' '\000' | \
> > 			xargs -0 git update-index --add --replace --remove
> > git diff-index giving the filename as "foo/bar\303\251"
> 
> I have to wonder in what century filter-branch was written ;-)
> 
> Shouldn't those two lines be:
> 
> 	git diff-index -r --name-only $commit |
>         git update-index --add --replace --remove --stdin
> 
> these days, without any of the cut and cruft?

Maybe even using "-z" in both cases?

Having said that, I do not understand why the old code did not work.  Will 
have a look later today.

Ciao,
Dscho

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> I have to wonder in what century filter-branch was written ;-)
>> 
>> Shouldn't those two lines be:
>> 
>> 	git diff-index -r --name-only $commit |
>>         git update-index --add --replace --remove --stdin
>> 
>> these days, without any of the cut and cruft?
>
> Maybe even using "-z" in both cases?
>
> Having said that, I do not understand why the old code did not work.  Will 
> have a look later today.

The reason mine does not have to use -z is because both end
knows how to C-quote paths under non-z mode.

Now you mention it, it certainly is a bit puzzling why the old
one did not work.

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

[Johannes Schindelin]

Hi,

On Fri, 15 Feb 2008, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > Having said that, I do not understand why the old code did not work.  
> > Will have a look later today.
> 
> Now you mention it, it certainly is a bit puzzling why the old one did 
> not work.

Okay, so I will not manage today.  Tomorrow is another day.

Ciao,
Dscho

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

[Johannes Schindelin]

Hi,

On Sat, 16 Feb 2008, Johannes Schindelin wrote:

> On Fri, 15 Feb 2008, Junio C Hamano wrote:
> 
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> > 
> > > Having said that, I do not understand why the old code did not work.  
> > > Will have a look later today.
> > 
> > Now you mention it, it certainly is a bit puzzling why the old one did 
> > not work.
> 
> Okay, so I will not manage today.  Tomorrow is another day.

Ah, now I understand.  The accents are shown quoted, but xargs does not 
know how to unquote them.  Cute.

Ciao,
Dscho

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

[Remi Vanicat]

Junio C Hamano <gitster@pobox.com> writes:

> Remi Vanicat <vanicat@debian.org> writes:
>
>> The culprit seem to be those line of filter-branch: (around line 279) 
>> 		git diff-index -r $commit | cut -f 2- | tr '\012' '\000' | \
>> 			xargs -0 git update-index --add --replace --remove
>> git diff-index giving the filename as "foo/bar\303\251"
>
> I have to wonder in what century filter-branch was written ;-)
>
> Shouldn't those two lines be:
>
> 	git diff-index -r --name-only $commit |
>         git update-index --add --replace --remove --stdin

That solve the bug for me.
-- 
Rémi Vanicat

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

[Junio C Hamano]

Not just those two lines but the next two lines were also very
old fashioned.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---

 * Could you try this one?

 git-filter-branch.sh     |    9 +++++----
 t/t7003-filter-branch.sh |   14 ++++++++++++++
 2 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index ff716ca..49e13f0 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -276,10 +276,11 @@ while read commit parents; do
 		eval "$filter_tree" < /dev/null ||
 			die "tree filter failed: $filter_tree"
 
-		git diff-index -r $commit | cut -f 2- | tr '\012' '\000' | \
-			xargs -0 git update-index --add --replace --remove
-		git ls-files -z --others | \
-			xargs -0 git update-index --add --replace --remove
+		(
+			git diff-index -r --name-only $commit
+			git ls-files --others
+		) |
+		git update-index --add --replace --remove --stdin
 	fi
 
 	eval "$filter_index" < /dev/null ||
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 5f60b22..868babc 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -165,4 +165,18 @@ test_expect_success '"map" works in commit filter' '
 	git rev-parse --verify master
 '
 
+test_expect_success 'Name needing quotes' '
+
+	git checkout -b rerere A &&
+	mkdir foo &&
+	name="れれれ" &&
+	>foo/$name &&
+	git add foo &&
+	git commit -m "Adding a file" &&
+	git filter-branch --tree-filter "rm -fr foo" &&
+	! git ls-files --error-unmatch "foo/$name" &&
+	test $(git rev-parse --verify rerere) != $(git rev-parse --verify A)
+
+'
+
 test_done

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

[Remi Vanicat]

Junio C Hamano <gitster@pobox.com> writes:

> Not just those two lines but the next two lines were also very
> old fashioned.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>
>  * Could you try this one?

It work both on my real word case and on my tests git repositories. 


-- 
Rémi Vanicat