buglet: 'git apply' doesn't handle criss-cross renames

buglet: 'git apply' doesn't handle criss-cross renames

[Linus Torvalds]

Ok, this doesn't come up in real life very much, but it's not impossible, 
and I was generating this test-case to show Al how it's probably going to 
be a real bitch to teach 'patch' how to handle git rename patches (or git 
copy patches) in the general case.

Of course, the way git patches work is that the same source file may show 
up several times (due to copies) or as both a source and a destination 
(criss-cross renames or just rename + create new file), and we always look 
at the _original_ state when we apply a patch fragment. Which makes it 
hard for something like GNU "patch" that is purely linear.

And I was going to send Al an example of git doing that, and I was sure 
that we had handled this at some point, but maybe I'm just wrong. I didn't 
really go back and test it.

The point is, git itself fails here. Although not for any really 
fundamental reason - but simply because we are a bit too tight on some 
error handling, and do some checks wrong.

Git can _generate_ criss-cross merges correctly (although you have to use 
the -B flag to make git try to break the names up):

	mv kernel/sched.c a
	mv kernel/exit.c kernel/sched.c
	mv a kernel/exit.c
	git diff -B -M > diff

and we get a nice correct diff:

	diff --git a/kernel/sched.c b/kernel/exit.c
	similarity index 100%
	rename from kernel/sched.c
	rename to kernel/exit.c
	diff --git a/kernel/exit.c b/kernel/sched.c
	similarity index 100%
	rename from kernel/exit.c
	rename to kernel/sched.c

but if you actually try to apply such a patch, we fail miserably (you need 
to got a 'git reset --hard' or similar to get back the original tree to 
apply the diff on, obviously):

	[torvalds@nehalem linux]$ git apply diff
	error: kernel/exit.c: already exists in working directory
	error: kernel/sched.c: already exists in working directory

which is really sad.

I don't have a patch for it, and I don't think it's a huge deal in 
practice, but I do think it's seriously wrong how we can generate a diff 
that we then refuse to apply.

			Linus

[PATCH] tests: test applying criss-cross rename patch

[Michał Kiedrowicz]

Signed-off-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
---
 t/t4130-apply-criss-cross-rename.sh |   35 +++++++++++++++++++++++++++++++++++
 1 files changed, 35 insertions(+), 0 deletions(-)
 create mode 100755 t/t4130-apply-criss-cross-rename.sh

diff --git a/t/t4130-apply-criss-cross-rename.sh b/t/t4130-apply-criss-cross-rename.sh
new file mode 100755
index 0000000..30187ff
--- /dev/null
+++ b/t/t4130-apply-criss-cross-rename.sh
@@ -0,0 +1,35 @@
+#!/bin/sh
+
+test_description='git apply handling criss-cross rename patch.'
+. ./test-lib.sh
+
+create_file() {
+	for ((i=0; i<100; i++)); do
+		echo "$2" >> "$1"
+	done
+}
+
+test_expect_success 'setup' '
+	create_file file1 "File1 contents" &&
+	create_file file2 "File2 contents" &&
+	git add file1 file2 &&
+	git commit -m 1
+'
+
+test_expect_success 'criss-cross rename' '
+	mv file1 tmp &&
+	mv file2 file1 &&
+	mv tmp file2
+'
+
+test_expect_success 'diff -M -B' '
+	git diff -M -B > diff &&
+	git reset --hard
+
+'
+
+test_expect_failure 'apply' '
+	git apply diff
+'
+
+test_done
-- 
1.6.0.6

Re: [PATCH] tests: test applying criss-cross rename patch

[Junio C Hamano]

Michał Kiedrowicz <michal.kiedrowicz@gmail.com> writes:

> Signed-off-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
> ---
>  t/t4130-apply-criss-cross-rename.sh |   35 +++++++++++++++++++++++++++++++++++
>  1 files changed, 35 insertions(+), 0 deletions(-)
>  create mode 100755 t/t4130-apply-criss-cross-rename.sh
>
> diff --git a/t/t4130-apply-criss-cross-rename.sh b/t/t4130-apply-criss-cross-rename.sh
> new file mode 100755
> index 0000000..30187ff
> --- /dev/null
> +++ b/t/t4130-apply-criss-cross-rename.sh
> @@ -0,0 +1,35 @@
> +#!/bin/sh
> +
> +test_description='git apply handling criss-cross rename patch.'
> +. ./test-lib.sh
> +
> +create_file() {
> +	for ((i=0; i<100; i++)); do

Please don't; isn't this a bashism?

> +		echo "$2" >> "$1"
> +	done
> +}
> +
> +test_expect_success 'setup' '
> +	create_file file1 "File1 contents" &&
> +	create_file file2 "File2 contents" &&
> +	git add file1 file2 &&
> +	git commit -m 1
> +'
> +
> +test_expect_success 'criss-cross rename' '
> +	mv file1 tmp &&
> +	mv file2 file1 &&
> +	mv tmp file2
> +'
> +
> +test_expect_success 'diff -M -B' '
> +	git diff -M -B > diff &&
> +	git reset --hard
> +
> +'
> +
> +test_expect_failure 'apply' '
> +	git apply diff
> +'
> +
> +test_done
> -- 
> 1.6.0.6

Re: [PATCH] tests: test applying criss-cross rename patch

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Michał Kiedrowicz <michal.kiedrowicz@gmail.com> writes:
>
>> Signed-off-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
>> ---
>>  t/t4130-apply-criss-cross-rename.sh |   35 +++++++++++++++++++++++++++++++++++
>>  1 files changed, 35 insertions(+), 0 deletions(-)
>>  create mode 100755 t/t4130-apply-criss-cross-rename.sh
>>
>> diff --git a/t/t4130-apply-criss-cross-rename.sh b/t/t4130-apply-criss-cross-rename.sh
>> new file mode 100755
>> index 0000000..30187ff
>> --- /dev/null
>> +++ b/t/t4130-apply-criss-cross-rename.sh
>> @@ -0,0 +1,35 @@
>> +#!/bin/sh
>> +
>> +test_description='git apply handling criss-cross rename patch.'
>> +. ./test-lib.sh
>> +
>> +create_file() {
>> +	for ((i=0; i<100; i++)); do
>
> Please don't; isn't this a bashism?

I've queued your two patches with minimum fix-up (not covering all the
points in my comments) to 'pu'.  You may want to take a look and send in
improvement to replace them.

Thanks.

Re: [PATCH] tests: test applying criss-cross rename patch

[Alex Riesen]

2009/4/11 Michał Kiedrowicz <michal.kiedrowicz@gmail.com>:
> +test_expect_success 'criss-cross rename' '
> +       mv file1 tmp &&
> +       mv file2 file1 &&
> +       mv tmp file2
> +'
> +
> +test_expect_success 'diff -M -B' '
> +       git diff -M -B > diff &&
> +       git reset --hard
> +
> +'

This cannot work on systems where ctime is not trusted:
git diff will produce no data, as there are no changes in
metadata (the files are of the same size). Either make
the file sizes different or add a "touch file1 file2".

diff --git a/t/t4130-apply-criss-cross-rename.sh
b/t/t4130-apply-criss-cross-rename.sh
index 8623dbe..1ff049a 100755
--- a/t/t4130-apply-criss-cross-rename.sh
+++ b/t/t4130-apply-criss-cross-rename.sh
@@ -14,7 +14,7 @@ create_file() {

 test_expect_success 'setup' '
 	create_file file1 "File1 contents" &&
-	create_file file2 "File2 contents" &&
+	create_file file2 "Contents of File2" &&
 	git add file1 file2 &&
 	git commit -m 1
 '

Re: [PATCH] tests: test applying criss-cross rename patch

[Junio C Hamano]

Alex Riesen <raa.lkml@gmail.com> writes:

> 2009/4/11 Michał Kiedrowicz <michal.kiedrowicz@gmail.com>:
>> +test_expect_success 'criss-cross rename' '
>> +       mv file1 tmp &&
>> +       mv file2 file1 &&
>> +       mv tmp file2
>> +'
>> +
>> +test_expect_success 'diff -M -B' '
>> +       git diff -M -B > diff &&
>> +       git reset --hard
>> +
>> +'
>
> This cannot work on systems where ctime is not trusted:
> git diff will produce no data, as there are no changes in
> metadata (the files are of the same size). Either make
> the file sizes different or add a "touch file1 file2".

You seem to be saying that we still have a racy-git bug somewhere.  Is
your statement from an actual experience or a speculation?  If the former
we have a bug to kill, not a workaround to avoid the issue in this test.

>
> diff --git a/t/t4130-apply-criss-cross-rename.sh
> b/t/t4130-apply-criss-cross-rename.sh
> index 8623dbe..1ff049a 100755
> --- a/t/t4130-apply-criss-cross-rename.sh
> +++ b/t/t4130-apply-criss-cross-rename.sh
> @@ -14,7 +14,7 @@ create_file() {
>
>  test_expect_success 'setup' '
>  	create_file file1 "File1 contents" &&
> -	create_file file2 "File2 contents" &&
> +	create_file file2 "Contents of File2" &&
>  	git add file1 file2 &&
>  	git commit -m 1
>  '

Re: [PATCH] tests: test applying criss-cross rename patch

[Alex Riesen]

2009/4/21 Junio C Hamano <gitster@pobox.com>:
> Alex Riesen <raa.lkml@gmail.com> writes:
>
>> 2009/4/11 Michał Kiedrowicz <michal.kiedrowicz@gmail.com>:
>>> +test_expect_success 'criss-cross rename' '
>>> +       mv file1 tmp &&
>>> +       mv file2 file1 &&
>>> +       mv tmp file2
>>> +'
>>> +
>>> +test_expect_success 'diff -M -B' '
>>> +       git diff -M -B > diff &&
>>> +       git reset --hard
>>> +
>>> +'
>>
>> This cannot work on systems where ctime is not trusted:
>> git diff will produce no data, as there are no changes in
>> metadata (the files are of the same size). Either make
>> the file sizes different or add a "touch file1 file2".
>
> You seem to be saying that we still have a racy-git bug somewhere.  Is
> your statement from an actual experience or a speculation?

The test reproducibly fails for me.

> If the former we have a bug to kill, not a workaround to avoid the
> issue in this test.

Maybe. As I see it, there is just not _enough_ metadata in this
particular case to notice the change without doing complete
content comparison: the size is the same, mtime is the same,
dev/ino aren't available, and ctime is marked untrusted and thus
is not used. What's left to notice the change?