[PATCH] t4130: work around Windows limitation

[PATCH] t4130: work around Windows limitation

[Johannes Schindelin]

On Windows, it is already pretty expensive to try to recreate the stat()
data that Git assumes is cheap to obtain. To make things halfway decent
in performance, we even have to skip emulating the inode and to
determine the number of hard links.

This is not a huge problem, usually, as either the size or the mtime or
the ctime are tell-tale enough to say when a file has changed, and even
if not, those changes are typically made after the index file was
written, triggering a rehashing of the files' contents.

The t4130-apply-criss-cross-rename test case, however, requires the
inode to determine that files of equal size were swapped, as renaming
files does not update their mtime. And even if we use
nanosecond-precision mtimes on Windows, the file system's time
granularity is typically much coarser (100ms for NTFS, 2s for FAT).

That means that every once in a while, t4130 fails on Windows.

This patch provides the work-around by pretending that the index file
was written earlier than it actually was.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
Published-As: https://github.com/dscho/git/releases/tag/t4130-mingw-v1
 t/t4130-apply-criss-cross-rename.sh | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/t/t4130-apply-criss-cross-rename.sh b/t/t4130-apply-criss-cross-rename.sh
index d173acd..ee91af8 100755
--- a/t/t4130-apply-criss-cross-rename.sh
+++ b/t/t4130-apply-criss-cross-rename.sh
@@ -29,6 +29,14 @@ test_expect_success 'criss-cross rename' '
 '
 
 test_expect_success 'diff -M -B' '
+	if test_have_prereq MINGW
+	then
+		# On Windows it is prohbitively expensive to retrieve the
+		# equivalent of an "inode" when calling stat(), therefore we
+		# rely on mtime/ctime/size changes to let us know whether a
+		# file has changed, including the mtime relative to the index.
+		test-chmtime -1 .git/index
+	fi &&
 	git diff -M -B > diff &&
 	git reset --hard
 
@@ -52,6 +60,14 @@ test_expect_success 'criss-cross rename' '
 '
 
 test_expect_success 'diff -M -B' '
+	if test_have_prereq MINGW
+	then
+		# On Windows it is prohbitively expensive to retrieve the
+		# equivalent of an "inode" when calling stat(), therefore we
+		# rely on mtime/ctime/size changes to let us know whether a
+		# file has changed, including the mtime relative to the index.
+		test-chmtime -1 .git/index
+	fi &&
 	git diff -M -B > diff &&
 	git reset --hard
 '
-- 
2.9.0.281.g286a8d9

base-commit: 8c6d1f9807c67532e7fb545a944b064faff0f70b

Re: [PATCH] t4130: work around Windows limitation

[Johannes Sixt]

Am 27.07.2016 um 14:39 schrieb Johannes Schindelin:
> On Windows, it is already pretty expensive to try to recreate the stat()
> data that Git assumes is cheap to obtain. To make things halfway decent
> in performance, we even have to skip emulating the inode and to
> determine the number of hard links.
> 
> This is not a huge problem, usually, as either the size or the mtime or
> the ctime are tell-tale enough to say when a file has changed, and even
> if not, those changes are typically made after the index file was
> written, triggering a rehashing of the files' contents.
> 
> The t4130-apply-criss-cross-rename test case, however, requires the
> inode to determine that files of equal size were swapped, as renaming
> files does not update their mtime. And even if we use
> nanosecond-precision mtimes on Windows, the file system's time
> granularity is typically much coarser (100ms for NTFS, 2s for FAT).
> 
> That means that every once in a while, t4130 fails on Windows.
> 
> This patch provides the work-around by pretending that the index file
> was written earlier than it actually was.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> Published-As: https://github.com/dscho/git/releases/tag/t4130-mingw-v1
>  t/t4130-apply-criss-cross-rename.sh | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
> 
> diff --git a/t/t4130-apply-criss-cross-rename.sh b/t/t4130-apply-criss-cross-rename.sh
> index d173acd..ee91af8 100755
> --- a/t/t4130-apply-criss-cross-rename.sh
> +++ b/t/t4130-apply-criss-cross-rename.sh
> @@ -29,6 +29,14 @@ test_expect_success 'criss-cross rename' '
>  '
>  
>  test_expect_success 'diff -M -B' '
> +	if test_have_prereq MINGW
> +	then
> +		# On Windows it is prohbitively expensive to retrieve the
> +		# equivalent of an "inode" when calling stat(), therefore we
> +		# rely on mtime/ctime/size changes to let us know whether a
> +		# file has changed, including the mtime relative to the index.
> +		test-chmtime -1 .git/index
> +	fi &&
>  	git diff -M -B > diff &&
>  	git reset --hard
>  
> @@ -52,6 +60,14 @@ test_expect_success 'criss-cross rename' '
>  '
>  
>  test_expect_success 'diff -M -B' '
> +	if test_have_prereq MINGW
> +	then
> +		# On Windows it is prohbitively expensive to retrieve the
> +		# equivalent of an "inode" when calling stat(), therefore we
> +		# rely on mtime/ctime/size changes to let us know whether a
> +		# file has changed, including the mtime relative to the index.
> +		test-chmtime -1 .git/index
> +	fi &&
>  	git diff -M -B > diff &&
>  	git reset --hard
>  '
> 

How about this instead? It makes the sizes of the files different,
which is a bit less obscure, IMHO.

---- 8< ----
[PATCH] t4130: work around Windows limitation

On Windows, it is already pretty expensive to try to recreate the stat()
data that Git assumes is cheap to obtain. To make things halfway decent
in performance, we even have to skip emulating the inode and to
determine the number of hard links.

This is not a huge problem, usually, as either the size or the mtime or
the ctime are tell-tale enough to say when a file has changed, and even
if not, those changes are typically made after the index file was
written, triggering a rehashing of the files' contents.

The t4130-apply-criss-cross-rename test case, however, requires the
inode to determine that files of equal size were swapped, as renaming
files does not update their mtime. Every once in a while, t4130 fails
on Windows of this missing piece.

Equal file sizes are not crucial for the test cases, however. Hence,
generate files with different sizes so that there is some property that
the swapped files can be discovered reliably even on Windows.

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 t/t4130-apply-criss-cross-rename.sh | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/t/t4130-apply-criss-cross-rename.sh b/t/t4130-apply-criss-cross-rename.sh
index d173acd..f8a313b 100755
--- a/t/t4130-apply-criss-cross-rename.sh
+++ b/t/t4130-apply-criss-cross-rename.sh
@@ -13,9 +13,13 @@ create_file() {
 }
 
 test_expect_success 'setup' '
-	create_file file1 "File1 contents" &&
-	create_file file2 "File2 contents" &&
-	create_file file3 "File3 contents" &&
+	# Ensure that file sizes are different, because on Windows
+	# lstat() does not discover inode numbers, and we need
+	# other properties to discover swapped files
+	# (mtime is not always different, either).
+	create_file file1 "some content" &&
+	create_file file2 "some other content" &&
+	create_file file3 "again something else" &&
 	git add file1 file2 file3 &&
 	git commit -m 1
 '
-- 
2.9.0.443.ga8520ad

Re: [PATCH] t4130: work around Windows limitation

[Junio C Hamano]

On Wed, Jul 27, 2016 at 9:37 AM, Johannes Sixt <j6t@kdbg.org> wrote:
>
> How about this instead? It makes the sizes of the files different,
> which is a bit less obscure, IMHO.
> ...
> Equal file sizes are not crucial for the test cases, however.

That does sounds like a better solution.

Re: [PATCH] t4130: work around Windows limitation

[Johannes Schindelin]

Hi,

On Wed, 27 Jul 2016, Junio C Hamano wrote:

> On Wed, Jul 27, 2016 at 9:37 AM, Johannes Sixt <j6t@kdbg.org> wrote:
> >
> > How about this instead? It makes the sizes of the files different,
> > which is a bit less obscure, IMHO.
> > ...
> > Equal file sizes are not crucial for the test cases, however.
> 
> That does sounds like a better solution.

Fine by me.

Thanks,
Dscho

[PATCH v2] t4130: work around Windows limitation

[Johannes Sixt]

On Windows, it is already pretty expensive to try to recreate the stat()
data that Git assumes is cheap to obtain. To make things halfway decent
in performance, we even have to skip emulating the inode and to
determine the number of hard links.

This is not a huge problem, usually, as either the size or the mtime or
the ctime are tell-tale enough to say when a file has changed, and even
if not, those changes are typically made after the index file was
written, triggering a rehashing of the files' contents.

The t4130-apply-criss-cross-rename test case, however, requires the
inode to determine that files of equal size were swapped, as renaming
files does not update their mtime. Every once in a while, t4130 fails
on Windows because of this missing piece.

Equal file sizes are not crucial for the test cases, however. Hence,
generate files with different sizes so that there is some property that
the swapped files can be discovered reliably even on Windows.

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 This fell through the cracks, I think. I marked it as v2 because
 there is a minor fixup in the commit message.

 t/t4130-apply-criss-cross-rename.sh | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/t/t4130-apply-criss-cross-rename.sh b/t/t4130-apply-criss-cross-rename.sh
index d173acd..f8a313b 100755
--- a/t/t4130-apply-criss-cross-rename.sh
+++ b/t/t4130-apply-criss-cross-rename.sh
@@ -13,9 +13,13 @@ create_file() {
 }
  test_expect_success 'setup' '
-	create_file file1 "File1 contents" &&
-	create_file file2 "File2 contents" &&
-	create_file file3 "File3 contents" &&
+	# Ensure that file sizes are different, because on Windows
+	# lstat() does not discover inode numbers, and we need
+	# other properties to discover swapped files
+	# (mtime is not always different, either).
+	create_file file1 "some content" &&
+	create_file file2 "some other content" &&
+	create_file file3 "again something else" &&
 	git add file1 file2 file3 &&
 	git commit -m 1
 '
-- 
2.9.0.443.ga8520ad

Re: [PATCH v2] t4130: work around Windows limitation

[Johannes Schindelin]

Hi Hannes,

On Wed, 3 Aug 2016, Johannes Sixt wrote:

>  This fell through the cracks, I think. I marked it as v2 because
>  there is a minor fixup in the commit message.

It appears that it did fall through the cracks.

Given that it would make my life substantially easier (because I run the
test suite often enough to hit this edge case multiple times a day), I
would like to ask to fast-track it into v2.9.3.

Thanks,
Dscho

Re: [PATCH v2] t4130: work around Windows limitation

[Junio C Hamano]

Johannes Sixt <j6t@kdbg.org> writes:

>  This fell through the cracks, I think. I marked it as v2 because
>  there is a minor fixup in the commit message.

Thanks.  The patch itself seems to got whitespace damaged somewhere
between you and me, which I fixed up, but there may be similar
damage to the proposed log message as well.  What I read may already
have whitespace damaged but I read the message twice and it made
sense to me, so let's assume all is well ;-)

Thanks.

>
>  t/t4130-apply-criss-cross-rename.sh | 10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/t/t4130-apply-criss-cross-rename.sh b/t/t4130-apply-criss-cross-rename.sh
> index d173acd..f8a313b 100755
> --- a/t/t4130-apply-criss-cross-rename.sh
> +++ b/t/t4130-apply-criss-cross-rename.sh
> @@ -13,9 +13,13 @@ create_file() {
>  }
>   test_expect_success 'setup' '
> -	create_file file1 "File1 contents" &&
> -	create_file file2 "File2 contents" &&
> -	create_file file3 "File3 contents" &&
> +	# Ensure that file sizes are different, because on Windows
> +	# lstat() does not discover inode numbers, and we need
> +	# other properties to discover swapped files
> +	# (mtime is not always different, either).
> +	create_file file1 "some content" &&
> +	create_file file2 "some other content" &&
> +	create_file file3 "again something else" &&
>  	git add file1 file2 file3 &&
>  	git commit -m 1
>  '

Re: [PATCH v2] t4130: work around Windows limitation

[Johannes Sixt]

Am 03.08.2016 um 17:50 schrieb Junio C Hamano:
> Johannes Sixt <j6t@kdbg.org> writes:
> The patch itself seems to got whitespace damaged somewhere
> between you and me, which I fixed up,

Sorry for the damaged patch. I forgot that Thunderbird's "Send again" 
feature as well as copying and pasting between Thunderbird windows is 
poison for patch text.

What you have queued in js/t4130-rename-without-ino is correct. Thank 
you very much!

-- Hannes