[PATCH 0/2] fix t3010 failure when core.ignorecase=true

[PATCH 0/2] fix t3010 failure when core.ignorecase=true

[Eric Sunshine]

This series fixes a bug in dir.c which causes t3010 to fail [1] when
core.ignorecase is true. The problem is that
directory_exists_in_index(dirname,len) and
directory_exists_in_index_icase() behave differently if dirname[len] is
not a '/', even though this is beyond end-of-string.  2eac2a4cc4bdc8d7
(ls-files -k: a directory only can be killed if the index has a
non-directory; 2013-08-15) adds a caller which neglects to ensure that
the the required '/' is present, hence the failure.

I am not happy with the fix, which is too add a '/' after the last
character in dirname just at the call site introduced by
2eac2a4cc4bdc8d7.

The reason for my unhappiness is that directory_exists_in_index_icase()
makes the assumption, not only that it can access the character beyond
the end-of-string, but also that that character will unconditionally be
'/'. I presume that this was done for the sake of speed (existing
callers always had a '/' beyond end-of-string), but it feels like an
ugly wart. Since the required trailing '/' is purely an implementation
detail of directory_exists_in_index_icase(), and not of
directory_exists_in_index(), a cleaner fix would be for
directory_exists_in_index_icase() to add the '/' it needs, and not
expect the passed in dirname to have a '/' after its last character.
Unfortunately, such a fix would probably negate any optimization benefit
gained by the present implementation.

[1]: http://thread.gmane.org/gmane.comp.version-control.git/232727

Eric Sunshine (2):
  t3103: demonstrate dir.c:treat_one_path() core.ignorecase failure
  dir: test_one_path: fix inconsistent behavior due to missing '/'

 dir.c                   | 12 +++++++++---
 t/t3103-ls-tree-misc.sh | 15 +++++++++++++++
 2 files changed, 24 insertions(+), 3 deletions(-)

-- 
1.8.4.rc4.529.g78818d7

[PATCH 1/2] t3103: demonstrate dir.c:treat_one_path() core.ignorecase failure

[Eric Sunshine]

2eac2a4cc4bdc8d7 (ls-files -k: a directory only can be killed if the
index has a non-directory; 2013-08-15) adds a caller of
directory_exists_in_index(dirname,len) which forgets to satisfy the
undocumented requirement that a '/' must be present at dirname[len]
(despite being past the end-of-string). This oversight leads to
incorrect behavior when core.ignorecase is true. Demonstrate this.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 t/t3103-ls-tree-misc.sh | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/t/t3103-ls-tree-misc.sh b/t/t3103-ls-tree-misc.sh
index 09dcf04..fd95991 100755
--- a/t/t3103-ls-tree-misc.sh
+++ b/t/t3103-ls-tree-misc.sh
@@ -4,6 +4,7 @@ test_description='
 Miscellaneous tests for git ls-tree.
 
 	      1. git ls-tree fails in presence of tree damage.
+	      2. git ls-tree acts sanely with core.ignorecase.
 
 '
 
@@ -21,4 +22,18 @@ test_expect_success 'ls-tree fails with non-zero exit code on broken tree' '
 	test_must_fail git ls-tree -r HEAD
 '
 
+test_expect_failure 'ls-tree directory core.ignorecase' '
+	cat >expect <<-\EOF &&
+	d/e/f
+	EOF
+	mkdir d &&
+	>d/e &&
+	git update-index --add -- d/e &&
+	rm d/e &&
+	mkdir d/e &&
+	>d/e/f &&
+	git -c core.ignorecase=true ls-files -k >actual &&
+	test_cmp expect actual
+'
+
 test_done
-- 
1.8.4.rc4.529.g78818d7

Re: [PATCH 1/2] t3103: demonstrate dir.c:treat_one_path() core.ignorecase failure

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> @@ -21,4 +22,18 @@ test_expect_success 'ls-tree fails with non-zero exit code on broken tree' '
>  	test_must_fail git ls-tree -r HEAD
>  '
>  
> +test_expect_failure 'ls-tree directory core.ignorecase' '
> +	cat >expect <<-\EOF &&
> +	d/e/f
> +	EOF
> +	mkdir d &&
> +	>d/e &&
> +	git update-index --add -- d/e &&
> +	rm d/e &&
> +	mkdir d/e &&
> +	>d/e/f &&
> +	git -c core.ignorecase=true ls-files -k >actual &&
> +	test_cmp expect actual
> +'

Hmm. Wouldn't it be a clearer demonstration to add to t3010 another
invocation of "ls-files -k" that was added by 3c568751 (t3010:
update to demonstrate "ls-files -k" optimization pitfalls,
2013-08-15) but with core.ignorecase=true? Something like...

 t/t3010-ls-files-killed-modified.sh | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/t/t3010-ls-files-killed-modified.sh b/t/t3010-ls-files-killed-modified.sh
index 6ea7ca8..f4783ed 100755
--- a/t/t3010-ls-files-killed-modified.sh
+++ b/t/t3010-ls-files-killed-modified.sh
@@ -90,6 +90,17 @@ pathx/ju/nk
 EOF
 
 test_expect_success \
+    'git ls-files -k to show killed files (with icase)' \
+    'git -c core.ignorecase=true ls-files -k >.output'
+cat >.expected <<EOF
+path0/file0
+path1/file1
+path2
+path3
+pathx/ju/nk
+EOF
+
+test_expect_success \
     'validate git ls-files -k output.' \
     'test_cmp .expected .output'
 

[PATCH 2/2] dir: test_one_path: fix inconsistent behavior due to missing '/'

[Eric Sunshine]

Although undocumented, directory_exists_in_index_icase(dirname,len)
unconditionally assumes the presence of a '/' at dirname[len] (despite
being past the end-of-string). Callers are expected to respect this
assumption by ensuring that a '/' is present beyond the last character
of the passed path. directory_exists_in_index(), on the other hand,
does not assume nor care about a trailing '/' beyond end-of-string.

2eac2a4cc4bdc8d7 (ls-files -k: a directory only can be killed if the
index has a non-directory; 2013-08-15) adds a caller which forgets to
ensure the trailing '/', thus leading to inconsistent behavior between
directory_exists_in_index() and directory_exists_in_index_icase()
depending upon the setting of core.ignorecase. Fix this problem.

This also fixes an initially-unnoticed failure in a t3010 test added by
3c56875176390eee (t3010: update to demonstrate "ls-files -k"
optimization pitfalls; 2013-08-15) when core.ignorecase is true.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 dir.c                   | 12 +++++++++---
 t/t3103-ls-tree-misc.sh |  2 +-
 2 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/dir.c b/dir.c
index edd666a..a52c6f9 100644
--- a/dir.c
+++ b/dir.c
@@ -1160,9 +1160,15 @@ static enum path_treatment treat_one_path(struct dir_struct *dir,
 	 */
 	if ((dir->flags & DIR_COLLECT_KILLED_ONLY) &&
 	    (dtype == DT_DIR) &&
-	    !has_path_in_index &&
-	    (directory_exists_in_index(path->buf, path->len) == index_nonexistent))
-		return path_none;
+	    !has_path_in_index) {
+		strbuf_addch(path, '/');
+		if (directory_exists_in_index(path->buf, path->len - 1) ==
+		    index_nonexistent) {
+			strbuf_setlen(path, path->len - 1);
+			return path_none;
+		}
+		strbuf_setlen(path, path->len - 1);
+	}
 
 	exclude = is_excluded(dir, path->buf, &dtype);
 
diff --git a/t/t3103-ls-tree-misc.sh b/t/t3103-ls-tree-misc.sh
index fd95991..9fb1706 100755
--- a/t/t3103-ls-tree-misc.sh
+++ b/t/t3103-ls-tree-misc.sh
@@ -22,7 +22,7 @@ test_expect_success 'ls-tree fails with non-zero exit code on broken tree' '
 	test_must_fail git ls-tree -r HEAD
 '
 
-test_expect_failure 'ls-tree directory core.ignorecase' '
+test_expect_success 'ls-tree directory core.ignorecase' '
 	cat >expect <<-\EOF &&
 	d/e/f
 	EOF
-- 
1.8.4.rc4.529.g78818d7

Re: [PATCH 2/2] dir: test_one_path: fix inconsistent behavior due to missing '/'

[Jonathan Nieder]

Eric Sunshine wrote:

> Although undocumented, directory_exists_in_index_icase(dirname,len)
> unconditionally assumes the presence of a '/' at dirname[len] (despite
> being past the end-of-string). Callers are expected to respect
[...]
>                                                Fix this problem.

So, does this fix the problem by changing
directory_exists_in_index_icase() to be more liberal in what it
accepts, or callers to be more conservative in what they pass in?

Please forgive my laziness.  I ask in order to save future readers the
time of digging into the code.

Thanks,
Jonathan

Re: [PATCH 2/2] dir: test_one_path: fix inconsistent behavior due to missing '/'

[Eric Sunshine]

On Sun, Aug 25, 2013 at 2:00 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Eric Sunshine wrote:
>
>> Although undocumented, directory_exists_in_index_icase(dirname,len)
>> unconditionally assumes the presence of a '/' at dirname[len] (despite
>> being past the end-of-string). Callers are expected to respect
> [...]
>>                                                Fix this problem.
>
> So, does this fix the problem by changing
> directory_exists_in_index_icase() to be more liberal in what it
> accepts, or callers to be more conservative in what they pass in?

It places the onus upon the caller. As mentioned in the cover letter
[1], I was not happy with this solution. Junio felt likewise. A
follow-up series [2] fixes both directory_exists_in_index() and
directory_exists_in_index_icase() to be more liberal in what they
accept, relieving the caller of the burden. By the time that series
was posted, however, Junio and Peff had decided that a fix at a more
fundamental level would be better (a conclusion with which I agree,
but for which I do not yet have sufficient knowledge about git
internals to implement). In the meantime, as an interim bug fix, Junio
decided [3] to apply the patch to which you responded (but with an
updated commit message).

[1]: http://thread.gmane.org/gmane.comp.version-control.git/232796
[2]: http://thread.gmane.org/gmane.comp.version-control.git/232833
[3]: http://thread.gmane.org/gmane.comp.version-control.git/232833/focus=232837

> Please forgive my laziness.  I ask in order to save future readers the
> time of digging into the code.

Re: [PATCH 2/2] dir: test_one_path: fix inconsistent behavior due to missing '/'

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> ... A
> follow-up series [2] fixes both directory_exists_in_index() and
> directory_exists_in_index_icase() to be more liberal in what they
> accept, relieving the caller of the burden. By the time that series
> was posted, however, Junio and Peff had decided that a fix at a more
> fundamental level would be better (a conclusion with which I agree,

Thanks for a summary; the last "would be better" is a qualified one,
though.

My understanding is that we agreed that it would be better *if* we
can fix it at a more fundamental level. I looked at the codepaths
involved just enough to make that off the cuff suggestion and no
deeper than that, and I suspect neither did Peff.

If any of us looked at the problem deep enough, we may realize that
it would affect too many things, and $gmane/232833/focus=232835
(your second round) may turn out to be a better solution.  I think
none of us know yet.