[PATCH] git-add--interactive: never skip files included in index

[PATCH] git-add--interactive: never skip files included in index

[Pauli Virtanen]

Make "git add -p" to not skip files that are in index even if they are
excluded (by .gitignore etc.). This fixes the contradictory behavior
that "git status" and "git commit -a" listed such files as modified, but
"git add -p FILENAME" ignored them.

Signed-off-by: Pauli Virtanen <pav@iki.fi>
---
 git-add--interactive.perl  |    2 +-
 t/t3701-add-interactive.sh |   18 ++++++++++++++++++
 2 files changed, 19 insertions(+), 1 deletions(-)

diff --git a/git-add--interactive.perl b/git-add--interactive.perl
index 392efb9..69aeaf0 100755
--- a/git-add--interactive.perl
+++ b/git-add--interactive.perl
@@ -259,7 +259,7 @@ sub list_modified {
 		@tracked = map {
 			chomp $_;
 			unquote_path($_);
-		} run_cmd_pipe(qw(git ls-files --exclude-standard --), @ARGV);
+		} run_cmd_pipe(qw(git ls-files --), @ARGV);
 		return if (!@tracked);
 	}
 
diff --git a/t/t3701-add-interactive.sh b/t/t3701-add-interactive.sh
index 62fd65e..89bfdcb 100755
--- a/t/t3701-add-interactive.sh
+++ b/t/t3701-add-interactive.sh
@@ -138,6 +138,24 @@ test_expect_success 'real edit works' '
 	test_cmp expected output
 '
 
+cat >.gitignore <<EOF
+file
+EOF
+cat >file <<EOF
+changed
+EOF
+test_expect_success 'skip files similarly as commit -a' '
+	git reset
+	echo y | git add -p file &&
+	git diff >output &&
+	git reset &&
+	git commit -am commit &&
+	git diff >expected &&
+	test_cmp expected output &&
+	git reset --hard HEAD^
+'
+rm -f .gitignore
+
 if test "$(git config --bool core.filemode)" = false
 then
 	say 'skipping filemode tests (filesystem does not properly support modes)'
-- 
1.6.3.3

Re: [PATCH] git-add--interactive: never skip files included in index

[Junio C Hamano]

Thanks.

The change looks innocent enough and I do not expect to see any unexpected
regressions from it, but it is a bit too late for 1.6.5 cycle, so let's
queue this fix and aim for 1.6.5.1.

Re: [PATCH] git-add--interactive: never skip files included in index

[Jeff King]

On Sun, Oct 11, 2009 at 11:52:10AM -0700, Junio C Hamano wrote:

> Thanks.
> 
> The change looks innocent enough and I do not expect to see any unexpected
> regressions from it, but it is a bit too late for 1.6.5 cycle, so let's
> queue this fix and aim for 1.6.5.1.

I think this patch is good to apply, as there is no conceivable reason
to even look at excludes when listing modified files.

But this triggered my spider sense; shouldn't --exclude-standard simply
be a no-op for ls-files when we are not listing untracked files? And
bisecting, it seems that it is a very old regression caused by 63d285c
(per-directory-exclude: lazily read .gitignore files, 2007-11-29).

I don't know if it is worth fixing now or not. It does seem a bit
inconsistent to me (since everything else is very clear that .gitignore
is only about untracked files), but nobody seems to have been
complaining for the last two years (and they may have, in fact, been
coding to the new behavior).

-Peff

Re: [PATCH] git-add--interactive: never skip files included in index

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Sun, Oct 11, 2009 at 11:52:10AM -0700, Junio C Hamano wrote:
>
>> Thanks.
>> 
>> The change looks innocent enough and I do not expect to see any unexpected
>> regressions from it, but it is a bit too late for 1.6.5 cycle, so let's
>> queue this fix and aim for 1.6.5.1.
>
> I think this patch is good to apply, as there is no conceivable reason
> to even look at excludes when listing modified files.
>
> But this triggered my spider sense; shouldn't --exclude-standard simply
> be a no-op for ls-files when we are not listing untracked files? And
> bisecting, it seems that it is a very old regression caused by 63d285c
> (per-directory-exclude: lazily read .gitignore files, 2007-11-29).
>
> I don't know if it is worth fixing now or not. It does seem a bit
> inconsistent to me (since everything else is very clear that .gitignore
> is only about untracked files), but nobody seems to have been
> complaining for the last two years (and they may have, in fact, been
> coding to the new behavior).

This is one of those moments when I feel very blessed to have competent
and diligent people around me ;-)

I think you are right; that we shouldn't filter the output with gitignore
entries when showing what is _in_ the index.

Re: [PATCH] git-add--interactive: never skip files included in index

[Jeff King]

On Sun, Oct 11, 2009 at 03:22:05PM -0700, Junio C Hamano wrote:

> > I don't know if it is worth fixing now or not. It does seem a bit
> > inconsistent to me (since everything else is very clear that .gitignore
> > is only about untracked files), but nobody seems to have been
> > complaining for the last two years (and they may have, in fact, been
> > coding to the new behavior).
> 
> This is one of those moments when I feel very blessed to have competent
> and diligent people around me ;-)

Well, I have to do _something_ to make up for all the brown paper bag
bugs. ;)

> I think you are right; that we shouldn't filter the output with gitignore
> entries when showing what is _in_ the index.

So being the diligent and competent soul that I am, I started to prepare
a patch for this, which is included below. But the plot thickens.

I bisected the change of behavior to 63d285c, as I mentioned. But when I
found the code that needed to be tweaked, it actually git-blames to a
much earlier date in "git show-files", which later became ls-files.  See
9ff768e.

So now I am doubly confused. Did this feature exist, and was broken, and
you actually fixed it in 63d285c, creating what looked like a regression
but was actually intentional?

And more importantly, what do you want to do with it now?

---
 builtin-ls-files.c          |    8 --------
 t/t3003-ls-files-exclude.sh |   32 ++++++++++++++++++++++++++++++++
 2 files changed, 32 insertions(+), 8 deletions(-)
 create mode 100755 t/t3003-ls-files-exclude.sh

diff --git a/builtin-ls-files.c b/builtin-ls-files.c
index 2c95ca6..c5c0407 100644
--- a/builtin-ls-files.c
+++ b/builtin-ls-files.c
@@ -170,10 +170,6 @@ static void show_files(struct dir_struct *dir, const char *prefix)
 	if (show_cached | show_stage) {
 		for (i = 0; i < active_nr; i++) {
 			struct cache_entry *ce = active_cache[i];
-			int dtype = ce_to_dtype(ce);
-			if (excluded(dir, ce->name, &dtype) !=
-					!!(dir->flags & DIR_SHOW_IGNORED))
-				continue;
 			if (show_unmerged && !ce_stage(ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
@@ -186,10 +182,6 @@ static void show_files(struct dir_struct *dir, const char *prefix)
 			struct cache_entry *ce = active_cache[i];
 			struct stat st;
 			int err;
-			int dtype = ce_to_dtype(ce);
-			if (excluded(dir, ce->name, &dtype) !=
-					!!(dir->flags & DIR_SHOW_IGNORED))
-				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
 			err = lstat(ce->name, &st);
diff --git a/t/t3003-ls-files-exclude.sh b/t/t3003-ls-files-exclude.sh
new file mode 100755
index 0000000..fc1e379
--- /dev/null
+++ b/t/t3003-ls-files-exclude.sh
@@ -0,0 +1,32 @@
+#!/bin/sh
+
+test_description='ls-files --exclude does not affect index files'
+. ./test-lib.sh
+
+test_expect_success 'create repo with file' '
+	echo content >file &&
+	git add file &&
+	git commit -m file &&
+	echo modification >file
+'
+
+check_output() {
+test_expect_success "ls-files output contains file ($1)" "
+	echo '$2' >expect &&
+	git ls-files --exclude-standard --$1 >output &&
+	test_cmp expect output
+"
+}
+
+check_all_output() {
+	check_output 'cached' 'file'
+	check_output 'modified' 'file'
+}
+
+check_all_output
+test_expect_success 'add file to gitignore' '
+	echo file >.gitignore
+'
+check_all_output
+
+test_done
-- 
1.6.5.rc3.240.g77692

Re: [PATCH] git-add--interactive: never skip files included in index

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> So now I am doubly confused. Did this feature exist, and was broken, and
> you actually fixed it in 63d285c, creating what looked like a regression
> but was actually intentional?

I do not think it was an intentional change.  I _think_ the comment at the
beginning of show_files() tells us quite a bit---we don't do read-dir when
showing the indexed files, and I suspect that we used to rely on the fact
that not doing the read-dir made exclusion mechanism a no-op.  After the
lazy .gitignore reading, I suspect that excluded() call started to
initialize the exclude mechanism lazily, and that is how the bug was
introduced, isn't it?

Regarding your patch, the loops deal with paths that are already in the
index, so removing the conditional skip looks like a sane thing to do.

Re: [PATCH] git-add--interactive: never skip files included in index

[Jeff King]

On Sun, Oct 11, 2009 at 07:46:06PM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > So now I am doubly confused. Did this feature exist, and was broken, and
> > you actually fixed it in 63d285c, creating what looked like a regression
> > but was actually intentional?
> 
> I do not think it was an intentional change.  I _think_ the comment at the
> beginning of show_files() tells us quite a bit---we don't do read-dir when
> showing the indexed files, and I suspect that we used to rely on the fact
> that not doing the read-dir made exclusion mechanism a no-op.  After the
> lazy .gitignore reading, I suspect that excluded() call started to
> initialize the exclude mechanism lazily, and that is how the bug was
> introduced, isn't it?

I did a bit more looking, and the situation is a bit more complex than
that. Hopefully the commit message below explains it.

-- >8 --
Subject: [PATCH] ls-files: excludes should not impact tracked files

In all parts of git, .gitignore and other exclude files
impact only how we treat untracked files; they should have
no effect on files listed in the index.

This behavior was originally implemented very early on in
9ff768e, but only for --exclude-from. Later, commit 63d285c
accidentally caused us to trigger the behavior for
--exclude-per-directory.

This patch totally ignores excludes for files found in the
index. This means we are reversing the original intent of
9ff768e, while at the same time fixing the accidental
behavior of 63d285c. This is a good thing, though, as the
way that 9ff768e behaved does not really make sense with the
way exclusions are used in modern git.

Signed-off-by: Jeff King <peff@peff.net>
---
One other option would be to retain the --exclude-from behavior but
eliminate the --exclude-per-directory behavior. I don't think this is
very easy, though, as it involves separating out which excludes came
from which file. And I think expecting excludes to impact the list of
index files is crazy, anyway, since no other part of git does so. But we
should recognize that we are changing existing behavior; I consider it a
bug fix, though.

I also still think that Pauli's patch makes sense; there is no point in
passing --exclude-standard there. It should be a no-op.

 builtin-ls-files.c          |    8 --------
 t/t3003-ls-files-exclude.sh |   32 ++++++++++++++++++++++++++++++++
 2 files changed, 32 insertions(+), 8 deletions(-)
 create mode 100755 t/t3003-ls-files-exclude.sh

diff --git a/builtin-ls-files.c b/builtin-ls-files.c
index 2c95ca6..c5c0407 100644
--- a/builtin-ls-files.c
+++ b/builtin-ls-files.c
@@ -170,10 +170,6 @@ static void show_files(struct dir_struct *dir, const char *prefix)
 	if (show_cached | show_stage) {
 		for (i = 0; i < active_nr; i++) {
 			struct cache_entry *ce = active_cache[i];
-			int dtype = ce_to_dtype(ce);
-			if (excluded(dir, ce->name, &dtype) !=
-					!!(dir->flags & DIR_SHOW_IGNORED))
-				continue;
 			if (show_unmerged && !ce_stage(ce))
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
@@ -186,10 +182,6 @@ static void show_files(struct dir_struct *dir, const char *prefix)
 			struct cache_entry *ce = active_cache[i];
 			struct stat st;
 			int err;
-			int dtype = ce_to_dtype(ce);
-			if (excluded(dir, ce->name, &dtype) !=
-					!!(dir->flags & DIR_SHOW_IGNORED))
-				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
 			err = lstat(ce->name, &st);
diff --git a/t/t3003-ls-files-exclude.sh b/t/t3003-ls-files-exclude.sh
new file mode 100755
index 0000000..fc1e379
--- /dev/null
+++ b/t/t3003-ls-files-exclude.sh
@@ -0,0 +1,32 @@
+#!/bin/sh
+
+test_description='ls-files --exclude does not affect index files'
+. ./test-lib.sh
+
+test_expect_success 'create repo with file' '
+	echo content >file &&
+	git add file &&
+	git commit -m file &&
+	echo modification >file
+'
+
+check_output() {
+test_expect_success "ls-files output contains file ($1)" "
+	echo '$2' >expect &&
+	git ls-files --exclude-standard --$1 >output &&
+	test_cmp expect output
+"
+}
+
+check_all_output() {
+	check_output 'cached' 'file'
+	check_output 'modified' 'file'
+}
+
+check_all_output
+test_expect_success 'add file to gitignore' '
+	echo file >.gitignore
+'
+check_all_output
+
+test_done
-- 
1.6.5.rc3.240.g77692

Re: [PATCH] git-add--interactive: never skip files included in index

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Subject: [PATCH] ls-files: excludes should not impact tracked files
>
> In all parts of git, .gitignore and other exclude files
> impact only how we treat untracked files; they should have
> no effect on files listed in the index.
>
> This behavior was originally implemented very early on in
> 9ff768e, but only for --exclude-from. Later, commit 63d285c
> accidentally caused us to trigger the behavior for
> --exclude-per-directory.
>
> This patch totally ignores excludes for files found in the
> index. This means we are reversing the original intent of
> 9ff768e, while at the same time fixing the accidental
> behavior of 63d285c. This is a good thing, though, as the
> way that 9ff768e behaved does not really make sense with the
> way exclusions are used in modern git.
>
> Signed-off-by: Jeff King <peff@peff.net>
> ---

Makes sense; thanks.

> I also still think that Pauli's patch makes sense; there is no point in
> passing --exclude-standard there. It should be a no-op.

Yeah, that is also queued independent from this one.