[PATCH] pathspec: prevent empty strings as pathspecs

[PATCH] pathspec: prevent empty strings as pathspecs

[Emily Xie]

For any command that takes a pathspec, passing an empty string will
execute the command on all files in the current directory. This
results in unexpected behavior. For example, git add "" adds all
files to staging, while git rm -rf "" recursively removes all files
from the working tree and index. This patch prevents such cases by
throwing an error message whenever an empty string is detected as a
pathspec.

Signed-off-by: Emily Xie <emilyxxie@gmail.com>
Reported-by: David Turner <novalis@novalis.org>
Mentored-by: Michail Denchev <mdenchev@gmail.com>
Thanks-to: Sarah Sharp <sarah@thesharps.us> and James Sharp <jamey@minilop.net>
---
 pathspec.c     | 6 +++++-
 t/t3600-rm.sh  | 4 ++++
 t/t3700-add.sh | 4 ++++
 3 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/pathspec.c b/pathspec.c
index c9e9b6c..11901a2 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -402,8 +402,12 @@ void parse_pathspec(struct pathspec *pathspec,
 	}
 
 	n = 0;
-	while (argv[n])
+	while (argv[n]) {
+		if (*argv[n] == '\0') {
+			die("Empty string is not a valid pathspec.");
+		}
 		n++;
+	}
 
 	pathspec->nr = n;
 	ALLOC_ARRAY(pathspec->items, n);
diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
index d046d98..1d7dc79 100755
--- a/t/t3600-rm.sh
+++ b/t/t3600-rm.sh
@@ -791,6 +791,10 @@ test_expect_success 'setup for testing rm messages' '
 	git add bar.txt foo.txt
 '
 
+test_expect_success 'rm files with empty string pathspec' '
+  test_must_fail git rm ""
+'
+
 test_expect_success 'rm files with different staged content' '
 	cat >expect <<-\EOF &&
 	error: the following files have staged content different from both the
diff --git a/t/t3700-add.sh b/t/t3700-add.sh
index f14a665..49cb080 100755
--- a/t/t3700-add.sh
+++ b/t/t3700-add.sh
@@ -207,6 +207,10 @@ test_expect_success POSIXPERM,SANITY 'git add should fail atomically upon an unr
 	! ( git ls-files foo1 | grep foo1 )
 '
 
+test_expect_success 'git add empty string as pathspec must fail' '
+  test_must_fail git add ""
+'
+
 rm -f foo2
 
 test_expect_success POSIXPERM,SANITY 'git add --ignore-errors' '
-- 
2.8.4

Re: [PATCH] pathspec: prevent empty strings as pathspecs

[David Turner]

On Sat, 2016-06-18 at 20:57 -0400, Emily Xie wrote:
> For any command that takes a pathspec, passing an empty string will
> execute the command on all files in the current directory. This
> results in unexpected behavior. For example, git add "" adds all
> files to staging, while git rm -rf "" recursively removes all files
> from the working tree and index. This patch prevents such cases by
> throwing an error message whenever an empty string is detected as a
> pathspec.
> 
> Signed-off-by: Emily Xie <emilyxxie@gmail.com>
> Reported-by: David Turner <novalis@novalis.org>
> Mentored-by: Michail Denchev <mdenchev@gmail.com>
> Thanks-to: Sarah Sharp <sarah@thesharps.us> and James Sharp <jamey@minilop.net>
> ---

Overall, lgtm.

The reason for this behavior is that, generally, traversals start at
some tree, and if there are no elements in the pathspec, the recursion
ends with that tree. Because this is a UX issue, fixing it at the
pathspec level seems correct to me.


>  pathspec.c     | 6 +++++-
>  t/t3600-rm.sh  | 4 ++++
>  t/t3700-add.sh | 4 ++++
>  3 files changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/pathspec.c b/pathspec.c
> index c9e9b6c..11901a2 100644
> --- a/pathspec.c
> +++ b/pathspec.c
> @@ -402,8 +402,12 @@ void parse_pathspec(struct pathspec *pathspec,
>  	}
>  
>  	n = 0;
> -	while (argv[n])
> +	while (argv[n]) {
> +		if (*argv[n] == '\0') {
> +			die("Empty string is not a valid pathspec.");
> +		}

nit: git style doesn't use {} on one-statement ifs.  

>  		n++;
> +	}
>  
>  	pathspec->nr = n;
>  	ALLOC_ARRAY(pathspec->items, n);
> diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
> index d046d98..1d7dc79 100755
> --- a/t/t3600-rm.sh
> +++ b/t/t3600-rm.sh
> @@ -791,6 +791,10 @@ test_expect_success 'setup for testing rm messages' '
>  	git add bar.txt foo.txt
>  '
>  
> +test_expect_success 'rm files with empty string pathspec' '
> +  test_must_fail git rm ""
> +'
> +

Maybe check the error message here?

Re: [PATCH] pathspec: prevent empty strings as pathspecs

[Eric Sunshine]

On Sat, Jun 18, 2016 at 9:49 PM, David Turner <novalis@novalis.org> wrote:
> On Sat, 2016-06-18 at 20:57 -0400, Emily Xie wrote:
>> +     while (argv[n]) {
>> +             if (*argv[n] == '\0') {
>> +                     die("Empty string is not a valid pathspec.");
>> +             }
>
> nit: git style doesn't use {} on one-statement ifs.

Also, drop the full-stop (period) from the error message and, to be
consistent with modern practice, don't capitalize the first word.
Finally, the string should be localizable, so wrap it in _(...):

    die(_("empty string not valid pathspec"));

Re: [PATCH] pathspec: prevent empty strings as pathspecs

[Junio C Hamano]

Emily Xie <emilyxxie@gmail.com> writes:

> For any command that takes a pathspec, passing an empty string will
> execute the command on all files in the current directory. This
> results in unexpected behavior. For example, git add "" adds all
> files to staging, while git rm -rf "" recursively removes all files
> from the working tree and index. This patch prevents such cases by
> throwing an error message whenever an empty string is detected as a
> pathspec.

This change is likely to break people's existing scripts.

For example, a script may want to inspect some state of the whole
working tree and then after the inspection was satisfactory do
something that affects paths that are inside the directory you were
originally in.  You would write something like this:

	#!/bin/sh
	original=$(git rev-parse --show-prefix)

        cd "$(git rev-parse --show-toplevel)"
        ... do the whole-tree pre-inspection
        ... using various git commands here

	# Then finally do the "action"
        git add "$original"

This works even if you happen to be at the top-level of the working
tree, but with your patch it suddenly breaks.  It may be trivial to
update the script to use "." when $original is empty, but that is
not an excuse.  The important thing is that they need to be told
about having to change their script before this change actually
happens.  Suddenly breaking other people without telling is simply
rude and unacceptable.

At least you would need two-step process to introduce a change like
this to warn the people whose tools and workflows you are breaking.
That is, (1) in one release, you add code to only detect the case
you will be changing the behaviour in a later version and give
warning messages, and (2) in another release that is several release
cycles later, stop warning and actually change the behaviour.

That is, if we assume that passing an empty string by mistake is a
problem in the real world that we want to save users from.  After
all, even though there is _no_ good reason why we _need_ to take
'git add ""' as the same as 'git add .', there is no need not to,
either.  When a user says 'git add sub/sub/' we would add everything
in that directory, 'git add sub/' would add even more, and it is
logically consistent for 'git add ""' to add everything.  This is a
border-line "doctor, if I pass an empty string, it is taken the same
way as a dot. -- don't do that then."

A kind of change that satisifies the following criteria:

 (1) I can tell that the author of the change really means well.

 (2) The problem the change is fixing does not seem to be a huge
     issue in real life.

 (3) The code with the patch would behave is not wrong per-se; it
     probably is how we would have designed the feature if we knew
     better.

 (4) However, we are not designing things from scratch without
     existing users, and the change potentially breaks uncountable
     number of existing users.

is something I really hate having to evaluate.  This is one of them.

So, I am of two minds.

I do not mind a two-step breaking of compatibility to address this
issue; I would also understand if the author thinks it is not worth
the hassle to do so.  The sudden behaviour change with this patch
alone is however not acceptable, I would think.

Re: [PATCH] pathspec: prevent empty strings as pathspecs

[David Turner]

On 06/18/2016 11:10 PM, Junio C Hamano wrote:
> Emily Xie <emilyxxie@gmail.com> writes:
>
>> For any command that takes a pathspec, passing an empty string will
>> execute the command on all files in the current directory. This
>> results in unexpected behavior. For example, git add "" adds all
>> files to staging, while git rm -rf "" recursively removes all files
>> from the working tree and index. This patch prevents such cases by
>> throwing an error message whenever an empty string is detected as a
>> pathspec.
>
> This change is likely to break people's existing scripts.
>
> For example, a script may want to inspect some state of the whole
> working tree and then after the inspection was satisfactory do
> something that affects paths that are inside the directory you were
> originally in.  You would write something like this:
>
> 	#!/bin/sh
> 	original=$(git rev-parse --show-prefix)
>
>          cd "$(git rev-parse --show-toplevel)"
>          ... do the whole-tree pre-inspection
>          ... using various git commands here
>
> 	# Then finally do the "action"
>          git add "$original"
>
> This works even if you happen to be at the top-level of the working
> tree, but with your patch it suddenly breaks.  It may be trivial to
> update the script to use "." when $original is empty, but that is
> not an excuse.  The important thing is that they need to be told
> about having to change their script before this change actually
> happens.  Suddenly breaking other people without telling is simply
> rude and unacceptable.

The help for rev-parse --show-prefix says:
"When the command is invoked from a subdirectory, show the path of
the current directory relative to the top-level directory."

This doesn't say what it does when invoked at the top-level.  Maybe we 
should just change it to output "./", which would make such scripts 
work?  It would not necessarily fix all possible scripts, but it would 
fix these.  It seem odd to insist on preserving this undocumented 
behavior of git add/rm.

> At least you would need two-step process to introduce a change like
> this to warn the people whose tools and workflows you are breaking.
> That is, (1) in one release, you add code to only detect the case
> you will be changing the behaviour in a later version and give
> warning messages, and (2) in another release that is several release
> cycles later, stop warning and actually change the behaviour.
>
> That is, if we assume that passing an empty string by mistake is a
> problem in the real world that we want to save users from.

I discovered this when one of our shell-scripts had a variable was 
accidentally empty.  The user I was supporting was very confused.  So I 
think it is a problem in the real world.

Re: [PATCH] pathspec: prevent empty strings as pathspecs

[Junio C Hamano]

David Turner <novalis@novalis.org> writes:

> The help for rev-parse --show-prefix says:
> "When the command is invoked from a subdirectory, show the path of
> the current directory relative to the top-level directory."
>
> This doesn't say what it does when invoked at the top-level.  Maybe we
> should just change it to output "./", which would make such scripts
> work?  It would not necessarily fix all possible scripts, but it would
> fix these.  It seem odd to insist on preserving this undocumented
> behavior of git add/rm.

For two obvious reasons, you do not want to go there.

As you seem to understand in the above, "--show-prefix" was given
merely as an example, so updating its behaviour would not change the
fundamental problem with the patch under discussion at all.  It will
break scripts that knew, documented or not, that pathspec match is a
prefix substring match that honors directory boundary and an empty
string is defined long time ago to match any path and it does not
matter if that definition was made by mistake.

Also, if you change "--show-prefix" output, documented or not, that
will break other scripts that knew that checking it against an empty
string is a valid way to check if you are already at the top level
of the working tree.  And I highly suspect that such a change is
much harder to advertise and warn existing scripts, compared to the
change proposed by the patch under discussion.  The output routine
in show-prefix knows what it is giving to the caller, but does not
know how the caller uses it and what for.  Compared to that, ...

>> At least you would need two-step process to introduce a change like
>> this to warn the people whose tools and workflows you are breaking.
>> That is, (1) in one release, you add code to only detect the case
>> you will be changing the behaviour in a later version and give
>> warning messages, and (2) in another release that is several release
>> cycles later, stop warning and actually change the behaviour.

... is a lot easier transition (again, assuming that this change is
worth doing in the first place).

Re: [PATCH] pathspec: prevent empty strings as pathspecs

[Emily Xie]

Thanks for the quick responses on this patch!

In response to Junio:

> "At least you would need two-step process to introduce a change like
> this to warn the people whose tools and workflows you are breaking.
> That is, (1) in one release, you add code to only detect the case
> you will be changing the behaviour in a later version and give
> warning messages, and (2) in another release that is several release
> cycles later, stop warning and actually change the behaviour....

> I do not mind a two-step breaking of compatibility to address this
> issue; I would also understand if the author thinks it is not worth
> the hassle to do so.  The sudden behaviour change with this patch
> alone is however not acceptable, I would think."

I understand your hesitance with the original method. If the agreed
solution is a two-step implementation, I think it would definitely be
worth my time and hassle----in part because I'm particularly excited
about the prospect of contributing to Git, but mostly because I do
believe that it would be a good improvement.

Given this, I'll edit the patch and re-submit to only emit warning
messages for now.

- Emily