[PATCH] Make !pattern in .gitattributes non-fatal

[PATCH] Make !pattern in .gitattributes non-fatal

[Thomas Rast]

Before 82dce99 (attr: more matching optimizations from .gitignore,
2012-10-15), .gitattributes did not have any special treatment of a
leading '!'.  The docs, however, always said

  The rules how the pattern matches paths are the same as in
  `.gitignore` files; see linkgit:gitignore[5].

By those rules, leading '!' means pattern negation.  So 82dce99
correctly determined that this kind of line makes no sense and should
be disallowed.

However, users who actually had a rule for files starting with a '!'
are in a bad position: before 82dce99 '!' matched that literal
character, so it is conceivable that users have .gitattributes with
such lines in them.  After 82dce99 the unescaped version was
disallowed in such a way that git outright refuses to run(!) most
commands in the presence of such a .gitattributes.  It therefore
becomes very hard to fix, let alone work with, such repositories.

Let's at least allow the users to fix their repos: change the fatal
error into a warning.

Reported-by: mathstuf@gmail.com
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
---
 attr.c                | 8 +++++---
 t/t0003-attributes.sh | 3 ++-
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/attr.c b/attr.c
index 4657cc2..e2f9377 100644
--- a/attr.c
+++ b/attr.c
@@ -255,9 +255,11 @@ static struct match_attr *parse_attr_line(const char *line, const char *src,
 				      &res->u.pat.patternlen,
 				      &res->u.pat.flags,
 				      &res->u.pat.nowildcardlen);
-		if (res->u.pat.flags & EXC_FLAG_NEGATIVE)
-			die(_("Negative patterns are forbidden in git attributes\n"
-			      "Use '\\!' for literal leading exclamation."));
+		if (res->u.pat.flags & EXC_FLAG_NEGATIVE) {
+			warning(_("Negative patterns are ignored in git attributes\n"
+				  "Use '\\!' for literal leading exclamation."));
+			return NULL;
+		}
 	}
 	res->is_macro = is_macro;
 	res->num_attr = num_attr;
diff --git a/t/t0003-attributes.sh b/t/t0003-attributes.sh
index 43b2513..0b98b6f 100755
--- a/t/t0003-attributes.sh
+++ b/t/t0003-attributes.sh
@@ -198,7 +198,8 @@ test_expect_success 'root subdir attribute test' '
 
 test_expect_success 'negative patterns' '
 	echo "!f test=bar" >.gitattributes &&
-	test_must_fail git check-attr test -- f
+	git check-attr test -- '"'"'!f'"'"' 2>errors &&
+	test_i18ngrep "Negative patterns are ignored" errors
 '
 
 test_expect_success 'patterns starting with exclamation' '
-- 
1.8.2.rc1.392.gf57d6b8.dirty

Re: [PATCH] Make !pattern in .gitattributes non-fatal

[Junio C Hamano]

Makes sense.  Duy?

Re: [PATCH] Make !pattern in .gitattributes non-fatal

[Junio C Hamano]

Thomas Rast <trast@student.ethz.ch> writes:

> Before 82dce99 (attr: more matching optimizations from .gitignore,
> 2012-10-15), .gitattributes did not have any special treatment of a
> leading '!'.  The docs, however, always said
>
>   The rules how the pattern matches paths are the same as in
>   `.gitignore` files; see linkgit:gitignore[5].
>
> By those rules, leading '!' means pattern negation.  So 82dce99
> correctly determined that this kind of line makes no sense and should
> be disallowed.
>
> However, users who actually had a rule for files starting with a '!'
> are in a bad position: before 82dce99 '!' matched that literal
> character, so it is conceivable that users have .gitattributes with
> such lines in them.  After 82dce99 the unescaped version was
> disallowed in such a way that git outright refuses to run(!) most
> commands in the presence of such a .gitattributes.  It therefore
> becomes very hard to fix, let alone work with, such repositories.

Fixing the working tree is easy, but when we read from a history
that already records such an entry in an attribute file, it would
become somewhat cumbersome.  I wouldn't use "very hard to fix" to
describe such a case.

But the demotion to warning does make sense; let's do that in
v1.8.1.5.

Re: [PATCH] Make !pattern in .gitattributes non-fatal

[Thomas Rast]

Junio C Hamano <gitster@pobox.com> writes:

> Thomas Rast <trast@student.ethz.ch> writes:
>
>> Before 82dce99 (attr: more matching optimizations from .gitignore,
>> 2012-10-15), .gitattributes did not have any special treatment of a
>> leading '!'.  The docs, however, always said
>>
>>   The rules how the pattern matches paths are the same as in
>>   `.gitignore` files; see linkgit:gitignore[5].
>>
>> By those rules, leading '!' means pattern negation.  So 82dce99
>> correctly determined that this kind of line makes no sense and should
>> be disallowed.
>>
>> However, users who actually had a rule for files starting with a '!'
>> are in a bad position: before 82dce99 '!' matched that literal
>> character, so it is conceivable that users have .gitattributes with
>> such lines in them.  After 82dce99 the unescaped version was
>> disallowed in such a way that git outright refuses to run(!) most
>> commands in the presence of such a .gitattributes.  It therefore
>> becomes very hard to fix, let alone work with, such repositories.
>
> Fixing the working tree is easy, but when we read from a history
> that already records such an entry in an attribute file, it would
> become somewhat cumbersome.  I wouldn't use "very hard to fix" to
> describe such a case.

Well, I'm sorry if I hurt any feelings there, but...

  ~/tmp/badattr(master)$ git show bad:.gitattributes
  !bad text
  ~/tmp/badattr(master)$ ~/g/git-checkout bad  # a git without my patch
  fatal: Negative patterns are forbidden in git attributes
  Use '\!' for literal leading exclamation.
  ~/tmp/badattr(master)$ git status
  # On branch master
  nothing to commit (use -u to show untracked files)

Notice how it remains on master.  I suppose with enough knowledge of the
internals I could manage, but after seeing how hard it was to *build*
such broken history with a git that dies, I don't really want to try.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Re: [PATCH] Make !pattern in .gitattributes non-fatal

[Junio C Hamano]

Thomas Rast <trast@student.ethz.ch> writes:

> Junio C Hamano <gitster@pobox.com> writes:
>
>> Thomas Rast <trast@student.ethz.ch> writes:
>>
>>> Before 82dce99 (attr: more matching optimizations from .gitignore,
>>> 2012-10-15), .gitattributes did not have any special treatment of a
>>> leading '!'.  The docs, however, always said
>>>
>>>   The rules how the pattern matches paths are the same as in
>>>   `.gitignore` files; see linkgit:gitignore[5].
>>>
>>> By those rules, leading '!' means pattern negation.  So 82dce99
>>> correctly determined that this kind of line makes no sense and should
>>> be disallowed.
>>>
>>> However, users who actually had a rule for files starting with a '!'
>>> are in a bad position: before 82dce99 '!' matched that literal
>>> character, so it is conceivable that users have .gitattributes with
>>> such lines in them.  After 82dce99 the unescaped version was
>>> disallowed in such a way that git outright refuses to run(!) most
>>> commands in the presence of such a .gitattributes.  It therefore
>>> becomes very hard to fix, let alone work with, such repositories.
>>
>> Fixing the working tree is easy, but when we read from a history
>> that already records such an entry in an attribute file, it would
>> become somewhat cumbersome.  I wouldn't use "very hard to fix" to
>> describe such a case.
>
> Well, I'm sorry if I hurt any feelings there, but...
>
>   ~/tmp/badattr(master)$ git show bad:.gitattributes
>   !bad text
>   ~/tmp/badattr(master)$ ~/g/git-checkout bad  # a git without my patch
>   fatal: Negative patterns are forbidden in git attributes
>   Use '\!' for literal leading exclamation.
>   ~/tmp/badattr(master)$ git status
>   # On branch master
>   nothing to commit (use -u to show untracked files)
>
> Notice how it remains on master.  I suppose with enough knowledge of the
> internals I could manage,...

That would have been nicer to have in the log message.

In any case, 1.8.1.5 will go out with this fix (this is v1.8.1
regression IIUC) this afternoon.  Thanks for a fix.

Re: [PATCH] Make !pattern in .gitattributes non-fatal

[Duy Nguyen]

On Sat, Mar 2, 2013 at 3:06 AM, Thomas Rast <trast@student.ethz.ch> wrote:
> Before 82dce99 (attr: more matching optimizations from .gitignore,
> 2012-10-15), .gitattributes did not have any special treatment of a
> leading '!'.  The docs, however, always said
>
>   The rules how the pattern matches paths are the same as in
>   `.gitignore` files; see linkgit:gitignore[5].
>
> By those rules, leading '!' means pattern negation.  So 82dce99
> correctly determined that this kind of line makes no sense and should
> be disallowed.
>
> However, users who actually had a rule for files starting with a '!'
> are in a bad position: before 82dce99 '!' matched that literal
> character, so it is conceivable that users have .gitattributes with
> such lines in them.  After 82dce99 the unescaped version was
> disallowed in such a way that git outright refuses to run(!) most
> commands in the presence of such a .gitattributes.  It therefore
> becomes very hard to fix, let alone work with, such repositories.
>
> Let's at least allow the users to fix their repos: change the fatal
> error into a warning.

I agree we should not break existing .gitattributes. So yes, die()ing
here sounds bad. But..

> @@ -255,9 +255,11 @@ static struct match_attr *parse_attr_line(const char *line, const char *src,
>                                       &res->u.pat.patternlen,
>                                       &res->u.pat.flags,
>                                       &res->u.pat.nowildcardlen);
> -               if (res->u.pat.flags & EXC_FLAG_NEGATIVE)
> -                       die(_("Negative patterns are forbidden in git attributes\n"
> -                             "Use '\\!' for literal leading exclamation."));
> +               if (res->u.pat.flags & EXC_FLAG_NEGATIVE) {
> +                       warning(_("Negative patterns are ignored in git attributes\n"
> +                                 "Use '\\!' for literal leading exclamation."));
> +                       return NULL;
> +               }

This "return NULL;" means we ignore "!blah" pattern, which is a
regression, isn't it? Should we treat '!' as literal here?
-- 
Duy

Re: [PATCH] Make !pattern in .gitattributes non-fatal

[Junio C Hamano]

Duy Nguyen <pclouds@gmail.com> writes:

> This "return NULL;" means we ignore "!blah" pattern, which is a
> regression, isn't it? Should we treat '!' as literal here?

Probably not.  Can you point to a project everybody has heard of
that keeps track of a path that begins with an exclamation point?

With clarification to the documentation that says "you cannot use !
to negate" and your "die on such an entry", we have been going in
the direction that forbids the use of such an entry, and making it
mean literal exclamation point is going sideways in the middle of
the road.

Besides, if you want to match a path that begins with an exclam, you
can always say "[!]", no?

[PATCH] attr: always treat the leading exclamation mark as literal

[Nguyễn Thái Ngọc Duy]

8b1bd02 (Make !pattern in .gitattributes non-fatal - 2013-03-01)
raises the fact that even though '!' should have been a negative
pattern indicator, people have been misusing for plain '!' and old Git
versions accept that. This makes the leading '!' a de facto
literal. We can't ever make it negative indicator again because we
can't distinguish the literal '!' in existing repos and the negative
'!'  in future repos.

Acknowledge it in document and code that the leading '!' is forever
literal.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Perhaps this on top of Thomas' patch? I see no way else that we can
 change the meaning of '!' again if such patterns are already stored
 in repositories.

 Documentation/gitattributes.txt | 3 ++-
 attr.c                          | 8 ++------
 dir.c                           | 7 ++++---
 dir.h                           | 6 +++++-
 t/t0003-attributes.sh           | 5 ++---
 5 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
index 2698f63..02bff9d 100644
--- a/Documentation/gitattributes.txt
+++ b/Documentation/gitattributes.txt
@@ -56,7 +56,8 @@ When more than one pattern matches the path, a later line
 overrides an earlier line.  This overriding is done per
 attribute.  The rules how the pattern matches paths are the
 same as in `.gitignore` files; see linkgit:gitignore[5].
-Unlike `.gitignore`, negative patterns are forbidden.
+Unlike `.gitignore`, the leading `!` is treated as a literal
+exclamation, not an indicator of negative patterns.
 
 When deciding what attributes are assigned to a path, git
 consults `$GIT_DIR/info/attributes` file (which has the highest
diff --git a/attr.c b/attr.c
index d181d98..838ec21 100644
--- a/attr.c
+++ b/attr.c
@@ -254,12 +254,8 @@ static struct match_attr *parse_attr_line(const char *line, const char *src,
 		parse_exclude_pattern(&res->u.pat.pattern,
 				      &res->u.pat.patternlen,
 				      &res->u.pat.flags,
-				      &res->u.pat.nowildcardlen);
-		if (res->u.pat.flags & EXC_FLAG_NEGATIVE) {
-			warning(_("Negative patterns are ignored in git attributes\n"
-				  "Use '\\!' for literal leading exclamation."));
-			return NULL;
-		}
+				      &res->u.pat.nowildcardlen,
+				      1);
 	}
 	res->is_macro = is_macro;
 	res->num_attr = num_attr;
diff --git a/dir.c b/dir.c
index a473ca2..609ab99 100644
--- a/dir.c
+++ b/dir.c
@@ -311,13 +311,14 @@ static int no_wildcard(const char *string)
 void parse_exclude_pattern(const char **pattern,
 			   int *patternlen,
 			   int *flags,
-			   int *nowildcardlen)
+			   int *nowildcardlen,
+			   int gitattributes)
 {
 	const char *p = *pattern;
 	size_t i, len;
 
 	*flags = 0;
-	if (*p == '!') {
+	if (!gitattributes && *p == '!') {
 		*flags |= EXC_FLAG_NEGATIVE;
 		p++;
 	}
@@ -354,7 +355,7 @@ void add_exclude(const char *string, const char *base,
 	int flags;
 	int nowildcardlen;
 
-	parse_exclude_pattern(&string, &patternlen, &flags, &nowildcardlen);
+	parse_exclude_pattern(&string, &patternlen, &flags, &nowildcardlen, 0);
 	if (flags & EXC_FLAG_MUSTBEDIR) {
 		char *s;
 		x = xmalloc(sizeof(*x) + patternlen + 1);
diff --git a/dir.h b/dir.h
index f5c89e3..3aeb519 100644
--- a/dir.h
+++ b/dir.h
@@ -107,7 +107,11 @@ extern int path_excluded(struct path_exclude_check *, const char *, int namelen,
 extern int add_excludes_from_file_to_list(const char *fname, const char *base, int baselen,
 					  char **buf_p, struct exclude_list *which, int check_index);
 extern void add_excludes_from_file(struct dir_struct *, const char *fname);
-extern void parse_exclude_pattern(const char **string, int *patternlen, int *flags, int *nowildcardlen);
+extern void parse_exclude_pattern(const char **string,
+				  int *patternlen,
+				  int *flags,
+				  int *nowildcardlen,
+				  int gitattributes);
 extern void add_exclude(const char *string, const char *base,
 			int baselen, struct exclude_list *which);
 extern void free_excludes(struct exclude_list *el);
diff --git a/t/t0003-attributes.sh b/t/t0003-attributes.sh
index 1035a14..7ca283f 100755
--- a/t/t0003-attributes.sh
+++ b/t/t0003-attributes.sh
@@ -197,9 +197,8 @@ test_expect_success 'root subdir attribute test' '
 '
 
 test_expect_success 'negative patterns' '
-	echo "!f test=bar" >.gitattributes &&
-	git check-attr test -- '"'"'!f'"'"' 2>errors &&
-	test_i18ngrep "Negative patterns are ignored" errors
+	echo "!f test=foo" >.gitattributes &&
+	attr_check "!f" foo
 '
 
 test_expect_success 'patterns starting with exclamation' '
-- 
1.8.1.2.536.gf441e6d