[PATCH] grep: add known breakage of coloring when using extended patterns

[PATCH] grep: add known breakage of coloring when using extended patterns

[Bert Wesarg]

The coloring code does not respect the extended pattern format, expose
this with an known breakage test.

Signed-off-by: Bert Wesarg <bert.wesarg@googlemail.com>
---
 t/t7812-grep-color.sh |   25 +++++++++++++++++++++++++
 1 files changed, 25 insertions(+), 0 deletions(-)
 create mode 100755 t/t7812-grep-color.sh

diff --git a/t/t7812-grep-color.sh b/t/t7812-grep-color.sh
new file mode 100755
index 0000000..2cffff3
--- /dev/null
+++ b/t/t7812-grep-color.sh
@@ -0,0 +1,25 @@
+#!/bin/sh
+
+test_description='git grep --color
+'
+
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	test_commit initial input "foo bar baz
+bar baz
+"
+'
+
+cat >expected <<EOF
+input<CYAN>:<RESET><BOLD;RED>foo<RESET> bar baz
+EOF
+
+
+test_expect_failure 'grep coloring does not honor boolean algebra' '
+	git grep --color -e foo --or \( -e bar --and --not -e baz \) |
+		test_decode_color >actual
+    test_cmp expected actual
+'
+
+test_done
-- 
1.7.5.349.gfeb1a

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

[Johannes Sixt]

Am 5/2/2011 13:35, schrieb Bert Wesarg:
> +test_expect_failure 'grep coloring does not honor boolean algebra' '

I'm sure you meant to say 'grep coloring honors boolean algebra', no?

-- Hannes

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

[Bert Wesarg]

On Mon, May 2, 2011 at 13:41, Johannes Sixt <j.sixt@viscovery.net> wrote:
> Am 5/2/2011 13:35, schrieb Bert Wesarg:
>> +test_expect_failure 'grep coloring does not honor boolean algebra' '
>
> I'm sure you meant to say 'grep coloring honors boolean algebra', no?

You're right, I'm not so good in double-negations.

Bert

>
> -- Hannes
>

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

[René Scharfe]

Am 02.05.2011 13:35, schrieb Bert Wesarg:
> The coloring code does not respect the extended pattern format, expose
> this with an known breakage test.
>
> Signed-off-by: Bert Wesarg<bert.wesarg@googlemail.com>
> ---
>   t/t7812-grep-color.sh |   25 +++++++++++++++++++++++++
>   1 files changed, 25 insertions(+), 0 deletions(-)
>   create mode 100755 t/t7812-grep-color.sh
>
> diff --git a/t/t7812-grep-color.sh b/t/t7812-grep-color.sh
> new file mode 100755
> index 0000000..2cffff3
> --- /dev/null
> +++ b/t/t7812-grep-color.sh
> @@ -0,0 +1,25 @@
> +#!/bin/sh
> +
> +test_description='git grep --color
> +'
> +
> +. ./test-lib.sh
> +
> +test_expect_success 'setup' '
> +	test_commit initial input "foo bar baz
> +bar baz
> +"
> +'
> +
> +cat>expected<<EOF
> +input<CYAN>:<RESET><BOLD;RED>foo<RESET>  bar baz
> +EOF
> +
> +
> +test_expect_failure 'grep coloring does not honor boolean algebra' '
> +	git grep --color -e foo --or \( -e bar --and --not -e baz \) |
> +		test_decode_color>actual
> +    test_cmp expected actual
> +'
> +
> +test_done

The current code highlights the given search terms ("atoms").  You 
expect the matching part of a line to be highlighted instead.  Another 
example with differing results between the two ways is this:

	$ git grep -v -e foo --and -e bar

The first one was easy to implement and useful already.  I wonder how to 
implement the second one (I couldn't think of an easy way back then) and 
also if we really prefer it to the current behaviour in the first place.

The twisted logic that convinced me to forget about the issue goes like 
this: Even by giving negative search terms, the user is expressing a 
certain value of interest in them; highlighting these atoms even in 
lines which were not matched by them is a courtesy.

René

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

[Junio C Hamano]

René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:

>> +	test_commit initial input "foo bar baz
...
>> +	git grep --color -e foo --or \( -e bar --and --not -e baz \) |
...
> The current code highlights the given search terms ("atoms").

Hmm, I was probably not paying attention to "coloring the parts that
matched" topic at all, but wouldn't it be easier and more efficient to
paint only "foo" without painting "bar baz"?  We know the first term "foo"
matches, and the rest \(...\) that is --or'ed does not have to even be
evaluated, no?

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

[René Scharfe]

Am 02.05.2011 19:37, schrieb Junio C Hamano:
> René Scharfe<rene.scharfe@lsrfire.ath.cx>  writes:
>
>>> +	test_commit initial input "foo bar baz
> ...
>>> +	git grep --color -e foo --or \( -e bar --and --not -e baz \) |
> ...
>> The current code highlights the given search terms ("atoms").
>
> Hmm, I was probably not paying attention to "coloring the parts that
> matched" topic at all, but wouldn't it be easier and more efficient to
> paint only "foo" without painting "bar baz"?  We know the first term "foo"
> matches, and the rest \(...\) that is --or'ed does not have to even be
> evaluated, no?

Aggregating the set of matching characters and passing them back during 
expression evaluation can be more efficient, yes, as it would avoid 
calling regexec() on the printed lines again only to find out what to 
color.  But I wouldn't call it easy.  E.g. how to return the set of 
matching characters in the following case?

	$ git grep --color -e foo --and -e bar

I can only think of using lists or, even uglier, perhaps a bitmap to 
remember which characters to highlight.

Also GNU grep doesn't only color the first sufficient match.  E.g. this 
will color both b and d:

	$ echo abcde | grep --color -e b -e d

And I think this makes sense and meets my expectations.

But the other mode, which only colors the exact characters that were 
used to determine that the shown line matches (or doesn't match, if -v 
is given) may be interesting as well.  I'm curious to see an 
implementation and wonder if the results are really better than the ones 
of the current (cheating) approach. :)

René

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

[Junio C Hamano]

René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:

> Aggregating the set of matching characters and passing them back
> during expression evaluation can be more efficient, yes, as it would
> avoid calling regexec() on the printed lines again only to find out
> what to color.  But I wouldn't call it easy.  E.g. how to return the
> set of matching characters in the following case?
>
> 	$ git grep --color -e foo --and -e bar

I would naïvely expect each of the "grep_expr" in the parsed grep tree to
have not just a one-bit "hit", but rm_so/rm_eo pair, but as I said, I was
asleep while "coloring the parts that matched" discussion was in progress
;-)

But

> Also GNU grep doesn't only color the first sufficient match.
> E.g. this will color both b and d:
>
> 	$ echo abcde | grep --color -e b -e d

that is probably a useful thing to have.  Obviously, my suggestion should
optimize away the match with 'd' to begin with, so keeping rm_so/rm_eo
pair would not help painting d at all.  Worse yet, I would expect that

	$ echo abcdeabcde | grep --color -e b -e d

would need to paint all occurrences for such a purpose, so obviously you
would need an extra pass to find all potential hits.  Ugly and makes me
personally wonder if it is really worth it, but the user is explicitly
asking for --color and is prepared to pay the price, so it probably is Ok.

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

[René Scharfe]

Am 03.05.2011 00:04, schrieb Junio C Hamano:
> René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:
> 
>> Aggregating the set of matching characters and passing them back
>> during expression evaluation can be more efficient, yes, as it would
>> avoid calling regexec() on the printed lines again only to find out
>> what to color.  But I wouldn't call it easy.  E.g. how to return the
>> set of matching characters in the following case?
>>
>> 	$ git grep --color -e foo --and -e bar
> 
> I would naïvely expect each of the "grep_expr" in the parsed grep tree to
> have not just a one-bit "hit", but rm_so/rm_eo pair, but as I said, I was
> asleep while "coloring the parts that matched" discussion was in progress
> ;-)

That's true, but each one represents a single contiguous run of
characters to color.  But I meant to say that returning non-contiguous
ranges, as you can get in the --and example above, is not that easy.  To me.

Hmm, if one just wants to color the parts that triggered a match it gets
a lot easier.  Each pattern just needs to remember the last matching
offsets and then show_line can walk the expression tree to find out what
needs coloring.  Not sure about the details, though. :)

> But
> 
>> Also GNU grep doesn't only color the first sufficient match.
>> E.g. this will color both b and d:
>>
>> 	$ echo abcde | grep --color -e b -e d
> 
> that is probably a useful thing to have.  Obviously, my suggestion should
> optimize away the match with 'd' to begin with, so keeping rm_so/rm_eo
> pair would not help painting d at all.  Worse yet, I would expect that
> 
> 	$ echo abcdeabcde | grep --color -e b -e d
> 
> would need to paint all occurrences for such a purpose, so obviously you
> would need an extra pass to find all potential hits.  Ugly and makes me
> personally wonder if it is really worth it, but the user is explicitly
> asking for --color and is prepared to pay the price, so it probably is Ok.

The performance impact is low as long as there are lots more lines that
don't match than matching ones.  Which should be a given for most uses
of grep.

I always miss hit coloring if I have to use an environment that doesn't
provide , similarly to syntax highlighting, so I do think it's worth it,
for me at least.

René