Add "git grep" helper

Add "git grep" helper

[Linus Torvalds]

Very convenient shorthand for

	git-ls-files [file-patterns] | xargs grep <pattern>

which I tend to do all the time.

Yes, it's trivial, but it's really nice. I can do

	git grep '\<some_variable\>' arch/i386 include/asm-i386

and it does exactly what you'd think it does. And since it just uses the 
normal git-ls-files file patterns, you can do things like

	git grep something 'include/*.h'

and it will search all header files under the include/ subdirectory.

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
---

diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -76,7 +76,7 @@ SCRIPT_SH = \
 	git-tag.sh git-verify-tag.sh git-whatchanged.sh git.sh \
 	git-applymbox.sh git-applypatch.sh \
 	git-merge.sh git-merge-stupid.sh git-merge-octopus.sh \
-	git-merge-resolve.sh
+	git-merge-resolve.sh git-grep.sh
 
 SCRIPT_PERL = \
 	git-archimport.perl git-cvsimport.perl git-relink.perl \
diff --git a/git-grep.sh b/git-grep.sh
new file mode 100755
--- /dev/null
+++ b/git-grep.sh
@@ -0,0 +1,20 @@
+#!/bin/sh
+flags=
+while :; do
+  pattern="$1"
+  case "$pattern" in
+  -i|-I|-a|-E|-H|-h|-l)
+    flags="$flags $pattern"
+    shift
+    ;;
+  -*)
+    echo "unknown flag $pattern" >&2
+    exit 1
+    ;;
+  *)
+    break
+    ;;
+  esac
+done
+shift
+git-ls-files -z "$@" | xargs -0 grep $flags "$pattern"

Re: Add "git grep" helper

[Morten Welinder]

Cute, but what about grep flags like "-w", "-n", and "-c"?

M.

Re: Add "git grep" helper

[Linus Torvalds]

On Mon, 12 Sep 2005, Morten Welinder wrote:
>
> Cute, but what about grep flags like "-w", "-n", and "-c"?

Hey, add them to the flags list. Or just pass in all flags to grep, I
dunno (I was kind of expecting some flags to go to git-ls-files, but that
may or may not make any sense, and my script obviously didn't end up doing
that).

		Linus

Re: Add "git grep" helper

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> Very convenient shorthand for
>
> 	git-ls-files [file-patterns] | xargs grep <pattern>
>
> which I tend to do all the time.

Great minds think alike.  I stopped using 'grep-find' in Emacs
and use "git-ls-fiels -z | xargs -0 grep" instead these days.
Thanks for the patch; applied.

I do not much care about other grep flags but I think you forgot
to special case '-e', so what is in the "master" branch has one
extra commit on top of it for that (it does not seem to have
percolated down yet).

Re: Add "git grep" helper

[Linus Torvalds]

On Mon, 12 Sep 2005, Junio C Hamano wrote:
> 
> I do not much care about other grep flags but I think you forgot
> to special case '-e', so what is in the "master" branch has one
> extra commit on top of it for that (it does not seem to have
> percolated down yet).

Actually, the more I think about it, the more I think Morten was right, 
and the git-grep case statement should just put all flags in "$flags", and 
any git-ls-files flags can be handled specially before.

We also need special casing for grep flags that take an argument.  So the
end result might be something like the following..

  ...
  case "$pattern" in

    # git-ls-file specific flags
    --others|--exclude=*|--exclude-from=*|--exclude-per-directory=*)
	git_flags="$git_flags $pattern"
	shift
	;;

    # grep flags with an argument
    -B|-C|-m)
	flags="$flags $pattern $2"
	shift
	shift
	;;

    # grep 'pattern' argument
    -e)
	pattern="$2"
	shift
	break
	;;

    # We assume everything else is a regular grep pattern
    -*)
	flags="$flags $pattern"
	shift
	;;
  ...

instead. And it's entirely possible that we'd never want to even bother 
with things like --others and --exclude* at all.

		Linus

Improve "git grep" flags handling

[Linus Torvalds]

This allows any arbitrary flags to "grep", and knows about the few special 
grep flags that take an argument too.

It also allows some flags for git-ls-files, although their usefulness is 
questionable.

With this, something line

	git grep -w -1 pattern

works, without the script enumerating every possible flag.

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
---

On Mon, 12 Sep 2005, Linus Torvalds wrote:
> 
> We also need special casing for grep flags that take an argument.  So the
> end result might be something like the following..

diff --git a/git-grep.sh b/git-grep.sh
--- a/git-grep.sh
+++ b/git-grep.sh
@@ -1,25 +1,38 @@
 #!/bin/sh
 flags=
+git_flags=
 while :; do
   pattern="$1"
   case "$pattern" in
-  -i|-I|-a|-E|-H|-h|-l)
-    flags="$flags $pattern"
-    shift
-    ;;
+  # git-ls-file specific flags
+  --others|--exclude=*|--exclude-from=*|--exclude-per-directory=*)
+      git_flags="$git_flags $pattern"
+      shift
+      ;;
+
+  # grep flags with an argument
+  -B|-C|-m)
+      flags="$flags $pattern $2"
+      shift
+      shift
+      ;;
+
+  # grep 'pattern' argument
   -e)
-    pattern="$2"
-    shift
-    break
-    ;;
+      pattern="$2"
+      shift
+      break
+      ;;
+
+  # We assume everything else is a regular grep pattern
   -*)
-    echo "unknown flag $pattern" >&2
-    exit 1
-    ;;
+      flags="$flags $pattern"
+      shift
+      ;;
   *)
     break
     ;;
   esac
 done
 shift
-git-ls-files -z "$@" | xargs -0 grep $flags -e "$pattern"
+git-ls-files -z $git_flags "$@" | xargs -0 grep $flags -e "$pattern"

Re: Add "git grep" helper

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> Actually, the more I think about it, the more I think Morten was right, 
> and the git-grep case statement should just put all flags in "$flags", and 
> any git-ls-files flags can be handled specially before.
>
> We also need special casing for grep flags that take an argument.  So the
> end result might be something like the following..

Arrrrrrrrrrrrrrrrrrrgh.

Now, whose grep are we going to support?  Should we teach every
flag GNU grep supports that can take a parameter to the script?

Can we cheat and do something like this instead?

    git grep -B 6 -A 2 frotz -- arch/i386
    git grep -B 6 -A 2 frotz --others arch/i386

That is, things up to a known git-ls-files flag  or -- are
passed to grep and the rest is given to git-ls-files.

Re: Improve "git grep" flags handling

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> +  # git-ls-file specific flags
> +  --others|--exclude=*|--exclude-from=*|--exclude-per-directory=*)
> +      git_flags="$git_flags $pattern"
> +      shift
> +      ;;

This would not fly well, I am afraid.  --exclude=* are usually
shell globs which would be expanded before you use them
unquoted.

Re: Improve "git grep" flags handling

[Linus Torvalds]

On Mon, 12 Sep 2005, Junio C Hamano wrote:
> 
> This would not fly well, I am afraid.  --exclude=* are usually
> shell globs which would be expanded before you use them
> unquoted.

Good point. However, since it's unlikely that anybody will use it, maybe 
it's not important - more of a "this is how you could do it if you fixed 
the globbing problem" ;)

It's easy enough to fix if you use bash array variables, but I thought 
adding that was not supposed to be done? Or have people resigned 
themselves to bash extensions ;)

		Linus

Re: Improve "git grep" flags handling

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> Good point. However, since it's unlikely that anybody will use it, maybe 
> it's not important - more of a "this is how you could do it if you fixed 
> the globbing problem" ;)

Yeah, let's see how tolerant your barf-o-meter is today.  This
would make even *me* barf ;-), but it sometimes may be useful to
be able to say:

    git grep --others --exclude='*.[oa]' '@@GIT_'

------------
#!/bin/sh

# git grep <flags>... <pattern> <path>...

{
    # Pick ls-files arguments only.
    first=yes skip=no pattern_found=no
    for x
    do
	case "$skip" in yes) skip=no; continue ;; esac 
    	case "$x" in
	--cached|--deleted|--others|--killed|--ignored|--exclude=*|\
	--exclude-from=*|\--exclude-per-directory=*)
		test 'yes' = "$first" && set x
		set "$@" "$x"
		first=no
		;;
	-A|-B|-C|-D|-d|-f|-m)
		skip=yes
		;;
	-e)
		skip=yes
		pattern_found=yes
		;;
	-*)
		;;
	*)
		case "$pattern_found" in
		yes)
			test 'yes' = "$first" && set x
			set "$@" "$x"
			;;
		no)
			;; # not for us, but for grep
		esac
		;;
	esac
    done
    case "$first" in
    yes) # we did not get *any* ls-files parameter.
    	set x ;;
    esac
    shift
    git-ls-files -z "$@"
} | {
    first=yes eat=no pattern_found=no
    for x
    do
	case "$eat" in
	yes)
		eat=no;
		set "$@" "$x"
		continue
		;;
	esac 
    	case "$x" in
	--cached|--deleted|--others|--killed|--ignored|--exclude=*|\
	--exclude-from=*|\--exclude-per-directory=*)
		;;
	-A|-B|-C|-D|-d|-f|-m)
		test 'yes' = "$first" && set x
		set "$@" "$x"
		first=no
		eat=yes
		;;
	-e)
		test 'yes' = "$first" && set x
		set "$@" "$x"
		eat=yes
		pattern_found=yes
		;;
	-*)
		test 'yes' = "$first" && set x
		set "$@" "$x"
		first=no
		;;
	*)
		case "$pattern_found" in
		yes)
			;; # not for us, but for ls-files
		no)
			test 'yes' = "$first" && set x
			set "$@" "$x"
			pattern_found=yes
		esac
		;;
	esac
    done
    case "$eat,$pattern_found" in
    yes,* | *,no)
    	echo >&2 "* malformed grep parameters"
	exit 1
    esac
    shift
    xargs -0 grep "$@"
}

Re: Improve "git grep" flags handling

[Linus Torvalds]

On Mon, 12 Sep 2005, Junio C Hamano wrote:
> 
> Yeah, let's see how tolerant your barf-o-meter is today.

I'm not barfing, but that's probably because my brain just shut down and 
is desperately trying to gouge my eyes out with a spoon.

Wouldn't it be _much_ nicer to just do

	pattern=
	flags=()
	git_flags=()
	while : ; do
		case "$1" in
		--cached|--deleted|--others|--killed|\
		--ignored|--exclude=*|\
		--exclude-from=*|\--exclude-per-directory=*)
			git_flags=("${git_flags[@]}" "$1")
			;;
		-e)
			pattern="$2"
			shift
			;;
		-A|-B|-C|-D|-d|-f|-m)
			flags=("${flags[@]}" "$1" "$2")
			shift
			;;
		--)
			shift
			break
			;;
		-*)
			flags=("${flags[@]}" "$1")
			;;
		*)
			if [ -z "$pattern" ]; then
				pattern="$1"
				shift
			fi
			break
			;;
		esac
		shift
	done
	git-ls-files -z "${git_flags[@]}" | 
		xargs -0 grep "${flags[@]}" "$pattern"

which does use bash array variables, but dang, it does so for a reason: 
they really are very very useful, and they make it _so_ much more pleasant 
to do these things..

		Linus

Re: Improve "git grep" flags handling

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> On Mon, 12 Sep 2005, Junio C Hamano wrote:
>> 
>> Yeah, let's see how tolerant your barf-o-meter is today.
>
> I'm not barfing, but that's probably because my brain just shut down and 
> is desperately trying to gouge my eyes out with a spoon.
>
> Wouldn't it be _much_ nicer to just do

Yes.

[PATCH] Improve "git grep" flags handling

[Junio C Hamano]

This allows any arbitrary flags to "grep", and knows about the few
special grep flags that take an argument too.

It also allows some flags for git-ls-files, although their usefulness
is questionable.

With this, something line

	git grep -w -1 pattern

works, without the script enumerating every possible flag.

[jc: this was the version after I showed Linus a barf-o-meter test
 version that avoids shell arrays.  The emperor penguin must have
 typed this version blindly, since he said:

    I'm not barfing, but that's probably because my brain just shut
    down and is desperately trying to gouge my eyes out with a spoon.

 I slightly fixed it to catch the remaining arguments meant to be
 given git-ls-files.]

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>

---

    Linus Torvalds <torvalds@osdl.org> writes:

    Wouldn't it be _much_ nicer to just do which does use bash
    array variables, but dang, it does so for a reason: they
    really are very very useful, and they make it _so_ much more
    pleasant to do these things..

 Thanks, Linus.  This is what will go into "master" tonight.

 git-grep.sh |   64 ++++++++++++++++++++++++++++++++++++++---------------------
 1 files changed, 41 insertions(+), 23 deletions(-)

6df4eef9b10c8de2b9bc3dc769f3a008a1200df7
diff --git a/git-grep.sh b/git-grep.sh
--- a/git-grep.sh
+++ b/git-grep.sh
@@ -1,25 +1,43 @@
 #!/bin/sh
-flags=
-while :; do
-  pattern="$1"
-  case "$pattern" in
-  -i|-I|-a|-E|-H|-h|-l)
-    flags="$flags $pattern"
-    shift
-    ;;
-  -e)
-    pattern="$2"
-    shift
-    break
-    ;;
-  -*)
-    echo "unknown flag $pattern" >&2
-    exit 1
-    ;;
-  *)
-    break
-    ;;
-  esac
+#
+# Copyright (c) Linus Torvalds, 2005
+#
+
+pattern=
+flags=()
+git_flags=()
+while : ; do
+	case "$1" in
+	--cached|--deleted|--others|--killed|\
+	--ignored|--exclude=*|\
+	--exclude-from=*|\--exclude-per-directory=*)
+		git_flags=("${git_flags[@]}" "$1")
+		;;
+	-e)
+		pattern="$2"
+		shift
+		;;
+	-A|-B|-C|-D|-d|-f|-m)
+		flags=("${flags[@]}" "$1" "$2")
+		shift
+		;;
+	--)
+		# The rest are git-ls-files paths (or flags)
+		shift
+		break
+		;;
+	-*)
+		flags=("${flags[@]}" "$1")
+		;;
+	*)
+		if [ -z "$pattern" ]; then
+			pattern="$1"
+			shift
+		fi
+		break
+		;;
+	esac
+	shift
 done
-shift
-git-ls-files -z "$@" | xargs -0 grep $flags -e "$pattern"
+git-ls-files -z "${git_flags[@]}" "$@" |
+	xargs -0 grep "${flags[@]}" "$pattern"

Re: [PATCH] Improve "git grep" flags handling

[Horst von Brand]

Junio C Hamano <junkio@cox.net> wrote:

[...]

>  Thanks, Linus.  This is what will go into "master" tonight.
> 
>  git-grep.sh |   64 ++++++++++++++++++++++++++++++++++++++---------------------
>  1 files changed, 41 insertions(+), 23 deletions(-)
> 
> 6df4eef9b10c8de2b9bc3dc769f3a008a1200df7
> diff --git a/git-grep.sh b/git-grep.sh
> --- a/git-grep.sh
> +++ b/git-grep.sh
> @@ -1,25 +1,43 @@
>  #!/bin/sh

Shouldn't shebang go /bin/bash, as the script uses bash-isms now?
(For portability to non-enlightened systems the installation would have to
locate bash too... and/or mention this in the INSTALL file)

Or perhaps redo the mess in Perl or some such?
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: [PATCH] Improve "git grep" flags handling

[Linus Torvalds]

On Tue, 13 Sep 2005, Horst von Brand wrote:
> 
> Shouldn't shebang go /bin/bash, as the script uses bash-isms now?
> (For portability to non-enlightened systems the installation would have to
> locate bash too... and/or mention this in the INSTALL file)

Some bash installs only install in /bin/sh, so..

> Or perhaps redo the mess in Perl or some such?

Hey, the code isn't a mess. The fact that there are tons of different 
shells and they don't support it is the mess.

So we should strive for bash syntax to be so common that other shells 
follow suit ;)

I personally find perl to be a really bad language. It has more of a 
unified base (different versions, but at least not totally different and 
unrelated implementations), and it's clearly more powerful, but as a 
_language_ I don't understand how anybody can accept that crap.

The "there's more than one way to do something" slogan may be cute and
sound good to people who are drawn to that thing, but it's actually bad.  
The language is designed to be write-only, and the "you can do it fifty
different ways" is part of it (and line noise characters is another part
of it).

So sh is actually often a much better language. Too bad some of the 
features end up being outside the standard language.

I know, I know, people will consider me crazy for saying that. 

Oh, well. As long as all the _important_ stuff is in C, we're ok.

			Linus

Re: [PATCH] Improve "git grep" flags handling

[Junio C Hamano]

Horst von Brand <vonbrand@inf.utfsm.cl> writes:

> Shouldn't shebang go /bin/bash, as the script uses bash-isms now?
> (For portability to non-enlightened systems the installation would have to
> locate bash too... and/or mention this in the INSTALL file)

I heard somebody say the shell arrays actually came from Korn,
so /bin/bash is a wrong thing to do if that is the case.

> Or perhaps redo the mess in Perl or some such?

Especially given 'xargs grep' part is moderately expensive
anyway and startup overhead between shell and perl does not
matter here very much, the suggestion is mildly tempting.

Re: [PATCH] Improve "git grep" flags handling

[Linus Torvalds]

On Tue, 13 Sep 2005, Junio C Hamano wrote:
> 
> I heard somebody say the shell arrays actually came from Korn,
> so /bin/bash is a wrong thing to do if that is the case.

Me. 

There are differences in indexing the arrays between ksh and bash, but I
think the git-grep.sh style of usage should work on both bash and ksh.

[ goes off and tests ]

Indeed. I just tested the thing I sent out with "ksh git-grep.sh .." and 
it worked fine.

Using "zsh" the grep flags don't work, and "tcsh" obviously will never run
_any_ valid shell code ;)

		Linus