git-grep ignores untracked files

git-grep ignores untracked files

[Mark Lodato]

Currently, git-grep ignores untracked files in the working directory.
For example:

    $ pattern=qazxswedc
    $ echo $pattern > foo
    $ git grep $pattern || echo not found
    not found

This is annoying and counter-intuitive, since it *does* search modified
files with unstaged changes.  More importantly, this is undocumented.

The situation is much worse when untracked files are given on the
command line.  For example, the following command warns me that a file
does not exist, as expected.

    $ git grep $pattern does_not_exist
    fatal: ambiguous argument 'does_not_exist': unknown revision or path not in the working tree.
    Use '--' to separate paths from revisions

However, if I give it an untracked file, it just tells me there's no match.

    $ git grep $pattern foo || echo not found
    not found

In this case, I would expect the program to either search that file
(would be nice!) or die with an error message (I'll take this for now).
Furthermore, when '--' is given, even the non-existent case fails
silently.

    $ git grep $pattern -- does_not_exist || echo not found
    not found

So, what I would I like to see?  Ideally, something like the following
series of patches:

1. Document the current behavior in the man page.

2. Issue a warning if any paths (pattern or non-pattern) fail to match
any files.  Ideally, this would work in all cases (work tree, --cached,
and trees).  If an untracked file matches a pattern that also matches
a tracked file, there would be no warning - not sure what's best in this
case.  This warning should probably be disable-able with a flag, say
--no-warn-if-unmatched.

3. If neither --cached nor trees are given, and a non-pattern filename
is given that does not not exist in the cache, search this file as well.
(That is, instead of issuing a warning, just search it.)  If the
filename is a directory, issue a warning.

4. Add an --untracked flag that causes grep to search *all* files in the
working directory, not just tracked ones.  (This includes searches with
and without a pathspec.)  The option would be incompatible with --cached
or given trees.


I post this wish list for several reasons.  First, to see if anyone
agrees with me.  Second, to see if this patch series has a chance of
ever being accepted.  Third, to implant this idea into someones head so
they write it for me :).  I may try to do it myself, but it will take me
a while.

To get started, here's a shot at #1.

Mark

---- 8< ----
From 697d97ca45b5c7abe8c84c3caae28cf839c668ac Mon Sep 17 00:00:00 2001
From: Mark Lodato <lodatom@gmail.com>
Date: Sun, 14 Feb 2010 19:28:55 -0500
Subject: [PATCH] Documentation: clarify untracked files and <path>

Note that only tracked files are searched, that non-matching paths are
silently ignored, and that <path> is a glob(7) pattern.

Signed-off-by: Mark Lodato <lodatom@gmail.com>
---
 Documentation/git-grep.txt |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)

diff --git a/Documentation/git-grep.txt b/Documentation/git-grep.txt
index e019e76..fdc05ec 100644
--- a/Documentation/git-grep.txt
+++ b/Documentation/git-grep.txt
@@ -27,7 +27,9 @@ SYNOPSIS
 DESCRIPTION
 -----------
 Look for specified patterns in the working tree files, blobs
-registered in the index file, or given tree objects.
+registered in the index file, or given tree objects.  Only tracked files in
+the working tree are searched.  Paths that do not match are silently ignored,
+including paths to untracked files.
 
 
 OPTIONS
@@ -170,6 +172,10 @@ OPTIONS
 	Signals the end of options; the rest of the parameters
 	are <path> limiters.
 
+<path>...::
+	Only search files matching these wildcard patterns; see glob(7) for
+	the format.  If not given, all tracked files in the tree are searched.
+
 
 Example
 -------
-- 
1.7.0

Re: git-grep ignores untracked files

[Junio C Hamano]

>  Look for specified patterns in the working tree files, blobs
> -registered in the index file, or given tree objects.
> +registered in the index file, or given tree objects.  Only tracked files in
> +the working tree are searched.  Paths that do not match are silently ignored,
> +including paths to untracked files.

Strictly speaking, because git commands are about tracked files unless
explicitly stated otherwise, the phrase "files in the work tree" already
means "tracked" ones.  It is understandable (but not excusable) that
people who wrote manual assumed the readers would have learned that from
the user manual without repeating.  Adding explicit description would be a
good thing to do.

I think rewording the paragraph to "... patterns in the tracked files in
the work tree, blobs registered in the index file, ... or given tree
objects."  would be a good balance to strike.  It gives enough information
without making it too verbose.

I doubt we want to have "Only tracked files blah blah".  Like all the
normal git commands, "grep" is about tracked contents, and I don't think
it would help to repeat the obvious like pathspec filter will act as a
filter.  "add <pathspec>" is an exception in that it _is_ about untracked
paths and that is why you get warnings for unmatched ones.

    Side note: there will be --no-index option to let you run "git grep"
    over files in a random directory.

> +<path>...::
> +	Only search files matching these wildcard patterns; see glob(7) for
> +	the format.  If not given, all tracked files in the tree are searched.

Please do *not* "see glob(7) for the format".  Pathspec used for "grep"
(and "ls-files") are "leading path match or glob(7)".  E.g. "git grep
frotz t/" looks for frotz in all files under "t/" recursively, and that
does not have much to do with glob(7).  If we do not have a description
already, we may want to add these basics to git(1) or the user manual.

Thanks.

Re: git-grep ignores untracked files

[Mark Lodato]

On Sun, Feb 14, 2010 at 8:45 PM, Junio C Hamano <gitster@pobox.com> wrote:
> I doubt we want to have "Only tracked files blah blah".  Like all the
> normal git commands, "grep" is about tracked contents, and I don't think
> it would help to repeat the obvious like pathspec filter will act as a
> filter.  "add <pathspec>" is an exception in that it _is_ about untracked
> paths and that is why you get warnings for unmatched ones.

I think adding this information to the description of <path> would be
sufficient.  I'll send a new patch in a minute.

>    Side note: there will be --no-index option to let you run "git grep"
>    over files in a random directory.

Ah, didn't see this.  So, it looks like most of my requests were
already done.  However, it may still be a good idea to DWIM when none
of --cached, --no-index, or trees are given but a pathspec is given
that matches an untracked file in the working directory.  For example:

$ git grep $pattern -- untracked_file

It is obvious that the user meant to specify --no-index.   However,
I'm not sure where to draw the line.  What if they give tracked and
untracked files, or if they given a glob pattern that matches both?

Still, the simple, "one path is given, it's not a pattern, and it
matches exactly to an untracked file" case should probably run with
--no-index.

>> +<path>...::
>> +     Only search files matching these wildcard patterns; see glob(7) for
>> +     the format.  If not given, all tracked files in the tree are searched.
>
> Please do *not* "see glob(7) for the format".  Pathspec used for "grep"
> (and "ls-files") are "leading path match or glob(7)".  E.g. "git grep
> frotz t/" looks for frotz in all files under "t/" recursively, and that
> does not have much to do with glob(7).  If we do not have a description
> already, we may want to add these basics to git(1) or the user manual.

Ah.  This is a major inconsistency in the documentation.  I have a lot
to say about this, so I'll make this a separate thread.  For the
patch, I'll just reference git-add(1), which is the does have a
description, and then we can fix this properly in a future patch.

[PATCHv2] grep documentation: clarify what files match

[Mark Lodato]

Clarify that git-grep(1) searches only tracked files, and that each
<path> is a glob, as in git-add(1).  Add an example to show a simple use
case for searching all .c and .h files.

The meta-variable <path> should be changed to an official term for
a path glob, and the description for this should be in git(1), not
git-add(1).  However, we don't yet have such an official term, so just
use <path> and reference git-add(1) for now.

Signed-off-by: Mark Lodato <lodatom@gmail.com>
---
 Documentation/git-grep.txt |   13 ++++++++++---
 1 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-grep.txt b/Documentation/git-grep.txt
index e019e76..7f24032 100644
--- a/Documentation/git-grep.txt
+++ b/Documentation/git-grep.txt
@@ -26,8 +26,8 @@ SYNOPSIS
 
 DESCRIPTION
 -----------
-Look for specified patterns in the working tree files, blobs
-registered in the index file, or given tree objects.
+Look for specified patterns in the tracked files in the working tree, blobs
+registered in the index file, or blobs in given tree objects.
 
 
 OPTIONS
@@ -49,7 +49,7 @@ OPTIONS
 	Don't match the pattern in binary files.
 
 --max-depth <depth>::
-	For each pathspec given on command line, descend at most <depth>
+	For each <path> given on command line, descend at most <depth>
 	levels of directories. A negative value means no limit.
 
 -w::
@@ -170,10 +170,17 @@ OPTIONS
 	Signals the end of options; the rest of the parameters
 	are <path> limiters.
 
+<path>...::
+	If given, limit the search to paths matching at least one pattern.
+	Each pattern is the same as <filepattern> of linkgit:git-add[1].
 
 Example
 -------
 
+git grep 'time_t' -- '*.[ch]'::
+	Looks for `time_t` in all tracked .c and .h files in the working
+	directory.
+
 git grep -e \'#define\' --and \( -e MAX_PATH -e PATH_MAX \)::
 	Looks for a line that has `#define` and either `MAX_PATH` or
 	`PATH_MAX`.
-- 
1.7.0

Re: [PATCHv2] grep documentation: clarify what files match

[Junio C Hamano]

Mark Lodato <lodatom@gmail.com> writes:

> Clarify that git-grep(1) searches only tracked files, and that each
> <path> is a glob, as in git-add(1).  Add an example to show a simple use
> case for searching all .c and .h files.
>
> The meta-variable <path> should be changed to an official term for
> a path glob, and the description for this should be in git(1), not
> git-add(1).  However, we don't yet have such an official term, so just
> use <path> and reference git-add(1) for now.
>
> Signed-off-by: Mark Lodato <lodatom@gmail.com>
> ---

Looks much better, even though I personally prefer a shorter "work tree"
and also think the official name of the stuff is "pathspec".

Will queue until an alternative materializes (if ever) ;-).

Thanks.