git-grep to operate across who repository and not just CWD?

git-grep to operate across who repository and not just CWD?

[David Chanters]

Hi all,

[ Please Cc me as I am not subscribed to this list, thanks. ]

I'm wondering if there's an easy way to get git-grep (and I suppose
other commands which operate on a per-repository level rather than
per-tree) to work across the whole repository?

Often I will be in the depths of my git repository, run "git grep
--options 'search string'", to find no results.  Of course, then I
remember that git grep doesn't work across the whole repository, it
works like normal grep, and only considers the CWD onwards.
Typically I end up cursing, using {push,pop}d to recall where I am,
cd'ing to the root of the repository and running "git grep" from there
and then poping my CWD to go back to where I was.

Is there some clever trickery or command-line flag I've not read about
in the "git-grep" man page to make this idea more seamless?

TIA,

David

Re: git-grep to operate across who repository and not just CWD?

[Michael J Gruber]

David Chanters venit, vidit, dixit 28.02.2011 01:17:
> Hi all,
> 
> [ Please Cc me as I am not subscribed to this list, thanks. ]
> 
> I'm wondering if there's an easy way to get git-grep (and I suppose
> other commands which operate on a per-repository level rather than
> per-tree) to work across the whole repository?
> 
> Often I will be in the depths of my git repository, run "git grep
> --options 'search string'", to find no results.  Of course, then I
> remember that git grep doesn't work across the whole repository, it
> works like normal grep, and only considers the CWD onwards.
> Typically I end up cursing, using {push,pop}d to recall where I am,
> cd'ing to the root of the repository and running "git grep" from there
> and then poping my CWD to go back to where I was.
> 
> Is there some clever trickery or command-line flag I've not read about
> in the "git-grep" man page to make this idea more seamless?

git grep -- $(git rev-parse --show-cdup)

is the best we have right now. I think we're still looking for a good
way to denote "root of repo" (like "." for cwd).

Also, we're thinking of changing a few defaults (to repo-wide), but "git
grep" is meant to stay close to ordinary grep.

Michael

Re: git-grep to operate across who repository and not just CWD?

[Jay Soffian]

On Mon, Feb 28, 2011 at 4:27 AM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
> git grep -- $(git rev-parse --show-cdup)
>
> is the best we have right now. I think we're still looking for a good
> way to denote "root of repo" (like "." for cwd).
>
> Also, we're thinking of changing a few defaults (to repo-wide), but "git
> grep" is meant to stay close to ordinary grep.

I had the crazy thought that if git had a --cdup option, then this
would work with any command you wanted to run from the top:

  git --cdup grep ...

Maybe that's the best way to expose "from the top please" generically?

j.

Re: git-grep to operate across who repository and not just CWD?

[Junio C Hamano]

Jay Soffian <jaysoffian@gmail.com> writes:

> I had the crazy thought that if git had a --cdup option, then this
> would work with any command you wanted to run from the top:
>
>   git --cdup grep ...
>
> Maybe that's the best way to expose "from the top please" generically?

We tend to give --full-tree option to individual subcommands where it
makes sense to get the same effect.

One problem with a global "git --cdup" is to sensibly handle pathspecs
given from the command line.  You would likely want

	$ cd Documentation
        $ git --cdup grep -e info pu Makefile '*.txt'

to still refer to Makefile relative to Documentation/ directory (imagine
typing "Makef<TAB>" to complete while forming that command line) while
looking for '*.txt' files everywhere in the tree, but only individual
commands can tell which parameters are pathspec in argv[].  Hence the
subcommands have to be aware of the wish (--cdup or --full-tree) of the
user to affect the full tree, not limited to the current working
directory.

Re: git-grep to operate across who repository and not just CWD?

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Jay Soffian <jaysoffian@gmail.com> writes:
>
>> I had the crazy thought that if git had a --cdup option, then this
>> would work with any command you wanted to run from the top:
>>
>>   git --cdup grep ...
>>
>> Maybe that's the best way to expose "from the top please" generically?
>
> We tend to give --full-tree option to individual subcommands where it
> makes sense to get the same effect.
>
> One problem with a global "git --cdup" is ...

Having said all that, here is a tip of the day.  I have this in my .bashrc
for interactive shells.

        cdup () {
                local eh
                eh=$(git rev-parse --is-inside-work-tree) || return
                case "$eh" in
                true)
                        eh=$(git rev-parse --show-toplevel)
                        test -z "$eh" || cd "$eh"
                        ;;
                false)
                        eh=$(git rev-parse --git-dir) || return
                        cd "$eh" || return
                        eh=$(git rev-parse --is-bare-repository) || return
                        test "z$eh" = ztrue || cd ..
                        ;;
                esac
        }

and I can say

	$ cd Documentation/howto
        $ editor *.txt
        $ cdup

to come back to the toplevel after having worked somewhere deep in the
mine.

Re: git-grep to operate across who repository and not just CWD?

[Phil Hord]

(cc list: Pardon this duplicate attempt)

On 02/28/2011 04:27 AM, Michael J Gruber wrote:
> David Chanters venit, vidit, dixit 28.02.2011 01:17:
>> > Hi all,
>> > 
>> > [ Please Cc me as I am not subscribed to this list, thanks. ]
>> > 
>> > I'm wondering if there's an easy way to get git-grep (and I suppose
>> > other commands which operate on a per-repository level rather than
>> > per-tree) to work across the whole repository?
> git grep -- $(git rev-parse --show-cdup)
>
> is the best we have right now. I think we're still looking for a good
> way to denote "root of repo" (like "." for cwd).
>
> Also, we're thinking of changing a few defaults (to repo-wide), but "git
> grep" is meant to stay close to ordinary grep.

But git grep is different than grep in exactly the "files selection"
area.  With grep, I always have to specify the files to search.  With
git-grep, I don't.

Oridinary grep with no paths fails (reads stdin), so when I make this
mistake it is always immediately evident. I retry the command with
paths/wildcards.  But git-grep with no path "works" and I am likely to
forget that it worked only on my $PWD.

git-grep also includes subdirectories and excludes untracked files by
default.  This makes git-grep feel like the "repository grep" tool that
it is. The fact that it does _not_ search from the top of the repository
by default seems (to me) to be the only oddball case.

I would be much more comfortable with David's proposed option turned on
always.  When I want to search "here", I can add a dot.

  git grep foo       # search the whole repository
  git grep foo -- .  # Search only from $PWD

Maybe it's dangerous as an always-on option, as it can break scripts. 
And I'd be happy-ish even with a --full-tree option.  But I think I
would eventually alias it so it always does what I expect.

I know that's not what the original question was, but it's the behavior
I often erroneously expect.

Thoughts?

Phil

Re: git-grep to operate across who repository and not just CWD?

[Michael J Gruber]

Phil Hord venit, vidit, dixit 28.02.2011 23:25:
> (cc list: Pardon this duplicate attempt)
> 
> On 02/28/2011 04:27 AM, Michael J Gruber wrote:
>> David Chanters venit, vidit, dixit 28.02.2011 01:17:
>>>> Hi all,
>>>>
>>>> [ Please Cc me as I am not subscribed to this list, thanks. ]
>>>>
>>>> I'm wondering if there's an easy way to get git-grep (and I suppose
>>>> other commands which operate on a per-repository level rather than
>>>> per-tree) to work across the whole repository?
>> git grep -- $(git rev-parse --show-cdup)
>>
>> is the best we have right now. I think we're still looking for a good
>> way to denote "root of repo" (like "." for cwd).
>>
>> Also, we're thinking of changing a few defaults (to repo-wide), but "git
>> grep" is meant to stay close to ordinary grep.
> 
> But git grep is different than grep in exactly the "files selection"
> area.  With grep, I always have to specify the files to search.  With
> git-grep, I don't.
> 
> Oridinary grep with no paths fails (reads stdin), so when I make this
> mistake it is always immediately evident. I retry the command with
> paths/wildcards.  But git-grep with no path "works" and I am likely to
> forget that it worked only on my $PWD.
> 
> git-grep also includes subdirectories and excludes untracked files by
> default.  This makes git-grep feel like the "repository grep" tool that
> it is. The fact that it does _not_ search from the top of the repository
> by default seems (to me) to be the only oddball case.
> 
> I would be much more comfortable with David's proposed option turned on
> always.  When I want to search "here", I can add a dot.
> 
>   git grep foo       # search the whole repository
>   git grep foo -- .  # Search only from $PWD
> 
> Maybe it's dangerous as an always-on option, as it can break scripts. 
> And I'd be happy-ish even with a --full-tree option.  But I think I
> would eventually alias it so it always does what I expect.
> 
> I know that's not what the original question was, but it's the behavior
> I often erroneously expect.

I would love "git grep" to be repowise, with the simple "git grep ." to
make it relative to cwd. We've discussed making more commands repowise,
and the consensus (which I've stated above, accepting the majority vote)
was that some should stay, e.g. git-grep. The discussion started with

<http://permalink.gmane.org/gmane.comp.version-control.git/166135>

and the consensus is stated here:

<http://permalink.gmane.org/gmane.comp.version-control.git/167149>

This does not prevent you from submitting a "--full-tree" patch for
git-grep, of course.

Michael

Re: git-grep to operate across who repository and not just CWD?

[Nguyen Thai Ngoc Duy]

On Tue, Mar 1, 2011 at 3:05 PM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
> I would love "git grep" to be repowise, with the simple "git grep ." to
> make it relative to cwd. We've discussed making more commands repowise,
> and the consensus (which I've stated above, accepting the majority vote)
> was that some should stay, e.g. git-grep. The discussion started with
>
> <http://permalink.gmane.org/gmane.comp.version-control.git/166135>
>
> and the consensus is stated here:
>
> <http://permalink.gmane.org/gmane.comp.version-control.git/167149>
>
> This does not prevent you from submitting a "--full-tree" patch for
> git-grep, of course.

In fact Junio wrote one:

http://mid.gmane.org/7vk4xggv27.fsf@alter.siamese.dyndns.org

If I remember correctly, it was dropped because of the interaction
with pathspecs (relative to cwd vs to worktree's root). I'd be great
if someone can pick it up and finish it.
-- 
Duy

Re: git-grep to operate across who repository and not just CWD?

[Michael J Gruber]

Nguyen Thai Ngoc Duy venit, vidit, dixit 01.03.2011 09:16:
> On Tue, Mar 1, 2011 at 3:05 PM, Michael J Gruber
> <git@drmicha.warpmail.net> wrote:
>> I would love "git grep" to be repowise, with the simple "git grep ." to
>> make it relative to cwd. We've discussed making more commands repowise,
>> and the consensus (which I've stated above, accepting the majority vote)
>> was that some should stay, e.g. git-grep. The discussion started with
>>
>> <http://permalink.gmane.org/gmane.comp.version-control.git/166135>
>>
>> and the consensus is stated here:
>>
>> <http://permalink.gmane.org/gmane.comp.version-control.git/167149>
>>
>> This does not prevent you from submitting a "--full-tree" patch for
>> git-grep, of course.
> 
> In fact Junio wrote one:
> 
> http://mid.gmane.org/7vk4xggv27.fsf@alter.siamese.dyndns.org

Oh yes, thanks for reminding me. And guess who replied first back then?
Oh well...

> If I remember correctly, it was dropped because of the interaction
> with pathspecs (relative to cwd vs to worktree's root). I'd be great
> if someone can pick it up and finish it.

Rereading that thread, I don't think there was any objection against
"--full-tree", but it suffered from DTD (discussed-to-death). To make it
really usefull, one would need a short-cut/option/whatever, and that's
where the discussion went astray. I have an idea, though :)

Michael

Re: git-grep to operate across who repository and not just CWD?

[Nguyen Thai Ngoc Duy]

On Tue, Mar 1, 2011 at 3:54 PM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
>> If I remember correctly, it was dropped because of the interaction
>> with pathspecs (relative to cwd vs to worktree's root). I'd be great
>> if someone can pick it up and finish it.
>
> Rereading that thread, I don't think there was any objection against
> "--full-tree", but it suffered from DTD (discussed-to-death). To make it
> really usefull, one would need a short-cut/option/whatever, and that's
> where the discussion went astray. I have an idea, though :)

No it was in next or pu for a while, then got dropped out. I don't
remember what "what's cooking" mail though.
-- 
Duy

Re: git-grep to operate across who repository and not just CWD?

[Nguyen Thai Ngoc Duy]

On Tue, Mar 1, 2011 at 4:32 PM, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
> No it was in next or pu for a while, then got dropped out. I don't
> remember what "what's cooking" mail though.

I got it, 1dbdbcb (What's cooking (2009/12 #04)):

+The interaction with this option and pathspecs need to be worked out
+better.  I _think_ "grep --full-tree -e pattern -- '*.h'" should find from
+all the header files in the tree, for example.
-- 
Duy

[PATCH/POC 0/2] grep --full-tree

[Michael J Gruber]

I think we could make --full-tree useful with pathspecs like below. There are
no tests yet and no doc because I actually favour a pathspec based solution,
and that would possibly lead to mixed pathspecs (relative+absolute),
interacting badly with an overall option.

But I have to look at the pathspec stuff first. Or maybe at format-patch, if I
interpret the latest What's Cooking correctly...

Junio C Hamano (1):
  grep: --full-tree

Michael J Gruber (1):
  grep: make --full-tree work with pathspecs

 builtin/grep.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

-- 
1.7.4.1.257.gb09fa

[PATCH/RFC 1/2] grep: --full-tree

[Michael J Gruber]

From: Junio C Hamano <gitster@pobox.com>

While working inside a deep subdirectory, it sometimes is necessary to
find a string you see in a file you are working on from the files in the
entire project.  This is especially true when you are dipping your toe
into an unfamiliar project.

By default, "git grep" limits its search space to the current directory
and below (i.e. as if "-r ." is specified), and it is rather cumbersome to
repeat ../ as many times as necessary.  This new option tells "git grep"
not to limit the search space to the current directory.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 builtin/grep.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/builtin/grep.c b/builtin/grep.c
index 5afee2f..d005528 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -728,6 +728,7 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 {
 	int hit = 0;
 	int cached = 0;
+	int full_tree = 0;
 	int seen_dashdash = 0;
 	int external_grep_allowed__ignored;
 	const char *show_in_pager = NULL, *default_pager = "dummy";
@@ -773,6 +774,8 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 		OPT_BIT('H', NULL, &opt.pathname, "show filenames", 1),
 		OPT_NEGBIT(0, "full-name", &opt.relative,
 			"show filenames relative to top directory", 1),
+		OPT_BIT(0, "full-tree", &full_tree,
+			"search from the top of the tree", 1),
 		OPT_BOOLEAN('l', "files-with-matches", &opt.name_only,
 			"show only filenames instead of matching lines"),
 		OPT_BOOLEAN(0, "name-only", &opt.name_only,
@@ -957,7 +960,7 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 
 	if (i < argc)
 		paths = get_pathspec(prefix, argv + i);
-	else if (prefix) {
+	else if (prefix && !full_tree) {
 		paths = xcalloc(2, sizeof(const char *));
 		paths[0] = prefix;
 		paths[1] = NULL;
-- 
1.7.4.1.257.gb09fa

[PATCH/RFC 2/2] grep: make --full-tree work with pathspecs

[Michael J Gruber]

When --full-tree is given, make the pathspecs be applied relative to the
root. That way, "git grep --full-tree expr -- *.c" looks in all C files in
the repo.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
 builtin/grep.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/builtin/grep.c b/builtin/grep.c
index d005528..e0a75a7 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -959,7 +959,7 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 	}
 
 	if (i < argc)
-		paths = get_pathspec(prefix, argv + i);
+		paths = get_pathspec(!full_tree ? prefix : NULL, argv + i);
 	else if (prefix && !full_tree) {
 		paths = xcalloc(2, sizeof(const char *));
 		paths[0] = prefix;
-- 
1.7.4.1.257.gb09fa

[PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Michael J Gruber]

Introduce a leading ':' as the notation for repo-wide pathspecs.

This is in line with our treeish:path notation which defaults to
repowide paths.

Heck: Even ':./path' works for pathspecs, and I have no clue why!

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
I have not tested this for adverse side effects, only for the intended ones
with "git grep".

It's an alternative to "--full-tree", more far reaching, maybe too far.
But we seem to accept that paths do not start with ':', and that notation
should make msys happy, but what do I know about msys.

(one could stick this into prefix_path() rather than get_pathspec(), but that
would be way too far)

 setup.c |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/setup.c b/setup.c
index 021d013..e7b05f0 100644
--- a/setup.c
+++ b/setup.c
@@ -147,7 +147,11 @@ const char **get_pathspec(const char *prefix, const char **pathspec)
 	dst = pathspec;
 	prefixlen = prefix ? strlen(prefix) : 0;
 	while (*src) {
-		const char *p = prefix_path(prefix, prefixlen, *src);
+		const char *p;
+		if ((*src)[0] == ':')
+			p = prefix_path(NULL, 0, (*src)+1);
+		else
+			p = prefix_path(prefix, prefixlen, *src);
 		*(dst++) = p;
 		src++;
 	}
-- 
1.7.4.1.257.gb09fa

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Nguyen Thai Ngoc Duy]

2011/3/1 Michael J Gruber <git@drmicha.warpmail.net>:
> Introduce a leading ':' as the notation for repo-wide pathspecs.
>
> This is in line with our treeish:path notation which defaults to
> repowide paths.
>
> Heck: Even ':./path' works for pathspecs, and I have no clue why!

If you are going to turn pathspecs into something more complex,
reserve room for future extension. I have negative pathspecs that can
utilize it.

I take it, from now on people must refer file name ':foo' as './:foo'
with your patch?
-- 
Duy

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Michael J Gruber]

Nguyen Thai Ngoc Duy venit, vidit, dixit 01.03.2011 12:13:
> 2011/3/1 Michael J Gruber <git@drmicha.warpmail.net>:
>> Introduce a leading ':' as the notation for repo-wide pathspecs.
>>
>> This is in line with our treeish:path notation which defaults to
>> repowide paths.
>>
>> Heck: Even ':./path' works for pathspecs, and I have no clue why!
> 
> If you are going to turn pathspecs into something more complex,
> reserve room for future extension. I have negative pathspecs that can
> utilize it.
> 
> I take it, from now on people must refer file name ':foo' as './:foo'
> with your patch?

That is up for discussion, of course. When discussing a new approach for
file mode dependent attributes, I was hoping to get through with
symlink:path, and did not. But it was decided that something like
:symlink:path would be good enough, in the sense of avoiding enough
possible conflicts. That made me hope that :path would be, too.

(I have not checked for interaction of those two, which are in flight.)

I would think that file names like ":foo" are problematic on msys
already, so in a sense they are no-no already, and free to take as
special notation.

Michael

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Michael J Gruber]

Michael J Gruber venit, vidit, dixit 01.03.2011 11:21:
> Introduce a leading ':' as the notation for repo-wide pathspecs.
> 
> This is in line with our treeish:path notation which defaults to
> repowide paths.
> 
> Heck: Even ':./path' works for pathspecs, and I have no clue why!
> 
> Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
> ---
> I have not tested this for adverse side effects, only for the intended ones
> with "git grep".

To follow up:

Of all callers of get_pathspec() (add, checkout, clean, commit, grep,
ls-files, ls-tree, mv, rerere, reset, rm), all work perfectly, except:

ls-tree seems to be special not only in this respect.

rerere I'm too dumb to test.

Even things like

git mv :Makefile Makefile

work and mv Makefile from the root to your current work dir!

This may even provide an alternative to switching some defaults to "repo
wide".

Michael

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Nguyen Thai Ngoc Duy]

On Tue, Mar 1, 2011 at 6:16 PM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
> Nguyen Thai Ngoc Duy venit, vidit, dixit 01.03.2011 12:13:
>> 2011/3/1 Michael J Gruber <git@drmicha.warpmail.net>:
>>> Introduce a leading ':' as the notation for repo-wide pathspecs.
>>>
>>> This is in line with our treeish:path notation which defaults to
>>> repowide paths.
>>>
>>> Heck: Even ':./path' works for pathspecs, and I have no clue why!
>>
>> If you are going to turn pathspecs into something more complex,
>> reserve room for future extension. I have negative pathspecs that can
>> utilize it.
>>
>> I take it, from now on people must refer file name ':foo' as './:foo'
>> with your patch?
>
> That is up for discussion, of course. When discussing a new approach for
> file mode dependent attributes, I was hoping to get through with
> symlink:path, and did not. But it was decided that something like
> :symlink:path would be good enough, in the sense of avoiding enough
> possible conflicts. That made me hope that :path would be, too.

Take me down. I'm going crazy now.

Another, less cryptic choice, is to make these special notations
separate from true pathspecs. For example, instead of ":foo" we can
say "--root foo". get_pathspec() and friends can be updated to remove
--root and rewrite the next pathspec. Extensibility is obvious.
Problems are plenty:

 - may be confused with command line options without "--" as separator
("-:" on the other hand is not, but looks weird)
 - '-' is not a reserved letter, the same as ':'
 - ...

> (I have not checked for interaction of those two, which are in flight.)
>
> I would think that file names like ":foo" are problematic on msys
> already, so in a sense they are no-no already, and free to take as
> special notation.

Back to what I'm writing above, '-' may be chosen over ':' even
without separation because UNIXers are trained that '-' is usually the
beginning of something special, I suppose most of us would go with
./-blah for file names.
-- 
Duy

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Michael J Gruber]

Nguyen Thai Ngoc Duy venit, vidit, dixit 01.03.2011 12:50:
> On Tue, Mar 1, 2011 at 6:16 PM, Michael J Gruber
> <git@drmicha.warpmail.net> wrote:
>> Nguyen Thai Ngoc Duy venit, vidit, dixit 01.03.2011 12:13:
>>> 2011/3/1 Michael J Gruber <git@drmicha.warpmail.net>:
>>>> Introduce a leading ':' as the notation for repo-wide pathspecs.
>>>>
>>>> This is in line with our treeish:path notation which defaults to
>>>> repowide paths.
>>>>
>>>> Heck: Even ':./path' works for pathspecs, and I have no clue why!
>>>
>>> If you are going to turn pathspecs into something more complex,
>>> reserve room for future extension. I have negative pathspecs that can
>>> utilize it.
>>>
>>> I take it, from now on people must refer file name ':foo' as './:foo'
>>> with your patch?
>>
>> That is up for discussion, of course. When discussing a new approach for
>> file mode dependent attributes, I was hoping to get through with
>> symlink:path, and did not. But it was decided that something like
>> :symlink:path would be good enough, in the sense of avoiding enough
>> possible conflicts. That made me hope that :path would be, too.
> 
> Take me down. I'm going crazy now.

What is crazy about that?

HEAD:path is repo wide already

:path is also, after this patch

Note that when you have a file named :foo now, it can already be
mistaken as the blob at "foo" in the index (or HEAD) already, in places
where rev:path makes sense. So you would need quotation before my patch.

> Another, less cryptic choice, is to make these special notations
> separate from true pathspecs. For example, instead of ":foo" we can
> say "--root foo". get_pathspec() and friends can be updated to remove
> --root and rewrite the next pathspec. Extensibility is obvious.

Only that some commands have "--root" as an option, and even if not,
it's just too much to type.

> Problems are plenty:
> 
>  - may be confused with command line options without "--" as separator
> ("-:" on the other hand is not, but looks weird)
>  - '-' is not a reserved letter, the same as ':'
>  - ...
> 
>> (I have not checked for interaction of those two, which are in flight.)
>>
>> I would think that file names like ":foo" are problematic on msys
>> already, so in a sense they are no-no already, and free to take as
>> special notation.
> 
> Back to what I'm writing above, '-' may be chosen over ':' even
> without separation because UNIXers are trained that '-' is usually the
> beginning of something special, I suppose most of us would go with
> ./-blah for file names.

If ":" is crazy which is in line with our current notation, then how do
you call "-"? "-" is

- a short option identifier
- a negation (attributes)
- a notation for stdin

Michael

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Nguyen Thai Ngoc Duy]

On Tue, Mar 1, 2011 at 6:57 PM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
> HEAD:path is repo wide already
>
> :path is also, after this patch
>
> Note that when you have a file named :foo now, it can already be
> mistaken as the blob at "foo" in the index (or HEAD) already, in places
> where rev:path makes sense. So you would need quotation before my patch.

No. ':foo' as a reference to 'foo' in index is a SHA1-extended syntax
and I think we try to avoid ambiguation when a sha1-extended syntax
may look like a path or vice versa.

>> Another, less cryptic choice, is to make these special notations
>> separate from true pathspecs. For example, instead of ":foo" we can
>> say "--root foo". get_pathspec() and friends can be updated to remove
>> --root and rewrite the next pathspec. Extensibility is obvious.
>
> Only that some commands have "--root" as an option, and even if not,
> it's just too much to type.

Yes, choose one between cryptic/short and descriptive/long :)

>> Back to what I'm writing above, '-' may be chosen over ':' even
>> without separation because UNIXers are trained that '-' is usually the
>> beginning of something special, I suppose most of us would go with
>> ./-blah for file names.
>
> If ":" is crazy which is in line with our current notation, then how do
> you call "-"? "-" is
>
> - a short option identifier
> - a negation (attributes)
> - a notation for stdin

'-' is crazy, not ':'. Perhaps I'm embracing '-' too much.
-- 
Duy

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Phil Hord]

On 03/01/2011 06:49 AM, Michael J Gruber wrote:
> Michael J Gruber venit, vidit, dixit 01.03.2011 11:21:
>> Introduce a leading ':' as the notation for repo-wide pathspecs.
>>
>> This is in line with our treeish:path notation which defaults to
>> repowide paths.
>>
>> Heck: Even ':./path' works for pathspecs, and I have no clue why!

I like it.  Thanks for looking at this.

Phil

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Junio C Hamano]

Nguyen Thai Ngoc Duy <pclouds@gmail.com> writes:

> No. ':foo' as a reference to 'foo' in index is a SHA1-extended syntax
> and I think we try to avoid ambiguation when a sha1-extended syntax
> may look like a path or vice versa.

Very true.

Just as a thought experiment (I am skeptical about this whole "this is
from root" prefix idea to begin with, but I don't want to shoot an idea
down prematurely when there may still be untold gems I haven't seen in
it):

    $ git grep -e frotz .../

to abbreviate "I don't bother to count my ../" might be an alternative,
though.

The reason I am skeptical about the "from root prefix" is because I do not
see a way to make it compatible with other meaningful pathspecs.

    $ cd Documentation
    $ git grep -e frotz '*.txt'

would find frotz in all *.txt files in Documentation (and its
subdirectories), if the command takes "relatigve to cwd".

It also is very clear that

    $ cd Documentation
    $ git grep --full-tree -e frotz '*.txt'

would find those anywhere, inside or outside Documentation.

On the other hand, it is natural to expect that

    $ git grep -e frotz ".../*.txt"

should find *.txt files _only_ at the root level, so it is not as useful as
the --full-tree (or --root).

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Michael J Gruber]

Junio C Hamano venit, vidit, dixit 01.03.2011 15:50:
> Nguyen Thai Ngoc Duy <pclouds@gmail.com> writes:
> 
>> No. ':foo' as a reference to 'foo' in index is a SHA1-extended syntax
>> and I think we try to avoid ambiguation when a sha1-extended syntax
>> may look like a path or vice versa.
> 
> Very true.
> 
> Just as a thought experiment (I am skeptical about this whole "this is
> from root" prefix idea to begin with, but I don't want to shoot an idea
> down prematurely when there may still be untold gems I haven't seen in
> it):
> 
>     $ git grep -e frotz .../
> 
> to abbreviate "I don't bother to count my ../" might be an alternative,
> though.
> 
> The reason I am skeptical about the "from root prefix" is because I do not
> see a way to make it compatible with other meaningful pathspecs.
> 
>     $ cd Documentation
>     $ git grep -e frotz '*.txt'
> 
> would find frotz in all *.txt files in Documentation (and its
> subdirectories), if the command takes "relatigve to cwd".
> 
> It also is very clear that
> 
>     $ cd Documentation
>     $ git grep --full-tree -e frotz '*.txt'
> 
> would find those anywhere, inside or outside Documentation.
> 
> On the other hand, it is natural to expect that
> 
>     $ git grep -e frotz ".../*.txt"
> 
> should find *.txt files _only_ at the root level, so it is not as useful as
> the --full-tree (or --root).

Exactly that is (one of the reasons) why I used something which does not
look like "as many ../ as necessary" nor like "/". With my implementation,

git grep -e frotz ":*.txt"

from a subdir will grep the exact same files as

(cd $(git rev-parse --cdup) && git grep -e frotz "*.txt")

will (it is --full-tree!), and will output the results relative to the
current workdir.

Note that we already have to disambiguate between revspecs and pathspecs
with -- in several places; that is not different with the new notation,
and even not more frequent if it is not used.

I have to say I'm really excited about how transparently this works
across all kinds of commands, and how suggestive this is with rev:path
in mind.

Also, e.g.,

git grep -e frotz "*.c" ":*.h"

will look in all C files in the cwd and and all headers everywhere. Just
think of the possibilities, and of the usefulness with clean, add,
commit, reset,...!

Michael

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Phil Hord]

On 03/01/2011 09:50 AM, Junio C Hamano wrote:
> The reason I am skeptical about the "from root prefix" is because I do not
> see a way to make it compatible with other meaningful pathspecs.
>
>     $ cd Documentation
>     $ git grep -e frotz '*.txt'
>
> would find frotz in all *.txt files in Documentation (and its
> subdirectories), if the command takes "relatigve to cwd".
>
> It also is very clear that
>
>     $ cd Documentation
>     $ git grep --full-tree -e frotz '*.txt'
>
> would find those anywhere, inside or outside Documentation.
>
> On the other hand, it is natural to expect that
>
>     $ git grep -e frotz ".../*.txt"
>
> should find *.txt files _only_ at the root level, so it is not as
useful as
> the --full-tree (or --root).

I don't understand this last statement.  I think it implies that it is
also natural to expect that

    $ git grep -e frotz -- "../*.txt"

should find *.txt files _only_ in the parent directory.  But this is not
the case.  It returns the same search results as

    $ ( cd .. ; git grep -e frotz -- "*.txt" )

Phil

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[James Pickens]

Junio C Hamano <gitster@pobox.com> wrote:
> On the other hand, it is natural to expect that
>
>    $ git grep -e frotz ".../*.txt"
>
> should find *.txt files _only_ at the root level, so it is not as useful as
> the --full-tree (or --root).

I've often wished Git supported the zsh '**' wildcard to match anything,
including slashes.  Here's another case where it would be useful - to
match *.txt everywhere, you'd use ".../**.txt", and it leaves you the
option of using ".../*.txt" if you really want to find *.txt only at the
root level.  Of course, it may be very difficult to implement...

James

Re: [PATCH/RFC 2/2] grep: make --full-tree work with pathspecs

[Junio C Hamano]

Michael J Gruber <git@drmicha.warpmail.net> writes:

> When --full-tree is given, make the pathspecs be applied relative to the
> root. That way, "git grep --full-tree expr -- *.c" looks in all C files in
> the repo.

This is basically Ok, but I wonder if this has funny interaction with
rev/path disambiguation.  What happens to these two "git grep", and what
should happen?

	cd Documentation
        git grep --full-tree index technical
        git grep --full-tree index Documentation/technical

Once you said --full-tree, "technical" does not refer to Documentation/technical
so the first one should probably say "technical is not a rev and there is
no such path" while the second one should know Documentation/technical/ is
a pathspec, perhaps?

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Junio C Hamano]

Michael J Gruber <git@drmicha.warpmail.net> writes:

> Also, e.g.,
>
> git grep -e frotz "*.c" ":*.h"
>
> will look in all C files in the cwd and and all headers everywhere.

Ah, that is cute.  I don't know if the syntax is acceptable to the general
public, but I do see the beauty in that approach of marking individual
pathspec as "(the rest is) from root".

Nice.

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Nguyen Thai Ngoc Duy]

On Tue, Mar 1, 2011 at 6:16 PM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
> Nguyen Thai Ngoc Duy venit, vidit, dixit 01.03.2011 12:13:
>> 2011/3/1 Michael J Gruber <git@drmicha.warpmail.net>:
>>> Introduce a leading ':' as the notation for repo-wide pathspecs.
>>>
>>> This is in line with our treeish:path notation which defaults to
>>> repowide paths.
>>>
>>> Heck: Even ':./path' works for pathspecs, and I have no clue why!
>>
>> If you are going to turn pathspecs into something more complex,
>> reserve room for future extension. I have negative pathspecs that can
>> utilize it.
>>
>> I take it, from now on people must refer file name ':foo' as './:foo'
>> with your patch?
>
> That is up for discussion, of course. When discussing a new approach for
> file mode dependent attributes, I was hoping to get through with
> symlink:path, and did not. But it was decided that something like
> :symlink:path would be good enough, in the sense of avoiding enough
> possible conflicts. That made me hope that :path would be, too.

Good morning! I'm saner now. How about :/path? That would reserve
anything next to ':' except '/'.
-- 
Duy

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Sverre Rabbelier]

Heya,

On Tue, Mar 1, 2011 at 16:01, Michael J Gruber <git@drmicha.warpmail.net> wrote:
> I have to say I'm really excited about how transparently this works
> across all kinds of commands, and how suggestive this is with rev:path
> in mind.

I like it, especially considering how small the impact on the codebase
is. The downside is (once again) backwards compatibility though, I
haven't heard much on how to address that, other than "just quote it"
(which _I_ think is fine, people with filenames that start with fancy
characters are probably used to quoting them anyway), but what does
Junio think?

-- 
Cheers,

Sverre Rabbelier

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Nguyen Thai Ngoc Duy]

On Wed, Mar 2, 2011 at 7:34 PM, Sverre Rabbelier <srabbelier@gmail.com> wrote:
> Heya,
>
> On Tue, Mar 1, 2011 at 16:01, Michael J Gruber <git@drmicha.warpmail.net> wrote:
>> I have to say I'm really excited about how transparently this works
>> across all kinds of commands, and how suggestive this is with rev:path
>> in mind.
>
> I like it, especially considering how small the impact on the codebase
> is. The downside is (once again) backwards compatibility though, I
> haven't heard much on how to address that, other than "just quote it"
> (which _I_ think is fine, people with filenames that start with fancy
> characters are probably used to quoting them anyway)

Yeah. And if this is accepted, the "git add -u (without dot)" issue
may cool down. I personally don't mind typing "git add -u :" (or "git
add -u :/").
-- 
Duy

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Michael J Gruber]

Nguyen Thai Ngoc Duy venit, vidit, dixit 02.03.2011 13:57:
> On Wed, Mar 2, 2011 at 7:34 PM, Sverre Rabbelier <srabbelier@gmail.com> wrote:
>> Heya,
>>
>> On Tue, Mar 1, 2011 at 16:01, Michael J Gruber <git@drmicha.warpmail.net> wrote:
>>> I have to say I'm really excited about how transparently this works
>>> across all kinds of commands, and how suggestive this is with rev:path
>>> in mind.
>>
>> I like it, especially considering how small the impact on the codebase
>> is. The downside is (once again) backwards compatibility though, I
>> haven't heard much on how to address that, other than "just quote it"
>> (which _I_ think is fine, people with filenames that start with fancy
>> characters are probably used to quoting them anyway)
> 
> Yeah. And if this is accepted, the "git add -u (without dot)" issue
> may cool down. I personally don't mind typing "git add -u :" (or "git
> add -u :/").

Why not even ":)"

Seriously, I'm glad this is gaining support. As for the notation, I
tried to take several things into account, which is only possible by
compromising somewhat on some:

- usability (as short as possible - 1 char optimum, 2 at most)

- suggestiveness, e.g. ":path" like in "rev:path" in line with git
usage, or "/path" in line with unix usage (although this has the wrong
connotation of being anchored at root)

- backward compatibility (new code does not misinterpret old notation)

- msysgit compatibility (I think "/" has issues)

- disambiguation from other notation (notably rev:path)

I ended up compromising slightly on the last one. Note that this does
not introduce additional ambiguities for existing use cases[*], only for
the new notation, i.e. commands expecting "treeish pathspec" need a
helping double dash when they are feed the new :pathspec without a treeish.

Michael

[*] I keep forgetting that some people may have files whose names begin
with ":". They are ambiguous now already with "treeish pathspec"
commands, but not with "pathspec" commands. The latter would change.

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Junio C Hamano]

Michael J Gruber <git@drmicha.warpmail.net> writes:

> [*] I keep forgetting that some people may have files whose names begin
> with ":". They are ambiguous now already with "treeish pathspec"
> commands, but not with "pathspec" commands. The latter would change.

Just to make sure I understand that they have easy workarounds:

 - If you have a path foo/:bar, you can say

   git log master -- foo/:bar

   because ':' signals the magic and gets stripped only when it is at the
   beginning (i.e. not affecting foo/:bar); and

 - For :boz at the root level, you can say

   git log master -- '\:boz'

   because the backslash in '\:boz' makes the colon not at the beginning and
   the glob match sees '\:boz' and then matches '\:' with literal ':' at the
   beginning of the pathname ":boz".

In very old times, git used to work only from the top-level of the working
tree.

The way we give an illusion that a command is restricted within the
current working directory was by learning the "prefix" returned by
setup_git_directory() while it chdir(2)'s up to the root level of the
working tree, and then by limiting the operation to the pathspec given
from the command line (each of whose elements prefixed by "prefix" by
calling get_pathspec()).

Your ':'-prefix trick will naturally work very well with this arrangement.
Instead of prefixing the "prefix", you would just strip ':' from the front
for such a magic pathspec element, and that should be all that is necessary.

There is a small worry, though.  Some codepaths have tricks that take
advantage of the knowledge of the current behaviour that the resulting
pathspec elements all refer to subtree under the "prefix", and try to
optimize their tree traversal.  I think dir.c:fill_directory()'s use of
common_prefix() is safe (it recomputes what is common based on the result
of get_pathspec(), not blindly using the original "prefix"), but we need
to make sure there isn't a codepath that blindly believes that the
original "prefix" defines the extent of the operation.  Anything that
understands "../" to step outside the cwd should be already safe, so I
hopefully am being worried too much.

Earlier, the list consensus was that if we were to aim for uniformity, we
should make everything relative to the root of the working tree when there
is no pathspec by default, because you can always give a single '.' to
restrict the extent of the operation to the cwd, but you cannot extend the
extent of the operation without tediously counting "../".  Would this ':'
trick affect that argument?  If a command is relative to the cwd with no
pathspec, you can now give a single ':' to affect the whole tree.

As I wrote in my response to Jeff in

  http://thread.gmane.org/gmane.comp.version-control.git/133570/focus=133874

I always thought that it would be the best solution that makes the choice
of the default irrelevant, and this ":" trick certainly feels like this is
that solution (I also think having a good default matters).

And we can start thinking about deprecating --full-tree option, no?  I
like that, too ;-).

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Michael J Gruber]

Junio C Hamano venit, vidit, dixit 02.03.2011 17:53:
> Michael J Gruber <git@drmicha.warpmail.net> writes:
> 
>> [*] I keep forgetting that some people may have files whose names begin
>> with ":". They are ambiguous now already with "treeish pathspec"
>> commands, but not with "pathspec" commands. The latter would change.
> 
> Just to make sure I understand that they have easy workarounds:
> 
>  - If you have a path foo/:bar, you can say
> 
>    git log master -- foo/:bar
> 
>    because ':' signals the magic and gets stripped only when it is at the
>    beginning (i.e. not affecting foo/:bar); and

Yes.

> 
>  - For :boz at the root level, you can say
> 
>    git log master -- '\:boz'
> 
>    because the backslash in '\:boz' makes the colon not at the beginning and
>    the glob match sees '\:boz' and then matches '\:' with literal ':' at the
>    beginning of the pathname ":boz".

Yes. Due to the shell escaping, a shorter way is

git log master -- ::boz

:)

> 
> In very old times, git used to work only from the top-level of the working
> tree.
> 
> The way we give an illusion that a command is restricted within the
> current working directory was by learning the "prefix" returned by
> setup_git_directory() while it chdir(2)'s up to the root level of the
> working tree, and then by limiting the operation to the pathspec given
> from the command line (each of whose elements prefixed by "prefix" by
> calling get_pathspec()).
> 
> Your ':'-prefix trick will naturally work very well with this arrangement.
> Instead of prefixing the "prefix", you would just strip ':' from the front
> for such a magic pathspec element, and that should be all that is necessary.

and pretend prefix == NULL, exactly.

> 
> There is a small worry, though.  Some codepaths have tricks that take
> advantage of the knowledge of the current behaviour that the resulting
> pathspec elements all refer to subtree under the "prefix", and try to
> optimize their tree traversal.  I think dir.c:fill_directory()'s use of
> common_prefix() is safe (it recomputes what is common based on the result
> of get_pathspec(), not blindly using the original "prefix"), but we need
> to make sure there isn't a codepath that blindly believes that the
> original "prefix" defines the extent of the operation.  Anything that
> understands "../" to step outside the cwd should be already safe, so I
> hopefully am being worried too much.

Except for rerere, I've tried all callers, and all work. ls-tree is a
bit strange, but that was true already for rev:path. I think it's OK if
ls-tree does not grok this, but I'll have another look.

> 
> Earlier, the list consensus was that if we were to aim for uniformity, we
> should make everything relative to the root of the working tree when there
> is no pathspec by default, because you can always give a single '.' to
> restrict the extent of the operation to the cwd, but you cannot extend the
> extent of the operation without tediously counting "../".

Hadn't we decided there were exceptions (e.g. grep), and there weren't
that many suggested changes (to repo-wide) left?

>  Would this ':'
> trick affect that argument?  If a command is relative to the cwd with no
> pathspec, you can now give a single ':' to affect the whole tree.

In my view yes. I would even say: If we don't change every single
command to repo-wide default there is no need to change (and break
things) if we have an easy one-character way of saying "repo-wide".

> 
> As I wrote in my response to Jeff in
> 
>   http://thread.gmane.org/gmane.comp.version-control.git/133570/focus=133874
> 
> I always thought that it would be the best solution that makes the choice
> of the default irrelevant, and this ":" trick certainly feels like this is
> that solution (I also think having a good default matters).
> 
> And we can start thinking about deprecating --full-tree option, no?  I
> like that, too ;-).
> 

:)

Michael

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Miles Bader]

Michael J Gruber <git@drmicha.warpmail.net> writes:
>>  Would this ':'
>> trick affect that argument?  If a command is relative to the cwd with no
>> pathspec, you can now give a single ':' to affect the whole tree.
>
> In my view yes. I would even say: If we don't change every single
> command to repo-wide default there is no need to change (and break
> things) if we have an easy one-character way of saying "repo-wide".

... except, of course that the current state is still confusingly
inconsistent.  Even if ":" is available, and even if somebody knows
about it, they won't use it unless they know they have to because people
are lazy, particularly when typing at the command-line.

There will _still_ be tons of times when people don't realize they're in
a subdirectory, and so need ":", or don't realize that command X doesn't
follow the majority of commands in using the "no args = root relative"
behavior.  So the current state of things is still somewhat dangerous
for users.

Something like ":" would be a great feature for scripting though.

-Miles

-- 
Discriminate, v.i. To note the particulars in which one person or thing is,
if possible, more objectionable than another.

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Phil Hord]

On 03/02/2011 11:53 AM, Junio C Hamano wrote:
> Michael J Gruber <git@drmicha.warpmail.net> writes:
>> [*] I keep forgetting that some people may have files whose names begin
>> with ":". They are ambiguous now already with "treeish pathspec"
>> commands, but not with "pathspec" commands. The latter would change.
> Just to make sure I understand that they have easy workarounds:
>
>  - If you have a path foo/:bar, you can say
>
>    git log master -- foo/:bar
>
>    because ':' signals the magic and gets stripped only when it is at the
>    beginning (i.e. not affecting foo/:bar); and
>
>  - For :boz at the root level, you can say
>
>    git log master -- '\:boz'
>
>    because the backslash in '\:boz' makes the colon not at the beginning and
>    the glob match sees '\:boz' and then matches '\:' with literal ':' at the
>    beginning of the pathname ":boz".

Easy workaround, maybe, but still a potential problem for unsuspecting
scripts.

  - I think this fails in a directory with :foo.c

    git log master -- *.c


  - Would this work, though?

    git log master -- "*.c"

Phil

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Phil Hord]

On 03/01/2011 07:12 PM, Nguyen Thai Ngoc Duy wrote:
> On Tue, Mar 1, 2011 at 6:16 PM, Michael J Gruber
> <git@drmicha.warpmail.net> wrote:
>> Nguyen Thai Ngoc Duy venit, vidit, dixit 01.03.2011 12:13:
>>> If you are going to turn pathspecs into something more complex,
>>> reserve room for future extension. I have negative pathspecs that can
>>> utilize it.
>>>
>>> I take it, from now on people must refer file name ':foo' as './:foo'
>>> with your patch?
>> That is up for discussion, of course. When discussing a new approach for
>> file mode dependent attributes, I was hoping to get through with
>> symlink:path, and did not. But it was decided that something like
>> :symlink:path would be good enough, in the sense of avoiding enough
>> possible conflicts. That made me hope that :path would be, too.
> Good morning! I'm saner now. How about :/path? That would reserve
> anything next to ':' except '/'.

I like this.  The only 'failure' that comes to mind is something like

     git log -- */*.c

when there's a subdirectory named ':'. 

Phil

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Junio C Hamano]

Miles Bader <miles@gnu.org> writes:

> Michael J Gruber <git@drmicha.warpmail.net> writes:
>>>  Would this ':'
>>> trick affect that argument?  If a command is relative to the cwd with no
>>> pathspec, you can now give a single ':' to affect the whole tree.
>>
>> In my view yes. I would even say: If we don't change every single
>> command to repo-wide default there is no need to change (and break
>> things) if we have an easy one-character way of saying "repo-wide".
>
> ... except, of course that the current state is still confusingly
> inconsistent....

You should know that we are already in violent agreement, if you re-read
my message where I say "a good default matters".

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Michael J Gruber]

Phil Hord venit, vidit, dixit 03.03.2011 04:44:
> On 03/02/2011 11:53 AM, Junio C Hamano wrote:
>> Michael J Gruber <git@drmicha.warpmail.net> writes:
>>> [*] I keep forgetting that some people may have files whose names begin
>>> with ":". They are ambiguous now already with "treeish pathspec"
>>> commands, but not with "pathspec" commands. The latter would change.
>> Just to make sure I understand that they have easy workarounds:
>>
>>  - If you have a path foo/:bar, you can say
>>
>>    git log master -- foo/:bar
>>
>>    because ':' signals the magic and gets stripped only when it is at the
>>    beginning (i.e. not affecting foo/:bar); and
>>
>>  - For :boz at the root level, you can say
>>
>>    git log master -- '\:boz'
>>
>>    because the backslash in '\:boz' makes the colon not at the beginning and
>>    the glob match sees '\:boz' and then matches '\:' with literal ':' at the
>>    beginning of the pathname ":boz".
> 
> Easy workaround, maybe, but still a potential problem for unsuspecting
> scripts.
> 
>   - I think this fails in a directory with :foo.c
> 
>     git log master -- *.c
> 
> 
>   - Would this work, though?
> 
>     git log master -- "*.c"

I hope you are aware that these two are completely different before my
patch already, are you?

The second one will match ":foo.c" and any other .c-file at cwd in any
commit in master (which changes it), of course. No ambiguity here. This
is almost always what you want.

The first one would match ":foo.c" and any other .c file which you
currently have at cwd in your working tree (!), before my patch (unless
you don't have any in your wt), and is almost never what you want.

After my patch, it would interpret the ":foo.c" which the shell glob
expands to differently. That is exactly the ambiguity that I mentioned.

Michael

Re: [PATCH/alternative/raw and rough] setup.c: denote repo wide pathspecs by ':'

[Michael J Gruber]

Phil Hord venit, vidit, dixit 03.03.2011 04:51:
> On 03/01/2011 07:12 PM, Nguyen Thai Ngoc Duy wrote:
>> On Tue, Mar 1, 2011 at 6:16 PM, Michael J Gruber
>> <git@drmicha.warpmail.net> wrote:
>>> Nguyen Thai Ngoc Duy venit, vidit, dixit 01.03.2011 12:13:
>>>> If you are going to turn pathspecs into something more complex,
>>>> reserve room for future extension. I have negative pathspecs that can
>>>> utilize it.
>>>>
>>>> I take it, from now on people must refer file name ':foo' as './:foo'
>>>> with your patch?
>>> That is up for discussion, of course. When discussing a new approach for
>>> file mode dependent attributes, I was hoping to get through with
>>> symlink:path, and did not. But it was decided that something like
>>> :symlink:path would be good enough, in the sense of avoiding enough
>>> possible conflicts. That made me hope that :path would be, too.
>> Good morning! I'm saner now. How about :/path? That would reserve
>> anything next to ':' except '/'.
> 
> I like this.  The only 'failure' that comes to mind is something like
> 
>      git log -- */*.c
> 
> when there's a subdirectory named ':'. 

msysgit anyone?

Michael

[PATCH] pathspec: reserve some letters after a colon pathspec

[Nguyễn Thái Ngọc Duy]

Pathspec ':something' means 'something' at top directory. Limit it a
bit so that ':<non-alnum>something' can be reserved for future
extensions. ':\<non-alnum>something' can be used to achieve
':something' before this patch.

All non-alphanumeric chars on the en_US keyboard, except \ and ., are
currently reserved.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 This is the better, non-whitespace-damaged version. While I mark
 colon_pathspec_type() static, you can export it to use in git-attr.c

 setup.c |   31 +++++++++++++++++++++++++++++--
 1 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/setup.c b/setup.c
index 3bbb01a..684abb5 100644
--- a/setup.c
+++ b/setup.c
@@ -123,6 +123,27 @@ void verify_non_filename(const char *prefix, const char *arg)
 	    "Use '--' to separate filenames from revisions", arg);
 }
 
+static int colon_pathspec_type(const char **pathspec)
+{
+	const char *reserved = "~`!@#$%^&*()-_=+[{]}|;:'\",<>/?";
+	const char *s = *pathspec;
+	int ret;
+
+	if (*s++ != ':')
+		return -1;
+	if (*s == '\\') {
+		s++;
+		ret = 0;
+	}
+	else if (*s && strchr(reserved, *s))
+		ret = -1;
+	else
+		ret = 0;
+
+	*pathspec = s;
+	return ret;
+}
+
 const char **get_pathspec(const char *prefix, const char **pathspec)
 {
 	const char *entry = *pathspec;
@@ -145,8 +166,14 @@ const char **get_pathspec(const char *prefix, const char **pathspec)
 	prefixlen = prefix ? strlen(prefix) : 0;
 	while (*src) {
 		const char *p;
-		if ((*src)[0] == ':')
-			p = prefix_path(NULL, 0, (*src)+1);
+
+		if ((*src)[0] == ':') {
+			const char **s = src;
+			if (colon_pathspec_type(s) != 0)
+				die("Pathspec syntax ':%c' is not supported. %s"
+				    "Quote it for literally match.", (*s)[0], *s);
+			p = prefix_path(NULL, 0, *s);
+		}
 		else
 			p = prefix_path(prefix, prefixlen, *src);
 		*(dst++) = p;
-- 
1.7.4.74.g639db

Re: [PATCH] pathspec: reserve some letters after a colon pathspec

[Junio C Hamano]

Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:

> Pathspec ':something' means 'something' at top directory. Limit it a
> bit so that ':<non-alnum>something' can be reserved for future
> extensions. ':\<non-alnum>something' can be used to achieve
> ':something' before this patch.
>
> All non-alphanumeric chars on the en_US keyboard, except \ and ., are
> currently reserved.

While I was writing the other message, I really was hoping that people
would notice that trying to limit the magic signature (i.e. "which magic I
want" in my previous message) to a non-alnum letter that cannot easily be
remembered would be a bad direction.  A set of short mnemonic is fine, but
we probably should prepare the syntax framework to reserve spelled out
magic names for readability.

Here is a weather-baloon.  I will use colon below as the magic introducer,
as I don't care very deeply about the choice of it.

 - "^:([^\w\d]+)(.*)$", that is "a magic introducer followed by a sequence
   of non-alnum followed by the remainder" means that the part that is
   given to the matching engine is $2, and each gibberish character in $1
   determines what magic is requested when the matching engine does its
   work.  Among the gibberish that can be in $1, we currently would want
   to support:

    . '/' denotes that $2 is relative to root of the working tree, i.e. do
      not add 'prefix' to it at the left.

    . '!' denotes that the matching with $2 should not honor globbing.

   e.g.

    ":/*lib/**/foo.h", if '*' denoted recursive glob support for '**/' to
    mean "zero-or-more levels of any directory" [*1*], it would find any
    foo.h in a directory 'lib' or its subdirectory that is found in
    anywhere in the working tree.

 - "^:((?:[-a-z]+)(?:,[-a-z+]+)*):(.*)$", that is "a magic introducer,
   followed by one or more alpha-string separated with comma, followed
   by a magic terminator, and the remainder" means that the remainder is
   what is given to the matching engine, and the alpha-strings spell out
   the name of the magic.  We currently would want to support:

    . 'full-tree' means exactly the same as '/' mnemonic above.
    . 'noglob' means exactly the same as '!' mnemonic.

   e.g.

   ":full-tree,recursive-glob:lib/**/foo.h" would be how you fully spell
   the above example in the mnemonic section [*2*].


[Footnote]

*1* "man zshexpn" and look for "Recursive Globbing".

*2* It would be "/full-tree,recursive-glob/lib/**/foo.h" if the magic
introducer were '/', which might be easier to the eye.

Re: [PATCH] pathspec: reserve some letters after a colon pathspec

[Michael J Gruber]

Junio C Hamano venit, vidit, dixit 23.03.2011 19:04:
> Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:
> 
>> Pathspec ':something' means 'something' at top directory. Limit it a
>> bit so that ':<non-alnum>something' can be reserved for future
>> extensions. ':\<non-alnum>something' can be used to achieve
>> ':something' before this patch.
>>
>> All non-alphanumeric chars on the en_US keyboard, except \ and ., are
>> currently reserved.
> 
> While I was writing the other message, I really was hoping that people
> would notice that trying to limit the magic signature (i.e. "which magic I
> want" in my previous message) to a non-alnum letter that cannot easily be
> remembered would be a bad direction.  A set of short mnemonic is fine, but
> we probably should prepare the syntax framework to reserve spelled out
> magic names for readability.
> 
> Here is a weather-baloon.  I will use colon below as the magic introducer,
> as I don't care very deeply about the choice of it.
> 
>  - "^:([^\w\d]+)(.*)$", that is "a magic introducer followed by a sequence
>    of non-alnum followed by the remainder" means that the part that is
>    given to the matching engine is $2, and each gibberish character in $1
>    determines what magic is requested when the matching engine does its
>    work.  Among the gibberish that can be in $1, we currently would want
>    to support:
> 
>     . '/' denotes that $2 is relative to root of the working tree, i.e. do
>       not add 'prefix' to it at the left.
> 
>     . '!' denotes that the matching with $2 should not honor globbing.
> 
>    e.g.
> 
>     ":/*lib/**/foo.h", if '*' denoted recursive glob support for '**/' to
>     mean "zero-or-more levels of any directory" [*1*], it would find any
>     foo.h in a directory 'lib' or its subdirectory that is found in
>     anywhere in the working tree.
> 
>  - "^:((?:[-a-z]+)(?:,[-a-z+]+)*):(.*)$", that is "a magic introducer,
>    followed by one or more alpha-string separated with comma, followed
>    by a magic terminator, and the remainder" means that the remainder is
>    what is given to the matching engine, and the alpha-strings spell out
>    the name of the magic.  We currently would want to support:
> 
>     . 'full-tree' means exactly the same as '/' mnemonic above.
>     . 'noglob' means exactly the same as '!' mnemonic.
> 
>    e.g.
> 
>    ":full-tree,recursive-glob:lib/**/foo.h" would be how you fully spell
>    the above example in the mnemonic section [*2*].

I like this a lot, especially the fact that we would have descriptive
long names as well as short versions for a subset! Two remarks:

:(symlink|submodule|directory|file): would fit into that scheme (for use
in .gitattributes), though I'm not sure we want that for general
pathspecs. We probably want textconv applied to :file: only by default,
attributes to match with :file only?

We already have ":./cdwfile" as in "commit:./cwdfile", and this looks
like a preexisting instance, although it is not ("commit:" gets stripped
and "./cwdfile" is the pathspec). People will probably try something
like "commit:/rootfile", and we may or may not want to support this.
That particular one is easy, but "commit:full-tree:name" has a defined
meaning now...

Michael

Re: [PATCH] pathspec: reserve some letters after a colon pathspec

[Nguyen Thai Ngoc Duy]

2011/3/24 Michael J Gruber <git@drmicha.warpmail.net>:
>> Here is a weather-baloon.  I will use colon below as the magic introducer,
>> as I don't care very deeply about the choice of it.
>>
>>  - "^:([^\w\d]+)(.*)$", that is "a magic introducer followed by a sequence
>>    of non-alnum followed by the remainder" means that the part that is
>>    given to the matching engine is $2, and each gibberish character in $1
>>    determines what magic is requested when the matching engine does its
>>    work.  Among the gibberish that can be in $1, we currently would want
>>    to support:
>>
>>     . '/' denotes that $2 is relative to root of the working tree, i.e. do
>>       not add 'prefix' to it at the left.
>>
>>     . '!' denotes that the matching with $2 should not honor globbing.
>>

And maybe:

    . ':' to reach the superproject if user's inside a subproject. So
'::/foo' means foo at superproject while ':/foo' means foo in the
current project, both at root.

>>  ...
>
> I like this a lot, especially the fact that we would have descriptive
> long names as well as short versions for a subset!

I'll leave it to you to come up with something we can test :)

> Two remarks:
>
> :(symlink|submodule|directory|file): would fit into that scheme (for use
> in .gitattributes), though I'm not sure we want that for general
> pathspecs. We probably want textconv applied to :file: only by default,
> attributes to match with :file only?

It does not hurt to have generic support for everything. 'git ls-files
-- :executable:' would be nice, though I'm not sure if I will ever use
it.

> We already have ":./cdwfile" as in "commit:./cwdfile", and this looks
> like a preexisting instance, although it is not ("commit:" gets stripped
> and "./cwdfile" is the pathspec). People will probably try something
> like "commit:/rootfile", and we may or may not want to support this.
> That particular one is easy, but "commit:full-tree:name" has a defined
> meaning now...

I think we should leave this one out. It's to address a single path.
If you bring full pathspec support to it, a pathspec may resolve to
multiple paths, which is unwanted. If people want pathspecs, they can
do "git cmd commit -- pathspecs" most of the time.
-- 
Duy

Re: [PATCH] pathspec: reserve some letters after a colon pathspec

[Junio C Hamano]

Nguyen Thai Ngoc Duy <pclouds@gmail.com> writes:

> And maybe:
>
>     . ':' to reach the superproject if user's inside a subproject. So
> '::/foo' means foo at superproject while ':/foo' means foo in the
> current project, both at root.

A magic with that meaning may be fine (or may be not---I don't care too
much about "because we could" at this point), but if you are going to use
':' as the magic introducer, you cannot use ':' as one of the magic
signatures, as it would make it ambiguous when you said '::'.  Did you
write a long-form with 0 spelled-out magic (perhaps to defeat some other
future settings like --option or config)?  Or did you mean that magic
signature?

Re: [PATCH] pathspec: reserve some letters after a colon pathspec

[Junio C Hamano]

Michael J Gruber <git@drmicha.warpmail.net> writes:

> Junio C Hamano venit, vidit, dixit 23.03.2011 19:04:
> ...
>> Here is a weather-baloon.  I will use colon below as the magic introducer,
>> as I don't care very deeply about the choice of it.
>> 
>>  - "^:([^\w\d]+)(.*)$", that is "a magic introducer followed by a sequence
>>    of non-alnum followed by the remainder" means that the part that is
>>    given to the matching engine is $2, and each gibberish character in $1
>>    determines what magic is requested when the matching engine does its
>>    work.  Among the gibberish that can be in $1, we currently would want
>>    to support:
>> 
>>     . '/' denotes that $2 is relative to root of the working tree, i.e. do
>>       not add 'prefix' to it at the left.
>> 
>>     . '!' denotes that the matching with $2 should not honor globbing.
>> 
>>    e.g.
>> 
>>     ":/*lib/**/foo.h", if '*' denoted recursive glob support for '**/' to
>>     mean "zero-or-more levels of any directory" [*1*], it would find any
>>     foo.h in a directory 'lib' or its subdirectory that is found in
>>     anywhere in the working tree.
>> 
>>  - "^:((?:[-a-z]+)(?:,[-a-z+]+)*):(.*)$", that is "a magic introducer,
>>    followed by one or more alpha-string separated with comma, followed
>>    by a magic terminator, and the remainder" means that the remainder is
>>    what is given to the matching engine, and the alpha-strings spell out
>>    the name of the magic.  We currently would want to support:
>> 
>>     . 'full-tree' means exactly the same as '/' mnemonic above.
>>     . 'noglob' means exactly the same as '!' mnemonic.
>> 
>>    e.g.
>> 
>>    ":full-tree,recursive-glob:lib/**/foo.h" would be how you fully spell
>>    the above example in the mnemonic section [*2*].
>
> I like this a lot, especially the fact that we would have descriptive
> long names as well as short versions for a subset! Two remarks:
>
> :(symlink|submodule|directory|file): would fit into that scheme (for use
> in .gitattributes), though I'm not sure we want that for general
> pathspecs.

I do not offhand think it is a good idea.  While traversing history the
pathspec matcher often does not have the mode information extracted from
the tree object in the codepath it inspects the name, so it would be very
costly, I don't think it would particularly be useful, and pathspec is
about names and not about types.

A magic that says "please match case insensitively", so that we do not
have to write "git log -- '[Rr][Ee][Aa][Dd][Mm][Ee]'" would be very
useful.  Perhaps "gibberish" is not a good long term solution after all,
as a natural short-hand for that magic would be a single letter 'i'
somewhere, similar to (?i) in pcre.