let git grep consider sub projects

let git grep consider sub projects

[Olaf Hering]

How can I teach 'git grep $string' to also consider subprojects?

The build system of xen.git clones 4 other trees into a directory in its
source tree during build. It would be nice if my 'git grep' searches
also in these cloned subdirs. Somehow there must be some knowledge
because 'git clean -dfx' leaves them alone, while 'git clean -dffx'
wipes everything.

Olaf

PS: Sometimes I miss a 'git clean -dfx --also-sub-repos' useful to
really clean everything before starting over.

Re: let git grep consider sub projects

[Junio C Hamano]

Olaf Hering <olaf@aepfle.de> writes:

> How can I teach 'git grep $string' to also consider subprojects?
>
> The build system of xen.git clones 4 other trees into a directory in its
> source tree during build. It would be nice if my 'git grep' searches
> also in these cloned subdirs. Somehow there must be some knowledge
> because 'git clean -dfx' leaves them alone, while 'git clean -dffx'
> wipes everything.
>
> Olaf
>
> PS: Sometimes I miss a 'git clean -dfx --also-sub-repos' useful to
> really clean everything before starting over.

Is "submodule foreach" under-advertised or with less than adequate
features?

Re: let git grep consider sub projects

[Mikael Magnusson]

On 7 October 2014 20:25, Junio C Hamano <gitster@pobox.com> wrote:
> Olaf Hering <olaf@aepfle.de> writes:
>
>> How can I teach 'git grep $string' to also consider subprojects?
>>
>> The build system of xen.git clones 4 other trees into a directory in its
>> source tree during build. It would be nice if my 'git grep' searches
>> also in these cloned subdirs. Somehow there must be some knowledge
>> because 'git clean -dfx' leaves them alone, while 'git clean -dffx'
>> wipes everything.
>>
>> Olaf
>>
>> PS: Sometimes I miss a 'git clean -dfx --also-sub-repos' useful to
>> really clean everything before starting over.
>
> Is "submodule foreach" under-advertised or with less than adequate
> features?

It sounds like in these use cases, you would want the commands to run
on all the submodules but also in the parent repo, am I wrong in
thinking that git submodule foreach does only the former part? So you
would either need to make a wrapper thing yourself or run the command
twice.

In the first case with the git grep, I can also imagine that with some
nontrivial patterns, having to quote the metacharacters not only once,
but twice, can be a significant annoyance. Eg, first protect it from
git submodule foreach parsing it, and then from the shell running the
individual commands.

-- 
Mikael Magnusson

Re: let git grep consider sub projects

[Jens Lehmann]

Am 07.10.2014 um 21:12 schrieb Mikael Magnusson:
> On 7 October 2014 20:25, Junio C Hamano <gitster@pobox.com> wrote:
>> Olaf Hering <olaf@aepfle.de> writes:
>>
>>> How can I teach 'git grep $string' to also consider subprojects?
>>>
>>> The build system of xen.git clones 4 other trees into a directory in its
>>> source tree during build. It would be nice if my 'git grep' searches
>>> also in these cloned subdirs. Somehow there must be some knowledge
>>> because 'git clean -dfx' leaves them alone, while 'git clean -dffx'
>>> wipes everything.
>>>
>>> Olaf
>>>
>>> PS: Sometimes I miss a 'git clean -dfx --also-sub-repos' useful to
>>> really clean everything before starting over.
>>
>> Is "submodule foreach" under-advertised or with less than adequate
>> features?
>
> It sounds like in these use cases, you would want the commands to run
> on all the submodules but also in the parent repo, am I wrong in
> thinking that git submodule foreach does only the former part? So you
> would either need to make a wrapper thing yourself or run the command
> twice.

Yep, the workaround for a command "git foo" that hasn't learned the
--recurse-submodules option yet is

    git foo && git submodule foreach --recursive git foo

> In the first case with the git grep, I can also imagine that with some
> nontrivial patterns, having to quote the metacharacters not only once,
> but twice, can be a significant annoyance. Eg, first protect it from
> git submodule foreach parsing it, and then from the shell running the
> individual commands.

The long term goal is to teach git grep and git clean - and others -
the --recurse-submodules option. I'd be happy to help if someone
wants to work on that, as I'm busy working on another topic.

Re: let git grep consider sub projects

[Olaf Hering]

On Wed, Oct 08, Jens Lehmann wrote:

>    git foo && git submodule foreach --recursive git foo

Looks like a submodule is yet another thing, or I have to learn what a
submodule is:

olaf@bax:~/xen $ find . -name .git
./.git
./tools/qemu-xen-dir-remote/.git
./tools/qemu-xen-traditional-dir-remote/.git
./tools/firmware/seabios-dir-remote/.git
olaf@bax:~/xen $ git submodule foreach --recursive git show
olaf@bax:~/xen $

xen does essentially 'git clone $url1 dir/url1-dir;git clone $url2
dir/url2-dir'. So the top level .git does not really know about
dir/*-dir.

Olaf

Re: let git grep consider sub projects

[Jens Lehmann]

Am 08.10.2014 um 08:59 schrieb Olaf Hering:
> On Wed, Oct 08, Jens Lehmann wrote:
>
>>     git foo && git submodule foreach --recursive git foo
>
> Looks like a submodule is yet another thing, or I have to learn what a
> submodule is:
>
> olaf@bax:~/xen $ find . -name .git
> ./.git
> ./tools/qemu-xen-dir-remote/.git
> ./tools/qemu-xen-traditional-dir-remote/.git
> ./tools/firmware/seabios-dir-remote/.git
> olaf@bax:~/xen $ git submodule foreach --recursive git show
> olaf@bax:~/xen $
>
> xen does essentially 'git clone $url1 dir/url1-dir;git clone $url2
> dir/url2-dir'. So the top level .git does not really know about
> dir/*-dir.

Okay, I just checked that xen doesn't use submodules but uses simple
embedded repos ignored by the .gitignore file. From a quick glance
it looks like handling the sub projects is scripted in the Makefiles.
This is perfectly fine, but then you can't really expect git commands
to know about these sub projects. Or am I missing something?

Re: let git grep consider sub projects

[Olaf Hering]

On Wed, Oct 08, Jens Lehmann wrote:

> Okay, I just checked that xen doesn't use submodules but uses simple
> embedded repos ignored by the .gitignore file. From a quick glance
> it looks like handling the sub projects is scripted in the Makefiles.
> This is perfectly fine, but then you can't really expect git commands
> to know about these sub projects. Or am I missing something?

I just did not know about the submodule command. And I did not realize
that they are in .gitignore, which should have been obvious.

I will check if using submodule instead of plain clone can be done,
without breaking anyones workflow.


Olaf