git diff --submodule=diff and --stat/--dirstat/--name-only

git diff --submodule=diff and --stat/--dirstat/--name-only

[Marc Sune]

Hello all,

First, thank you to the community for the great work. Worth saying it
from time to time, I think.

I am dealing with a couple of big repositories that use git
submodules. They have nested submodules, and some of them are pretty
huge. I came across:

git diff --submodule=diff

Which is very handy for creating some tooling, but it's obviously slow
- in the order of tens of minutes in my case - for big diffs. I was
only interested in the list files that changed, in this particular
case, but:

git diff --stat --submodule=diff

doesn't seem to honour `--submodule=diff` and it doesn't go into the
submodule(s) nor recurses, of course. Other options like `--dirstat`
or `--name-only` seem to behave the same way.

I've tried this v2.20.1 and the HEAD of master (d4a392452e) with the
same results. Is this a missing feature, a bug or is it just the
intended behaviour?

Regards
marc

Re: git diff --submodule=diff and --stat/--dirstat/--name-only

[Philippe Blain]

Hi Marc,

> Le 14 oct. 2020 à 06:14, Marc Sune <marcdevel@gmail.com> a écrit :
> 
> Hello all,
> 
> First, thank you to the community for the great work. Worth saying it
> from time to time, I think.
> 
> I am dealing with a couple of big repositories that use git
> submodules. They have nested submodules, and some of them are pretty
> huge. I came across:
> 
> git diff --submodule=diff
> 
> Which is very handy for creating some tooling, but it's obviously slow
> - in the order of tens of minutes in my case - for big diffs. I was
> only interested in the list files that changed, in this particular
> case, but:
> 
> git diff --stat --submodule=diff
> 
> doesn't seem to honour `--submodule=diff` and it doesn't go into the
> submodule(s) nor recurses, of course. Other options like `--dirstat`
> or `--name-only` seem to behave the same way.
> 
> I've tried this v2.20.1 and the HEAD of master (d4a392452e) with the
> same results. Is this a missing feature, a bug or is it just the
> intended behaviour?
> 
> Regards
> marc

This would indeed be useful. It's a missing feature, and so intended behaviour
for the moment, I would say. It was discussed recently on the list [1] :


[1] https://lore.kernel.org/git/20200924063829.GA1851751@coredump.intra.peff.net/t/#u

Cheers,

Philippe.

Re: git diff --submodule=diff and --stat/--dirstat/--name-only

[Marc Sune]

Philippe,

Missatge de Philippe Blain <levraiphilippeblain@gmail.com> del dia
dc., 14 d’oct. 2020 a les 15:29:
>
> Hi Marc,
>
> > Le 14 oct. 2020 à 06:14, Marc Sune <marcdevel@gmail.com> a écrit :
> >
> > Hello all,
> >
> > First, thank you to the community for the great work. Worth saying it
> > from time to time, I think.
> >
> > I am dealing with a couple of big repositories that use git
> > submodules. They have nested submodules, and some of them are pretty
> > huge. I came across:
> >
> > git diff --submodule=diff
> >
> > Which is very handy for creating some tooling, but it's obviously slow
> > - in the order of tens of minutes in my case - for big diffs. I was
> > only interested in the list files that changed, in this particular
> > case, but:
> >
> > git diff --stat --submodule=diff
> >
> > doesn't seem to honour `--submodule=diff` and it doesn't go into the
> > submodule(s) nor recurses, of course. Other options like `--dirstat`
> > or `--name-only` seem to behave the same way.
> >
> > I've tried this v2.20.1 and the HEAD of master (d4a392452e) with the
> > same results. Is this a missing feature, a bug or is it just the
> > intended behaviour?
> >
> > Regards
> > marc
>
> This would indeed be useful. It's a missing feature, and so intended behaviour
> for the moment, I would say. It was discussed recently on the list [1] :
>
>
> [1] https://lore.kernel.org/git/20200924063829.GA1851751@coredump.intra.peff.net/t/#u

Thank you for the pointer and the clarification.

I am not sure extending `git diff` options (only) under
`--submodule=`, e.g. `--submodule=stat` is the way to go. It seems
redundant to me.

I am wondering if it would make sense to have basic options of git
commands, e.g. `git diff --stat`, be honoured for submodules too, and
just control whether git commands should get in the submodule(s) or
not, and perhaps the depth of the recursion (optionally).

For instance, I don't fully get the use-case(s) in which parts of the
output are --stat (supermodule) and submodules show something
different:

git diff --stat --submodule=diff
git diff --stat --submodule=log

Specially for log, this sounds to me like it should be under `git
log`, with the appropriate options. Perhaps you can shed some light on
the use-cases these combinations support.

I would think something like:

git diff --stat --submodule-follow
git diff --stat --submodule-follow-depth=4

git diff --names-only --submodule-follow
git diff --dirstat --submodule-follow

and for other commands (some sort of header line should be printed to
note the history is under the submodule X):

git log --submodule-follow
git log --submodule-follow-depth=4

and of course allowing `--` modifier:

git log --submodule-follow -- libs/library1

would be easier to use.

Let me know if any of this resonates, and if some patches along these
lines would be welcomed (might ask some help offlist).

Regards
marc

>
> Cheers,
>
> Philippe.
>

Re: git diff --submodule=diff and --stat/--dirstat/--name-only

[Philippe Blain]

Hi Marc,

> Le 14 oct. 2020 à 10:35, Marc Sune <marcdevel@gmail.com> a écrit :
> 
> Philippe,
> 
> Missatge de Philippe Blain <levraiphilippeblain@gmail.com> del dia
> dc., 14 d’oct. 2020 a les 15:29:
>> 
>> Hi Marc,
>> 
>>> Le 14 oct. 2020 à 06:14, Marc Sune <marcdevel@gmail.com> a écrit :
>>> 
>>> Hello all,
>>> 
>>> First, thank you to the community for the great work. Worth saying it
>>> from time to time, I think.
>>> 
>>> I am dealing with a couple of big repositories that use git
>>> submodules. They have nested submodules, and some of them are pretty
>>> huge. I came across:
>>> 
>>> git diff --submodule=diff
>>> 
>>> Which is very handy for creating some tooling, but it's obviously slow
>>> - in the order of tens of minutes in my case - for big diffs. I was
>>> only interested in the list files that changed, in this particular
>>> case, but:
>>> 
>>> git diff --stat --submodule=diff
>>> 
>>> doesn't seem to honour `--submodule=diff` and it doesn't go into the
>>> submodule(s) nor recurses, of course. Other options like `--dirstat`
>>> or `--name-only` seem to behave the same way.
>>> 
>>> I've tried this v2.20.1 and the HEAD of master (d4a392452e) with the
>>> same results. Is this a missing feature, a bug or is it just the
>>> intended behaviour?
>>> 
>>> Regards
>>> marc
>> 
>> This would indeed be useful. It's a missing feature, and so intended behaviour
>> for the moment, I would say. It was discussed recently on the list [1] :
>> 
>> 
>> [1] https://lore.kernel.org/git/20200924063829.GA1851751@coredump.intra.peff.net/t/#u
> 
> Thank you for the pointer and the clarification.
> 
> I am not sure extending `git diff` options (only) under
> `--submodule=`, e.g. `--submodule=stat` is the way to go. It seems
> redundant to me.
> 
> I am wondering if it would make sense to have basic options of git
> commands, e.g. `git diff --stat`, be honoured for submodules too, and
> just control whether git commands should get in the submodule(s) or
> not, and perhaps the depth of the recursion (optionally).

Did you try `git submodule foreach` ? If you are scripting, you could do:

    git diff --stat && git submodule foreach [--recursive] git diff --stat

Although here the recursion is an all-or-nothing business.

> For instance, I don't fully get the use-case(s) in which parts of the
> output are --stat (supermodule) and submodules show something
> different:
> 
> git diff --stat --submodule=diff
> git diff --stat --submodule=log
> 
> Specially for log, this sounds to me like it should be under `git
> log`, with the appropriate options.

I think 'log' is useful; it's more descriptive then having just a hash,
which is the default...

> Perhaps you can shed some light on
> the use-cases these combinations support.
> 
> I would think something like:
> 
> git diff --stat --submodule-follow
> git diff --stat --submodule-follow-depth=4
> 
> git diff --names-only --submodule-follow
> git diff --dirstat --submodule-follow
> and for other commands (some sort of header line should be printed to
> note the history is under the submodule X):
> 
> git log --submodule-follow
> git log --submodule-follow-depth=4
> 
> and of course allowing `--` modifier:
> 
> git log --submodule-follow -- libs/library1
> 
> would be easier to use.

I'm not sure of the use case for these... The history
of the submodule project can be seen with `git -C path/to/submodule log` 
(or just `cd` there and `git log`), and the history of the gitlink in the superproject
with `git log -- path/to/submodule`... 

It does not really make sense to interleave the history of the superproject and
the submodule in the same output, in my opinion at least.

> Let me know if any of this resonates, and if some patches along these
> lines would be welcomed

I can't really answer that, I'm just a small time contributor :)

> (might ask some help offlist).

You should ask on the list, on the mentoring list, or on IRC (see [1]).

[1] https://git-scm.com/docs/MyFirstContribution#getting-help

Cheers,
Philippe.

Re: git diff --submodule=diff and --stat/--dirstat/--name-only

[Marc Sune]

Philippe,

Missatge de Philippe Blain <levraiphilippeblain@gmail.com> del dia
dj., 15 d’oct. 2020 a les 0:51:
>
> Hi Marc,
>
> > Le 14 oct. 2020 à 10:35, Marc Sune <marcdevel@gmail.com> a écrit :
> >
> > Philippe,
> >
> > Missatge de Philippe Blain <levraiphilippeblain@gmail.com> del dia
> > dc., 14 d’oct. 2020 a les 15:29:
> >>
> >> Hi Marc,
> >>
> >>> Le 14 oct. 2020 à 06:14, Marc Sune <marcdevel@gmail.com> a écrit :
> >>>
> >>> Hello all,
> >>>
> >>> First, thank you to the community for the great work. Worth saying it
> >>> from time to time, I think.
> >>>
> >>> I am dealing with a couple of big repositories that use git
> >>> submodules. They have nested submodules, and some of them are pretty
> >>> huge. I came across:
> >>>
> >>> git diff --submodule=diff
> >>>
> >>> Which is very handy for creating some tooling, but it's obviously slow
> >>> - in the order of tens of minutes in my case - for big diffs. I was
> >>> only interested in the list files that changed, in this particular
> >>> case, but:
> >>>
> >>> git diff --stat --submodule=diff
> >>>
> >>> doesn't seem to honour `--submodule=diff` and it doesn't go into the
> >>> submodule(s) nor recurses, of course. Other options like `--dirstat`
> >>> or `--name-only` seem to behave the same way.
> >>>
> >>> I've tried this v2.20.1 and the HEAD of master (d4a392452e) with the
> >>> same results. Is this a missing feature, a bug or is it just the
> >>> intended behaviour?
> >>>
> >>> Regards
> >>> marc
> >>
> >> This would indeed be useful. It's a missing feature, and so intended behaviour
> >> for the moment, I would say. It was discussed recently on the list [1] :
> >>
> >>
> >> [1] https://lore.kernel.org/git/20200924063829.GA1851751@coredump.intra.peff.net/t/#u
> >
> > Thank you for the pointer and the clarification.
> >
> > I am not sure extending `git diff` options (only) under
> > `--submodule=`, e.g. `--submodule=stat` is the way to go. It seems
> > redundant to me.
> >
> > I am wondering if it would make sense to have basic options of git
> > commands, e.g. `git diff --stat`, be honoured for submodules too, and
> > just control whether git commands should get in the submodule(s) or
> > not, and perhaps the depth of the recursion (optionally).
>
> Did you try `git submodule foreach` ? If you are scripting, you could do:
>
>     git diff --stat && git submodule foreach [--recursive] git diff --stat

Yes, ofc. There are multiple reasons for not using it in this specific
case I am facing (specially performance), but this is probably
offtopic.

>
> Although here the recursion is an all-or-nothing business.
>
> > For instance, I don't fully get the use-case(s) in which parts of the
> > output are --stat (supermodule) and submodules show something
> > different:
> >
> > git diff --stat --submodule=diff
> > git diff --stat --submodule=log
> >
> > Specially for log, this sounds to me like it should be under `git
> > log`, with the appropriate options.
>
> I think 'log' is useful; it's more descriptive then having just a hash,
> which is the default...
>
> > Perhaps you can shed some light on
> > the use-cases these combinations support.
> >
> > I would think something like:
> >
> > git diff --stat --submodule-follow
> > git diff --stat --submodule-follow-depth=4
> >
> > git diff --names-only --submodule-follow
> > git diff --dirstat --submodule-follow
> > and for other commands (some sort of header line should be printed to
> > note the history is under the submodule X):
> >
> > git log --submodule-follow
> > git log --submodule-follow-depth=4
> >
> > and of course allowing `--` modifier:
> >
> > git log --submodule-follow -- libs/library1
> >
> > would be easier to use.
>
> I'm not sure of the use case for these... The history
> of the submodule project can be seen with `git -C path/to/submodule log`
> (or just `cd` there and `git log`), and the history of the gitlink in the superproject
> with `git log -- path/to/submodule`...
>
> It does not really make sense to interleave the history of the superproject and
> the submodule in the same output, in my opinion at least.
>
> > Let me know if any of this resonates, and if some patches along these
> > lines would be welcomed
>
> I can't really answer that, I'm just a small time contributor :)

I will wait for some thoughts on the proposal, and to see if someone
can shed some light on the reasons why `--submodule=` option works the
way it does, today, in git diff.

Regards
marc

>
> > (might ask some help offlist).
>
> You should ask on the list, on the mentoring list, or on IRC (see [1]).
>
> [1] https://git-scm.com/docs/MyFirstContribution#getting-help
>
> Cheers,
> Philippe.
>