Git pull confusing output

Git pull confusing output

[Will]

I’m far from being a guru, but I consider myself a competent Git user. 
Yet, here’s my understanding of the output of one the most-used 
commands, `git push`:
> Counting objects: 6, done.
No idea what an “object” is. Apparently there’s 6 of them here. 
What does “counting” them means? Should I care?
> Delta compression using up to 4 threads.
No idea what is “delta compression”, I suppose something is being 
compressed. It’s using anything between 1 and 4 threads, which is not 
a very precise or useful information. Should I care?
> Compressing objects: 100% (6/6), done.
I still don’t know what objects are, but I appreciate having feedback 
on progress
> Writing objects: 100% (6/6), 656 bytes | 656.00 KiB/s, done.
Writing what, where? Should I care? Still good to have feedback
> Total 6 (delta 4), reused 0 (delta 0)
No idea what any of those numbers mean. Should I care?
> remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
I do know what’s a remote, but I don’t know what “resolving 
deltas” means. There’s local objects now? I don’t understand what 
happened to those local objects, are they the byproduct of the delta 
resolving or the input or something else? Should I care?
> To github.com:williamdclt/some-repo.git
Fair enough
> 1ca9aaa..4320d30  master -> master
Fair enough


All in all, I didn’t understand most of what I’ve been told, and 
don’t seem to care about it. Don’t take my sassiness for disrespect, 
I really appreciate (and am impressed by) everything that happens here, 
but I feel like a less confusing UI is such a low-hanging fruit. How 
many devs understand what all of this means, 1%-2% if even that? And 
even them, do they need this info every time they push?

I feel like a less intimidating output would help, while showing info 
about objects and deltas with the verbose flag:
> Compressing… done
> Pushing to github.com:williamdclt/some-repo.git… done
> 1ca9aaa..4320d30  master -> master


I’d be more than happy to work on this (`git push` is an example 
amongst so many other), but want the mailing list’s opinion on it. Am 
I wrong in thinking that this output is not something users want, am I 
fighting windmills or maybe just being ignorant?

Re: Git pull confusing output

[Stefan Beller]

On Tue, Nov 27, 2018 at 8:52 AM Will <william.duclot@gmail.com> wrote:

> And even them, do they need this info every time they push?

I agree that we should make the output a bit more user friendly,
which means we'd only want to output relevant data for the user.

The different phases taking each one line takes up precious
screen real estate, so another approach would be delete the line
after one phase is finished, such that you'd only see the currently
active phase (that can be useful for debugging as in "The phase of
'Writing objects' takes very long" -> slow network connection).

> I feel like a less intimidating output would help, while showing info
> about objects and deltas with the verbose flag:

I agree that most information in pushing is not very useful
and could be omitted. This helps in multiple ways:
* it keeps the focus on the actually important information,
   see bf1a11f0a1 (sideband: highlight keywords in remote
   sideband output, 2018-08-07)
* less space in a terminal wasted, such that you can scroll over
   it better

> > Compressing… done

After the push succeeded this information would not be useful
any more, it is only useful during the compression phase
(Does it progress quickly enough? or does it error out?)

Slightly related (but applies mostly to fetch, for which this
discussion can also be had):
When fetching, these informations are generated on the
remote side (as the server needs to create the packfile
according to your local state that you negotiated with the
server), which takes some time. Sending over this
information also keeps the connection alive. This is only
relevant in corner cases depending on the setup of the
hosting provider/repository, but it led to commits such as
https://eclipse.googlesource.com/jgit/jgit/+/a38531b21c7e2b0dc956e0ed1bfc9513f604273c
in the java implementation of Git.

> > Pushing to github.com:williamdclt/some-repo.git… done
> > 1ca9aaa..4320d30  master -> master
>
>
> I’d be more than happy to work on this (`git push` is an example
> amongst so many other), but want the mailing list’s opinion on it. Am
> I wrong in thinking that this output is not something users want, am I
> fighting windmills or maybe just being ignorant?

I think this would be a useful patch, but it could get complicated
quickly: push uses other low level git commands to prepare the
packfile to be sent to the server, currently it only needs to pipe
through the output of the low level command (or even have the
low level command directly write to the terminal).

The output of those low level commands should not be changed
when run on their own, I would assume.

So maybe the best way to dive into understanding what happens
under the hood in git-push is to run

  GIT_TRACE=1 git push ...

and see what child processes are invoked (e.g.
run_command: git pack-objects --all-progress-implied)
and then we'd need to change the output of iff the
specific progress flag is given.

Stefan

Re: Git pull confusing output

[Will]

On 27 Nov 2018, at 19:24, Stefan Beller wrote:

> The different phases taking each one line takes up precious
> screen real estate, so another approach would be delete the line
> after one phase is finished, such that you'd only see the currently
> active phase (that can be useful for debugging as in "The phase of
> 'Writing objects' takes very long" -> slow network connection).

I like this idea

>>> Pushing to github.com:williamdclt/some-repo.git… done
>>> 1ca9aaa..4320d30  master -> master
>>
>>
>> I’d be more than happy to work on this (`git push` is an example
>> amongst so many other), but want the mailing list’s opinion on it. Am
>> I wrong in thinking that this output is not something users want, am I
>> fighting windmills or maybe just being ignorant?
>
> I think this would be a useful patch, but it could get complicated
> quickly: push uses other low level git commands to prepare the
> packfile to be sent to the server, currently it only needs to pipe
> through the output of the low level command (or even have the
> low level command directly write to the terminal).
>
> The output of those low level commands should not be changed
> when run on their own, I would assume.

Agreed. I didn’t expect it to be so subtle, but I’ll look into it
and see if that’s something within my reach.

Re: Git pull confusing output

[Ævar Arnfjörð Bjarmason]

On Tue, Nov 27 2018, Will wrote:

> On 27 Nov 2018, at 19:24, Stefan Beller wrote:
>
>> The different phases taking each one line takes up precious
>> screen real estate, so another approach would be delete the line
>> after one phase is finished, such that you'd only see the currently
>> active phase (that can be useful for debugging as in "The phase of
>> 'Writing objects' takes very long" -> slow network connection).
>
> I like this idea
>
>>>> Pushing to github.com:williamdclt/some-repo.git… done
>>>> 1ca9aaa..4320d30  master -> master
>>>
>>>
>>> I’d be more than happy to work on this (`git push` is an example
>>> amongst so many other), but want the mailing list’s opinion on it. Am
>>> I wrong in thinking that this output is not something users want, am I
>>> fighting windmills or maybe just being ignorant?
>>
>> I think this would be a useful patch, but it could get complicated
>> quickly: push uses other low level git commands to prepare the
>> packfile to be sent to the server, currently it only needs to pipe
>> through the output of the low level command (or even have the
>> low level command directly write to the terminal).
>>
>> The output of those low level commands should not be changed
>> when run on their own, I would assume.
>
> Agreed. I didn’t expect it to be so subtle, but I’ll look into it
> and see if that’s something within my reach.

It's not *quite* the same topic, but a related WIP patch that got
dropped (and it would be great if someone looking at this area could
pick it up) is
https://public-inbox.org/git/20180902085503.GA25391@sigill.intra.peff.net/

I think it's also worth looking into making the progress code able to
emit stuff like:

    Counting / Compressing / Writing objects A% (X_1/Y_2) B% (X_2/Y_2) C% (X_3/Y_3)

That would allow for splitting up some of these cases where our progress
bars are overly verbose across multiple lines without losing info by
completely erasing the line, and would also support other cases where
sometimes we do this stuff concurrently. See e.g. my recent commit-graph
progress patches for something that would benefit from this type of
thing.

It would need a patch to the progress code to make it able to juggle N
number of progress bars with some format to stitch them all together.

Re: Git pull confusing output

[Junio C Hamano]

Will <william.duclot@gmail.com> writes:

> I’m far from being a guru, but I consider myself a competent Git
> user. Yet, here’s my understanding of the output of one the most-used
> commands, `git push`:
>> Counting objects: 6, done.
> No idea what an “object” is. Apparently there’s 6 of them
> here. What does “counting” them means? Should I care?

You vaguely recall that the last time you pushed you saw ~400
objects counted there, so you get the feeling how active you were
since then.

It is up to you if you are interested in such a feel of the level of
activity.  "git fetch" (hence "git pull") would also give you a
similar "feel", e.g. "the last fetch was ~1200 objects and today's
is mere ~200, so it seems it is a relatively slow day".

As to "what is an object?", there are plenty of Git tutorials and
books to learn the basics from.  Again, it is up to you if you care.

>> Delta compression using up to 4 threads.
> No idea what is “delta compression”, I suppose something is being
> compressed. It’s using anything between 1 and 4 threads, which is not
> a very precise or useful information. Should I care?

Likewise.

>> Compressing objects: 100% (6/6), done.
> I still don’t know what objects are, but I appreciate having feedback
> on progress

Exactly.

>> Writing objects: 100% (6/6), 656 bytes | 656.00 KiB/s, done.
> Writing what, where? Should I care? Still good to have feedback

You are pushing the data in commits you wrote, modifications you
made to files, etc., to the other side, so that is what is written
to the other side.  Is there any other thing you might suspect that
is written in this context, to make you think a clarification is
needed in the above message?

>> Total 6 (delta 4), reused 0 (delta 0)
> No idea what any of those numbers mean. Should I care?

It is up to you to get interested in these details and learn what
they mean.  In this case, among these 6 objects transferred, Git
managed to find that 4 are similar to other objects the other side
already has or being sent by this push and can be transferred very
efficiently by sending only the difference, which is what "delta"
means.

>> remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
> I do know what’s a remote, but I don’t know what “resolving
> deltas” means. There’s local objects now? I don’t understand what
> happened to those local objects, are they the byproduct of the delta
> resolving or the input or something else? Should I care?

The "remote:" prefix is "the other side said the following".  IOW,
you are seeing the message from the receiving end.  As you sent 4
objects as mere "difference" (not the whole data needed to know
every byte of the file or directory), the receiving side needed to
find the object the "difference" was relative to, and reassemble
what you would have sent if there weren't delta compression.  These
4 local objects were local from the point of view of the other side,
i.e. the repository that received your push.

The information density of this one is much lower than the previous
progress output lines.  This one is primarily to give you the feeling 
of relative speed (you've seen how fast the "writing" phase which is
constrained mostly by over-the-wire speed already, and now you are
observing how many more seconds are spent to post-process the data
sent over the wire) and avoiding to get you bored.

I think we have "--quiet" option for those who do not care.

Re: Git pull confusing output

[Stefan Beller]

On Tue, Nov 27, 2018 at 10:31 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Will <william.duclot@gmail.com> writes:
>
> > I’m far from being a guru, but I consider myself a competent Git
> > user. Yet, here’s my understanding of the output of one the most-used
> > commands, `git push`:
> >> Counting objects: 6, done.
> > No idea what an “object” is. Apparently there’s 6 of them
> > here. What does “counting” them means? Should I care?
>
> You vaguely recall that the last time you pushed you saw ~400
> objects counted there, so you get the feeling how active you were
> since then.
>
> It is up to you if you are interested in such a feel of the level of
> activity.  "git fetch" (hence "git pull") would also give you a
> similar "feel", e.g. "the last fetch was ~1200 objects and today's
> is mere ~200, so it seems it is a relatively slow day".
>
> As to "what is an object?", there are plenty of Git tutorials and
> books to learn the basics from.  Again, it is up to you if you care.

While this is very interesting to the experienced git user, the
approximation of activity by object count is very coarse to say at least.

As It approximates changes in the DAG object count and nothing
about the deltas (which as we learn comes later and it comes with
a progress meter in bytes), it only provides the basics.

>
> I think we have "--quiet" option for those who do not care.

I think some users are not focused as much on the version control as
much as they are focused on another problem that is solved with
the content inside the repo.

Which means they only care about 'actionable' output, such as
* errors
* information provided by remote
  (e.g. links to click to start a code review)
* too long waiting time
  (so they can abort and inspect the problem)

I would suggest we come up with a mode that is "not quiet", but
cuts down to only the basic actionable items [and make that
the default eventually].

Now these actionable items depend on the workflow used,
for which I think an email based maintainers workflow is not
the norm. The vast majority of people uses git-push to
upload their change to a code review system instead.
And for such a workflow the size (as proxied by
object/delta count) is not as important as the target ref
that you push to or potentially the diffstat output of
a potential merge to a target branch.

TLDR: I still think making git-push a bit more quiet is
beneficial to the user base at large.

Stefan