script for reproducing history example in git-log(1) man page

script for reproducing history example in git-log(1) man page

[Adam Spiers]

I wanted to be able to experiment with the TREESAME example given in
the git-log(1) man page, so I built this script which recreates it:

--------- 8< --------- 8< --------- 8< --------- 8< --------- 8< ---------
#!/bin/bash

mkdir git-log-example
cd git-log-example

git init

# I
echo asdf > foo; echo quux > quux
git add .
git commit -mI; git tag I

# A
echo foo > foo
git add .
git commit -mA; git tag A

# B
git checkout -b b I
echo foo > foo
git add .
git commit -mB; git tag B

# M
git checkout master
git merge --no-commit b
git commit -m"M: merge of A and B"; git tag M

# C
git checkout -b c I
git commit --allow-empty -mC; git tag C

# N
git checkout master
git merge --no-commit c
git commit -m"N: merge of M and C"; git tag N

# D
git checkout -b d I
echo baz > foo
git add .
git commit -mD; git tag D

# O
git checkout master
git merge --no-commit d
echo foobarbaz > foo
git add .
git commit -m"O: merge of N and D"; git tag O

# E
git checkout -b e I
echo xyzzy > quux
git add .
git commit -mE; git tag E

# P
git checkout master
git merge --no-commit e
echo "quux xyzzy" > quux
git add .
git commit -m"P: merge of O and E"; git tag P

# X
git checkout -b x I
rm foo quux; echo side > side
git add -A .
git commit --amend -m"X"; git tag X

# Y
git checkout -b y x
echo side2 > side
git add .
git commit -m"Y"; git tag Y

# Q
git checkout master
git merge --no-commit y
git commit -mQ; git tag Q

# cleanup unneeded branches
git branch -D b c d e x y
--------- 8< --------- 8< --------- 8< --------- 8< --------- 8< ---------

Would it be worth including this in (say) contrib/, and then referring
to it from the man page, in case anyone else feels a similar urge?

questions / suggestions about history simplification

[Adam Spiers]

On Thu, Dec 19, 2013 at 06:36:45PM +0000, Adam Spiers wrote:
> I wanted to be able to experiment with the TREESAME example given in
> the git-log(1) man page, so I built this script which recreates it:

[snipped]

> Would it be worth including this in (say) contrib/, and then referring
> to it from the man page, in case anyone else feels a similar urge?

Hmm, another related option would be to add a new test case which
tests that git log behaves in the way the man page says it does, in
this case.  Although to some extent this would duplicate what
t6012-rev-list-simplify.sh already tests.

I still don't understand a few things about history simplification:

1. The "--full-history without parent rewriting" correctly asserts
   that commit Q will be shown.  But AFAICS this contradicts the
   documented behaviour "Commits are included if they are not TREESAME
   to any parent" which is implied by "This mode differs from the
   default in one point:", because Q is TREESAME to P.

2. What difference does --dense ever make?  In all three of the modes
   described above it ("Default", "--full-history without parent
   rewriting", and "--full-history with parent rewriting"), walked
   commits are already included if they are not TREESAME to any
   parent.

3. Why is --sparse so called, given that it increases rather than
   decreases the number of commits shown?

I have to say I find this section of the man page really quite hard to
grok, partially due to the choice of "TREESAME" word.  I'm guessing
that this was used because it reflects the name of the constant used
in the code, but it does not help legibility of the man page at all.

I think it could help to add descriptions of the behaviour which are
less formal and more intuitive from a pragmatic real world point of
view.  For example:

    "Each commit walked will only be shown in the default output mode
     if it changed the given path(s) relative to *all* its parents.
     When walking the commit graph, if a merge didn't change the given
     path relative to at least one of its parents, then only one of
     those parents would be walked.  This reduces the number of
     commits shown, but pruning commit chains whose changes
     effectively died out during merges."

This sort of text could then be followed by the examples, for those
who want to check they understood it fully.

Hope this feedback is useful,

Adam

Re: questions / suggestions about history simplification

[Jonathan Nieder]

Adam Spiers wrote:

> Hmm, another related option would be to add a new test case which
> tests that git log behaves in the way the man page says it does, in
> this case.

Yes, please!  If you have a rough patch in that direction, that
would be welcome.

Thanks,
Jonathan

Re: questions / suggestions about history simplification

[Junio C Hamano]

Adam Spiers <git@adamspiers.org> writes:

> 2. What difference does --dense ever make?

It is set by default, and --sparse is its opposite option, i.e. it
turns revs->dense off.

When revs->dense is turned off, the usual treesame logic does not
kick in to rewrite parents in a single strand of pearls (i.e. a
stretch of history that solely consists of non-merge commits).
	
> 3. Why is --sparse so called, given that it increases rather than
>    decreases the number of commits shown?

The number of commits in the output will increase by including
commits that are irrelevant to explain the history of paths
specified by pathspec in revs->prune.  The information density
decreases as the result, and that is what "sparse" signifies.

Re: questions / suggestions about history simplification

[Adam Spiers]

On Thu, Dec 19, 2013 at 12:20:52PM -0800, Junio C Hamano wrote:
> Adam Spiers <git@adamspiers.org> writes:
> 
> > 2. What difference does --dense ever make?
> 
> It is set by default, and --sparse is its opposite option, i.e. it
> turns revs->dense off.

Ah.  It appears to be missing from the man page that it's the default.

> When revs->dense is turned off, the usual treesame logic does not
> kick in to rewrite parents in a single strand of pearls (i.e. a
> stretch of history that solely consists of non-merge commits).

I see.

> > 3. Why is --sparse so called, given that it increases rather than
> >    decreases the number of commits shown?
> 
> The number of commits in the output will increase by including
> commits that are irrelevant to explain the history of paths
> specified by pathspec in revs->prune.  The information density
> decreases as the result, and that is what "sparse" signifies.

Ah OK, that makes sense now, but not the most intuitive choice of name
IMHO.  I would have gone for something like --all-commits, but I guess
it's way too late to change now.

Re: questions / suggestions about history simplification

[Junio C Hamano]

Adam Spiers <git@adamspiers.org> writes:

> On Thu, Dec 19, 2013 at 06:36:45PM +0000, Adam Spiers wrote:
>> I wanted to be able to experiment with the TREESAME example given in
>> the git-log(1) man page, so I built this script which recreates it:
>
> [snipped]
>
>> Would it be worth including this in (say) contrib/, and then referring
>> to it from the man page, in case anyone else feels a similar urge?

I doubt it.  75% of the work for such a person to understand the
behaviour from such an example is to understand what kind of history
the example is building.  As you noticed, we do have existing tests
to build "interesting" sample histories, but the fact that you did
not bother with them and instead rolled your own should tell us
something ;-)  The next person is unlikely to read your sample in
contrib/ but will roll his own, which is probably more efficient way
than learning from a series of commands.

What we _could_ do instead may be to better annotate sample
histories in the existing tests.  Some of them (e.g. 6004, 6007) do
have topology illustrations with what paths are changed at each node
in the graph, but many lack such a visual aid to help readers
understand what is going on at a glance.

Re: questions / suggestions about history simplification

[Adam Spiers]

On Thu, Dec 19, 2013 at 11:10:44AM -0800, Jonathan Nieder wrote:
> Adam Spiers wrote:
> 
> > Hmm, another related option would be to add a new test case which
> > tests that git log behaves in the way the man page says it does, in
> > this case.
> 
> Yes, please!  If you have a rough patch in that direction, that
> would be welcome.

Unfortunately I've already exceeded this month's time budget for git
hackery :-/  But it would be a pretty short job for someone else.

Re: questions / suggestions about history simplification

[Junio C Hamano]

Adam Spiers <git@adamspiers.org> writes:

> Ah OK, that makes sense now, but not the most intuitive choice of name
> IMHO.  I would have gone for something like --all-commits, but I guess
> it's way too late to change now.

Besides, it is not --all-commits, is it?  We do cull irrelevant side
branches IIRC.

Re: questions / suggestions about history simplification

[Adam Spiers]

On Thu, Dec 19, 2013 at 12:39:05PM -0800, Junio C Hamano wrote:
> Adam Spiers <git@adamspiers.org> writes:
> 
> > Ah OK, that makes sense now, but not the most intuitive choice of name
> > IMHO.  I would have gone for something like --all-commits, but I guess
> > it's way too late to change now.
> 
> Besides, it is not --all-commits, is it?  We do cull irrelevant side
> branches IIRC.

True, unless (IIUC) --full-history is also specified, in which case no
commit chains are pruned from the walk.

So --all-walked-commits sounds about right.

Re: questions / suggestions about history simplification

[Adam Spiers]

On Thu, Dec 19, 2013 at 12:37:53PM -0800, Junio C Hamano wrote:
> Adam Spiers <git@adamspiers.org> writes:
> 
> > On Thu, Dec 19, 2013 at 06:36:45PM +0000, Adam Spiers wrote:
> >> I wanted to be able to experiment with the TREESAME example given in
> >> the git-log(1) man page, so I built this script which recreates it:
> >
> > [snipped]
> >
> >> Would it be worth including this in (say) contrib/, and then referring
> >> to it from the man page, in case anyone else feels a similar urge?
> 
> I doubt it.  75% of the work for such a person to understand the
> behaviour from such an example is to understand what kind of history
> the example is building.

Agreed.  And that's precisely why I wanted a real repository
manifesting the given example: being able to view it in gitk makes
that a lot easier to understand, for obvious reasons.

> As you noticed, we do have existing tests
> to build "interesting" sample histories, but the fact that you did
> not bother with them and instead rolled your own should tell us
> something ;-)

Well I didn't roll my own; I just copied the example from the man
page.  So it only tells you that I was spending a fair amount of
effort trying to understand the man page ;-)  A user should not have to
read the test suite to understand how the thing works - that's only
for developers (conveniently ignoring for the sake of this argument
that I am occasionally a git developer too ;-)

> The next person is unlikely to read your sample in
> contrib/ but will roll his own,

Not if the man page says "if you wish to experiment with these options
yourself, you can easily recreate the repository in this example by
running the script contrib/foo bundled in the source distribution".

> which is probably more efficient way
> than learning from a series of commands.

The goal of sharing the series of commands is not to educate users
through reading them, but simply to save them the time they would have
to spend manually recreating the example scenario given in the man
page.  After all, the useful information is not how to set up a
repository reflecting the scenario, but rather, how the various
git-log options affect behaviour when run on that repository.

> What we _could_ do instead may be to better annotate sample
> histories in the existing tests.  Some of them (e.g. 6004, 6007) do
> have topology illustrations with what paths are changed at each node
> in the graph, but many lack such a visual aid to help readers
> understand what is going on at a glance.

Again, this has the flaw of requiring non-developer users to read the
test suite.  On most distributions, the test suite code isn't even
installed, so this means they would have to accurately recreate the
source from which their installed binary packages were built.  Surely
that is considerably more effort than most users should reasonably be
expected to spend.  In contrast, it would be trivial to extend
standard distro packages to include a file e.g.

  /usr/share/git-core/examples/git-log-history-simplification.sh

Re: questions / suggestions about history simplification

[Adam Spiers]

On Thu, Dec 19, 2013 at 07:03:33PM +0000, Adam Spiers wrote:
> I still don't understand a few things about history simplification:
> 
> 1. The "--full-history without parent rewriting" correctly asserts
>    that commit Q will be shown.  But AFAICS this contradicts the
>    documented behaviour "Commits are included if they are not TREESAME
>    to any parent" which is implied by "This mode differs from the
>    default in one point:", because Q is TREESAME to P.

I'm *guessing* that this apparent exception to the stated rules could
be because Q is the start of the walk.  Or maybe I'm just missing a
key detail.  I'd love to hear confirmation either way.

[Questions 2 and 3 snipped]

4. --dense (the default behaviour) includes walked commits iff they are not
   TREESAME to any parent.  --sparse includes all walked commits.  But
   how do I get the middle ground, i.e. all walked commits which are
   not TREESAME to at least one parent?

This seems like an obvious use case, but I haven't figured out how to
do it yet :-/

Thanks!  (I probably don't say that often enough, but the sentiment is
always there ;-)

Re: questions / suggestions about history simplification

[Junio C Hamano]

Adam Spiers <git@adamspiers.org> writes:

>> I doubt it.  75% of the work for such a person to understand the
>> behaviour from such an example is to understand what kind of history
>> the example is building.
>
> Agreed.  And that's precisely why I wanted a real repository
> manifesting the given example: being able to view it in gitk makes
> that a lot easier to understand, for obvious reasons.
> ...
> Well I didn't roll my own; I just copied the example from the man
> page.  So it only tells you that I was spending a fair amount of
> effort trying to understand the man page ;-)

Oh, that part I would agree, but then ...

> Not if the man page says "if you wish to experiment with these options
> yourself, you can easily recreate the repository in this example by
> running the script contrib/foo bundled in the source distribution".
> ...
> The goal of sharing the series of commands is not to educate users
> through reading them, but simply to save them the time they would have
> to spend manually recreating the example scenario given in the man
> page.

... this part could be even easier if distro ships a sample repository,
not a recipe to create such a sample repository, no?

Re: questions / suggestions about history simplification

[Adam Spiers]

On Sat, Dec 21, 2013 at 10:44:43PM -0800, Junio C Hamano wrote:
> Adam Spiers <git@adamspiers.org> writes:
> 
> >> I doubt it.  75% of the work for such a person to understand the
> >> behaviour from such an example is to understand what kind of history
> >> the example is building.
> >
> > Agreed.  And that's precisely why I wanted a real repository
> > manifesting the given example: being able to view it in gitk makes
> > that a lot easier to understand, for obvious reasons.
> > ...
> > Well I didn't roll my own; I just copied the example from the man
> > page.  So it only tells you that I was spending a fair amount of
> > effort trying to understand the man page ;-)
> 
> Oh, that part I would agree, but then ...
> 
> > Not if the man page says "if you wish to experiment with these options
> > yourself, you can easily recreate the repository in this example by
> > running the script contrib/foo bundled in the source distribution".
> > ...
> > The goal of sharing the series of commands is not to educate users
> > through reading them, but simply to save them the time they would have
> > to spend manually recreating the example scenario given in the man
> > page.
> 
> ... this part could be even easier if distro ships a sample repository,
> not a recipe to create such a sample repository, no?

It could ship either or both, but shipping a repository only saves the
user from having to run a single command to create the repository from
the script, and even that advantage is negated if the user wishes the
repository to be read/write (since then they would need "cp -a ...").

However, the question of how the distro should ship it is separate to
the question of how the git source repository and release tarballs
should provide it.  Including a script (e.g. in contrib/) means that
any changes to the man page can trivially be mirrored in that script
within a single commit, giving all the normal advantages git offers
for tracking source.  And then the distro can easily create the sample
repository from the script at package build-time, if they want to.

OTOH, including a sample repository embedded within the git repository
is either impossible or very ugly (e.g. having a non-default value of
GIT_DIR for the embedded repository).  But I doubt you were suggesting
that ;-)

Re: questions / suggestions about history simplification

[Junio C Hamano]

Adam Spiers <git@adamspiers.org> writes:

> OTOH, including a sample repository embedded within the git repository
> is either impossible or very ugly (e.g. having a non-default value of
> GIT_DIR for the embedded repository).  But I doubt you were suggesting
> that ;-)

No.

By the way, t/t1200-tutorial.sh was meant to follow what the
tutorial manual tells the reader to try. We may want to update
and/or enhance it to cover more materials.

Thanks.