[PATCH 0/2] Documentation/howto/maintain-git.txt: a pair of bugfixes

[PATCH 0/2] Documentation/howto/maintain-git.txt: a pair of bugfixes

[Taylor Blau]

The documentation on how to use the scripts in 'todo' has a couple of
small typos that make it confusing when generating the Meta/redo-jch.sh
and Meta/redo-seen.sh scripts the first time.

Correct these to avoid any confusion in the future.

Note that this branch is based off of 'tb/howto-using-redo-script'.

Thanks in advance for your review.

Taylor Blau (2):
  Documentation: build redo-jch.sh from master..jch
  Documentation: build redo-seen.sh from jch..seen

 Documentation/howto/maintain-git.txt | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

-- 
2.38.0.16.g393fd4c6db

[PATCH 1/2] Documentation: build redo-jch.sh from master..jch

[Taylor Blau]

Rebuilding the 'jch' branch begins by reintegrating any topics between
'master' and 'jch', not 'master' and 'seen'.

In the maintainer guide, the documentation isn't quite right, since the
initial input to Meta/Reintegrate is "master..seen", not "master..jch".
This can lead to confusing results when generating the Meta/redo-jch.sh
script for the first time.

Additionally, rebuilding 'jch' takes place in two steps. First, running
the script up to the first "### match next" cut-line, and then comparing
the result with what's on 'next' (i.e. with "git diff jch next"). Then,
the remaining set of topics get merged down to 'jch' (which aren't on
'next') by running the entire "redo-jch.sh" script.

Clarify the documentation to reflect this.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
 Documentation/howto/maintain-git.txt | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/Documentation/howto/maintain-git.txt b/Documentation/howto/maintain-git.txt
index 215e2edb0f..bac6429fbc 100644
--- a/Documentation/howto/maintain-git.txt
+++ b/Documentation/howto/maintain-git.txt
@@ -231,7 +231,7 @@ by doing the following:
  - Prepare 'jch' branch, which is used to represent somewhere
    between 'master' and 'seen' and often is slightly ahead of 'next'.
 
-     $ Meta/Reintegrate master..seen >Meta/redo-jch.sh
+     $ Meta/Reintegrate master..jch >Meta/redo-jch.sh
 
    The result is a script that lists topics to be merged in order to
    rebuild 'seen' as the input to Meta/Reintegrate script.  Remove
@@ -283,6 +283,11 @@ by doing the following:
 
      $ git diff jch next
 
+   Then build the rest of 'jch':
+
+     $ git checkout jch
+     $ sh Meta/redo-jch.sh
+
    When all is well, clean up the redo-jch.sh script with
 
      $ sh Meta/redo-jch.sh -u
-- 
2.38.0.16.g393fd4c6db

[PATCH 2/2] Documentation: build redo-seen.sh from jch..seen

[Taylor Blau]

In a similar spirit as the previous commit, the 'seen' branch gets
rebuilt by reintegrating topics between 'jch' and the (old) tip of
'seen'.

Update the instructions on how to generate Meta/redo-seen.sh for the
first time to reflect this.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
 Documentation/howto/maintain-git.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/howto/maintain-git.txt b/Documentation/howto/maintain-git.txt
index bac6429fbc..d07c6d44e5 100644
--- a/Documentation/howto/maintain-git.txt
+++ b/Documentation/howto/maintain-git.txt
@@ -298,7 +298,7 @@ by doing the following:
 
  - Rebuild 'seen'.
 
-     $ Meta/Reintegrate master..seen >Meta/redo-seen.sh
+     $ Meta/Reintegrate jch..seen >Meta/redo-seen.sh
 
    Edit the result by adding new topics that are not still in 'seen'
    in the script.  Then
-- 
2.38.0.16.g393fd4c6db

Re: [PATCH 0/2] Documentation/howto/maintain-git.txt: a pair of bugfixes

[Jeff King]

On Mon, Oct 31, 2022 at 07:47:33PM -0400, Taylor Blau wrote:

> The documentation on how to use the scripts in 'todo' has a couple of
> small typos that make it confusing when generating the Meta/redo-jch.sh
> and Meta/redo-seen.sh scripts the first time.
> 
> Correct these to avoid any confusion in the future.

Thanks for writing these up. A little back-story: Taylor and I had
looked at this together off-list, because the results of following the
documentation to the letter made little sense.

So both look good to me, with the giant grain of salt that I was party
to the original head-scratching that led to the patches. Given that
there's no urgency (unless you are hit by a bus in the next few weeks),
it probably makes sense to queue these fixes and wait for Junio to
review and (hopefully) merge them.

There is one thing we wondered about, and I think only Junio can answer.
The instructions after these patches yield two files, redo-jch.sh and
redo-seen.sh. The first one has two sections separated by a "###"
marker: the ones that are in 'next', and the ones that are in 'jch'.
The third ones has topics that are only in 'seen' (which also contains
all of the others).

Why not have a single file with all of the topics, with two "###"
markers? I.e., something like:

  # build the topic list from what's in 'seen' now
  Meta/Reintegrate master..seen >Meta/redo.sh

  # rebuild the first part of jch, which is everything in next
  git checkout -B master jch
  Meta/redo.sh -c1

  # now build actual next, which should end up with the same tree
  git checkout next
  git merge master ;# if necessary
  Meta/redo.sh -c1 -e ;# -e to annotate any topics
  git diff jch next ;# should be identical

  # now build the rest of jch
  git checkout jch
  Meta/redo.sh -c2

  # now build seen
  git checkout -B seen master
  Meta/redo.sh ;# no -c, do it all

I _think_ the two are equivalent, and it seems more convenient to keep
all of the topic names in one file (since they'd migrate across the cut
to go from seen to jch, then jch to next). But maybe you (Junio) find it
more convenient to keep them separate. The 'seen' list is a bit more
fluid, and I'm sure you figured out how to copy lines between two
different buffers in emacs. :)

Mostly just curious.

-Peff

Re: [PATCH 0/2] Documentation/howto/maintain-git.txt: a pair of bugfixes

[Taylor Blau]

On Tue, Nov 01, 2022 at 04:01:12AM -0400, Jeff King wrote:
> On Mon, Oct 31, 2022 at 07:47:33PM -0400, Taylor Blau wrote:
>
> > The documentation on how to use the scripts in 'todo' has a couple of
> > small typos that make it confusing when generating the Meta/redo-jch.sh
> > and Meta/redo-seen.sh scripts the first time.
> >
> > Correct these to avoid any confusion in the future.
>
> Thanks for writing these up. A little back-story: Taylor and I had
> looked at this together off-list, because the results of following the
> documentation to the letter made little sense.
>
> So both look good to me, with the giant grain of salt that I was party
> to the original head-scratching that led to the patches. Given that
> there's no urgency (unless you are hit by a bus in the next few weeks),
> it probably makes sense to queue these fixes and wait for Junio to
> review and (hopefully) merge them.

All true :-).

And indeed, if I'm hit by a bus in the next couple of weeks, I think
that whoever steps up will be clever enough to find this thread on the
list and work it out from there.

Let's hope that we don't have to go there. I'm mainly interested in
Junio's review, anyway, so I'll happily let this one sit until we hear
from him.

> There is one thing we wondered about, and I think only Junio can answer.
> The instructions after these patches yield two files, redo-jch.sh and
> redo-seen.sh. The first one has two sections separated by a "###"
> marker: the ones that are in 'next', and the ones that are in 'jch'.
> The third ones has topics that are only in 'seen' (which also contains
> all of the others).
>
> Why not have a single file with all of the topics, with two "###"
> markers? I.e., something like:

Thanks for writing our confusion up. I, too, think that this would be an
easier process to follow than juggling the topics between two files. I
could imagine that there was some circumstance that bit Junio in the
past (for which it really *is* easier to manage the set of topics broken
out across two files).

I could equally imagine the answer being, "it's always been this way",
and so the process is baked into his muscle memory. Whatever the reason
is, I'm definitely curious in the rationale behind it.

>   # build the topic list from what's in 'seen' now
>   Meta/Reintegrate master..seen >Meta/redo.sh
>
>   # rebuild the first part of jch, which is everything in next
>   git checkout -B master jch
>   Meta/redo.sh -c1
>
>   # now build actual next, which should end up with the same tree
>   git checkout next
>   git merge master ;# if necessary
>   Meta/redo.sh -c1 -e ;# -e to annotate any topics
>   git diff jch next ;# should be identical
>
>   # now build the rest of jch
>   git checkout jch
>   Meta/redo.sh -c2
>
>   # now build seen
>   git checkout -B seen master
>   Meta/redo.sh ;# no -c, do it all
>
> I _think_ the two are equivalent, and it seems more convenient to keep
> all of the topic names in one file (since they'd migrate across the cut
> to go from seen to jch, then jch to next). But maybe you (Junio) find it
> more convenient to keep them separate. The 'seen' list is a bit more
> fluid, and I'm sure you figured out how to copy lines between two
> different buffers in emacs. :)

Based on my own understanding of the scripts in 'todo', I agree that
they should be equivalent.

Thanks,
Taylor

Re: [PATCH 0/2] Documentation/howto/maintain-git.txt: a pair of bugfixes

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Why not have a single file with all of the topics, with two "###"
> markers?

Mostly because jch and seen are built in two distinct steps, and I
need to be able to rearrange lines in the redo-jch.sh file either in
the editor or by running "Meta/redo-jch.sh -u", without touching
what is in redo-seen.sh file (meaning: what comes after your final
"###" cut lines).

After replacing the existing topic branches and creating new ones
for new topics, decide which topics to merge down to master and next
and edit redo-jch.sh (while looking at the what's cooking report
and/or output from "Meta/cook -w").  If we merge some to 'master',
then tentatively write '###' marker before these topics.  Then

    $ git checkout --detach master
    $ Meta/redo-jch.sh -c1
    $ edit RelNotes
    $ git commit -a -s -m "${N}th batch"
    $ Meta/round -coccinelle ;# and other tests as necessary
    $ Meta/Reintegrate master.. >P
    $ git checkout master
    $ sh P && git commit --amend --no-edit --reset-author

will rebuild the 'master' (but this is needed only when 'master'
gets updated in the day's integration).  After that

    $ git checkout -B jch master
    $ Meta/redo-jch.sh

and use 

    $ git branch --no-merged jch --no-merged seen --sort=-committerdate '??/*'

to see any topics that are not in last 'seen' and 'jch' we just
rebuilt.  They are either replaced in 'seen' or new ones.  Then
merge some of them that you are more confident than others to 'jch'
and test and update the redo-jch.sh script.

    $ git merge xy/xxy
    $ git merge fr/otz
    ...
    $ Meta/round ;# or whatever tests that are appropriate
    $ Meta/redo-jch.sh -u

This "-u"pdate step can be done without disturbing what should later
build on top for 'seen' by having the script separately.  

When day's integration contains an update to 'next', there is
another step here:

    $ git checkout --detach next
    $ Meta/redo-jch.sh -c1
    $ git merge -m "Sync with 'master'" --no-log master
    $ Meta/round ;# or whatever tests that are appropriate
    $ git diff 'jch^{/^### match next}' ;# must be empty

After that, queue new topics that are more questionable on 'seen',
and the rest:

    $ git checkotu -B seen
    $ git merge ni/tfol
    $ git merge yo/min
    ...
    $ Meta/round ;# or whatever tests that are appropriate
    ... here, new topics may be worse than what I initially thought
    ... that they need to be ejected from 'seen'
    $ git reset --hard jch && git merge yo/min ... && Meta/round

And once satisfied with the new topics, queue the remainder.

    $ Meta/redo-seen.sh
    $ Meta/round ;# or whatever tests that are appropriate
    $ Meta/redo-seen.sh -u ;# finally

Being able to rebuild the redo-* script for only the 'jch' part,
independent from the 'seen' part, is quite essential in the
workflow, because I may not yet know how day's 'seen' would look
like, when I am recording the topics and integration order for 'jch'.

Re: [PATCH 0/2] Documentation/howto/maintain-git.txt: a pair of bugfixes

[Taylor Blau]

On Tue, Nov 01, 2022 at 03:23:25PM -0700, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > Why not have a single file with all of the topics, with two "###"
> > markers?
>
> Mostly because jch and seen are built in two distinct steps, and I
> need to be able to rearrange lines in the redo-jch.sh file either in
> the editor or by running "Meta/redo-jch.sh -u", without touching
> what is in redo-seen.sh file (meaning: what comes after your final
> "###" cut lines).

Very helpful. Thank you :-).

Thanks,
Taylor

Re: [PATCH 0/2] Documentation/howto/maintain-git.txt: a pair of bugfixes

[Jeff King]

On Tue, Nov 01, 2022 at 03:23:25PM -0700, Junio C Hamano wrote:

> and use 
> 
>     $ git branch --no-merged jch --no-merged seen --sort=-committerdate '??/*'
> 
> to see any topics that are not in last 'seen' and 'jch' we just
> rebuilt.  They are either replaced in 'seen' or new ones.  Then
> merge some of them that you are more confident than others to 'jch'
> and test and update the redo-jch.sh script.
> 
>     $ git merge xy/xxy
>     $ git merge fr/otz
>     ...
>     $ Meta/round ;# or whatever tests that are appropriate
>     $ Meta/redo-jch.sh -u
> 
> This "-u"pdate step can be done without disturbing what should later
> build on top for 'seen' by having the script separately.

Ah, OK, this is the part I was missing. I assumed you would write the
new topic names into the redo-jch file, then run it to create the merged
state (since with rerere, it's easy-ish to reset jch back to master and
just rebuild to the desired spot).

But that is inside-out from your workflow, which is to treat topics
individually. Updating redo-jch.sh is done by the script and explains
what you did manually, rather than manually updating redo-jch.sh and
having it build automatically. Your way makes much more sense, since you
get the opportunity to examine the intermediate states more closely.

Thanks for explaining!

-Peff