How does Git's maintenance policy handle topics that don't start from "master?"

How does Git's maintenance policy handle topics that don't start from "master?"

[Steven E. Harris]

I've read the /Addendum to "MaintNotes"/ document¹ several times in the
last few years, but in the process of trying to employ the policy with
my current team, our progress is stuck on a case that isn't addressed by
the policy -- directly, anyway.

In the policy section "Handle the remaining patches," the first clause
reads as follows:

,----[ First case for remaining patches ]
| Anything unobvious that is applicable to 'master' (in other
| words, does not depend on anything that is still in 'next'
| and not in 'master') is applied to a new topic branch that
| is forked from the tip of 'master'.  This includes both
| enhancements and unobvious fixes to 'master'.
`----

It addresses topics that can be built on top of the "master" branch,
these topics not depending on anything only available outside the
"master" branch, such as in the "next" branch. This policy is focusing
on the receiver and integrator of patches, rather than the author, but
it's not hard to infer that an author should start his work from the
"master" branch in order for his patches to be eligible for treatment by
this clause.

What about the case where an author started his work from the "next"
branch instead? He may have submitted an earlier batch of work that's
still cooking in "next," and now he needs to build something else that
can take advantage of that earlier work. It's clear that if he starts
from "next" and relies on that earlier work, then his later work is not
independent and cannot possibly graduate to the "master" branch unless
and until his earlier work graduates too.

Is the Git policy on such dependency simply, "Don't do that?"

Consider a situation where the earlier topic branch's contribution
cooking in "next" is looking good and everyone is feeling confident that
it's going to graduate, and our poor author /needs/ to get started on
his next task that would make use of the earlier work. If he does start
his new topic branch from "next" -- or maybe starts it from his earlier
topic branch instead -- what will go wrong later? Is there a part of the
policy that addresses this case that I missed?


Footnotes: 
¹ http://www.kernel.org/pub/software/scm/git/docs/v1.7.10.1/howto/maintain-git.txt

-- 
Steven E. Harris

Re: How does Git's maintenance policy handle topics that don't start from "master?"

[Steven E. Harris]

"Steven E. Harris" <seh@panix.com> writes:

> Is there a part of the policy that addresses this case that I missed?

After posting I discovered the "SubmittingPatches" document¹, which does
address part of my inquiry:

,----[ Decide what to base your work on ]
| - A new feature should be based on 'master' in general. If the new
|    feature depends on a topic that is in 'pu', but not in 'master',
|    base your work on the tip of that topic.
| 
| - Corrections and enhancements to a topic not yet in 'master' should
|    be based on the tip of that topic. If the topic has not been merged
|    to 'next', it's alright to add a note to squash minor corrections
|    into the series.
| 
| - In the exceptional case that a new feature depends on several topics
|    not in 'master', start working on 'next' or 'pu' privately and send
|    out patches for discussion. Before the final merge, you may have to
|    wait until some of the dependent topics graduate to 'master', and
|    rebase your work.
`----

The last one in particular implies that the topic branch will not be fit
for serious consideration until its dependencies have themselves
graduated to "master."

Footnotes: 
¹ https://raw.github.com/git/git/master/Documentation/SubmittingPatches
-- 
Steven E. Harris

Re: How does Git's maintenance policy handle topics that don't start from "master?"

[Junio C Hamano]

"Steven E. Harris" <seh@panix.com> writes:

> What about the case where an author started his work from the "next"
> branch instead? He may have submitted an earlier batch of work that's
> still cooking in "next," and now he needs to build something else that
> can take advantage of that earlier work.

It often is clear that the follow-on topic depends on an earlier
topic branch (mostly because the contributor is aware of it and
state it in the message).  An obvious thing to do in such a case is
to create a new branch to queue that topic starting at the tip of
the earlier topic.  Note that this is never from the tip of "next",
as it is very unlikely that such a follow-on topic depends on
everything that is not in "master" yet.

Sometimes a new work depends on multiple topics that are still
cooking, and *all* of these topics that the new work depends on are
in good shape.  In such a case, I create a new branch by merging
these prerequisite topics and then queue new work there.  Obviously
the new work is taken hostage to *all* of its dependent topics, and
cannot graduate until all of the base topics graduate.

Sometimes a new work depends on one topic that is still cooking in
"next", *and* also needs updates made by other topics that are
already in "master".  You can guess what should happen---take the
tip of that topic that is still cooking in "next", merge the commit
on "master" that adds other necessary bits, and then that becomes
the base of the new topic.  The "commit that adds other necessary
bits" could be the tip of "master" (easiest for me, but it makes the
new topic unmergeable to "maint" later) or the tip of an old topic
that was merged to "master" (more work for me, but it is worth if
both the other old topic and the topic that is cooking in "next" are
meant to be merged to "maint" later, and the new work is also meant
to eventually be merged to "maint").

In a rare cases where a new work depends on millions of uncooked
topics, we simply reject the follow-on series and tell the submitter
to wait until the dust settles, but in practice it does not happen
very often.

In other words, the "policy" is not a mechanical recipe to be
followed by brainless monkeys; the integrator needs to follow the
common sense of keeping the resulting topic branch mergeable to as
many relevant contexts as necessary.

And the contributor can help in this process, as well.

Re: How does Git's maintenance policy handle topics that don't start from "master?"

[Steven E. Harris]

Junio C Hamano <gitster@pobox.com> writes:

> It often is clear that the follow-on topic depends on an earlier topic
> branch (mostly because the contributor is aware of it and state it in
> the message).  An obvious thing to do in such a case is to create a
> new branch to queue that topic starting at the tip of the earlier
> topic.  Note that this is never from the tip of "next", as it is very
> unlikely that such a follow-on topic depends on everything that is not
> in "master" yet.

Thank you for the thorough reply.

I want to make sure that I understand the main hazard that the policy
aims to avoid. If an author bases a topic branch on the tip of "next,"
then, later, it will not be possible to merge his work to "master"
without implicitly accepting all of "next" (or, at least the part of
"next" the precedes this topic branch) into "master." We are trying to
avoid merging "upward" from "next" to "master" like that, and prefer to
take the topics to "master" directly rather than implicitly by way of
their inclusion in "next."

What isn't so clear to me, though, is /why/ this don't-merge-from-"next"
rule is so important. Say that we had one topic "t1" depart from "next,"
and then another topic "t2" depart from "t1," and both have been cooking
in "next," with good results.

  ---o---o---o---o  master
                  \
                   o---o---o---o---o---M---o---o next
                        \     /       /
                         o---o t1    /
                              \     /
                               o---o t2

If we wanted to graduate these two topics to "master," we /could/ merge
from commit M back to "master," though here I deliberately included the
nefarious commit X, which shows other interleaved contributions along
"next" that are also part of the M commit.

What about this case, where topics "t1" and "t2" did depart from
"master," and are doing well along "next" together as of commit M.

  ---o---o---o---o  master
      \   \       \
       \   o---o---o---M---o---o next
        \     /       /
         o---o t1    /
          \         /
           o---o---o t2

The Git policy as I understand it prescribes that we merge from the tips
of "t1" and "t2" back to master, not from a commit like M. What harm
would come from merging from M in this case? Future archaeology of topic
provenance?

-- 
Steven E. Harris

Re: How does Git's maintenance policy handle topics that don't start from "master?"

[Junio C Hamano]

"Steven E. Harris" <seh@panix.com> writes:

> What isn't so clear to me, though, is /why/ this don't-merge-from-"next"
> rule is so important. Say that we had one topic "t1" depart from "next,"
> and then another topic "t2" depart from "t1," and both have been cooking
> in "next," with good results.
>
>   ---o---o---o---o  master
>                   \
>                    o---o---o---o---o---M---o---o next
>                         \     /       /
>                          o---o t1    /
>                               \     /
>                                o---o t2
>
> If we wanted to graduate these two topics to "master," we /could/ merge
> from commit M back to "master," though here I deliberately included the
> nefarious commit X, which shows other interleaved contributions along
> "next" that are also part of the M commit.

I do not see any X above, but I think you meant the commits in
master..M that are not reachable from either t1 and t2.

And I think you answered your own question.  These "master..M ^t1
^t2" commits are topics that are *not* part of t1 nor t2.  If you
deem all of them are good enough for your master, it is perfectly
fine to merge M to master.  In reality, it is more cumbersome to
think about what is and what is not yet in M and decide if the set
of changes that happen to be in M match exactly what you want to
merge, than knowing that you exactly want to have t1 and t2 and
nothing else in your master during this integration run and merge
only these two topics.

The same answer to the other picture.  If you can figure out an
appropriate commit M that has what you want, go right ahead and
merge that to 'master'; I do not see any harm there.  I personally
do not think it is worth the effort to figure out which commit
between master..next that M is, and verify master..M contains
everything you want and nothing you don't.  Merging t1 and t2
explicitly, when you know they are the only thing you want to merge,
is much simpler and less error prone.

> What about this case, where topics "t1" and "t2" did depart from
> "master," and are doing well along "next" together as of commit M.
>
>   ---o---o---o---o  master
>       \   \       \
>        \   o---o---o---M---o---o next
>         \     /       /
>          o---o t1    /
>           \         /
>            o---o---o t2
>
> The Git policy as I understand it prescribes that we merge from the tips
> of "t1" and "t2" back to master, not from a commit like M. What harm
> would come from merging from M in this case? Future archaeology of topic
> provenance?

Re: How does Git's maintenance policy handle topics that don't start from "master?"

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> "Steven E. Harris" <seh@panix.com> writes:
>
>> What about this case, where topics "t1" and "t2" did depart from
>> "master," and are doing well along "next" together as of commit M.
>>
>>   ---o---o---o---o  master
>>       \   \       \
>>        \   o---o---o---M---o---o next
>>         \     /       /
>>          o---o t1    /
>>           \         /
>>            o---o---o t2
>>
>> The Git policy as I understand it prescribes that we merge from the tips
>> of "t1" and "t2" back to master, not from a commit like M. What harm
>> would come from merging from M in this case? Future archaeology of topic
>> provenance?

You would also have to think about what you write to explain that
merge of M into 'master' in the commit log message.  In the above
special case where 'next' happened to have had only t1 and t2 when
you decided to merge to 'master', you could say "Merge t1 and t2 to
achieve X", but in the earlier example (elided), it is not clear
what random bits you are merging with it.

Compared to that, if you merge t1 and t2 separately, you can say "I
am merging t1 topic that does X", and readers of "git log master"
(better yet "git log --first-parent master") would understand that
the master branch after that commit can do X (and everything before
that commit does not).