[RFC] How to accellerate the patch flow (or should we?)

[RFC] How to accellerate the patch flow (or should we?)

[Junio C Hamano]

A typical lifecycle of a patch series sent to the list ought to be
as follows:

 1. You write a solution, describe the problem and solution in a
    patch series, send it to the list.

 2. People notice, take interest in, and comment on these patches.

 3. After #2 above, you may find it necessary to produce an updated
    version.  Go back to #1.

 4. While the above cycle is running, the maintainer may queue it in
    'seen', for two purposes.  (1) not to lose sight and forget
    about the change.  (2) to catch potential conflicts and overlaps
    with other in-flight topics to keep their interaction manageable.

 5. The maintainer notices that the discussion has reached a
    conclusion that the patch series is in a good shape, and merges
    it to 'next'.

 6. If a problem is found while the topic is in 'next', the topic
    may be marked as 'on hold' until an incremental solution to the
    problem is created.  This step may repeat while the topic is in
    'next'.  If the problem, which hasn't been noticed before #5
    happend, turns out to be so severe, the topic may have to be
    ejected out of 'next' for fresh restart.

 7. If 7 calendar days passes since the topic was merged to 'next'
    (or any incremental fixes are also added and merged to 'next')
    without any further finding of more problems, the topic gets
    merged to 'master'.  We often call this 'graduating'.

 8. Any further updates (enhancements and bugfixes) to the already
    graduated topic goes through the same cycle as a new and
    independent topic, starting from #1.

The time taken during the 1..3 cycle and number of iterations to get
right cannot be really shrunk without making our developers work
harder and there are only 24 hours in a day.  And I do not think we
can realistically "make our developers work hader", as everybody's
priority is different.

The time taken during 7. is pretty much fixed and unless we are
willing to sacrifice the quality of the end result, cannot
reasonably be shortened (note that this is based on the assumption
that "find any remaining bugs while it is in 'next' before it hits
'master'" philosophy is working, but we have never run experiments
to shorten this to say 3 days to see if we see more bugs on 'master'
yet).

But I wonder if 5. can be accellerated without sacrificing the
quality of the end result, every time I send out a "ping" e-mail to
old discussion thread, with "The discussion seems to have petered
out at this point.  Is everybody happy?"  Either I could have sent
such "ping"s earlier, or reviewers who were involved in the review
could have given a positive "oh, this looks good. let's declare a
victory" sooner.  Of course, from the point of view of distributing
the load and making the process as parallelized as possible, the
latter is more preferrable.

Let's take a handful of topics that graduated to 'master' on
2025-09-23 and examine how much they spent in each phase.

  * cs/subtree-squash-split-fix: The topic took 3 iterations (v1:
    2025-08-24, v2: 2025-09-05, v3: 2025-09-10).

  After <dd71ebee-8629-43c3-aa2a-40124400f262@gmail.com> gave an Ack,
  the topic was picked up to 'seen' on 2025-09-11, and then 42063163
  (Merge branch 'cs/subtree-squash-split-fix' into next, 2025-09-15)
  merged it to 'next'.  We probably could have merged it to 'next' a
  few days earlier.

  * rs/get-oid-with-flags-cleanup: a single iteration (2025-09-10)

  The topic was picked up to 'seen' on the same day,
  <aMevTM6YESMDdWPh@pks.im> gave an ack on 2025-09-15, triggering
  its merge to 'next' on the same day at ff8d1faa (Merge branch
  'rs/get-oid-with-flags-cleanup' into next, 2025-09-15).  I think
  this is an example of the process working perfectly for an
  uncontroversially good small change.

  * jk/add-i-color: 2 iterations (v1: 2025-08-21, v2: 2025-09-08).

  The first iteration was sent on 2025-08-21 after a bug report filed
  on the previous day.  After a few review exchanges starting
  2025-09-03, the second iteration sent on 2025-09-08 was acked on the
  next day.  The topic was merged on 2025-09-15, after spending a week
  in 'seen' outside 'next'.  I do not recall the details, but perhaps
  it was during relatively busy period, or there were topics that
  touched overlapping areas, or something?  Or perhaps maintainer
  fatigue?

  * cc/promisor-remote-capability: 8 iterations (v1: 2025-04-14,
    v2: 2025-04-29, v3: 2025-05-19, v4: 2025-06-11, v5: 2025-06-25,
    v6: 2025-07-21, v7: 2025-07-31, v8: 2025-09-08).

  Past round 4 or so we started seeing fewer reviews if any.  The
  topic has lived almost since its inception in 'seen' until the last
  round was merged to 'next' at 367d83c0 (Merge branch
  'cc/promisor-remote-capability' into next, 2025-09-15).  Again, this
  last round wasted 7 days in 'seen' without any activity, which I
  wonder if we could have somehow avoided?

Opinions and ideas to help topics not to waste too much time waiting
for step 5. is very much appreciated.  The most radical idea I can
think if is to introduce two more "maintainers" and any two of them
can vote to merge a topic in 'seen' to 'next' without waiting for
the other maintainer, but there may be simpler approaches that do
not require huge workflow changes.

Thanks.

Re: [RFC] How to accellerate the patch flow (or should we?)

[Taylor Blau]

On Fri, Sep 26, 2025 at 03:24:04PM -0700, Junio C Hamano wrote:
>  4. While the above cycle is running, the maintainer may queue it in
>     'seen', for two purposes.  (1) not to lose sight and forget
>     about the change.  (2) to catch potential conflicts and overlaps
>     with other in-flight topics to keep their interaction manageable.

Perhaps a third purpose is to let the maintainer (or those who use
and/or build off of 'seen' as their daily driver) detect any bugs in
that topic, or via interaction with other topics in 'seen'.

> The time taken during 7. is pretty much fixed and unless we are
> willing to sacrifice the quality of the end result, cannot
> reasonably be shortened (note that this is based on the assumption
> that "find any remaining bugs while it is in 'next' before it hits
> 'master'" philosophy is working, but we have never run experiments
> to shorten this to say 3 days to see if we see more bugs on 'master'
> yet).

I have mixed feelings about this. On the one hand, I am a little
uncomfortable with the idea of shortening the time in 'next' to fewer
than 7 days. I, too, have the feeling that having more time in 'next'
gives us a greater chance of spotting bugs in a topic that is otherwise
destined for 'master'.

On the other hand, how many people are using 'next' as their daily
driver? Of those, how many are actively looking for bugs in the topics
that are in master..next. And of those, how many are actually triggering
unique code paths that would expose those bugs in the first place?

I have a vague recollection that Google internally has their engineers
run a version of Git that is based on 'next'. But after spending a few
minutes searching through the list archives, I can't seem to find any
record of that.

Assuming that Google does provide a large user-base for 'next', I think
we could reasonably experiment with shortening the time that topics cook
in 'next' before graduating, and compare how many bug reports we get.
Otherwise (if Google is not daily-driving 'next'), I think that we could
still run that experiment, but I think it carries somewhat higher risk.

(In either case, if we were to run that experiment, I'd suggest that we
do it towards the beginning of a release cycle. That will give us as
much time as possible to squash any extra bugs that might make it to
'master' without having to quickly revert out topics towards the end of
a cycle.)

> But I wonder if 5. can be accellerated without sacrificing the
> quality of the end result, every time I send out a "ping" e-mail to
> old discussion thread, with "The discussion seems to have petered
> out at this point.  Is everybody happy?"  Either I could have sent
> such "ping"s earlier, or reviewers who were involved in the review
> could have given a positive "oh, this looks good. let's declare a
> victory" sooner.  Of course, from the point of view of distributing
> the load and making the process as parallelized as possible, the
> latter is more preferrable.

I found that this was somewhat burdensome the few times that I have
served as interim-maintainer. I think that too often (and I am guilty of
this myself) reviewers will comment on a handful of patches, and, if
they don't have any blocking feedback, leave it at that when they would
otherwise be happy to see it merged.

I know that we have talked about encouraging more reviewers to give a
"stamp of approval" by responding to the cover letter with an "ack" or
"this looks ready to merge". When that does happen, it's very useful for
the maintainer. But when it doesn't, it leaves the maintainer in an
awkward spot of having to keep track of all pending topics on the list
and how long they have been quiet for.

Maybe tooling could help with this. But I think that we could also push
ourselves to be more proactive through policy changes. What if we
required that (unless the maintainer wants to unilaterally merge a
topic) that at least one other reviewer must give it a positive "ack"
before the maintainer starts merging it down?

(As a side-observation, that matches with my expeirence in how code
review is done at GitHub, where at least one other person must provide
an approving review, as opposed to just not leaving any blocking
feedback.)

That would leave topics which do not have active reviewers in a lurch,
but I think that is sort of the point. Contributors should expect to do
some of the work of making sure their topics get reviewed themselves,
and that shouldn't fall entirely on the maintainer.

>   * cc/promisor-remote-capability: 8 iterations (v1: 2025-04-14,
>     v2: 2025-04-29, v3: 2025-05-19, v4: 2025-06-11, v5: 2025-06-25,
>     v6: 2025-07-21, v7: 2025-07-31, v8: 2025-09-08).
>
>   Past round 4 or so we started seeing fewer reviews if any.  The
>   topic has lived almost since its inception in 'seen' until the last
>   round was merged to 'next' at 367d83c0 (Merge branch
>   'cc/promisor-remote-capability' into next, 2025-09-15).  Again, this
>   last round wasted 7 days in 'seen' without any activity, which I
>   wonder if we could have somehow avoided?

I haven't followed this series closely, but I feel like subjectively
having had my own series get into the v5-and-beyond range, that is
usually a signal that the topic is too complex/large and needs to be
split up. Not a critique of cc/promisor-remote-capability in particular,
just a general observation.

> Opinions and ideas to help topics not to waste too much time waiting
> for step 5. is very much appreciated.  The most radical idea I can
> think if is to introduce two more "maintainers" and any two of them
> can vote to merge a topic in 'seen' to 'next' without waiting for
> the other maintainer, but there may be simpler approaches that do
> not require huge workflow changes.

The best that I can come up with is the above (one or more contributors
must provide a positive "ack" before the maintainer merges a topic, up
to the maintainer's discretion). I think something like the following
would be worth trying:

--- 8< ---
diff --git a/Documentation/SubmittingPatches b/Documentation/SubmittingPatches
index 86ca7f6a78a..789febefff8 100644
--- a/Documentation/SubmittingPatches
+++ b/Documentation/SubmittingPatches
@@ -506,7 +506,10 @@ After the list reached a consensus that it is a good idea to apply the
 patch, re-send it with "To:" set to the maintainer{current-maintainer}
 and "cc:" the list{git-ml} for inclusion.  This is especially relevant
 when the maintainer did not heavily participate in the discussion and
-instead left the review to trusted others.
+instead left the review to trusted others.  Patch series must receive
+a positive "ack" from at least one contributor other than the primary
+patch series author in order to begin integrating it, subject to the
+maintainer's discretion.

 Do not forget to add trailers such as `Acked-by:`, `Reviewed-by:` and
 `Tested-by:` lines as necessary to credit people who helped your
--- >8 ---

I am not sure the idea of adding more maintainers is a good one or not.
Since I am not sure exactly what you are envisioning here, I think there
are a couple of cases:

 - There is a quorum of maintainers, who are all collectively
   responsible for building 'jch', 'seen', 'next', and 'master'. Any two
   of them are required to agree to move a topic down in order to do so,
   without needing the other maintainer to weigh in.

 - There are sub-system maintainers who are responsible for their own
   trees, and who send pull requests to the primary maintainer to
   integrate their trees back into the primary maintainer's.

The "quorum of maintainers" idea has a couple of drawbacks. Unless one
maintainer has the special role of being able to unilaterally merge
topics, each topic would require twice as many maintainers as we
currently have in order to merge it down ;-). I could also imagine there
being some coordination overhead between maintainers when shuffling who
is building the integration branches to make sure that topics don't
mysteriously get added or dropped when shuffling between maintainers.

I don't have strong feelings against the idea of having sub-system
maintainers, but I am also skeptical that it will meaningfully speed up
development. For any sub-system that has enough active contributors to
find a suitable sub-system maintainer, it is likely that patches in that
area are already getting reviewed (and hopefully merged) quickly.

It's the areas that don't have enough active contributors to have a
reliable sub-system maintainer that I worry about. Those areas are
already likely struggling to get patches reviewed and merged, but I am
not sure where we would even find a sub-system maintainer for those
areas in the hopes of speeding it up.

I think that last point is an argument in favor of the "must have a
positive 'ack'" rule above. It flips the responsibility of finding
reviewers back onto the contributors, and encourages more people to
become reviewers in new areas. In other words, if a contributor wants to
write patches in a area that traditionally struggles to get timely
reviews, they are incentivized to find and encourage new reviewers to
work in that area, which is a net benefit to the project.

So, I dunno. I guess I had more thoughts about this than I expected to,
but I would be interested in trying either shortening the period topics
cook in 'next', or trying out the "must receive a positive 'ack'" rule.

Thanks,
Taylor

Re: [RFC] How to accellerate the patch flow (or should we?)

[Junio C Hamano]

Taylor Blau <me@ttaylorr.com> writes:

>> ... (note that this is based on the assumption
>> that "find any remaining bugs while it is in 'next' before it hits
>> 'master'" philosophy is working, but we have never run experiments
>> to shorten this to say 3 days to see if we see more bugs on 'master'
>> yet).
> ...
> I have a vague recollection that Google internally has their engineers
> run a version of Git that is based on 'next'. But after spending a few
> minutes searching through the list archives, I can't seem to find any
> record of that.

They do, but the frequency they update desktop installations is lower
than the frequency I merge new topics to update the tip of 'next', so
I suspect they alone would not be sufficient guinea pigs.

> Maybe tooling could help with this. But I think that we could also push
> ourselves to be more proactive through policy changes. What if we
> required that (unless the maintainer wants to unilaterally merge a
> topic) that at least one other reviewer must give it a positive "ack"
> before the maintainer starts merging it down?
> ...
> The best that I can come up with is the above (one or more contributors
> must provide a positive "ack" before the maintainer merges a topic, up
> to the maintainer's discretion). I think something like the following
> would be worth trying:
>
> --- 8< ---
> diff --git a/Documentation/SubmittingPatches b/Documentation/SubmittingPatches
> index 86ca7f6a78a..789febefff8 100644
> --- a/Documentation/SubmittingPatches
> +++ b/Documentation/SubmittingPatches
> @@ -506,7 +506,10 @@ After the list reached a consensus that it is a good idea to apply the
>  patch, re-send it with "To:" set to the maintainer{current-maintainer}
>  and "cc:" the list{git-ml} for inclusion.  This is especially relevant
>  when the maintainer did not heavily participate in the discussion and
> -instead left the review to trusted others.
> +instead left the review to trusted others.  Patch series must receive
> +a positive "ack" from at least one contributor other than the primary
> +patch series author in order to begin integrating it, subject to the
> +maintainer's discretion.
>
>  Do not forget to add trailers such as `Acked-by:`, `Reviewed-by:` and
>  `Tested-by:` lines as necessary to credit people who helped your
> --- >8 ---

It would lead us into ugly awkwardness when we start clarifying what
exactly "contributor" is in the new sentence, though.  If a person,
whom none of us have ever heard of, sends their first message to
this list saying "Ack", does that count?  If an active developer,
who is known to be sloppier than others, sends an "Ack" to somebody
else's patch that was posted 3 hours before (hence there wouldn't
have sufficient time to think through the issues), how much should
that "Ack" weigh?

Perhaps rephrasing it to "those who have helped in polishing the
patches with their reviews and discussing the issues with the patch
author" to tighten the language a bit may help?

I dunno, as that would still give the "ack right" to a random
noisemaker who threw a drive-by "review" that did not add much value
to the patches, if the original author responded "Thanks" out of
courtesy.

> I am not sure the idea of adding more maintainers is a good one or not.
> Since I am not sure exactly what you are envisioning here, I think there
> are a couple of cases:
>
>  - There is a quorum of maintainers, who are all collectively
>    responsible for building 'jch', 'seen', 'next', and 'master'. Any two
>    of them are required to agree to move a topic down in order to do so,
>    without needing the other maintainer to weigh in.
>
>  - There are sub-system maintainers who are responsible for their own
>    trees, and who send pull requests to the primary maintainer to
>    integrate their trees back into the primary maintainer's.

Git is not large enough to benefit from the latter arrangement.
What I had in mind was the former one.

> The "quorum of maintainers" idea has a couple of drawbacks. Unless one
> maintainer has the special role of being able to unilaterally merge
> topics, each topic would require twice as many maintainers as we
> currently have in order to merge it down ;-). I could also imagine there
> being some coordination overhead between maintainers when shuffling who
> is building the integration branches to make sure that topics don't
> mysteriously get added or dropped when shuffling between maintainers.

True.  It was a strawman to invite other more realistic ideas (like
your "positive ack required"), and was not necessarily designed to
be workable ;-).

Re: [RFC] How to accellerate the patch flow (or should we?)

[Taylor Blau]

On Sat, Sep 27, 2025 at 05:19:03PM -0700, Junio C Hamano wrote:
> Taylor Blau <me@ttaylorr.com> writes:
>
> >> ... (note that this is based on the assumption
> >> that "find any remaining bugs while it is in 'next' before it hits
> >> 'master'" philosophy is working, but we have never run experiments
> >> to shorten this to say 3 days to see if we see more bugs on 'master'
> >> yet).
> > ...
> > I have a vague recollection that Google internally has their engineers
> > run a version of Git that is based on 'next'. But after spending a few
> > minutes searching through the list archives, I can't seem to find any
> > record of that.
>
> They do, but the frequency they update desktop installations is lower
> than the frequency I merge new topics to update the tip of 'next', so
> I suspect they alone would not be sufficient guinea pigs.

Good to know, and yeah, if Googlers aren't receiving 'next' updates as
frequently as the maintainer is producing them, then I don't think that
increases the risk of shortening the period for which topics cook on
'next' before graduating.

> It would lead us into ugly awkwardness when we start clarifying what
> exactly "contributor" is in the new sentence, though.  If a person,
> whom none of us have ever heard of, sends their first message to
> this list saying "Ack", does that count?  If an active developer,
> who is known to be sloppier than others, sends an "Ack" to somebody
> else's patch that was posted 3 hours before (hence there wouldn't
> have sufficient time to think through the issues), how much should
> that "Ack" weigh?
>
> Perhaps rephrasing it to "those who have helped in polishing the
> patches with their reviews and discussing the issues with the patch
> author" to tighten the language a bit may help?
>
> I dunno, as that would still give the "ack right" to a random
> noisemaker who threw a drive-by "review" that did not add much value
> to the patches, if the original author responded "Thanks" out of
> courtesy.

Good point. Having a REVIEWERS file might help with that. Perhaps that
file starts with just you on it, and then it can be expanded to form a
network of trusted reviewers over time.

Building on the "how code review is done at GitHub" thing... GitHub has
a concept of required reviewers for PRs based on what file(s) are
modified as a part of the PR via its CODEOWNERS file. I share your
feeling below that the project is not large enough to have individual
areas have separate groups of reviewers, so perhaps just a single list
is fine.

> True.  It was a strawman to invite other more realistic ideas (like
> your "positive ack required"), and was not necessarily designed to
> be workable ;-).

;-).

Thanks,
Taylor

Re: [RFC] How to accellerate the patch flow (or should we?)

[Kristoffer Haugsbakk]

On Sat, Sep 27, 2025, at 00:24, Junio C Hamano wrote:
> A typical lifecycle of a patch series sent to the list ought to be
> as follows:
>
>[snip]
>  7. If 7 calendar days passes since the topic was merged to 'next'
>     (or any incremental fixes are also added and merged to 'next')
>     without any further finding of more problems, the topic gets
>     merged to 'master'.  We often call this 'graduating'.
>
>[snip]
> The time taken during 7. is pretty much fixed and unless we are
> willing to sacrifice the quality of the end result, cannot
> reasonably be shortened (note that this is based on the assumption
> that "find any remaining bugs while it is in 'next' before it hits
> 'master'" philosophy is working, but we have never run experiments
> to shorten this to say 3 days to see if we see more bugs on 'master'
> yet).

Maybe pure documentation changes could cook for less than seven days.
Not for the benefit of graduating to `master` faster but for the
benefits that less integration time brings:

- Less integration overhead since things are less likely to touch the
  same code (docs) at the same time
- Less likely that you have to base new changes on pending series
  - Integration overhead that affects both the contributor and the
    maintainer
- Potential second order effects like not being afraid of splitting up a
  lengthy series into an uncontroversial preamble of pure fixes since
  the combined series won’t have such a long potential lifespan

Given a documentation series that either fixes something or subjectively
(by consensus) improves something, what’s the risk of cooking for
slightly less than seven days?  You might introduce rendering
regressions, which is not on the same order of badness that a software
bug is.  And as for the subjectively improved documentation: people who
disagree on a subjective basis with a topic in `next` are already
considered (?) a bit late to the party.

And you also might miss out on feedback from people who run
`Documentation/doc-diff origin/master origin/next` when they are
bored.[1]  But again it doesn’t seem like a big risk or downside.

You still keep the same standard of review.  But save integration time.

🔗 1: https://lore.kernel.org/git/df251b0c-c593-41ed-903e-8fb1c323b874@app.fastmail.com/

-- 
Kristoffer Haugsbakk

Re: [RFC] How to accellerate the patch flow (or should we?)

[Kristoffer Haugsbakk]

On Sat, Sep 27, 2025, at 23:32, Taylor Blau wrote:
> On Fri, Sep 26, 2025 at 03:24:04PM -0700, Junio C Hamano wrote:
>>  4. While the above cycle is running, the maintainer may queue it in
>>     'seen', for two purposes.  (1) not to lose sight and forget
>>     about the change.  (2) to catch potential conflicts and overlaps
>>     with other in-flight topics to keep their interaction manageable.
>
> Perhaps a third purpose is to let the maintainer (or those who use
> and/or build off of 'seen' as their daily driver) detect any bugs in
> that topic, or via interaction with other topics in 'seen'.
>
>> The time taken during 7. is pretty much fixed and unless we are
>> willing to sacrifice the quality of the end result, cannot
>> reasonably be shortened (note that this is based on the assumption
>> that "find any remaining bugs while it is in 'next' before it hits
>> 'master'" philosophy is working, but we have never run experiments
>> to shorten this to say 3 days to see if we see more bugs on 'master'
>> yet).
>
> I have mixed feelings about this. On the one hand, I am a little
> uncomfortable with the idea of shortening the time in 'next' to fewer
> than 7 days. I, too, have the feeling that having more time in 'next'
> gives us a greater chance of spotting bugs in a topic that is otherwise
> destined for 'master'.
>
> On the other hand, how many people are using 'next' as their daily
> driver? Of those, how many are actively looking for bugs in the topics
> that are in master..next. And of those, how many are actually triggering
> unique code paths that would expose those bugs in the first place?

Theoretically all the projects that make heavy use of git(1) could run
`next` (and `master`) as an alternative configuration of their
integration tests.

-- 
Kristoffer

Re: [RFC] How to accellerate the patch flow (or should we?)

[Ben Knoble]

> Le 29 sept. 2025 à 16:13, Kristoffer Haugsbakk <kristofferhaugsbakk@fastmail.com> a écrit :
> 
> On Sat, Sep 27, 2025, at 23:32, Taylor Blau wrote:
>>> On Fri, Sep 26, 2025 at 03:24:04PM -0700, Junio C Hamano wrote:
>>> 4. While the above cycle is running, the maintainer may queue it in
>>>    'seen', for two purposes.  (1) not to lose sight and forget
>>>    about the change.  (2) to catch potential conflicts and overlaps
>>>    with other in-flight topics to keep their interaction manageable.
>> 
>> Perhaps a third purpose is to let the maintainer (or those who use
>> and/or build off of 'seen' as their daily driver) detect any bugs in
>> that topic, or via interaction with other topics in 'seen'.
>> 
>>> The time taken during 7. is pretty much fixed and unless we are
>>> willing to sacrifice the quality of the end result, cannot
>>> reasonably be shortened (note that this is based on the assumption
>>> that "find any remaining bugs while it is in 'next' before it hits
>>> 'master'" philosophy is working, but we have never run experiments
>>> to shorten this to say 3 days to see if we see more bugs on 'master'
>>> yet).
>> 
>> I have mixed feelings about this. On the one hand, I am a little
>> uncomfortable with the idea of shortening the time in 'next' to fewer
>> than 7 days. I, too, have the feeling that having more time in 'next'
>> gives us a greater chance of spotting bugs in a topic that is otherwise
>> destined for 'master'.
>> 
>> On the other hand, how many people are using 'next' as their daily
>> driver? Of those, how many are actively looking for bugs in the topics
>> that are in master..next. And of those, how many are actually triggering
>> unique code paths that would expose those bugs in the first place?
> 
> Theoretically all the projects that make heavy use of git(1) could run
> `next` (and `master`) as an alternative configuration of their
> integration tests.

This reminds me of crater from Rust: could we work with know “big users” in the ecosystems to run their tests against our next/master Git periodically? On every commit?

Re: [RFC] How to accellerate the patch flow (or should we?)

[Junio C Hamano]

"Kristoffer Haugsbakk" <kristofferhaugsbakk@fastmail.com> writes:

>> The time taken during 7. is pretty much fixed and unless we are
>> willing to sacrifice the quality of the end result, cannot
>> reasonably be shortened (note that this is based on the assumption
>> that "find any remaining bugs while it is in 'next' before it hits
>> 'master'" philosophy is working, but we have never run experiments
>> to shorten this to say 3 days to see if we see more bugs on 'master'
>> yet).
>
> Maybe pure documentation changes could cook for less than seven days.

Yup, that kind of adjustment at the maintainer's discretion has
already been applied, depending on the nature of a topic.  Some
truly trivial typofixes may not even go through 'next' but get
applied to 'master' directly.

Thanks.

Re: [RFC] How to accellerate the patch flow (or should we?)

[Patrick Steinhardt]

On Sat, Sep 27, 2025 at 05:19:03PM -0700, Junio C Hamano wrote:
> Taylor Blau <me@ttaylorr.com> writes:
> > diff --git a/Documentation/SubmittingPatches b/Documentation/SubmittingPatches
> > index 86ca7f6a78a..789febefff8 100644
> > --- a/Documentation/SubmittingPatches
> > +++ b/Documentation/SubmittingPatches
> > @@ -506,7 +506,10 @@ After the list reached a consensus that it is a good idea to apply the
> >  patch, re-send it with "To:" set to the maintainer{current-maintainer}
> >  and "cc:" the list{git-ml} for inclusion.  This is especially relevant
> >  when the maintainer did not heavily participate in the discussion and
> > -instead left the review to trusted others.
> > +instead left the review to trusted others.  Patch series must receive
> > +a positive "ack" from at least one contributor other than the primary
> > +patch series author in order to begin integrating it, subject to the
> > +maintainer's discretion.
> >
> >  Do not forget to add trailers such as `Acked-by:`, `Reviewed-by:` and
> >  `Tested-by:` lines as necessary to credit people who helped your
> > --- >8 ---
> 
> It would lead us into ugly awkwardness when we start clarifying what
> exactly "contributor" is in the new sentence, though.  If a person,
> whom none of us have ever heard of, sends their first message to
> this list saying "Ack", does that count?  If an active developer,
> who is known to be sloppier than others, sends an "Ack" to somebody
> else's patch that was posted 3 hours before (hence there wouldn't
> have sufficient time to think through the issues), how much should
> that "Ack" weigh?
> 
> Perhaps rephrasing it to "those who have helped in polishing the
> patches with their reviews and discussing the issues with the patch
> author" to tighten the language a bit may help?
> 
> I dunno, as that would still give the "ack right" to a random
> noisemaker who threw a drive-by "review" that did not add much value
> to the patches, if the original author responded "Thanks" out of
> courtesy.

Despite the potential awkwardness I have to wonder whether this would
even help us with the goal to speed up the overall process. To me it
rather feels like there's another step now that a patch series has to go
through, so my naive expectation is that it will rather slow the process
down even more.

Am I missing something?

> > I am not sure the idea of adding more maintainers is a good one or not.
> > Since I am not sure exactly what you are envisioning here, I think there
> > are a couple of cases:
> >
> >  - There is a quorum of maintainers, who are all collectively
> >    responsible for building 'jch', 'seen', 'next', and 'master'. Any two
> >    of them are required to agree to move a topic down in order to do so,
> >    without needing the other maintainer to weigh in.
> >
> >  - There are sub-system maintainers who are responsible for their own
> >    trees, and who send pull requests to the primary maintainer to
> >    integrate their trees back into the primary maintainer's.
> 
> Git is not large enough to benefit from the latter arrangement.
> What I had in mind was the former one.

True. The other problem I see is that it might also skew the rate of
patches towards subsystems that have more active contribtors (or rather
subsystem maintainers). Other subsystems that don't have the luxury
would then potentially be disfavored.

I guess this is especially an issue as the most active subsystems tend
to be the ones maintained by hosting providers like GitLab and GitHub.
Drive-by contributions on the other hand tend to be more cluttered
around different parts of Git, and here it's unlikely that we have
enough people to serve as subsystem maintainers.

Patrick

Re: [RFC] How to accellerate the patch flow (or should we?)

[Patrick Steinhardt]

On Sat, Sep 27, 2025 at 05:32:36PM -0400, Taylor Blau wrote:
> That would leave topics which do not have active reviewers in a lurch,
> but I think that is sort of the point. Contributors should expect to do
> some of the work of making sure their topics get reviewed themselves,
> and that shouldn't fall entirely on the maintainer.

I guess this is a realistic expectation for oldtimers on the mailing
list. And it especially is a possibility for developers that work in a
team, as they can basically tell others to do a review. But newcomers
already have a hard time to contribute to Git, so we should make sure
that we don't put more of the burden on them.

Patrick

Re: [RFC] How to accellerate the patch flow (or should we?)

[Patrick Steinhardt]

On Mon, Sep 29, 2025 at 10:12:31PM +0200, Kristoffer Haugsbakk wrote:
> On Sat, Sep 27, 2025, at 23:32, Taylor Blau wrote:
> > On Fri, Sep 26, 2025 at 03:24:04PM -0700, Junio C Hamano wrote:
> >>  4. While the above cycle is running, the maintainer may queue it in
> >>     'seen', for two purposes.  (1) not to lose sight and forget
> >>     about the change.  (2) to catch potential conflicts and overlaps
> >>     with other in-flight topics to keep their interaction manageable.
> >
> > Perhaps a third purpose is to let the maintainer (or those who use
> > and/or build off of 'seen' as their daily driver) detect any bugs in
> > that topic, or via interaction with other topics in 'seen'.
> >
> >> The time taken during 7. is pretty much fixed and unless we are
> >> willing to sacrifice the quality of the end result, cannot
> >> reasonably be shortened (note that this is based on the assumption
> >> that "find any remaining bugs while it is in 'next' before it hits
> >> 'master'" philosophy is working, but we have never run experiments
> >> to shorten this to say 3 days to see if we see more bugs on 'master'
> >> yet).
> >
> > I have mixed feelings about this. On the one hand, I am a little
> > uncomfortable with the idea of shortening the time in 'next' to fewer
> > than 7 days. I, too, have the feeling that having more time in 'next'
> > gives us a greater chance of spotting bugs in a topic that is otherwise
> > destined for 'master'.
> >
> > On the other hand, how many people are using 'next' as their daily
> > driver? Of those, how many are actively looking for bugs in the topics
> > that are in master..next. And of those, how many are actually triggering
> > unique code paths that would expose those bugs in the first place?
> 
> Theoretically all the projects that make heavy use of git(1) could run
> `next` (and `master`) as an alternative configuration of their
> integration tests.

At GitLab we do: we have nightly CI runs that execute our integration
tests with both `next` and `master`. This has already surfaced issues at
several points in time. We also have plans to roll out `next` to all of
our developers, but we haven't gotten around to this one yet.

Given the daily schedule of the CI runs it would be _okayish_ to reduce
the 7 days to something shorter. But realistically, not every pipeline
failure is noticed immediately, and sometimes we have different
priorities before we can dive into the pipeline failure.

The same is basically true for those companies that deploy to their devs
from `next`. There is a certain roundtrip time: the new version has to
be deployed, people have to start using it, and the bug reports have to
trickle back to the Git teams before they become actionable.

So I don't really feel like we can significantly reduce these 7 days
without losing some of the safety.

Patrick

Re: [RFC] How to accellerate the patch flow (or should we?)

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> Despite the potential awkwardness I have to wonder whether this would
> even help us with the goal to speed up the overall process. To me it
> rather feels like there's another step now that a patch series has to go
> through, so my naive expectation is that it will rather slow the process
> down even more.
>
> Am I missing something?

The above is an effort to reduce the chance/risk that the maintainer
is *not* made aware of the fact that a topic has already been well
reviewed, or the latest reroll has addressed all the issues
previously have pointed out and the reviewers are happy with its
shape, and ready to be merged to 'next', just left in 'seen' labeled
with "Will merge to 'next'?", etc.

Re: [RFC] How to accellerate the patch flow (or should we?)

[Patrick Steinhardt]

On Mon, Sep 29, 2025 at 03:46:47PM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > Despite the potential awkwardness I have to wonder whether this would
> > even help us with the goal to speed up the overall process. To me it
> > rather feels like there's another step now that a patch series has to go
> > through, so my naive expectation is that it will rather slow the process
> > down even more.
> >
> > Am I missing something?
> 
> The above is an effort to reduce the chance/risk that the maintainer
> is *not* made aware of the fact that a topic has already been well
> reviewed, or the latest reroll has addressed all the issues
> previously have pointed out and the reviewers are happy with its
> shape, and ready to be merged to 'next', just left in 'seen' labeled
> with "Will merge to 'next'?", etc.

Ah, okay, that makes more sense then. So basically, reviewers should
more explicitly state their overall opinion on a patch series so that it
becomes easier for you to figure out the current state at a glance?

Patrick

Re: [RFC] How to accellerate the patch flow (or should we?)

[Taylor Blau]

On Tue, Sep 30, 2025 at 12:23:32AM +0200, Patrick Steinhardt wrote:
> > It would lead us into ugly awkwardness when we start clarifying what
> > exactly "contributor" is in the new sentence, though.  If a person,
> > whom none of us have ever heard of, sends their first message to
> > this list saying "Ack", does that count?  If an active developer,
> > who is known to be sloppier than others, sends an "Ack" to somebody
> > else's patch that was posted 3 hours before (hence there wouldn't
> > have sufficient time to think through the issues), how much should
> > that "Ack" weigh?
> >
> > Perhaps rephrasing it to "those who have helped in polishing the
> > patches with their reviews and discussing the issues with the patch
> > author" to tighten the language a bit may help?
> >
> > I dunno, as that would still give the "ack right" to a random
> > noisemaker who threw a drive-by "review" that did not add much value
> > to the patches, if the original author responded "Thanks" out of
> > courtesy.
>
> Despite the potential awkwardness I have to wonder whether this would
> even help us with the goal to speed up the overall process. To me it
> rather feels like there's another step now that a patch series has to go
> through, so my naive expectation is that it will rather slow the process
> down even more.
>
> Am I missing something?

The idea is that it would make things possibly quicker, but never
slower. Since the maintainer is also part of the "trusted ack-ers", the
only process change would be that the maintainer could choose to avoid
reading a series closely if it has already received an ack.

(As an aside, in the pseudo-patch I wrote above, the maintainer could
choose to bypass the process entirely, since it would continue to be at
their discretion. Perhaps that's just a semantics thing, since by
merging the series the maintainer is implicitly providing their own
"ack", without actually saying so.)

> I guess this is especially an issue as the most active subsystems tend
> to be the ones maintained by hosting providers like GitLab and GitHub.
> Drive-by contributions on the other hand tend to be more cluttered
> around different parts of Git, and here it's unlikely that we have
> enough people to serve as subsystem maintainers.

Yeah... to be clear, the "subsystem maintainers" idea was really just me
wondering aloud which of the two possible interpretations of what Junio
said above would make the most sense to me.

Thanks,
Taylor

Re: [RFC] How to accellerate the patch flow (or should we?)

[Taylor Blau]

On Tue, Sep 30, 2025 at 12:23:44AM +0200, Patrick Steinhardt wrote:
> On Sat, Sep 27, 2025 at 05:32:36PM -0400, Taylor Blau wrote:
> > That would leave topics which do not have active reviewers in a lurch,
> > but I think that is sort of the point. Contributors should expect to do
> > some of the work of making sure their topics get reviewed themselves,
> > and that shouldn't fall entirely on the maintainer.
>
> I guess this is a realistic expectation for oldtimers on the mailing
> list. And it especially is a possibility for developers that work in a
> team, as they can basically tell others to do a review. But newcomers
> already have a hard time to contribute to Git, so we should make sure
> that we don't put more of the burden on them.

I think the roles would change slightly. I agree that newcomers should
not be entirely responsible for finding reviewers, and I don't think
that asking them to do that is entirely practical either.

But it's also impractical IMHO to expect that the maintainer doesn't
happen to lose track of a topic from time to time. Perhaps other
seasoned contributors could step in to say, "hey, this topic hasn't
gotten reviewed. Perhaps X, Y, or Z might want to chime in?".

I dunno.

Thanks,
Taylor

Re: [RFC] How to accellerate the patch flow (or should we?)

[Junio C Hamano]

Taylor Blau <me@ttaylorr.com> writes:

> (As an aside, in the pseudo-patch I wrote above, the maintainer could
> choose to bypass the process entirely, since it would continue to be at
> their discretion. Perhaps that's just a semantics thing, since by
> merging the series the maintainer is implicitly providing their own
> "ack", without actually saying so.)

This is exactly the reason why I keep saying "being in 'seen' has no
meaning other than I happened to have seen, not more than that".
A topic queued in 'seen' by default has no "ack" from me or anybody.

Re: [RFC] How to accellerate the patch flow (or should we?)

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> On Mon, Sep 29, 2025 at 03:46:47PM -0700, Junio C Hamano wrote:
>> Patrick Steinhardt <ps@pks.im> writes:
>> 
>> > Despite the potential awkwardness I have to wonder whether this would
>> > even help us with the goal to speed up the overall process. To me it
>> > rather feels like there's another step now that a patch series has to go
>> > through, so my naive expectation is that it will rather slow the process
>> > down even more.
>> >
>> > Am I missing something?
>> 
>> The above is an effort to reduce the chance/risk that the maintainer
>> is *not* made aware of the fact that a topic has already been well
>> reviewed, or the latest reroll has addressed all the issues
>> previously have pointed out and the reviewers are happy with its
>> shape, and ready to be merged to 'next', just left in 'seen' labeled
>> with "Will merge to 'next'?", etc.
>
> Ah, okay, that makes more sense then. So basically, reviewers should
> more explicitly state their overall opinion on a patch series so that it
> becomes easier for you to figure out the current state at a glance?

If they did so, it may reduce the time a topic wastes in limbo,
waiting for me to notice that it is ready.  What I do not know is if
such an additional burden on reviewers would slow us down overall.

Thanks.