Re: [PATCH 0/4] Run hooks in parallel

[Phillip Wood <phillip.wood123@gmail.com>]

On 12/02/2026 14:24, Adrian Ratiu wrote:
> On Thu, 12 Feb 2026, Phillip Wood <phillip.wood123@gmail.com> wrote:
>> Hi Adrian
>>
>> On 04/02/2026 17:33, Adrian Ratiu wrote:
>>> Hello everyone,
>>>
>>> This enables running hook commands in parallel and is based on the patch
>>> series enabling config hooks [1], which added the ability to run a list
>>> of hooks for each hook event.
>>>
>>> For context, hooks used to run sequentially due to hardcoded .jobs == 1
>>> in hook.c, leading to .processes == 1 in run-command.c. We're removing
>>> that restriction for hooks known to be safe to parallelize.
>>>
>>> The parallelism enabled here is to run multiple hook commands/scripts
>>> in parallel for a single event, for example the pre-push event might
>>> trigger linters / spell checkers / unit tests to run at the same time.
>>>
>>> Another kind of parallelism is to split the hook input to multiple
>>> child processes, running the same command in parallel on subsets of
>>> the hook input. This series does not do that. It might be a future
>>> addition on top of this, since it's kind of a lower-level parallelism.
>>
>> There's quite a lot of prior-art on parallelization from the various
>> hook managers - is there anything we can learn from them? For example I
>> know some of them serialize the pre-commit hook by default as it may
>> update the index but allow the user to configure a subset of scripts
>> that can be parallelized. They also allow for parallelization where
>> different scripts update different files (e.g. code formatters for
>> python and C can run in parallel). We don't need to implement all that
>> now but we should design our config so that we can support it in the future.
> 
> Yes, all the prior-art is very useful and it is possible to do
> finer-grained (or lower-level? :-) ) parallelism further with the
> new run-command parallelization design, APIs and config.
> 
> Obviously that will require more work and doing careful analysis on each
> hook-by-hook case. I'm just adding the basic buliding blocks and
> enabling the "highest-level" (most... independent?... level between
> tasks) of parallelism here.
> 
> My approach to this big problem was to simplify and break it down into
> smaller / easier-to-manage chunks. I'm still splitting up commits and
> untangling logic to ensure each part is done properly (also easier to
> review) and can work independently (no regresions etc) before building
> on top of it.

That sounds sensible and should make it easier to review, so long as we 
design the configuration in a way that it can be extended as we add more 
features.

> Thank you, and everyone else, so much for all the help and patience.
> 
>>> The pre-push hook is special because it is the only known hook to break
>>> backward compatibility when running in parallel, due to run-command
>>> collating its outputs via a pipe, so I added an extension for it.
>>> Users can opt-in to this extension with a runtime config.
>>
>> In the past we had a regression report [1] when the pre-commit hook
>> stopped having access to the terminal. I've not been following the hook
>> changes, is this series (or any of your preparatory series) in danger of
>> reintroducing that regression?
> 
> Thank you for raising this, it is a very valuable data point!
> 
> The preparatory series 100% will not reintroduce it.
> 
> This series might reintroduce it, depending how we set the defaults.
> 
> By that I mean:
>      -j1 will keep all hooks connected to the tty, just like before.
>      -jN with N>1 will disconnect the hooks from the tty and their
>          outputs will get buffered through run-command's pipes.
> 
> The design I followed (which to be transparent is Peff's design, I just
> implemented his ideas :), is the keep the original, serialized behavior
> exactly how it was before, to not introduce any regressions and to also
> keep identical "real-time" performance.
> 
> Taking this into account, together with Patrick's feedback on this
> series, I do intend keep the jobs == 1 default for all hooks in v2.

I think that would be safer. If we could opt-in to parallel execution on 
a per-hook basis would that be a solution for the "pre-push" hook? Users 
who want to keep the current behavior would avoid configuring parallel 
execution for that hook.

Thanks

Phillip

>>
>> Thanks for working on this - both config based hooks and parallel
>> execution are really nice improvements.
> 
> Thank you for the kind words. :)