All of lore.kernel.org
 help / color / mirror / Atom feed
From: Adrian Ratiu <adrian.ratiu@collabora.com>
To: Phillip Wood <phillip.wood123@gmail.com>,
	phillip.wood@dunelm.org.uk, git@vger.kernel.org
Cc: Jeff King <peff@peff.net>,
	Emily Shaffer <emilyshaffer@google.com>,
	Junio C Hamano <gitster@pobox.com>,
	Patrick Steinhardt <ps@pks.im>,
	Josh Steadmon <steadmon@google.com>,
	Kristoffer Haugsbakk <kristofferhaugsbakk@fastmail.com>
Subject: Re: [PATCH 0/4] Run hooks in parallel
Date: Fri, 13 Feb 2026 19:21:02 +0200	[thread overview]
Message-ID: <87h5rkpnk1.fsf@collabora.com> (raw)
In-Reply-To: <1dd09e04-dbdb-4a6c-933a-e4dc451878f9@gmail.com>

On Fri, 13 Feb 2026, Phillip Wood <phillip.wood123@gmail.com> wrote:
<snip>
>>>> The pre-push hook is special because it is the only known hook to break
>>>> backward compatibility when running in parallel, due to run-command
>>>> collating its outputs via a pipe, so I added an extension for it.
>>>> Users can opt-in to this extension with a runtime config.
>>>
>>> In the past we had a regression report [1] when the pre-commit hook
>>> stopped having access to the terminal. I've not been following the hook
>>> changes, is this series (or any of your preparatory series) in danger of
>>> reintroducing that regression?
>> 
>> Thank you for raising this, it is a very valuable data point!
>> 
>> The preparatory series 100% will not reintroduce it.
>> 
>> This series might reintroduce it, depending how we set the defaults.
>> 
>> By that I mean:
>>      -j1 will keep all hooks connected to the tty, just like before.
>>      -jN with N>1 will disconnect the hooks from the tty and their
>>          outputs will get buffered through run-command's pipes.
>> 
>> The design I followed (which to be transparent is Peff's design, I just
>> implemented his ideas :), is the keep the original, serialized behavior
>> exactly how it was before, to not introduce any regressions and to also
>> keep identical "real-time" performance.
>> 
>> Taking this into account, together with Patrick's feedback on this
>> series, I do intend keep the jobs == 1 default for all hooks in v2.
>
> I think that would be safer. If we could opt-in to parallel execution on 
> a per-hook basis would that be a solution for the "pre-push" hook? Users 
> who want to keep the current behavior would avoid configuring parallel 
> execution for that hook.

Yes, opting in on a per-hook basis is one of the mechanisms I'll be
adding in v2 (that's what I've been discussing with Patrick via the
other thread on this series).

However we also need a mechanism to enable more than just 1 hook at a
time, for users who want to enable by default a known-good set of hooks
to run in parallel.

That's what we did with `RUN_HOOKS_OPT_INIT_PARALLEL` at compile-time in
v1, however that's a dead end and I won't pursue it.

For v2 I'm thinking of a runtime/global config which can specify a list
of hook to default for parallel execution. That should be enough to
replace RUN_HOOKS_OPT_INIT_PARALLEL.

Suggestions welcome, as always. :)

  reply	other threads:[~2026-02-13 17:21 UTC|newest]

Thread overview: 113+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-02-04 17:33 [PATCH 0/4] Run hooks in parallel Adrian Ratiu
2026-02-04 17:33 ` [PATCH 1/4] config: add a repo_config_get_uint() helper Adrian Ratiu
2026-02-04 17:33 ` [PATCH 2/4] hook: allow parallel hook execution Adrian Ratiu
2026-02-11 12:41   ` Patrick Steinhardt
2026-02-12 12:25     ` Adrian Ratiu
2026-02-04 17:33 ` [PATCH 3/4] hook: introduce extensions.hookStdoutToStderr Adrian Ratiu
2026-02-04 17:33 ` [PATCH 4/4] hook: allow runtime enabling extensions.hookStdoutToStderr Adrian Ratiu
2026-02-12 10:43 ` [PATCH 0/4] Run hooks in parallel Phillip Wood
2026-02-12 14:24   ` Adrian Ratiu
2026-02-13 14:39     ` Phillip Wood
2026-02-13 17:21       ` Adrian Ratiu [this message]
2026-02-22  0:28 ` [PATCH v2 00/10] " Adrian Ratiu
2026-02-22  0:28   ` [PATCH v2 01/10] repository: fix repo_init() memleak due to missing _clear() Adrian Ratiu
2026-02-22  0:28   ` [PATCH v2 02/10] config: add a repo_config_get_uint() helper Adrian Ratiu
2026-02-22  0:28   ` [PATCH v2 03/10] hook: refactor hook_config_cache from strmap to named struct Adrian Ratiu
2026-02-22  0:28   ` [PATCH v2 04/10] hook: parse the hook.jobs config Adrian Ratiu
2026-02-22  0:28   ` [PATCH v2 05/10] hook: allow parallel hook execution Adrian Ratiu
2026-02-22  0:29   ` [PATCH v2 06/10] hook: mark non-parallelizable hooks Adrian Ratiu
2026-02-22  0:29   ` [PATCH v2 07/10] hook: add -j/--jobs option to git hook run Adrian Ratiu
2026-02-22  0:29   ` [PATCH v2 08/10] hook: add per-event jobs config Adrian Ratiu
2026-02-22  0:29   ` [PATCH v2 09/10] hook: introduce extensions.hookStdoutToStderr Adrian Ratiu
2026-02-22  0:29   ` [PATCH v2 10/10] hook: allow runtime enabling extensions.hookStdoutToStderr Adrian Ratiu
2026-03-09 13:37 ` [PATCH v3 0/9] Run hooks in parallel Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 1/9] repository: fix repo_init() memleak due to missing _clear() Adrian Ratiu
2026-03-15  4:55     ` Junio C Hamano
2026-03-15  5:05     ` Junio C Hamano
2026-03-09 13:37   ` [PATCH v3 2/9] config: add a repo_config_get_uint() helper Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 3/9] hook: parse the hook.jobs config Adrian Ratiu
2026-03-15 16:13     ` Junio C Hamano
2026-03-09 13:37   ` [PATCH v3 4/9] hook: allow parallel hook execution Adrian Ratiu
2026-03-15 20:46     ` Junio C Hamano
2026-03-18 18:02       ` Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 5/9] hook: mark non-parallelizable hooks Adrian Ratiu
2026-03-15 20:56     ` Junio C Hamano
2026-03-18 18:40       ` Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 6/9] hook: add -j/--jobs option to git hook run Adrian Ratiu
2026-03-15 21:00     ` Junio C Hamano
2026-03-18 19:00       ` Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 7/9] hook: add per-event jobs config Adrian Ratiu
2026-03-16 18:40     ` Junio C Hamano
2026-03-18 19:21       ` Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 8/9] hook: introduce extensions.hookStdoutToStderr Adrian Ratiu
2026-03-16 18:44     ` Junio C Hamano
2026-03-18 19:50       ` Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 9/9] hook: allow runtime enabling extensions.hookStdoutToStderr Adrian Ratiu
2026-03-20 13:53 ` [PATCH v4 0/9] Run hooks in parallel Adrian Ratiu
2026-03-20 13:53   ` [PATCH v4 1/9] config: add a repo_config_get_uint() helper Adrian Ratiu
2026-03-20 13:53   ` [PATCH v4 2/9] hook: parse the hook.jobs config Adrian Ratiu
2026-03-24  9:07     ` Patrick Steinhardt
2026-03-24 18:59       ` Adrian Ratiu
2026-03-20 13:53   ` [PATCH v4 3/9] hook: allow parallel hook execution Adrian Ratiu
2026-03-24  9:07     ` Patrick Steinhardt
2026-03-20 13:53   ` [PATCH v4 4/9] hook: allow pre-push parallel execution Adrian Ratiu
2026-03-20 13:53   ` [PATCH v4 5/9] hook: mark non-parallelizable hooks Adrian Ratiu
2026-03-20 13:53   ` [PATCH v4 6/9] hook: add -j/--jobs option to git hook run Adrian Ratiu
2026-03-24  9:07     ` Patrick Steinhardt
2026-03-20 13:53   ` [PATCH v4 7/9] hook: add per-event jobs config Adrian Ratiu
2026-03-24  9:08     ` Patrick Steinhardt
2026-03-20 13:53   ` [PATCH v4 8/9] hook: warn when hook.<friendly-name>.jobs is set Adrian Ratiu
2026-03-24  9:08     ` Patrick Steinhardt
2026-03-20 13:53   ` [PATCH v4 9/9] hook: add hook.<event>.enabled switch Adrian Ratiu
2026-03-24  9:08     ` Patrick Steinhardt
2026-03-25 18:43       ` Adrian Ratiu
2026-03-20 17:24   ` [PATCH v4 0/9] Run hooks in parallel Junio C Hamano
2026-03-23 15:07     ` Adrian Ratiu
2026-03-24  9:07       ` Patrick Steinhardt
2026-03-26 10:18 ` [PATCH v5 00/12] " Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 01/12] repository: fix repo_init() memleak due to missing _clear() Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 02/12] config: add a repo_config_get_uint() helper Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 03/12] hook: parse the hook.jobs config Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 04/12] hook: allow parallel hook execution Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 05/12] hook: allow pre-push parallel execution Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 06/12] hook: mark non-parallelizable hooks Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 07/12] hook: add -j/--jobs option to git hook run Adrian Ratiu
2026-03-27 14:46     ` Patrick Steinhardt
2026-03-26 10:18   ` [PATCH v5 08/12] hook: add per-event jobs config Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 09/12] hook: warn when hook.<friendly-name>.jobs is set Adrian Ratiu
2026-03-27 14:46     ` Patrick Steinhardt
2026-03-26 10:18   ` [PATCH v5 10/12] hook: move is_known_hook() to hook.c for wider use Adrian Ratiu
2026-03-27 14:46     ` Patrick Steinhardt
2026-03-27 15:59       ` Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 11/12] hook: add hook.<event>.enabled switch Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 12/12] hook: allow hook.jobs=-1 to use all available CPU cores Adrian Ratiu
2026-04-04  8:29 ` [PATCH v6 00/12] Run hooks in parallel Adrian Ratiu
2026-04-04  8:29   ` [PATCH v6 01/12] repository: fix repo_init() memleak due to missing _clear() Adrian Ratiu
2026-04-04  8:29   ` [PATCH v6 02/12] config: add a repo_config_get_uint() helper Adrian Ratiu
2026-04-04  8:29   ` [PATCH v6 03/12] hook: parse the hook.jobs config Adrian Ratiu
2026-04-04  8:29   ` [PATCH v6 04/12] hook: allow parallel hook execution Adrian Ratiu
2026-04-04  8:29   ` [PATCH v6 05/12] hook: allow pre-push parallel execution Adrian Ratiu
2026-04-04  8:29   ` [PATCH v6 06/12] hook: mark non-parallelizable hooks Adrian Ratiu
2026-04-04  8:29   ` [PATCH v6 07/12] hook: add -j/--jobs option to git hook run Adrian Ratiu
2026-04-04  8:29   ` [PATCH v6 08/12] hook: add per-event jobs config Adrian Ratiu
2026-04-04  8:29   ` [PATCH v6 09/12] hook: warn when hook.<friendly-name>.jobs is set Adrian Ratiu
2026-04-04  8:29   ` [PATCH v6 10/12] hook: move is_known_hook() to hook.c for wider use Adrian Ratiu
2026-04-04  8:29   ` [PATCH v6 11/12] hook: add hook.<event>.enabled switch Adrian Ratiu
2026-04-04  8:29   ` [PATCH v6 12/12] hook: allow hook.jobs=-1 to use all available CPU cores Adrian Ratiu
2026-04-06 16:24   ` [PATCH v6 00/12] Run hooks in parallel Junio C Hamano
2026-04-08 10:17   ` Patrick Steinhardt
2026-04-08 16:57     ` Junio C Hamano
2026-04-10  9:05 ` [PATCH v7 00/13] " Adrian Ratiu
2026-04-10  9:05   ` [PATCH v7 01/13] repository: fix repo_init() memleak due to missing _clear() Adrian Ratiu
2026-04-10  9:05   ` [PATCH v7 02/13] config: add a repo_config_get_uint() helper Adrian Ratiu
2026-04-10  9:05   ` [PATCH v7 03/13] hook: parse the hook.jobs config Adrian Ratiu
2026-04-10  9:05   ` [PATCH v7 04/13] hook: allow parallel hook execution Adrian Ratiu
2026-04-10  9:06   ` [PATCH v7 05/13] hook: allow pre-push parallel execution Adrian Ratiu
2026-04-10  9:06   ` [PATCH v7 06/13] hook: mark non-parallelizable hooks Adrian Ratiu
2026-04-10  9:06   ` [PATCH v7 07/13] hook: add -j/--jobs option to git hook run Adrian Ratiu
2026-04-10  9:06   ` [PATCH v7 08/13] hook: add per-event jobs config Adrian Ratiu
2026-04-10  9:06   ` [PATCH v7 09/13] hook: warn when hook.<friendly-name>.jobs is set Adrian Ratiu
2026-04-10  9:06   ` [PATCH v7 10/13] hook: move is_known_hook() to hook.c for wider use Adrian Ratiu
2026-04-10  9:06   ` [PATCH v7 11/13] hook: add hook.<event>.enabled switch Adrian Ratiu
2026-04-10  9:06   ` [PATCH v7 12/13] hook: allow hook.jobs=-1 to use all available CPU cores Adrian Ratiu
2026-04-10  9:06   ` [PATCH v7 13/13] t1800: test SIGPIPE with parallel hooks Adrian Ratiu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87h5rkpnk1.fsf@collabora.com \
    --to=adrian.ratiu@collabora.com \
    --cc=emilyshaffer@google.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=kristofferhaugsbakk@fastmail.com \
    --cc=peff@peff.net \
    --cc=phillip.wood123@gmail.com \
    --cc=phillip.wood@dunelm.org.uk \
    --cc=ps@pks.im \
    --cc=steadmon@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.