public inbox for git@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/4] Run hooks in parallel
@ 2026-02-04 17:33 Adrian Ratiu
  2026-02-04 17:33 ` [PATCH 1/4] config: add a repo_config_get_uint() helper Adrian Ratiu
                   ` (8 more replies)
  0 siblings, 9 replies; 83+ messages in thread
From: Adrian Ratiu @ 2026-02-04 17:33 UTC (permalink / raw)
  To: git
  Cc: Jeff King, Emily Shaffer, Junio C Hamano, Patrick Steinhardt,
	Josh Steadmon, Kristoffer Haugsbakk, Adrian Ratiu

Hello everyone,

This enables running hook commands in parallel and is based on the patch
series enabling config hooks [1], which added the ability to run a list
of hooks for each hook event.

For context, hooks used to run sequentially due to hardcoded .jobs == 1
in hook.c, leading to .processes == 1 in run-command.c. We're removing
that restriction for hooks known to be safe to parallelize.

The parallelism enabled here is to run multiple hook commands/scripts
in parallel for a single event, for example the pre-push event might
trigger linters / spell checkers / unit tests to run at the same time.

Another kind of parallelism is to split the hook input to multiple
child processes, running the same command in parallel on subsets of
the hook input. This series does not do that. It might be a future
addition on top of this, since it's kind of a lower-level parallelism.

The pre-push hook is special because it is the only known hook to break
backward compatibility when running in parallel, due to run-command
collating its outputs via a pipe, so I added an extension for it.
Users can opt-in to this extension with a runtime config.

Suggestions for alternative solutions to the extension are welcome.

Again, this is based on the latest v1 config hooks series [1] which
has not yet landed in next or master.

Branch pushed to GitHub containing all dependency patches: [2]
Successful CI run: [3]

Many thanks to all who contributed to this effort up to now, including
Emily, AEvar, Junio, Patrick, Peff and many others.

Thank you,
Adrian

1: https://lore.kernel.org/git/20260204165126.1548805-1-adrian.ratiu@collabora.com/T/#mdb138a39d332f234bc9068b7f4e05b10c400e572
2: https://github.com/10ne1/git/tree/refs/heads/dev/aratiu/parallel-hooks-v1
3: https://github.com/10ne1/git/actions/runs/21680184456

Adrian Ratiu (3):
  config: add a repo_config_get_uint() helper
  hook: introduce extensions.hookStdoutToStderr
  hook: allow runtime enabling extensions.hookStdoutToStderr

Emily Shaffer (1):
  hook: allow parallel hook execution

 Documentation/config/extensions.adoc |  15 ++
 Documentation/config/hook.adoc       |  14 ++
 Documentation/git-hook.adoc          |  14 +-
 builtin/am.c                         |  10 +-
 builtin/checkout.c                   |  13 +-
 builtin/clone.c                      |   6 +-
 builtin/hook.c                       |   7 +-
 builtin/receive-pack.c               |   9 +-
 builtin/worktree.c                   |   2 +-
 commit.c                             |   2 +-
 config.c                             |  28 +++
 config.h                             |  13 ++
 hook.c                               |  51 ++++-
 hook.h                               |  20 +-
 parse.c                              |   9 +
 parse.h                              |   1 +
 refs.c                               |   2 +-
 repository.c                         |   1 +
 repository.h                         |   1 +
 sequencer.c                          |   4 +-
 setup.c                              |  17 ++
 setup.h                              |   1 +
 t/t1800-hook.sh                      | 270 ++++++++++++++++++++++++++-
 transport.c                          |   9 +-
 24 files changed, 476 insertions(+), 43 deletions(-)

-- 
2.52.0.732.gb351b5166d.dirty


^ permalink raw reply	[flat|nested] 83+ messages in thread

end of thread, other threads:[~2026-03-27 16:00 UTC | newest]

Thread overview: 83+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-02-04 17:33 [PATCH 0/4] Run hooks in parallel Adrian Ratiu
2026-02-04 17:33 ` [PATCH 1/4] config: add a repo_config_get_uint() helper Adrian Ratiu
2026-02-04 17:33 ` [PATCH 2/4] hook: allow parallel hook execution Adrian Ratiu
2026-02-11 12:41   ` Patrick Steinhardt
2026-02-12 12:25     ` Adrian Ratiu
2026-02-04 17:33 ` [PATCH 3/4] hook: introduce extensions.hookStdoutToStderr Adrian Ratiu
2026-02-04 17:33 ` [PATCH 4/4] hook: allow runtime enabling extensions.hookStdoutToStderr Adrian Ratiu
2026-02-12 10:43 ` [PATCH 0/4] Run hooks in parallel Phillip Wood
2026-02-12 14:24   ` Adrian Ratiu
2026-02-13 14:39     ` Phillip Wood
2026-02-13 17:21       ` Adrian Ratiu
2026-02-22  0:28 ` [PATCH v2 00/10] " Adrian Ratiu
2026-02-22  0:28   ` [PATCH v2 01/10] repository: fix repo_init() memleak due to missing _clear() Adrian Ratiu
2026-02-22  0:28   ` [PATCH v2 02/10] config: add a repo_config_get_uint() helper Adrian Ratiu
2026-02-22  0:28   ` [PATCH v2 03/10] hook: refactor hook_config_cache from strmap to named struct Adrian Ratiu
2026-02-22  0:28   ` [PATCH v2 04/10] hook: parse the hook.jobs config Adrian Ratiu
2026-02-22  0:28   ` [PATCH v2 05/10] hook: allow parallel hook execution Adrian Ratiu
2026-02-22  0:29   ` [PATCH v2 06/10] hook: mark non-parallelizable hooks Adrian Ratiu
2026-02-22  0:29   ` [PATCH v2 07/10] hook: add -j/--jobs option to git hook run Adrian Ratiu
2026-02-22  0:29   ` [PATCH v2 08/10] hook: add per-event jobs config Adrian Ratiu
2026-02-22  0:29   ` [PATCH v2 09/10] hook: introduce extensions.hookStdoutToStderr Adrian Ratiu
2026-02-22  0:29   ` [PATCH v2 10/10] hook: allow runtime enabling extensions.hookStdoutToStderr Adrian Ratiu
2026-03-09 13:37 ` [PATCH v3 0/9] Run hooks in parallel Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 1/9] repository: fix repo_init() memleak due to missing _clear() Adrian Ratiu
2026-03-15  4:55     ` Junio C Hamano
2026-03-15  5:05     ` Junio C Hamano
2026-03-09 13:37   ` [PATCH v3 2/9] config: add a repo_config_get_uint() helper Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 3/9] hook: parse the hook.jobs config Adrian Ratiu
2026-03-15 16:13     ` Junio C Hamano
2026-03-09 13:37   ` [PATCH v3 4/9] hook: allow parallel hook execution Adrian Ratiu
2026-03-15 20:46     ` Junio C Hamano
2026-03-18 18:02       ` Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 5/9] hook: mark non-parallelizable hooks Adrian Ratiu
2026-03-15 20:56     ` Junio C Hamano
2026-03-18 18:40       ` Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 6/9] hook: add -j/--jobs option to git hook run Adrian Ratiu
2026-03-15 21:00     ` Junio C Hamano
2026-03-18 19:00       ` Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 7/9] hook: add per-event jobs config Adrian Ratiu
2026-03-16 18:40     ` Junio C Hamano
2026-03-18 19:21       ` Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 8/9] hook: introduce extensions.hookStdoutToStderr Adrian Ratiu
2026-03-16 18:44     ` Junio C Hamano
2026-03-18 19:50       ` Adrian Ratiu
2026-03-09 13:37   ` [PATCH v3 9/9] hook: allow runtime enabling extensions.hookStdoutToStderr Adrian Ratiu
2026-03-20 13:53 ` [PATCH v4 0/9] Run hooks in parallel Adrian Ratiu
2026-03-20 13:53   ` [PATCH v4 1/9] config: add a repo_config_get_uint() helper Adrian Ratiu
2026-03-20 13:53   ` [PATCH v4 2/9] hook: parse the hook.jobs config Adrian Ratiu
2026-03-24  9:07     ` Patrick Steinhardt
2026-03-24 18:59       ` Adrian Ratiu
2026-03-20 13:53   ` [PATCH v4 3/9] hook: allow parallel hook execution Adrian Ratiu
2026-03-24  9:07     ` Patrick Steinhardt
2026-03-20 13:53   ` [PATCH v4 4/9] hook: allow pre-push parallel execution Adrian Ratiu
2026-03-20 13:53   ` [PATCH v4 5/9] hook: mark non-parallelizable hooks Adrian Ratiu
2026-03-20 13:53   ` [PATCH v4 6/9] hook: add -j/--jobs option to git hook run Adrian Ratiu
2026-03-24  9:07     ` Patrick Steinhardt
2026-03-20 13:53   ` [PATCH v4 7/9] hook: add per-event jobs config Adrian Ratiu
2026-03-24  9:08     ` Patrick Steinhardt
2026-03-20 13:53   ` [PATCH v4 8/9] hook: warn when hook.<friendly-name>.jobs is set Adrian Ratiu
2026-03-24  9:08     ` Patrick Steinhardt
2026-03-20 13:53   ` [PATCH v4 9/9] hook: add hook.<event>.enabled switch Adrian Ratiu
2026-03-24  9:08     ` Patrick Steinhardt
2026-03-25 18:43       ` Adrian Ratiu
2026-03-20 17:24   ` [PATCH v4 0/9] Run hooks in parallel Junio C Hamano
2026-03-23 15:07     ` Adrian Ratiu
2026-03-24  9:07       ` Patrick Steinhardt
2026-03-26 10:18 ` [PATCH v5 00/12] " Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 01/12] repository: fix repo_init() memleak due to missing _clear() Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 02/12] config: add a repo_config_get_uint() helper Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 03/12] hook: parse the hook.jobs config Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 04/12] hook: allow parallel hook execution Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 05/12] hook: allow pre-push parallel execution Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 06/12] hook: mark non-parallelizable hooks Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 07/12] hook: add -j/--jobs option to git hook run Adrian Ratiu
2026-03-27 14:46     ` Patrick Steinhardt
2026-03-26 10:18   ` [PATCH v5 08/12] hook: add per-event jobs config Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 09/12] hook: warn when hook.<friendly-name>.jobs is set Adrian Ratiu
2026-03-27 14:46     ` Patrick Steinhardt
2026-03-26 10:18   ` [PATCH v5 10/12] hook: move is_known_hook() to hook.c for wider use Adrian Ratiu
2026-03-27 14:46     ` Patrick Steinhardt
2026-03-27 15:59       ` Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 11/12] hook: add hook.<event>.enabled switch Adrian Ratiu
2026-03-26 10:18   ` [PATCH v5 12/12] hook: allow hook.jobs=-1 to use all available CPU cores Adrian Ratiu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox