git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Adrian Ratiu <adrian.ratiu@collabora.com>
To: Patrick Steinhardt <ps@pks.im>
Cc: git@vger.kernel.org, "Emily Shaffer" <emilyshaffer@google.com>,
	"Rodrigo Damazio Bovendorp" <rdamazio@google.com>,
	"Junio C Hamano" <gitster@pobox.com>,
	"Josh Steadmon" <steadmon@google.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: Re: [PATCH 05/10] reference-transaction: use hook.h to run hooks
Date: Wed, 08 Oct 2025 15:26:32 +0300	[thread overview]
Message-ID: <87v7kp37gn.fsf@collabora.com> (raw)
In-Reply-To: <aN4c_DWtqBBScKEh@pks.im>

Hi Patrick and sorry for the delayed reply!

On Thu, 02 Oct 2025, Patrick Steinhardt <ps@pks.im> wrote:
> On Thu, Sep 25, 2025 at 03:53:48PM +0300, Adrian Ratiu wrote: 
>> diff --git a/refs.c b/refs.c index 4ff55cf24f..5a2b6ad1fc 
>> 100644 --- a/refs.c +++ b/refs.c @@ -2377,31 +2377,16 @@ static 
>> int ref_update_reject_duplicates(struct string_list *refnames, 
>>  	return 0; }  
>> -static int run_transaction_hook(struct ref_transaction 
>> *transaction, -				const char *state) 
>> +static int transaction_hook_feed_stdin(int hook_stdin_fd, void 
>> *pp_cb, void *pp_task_cb UNUSED) 
>>  { 
>> -	struct child_process proc = CHILD_PROCESS_INIT; + 
>> struct hook_cb_data *hook_cb = pp_cb; +	struct 
>> run_hooks_opt *opt = hook_cb->options; +	struct 
>> ref_transaction *transaction = opt->feed_pipe_ctx; 
>>  	struct strbuf buf = STRBUF_INIT; 
>> -	const char *hook; -	int ret = 0, i; - -	hook = 
>> find_hook(transaction->ref_store->repo, 
>> "reference-transaction"); -	if (!hook) -		return 
>> ret; - -	strvec_pushl(&proc.args, hook, state, NULL); - 
>> proc.in = -1; -	proc.stdout_to_stderr = 1; - 
>> proc.trace2_hook_name = "reference-transaction"; - -	ret = 
>> start_command(&proc); -	if (ret) -		return 
>> ret; - -	sigchain_push(SIGPIPE, SIG_IGN); 
>>   
>> -	for (i = 0; i < transaction->nr; i++) { +	for (int i 
>> = 0; i < transaction->nr; i++) { 
>>  		struct ref_update *update = 
>>  transaction->updates[i]; 
>> +		int ret; 
>>   if (update->flags & REF_LOG_ONLY) continue; 
> 
> Hm. In the "pre-push" hook you converted the callback to process 
> one ref per invocation. Why don't we do the same over here, with 
> one transaction per invocation? 
> 
> Not saying that either one of these is better, but it left me 
> puzzled why we use two different patterns now. 

Good catch! It's a good idea to make them consistent.

We should do it here like we did for pre-push, to avoid processing 
all data at once and risk blocking on the write side (one 
extreme).

However, writing just once/one-line per callback (the other 
extreme) might add unnecessary delay/waiting on the hook child 
side.

So I'll modify both to batch let's say 50-100 in one call, to 
ensure a good balance.

This ties into my previous message about how much data should we 
write in one batch to ensure good throughtput while also not 
blocking for too long. Hope it makes sense.

Will improve this in v2.

Again, many thanks for your careful review, really appreciate it.

  reply	other threads:[~2025-10-08 12:26 UTC|newest]

Thread overview: 137+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-09-25 12:53 [PATCH 00/10] Convert remaining hooks to hook.h Adrian Ratiu
2025-09-25 12:53 ` [PATCH 01/10] run-command: add stdin callback for parallelization Adrian Ratiu
2025-10-02  6:34   ` Patrick Steinhardt
2025-10-02 15:46     ` Junio C Hamano
2025-10-06 13:01       ` Adrian Ratiu
2025-10-06 12:59     ` Adrian Ratiu
2025-10-14 17:35       ` Adrian Ratiu
2025-09-25 12:53 ` [PATCH 02/10] hook: provide stdin via callback Adrian Ratiu
2025-09-25 20:05   ` Junio C Hamano
2025-09-26 12:03     ` Adrian Ratiu
2025-10-10 19:57   ` Emily Shaffer
2025-10-13 14:47     ` Adrian Ratiu
2025-09-25 12:53 ` [PATCH 03/10] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Adrian Ratiu
2025-09-25 20:15   ` Junio C Hamano
2025-09-26 12:29     ` Adrian Ratiu
2025-09-26 14:12   ` Phillip Wood
2025-09-26 15:53     ` Adrian Ratiu
2025-09-29 10:11       ` Phillip Wood
2025-09-26 17:52     ` Junio C Hamano
2025-09-29  7:33       ` Adrian Ratiu
2025-10-02  6:34   ` Patrick Steinhardt
2025-10-08  7:04     ` Adrian Ratiu
2025-09-25 12:53 ` [PATCH 04/10] transport: convert pre-push hook " Adrian Ratiu
2025-09-25 18:58   ` D. Ben Knoble
2025-09-26 13:02     ` Adrian Ratiu
2025-09-26 14:11   ` Phillip Wood
2025-09-29 11:33     ` Adrian Ratiu
2025-09-25 12:53 ` [PATCH 05/10] reference-transaction: use hook.h to run hooks Adrian Ratiu
2025-09-25 21:45   ` Junio C Hamano
2025-09-26 13:03     ` Adrian Ratiu
2025-10-02  6:34   ` Patrick Steinhardt
2025-10-08 12:26     ` Adrian Ratiu [this message]
2025-09-25 12:53 ` [PATCH 06/10] run-command: allow capturing of collated output Adrian Ratiu
2025-09-25 21:52   ` Junio C Hamano
2025-09-26 14:14     ` Adrian Ratiu
2025-09-25 12:53 ` [PATCH 07/10] hooks: allow callers to capture output Adrian Ratiu
2025-09-25 12:53 ` [PATCH 08/10] receive-pack: convert 'update' hook to hook.h Adrian Ratiu
2025-09-25 21:53   ` Junio C Hamano
2025-10-10 19:57   ` Emily Shaffer
2025-10-17  8:27     ` Adrian Ratiu
2025-09-25 12:53 ` [PATCH 09/10] post-update: use hook.h library Adrian Ratiu
2025-09-25 18:02 ` [PATCH 10/10] receive-pack: convert receive hooks to hook.h Adrian Ratiu
2025-10-10 19:57 ` [PATCH 00/10] Convert remaining " Emily Shaffer
2025-10-17 14:15 ` [PATCH v2 " Adrian Ratiu
2025-10-17 14:15   ` [PATCH v2 01/10] run-command: add stdin callback for parallelization Adrian Ratiu
2025-10-21  7:40     ` Patrick Steinhardt
2025-10-17 14:15   ` [PATCH v2 02/10] hook: provide stdin via callback Adrian Ratiu
2025-10-21  7:41     ` Patrick Steinhardt
2025-10-21  7:41     ` Patrick Steinhardt
2025-10-21 14:44       ` Adrian Ratiu
2025-10-17 14:15   ` [PATCH v2 03/10] hook: convert 'post-rewrite' hook in sequencer.c to hook API Adrian Ratiu
2025-10-21  7:41     ` Patrick Steinhardt
2025-10-21 15:44       ` Adrian Ratiu
2025-10-17 14:15   ` [PATCH v2 04/10] transport: convert pre-push " Adrian Ratiu
2025-10-21  7:41     ` Patrick Steinhardt
2025-10-21 16:04       ` Adrian Ratiu
2025-10-17 14:15   ` [PATCH v2 05/10] reference-transaction: use hook API instead of run-command Adrian Ratiu
2025-10-17 14:15   ` [PATCH v2 06/10] hook: allow overriding the ungroup option Adrian Ratiu
2025-10-17 14:15   ` [PATCH v2 07/10] run-command: allow capturing of collated output Adrian Ratiu
2025-10-21  7:41     ` Patrick Steinhardt
2025-10-21 16:25       ` Adrian Ratiu
2025-10-17 14:15   ` [PATCH v2 08/10] hooks: allow callers to capture output Adrian Ratiu
2025-10-17 14:15   ` [PATCH v2 09/10] receive-pack: convert update hooks to new API Adrian Ratiu
2025-10-28 18:39     ` Kristoffer Haugsbakk
2025-10-17 14:15   ` [PATCH v2 10/10] receive-pack: convert receive hooks to hook API Adrian Ratiu
2025-10-21  7:41     ` Patrick Steinhardt
2025-10-28 18:42     ` Kristoffer Haugsbakk
2025-10-29 13:46       ` Adrian Ratiu
2025-10-29 13:50         ` Kristoffer Haugsbakk
2025-11-15 19:48     ` Junio C Hamano
2025-11-17 16:51       ` Adrian Ratiu
2025-10-21  7:40   ` [PATCH v2 00/10] Convert remaining hooks to hook.h Patrick Steinhardt
2025-10-21 16:34     ` Adrian Ratiu
2025-11-24 17:20 ` [PATCH v3 " Adrian Ratiu
2025-11-24 17:20   ` [PATCH v3 01/10] run-command: add stdin callback for parallelization Adrian Ratiu
2025-11-25 23:15     ` Junio C Hamano
2025-11-27 12:00       ` Adrian Ratiu
2025-11-24 17:20   ` [PATCH v3 02/10] hook: provide stdin via callback Adrian Ratiu
2025-11-29 13:03     ` Adrian Ratiu
2025-11-29 22:21       ` Junio C Hamano
2025-12-01 13:26         ` Adrian Ratiu
2025-11-24 17:20   ` [PATCH v3 03/10] hook: convert 'post-rewrite' hook in sequencer.c to hook API Adrian Ratiu
2025-11-24 17:20   ` [PATCH v3 04/10] transport: convert pre-push " Adrian Ratiu
2025-11-24 22:55     ` Junio C Hamano
2025-11-27 14:24       ` Adrian Ratiu
2025-11-24 17:20   ` [PATCH v3 05/10] reference-transaction: use hook API instead of run-command Adrian Ratiu
2025-11-24 17:20   ` [PATCH v3 06/10] hook: allow overriding the ungroup option Adrian Ratiu
2025-11-24 17:20   ` [PATCH v3 07/10] run-command: allow capturing of collated output Adrian Ratiu
2025-11-24 17:20   ` [PATCH v3 08/10] hooks: allow callers to capture output Adrian Ratiu
2025-11-24 17:20   ` [PATCH v3 09/10] receive-pack: convert update hooks to new API Adrian Ratiu
2025-11-24 17:20   ` [PATCH v3 10/10] receive-pack: convert receive hooks to hook API Adrian Ratiu
2025-12-04 14:15 ` [PATCH v4 00/11] Convert remaining hooks to hook.h Adrian Ratiu
2025-12-04 14:15   ` [PATCH v4 01/11] run-command: add first helper for pp child states Adrian Ratiu
2025-12-04 14:15   ` [PATCH v4 02/11] run-command: add stdin callback for parallelization Adrian Ratiu
2025-12-04 14:15   ` [PATCH v4 03/11] hook: provide stdin via callback Adrian Ratiu
2025-12-16  8:08     ` Patrick Steinhardt
2025-12-04 14:15   ` [PATCH v4 04/11] hook: convert 'post-rewrite' hook in sequencer.c to hook API Adrian Ratiu
2025-12-04 14:15   ` [PATCH v4 05/11] transport: convert pre-push " Adrian Ratiu
2025-12-16  8:08     ` Patrick Steinhardt
2025-12-16  9:09       ` Adrian Ratiu
2025-12-16  9:30         ` Patrick Steinhardt
2025-12-17 23:07           ` Junio C Hamano
2025-12-04 14:15   ` [PATCH v4 06/11] reference-transaction: use hook API instead of run-command Adrian Ratiu
2025-12-04 14:15   ` [PATCH v4 07/11] hook: allow overriding the ungroup option Adrian Ratiu
2025-12-04 14:15   ` [PATCH v4 08/11] run-command: allow capturing of collated output Adrian Ratiu
2025-12-04 14:15   ` [PATCH v4 09/11] hooks: allow callers to capture output Adrian Ratiu
2025-12-04 14:15   ` [PATCH v4 10/11] receive-pack: convert update hooks to new API Adrian Ratiu
2025-12-16  8:08     ` Patrick Steinhardt
2025-12-16  9:22       ` Adrian Ratiu
2025-12-04 14:15   ` [PATCH v4 11/11] receive-pack: convert receive hooks to hook API Adrian Ratiu
2025-12-18 17:11 ` [PATCH v5 00/11] Convert remaining hooks to hook.h Adrian Ratiu
2025-12-18 17:11   ` [PATCH v5 01/11] run-command: add first helper for pp child states Adrian Ratiu
2025-12-18 17:11   ` [PATCH v5 02/11] run-command: add stdin callback for parallelization Adrian Ratiu
2025-12-18 17:11   ` [PATCH v5 03/11] hook: provide stdin via callback Adrian Ratiu
2025-12-18 17:11   ` [PATCH v5 04/11] hook: convert 'post-rewrite' hook in sequencer.c to hook API Adrian Ratiu
2025-12-18 17:11   ` [PATCH v5 05/11] transport: convert pre-push " Adrian Ratiu
2025-12-18 17:11   ` [PATCH v5 06/11] reference-transaction: use hook API instead of run-command Adrian Ratiu
2025-12-18 17:11   ` [PATCH v5 07/11] hook: allow overriding the ungroup option Adrian Ratiu
2025-12-18 17:11   ` [PATCH v5 08/11] run-command: allow capturing of collated output Adrian Ratiu
2025-12-18 17:11   ` [PATCH v5 09/11] hooks: allow callers to capture output Adrian Ratiu
2025-12-18 17:11   ` [PATCH v5 10/11] receive-pack: convert update hooks to new API Adrian Ratiu
2025-12-18 17:11   ` [PATCH v5 11/11] receive-pack: convert receive hooks to hook API Adrian Ratiu
2025-12-19 12:38     ` Patrick Steinhardt
2025-12-20 10:40       ` Adrian Ratiu
2025-12-26 12:23 ` [PATCH v6 00/11] Convert remaining hooks to hook.h Adrian Ratiu
2025-12-26 12:23   ` [PATCH v6 01/11] run-command: add first helper for pp child states Adrian Ratiu
2025-12-26 12:23   ` [PATCH v6 02/11] run-command: add stdin callback for parallelization Adrian Ratiu
2025-12-26 12:23   ` [PATCH v6 03/11] hook: provide stdin via callback Adrian Ratiu
2025-12-26 12:23   ` [PATCH v6 04/11] hook: convert 'post-rewrite' hook in sequencer.c to hook API Adrian Ratiu
2025-12-26 12:23   ` [PATCH v6 05/11] transport: convert pre-push " Adrian Ratiu
2025-12-26 12:23   ` [PATCH v6 06/11] reference-transaction: use hook API instead of run-command Adrian Ratiu
2025-12-26 12:23   ` [PATCH v6 07/11] hook: allow overriding the ungroup option Adrian Ratiu
2025-12-26 12:23   ` [PATCH v6 08/11] run-command: allow capturing of collated output Adrian Ratiu
2025-12-26 12:23   ` [PATCH v6 09/11] hooks: allow callers to capture output Adrian Ratiu
2025-12-26 12:23   ` [PATCH v6 10/11] receive-pack: convert update hooks to new API Adrian Ratiu
2025-12-26 12:23   ` [PATCH v6 11/11] receive-pack: convert receive hooks to hook API Adrian Ratiu
2025-12-28 11:32   ` [PATCH v6 00/11] Convert remaining hooks to hook.h Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87v7kp37gn.fsf@collabora.com \
    --to=adrian.ratiu@collabora.com \
    --cc=avarab@gmail.com \
    --cc=emilyshaffer@google.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=ps@pks.im \
    --cc=rdamazio@google.com \
    --cc=steadmon@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).