public inbox for git@vger.kernel.org
 help / color / mirror / Atom feed
From: Adrian Ratiu <adrian.ratiu@collabora.com>
To: Jeff King <peff@peff.net>
Cc: git@vger.kernel.org, Junio C Hamano <gitster@pobox.com>,
	Patrick Steinhardt <ps@pks.im>,
	Emily Shaffer <emilyshaffer@google.com>,
	Kristoffer Haugsbakk <kristofferhaugsbakk@fastmail.com>,
	Chris Darroch <chrisd@apache.org>,
	"brian m. carlson" <sandals@crustytoothpaste.net>
Subject: Re: [PATCH v2] hook: allow hooks to disable stdout_to_stderr
Date: Wed, 14 Jan 2026 19:56:23 +0200	[thread overview]
Message-ID: <87tswokri0.fsf@gentoo.mail-host-address-is-not-set> (raw)
In-Reply-To: <20260114171929.GC885771@coredump.intra.peff.net>

On Wed, 14 Jan 2026, Jeff King <peff@peff.net> wrote:
> On Wed, Jan 14, 2026 at 12:08:49PM -0500, Jeff King wrote:
>
>> I looked at what feed_receive_hook_cb() is doing and...it's kind of
>> horrifying. It arbitrarily sends 500 lines, and then yields to the
>> caller to pump stderr (assuming ungroup=0). So:
>> 
>>   1. It is assuming that 500 lines of input won't fill up the pipe
>>      buffer and block. Even if we compute the size of 500 lines we're
>>      sending, we don't know if the caller has cleared anything from the
>>      pipe in the last call. There might be zero bytes available!
>> 
>>   2. After 500 lines we'll go back to the caller, which will then
>>      poll(). But if there's nothing to read on stderr, it will wait for
>>      the 100ms timeout. So if you have, say, 501 lines to send, then
>>      there will be a pointless 100ms pause in the middle.
>> 
>> So here's an example hook setup that will deadlock due to (1):
>
> And just for fun, here's an example that shows problem (2):
>
> -- >8 --
> rm -rf repo
> git init repo
> cd repo
> git commit --allow-empty -m foo
> git init --bare dst.git
>
> cat >dst.git/hooks/pre-receive <<\EOF
> #!/bin/sh
> # We don't even need to do anything interesting here! Git
> # will send us 500 lines, then block waiting for stderr which
> # we'll never send, and then send us another batch of 500.
> cat >/dev/null
> EOF
> chmod +x dst.git/hooks/pre-receive
>
> # Now do a moderate push of 500 branches.
> seq --format='create refs/heads/small-%g HEAD' 500 |
> git update-ref --stdin
> time git push -q dst.git refs/heads/small-*
>
> # And compare with one that sends just one more.
> seq --format='create refs/heads/large-%g HEAD' 501 |
> git update-ref --stdin
> time git push -q dst.git refs/heads/large-*
> -- >8 --
>
> The second push always takes 100ms more! If we run the server side under
> strace by replacing the final line with this:
>
>   git push -q --receive-pack='strace -T git-receive-pack' dst.git refs/heads/large-*
>
> we can see the stall here as we write to the hook:
>
>   write(4, "00000000000000000000000000000000"..., 51393) = 51393 <0.000011>
>   poll([{fd=5, events=POLLIN|POLLHUP}], 1, 100) = 0 (Timeout) <0.100506>
>   write(4, "00000000000000000000000000000000"..., 102) = 102 <0.000057>
>
> That would likewise be solved by using ungroup=1 (in which case we do
> not poll, but just call the feed function immediately again) or by using
> a real poll() loop (which would see immediately that the hook is ready
> for more input, rather than hitting the 100ms timeout).

Thanks for the detailed examples.

For the server-side hooks, I think the way forward is to implement the
poll loop as you suggested so we can buffer stderr and for a single
(non-parallel) hook, we can keep the existing behavior (still need to
test this). I'll do that in a separate patch and drop the batching.

For the client side hooks, I'll send v3 of this series which fixes the
two regressions reported by Chris and Kristoffer.

  reply	other threads:[~2026-01-14 17:56 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-01-13 11:56 [PATCH] hook: make stdout_to_stderr optional Adrian Ratiu
2026-01-13 13:36 ` Patrick Steinhardt
2026-01-13 13:55   ` Adrian Ratiu
2026-01-13 14:00 ` Junio C Hamano
2026-01-13 14:06   ` Junio C Hamano
2026-01-13 14:59     ` Adrian Ratiu
2026-01-13 15:22       ` Junio C Hamano
2026-01-13 15:37         ` Adrian Ratiu
2026-01-13 14:11   ` Adrian Ratiu
2026-01-13 23:45 ` [PATCH v2] hook: allow hooks to disable stdout_to_stderr Adrian Ratiu
2026-01-14  3:12   ` Jeff King
2026-01-14  8:46     ` Adrian Ratiu
2026-01-14  8:59       ` Adrian Ratiu
2026-01-14  9:36       ` Kristoffer Haugsbakk
2026-01-14 17:08       ` Jeff King
2026-01-14 17:19         ` Jeff King
2026-01-14 17:56           ` Adrian Ratiu [this message]
2026-01-14  6:13   ` Kristoffer Haugsbakk
2026-01-14 18:57 ` [PATCH v3 0/2] Fix two hook conversion regressions Adrian Ratiu
2026-01-14 18:57   ` [PATCH v3 1/2] hook: allow hooks to disable stdout_to_stderr Adrian Ratiu
2026-01-14 18:57   ` [PATCH v3 2/2] hook: make ungroup opt-out instead of opt-in Adrian Ratiu
2026-01-14 21:27     ` Jeff King
2026-01-14 22:45       ` Adrian Ratiu
2026-01-18  8:44     ` Kristoffer Haugsbakk
2026-01-15 14:15   ` [PATCH v3 0/2] Fix two hook conversion regressions Junio C Hamano
2026-01-15 17:19     ` Adrian Ratiu
2026-01-15 17:33       ` Junio C Hamano
2026-01-15 17:53         ` Adrian Ratiu
2026-01-15 20:27           ` Junio C Hamano
2026-01-15 21:24             ` Adrian Ratiu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87tswokri0.fsf@gentoo.mail-host-address-is-not-set \
    --to=adrian.ratiu@collabora.com \
    --cc=chrisd@apache.org \
    --cc=emilyshaffer@google.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=kristofferhaugsbakk@fastmail.com \
    --cc=peff@peff.net \
    --cc=ps@pks.im \
    --cc=sandals@crustytoothpaste.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox