All of lore.kernel.org
 help / color / mirror / Atom feed
From: Patrick Steinhardt <ps@pks.im>
To: Jeff King <peff@peff.net>
Cc: git@vger.kernel.org, Matt Smiley <msmiley@gitlab.com>,
	"brian m. carlson" <sandals@crustytoothpaste.net>,
	Junio C Hamano <gitster@pobox.com>
Subject: Re: [PATCH 2/2] upload-pack: reduce lock contention when writing packfile data
Date: Tue, 3 Mar 2026 14:47:08 +0100	[thread overview]
Message-ID: <aabmXF5D1rMRZgP3@pks.im> (raw)
In-Reply-To: <20260303133540.GA818878@coredump.intra.peff.net>

On Tue, Mar 03, 2026 at 08:35:40AM -0500, Jeff King wrote:
> On Tue, Mar 03, 2026 at 10:31:46AM +0100, Patrick Steinhardt wrote:
> > We would likely hit this issue if we insist on the buffer being
> > completely filled before sending it out. But that's why I adapted the
> > logic to say that we send out once we've filled it at least 2/3rds of
> > the pktline limit. So in your case above we wouldn't face an issue as
> > we'd already send the first 50kB, as it is smaller than 2/3rds of the
> > maximum length (~42kB).
> > 
> > That being said, you'll still be able to construct cases where we have
> > weird edge cases. For example if you consistently send one byte less
> > than 2/3rds of the capacity.
> 
> Right, my numbers were just meant as examples. Whatever the values, it
> means that whatever is generating the pack data (pack-objects or
> otherwise) really wants to be in sync with how upload-pack is buffering.
> Or vice versa. If we just pass back whole chunks of what we read() in
> upload-pack, then that happens automatically.

Yeah, that's fair. I'll include this part in v2 for now so that we can
discuss once the bigger picture emerges :)

Thanks!

Patrick

  reply	other threads:[~2026-03-03 13:47 UTC|newest]

Thread overview: 64+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-02-27 11:22 [PATCH 0/2] upload-pack: reduce lock contention when writing packfile data Patrick Steinhardt
2026-02-27 11:23 ` [PATCH 1/2] upload-pack: fix debug statement when flushing " Patrick Steinhardt
2026-02-27 11:23 ` [PATCH 2/2] upload-pack: reduce lock contention when writing " Patrick Steinhardt
2026-02-27 13:04   ` brian m. carlson
2026-02-27 18:14     ` Patrick Steinhardt
2026-02-27 17:29   ` Junio C Hamano
2026-02-27 19:37   ` Jeff King
2026-03-02 12:12     ` Patrick Steinhardt
2026-03-02 18:20       ` Jeff King
2026-03-03  9:31         ` Patrick Steinhardt
2026-03-03 13:35           ` Jeff King
2026-03-03 13:47             ` Patrick Steinhardt [this message]
2026-03-03 15:00 ` [PATCH v2 00/10] " Patrick Steinhardt
2026-03-03 15:00   ` [PATCH v2 01/10] upload-pack: fix debug statement when flushing " Patrick Steinhardt
2026-03-03 15:00   ` [PATCH v2 02/10] upload-pack: adapt keepalives based on buffering Patrick Steinhardt
2026-03-05  0:56     ` Jeff King
2026-03-10 12:08       ` Patrick Steinhardt
2026-03-03 15:00   ` [PATCH v2 03/10] upload-pack: reduce lock contention when writing packfile data Patrick Steinhardt
2026-03-05  1:16     ` Jeff King
2026-03-10 12:14       ` Patrick Steinhardt
2026-03-03 15:00   ` [PATCH v2 04/10] git-compat-util: introduce `cast_size_t_to_ssize_t()` Patrick Steinhardt
2026-03-03 15:00   ` [PATCH v2 05/10] compat/posix: introduce writev(3p) wrapper Patrick Steinhardt
2026-03-04 22:01     ` Junio C Hamano
2026-03-05  0:37       ` Jeff King
2026-03-05  2:16         ` brian m. carlson
2026-03-05  6:39           ` Johannes Sixt
2026-03-05 22:22             ` brian m. carlson
2026-03-10 12:09               ` Patrick Steinhardt
2026-03-03 15:00   ` [PATCH v2 06/10] wrapper: introduce writev(3p) wrappers Patrick Steinhardt
2026-03-03 15:00   ` [PATCH v2 07/10] sideband: use writev(3p) to send pktlines Patrick Steinhardt
2026-03-04 22:05     ` Junio C Hamano
2026-03-03 15:00   ` [PATCH v2 08/10] csum-file: introduce `hashfd_ext()` Patrick Steinhardt
2026-03-04 22:11     ` Junio C Hamano
2026-03-10 12:09       ` Patrick Steinhardt
2026-03-03 15:00   ` [PATCH v2 09/10] csum-file: drop `hashfd_throughput()` Patrick Steinhardt
2026-03-03 15:00   ` [PATCH v2 10/10] builtin/pack-objects: reduce lock contention when writing packfile data Patrick Steinhardt
2026-03-10 13:24 ` [PATCH v3 00/10] upload-pack: " Patrick Steinhardt
2026-03-10 13:24   ` [PATCH v3 01/10] upload-pack: fix debug statement when flushing " Patrick Steinhardt
2026-03-10 13:24   ` [PATCH v3 02/10] upload-pack: adapt keepalives based on buffering Patrick Steinhardt
2026-03-10 13:24   ` [PATCH v3 03/10] upload-pack: prefer flushing data over sending keepalive Patrick Steinhardt
2026-03-10 17:09     ` Junio C Hamano
2026-03-10 17:43       ` Patrick Steinhardt
2026-03-10 13:25   ` [PATCH v3 04/10] upload-pack: reduce lock contention when writing packfile data Patrick Steinhardt
2026-03-10 13:25   ` [PATCH v3 05/10] compat/posix: introduce writev(3p) wrapper Patrick Steinhardt
2026-03-10 16:59     ` Junio C Hamano
2026-03-10 13:25   ` [PATCH v3 06/10] wrapper: introduce writev(3p) wrappers Patrick Steinhardt
2026-03-10 13:25   ` [PATCH v3 07/10] sideband: use writev(3p) to send pktlines Patrick Steinhardt
2026-03-10 13:25   ` [PATCH v3 08/10] csum-file: introduce `hashfd_ext()` Patrick Steinhardt
2026-03-10 13:25   ` [PATCH v3 09/10] csum-file: drop `hashfd_throughput()` Patrick Steinhardt
2026-03-10 13:25   ` [PATCH v3 10/10] builtin/pack-objects: reduce lock contention when writing packfile data Patrick Steinhardt
2026-03-10 17:11   ` [PATCH v3 00/10] upload-pack: " Junio C Hamano
2026-03-10 20:56   ` Johannes Sixt
2026-03-11  6:27     ` Patrick Steinhardt
2026-03-13  6:45 ` [PATCH v4 " Patrick Steinhardt
2026-03-13  6:45   ` [PATCH v4 01/10] upload-pack: fix debug statement when flushing " Patrick Steinhardt
2026-03-13  6:45   ` [PATCH v4 02/10] upload-pack: adapt keepalives based on buffering Patrick Steinhardt
2026-03-13  6:45   ` [PATCH v4 03/10] upload-pack: prefer flushing data over sending keepalive Patrick Steinhardt
2026-03-13  6:45   ` [PATCH v4 04/10] upload-pack: reduce lock contention when writing packfile data Patrick Steinhardt
2026-03-13  6:45   ` [PATCH v4 05/10] compat/posix: introduce writev(3p) wrapper Patrick Steinhardt
2026-03-13  6:45   ` [PATCH v4 06/10] wrapper: introduce writev(3p) wrappers Patrick Steinhardt
2026-03-13  6:45   ` [PATCH v4 07/10] sideband: use writev(3p) to send pktlines Patrick Steinhardt
2026-03-13  6:45   ` [PATCH v4 08/10] csum-file: introduce `hashfd_ext()` Patrick Steinhardt
2026-03-13  6:45   ` [PATCH v4 09/10] csum-file: drop `hashfd_throughput()` Patrick Steinhardt
2026-03-13  6:45   ` [PATCH v4 10/10] builtin/pack-objects: reduce lock contention when writing packfile data Patrick Steinhardt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aabmXF5D1rMRZgP3@pks.im \
    --to=ps@pks.im \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=msmiley@gitlab.com \
    --cc=peff@peff.net \
    --cc=sandals@crustytoothpaste.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.