From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
To: Patrick Steinhardt <ps@pks.im>
Cc: Johannes Schindelin via GitGitGadget <gitgitgadget@gmail.com>,
git@vger.kernel.org
Subject: Re: [PATCH 12/12] git-zlib: widen `git_deflate_bound()` to `size_t`
Date: Wed, 5 Aug 2026 15:58:43 +0200 (CEST) [thread overview]
Message-ID: <7fe7ca44-f3c4-7e99-b7fe-81e8467e294e@gmx.de> (raw)
In-Reply-To: <anMBBW_arzuri4Qo@pks.im>
Hi Patrick,
On Wed, 5 Aug 2026, Patrick Steinhardt wrote:
> On Thu, Jul 09, 2026 at 04:49:39PM +0000, Johannes Schindelin via GitGitGadget wrote:
> > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> >
> > All four `unsigned long`/`int`/`ssize_t` receivers across archive-zip,
> > diff, http-push and t/helper/test-pack-deltas were widened to `size_t`
> > in the prior commits, and remote-curl and fast-import were already
> > there. With every caller prepared, both the parameter and the return
> > type can now move without introducing any silent narrowing.
>
> Nit, feel free to ignore: I feel like all of these patches could've been
> squashed into a single one, as they're trivial enough.
I like them trivial and small ;-)
> > For inputs above zlib's `uLong` range (i.e. >4 GiB on platforms where
> > `uLong` is 32-bit, notably 64-bit Windows), defer to zlib's stored-block
> > formula (the same fallback it would itself use for an unknown stream
> > state) plus the worst-case wrapper overhead. The existing path through
> > `deflateBound()` is unchanged for inputs that fit.
>
> A link or something like that to the formula would've helped here, as
> I'm not familiar with this mechanism.
Right. I added two references to the commit message.
>
> > diff --git a/git-zlib.c b/git-zlib.c
> > index d21adb3bf5..ebbbcc6d1a 100644
> > --- a/git-zlib.c
> > +++ b/git-zlib.c
> > @@ -167,9 +167,21 @@ int git_inflate(git_zstream *strm, int flush)
> > return status;
> > }
> >
> > -unsigned long git_deflate_bound(git_zstream *strm, unsigned long size)
> > +size_t git_deflate_bound(git_zstream *strm, size_t size)
> > {
> > - return deflateBound(&strm->z, size);
> > +#if SIZE_MAX > ULONG_MAX
> > + if (size > maximum_unsigned_value_of_type(uLong))
> > + /*
> > + * deflateBound() takes uLong, which is 32-bit on
> > + * Windows. For inputs above that range, return zlib's
> > + * stored-block formula (the conservative path it would
> > + * itself use for an unknown stream state) plus the
> > + * worst-case wrapper overhead.
> > + */
> > + return size + (size >> 5) + (size >> 7) + (size >> 11)
> > + + 7 + 18;
> > +#endif
>
> So is the idea here that we estimate the highest number of bytes that
> the deflated size could end up with?
Precisely. And the formula in zlib is a bit complex, it calculates a
"fixedlen" and a "storelen" for two different ways to represent the worst
case size. But for large values, only `storelen` matters, therefore we can
get away with a much simpler logic here.
FWIW zlib v1.3.2 added `deflateBound_z()`, which accepts `size_t` (or more
precisely: `z_size_t`). However, v1.3.2 is only 7 months old, so I'll be
retired by the time Debian stable gets it :-P
Ciao,
Johannes
>
> Patrick
>
next prev parent reply other threads:[~2026-08-05 13:58 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
2026-08-05 9:22 ` Patrick Steinhardt
2026-08-05 13:51 ` Johannes Schindelin
2026-07-09 16:49 ` [PATCH 02/12] delta: widen `create_delta_index()` parameter " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 03/12] pack-objects: widen delta-cache accounting " Johannes Schindelin via GitGitGadget
2026-08-05 9:23 ` Patrick Steinhardt
2026-08-05 13:52 ` Johannes Schindelin
2026-07-09 16:49 ` [PATCH 04/12] pack-objects: widen `free_unpacked()` return " Johannes Schindelin via GitGitGadget
2026-08-05 9:23 ` Patrick Steinhardt
2026-07-09 16:49 ` [PATCH 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 06/12] delta: widen `create_delta()` and `diff_delta()` " Johannes Schindelin via GitGitGadget
2026-08-05 9:23 ` Patrick Steinhardt
2026-07-09 16:49 ` [PATCH 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 09/12] diff: widen `deflate_it()`'s bound local from int " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 10/12] http-push: widen `start_put()`'s size local from `ssize_t` " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 12/12] git-zlib: widen `git_deflate_bound()` " Johannes Schindelin via GitGitGadget
2026-08-05 9:23 ` Patrick Steinhardt
2026-08-05 13:58 ` Johannes Schindelin [this message]
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
2026-08-06 6:15 ` Patrick Steinhardt
2026-08-05 16:14 ` [PATCH v2 02/12] delta: widen `create_delta_index()` parameter " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 03/12] pack-objects: widen delta-cache accounting " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 04/12] pack-objects: widen `free_unpacked()` return " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 06/12] delta: widen `create_delta()` and `diff_delta()` " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields " Johannes Schindelin via GitGitGadget
2026-08-07 21:41 ` Junio C Hamano
2026-08-07 22:06 ` Junio C Hamano
2026-08-05 16:14 ` [PATCH v2 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 09/12] diff: widen `deflate_it()`'s bound local from int " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 10/12] http-push: widen `start_put()`'s size local from `ssize_t` " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 12/12] git-zlib: widen `git_deflate_bound()` " Johannes Schindelin via GitGitGadget
2026-08-06 6:15 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Patrick Steinhardt
2026-08-06 17:30 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7fe7ca44-f3c4-7e99-b7fe-81e8467e294e@gmx.de \
--to=johannes.schindelin@gmx.de \
--cc=git@vger.kernel.org \
--cc=gitgitgadget@gmail.com \
--cc=ps@pks.im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox