[PATCH] zlib: properly clamp to uLong

[PATCH] zlib: properly clamp to uLong

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

On platforms where `unsigned long` and `size_t` differ in bit size, we
want to clamp the buffers we pass to zlib to the former's size, as per
d05d666977 (git-zlib: handle data streams larger than 4GB, 2026-05-08).

The logic introduced in that commit performs a clamping to the bits,
though, which fails to do what is needed here: If too many bytes are
available in the buffers, we need to clamp to the maximum value of an
`unsigned long`. Otherwise, we ask zlib to use too small buffers, in the
worst case using 0 as the size (think: a value whose 32 lowest bits are
all zero).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
    zlib: properly clamp to uLong
    
    I re-read this logic earlier this week... and I am quite convinced that
    it needs to be fixed.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2153%2Fdscho%2Ffix-ulong-clamping-for-zlib-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2153/dscho/fix-ulong-clamping-for-zlib-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2153

 git-zlib.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/git-zlib.c b/git-zlib.c
index b91cb323ae..d21adb3bf5 100644
--- a/git-zlib.c
+++ b/git-zlib.c
@@ -38,12 +38,17 @@ static inline uInt zlib_buf_cap(unsigned long len)
 	return (ZLIB_BUF_MAX < len) ? ZLIB_BUF_MAX : len;
 }
 
+static inline uLong zlib_uLong_cap(size_t s)
+{
+	return s < ULONG_MAX_VALUE ? (uLong)s : ULONG_MAX_VALUE;
+}
+
 static void zlib_pre_call(git_zstream *s)
 {
 	s->z.next_in = s->next_in;
 	s->z.next_out = s->next_out;
-	s->z.total_in = (uLong)(s->total_in & ULONG_MAX_VALUE);
-	s->z.total_out = (uLong)(s->total_out & ULONG_MAX_VALUE);
+	s->z.total_in = zlib_uLong_cap(s->total_in);
+	s->z.total_out = zlib_uLong_cap(s->total_out);
 	s->z.avail_in = zlib_buf_cap(s->avail_in);
 	s->z.avail_out = zlib_buf_cap(s->avail_out);
 }
@@ -60,7 +65,7 @@ static void zlib_post_call(git_zstream *s, int status)
 	 * We track our own totals and verify only the low bits match.
 	 */
 	if ((s->z.total_out & ULONG_MAX_VALUE) !=
-	    ((s->total_out + bytes_produced) & ULONG_MAX_VALUE))
+	    ((zlib_uLong_cap(s->total_out) + bytes_produced) & ULONG_MAX_VALUE))
 		BUG("total_out mismatch");
 	/*
 	 * zlib does not update total_in when it returns Z_NEED_DICT,
@@ -68,7 +73,7 @@ static void zlib_post_call(git_zstream *s, int status)
 	 */
 	if (status != Z_NEED_DICT &&
 	    (s->z.total_in & ULONG_MAX_VALUE) !=
-	    ((s->total_in + bytes_consumed) & ULONG_MAX_VALUE))
+	    ((zlib_uLong_cap(s->total_in) + bytes_consumed) & ULONG_MAX_VALUE))
 		BUG("total_in mismatch");
 
 	s->total_out += bytes_produced;

base-commit: 7a094d68a27e321a99c8ab6b700909e503904bd9
-- 
gitgitgadget

Re: [PATCH] zlib: properly clamp to uLong

[Junio C Hamano]

The original in js/objects-larger-than-4gb-on-windows says things
like:

+	s->z.total_in = (uLong)(s->total_in & ULONG_MAX_VALUE);
+	s->z.total_out = (uLong)(s->total_out & ULONG_MAX_VALUE);

Your patch ...

> +static inline uLong zlib_uLong_cap(size_t s)
> +{
> +	return s < ULONG_MAX_VALUE ? (uLong)s : ULONG_MAX_VALUE;
> +}
> +
>  static void zlib_pre_call(git_zstream *s)
>  {
>  	s->z.next_in = s->next_in;
>  	s->z.next_out = s->next_out;
> -	s->z.total_in = (uLong)(s->total_in & ULONG_MAX_VALUE);
> -	s->z.total_out = (uLong)(s->total_out & ULONG_MAX_VALUE);
> +	s->z.total_in = zlib_uLong_cap(s->total_in);
> +	s->z.total_out = zlib_uLong_cap(s->total_out);

... is an obvious fix for that.

> @@ -60,7 +65,7 @@ static void zlib_post_call(git_zstream *s, int status)
>  	 * We track our own totals and verify only the low bits match.
>  	 */
>  	if ((s->z.total_out & ULONG_MAX_VALUE) !=
> -	    ((s->total_out + bytes_produced) & ULONG_MAX_VALUE))
> +	    ((zlib_uLong_cap(s->total_out) + bytes_produced) & ULONG_MAX_VALUE))
>  		BUG("total_out mismatch");

Because we now clamp (not "taking lower bits of") s->total_out to a
value between 0..4GB and store it in s->z.total_out in pre-call, let
zlib do its thing that increments s->z.total_out modulo 4GB, and we
clamp the s->total_out (before incrementing) the same way in
post_call here, both sides of "!=" above even out.  But the comment
before this comparison that claims that "we ... verify only the low
bits match" is a bit off the reality, I suspect.

> @@ -68,7 +73,7 @@ static void zlib_post_call(git_zstream *s, int status)
>  	 */
>  	if (status != Z_NEED_DICT &&
>  	    (s->z.total_in & ULONG_MAX_VALUE) !=
> -	    ((s->total_in + bytes_consumed) & ULONG_MAX_VALUE))
> +	    ((zlib_uLong_cap(s->total_in) + bytes_consumed) & ULONG_MAX_VALUE))
>  		BUG("total_in mismatch");
>  
>  	s->total_out += bytes_produced;
>
> base-commit: 7a094d68a27e321a99c8ab6b700909e503904bd9