All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 00/12] Next size_t stop: pack-objects/delta
@ 2026-07-09 16:49 Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
                   ` (11 more replies)
  0 siblings, 12 replies; 13+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
  To: git; +Cc: Johannes Schindelin

This patch series continues the effort to stop using unsigned long where
size_t should have been used in the first place. This makes a difference on
64-bit Windows, where unsigned long is 32-bit.

With these fixes, the pack-objects machinery works as intended on 64-bit
Windows (and any other 64-bit platform where unsigned long isn't 64-bit).

Johannes Schindelin (12):
  diff-delta: widen `struct delta_index`' size fields to `size_t`
  delta: widen `create_delta_index()` parameter to `size_t`
  pack-objects: widen delta-cache accounting to `size_t`
  pack-objects: widen `free_unpacked()` return to `size_t`
  pack-objects: widen `mem_usage` and `try_delta()`'s out-param to
    `size_t`
  delta: widen `create_delta()` and `diff_delta()` to `size_t`
  packfile, git-zlib: widen `use_pack()` and zstream avail fields to
    `size_t`
  archive-zip: widen `zlib_deflate_raw()`'s maxsize local to `size_t`
  diff: widen `deflate_it()`'s bound local from int to `size_t`
  http-push: widen `start_put()`'s size local from `ssize_t` to `size_t`
  t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local to
    `size_t`
  git-zlib: widen `git_deflate_bound()` to `size_t`

 archive-zip.c               |  2 +-
 builtin/fast-import.c       |  6 ++++--
 builtin/pack-objects.c      | 30 ++++++++++++++++--------------
 delta.h                     | 12 ++++++------
 diff-delta.c                | 12 ++++++------
 diff.c                      |  6 ++++--
 git-zlib.c                  | 16 ++++++++++++++--
 git-zlib.h                  |  6 +++---
 http-push.c                 |  2 +-
 pack-check.c                |  4 ++--
 packfile.c                  |  4 ++--
 packfile.h                  |  3 ++-
 t/helper/test-delta.c       |  2 +-
 t/helper/test-pack-deltas.c |  7 ++++---
 14 files changed, 66 insertions(+), 46 deletions(-)


base-commit: f85a7e662054a7b0d9070e432508831afa214b47
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2175%2Fdscho%2Fsize-t%2Fpack-objects-delta-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2175/dscho/size-t/pack-objects-delta-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2175
-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t`
  2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 02/12] delta: widen `create_delta_index()` parameter " Johannes Schindelin via GitGitGadget
                   ` (10 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
  To: git; +Cc: Johannes Schindelin, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Preparation for widening the delta-encoding API to `size_t` in
subsequent commits, which is what lets pack-objects drop the
`cast_size_t_to_ulong()` shims that 606c192380 (odb, packfile: use
size_t for streaming object sizes, 2026-05-08) had to leave behind in
`get_delta()` and `try_delta()` because their downstream consumers were
still narrow.

The struct is private to diff-delta.c, so widening its fields in
isolation is a no-op at runtime: the values stored continue to fit in 32
bits on Windows because the public API around it still truncates.
Splitting it out keeps the API-change commit focused on caller updates.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 diff-delta.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/diff-delta.c b/diff-delta.c
index 43c339f010..b6b65d7607 100644
--- a/diff-delta.c
+++ b/diff-delta.c
@@ -125,9 +125,9 @@ struct unpacked_index_entry {
 };
 
 struct delta_index {
-	unsigned long memsize;
+	size_t memsize;
 	const void *src_buf;
-	unsigned long src_size;
+	size_t src_size;
 	unsigned int hash_mask;
 	struct index_entry *hash[FLEX_ARRAY];
 };
@@ -140,7 +140,7 @@ struct delta_index * create_delta_index(const void *buf, unsigned long bufsize)
 	struct unpacked_index_entry *entry, **hash;
 	struct index_entry *packed_entry, **packed_hash;
 	void *mem;
-	unsigned long memsize;
+	size_t memsize;
 
 	if (!buf || !bufsize)
 		return NULL;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 02/12] delta: widen `create_delta_index()` parameter to `size_t`
  2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 03/12] pack-objects: widen delta-cache accounting " Johannes Schindelin via GitGitGadget
                   ` (9 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
  To: git; +Cc: Johannes Schindelin, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The sole caller (`try_delta()` in builtin/pack-objects.c) passes an
`unsigned long`, which promotes safely, so no caller fixups are needed.
Splitting it out keeps the `diff_delta()`/`create_delta()` widening,
which does ripple to several callers, in its own commit.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 delta.h      | 2 +-
 diff-delta.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/delta.h b/delta.h
index eb5c6d2fdb..a19586d789 100644
--- a/delta.h
+++ b/delta.h
@@ -14,7 +14,7 @@ struct delta_index;
  * using free_delta_index().
  */
 struct delta_index *
-create_delta_index(const void *buf, unsigned long bufsize);
+create_delta_index(const void *buf, size_t bufsize);
 
 /*
  * free_delta_index: free the index created by create_delta_index()
diff --git a/diff-delta.c b/diff-delta.c
index b6b65d7607..c93ac42594 100644
--- a/diff-delta.c
+++ b/diff-delta.c
@@ -132,7 +132,7 @@ struct delta_index {
 	struct index_entry *hash[FLEX_ARRAY];
 };
 
-struct delta_index * create_delta_index(const void *buf, unsigned long bufsize)
+struct delta_index * create_delta_index(const void *buf, size_t bufsize)
 {
 	unsigned int i, hsize, hmask, entries, prev_val, *hash_count;
 	const unsigned char *data, *buffer = buf;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 03/12] pack-objects: widen delta-cache accounting to `size_t`
  2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 02/12] delta: widen `create_delta_index()` parameter " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 04/12] pack-objects: widen `free_unpacked()` return " Johannes Schindelin via GitGitGadget
                   ` (8 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
  To: git; +Cc: Johannes Schindelin, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

These three are a single accounting tuple (the globals tracking
cumulative cached-delta bytes, plus the helper that compares them
against an incoming delta size) and are latently 32-bit on Windows where
`unsigned long` != `size_t`: a pack with many large cached deltas could
wrap silently.

The widening is internally consistent on its own: the additions and
subtractions against delta_cache_size already come from `size_t` sources
(`DELTA_SIZE()` returns `size_t`), and `delta_cacheable()`'s sole caller
in `try_delta()` still passes `unsigned long`, which promotes.

Prerequisite for dropping `try_delta()`'s `cast_size_t_to_ulong()`
shims, which becomes possible once 1create_delta()` and `diff_delta()`
are widened in a later commit.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/pack-objects.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index e3760b3492..f89628a760 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -260,8 +260,8 @@ static int exclude_promisor_objects_best_effort;
 
 static int use_delta_islands;
 
-static unsigned long delta_cache_size = 0;
-static unsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;
+static size_t delta_cache_size = 0;
+static size_t max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;
 static unsigned long cache_max_small_delta_size = 1000;
 
 static unsigned long window_memory_limit = 0;
@@ -2688,8 +2688,8 @@ struct unpacked {
 	unsigned depth;
 };
 
-static int delta_cacheable(unsigned long src_size, unsigned long trg_size,
-			   unsigned long delta_size)
+static int delta_cacheable(size_t src_size, size_t trg_size,
+			   size_t delta_size)
 {
 	if (max_delta_cache_size && delta_cache_size + delta_size > max_delta_cache_size)
 		return 0;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 04/12] pack-objects: widen `free_unpacked()` return to `size_t`
  2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
                   ` (2 preceding siblings ...)
  2026-07-09 16:49 ` [PATCH 03/12] pack-objects: widen delta-cache accounting " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param " Johannes Schindelin via GitGitGadget
                   ` (7 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
  To: git; +Cc: Johannes Schindelin, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

`free_unpacked()` sums two byte counts: `sizeof_delta_index()` and
`SIZE(n->entry)`. The latter has been `size_t` since the prior topic
"More work supporting objects larger than 4GB on Windows" widened
`SIZE()`/`oe_size()` to `size_t`, so accumulating it into an `unsigned
long` return was a silent Windows-only truncation on a packing run with
many large objects.

The sole caller, `find_deltas()`, still holds its own `mem_usage` in an
`unsigned long` for now, and therefore still truncates silently.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/pack-objects.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index f89628a760..4737a6a32c 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -2972,9 +2972,9 @@ static unsigned int check_delta_limit(struct object_entry *me, unsigned int n)
 	return m;
 }
 
-static unsigned long free_unpacked(struct unpacked *n)
+static size_t free_unpacked(struct unpacked *n)
 {
-	unsigned long freed_mem = sizeof_delta_index(n->index);
+	size_t freed_mem = sizeof_delta_index(n->index);
 	free_delta_index(n->index);
 	n->index = NULL;
 	if (n->data) {
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param to `size_t`
  2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
                   ` (3 preceding siblings ...)
  2026-07-09 16:49 ` [PATCH 04/12] pack-objects: widen `free_unpacked()` return " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 06/12] delta: widen `create_delta()` and `diff_delta()` " Johannes Schindelin via GitGitGadget
                   ` (6 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
  To: git; +Cc: Johannes Schindelin, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The pair must move together because `find_deltas()` passes `&mem_usage`
to `try_delta()`: widening either alone breaks the type match.

`mem_usage` accumulates per-object byte counts already computed in
`size_t` (`SIZE()` and `sizeof_delta_index()` reach here through
`free_unpacked()`, now `size_t`), and was the last 32-bit-on-Windows
narrowing point in the delta-window memory accounting chain. With this
commit, that chain uses `size_t` consistently except for
`sizeof_delta_index()`'s still-narrow return, whose value is bounded by
`create_delta_index()`'s entries cap.

`window_memory_limit` (config-driven via `git_config_ulong()`) stays
`unsigned long`: it is only compared against `mem_usage` and promotes.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/pack-objects.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 4737a6a32c..63ceeb736f 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -2804,7 +2804,7 @@ size_t oe_get_size_slow(struct packing_data *pack,
 }
 
 static int try_delta(struct unpacked *trg, struct unpacked *src,
-		     unsigned max_depth, unsigned long *mem_usage)
+		     unsigned max_depth, size_t *mem_usage)
 {
 	struct object_entry *trg_entry = trg->entry;
 	struct object_entry *src_entry = src->entry;
@@ -2991,7 +2991,7 @@ static void find_deltas(struct object_entry **list, unsigned *list_size,
 {
 	uint32_t i, idx = 0, count = 0;
 	struct unpacked *array;
-	unsigned long mem_usage = 0;
+	size_t mem_usage = 0;
 
 	CALLOC_ARRAY(array, window);
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 06/12] delta: widen `create_delta()` and `diff_delta()` to `size_t`
  2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
                   ` (4 preceding siblings ...)
  2026-07-09 16:49 ` [PATCH 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields " Johannes Schindelin via GitGitGadget
                   ` (5 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
  To: git; +Cc: Johannes Schindelin, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Last stop in the delta-encoding API widening for >4 GiB blobs on
Windows: with `create_delta_index()` done in the prior commit and
`create_delta()`/`diff_delta()` finished here, every byte count that
crosses delta.h is now `size_t`. The struct fields they store into have
been `size_t` since the diff-delta struct widening.

The API change must move with all callers in the same commit (the build
only passes when every `&delta_size` matches the new `size_t*`). Caller
updates are kept minimal:

  * builtin/pack-objects.c `get_delta()` and `try_delta()`: widen only
    the local `delta_size` variable; the surrounding unsigned-long
    locals and their `cast_size_t_to_ulong()` shims are out of scope
    here and will be cleaned up in their own commits.

  * builtin/fast-import.c, diff.c, t/helper/test-pack-deltas.c:
    keep the local unsigned-long delta size (each feeds a still-
    unsigned-long downstream consumer: zlib's `avail_in`,
    `deflate_it()`, the test helper's own `do_compress()`), and bridge
    via a temporary `size_t` plus `cast_size_t_to_ulong()`. The new
    casts are paid back in later topics that widen those consumers.

  * t/helper/test-delta.c: widen the local outright (no downstream
    consumer beyond the test's own `out_size`, which is already
    `size_t`).

Note that GCC struggles a bit to figure out that `deltalen` is always
initialized before it is used; To help it along, we initialize it to 0.
This work-around will go away in a later patch series when `deltalen`
can be widened to `size_t`.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/fast-import.c       |  6 ++++--
 builtin/pack-objects.c      |  6 ++++--
 delta.h                     | 10 +++++-----
 diff-delta.c                |  4 ++--
 diff.c                      |  4 +++-
 t/helper/test-delta.c       |  2 +-
 t/helper/test-pack-deltas.c |  5 +++--
 7 files changed, 22 insertions(+), 15 deletions(-)

diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index aa656c5195..1c6e5366c2 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -962,7 +962,7 @@ static int store_object(
 	struct object_entry *e;
 	unsigned char hdr[96];
 	struct object_id oid;
-	unsigned long hdrlen, deltalen;
+	unsigned long hdrlen, deltalen = 0;
 	struct git_hash_ctx c;
 	git_zstream s;
 	struct repo_config_values *cfg = repo_config_values(the_repository);
@@ -998,11 +998,13 @@ static int store_object(
 
 	if (last && last->data.len && last->data.buf && last->depth < max_depth
 		&& dat->len > the_hash_algo->rawsz) {
+		size_t deltalen_st;
 
 		delta_count_attempts_by_type[type]++;
 		delta = diff_delta(last->data.buf, last->data.len,
 			dat->buf, dat->len,
-			&deltalen, dat->len - the_hash_algo->rawsz);
+			&deltalen_st, dat->len - the_hash_algo->rawsz);
+		deltalen = cast_size_t_to_ulong(deltalen_st);
 	} else
 		delta = NULL;
 
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 63ceeb736f..315ea0ed7e 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -353,7 +353,8 @@ static void index_commit_for_bitmap(struct commit *commit)
 
 static void *get_delta(struct object_entry *entry)
 {
-	unsigned long size, base_size, delta_size;
+	unsigned long size, base_size;
+	size_t delta_size;
 	void *buf, *base_buf, *delta_buf;
 	enum object_type type;
 	size_t size_st = 0, base_size_st = 0;
@@ -2808,7 +2809,8 @@ static int try_delta(struct unpacked *trg, struct unpacked *src,
 {
 	struct object_entry *trg_entry = trg->entry;
 	struct object_entry *src_entry = src->entry;
-	unsigned long trg_size, src_size, delta_size, sizediff, max_size, sz;
+	unsigned long trg_size, src_size, sizediff, max_size, sz;
+	size_t delta_size;
 	unsigned ref_depth;
 	enum object_type type;
 	void *delta_buf;
diff --git a/delta.h b/delta.h
index a19586d789..59ccaaa0e0 100644
--- a/delta.h
+++ b/delta.h
@@ -42,8 +42,8 @@ unsigned long sizeof_delta_index(struct delta_index *index);
  */
 void *
 create_delta(const struct delta_index *index,
-	     const void *buf, unsigned long bufsize,
-	     unsigned long *delta_size, unsigned long max_delta_size);
+	     const void *buf, size_t bufsize,
+	     size_t *delta_size, size_t max_delta_size);
 
 /*
  * diff_delta: create a delta from source buffer to target buffer
@@ -54,9 +54,9 @@ create_delta(const struct delta_index *index,
  * updated with its size.  The returned buffer must be freed by the caller.
  */
 static inline void *
-diff_delta(const void *src_buf, unsigned long src_bufsize,
-	   const void *trg_buf, unsigned long trg_bufsize,
-	   unsigned long *delta_size, unsigned long max_delta_size)
+diff_delta(const void *src_buf, size_t src_bufsize,
+	   const void *trg_buf, size_t trg_bufsize,
+	   size_t *delta_size, size_t max_delta_size)
 {
 	struct delta_index *index = create_delta_index(src_buf, src_bufsize);
 	if (index) {
diff --git a/diff-delta.c b/diff-delta.c
index c93ac42594..15210e8381 100644
--- a/diff-delta.c
+++ b/diff-delta.c
@@ -318,8 +318,8 @@ unsigned long sizeof_delta_index(struct delta_index *index)
 
 void *
 create_delta(const struct delta_index *index,
-	     const void *trg_buf, unsigned long trg_size,
-	     unsigned long *delta_size, unsigned long max_size)
+	     const void *trg_buf, size_t trg_size,
+	     size_t *delta_size, size_t max_size)
 {
 	unsigned int i, val;
 	off_t outpos, moff;
diff --git a/diff.c b/diff.c
index 2a9d0d8687..69eb2f76a4 100644
--- a/diff.c
+++ b/diff.c
@@ -3647,9 +3647,11 @@ static void emit_binary_diff_body(struct diff_options *o,
 	delta = NULL;
 	deflated = deflate_it(two->ptr, two->size, &deflate_size);
 	if (one->size && two->size) {
+		size_t delta_size_st = 0;
 		delta = diff_delta(one->ptr, one->size,
 				   two->ptr, two->size,
-				   &delta_size, deflate_size);
+				   &delta_size_st, deflate_size);
+		delta_size = cast_size_t_to_ulong(delta_size_st);
 		if (delta) {
 			void *to_free = delta;
 			orig_size = delta_size;
diff --git a/t/helper/test-delta.c b/t/helper/test-delta.c
index 8223a60229..d807afef75 100644
--- a/t/helper/test-delta.c
+++ b/t/helper/test-delta.c
@@ -32,7 +32,7 @@ int cmd__delta(int argc, const char **argv)
 		die_errno("unable to read '%s'", argv[3]);
 
 	if (argv[1][1] == 'd') {
-		unsigned long delta_size;
+		size_t delta_size;
 		out_buf = diff_delta(from.buf, from.len,
 				     data.buf, data.len,
 				     &delta_size, 0);
diff --git a/t/helper/test-pack-deltas.c b/t/helper/test-pack-deltas.c
index 840797cf0d..5e0f726842 100644
--- a/t/helper/test-pack-deltas.c
+++ b/t/helper/test-pack-deltas.c
@@ -49,7 +49,7 @@ static void write_ref_delta(struct hashfile *f,
 {
 	unsigned char header[MAX_PACK_OBJECT_HEADER];
 	unsigned long delta_size, compressed_size, hdrlen;
-	size_t size, base_size;
+	size_t size, base_size, delta_size_st = 0;
 	enum object_type type;
 	void *base_buf, *delta_buf;
 	void *buf = odb_read_object(the_repository->objects,
@@ -65,7 +65,8 @@ static void write_ref_delta(struct hashfile *f,
 		die("unable to read %s", oid_to_hex(base));
 
 	delta_buf = diff_delta(base_buf, base_size,
-			       buf, size, &delta_size, 0);
+			       buf, size, &delta_size_st, 0);
+	delta_size = cast_size_t_to_ulong(delta_size_st);
 
 	compressed_size = do_compress(&delta_buf, delta_size);
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields to `size_t`
  2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
                   ` (5 preceding siblings ...)
  2026-07-09 16:49 ` [PATCH 06/12] delta: widen `create_delta()` and `diff_delta()` " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local " Johannes Schindelin via GitGitGadget
                   ` (4 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
  To: git; +Cc: Johannes Schindelin, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Bundling the two widenings: four call sites pass `&stream.avail_in`
directly to `use_pack()`, and widening either type fencepost alone would
force a bridge variable at each. Doing both together is the simpler end
state and is the prerequisite for the `do_compress()` widening in the
next commit, which is what lets `write_no_reuse_object()` lose its last
`cast_size_t_to_ulong()` shim.

The unsigned-long locals widened at the other `use_pack()` callers
(avail / remaining / left) hold pack-window sizes bounded by
`core.packedGitWindowSize`, so the change is type consistency rather
than a new >4GB capability. `git_zstream.avail_in`/`avail_out` likewise
reach zlib's `uInt` fields only after `zlib_buf_cap()`'s 1 GiB cap, so
the wrapper already accepted `size_t`-shaped inputs in practice.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/pack-objects.c | 8 ++++----
 git-zlib.h             | 4 ++--
 pack-check.c           | 4 ++--
 packfile.c             | 4 ++--
 packfile.h             | 3 ++-
 5 files changed, 12 insertions(+), 11 deletions(-)

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 315ea0ed7e..cedda6ba9c 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -488,7 +488,7 @@ static void copy_pack_data(struct hashfile *f,
 		off_t len)
 {
 	unsigned char *in;
-	unsigned long avail;
+	size_t avail;
 
 	while (len) {
 		in = use_pack(p, w_curs, offset, &avail);
@@ -2261,7 +2261,7 @@ static void check_object(struct object_entry *entry, uint32_t object_index)
 		struct object_id base_ref;
 		struct object_entry *base_entry;
 		unsigned long used, used_0;
-		unsigned long avail;
+		size_t avail;
 		off_t ofs;
 		unsigned char *buf, c;
 		enum object_type type;
@@ -2773,8 +2773,8 @@ size_t oe_get_size_slow(struct packing_data *pack,
 	struct pack_window *w_curs;
 	unsigned char *buf;
 	enum object_type type;
-	unsigned long used, avail;
-	size_t size;
+	unsigned long used;
+	size_t avail, size;
 
 	if (e->type_ != OBJ_OFS_DELTA && e->type_ != OBJ_REF_DELTA) {
 		size_t sz;
diff --git a/git-zlib.h b/git-zlib.h
index 44380e8ad3..0b24b15bd0 100644
--- a/git-zlib.h
+++ b/git-zlib.h
@@ -5,8 +5,8 @@
 
 typedef struct git_zstream {
 	struct z_stream_s z;
-	unsigned long avail_in;
-	unsigned long avail_out;
+	size_t avail_in;
+	size_t avail_out;
 	size_t total_in;
 	size_t total_out;
 	unsigned char *next_in;
diff --git a/pack-check.c b/pack-check.c
index 5adfb3f272..befb860472 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -34,7 +34,7 @@ int check_pack_crc(struct packed_git *p, struct pack_window **w_curs,
 	uint32_t data_crc = crc32(0, NULL, 0);
 
 	do {
-		unsigned long avail;
+		size_t avail;
 		void *data = use_pack(p, w_curs, offset, &avail);
 		if (avail > len)
 			avail = len;
@@ -71,7 +71,7 @@ static int verify_packfile(struct repository *r,
 
 	r->hash_algo->init_fn(&ctx);
 	do {
-		unsigned long remaining;
+		size_t remaining;
 		unsigned char *in = use_pack(p, w_curs, offset, &remaining);
 		offset += remaining;
 		if (!pack_sig_ofs)
diff --git a/packfile.c b/packfile.c
index 1d1b23b6cc..629fe46a6a 100644
--- a/packfile.c
+++ b/packfile.c
@@ -620,7 +620,7 @@ static int in_window(struct repository *r, struct pack_window *win,
 unsigned char *use_pack(struct packed_git *p,
 		struct pack_window **w_cursor,
 		off_t offset,
-		unsigned long *left)
+		size_t *left)
 {
 	struct pack_window *win = *w_cursor;
 
@@ -960,7 +960,7 @@ int unpack_object_header(struct packed_git *p,
 			 size_t *sizep)
 {
 	unsigned char *base;
-	unsigned long left;
+	size_t left;
 	unsigned long used;
 	enum object_type type;
 
diff --git a/packfile.h b/packfile.h
index 2329a69701..3cff8bdcb9 100644
--- a/packfile.h
+++ b/packfile.h
@@ -240,7 +240,8 @@ uint32_t get_pack_fanout(struct packed_git *p, uint32_t value);
 
 struct object_database;
 
-unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
+unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t,
+			size_t *);
 void close_pack_windows(struct packed_git *);
 void close_pack(struct packed_git *);
 void unuse_pack(struct pack_window **);
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local to `size_t`
  2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
                   ` (6 preceding siblings ...)
  2026-07-09 16:49 ` [PATCH 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 09/12] diff: widen `deflate_it()`'s bound local from int " Johannes Schindelin via GitGitGadget
                   ` (3 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
  To: git; +Cc: Johannes Schindelin, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Prep for the upcoming `git_deflate_bound()` widening to `size_t`: the
local that catches its return needs to be `size_t` too, otherwise the
widening would introduce a silent Windows narrowing here. No semantic
effect with the current unsigned-long-returning `git_deflate_bound()`
(`size_t == unsigned long` on this caller's platforms today).

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 archive-zip.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/archive-zip.c b/archive-zip.c
index 97ea8d60d6..a487d4c041 100644
--- a/archive-zip.c
+++ b/archive-zip.c
@@ -206,7 +206,7 @@ static void *zlib_deflate_raw(void *data, unsigned long size,
 			      unsigned long *compressed_size)
 {
 	git_zstream stream;
-	unsigned long maxsize;
+	size_t maxsize;
 	void *buffer;
 	int result;
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 09/12] diff: widen `deflate_it()`'s bound local from int to `size_t`
  2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
                   ` (7 preceding siblings ...)
  2026-07-09 16:49 ` [PATCH 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 10/12] http-push: widen `start_put()`'s size local from `ssize_t` " Johannes Schindelin via GitGitGadget
                   ` (2 subsequent siblings)
  11 siblings, 0 replies; 13+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
  To: git; +Cc: Johannes Schindelin, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Fixes a pre-existing silent narrowing from `git_deflate_bound()`'s
`unsigned long` return into an `int` local: anything past 2 GiB has
always wrapped negative here and then been re-extended to `size_t`
inside `xmalloc()`. Also prep for the upcoming `git_deflate_bound()`
widening to `size_t`, which would extend the narrowing further if
`bound` stayed `int`.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 diff.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/diff.c b/diff.c
index 69eb2f76a4..c14f69719b 100644
--- a/diff.c
+++ b/diff.c
@@ -3609,7 +3609,7 @@ static unsigned char *deflate_it(char *data,
 				 unsigned long size,
 				 unsigned long *result_size)
 {
-	int bound;
+	size_t bound;
 	unsigned char *deflated;
 	git_zstream stream;
 	struct repo_config_values *cfg = repo_config_values(the_repository);
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 10/12] http-push: widen `start_put()`'s size local from `ssize_t` to `size_t`
  2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
                   ` (8 preceding siblings ...)
  2026-07-09 16:49 ` [PATCH 09/12] diff: widen `deflate_it()`'s bound local from int " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local " Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 12/12] git-zlib: widen `git_deflate_bound()` " Johannes Schindelin via GitGitGadget
  11 siblings, 0 replies; 13+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
  To: git; +Cc: Johannes Schindelin, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The local is initialised from `git_deflate_bound()` (an unsigned upper
bound on the deflated output, never negative) and used in exactly three
places: the initialising assignment, `strbuf_grow(buf, size)` whose
parameter is already `size_t`, and `stream.avail_out` which became
`size_t` in the prior commit. There is no comparison against zero or a
negative value, no subtraction, no arithmetic that depends on
signedness, and no path that would assign a signed quantity to it.

The original `ssize_t` was the wrong type to begin with: a
`git_deflate_bound()` result above `SSIZE_MAX` would have wrapped
negative on assignment and then implicitly re-extended to a huge
`size_t` at `strbuf_grow()`/`stream.avail_out`, requesting an absurd
allocation. That is not a real-world concern for the object sizes
http-push pushes today, but it is also the reason the type needs to move
to `size_t` before `git_deflate_bound()` itself is widened.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 http-push.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/http-push.c b/http-push.c
index 3c23cbba27..2a07d14259 100644
--- a/http-push.c
+++ b/http-push.c
@@ -367,7 +367,7 @@ static void start_put(struct transfer_request *request)
 	void *unpacked;
 	size_t len;
 	int hdrlen;
-	ssize_t size;
+	size_t size;
 	git_zstream stream;
 	struct repo_config_values *cfg = repo_config_values(the_repository);
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local to `size_t`
  2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
                   ` (9 preceding siblings ...)
  2026-07-09 16:49 ` [PATCH 10/12] http-push: widen `start_put()`'s size local from `ssize_t` " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
  2026-07-09 16:49 ` [PATCH 12/12] git-zlib: widen `git_deflate_bound()` " Johannes Schindelin via GitGitGadget
  11 siblings, 0 replies; 13+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
  To: git; +Cc: Johannes Schindelin, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Prep for the upcoming `git_deflate_bound()` widening to `size_t`. The
local is only ever the return value of `git_deflate_bound()` and the
`xmalloc()`/`stream.avail_out` sizes derived from it; widening it has no
semantic effect today.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/helper/test-pack-deltas.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/helper/test-pack-deltas.c b/t/helper/test-pack-deltas.c
index 5e0f726842..959705feca 100644
--- a/t/helper/test-pack-deltas.c
+++ b/t/helper/test-pack-deltas.c
@@ -22,7 +22,7 @@ static unsigned long do_compress(void **pptr, unsigned long size)
 {
 	git_zstream stream;
 	void *in, *out;
-	unsigned long maxsize;
+	size_t maxsize;
 
 	git_deflate_init(&stream, 1);
 	maxsize = git_deflate_bound(&stream, size);
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 12/12] git-zlib: widen `git_deflate_bound()` to `size_t`
  2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
                   ` (10 preceding siblings ...)
  2026-07-09 16:49 ` [PATCH 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
  11 siblings, 0 replies; 13+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
  To: git; +Cc: Johannes Schindelin, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

All four `unsigned long`/`int`/`ssize_t` receivers across archive-zip,
diff, http-push and t/helper/test-pack-deltas were widened to `size_t`
in the prior commits, and remote-curl and fast-import were already
there. With every caller prepared, both the parameter and the return
type can now move without introducing any silent narrowing.

For inputs above zlib's `uLong` range (i.e. >4 GiB on platforms where
`uLong` is 32-bit, notably 64-bit Windows), defer to zlib's stored-block
formula (the same fallback it would itself use for an unknown stream
state) plus the worst-case wrapper overhead. The existing path through
`deflateBound()` is unchanged for inputs that fit.

Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 git-zlib.c | 16 ++++++++++++++--
 git-zlib.h |  2 +-
 2 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/git-zlib.c b/git-zlib.c
index d21adb3bf5..ebbbcc6d1a 100644
--- a/git-zlib.c
+++ b/git-zlib.c
@@ -167,9 +167,21 @@ int git_inflate(git_zstream *strm, int flush)
 	return status;
 }
 
-unsigned long git_deflate_bound(git_zstream *strm, unsigned long size)
+size_t git_deflate_bound(git_zstream *strm, size_t size)
 {
-	return deflateBound(&strm->z, size);
+#if SIZE_MAX > ULONG_MAX
+	if (size > maximum_unsigned_value_of_type(uLong))
+		/*
+		 * deflateBound() takes uLong, which is 32-bit on
+		 * Windows. For inputs above that range, return zlib's
+		 * stored-block formula (the conservative path it would
+		 * itself use for an unknown stream state) plus the
+		 * worst-case wrapper overhead.
+		 */
+		return size + (size >> 5) + (size >> 7) + (size >> 11)
+			+ 7 + 18;
+#endif
+	return deflateBound(&strm->z, (uLong)size);
 }
 
 void git_deflate_init(git_zstream *strm, int level)
diff --git a/git-zlib.h b/git-zlib.h
index 0b24b15bd0..9248d11ca9 100644
--- a/git-zlib.h
+++ b/git-zlib.h
@@ -25,6 +25,6 @@ void git_deflate_end(git_zstream *);
 int git_deflate_abort(git_zstream *);
 int git_deflate_end_gently(git_zstream *);
 int git_deflate(git_zstream *, int flush);
-unsigned long git_deflate_bound(git_zstream *, unsigned long);
+size_t git_deflate_bound(git_zstream *, size_t);
 
 #endif /* GIT_ZLIB_H */
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2026-07-09 16:50 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 02/12] delta: widen `create_delta_index()` parameter " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 03/12] pack-objects: widen delta-cache accounting " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 04/12] pack-objects: widen `free_unpacked()` return " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 06/12] delta: widen `create_delta()` and `diff_delta()` " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 09/12] diff: widen `deflate_it()`'s bound local from int " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 10/12] http-push: widen `start_put()`'s size local from `ssize_t` " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 12/12] git-zlib: widen `git_deflate_bound()` " Johannes Schindelin via GitGitGadget

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.