* [PATCH 00/12] Next size_t stop: pack-objects/delta
@ 2026-07-09 16:49 Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
` (12 more replies)
0 siblings, 13 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin
This patch series continues the effort to stop using unsigned long where
size_t should have been used in the first place. This makes a difference on
64-bit Windows, where unsigned long is 32-bit.
With these fixes, the pack-objects machinery works as intended on 64-bit
Windows (and any other 64-bit platform where unsigned long isn't 64-bit).
Johannes Schindelin (12):
diff-delta: widen `struct delta_index`' size fields to `size_t`
delta: widen `create_delta_index()` parameter to `size_t`
pack-objects: widen delta-cache accounting to `size_t`
pack-objects: widen `free_unpacked()` return to `size_t`
pack-objects: widen `mem_usage` and `try_delta()`'s out-param to
`size_t`
delta: widen `create_delta()` and `diff_delta()` to `size_t`
packfile, git-zlib: widen `use_pack()` and zstream avail fields to
`size_t`
archive-zip: widen `zlib_deflate_raw()`'s maxsize local to `size_t`
diff: widen `deflate_it()`'s bound local from int to `size_t`
http-push: widen `start_put()`'s size local from `ssize_t` to `size_t`
t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local to
`size_t`
git-zlib: widen `git_deflate_bound()` to `size_t`
archive-zip.c | 2 +-
builtin/fast-import.c | 6 ++++--
builtin/pack-objects.c | 30 ++++++++++++++++--------------
delta.h | 12 ++++++------
diff-delta.c | 12 ++++++------
diff.c | 6 ++++--
git-zlib.c | 16 ++++++++++++++--
git-zlib.h | 6 +++---
http-push.c | 2 +-
pack-check.c | 4 ++--
packfile.c | 4 ++--
packfile.h | 3 ++-
t/helper/test-delta.c | 2 +-
t/helper/test-pack-deltas.c | 7 ++++---
14 files changed, 66 insertions(+), 46 deletions(-)
base-commit: f85a7e662054a7b0d9070e432508831afa214b47
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2175%2Fdscho%2Fsize-t%2Fpack-objects-delta-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2175/dscho/size-t/pack-objects-delta-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2175
--
gitgitgadget
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t`
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
2026-08-05 9:22 ` Patrick Steinhardt
2026-07-09 16:49 ` [PATCH 02/12] delta: widen `create_delta_index()` parameter " Johannes Schindelin via GitGitGadget
` (11 subsequent siblings)
12 siblings, 1 reply; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Preparation for widening the delta-encoding API to `size_t` in
subsequent commits, which is what lets pack-objects drop the
`cast_size_t_to_ulong()` shims that 606c192380 (odb, packfile: use
size_t for streaming object sizes, 2026-05-08) had to leave behind in
`get_delta()` and `try_delta()` because their downstream consumers were
still narrow.
The struct is private to diff-delta.c, so widening its fields in
isolation is a no-op at runtime: the values stored continue to fit in 32
bits on Windows because the public API around it still truncates.
Splitting it out keeps the API-change commit focused on caller updates.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
diff-delta.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/diff-delta.c b/diff-delta.c
index 43c339f010..b6b65d7607 100644
--- a/diff-delta.c
+++ b/diff-delta.c
@@ -125,9 +125,9 @@ struct unpacked_index_entry {
};
struct delta_index {
- unsigned long memsize;
+ size_t memsize;
const void *src_buf;
- unsigned long src_size;
+ size_t src_size;
unsigned int hash_mask;
struct index_entry *hash[FLEX_ARRAY];
};
@@ -140,7 +140,7 @@ struct delta_index * create_delta_index(const void *buf, unsigned long bufsize)
struct unpacked_index_entry *entry, **hash;
struct index_entry *packed_entry, **packed_hash;
void *mem;
- unsigned long memsize;
+ size_t memsize;
if (!buf || !bufsize)
return NULL;
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH 02/12] delta: widen `create_delta_index()` parameter to `size_t`
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 03/12] pack-objects: widen delta-cache accounting " Johannes Schindelin via GitGitGadget
` (10 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
The sole caller (`try_delta()` in builtin/pack-objects.c) passes an
`unsigned long`, which promotes safely, so no caller fixups are needed.
Splitting it out keeps the `diff_delta()`/`create_delta()` widening,
which does ripple to several callers, in its own commit.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
delta.h | 2 +-
diff-delta.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/delta.h b/delta.h
index eb5c6d2fdb..a19586d789 100644
--- a/delta.h
+++ b/delta.h
@@ -14,7 +14,7 @@ struct delta_index;
* using free_delta_index().
*/
struct delta_index *
-create_delta_index(const void *buf, unsigned long bufsize);
+create_delta_index(const void *buf, size_t bufsize);
/*
* free_delta_index: free the index created by create_delta_index()
diff --git a/diff-delta.c b/diff-delta.c
index b6b65d7607..c93ac42594 100644
--- a/diff-delta.c
+++ b/diff-delta.c
@@ -132,7 +132,7 @@ struct delta_index {
struct index_entry *hash[FLEX_ARRAY];
};
-struct delta_index * create_delta_index(const void *buf, unsigned long bufsize)
+struct delta_index * create_delta_index(const void *buf, size_t bufsize)
{
unsigned int i, hsize, hmask, entries, prev_val, *hash_count;
const unsigned char *data, *buffer = buf;
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH 03/12] pack-objects: widen delta-cache accounting to `size_t`
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 02/12] delta: widen `create_delta_index()` parameter " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
2026-08-05 9:23 ` Patrick Steinhardt
2026-07-09 16:49 ` [PATCH 04/12] pack-objects: widen `free_unpacked()` return " Johannes Schindelin via GitGitGadget
` (9 subsequent siblings)
12 siblings, 1 reply; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
These three are a single accounting tuple (the globals tracking
cumulative cached-delta bytes, plus the helper that compares them
against an incoming delta size) and are latently 32-bit on Windows where
`unsigned long` != `size_t`: a pack with many large cached deltas could
wrap silently.
The widening is internally consistent on its own: the additions and
subtractions against delta_cache_size already come from `size_t` sources
(`DELTA_SIZE()` returns `size_t`), and `delta_cacheable()`'s sole caller
in `try_delta()` still passes `unsigned long`, which promotes.
Prerequisite for dropping `try_delta()`'s `cast_size_t_to_ulong()`
shims, which becomes possible once 1create_delta()` and `diff_delta()`
are widened in a later commit.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
builtin/pack-objects.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index e3760b3492..f89628a760 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -260,8 +260,8 @@ static int exclude_promisor_objects_best_effort;
static int use_delta_islands;
-static unsigned long delta_cache_size = 0;
-static unsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;
+static size_t delta_cache_size = 0;
+static size_t max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;
static unsigned long cache_max_small_delta_size = 1000;
static unsigned long window_memory_limit = 0;
@@ -2688,8 +2688,8 @@ struct unpacked {
unsigned depth;
};
-static int delta_cacheable(unsigned long src_size, unsigned long trg_size,
- unsigned long delta_size)
+static int delta_cacheable(size_t src_size, size_t trg_size,
+ size_t delta_size)
{
if (max_delta_cache_size && delta_cache_size + delta_size > max_delta_cache_size)
return 0;
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH 04/12] pack-objects: widen `free_unpacked()` return to `size_t`
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (2 preceding siblings ...)
2026-07-09 16:49 ` [PATCH 03/12] pack-objects: widen delta-cache accounting " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
2026-08-05 9:23 ` Patrick Steinhardt
2026-07-09 16:49 ` [PATCH 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param " Johannes Schindelin via GitGitGadget
` (8 subsequent siblings)
12 siblings, 1 reply; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
`free_unpacked()` sums two byte counts: `sizeof_delta_index()` and
`SIZE(n->entry)`. The latter has been `size_t` since the prior topic
"More work supporting objects larger than 4GB on Windows" widened
`SIZE()`/`oe_size()` to `size_t`, so accumulating it into an `unsigned
long` return was a silent Windows-only truncation on a packing run with
many large objects.
The sole caller, `find_deltas()`, still holds its own `mem_usage` in an
`unsigned long` for now, and therefore still truncates silently.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
builtin/pack-objects.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index f89628a760..4737a6a32c 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -2972,9 +2972,9 @@ static unsigned int check_delta_limit(struct object_entry *me, unsigned int n)
return m;
}
-static unsigned long free_unpacked(struct unpacked *n)
+static size_t free_unpacked(struct unpacked *n)
{
- unsigned long freed_mem = sizeof_delta_index(n->index);
+ size_t freed_mem = sizeof_delta_index(n->index);
free_delta_index(n->index);
n->index = NULL;
if (n->data) {
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param to `size_t`
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (3 preceding siblings ...)
2026-07-09 16:49 ` [PATCH 04/12] pack-objects: widen `free_unpacked()` return " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 06/12] delta: widen `create_delta()` and `diff_delta()` " Johannes Schindelin via GitGitGadget
` (7 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
The pair must move together because `find_deltas()` passes `&mem_usage`
to `try_delta()`: widening either alone breaks the type match.
`mem_usage` accumulates per-object byte counts already computed in
`size_t` (`SIZE()` and `sizeof_delta_index()` reach here through
`free_unpacked()`, now `size_t`), and was the last 32-bit-on-Windows
narrowing point in the delta-window memory accounting chain. With this
commit, that chain uses `size_t` consistently except for
`sizeof_delta_index()`'s still-narrow return, whose value is bounded by
`create_delta_index()`'s entries cap.
`window_memory_limit` (config-driven via `git_config_ulong()`) stays
`unsigned long`: it is only compared against `mem_usage` and promotes.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
builtin/pack-objects.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 4737a6a32c..63ceeb736f 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -2804,7 +2804,7 @@ size_t oe_get_size_slow(struct packing_data *pack,
}
static int try_delta(struct unpacked *trg, struct unpacked *src,
- unsigned max_depth, unsigned long *mem_usage)
+ unsigned max_depth, size_t *mem_usage)
{
struct object_entry *trg_entry = trg->entry;
struct object_entry *src_entry = src->entry;
@@ -2991,7 +2991,7 @@ static void find_deltas(struct object_entry **list, unsigned *list_size,
{
uint32_t i, idx = 0, count = 0;
struct unpacked *array;
- unsigned long mem_usage = 0;
+ size_t mem_usage = 0;
CALLOC_ARRAY(array, window);
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH 06/12] delta: widen `create_delta()` and `diff_delta()` to `size_t`
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (4 preceding siblings ...)
2026-07-09 16:49 ` [PATCH 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
2026-08-05 9:23 ` Patrick Steinhardt
2026-07-09 16:49 ` [PATCH 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields " Johannes Schindelin via GitGitGadget
` (6 subsequent siblings)
12 siblings, 1 reply; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Last stop in the delta-encoding API widening for >4 GiB blobs on
Windows: with `create_delta_index()` done in the prior commit and
`create_delta()`/`diff_delta()` finished here, every byte count that
crosses delta.h is now `size_t`. The struct fields they store into have
been `size_t` since the diff-delta struct widening.
The API change must move with all callers in the same commit (the build
only passes when every `&delta_size` matches the new `size_t*`). Caller
updates are kept minimal:
* builtin/pack-objects.c `get_delta()` and `try_delta()`: widen only
the local `delta_size` variable; the surrounding unsigned-long
locals and their `cast_size_t_to_ulong()` shims are out of scope
here and will be cleaned up in their own commits.
* builtin/fast-import.c, diff.c, t/helper/test-pack-deltas.c:
keep the local unsigned-long delta size (each feeds a still-
unsigned-long downstream consumer: zlib's `avail_in`,
`deflate_it()`, the test helper's own `do_compress()`), and bridge
via a temporary `size_t` plus `cast_size_t_to_ulong()`. The new
casts are paid back in later topics that widen those consumers.
* t/helper/test-delta.c: widen the local outright (no downstream
consumer beyond the test's own `out_size`, which is already
`size_t`).
Note that GCC struggles a bit to figure out that `deltalen` is always
initialized before it is used; To help it along, we initialize it to 0.
This work-around will go away in a later patch series when `deltalen`
can be widened to `size_t`.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
builtin/fast-import.c | 6 ++++--
builtin/pack-objects.c | 6 ++++--
delta.h | 10 +++++-----
diff-delta.c | 4 ++--
diff.c | 4 +++-
t/helper/test-delta.c | 2 +-
t/helper/test-pack-deltas.c | 5 +++--
7 files changed, 22 insertions(+), 15 deletions(-)
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index aa656c5195..1c6e5366c2 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -962,7 +962,7 @@ static int store_object(
struct object_entry *e;
unsigned char hdr[96];
struct object_id oid;
- unsigned long hdrlen, deltalen;
+ unsigned long hdrlen, deltalen = 0;
struct git_hash_ctx c;
git_zstream s;
struct repo_config_values *cfg = repo_config_values(the_repository);
@@ -998,11 +998,13 @@ static int store_object(
if (last && last->data.len && last->data.buf && last->depth < max_depth
&& dat->len > the_hash_algo->rawsz) {
+ size_t deltalen_st;
delta_count_attempts_by_type[type]++;
delta = diff_delta(last->data.buf, last->data.len,
dat->buf, dat->len,
- &deltalen, dat->len - the_hash_algo->rawsz);
+ &deltalen_st, dat->len - the_hash_algo->rawsz);
+ deltalen = cast_size_t_to_ulong(deltalen_st);
} else
delta = NULL;
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 63ceeb736f..315ea0ed7e 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -353,7 +353,8 @@ static void index_commit_for_bitmap(struct commit *commit)
static void *get_delta(struct object_entry *entry)
{
- unsigned long size, base_size, delta_size;
+ unsigned long size, base_size;
+ size_t delta_size;
void *buf, *base_buf, *delta_buf;
enum object_type type;
size_t size_st = 0, base_size_st = 0;
@@ -2808,7 +2809,8 @@ static int try_delta(struct unpacked *trg, struct unpacked *src,
{
struct object_entry *trg_entry = trg->entry;
struct object_entry *src_entry = src->entry;
- unsigned long trg_size, src_size, delta_size, sizediff, max_size, sz;
+ unsigned long trg_size, src_size, sizediff, max_size, sz;
+ size_t delta_size;
unsigned ref_depth;
enum object_type type;
void *delta_buf;
diff --git a/delta.h b/delta.h
index a19586d789..59ccaaa0e0 100644
--- a/delta.h
+++ b/delta.h
@@ -42,8 +42,8 @@ unsigned long sizeof_delta_index(struct delta_index *index);
*/
void *
create_delta(const struct delta_index *index,
- const void *buf, unsigned long bufsize,
- unsigned long *delta_size, unsigned long max_delta_size);
+ const void *buf, size_t bufsize,
+ size_t *delta_size, size_t max_delta_size);
/*
* diff_delta: create a delta from source buffer to target buffer
@@ -54,9 +54,9 @@ create_delta(const struct delta_index *index,
* updated with its size. The returned buffer must be freed by the caller.
*/
static inline void *
-diff_delta(const void *src_buf, unsigned long src_bufsize,
- const void *trg_buf, unsigned long trg_bufsize,
- unsigned long *delta_size, unsigned long max_delta_size)
+diff_delta(const void *src_buf, size_t src_bufsize,
+ const void *trg_buf, size_t trg_bufsize,
+ size_t *delta_size, size_t max_delta_size)
{
struct delta_index *index = create_delta_index(src_buf, src_bufsize);
if (index) {
diff --git a/diff-delta.c b/diff-delta.c
index c93ac42594..15210e8381 100644
--- a/diff-delta.c
+++ b/diff-delta.c
@@ -318,8 +318,8 @@ unsigned long sizeof_delta_index(struct delta_index *index)
void *
create_delta(const struct delta_index *index,
- const void *trg_buf, unsigned long trg_size,
- unsigned long *delta_size, unsigned long max_size)
+ const void *trg_buf, size_t trg_size,
+ size_t *delta_size, size_t max_size)
{
unsigned int i, val;
off_t outpos, moff;
diff --git a/diff.c b/diff.c
index 2a9d0d8687..69eb2f76a4 100644
--- a/diff.c
+++ b/diff.c
@@ -3647,9 +3647,11 @@ static void emit_binary_diff_body(struct diff_options *o,
delta = NULL;
deflated = deflate_it(two->ptr, two->size, &deflate_size);
if (one->size && two->size) {
+ size_t delta_size_st = 0;
delta = diff_delta(one->ptr, one->size,
two->ptr, two->size,
- &delta_size, deflate_size);
+ &delta_size_st, deflate_size);
+ delta_size = cast_size_t_to_ulong(delta_size_st);
if (delta) {
void *to_free = delta;
orig_size = delta_size;
diff --git a/t/helper/test-delta.c b/t/helper/test-delta.c
index 8223a60229..d807afef75 100644
--- a/t/helper/test-delta.c
+++ b/t/helper/test-delta.c
@@ -32,7 +32,7 @@ int cmd__delta(int argc, const char **argv)
die_errno("unable to read '%s'", argv[3]);
if (argv[1][1] == 'd') {
- unsigned long delta_size;
+ size_t delta_size;
out_buf = diff_delta(from.buf, from.len,
data.buf, data.len,
&delta_size, 0);
diff --git a/t/helper/test-pack-deltas.c b/t/helper/test-pack-deltas.c
index 840797cf0d..5e0f726842 100644
--- a/t/helper/test-pack-deltas.c
+++ b/t/helper/test-pack-deltas.c
@@ -49,7 +49,7 @@ static void write_ref_delta(struct hashfile *f,
{
unsigned char header[MAX_PACK_OBJECT_HEADER];
unsigned long delta_size, compressed_size, hdrlen;
- size_t size, base_size;
+ size_t size, base_size, delta_size_st = 0;
enum object_type type;
void *base_buf, *delta_buf;
void *buf = odb_read_object(the_repository->objects,
@@ -65,7 +65,8 @@ static void write_ref_delta(struct hashfile *f,
die("unable to read %s", oid_to_hex(base));
delta_buf = diff_delta(base_buf, base_size,
- buf, size, &delta_size, 0);
+ buf, size, &delta_size_st, 0);
+ delta_size = cast_size_t_to_ulong(delta_size_st);
compressed_size = do_compress(&delta_buf, delta_size);
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields to `size_t`
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (5 preceding siblings ...)
2026-07-09 16:49 ` [PATCH 06/12] delta: widen `create_delta()` and `diff_delta()` " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local " Johannes Schindelin via GitGitGadget
` (5 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Bundling the two widenings: four call sites pass `&stream.avail_in`
directly to `use_pack()`, and widening either type fencepost alone would
force a bridge variable at each. Doing both together is the simpler end
state and is the prerequisite for the `do_compress()` widening in the
next commit, which is what lets `write_no_reuse_object()` lose its last
`cast_size_t_to_ulong()` shim.
The unsigned-long locals widened at the other `use_pack()` callers
(avail / remaining / left) hold pack-window sizes bounded by
`core.packedGitWindowSize`, so the change is type consistency rather
than a new >4GB capability. `git_zstream.avail_in`/`avail_out` likewise
reach zlib's `uInt` fields only after `zlib_buf_cap()`'s 1 GiB cap, so
the wrapper already accepted `size_t`-shaped inputs in practice.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
builtin/pack-objects.c | 8 ++++----
git-zlib.h | 4 ++--
pack-check.c | 4 ++--
packfile.c | 4 ++--
packfile.h | 3 ++-
5 files changed, 12 insertions(+), 11 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 315ea0ed7e..cedda6ba9c 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -488,7 +488,7 @@ static void copy_pack_data(struct hashfile *f,
off_t len)
{
unsigned char *in;
- unsigned long avail;
+ size_t avail;
while (len) {
in = use_pack(p, w_curs, offset, &avail);
@@ -2261,7 +2261,7 @@ static void check_object(struct object_entry *entry, uint32_t object_index)
struct object_id base_ref;
struct object_entry *base_entry;
unsigned long used, used_0;
- unsigned long avail;
+ size_t avail;
off_t ofs;
unsigned char *buf, c;
enum object_type type;
@@ -2773,8 +2773,8 @@ size_t oe_get_size_slow(struct packing_data *pack,
struct pack_window *w_curs;
unsigned char *buf;
enum object_type type;
- unsigned long used, avail;
- size_t size;
+ unsigned long used;
+ size_t avail, size;
if (e->type_ != OBJ_OFS_DELTA && e->type_ != OBJ_REF_DELTA) {
size_t sz;
diff --git a/git-zlib.h b/git-zlib.h
index 44380e8ad3..0b24b15bd0 100644
--- a/git-zlib.h
+++ b/git-zlib.h
@@ -5,8 +5,8 @@
typedef struct git_zstream {
struct z_stream_s z;
- unsigned long avail_in;
- unsigned long avail_out;
+ size_t avail_in;
+ size_t avail_out;
size_t total_in;
size_t total_out;
unsigned char *next_in;
diff --git a/pack-check.c b/pack-check.c
index 5adfb3f272..befb860472 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -34,7 +34,7 @@ int check_pack_crc(struct packed_git *p, struct pack_window **w_curs,
uint32_t data_crc = crc32(0, NULL, 0);
do {
- unsigned long avail;
+ size_t avail;
void *data = use_pack(p, w_curs, offset, &avail);
if (avail > len)
avail = len;
@@ -71,7 +71,7 @@ static int verify_packfile(struct repository *r,
r->hash_algo->init_fn(&ctx);
do {
- unsigned long remaining;
+ size_t remaining;
unsigned char *in = use_pack(p, w_curs, offset, &remaining);
offset += remaining;
if (!pack_sig_ofs)
diff --git a/packfile.c b/packfile.c
index 1d1b23b6cc..629fe46a6a 100644
--- a/packfile.c
+++ b/packfile.c
@@ -620,7 +620,7 @@ static int in_window(struct repository *r, struct pack_window *win,
unsigned char *use_pack(struct packed_git *p,
struct pack_window **w_cursor,
off_t offset,
- unsigned long *left)
+ size_t *left)
{
struct pack_window *win = *w_cursor;
@@ -960,7 +960,7 @@ int unpack_object_header(struct packed_git *p,
size_t *sizep)
{
unsigned char *base;
- unsigned long left;
+ size_t left;
unsigned long used;
enum object_type type;
diff --git a/packfile.h b/packfile.h
index 2329a69701..3cff8bdcb9 100644
--- a/packfile.h
+++ b/packfile.h
@@ -240,7 +240,8 @@ uint32_t get_pack_fanout(struct packed_git *p, uint32_t value);
struct object_database;
-unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
+unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t,
+ size_t *);
void close_pack_windows(struct packed_git *);
void close_pack(struct packed_git *);
void unuse_pack(struct pack_window **);
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local to `size_t`
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (6 preceding siblings ...)
2026-07-09 16:49 ` [PATCH 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 09/12] diff: widen `deflate_it()`'s bound local from int " Johannes Schindelin via GitGitGadget
` (4 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Prep for the upcoming `git_deflate_bound()` widening to `size_t`: the
local that catches its return needs to be `size_t` too, otherwise the
widening would introduce a silent Windows narrowing here. No semantic
effect with the current unsigned-long-returning `git_deflate_bound()`
(`size_t == unsigned long` on this caller's platforms today).
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
archive-zip.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/archive-zip.c b/archive-zip.c
index 97ea8d60d6..a487d4c041 100644
--- a/archive-zip.c
+++ b/archive-zip.c
@@ -206,7 +206,7 @@ static void *zlib_deflate_raw(void *data, unsigned long size,
unsigned long *compressed_size)
{
git_zstream stream;
- unsigned long maxsize;
+ size_t maxsize;
void *buffer;
int result;
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH 09/12] diff: widen `deflate_it()`'s bound local from int to `size_t`
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (7 preceding siblings ...)
2026-07-09 16:49 ` [PATCH 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 10/12] http-push: widen `start_put()`'s size local from `ssize_t` " Johannes Schindelin via GitGitGadget
` (3 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Fixes a pre-existing silent narrowing from `git_deflate_bound()`'s
`unsigned long` return into an `int` local: anything past 2 GiB has
always wrapped negative here and then been re-extended to `size_t`
inside `xmalloc()`. Also prep for the upcoming `git_deflate_bound()`
widening to `size_t`, which would extend the narrowing further if
`bound` stayed `int`.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
diff.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/diff.c b/diff.c
index 69eb2f76a4..c14f69719b 100644
--- a/diff.c
+++ b/diff.c
@@ -3609,7 +3609,7 @@ static unsigned char *deflate_it(char *data,
unsigned long size,
unsigned long *result_size)
{
- int bound;
+ size_t bound;
unsigned char *deflated;
git_zstream stream;
struct repo_config_values *cfg = repo_config_values(the_repository);
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH 10/12] http-push: widen `start_put()`'s size local from `ssize_t` to `size_t`
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (8 preceding siblings ...)
2026-07-09 16:49 ` [PATCH 09/12] diff: widen `deflate_it()`'s bound local from int " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local " Johannes Schindelin via GitGitGadget
` (2 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
The local is initialised from `git_deflate_bound()` (an unsigned upper
bound on the deflated output, never negative) and used in exactly three
places: the initialising assignment, `strbuf_grow(buf, size)` whose
parameter is already `size_t`, and `stream.avail_out` which became
`size_t` in the prior commit. There is no comparison against zero or a
negative value, no subtraction, no arithmetic that depends on
signedness, and no path that would assign a signed quantity to it.
The original `ssize_t` was the wrong type to begin with: a
`git_deflate_bound()` result above `SSIZE_MAX` would have wrapped
negative on assignment and then implicitly re-extended to a huge
`size_t` at `strbuf_grow()`/`stream.avail_out`, requesting an absurd
allocation. That is not a real-world concern for the object sizes
http-push pushes today, but it is also the reason the type needs to move
to `size_t` before `git_deflate_bound()` itself is widened.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
http-push.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/http-push.c b/http-push.c
index 3c23cbba27..2a07d14259 100644
--- a/http-push.c
+++ b/http-push.c
@@ -367,7 +367,7 @@ static void start_put(struct transfer_request *request)
void *unpacked;
size_t len;
int hdrlen;
- ssize_t size;
+ size_t size;
git_zstream stream;
struct repo_config_values *cfg = repo_config_values(the_repository);
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local to `size_t`
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (9 preceding siblings ...)
2026-07-09 16:49 ` [PATCH 10/12] http-push: widen `start_put()`'s size local from `ssize_t` " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 12/12] git-zlib: widen `git_deflate_bound()` " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Prep for the upcoming `git_deflate_bound()` widening to `size_t`. The
local is only ever the return value of `git_deflate_bound()` and the
`xmalloc()`/`stream.avail_out` sizes derived from it; widening it has no
semantic effect today.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
t/helper/test-pack-deltas.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/t/helper/test-pack-deltas.c b/t/helper/test-pack-deltas.c
index 5e0f726842..959705feca 100644
--- a/t/helper/test-pack-deltas.c
+++ b/t/helper/test-pack-deltas.c
@@ -22,7 +22,7 @@ static unsigned long do_compress(void **pptr, unsigned long size)
{
git_zstream stream;
void *in, *out;
- unsigned long maxsize;
+ size_t maxsize;
git_deflate_init(&stream, 1);
maxsize = git_deflate_bound(&stream, size);
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH 12/12] git-zlib: widen `git_deflate_bound()` to `size_t`
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (10 preceding siblings ...)
2026-07-09 16:49 ` [PATCH 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local " Johannes Schindelin via GitGitGadget
@ 2026-07-09 16:49 ` Johannes Schindelin via GitGitGadget
2026-08-05 9:23 ` Patrick Steinhardt
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
12 siblings, 1 reply; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-07-09 16:49 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
All four `unsigned long`/`int`/`ssize_t` receivers across archive-zip,
diff, http-push and t/helper/test-pack-deltas were widened to `size_t`
in the prior commits, and remote-curl and fast-import were already
there. With every caller prepared, both the parameter and the return
type can now move without introducing any silent narrowing.
For inputs above zlib's `uLong` range (i.e. >4 GiB on platforms where
`uLong` is 32-bit, notably 64-bit Windows), defer to zlib's stored-block
formula (the same fallback it would itself use for an unknown stream
state) plus the worst-case wrapper overhead. The existing path through
`deflateBound()` is unchanged for inputs that fit.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
git-zlib.c | 16 ++++++++++++++--
git-zlib.h | 2 +-
2 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/git-zlib.c b/git-zlib.c
index d21adb3bf5..ebbbcc6d1a 100644
--- a/git-zlib.c
+++ b/git-zlib.c
@@ -167,9 +167,21 @@ int git_inflate(git_zstream *strm, int flush)
return status;
}
-unsigned long git_deflate_bound(git_zstream *strm, unsigned long size)
+size_t git_deflate_bound(git_zstream *strm, size_t size)
{
- return deflateBound(&strm->z, size);
+#if SIZE_MAX > ULONG_MAX
+ if (size > maximum_unsigned_value_of_type(uLong))
+ /*
+ * deflateBound() takes uLong, which is 32-bit on
+ * Windows. For inputs above that range, return zlib's
+ * stored-block formula (the conservative path it would
+ * itself use for an unknown stream state) plus the
+ * worst-case wrapper overhead.
+ */
+ return size + (size >> 5) + (size >> 7) + (size >> 11)
+ + 7 + 18;
+#endif
+ return deflateBound(&strm->z, (uLong)size);
}
void git_deflate_init(git_zstream *strm, int level)
diff --git a/git-zlib.h b/git-zlib.h
index 0b24b15bd0..9248d11ca9 100644
--- a/git-zlib.h
+++ b/git-zlib.h
@@ -25,6 +25,6 @@ void git_deflate_end(git_zstream *);
int git_deflate_abort(git_zstream *);
int git_deflate_end_gently(git_zstream *);
int git_deflate(git_zstream *, int flush);
-unsigned long git_deflate_bound(git_zstream *, unsigned long);
+size_t git_deflate_bound(git_zstream *, size_t);
#endif /* GIT_ZLIB_H */
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* Re: [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t`
2026-07-09 16:49 ` [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
@ 2026-08-05 9:22 ` Patrick Steinhardt
2026-08-05 13:51 ` Johannes Schindelin
0 siblings, 1 reply; 39+ messages in thread
From: Patrick Steinhardt @ 2026-08-05 9:22 UTC (permalink / raw)
To: Johannes Schindelin via GitGitGadget; +Cc: git, Johannes Schindelin
On Thu, Jul 09, 2026 at 04:49:28PM +0000, Johannes Schindelin via GitGitGadget wrote:
> diff --git a/diff-delta.c b/diff-delta.c
> index 43c339f010..b6b65d7607 100644
> --- a/diff-delta.c
> +++ b/diff-delta.c
> @@ -125,9 +125,9 @@ struct unpacked_index_entry {
> };
>
> struct delta_index {
> - unsigned long memsize;
> + size_t memsize;
> const void *src_buf;
> - unsigned long src_size;
> + size_t src_size;
> unsigned int hash_mask;
> struct index_entry *hash[FLEX_ARRAY];
> };
`sizeof_delta_index` returns `index->memsize`, so we'll also have to
adapt that function's return value and its callers.
> @@ -140,7 +140,7 @@ struct delta_index * create_delta_index(const void *buf, unsigned long bufsize)
I was about to complain that the input parameter here uses `unsigned
long`, too. But the next patch addresses that.
> struct unpacked_index_entry *entry, **hash;
> struct index_entry *packed_entry, **packed_hash;
> void *mem;
> - unsigned long memsize;
> + size_t memsize;
>
> if (!buf || !bufsize)
> return NULL;
Patrick
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH 03/12] pack-objects: widen delta-cache accounting to `size_t`
2026-07-09 16:49 ` [PATCH 03/12] pack-objects: widen delta-cache accounting " Johannes Schindelin via GitGitGadget
@ 2026-08-05 9:23 ` Patrick Steinhardt
2026-08-05 13:52 ` Johannes Schindelin
0 siblings, 1 reply; 39+ messages in thread
From: Patrick Steinhardt @ 2026-08-05 9:23 UTC (permalink / raw)
To: Johannes Schindelin via GitGitGadget; +Cc: git, Johannes Schindelin
On Thu, Jul 09, 2026 at 04:49:30PM +0000, Johannes Schindelin via GitGitGadget wrote:
> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> index e3760b3492..f89628a760 100644
> --- a/builtin/pack-objects.c
> +++ b/builtin/pack-objects.c
> @@ -260,8 +260,8 @@ static int exclude_promisor_objects_best_effort;
>
> static int use_delta_islands;
>
> -static unsigned long delta_cache_size = 0;
> -static unsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;
> +static size_t delta_cache_size = 0;
> +static size_t max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;
The only other site that assigns `max_delta_cache_size` does so via
`git_config_int()`, so we happily accept negative values for
"pack.deltacachesize". This will cause a change in behaviour here, even
though arguably the behaviour both before and after this patch is broken
in the same way.
Ideally we'd have something like `git_config_size_t()`, or at least use
`git_config_uint()` here. But that could potentially break the case
where somebody mistakenly configured a negative value and took it as
"infinite", which was mostly true before.
In any case, our docs only mention positive values. So maybe this is
something we could fix while at it.
Patrick
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH 04/12] pack-objects: widen `free_unpacked()` return to `size_t`
2026-07-09 16:49 ` [PATCH 04/12] pack-objects: widen `free_unpacked()` return " Johannes Schindelin via GitGitGadget
@ 2026-08-05 9:23 ` Patrick Steinhardt
0 siblings, 0 replies; 39+ messages in thread
From: Patrick Steinhardt @ 2026-08-05 9:23 UTC (permalink / raw)
To: Johannes Schindelin via GitGitGadget; +Cc: git, Johannes Schindelin
On Thu, Jul 09, 2026 at 04:49:31PM +0000, Johannes Schindelin via GitGitGadget wrote:
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> index f89628a760..4737a6a32c 100644
> --- a/builtin/pack-objects.c
> +++ b/builtin/pack-objects.c
> @@ -2972,9 +2972,9 @@ static unsigned int check_delta_limit(struct object_entry *me, unsigned int n)
> return m;
> }
>
> -static unsigned long free_unpacked(struct unpacked *n)
> +static size_t free_unpacked(struct unpacked *n)
> {
> - unsigned long freed_mem = sizeof_delta_index(n->index);
> + size_t freed_mem = sizeof_delta_index(n->index);
Okay. As mentioned on a preceding patch, the function itself still
returns `unsigned long`, which should probably also be corrected in this
patch series.
Patrick
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH 06/12] delta: widen `create_delta()` and `diff_delta()` to `size_t`
2026-07-09 16:49 ` [PATCH 06/12] delta: widen `create_delta()` and `diff_delta()` " Johannes Schindelin via GitGitGadget
@ 2026-08-05 9:23 ` Patrick Steinhardt
0 siblings, 0 replies; 39+ messages in thread
From: Patrick Steinhardt @ 2026-08-05 9:23 UTC (permalink / raw)
To: Johannes Schindelin via GitGitGadget; +Cc: git, Johannes Schindelin
On Thu, Jul 09, 2026 at 04:49:33PM +0000, Johannes Schindelin via GitGitGadget wrote:
[snip]
> Note that GCC struggles a bit to figure out that `deltalen` is always
> initialized before it is used; To help it along, we initialize it to 0.
> This work-around will go away in a later patch series when `deltalen`
> can be widened to `size_t`.
Thanks for putting this note here, I was wondering about that part.
Patrick
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH 12/12] git-zlib: widen `git_deflate_bound()` to `size_t`
2026-07-09 16:49 ` [PATCH 12/12] git-zlib: widen `git_deflate_bound()` " Johannes Schindelin via GitGitGadget
@ 2026-08-05 9:23 ` Patrick Steinhardt
2026-08-05 13:58 ` Johannes Schindelin
0 siblings, 1 reply; 39+ messages in thread
From: Patrick Steinhardt @ 2026-08-05 9:23 UTC (permalink / raw)
To: Johannes Schindelin via GitGitGadget; +Cc: git, Johannes Schindelin
On Thu, Jul 09, 2026 at 04:49:39PM +0000, Johannes Schindelin via GitGitGadget wrote:
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> All four `unsigned long`/`int`/`ssize_t` receivers across archive-zip,
> diff, http-push and t/helper/test-pack-deltas were widened to `size_t`
> in the prior commits, and remote-curl and fast-import were already
> there. With every caller prepared, both the parameter and the return
> type can now move without introducing any silent narrowing.
Nit, feel free to ignore: I feel like all of these patches could've been
squashed into a single one, as they're trivial enough.
> For inputs above zlib's `uLong` range (i.e. >4 GiB on platforms where
> `uLong` is 32-bit, notably 64-bit Windows), defer to zlib's stored-block
> formula (the same fallback it would itself use for an unknown stream
> state) plus the worst-case wrapper overhead. The existing path through
> `deflateBound()` is unchanged for inputs that fit.
A link or something like that to the formula would've helped here, as
I'm not familiar with this mechanism.
> diff --git a/git-zlib.c b/git-zlib.c
> index d21adb3bf5..ebbbcc6d1a 100644
> --- a/git-zlib.c
> +++ b/git-zlib.c
> @@ -167,9 +167,21 @@ int git_inflate(git_zstream *strm, int flush)
> return status;
> }
>
> -unsigned long git_deflate_bound(git_zstream *strm, unsigned long size)
> +size_t git_deflate_bound(git_zstream *strm, size_t size)
> {
> - return deflateBound(&strm->z, size);
> +#if SIZE_MAX > ULONG_MAX
> + if (size > maximum_unsigned_value_of_type(uLong))
> + /*
> + * deflateBound() takes uLong, which is 32-bit on
> + * Windows. For inputs above that range, return zlib's
> + * stored-block formula (the conservative path it would
> + * itself use for an unknown stream state) plus the
> + * worst-case wrapper overhead.
> + */
> + return size + (size >> 5) + (size >> 7) + (size >> 11)
> + + 7 + 18;
> +#endif
So is the idea here that we estimate the highest number of bytes that
the deflated size could end up with?
Patrick
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t`
2026-08-05 9:22 ` Patrick Steinhardt
@ 2026-08-05 13:51 ` Johannes Schindelin
0 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin @ 2026-08-05 13:51 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: Johannes Schindelin via GitGitGadget, git
Hi Patrick,
On Wed, 5 Aug 2026, Patrick Steinhardt wrote:
> On Thu, Jul 09, 2026 at 04:49:28PM +0000, Johannes Schindelin via GitGitGadget wrote:
> > diff --git a/diff-delta.c b/diff-delta.c
> > index 43c339f010..b6b65d7607 100644
> > --- a/diff-delta.c
> > +++ b/diff-delta.c
> > @@ -125,9 +125,9 @@ struct unpacked_index_entry {
> > };
> >
> > struct delta_index {
> > - unsigned long memsize;
> > + size_t memsize;
> > const void *src_buf;
> > - unsigned long src_size;
> > + size_t src_size;
> > unsigned int hash_mask;
> > struct index_entry *hash[FLEX_ARRAY];
> > };
>
> `sizeof_delta_index` returns `index->memsize`, so we'll also have to
> adapt that function's return value and its callers.
Good call! Will fix.
Ciao,
Johannes
>
> > @@ -140,7 +140,7 @@ struct delta_index * create_delta_index(const void *buf, unsigned long bufsize)
>
> I was about to complain that the input parameter here uses `unsigned
> long`, too. But the next patch addresses that.
>
> > struct unpacked_index_entry *entry, **hash;
> > struct index_entry *packed_entry, **packed_hash;
> > void *mem;
> > - unsigned long memsize;
> > + size_t memsize;
> >
> > if (!buf || !bufsize)
> > return NULL;
>
> Patrick
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH 03/12] pack-objects: widen delta-cache accounting to `size_t`
2026-08-05 9:23 ` Patrick Steinhardt
@ 2026-08-05 13:52 ` Johannes Schindelin
0 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin @ 2026-08-05 13:52 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: Johannes Schindelin via GitGitGadget, git
Hi Patrick,
On Wed, 5 Aug 2026, Patrick Steinhardt wrote:
> On Thu, Jul 09, 2026 at 04:49:30PM +0000, Johannes Schindelin via GitGitGadget wrote:
> > diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> > index e3760b3492..f89628a760 100644
> > --- a/builtin/pack-objects.c
> > +++ b/builtin/pack-objects.c
> > @@ -260,8 +260,8 @@ static int exclude_promisor_objects_best_effort;
> >
> > static int use_delta_islands;
> >
> > -static unsigned long delta_cache_size = 0;
> > -static unsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;
> > +static size_t delta_cache_size = 0;
> > +static size_t max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;
>
> The only other site that assigns `max_delta_cache_size` does so via
> `git_config_int()`, so we happily accept negative values for
> "pack.deltacachesize". This will cause a change in behaviour here, even
> though arguably the behaviour both before and after this patch is broken
> in the same way.
>
> Ideally we'd have something like `git_config_size_t()`, or at least use
> `git_config_uint()` here.
Heh, I looked for `git_config_size_t()` and it does not exist, but
`git_config_ssize_t()` exists... Pretty inconsistent. Anyway, I added that
function and use it in the other assignment.
Ciao,
Johannes
> But that could potentially break the case where somebody mistakenly
> configured a negative value and took it as "infinite", which was mostly
> true before.
>
> In any case, our docs only mention positive values. So maybe this is
> something we could fix while at it.
>
> Patrick
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH 12/12] git-zlib: widen `git_deflate_bound()` to `size_t`
2026-08-05 9:23 ` Patrick Steinhardt
@ 2026-08-05 13:58 ` Johannes Schindelin
0 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin @ 2026-08-05 13:58 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: Johannes Schindelin via GitGitGadget, git
Hi Patrick,
On Wed, 5 Aug 2026, Patrick Steinhardt wrote:
> On Thu, Jul 09, 2026 at 04:49:39PM +0000, Johannes Schindelin via GitGitGadget wrote:
> > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> >
> > All four `unsigned long`/`int`/`ssize_t` receivers across archive-zip,
> > diff, http-push and t/helper/test-pack-deltas were widened to `size_t`
> > in the prior commits, and remote-curl and fast-import were already
> > there. With every caller prepared, both the parameter and the return
> > type can now move without introducing any silent narrowing.
>
> Nit, feel free to ignore: I feel like all of these patches could've been
> squashed into a single one, as they're trivial enough.
I like them trivial and small ;-)
> > For inputs above zlib's `uLong` range (i.e. >4 GiB on platforms where
> > `uLong` is 32-bit, notably 64-bit Windows), defer to zlib's stored-block
> > formula (the same fallback it would itself use for an unknown stream
> > state) plus the worst-case wrapper overhead. The existing path through
> > `deflateBound()` is unchanged for inputs that fit.
>
> A link or something like that to the formula would've helped here, as
> I'm not familiar with this mechanism.
Right. I added two references to the commit message.
>
> > diff --git a/git-zlib.c b/git-zlib.c
> > index d21adb3bf5..ebbbcc6d1a 100644
> > --- a/git-zlib.c
> > +++ b/git-zlib.c
> > @@ -167,9 +167,21 @@ int git_inflate(git_zstream *strm, int flush)
> > return status;
> > }
> >
> > -unsigned long git_deflate_bound(git_zstream *strm, unsigned long size)
> > +size_t git_deflate_bound(git_zstream *strm, size_t size)
> > {
> > - return deflateBound(&strm->z, size);
> > +#if SIZE_MAX > ULONG_MAX
> > + if (size > maximum_unsigned_value_of_type(uLong))
> > + /*
> > + * deflateBound() takes uLong, which is 32-bit on
> > + * Windows. For inputs above that range, return zlib's
> > + * stored-block formula (the conservative path it would
> > + * itself use for an unknown stream state) plus the
> > + * worst-case wrapper overhead.
> > + */
> > + return size + (size >> 5) + (size >> 7) + (size >> 11)
> > + + 7 + 18;
> > +#endif
>
> So is the idea here that we estimate the highest number of bytes that
> the deflated size could end up with?
Precisely. And the formula in zlib is a bit complex, it calculates a
"fixedlen" and a "storelen" for two different ways to represent the worst
case size. But for large values, only `storelen` matters, therefore we can
get away with a much simpler logic here.
FWIW zlib v1.3.2 added `deflateBound_z()`, which accepts `size_t` (or more
precisely: `z_size_t`). However, v1.3.2 is only 7 months old, so I'll be
retired by the time Debian stable gets it :-P
Ciao,
Johannes
>
> Patrick
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v2 00/12] Next size_t stop: pack-objects/delta
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (11 preceding siblings ...)
2026-07-09 16:49 ` [PATCH 12/12] git-zlib: widen `git_deflate_bound()` " Johannes Schindelin via GitGitGadget
@ 2026-08-05 16:14 ` Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
` (12 more replies)
12 siblings, 13 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-08-05 16:14 UTC (permalink / raw)
To: git; +Cc: Patrick Steinhardt, Johannes Schindelin
This patch series continues the effort to stop using unsigned long where
size_t should have been used in the first place. This makes a difference on
64-bit Windows, where unsigned long is 32-bit.
With these fixes, the pack-objects machinery works as intended on 64-bit
Windows (and any other 64-bit platform where unsigned long isn't 64-bit).
Changes since v1:
* The return value of sizeof_delta_index() is now included in the unsigned
long -> size_t work.
* To assign correct values to the now-widened max_delta_cache_size, a new
pair of helpers are introduced and used: git_parse_size_t() and
git_config_size_t()
* There are now two references regarding the provenance of the
deflateBound() formula in the corresponding commit message.
Johannes Schindelin (12):
diff-delta: widen `struct delta_index`' size fields to `size_t`
delta: widen `create_delta_index()` parameter to `size_t`
pack-objects: widen delta-cache accounting to `size_t`
pack-objects: widen `free_unpacked()` return to `size_t`
pack-objects: widen `mem_usage` and `try_delta()`'s out-param to
`size_t`
delta: widen `create_delta()` and `diff_delta()` to `size_t`
packfile, git-zlib: widen `use_pack()` and zstream avail fields to
`size_t`
archive-zip: widen `zlib_deflate_raw()`'s maxsize local to `size_t`
diff: widen `deflate_it()`'s bound local from int to `size_t`
http-push: widen `start_put()`'s size local from `ssize_t` to `size_t`
t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local to
`size_t`
git-zlib: widen `git_deflate_bound()` to `size_t`
archive-zip.c | 2 +-
builtin/fast-import.c | 6 ++++--
builtin/pack-objects.c | 32 +++++++++++++++++---------------
config.c | 9 +++++++++
config.h | 3 +++
delta.h | 14 +++++++-------
diff-delta.c | 14 +++++++-------
diff.c | 6 ++++--
git-zlib.c | 16 ++++++++++++++--
git-zlib.h | 6 +++---
http-push.c | 2 +-
pack-check.c | 4 ++--
packfile.c | 4 ++--
packfile.h | 3 ++-
parse.c | 9 +++++++++
parse.h | 1 +
t/helper/test-delta.c | 2 +-
t/helper/test-pack-deltas.c | 7 ++++---
18 files changed, 91 insertions(+), 49 deletions(-)
base-commit: f85a7e662054a7b0d9070e432508831afa214b47
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2175%2Fdscho%2Fsize-t%2Fpack-objects-delta-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2175/dscho/size-t/pack-objects-delta-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/2175
Range-diff vs v1:
1: 69c2c21f05 ! 1: 0012c1007b diff-delta: widen `struct delta_index`' size fields to `size_t`
@@ Commit message
bits on Windows because the public API around it still truncates.
Splitting it out keeps the API-change commit focused on caller updates.
+ Since the `memsize` attribute is returned by the `sizeof_delta_index()`
+ function verbatim, that function's return type is adjusted, too.
+
Assisted-by: Opus 4.7
+ Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
+ ## delta.h ##
+@@ delta.h: void free_delta_index(struct delta_index *index);
+ *
+ * Given pointer must be what create_delta_index() returned, or NULL.
+ */
+-unsigned long sizeof_delta_index(struct delta_index *index);
++size_t sizeof_delta_index(struct delta_index *index);
+
+ /*
+ * create_delta: create a delta from given index for the given buffer
+
## diff-delta.c ##
@@ diff-delta.c: struct unpacked_index_entry {
};
@@ diff-delta.c: struct delta_index * create_delta_index(const void *buf, unsigned
if (!buf || !bufsize)
return NULL;
+@@ diff-delta.c: void free_delta_index(struct delta_index *index)
+ free(index);
+ }
+
+-unsigned long sizeof_delta_index(struct delta_index *index)
++size_t sizeof_delta_index(struct delta_index *index)
+ {
+ if (index)
+ return index->memsize;
2: d92a5d4dec = 2: 75500c5abb delta: widen `create_delta_index()` parameter to `size_t`
3: 4ef2886549 ! 3: 5b54041baf pack-objects: widen delta-cache accounting to `size_t`
@@ Commit message
shims, which becomes possible once 1create_delta()` and `diff_delta()`
are widened in a later commit.
+ Note: since `max_delta_cache_size` changes data type to `size_t`, a pair
+ of new helpers is introduced to parse config values of that type, too.
+
Assisted-by: Opus 4.7
+ Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
## builtin/pack-objects.c ##
@@ builtin/pack-objects.c: struct unpacked {
{
if (max_delta_cache_size && delta_cache_size + delta_size > max_delta_cache_size)
return 0;
+@@ builtin/pack-objects.c: static int git_pack_config(const char *k, const char *v,
+ return 0;
+ }
+ if (!strcmp(k, "pack.deltacachesize")) {
+- max_delta_cache_size = git_config_int(k, v, ctx->kvi);
++ max_delta_cache_size = git_config_size_t(k, v, ctx->kvi);
+ return 0;
+ }
+ if (!strcmp(k, "pack.deltacachelimit")) {
+
+ ## config.c ##
+@@ config.c: ssize_t git_config_ssize_t(const char *name, const char *value,
+ return ret;
+ }
+
++size_t git_config_size_t(const char *name, const char *value,
++ const struct key_value_info *kvi)
++{
++ size_t ret;
++ if (!git_parse_size_t(value, &ret))
++ die_bad_number(name, value, kvi);
++ return ret;
++}
++
+ double git_config_double(const char *name, const char *value,
+ const struct key_value_info *kvi)
+ {
+
+ ## config.h ##
+@@ config.h: unsigned long git_config_ulong(const char *, const char *,
+ ssize_t git_config_ssize_t(const char *, const char *,
+ const struct key_value_info *);
+
++size_t git_config_size_t(const char *, const char *,
++ const struct key_value_info *);
++
+ /**
+ * Identically to `git_config_double`, but for double-precision floating point
+ * values.
+
+ ## parse.c ##
+@@ parse.c: int git_parse_ssize_t(const char *value, ssize_t *ret)
+ return 1;
+ }
+
++int git_parse_size_t(const char *value, size_t *ret)
++{
++ uintmax_t tmp;
++ if (!git_parse_unsigned(value, &tmp, maximum_signed_value_of_type(size_t)))
++ return 0;
++ *ret = tmp;
++ return 1;
++}
++
+ int git_parse_double(const char *value, double *ret)
+ {
+ char *end;
+
+ ## parse.h ##
+@@
+ int git_parse_signed(const char *value, intmax_t *ret, intmax_t max);
+ int git_parse_unsigned(const char *value, uintmax_t *ret, uintmax_t max);
+ int git_parse_ssize_t(const char *, ssize_t *);
++int git_parse_size_t(const char *, size_t *);
+ int git_parse_ulong(const char *, unsigned long *);
+ int git_parse_uint(const char *value, unsigned int *ret);
+ int git_parse_int(const char *value, int *ret);
4: 07d01200a4 = 4: 9850de1a91 pack-objects: widen `free_unpacked()` return to `size_t`
5: 7dca160102 = 5: c301958284 pack-objects: widen `mem_usage` and `try_delta()`'s out-param to `size_t`
6: e1ae83ba03 ! 6: cfbf6c9567 delta: widen `create_delta()` and `diff_delta()` to `size_t`
@@ builtin/pack-objects.c: static int try_delta(struct unpacked *trg, struct unpack
void *delta_buf;
## delta.h ##
-@@ delta.h: unsigned long sizeof_delta_index(struct delta_index *index);
+@@ delta.h: size_t sizeof_delta_index(struct delta_index *index);
*/
void *
create_delta(const struct delta_index *index,
@@ delta.h: create_delta(const struct delta_index *index,
if (index) {
## diff-delta.c ##
-@@ diff-delta.c: unsigned long sizeof_delta_index(struct delta_index *index)
+@@ diff-delta.c: size_t sizeof_delta_index(struct delta_index *index)
void *
create_delta(const struct delta_index *index,
7: 8353bc03c1 = 7: ca928b4579 packfile, git-zlib: widen `use_pack()` and zstream avail fields to `size_t`
8: acffd232ac = 8: 9f379ee7aa archive-zip: widen `zlib_deflate_raw()`'s maxsize local to `size_t`
9: b89d28c8aa = 9: ff103a0ee1 diff: widen `deflate_it()`'s bound local from int to `size_t`
10: 2d4d19c5fb = 10: c701d2f9b2 http-push: widen `start_put()`'s size local from `ssize_t` to `size_t`
11: 617960d9ca = 11: e6175d2d87 t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local to `size_t`
12: ab911cf556 ! 12: 762e716afe git-zlib: widen `git_deflate_bound()` to `size_t`
@@ Commit message
For inputs above zlib's `uLong` range (i.e. >4 GiB on platforms where
`uLong` is 32-bit, notably 64-bit Windows), defer to zlib's stored-block
- formula (the same fallback it would itself use for an unknown stream
- state) plus the worst-case wrapper overhead. The existing path through
+ formula (the same fallback it would itself use, see
+ https://github.com/madler/zlib/blob/v1.3.2/deflate.c#L832-L928 keeping
+ in mind that for large sizes, the `storelen` would be relevant, also
+ compare with https://github.com/madler/zlib/issues/549 for a fuller
+ story) plus the worst-case wrapper overhead. The existing path through
`deflateBound()` is unchanged for inputs that fit.
Assisted-by: Opus 4.7
--
gitgitgadget
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v2 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t`
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
@ 2026-08-05 16:14 ` Johannes Schindelin via GitGitGadget
2026-08-06 6:15 ` Patrick Steinhardt
2026-08-05 16:14 ` [PATCH v2 02/12] delta: widen `create_delta_index()` parameter " Johannes Schindelin via GitGitGadget
` (11 subsequent siblings)
12 siblings, 1 reply; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-08-05 16:14 UTC (permalink / raw)
To: git; +Cc: Patrick Steinhardt, Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Preparation for widening the delta-encoding API to `size_t` in
subsequent commits, which is what lets pack-objects drop the
`cast_size_t_to_ulong()` shims that 606c192380 (odb, packfile: use
size_t for streaming object sizes, 2026-05-08) had to leave behind in
`get_delta()` and `try_delta()` because their downstream consumers were
still narrow.
The struct is private to diff-delta.c, so widening its fields in
isolation is a no-op at runtime: the values stored continue to fit in 32
bits on Windows because the public API around it still truncates.
Splitting it out keeps the API-change commit focused on caller updates.
Since the `memsize` attribute is returned by the `sizeof_delta_index()`
function verbatim, that function's return type is adjusted, too.
Assisted-by: Opus 4.7
Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
delta.h | 2 +-
diff-delta.c | 8 ++++----
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/delta.h b/delta.h
index eb5c6d2fdb..ab0279168c 100644
--- a/delta.h
+++ b/delta.h
@@ -28,7 +28,7 @@ void free_delta_index(struct delta_index *index);
*
* Given pointer must be what create_delta_index() returned, or NULL.
*/
-unsigned long sizeof_delta_index(struct delta_index *index);
+size_t sizeof_delta_index(struct delta_index *index);
/*
* create_delta: create a delta from given index for the given buffer
diff --git a/diff-delta.c b/diff-delta.c
index 43c339f010..9e1f9e6f95 100644
--- a/diff-delta.c
+++ b/diff-delta.c
@@ -125,9 +125,9 @@ struct unpacked_index_entry {
};
struct delta_index {
- unsigned long memsize;
+ size_t memsize;
const void *src_buf;
- unsigned long src_size;
+ size_t src_size;
unsigned int hash_mask;
struct index_entry *hash[FLEX_ARRAY];
};
@@ -140,7 +140,7 @@ struct delta_index * create_delta_index(const void *buf, unsigned long bufsize)
struct unpacked_index_entry *entry, **hash;
struct index_entry *packed_entry, **packed_hash;
void *mem;
- unsigned long memsize;
+ size_t memsize;
if (!buf || !bufsize)
return NULL;
@@ -302,7 +302,7 @@ void free_delta_index(struct delta_index *index)
free(index);
}
-unsigned long sizeof_delta_index(struct delta_index *index)
+size_t sizeof_delta_index(struct delta_index *index)
{
if (index)
return index->memsize;
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v2 02/12] delta: widen `create_delta_index()` parameter to `size_t`
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
@ 2026-08-05 16:14 ` Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 03/12] pack-objects: widen delta-cache accounting " Johannes Schindelin via GitGitGadget
` (10 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-08-05 16:14 UTC (permalink / raw)
To: git; +Cc: Patrick Steinhardt, Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
The sole caller (`try_delta()` in builtin/pack-objects.c) passes an
`unsigned long`, which promotes safely, so no caller fixups are needed.
Splitting it out keeps the `diff_delta()`/`create_delta()` widening,
which does ripple to several callers, in its own commit.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
delta.h | 2 +-
diff-delta.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/delta.h b/delta.h
index ab0279168c..12075c54c5 100644
--- a/delta.h
+++ b/delta.h
@@ -14,7 +14,7 @@ struct delta_index;
* using free_delta_index().
*/
struct delta_index *
-create_delta_index(const void *buf, unsigned long bufsize);
+create_delta_index(const void *buf, size_t bufsize);
/*
* free_delta_index: free the index created by create_delta_index()
diff --git a/diff-delta.c b/diff-delta.c
index 9e1f9e6f95..bcc331af3e 100644
--- a/diff-delta.c
+++ b/diff-delta.c
@@ -132,7 +132,7 @@ struct delta_index {
struct index_entry *hash[FLEX_ARRAY];
};
-struct delta_index * create_delta_index(const void *buf, unsigned long bufsize)
+struct delta_index * create_delta_index(const void *buf, size_t bufsize)
{
unsigned int i, hsize, hmask, entries, prev_val, *hash_count;
const unsigned char *data, *buffer = buf;
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v2 03/12] pack-objects: widen delta-cache accounting to `size_t`
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 02/12] delta: widen `create_delta_index()` parameter " Johannes Schindelin via GitGitGadget
@ 2026-08-05 16:14 ` Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 04/12] pack-objects: widen `free_unpacked()` return " Johannes Schindelin via GitGitGadget
` (9 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-08-05 16:14 UTC (permalink / raw)
To: git; +Cc: Patrick Steinhardt, Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
These three are a single accounting tuple (the globals tracking
cumulative cached-delta bytes, plus the helper that compares them
against an incoming delta size) and are latently 32-bit on Windows where
`unsigned long` != `size_t`: a pack with many large cached deltas could
wrap silently.
The widening is internally consistent on its own: the additions and
subtractions against delta_cache_size already come from `size_t` sources
(`DELTA_SIZE()` returns `size_t`), and `delta_cacheable()`'s sole caller
in `try_delta()` still passes `unsigned long`, which promotes.
Prerequisite for dropping `try_delta()`'s `cast_size_t_to_ulong()`
shims, which becomes possible once 1create_delta()` and `diff_delta()`
are widened in a later commit.
Note: since `max_delta_cache_size` changes data type to `size_t`, a pair
of new helpers is introduced to parse config values of that type, too.
Assisted-by: Opus 4.7
Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
builtin/pack-objects.c | 10 +++++-----
config.c | 9 +++++++++
config.h | 3 +++
parse.c | 9 +++++++++
parse.h | 1 +
5 files changed, 27 insertions(+), 5 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index e3760b3492..97246c69ae 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -260,8 +260,8 @@ static int exclude_promisor_objects_best_effort;
static int use_delta_islands;
-static unsigned long delta_cache_size = 0;
-static unsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;
+static size_t delta_cache_size = 0;
+static size_t max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;
static unsigned long cache_max_small_delta_size = 1000;
static unsigned long window_memory_limit = 0;
@@ -2688,8 +2688,8 @@ struct unpacked {
unsigned depth;
};
-static int delta_cacheable(unsigned long src_size, unsigned long trg_size,
- unsigned long delta_size)
+static int delta_cacheable(size_t src_size, size_t trg_size,
+ size_t delta_size)
{
if (max_delta_cache_size && delta_cache_size + delta_size > max_delta_cache_size)
return 0;
@@ -3701,7 +3701,7 @@ static int git_pack_config(const char *k, const char *v,
return 0;
}
if (!strcmp(k, "pack.deltacachesize")) {
- max_delta_cache_size = git_config_int(k, v, ctx->kvi);
+ max_delta_cache_size = git_config_size_t(k, v, ctx->kvi);
return 0;
}
if (!strcmp(k, "pack.deltacachelimit")) {
diff --git a/config.c b/config.c
index 6a0de86e3a..010a58d307 100644
--- a/config.c
+++ b/config.c
@@ -1268,6 +1268,15 @@ ssize_t git_config_ssize_t(const char *name, const char *value,
return ret;
}
+size_t git_config_size_t(const char *name, const char *value,
+ const struct key_value_info *kvi)
+{
+ size_t ret;
+ if (!git_parse_size_t(value, &ret))
+ die_bad_number(name, value, kvi);
+ return ret;
+}
+
double git_config_double(const char *name, const char *value,
const struct key_value_info *kvi)
{
diff --git a/config.h b/config.h
index 31fe3e2961..b66dd08007 100644
--- a/config.h
+++ b/config.h
@@ -282,6 +282,9 @@ unsigned long git_config_ulong(const char *, const char *,
ssize_t git_config_ssize_t(const char *, const char *,
const struct key_value_info *);
+size_t git_config_size_t(const char *, const char *,
+ const struct key_value_info *);
+
/**
* Identically to `git_config_double`, but for double-precision floating point
* values.
diff --git a/parse.c b/parse.c
index d77f28046a..266bbd539b 100644
--- a/parse.c
+++ b/parse.c
@@ -134,6 +134,15 @@ int git_parse_ssize_t(const char *value, ssize_t *ret)
return 1;
}
+int git_parse_size_t(const char *value, size_t *ret)
+{
+ uintmax_t tmp;
+ if (!git_parse_unsigned(value, &tmp, maximum_signed_value_of_type(size_t)))
+ return 0;
+ *ret = tmp;
+ return 1;
+}
+
int git_parse_double(const char *value, double *ret)
{
char *end;
diff --git a/parse.h b/parse.h
index a6dd37c4cb..db742f35fb 100644
--- a/parse.h
+++ b/parse.h
@@ -4,6 +4,7 @@
int git_parse_signed(const char *value, intmax_t *ret, intmax_t max);
int git_parse_unsigned(const char *value, uintmax_t *ret, uintmax_t max);
int git_parse_ssize_t(const char *, ssize_t *);
+int git_parse_size_t(const char *, size_t *);
int git_parse_ulong(const char *, unsigned long *);
int git_parse_uint(const char *value, unsigned int *ret);
int git_parse_int(const char *value, int *ret);
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v2 04/12] pack-objects: widen `free_unpacked()` return to `size_t`
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (2 preceding siblings ...)
2026-08-05 16:14 ` [PATCH v2 03/12] pack-objects: widen delta-cache accounting " Johannes Schindelin via GitGitGadget
@ 2026-08-05 16:14 ` Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param " Johannes Schindelin via GitGitGadget
` (8 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-08-05 16:14 UTC (permalink / raw)
To: git; +Cc: Patrick Steinhardt, Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
`free_unpacked()` sums two byte counts: `sizeof_delta_index()` and
`SIZE(n->entry)`. The latter has been `size_t` since the prior topic
"More work supporting objects larger than 4GB on Windows" widened
`SIZE()`/`oe_size()` to `size_t`, so accumulating it into an `unsigned
long` return was a silent Windows-only truncation on a packing run with
many large objects.
The sole caller, `find_deltas()`, still holds its own `mem_usage` in an
`unsigned long` for now, and therefore still truncates silently.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
builtin/pack-objects.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 97246c69ae..503ebbf091 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -2972,9 +2972,9 @@ static unsigned int check_delta_limit(struct object_entry *me, unsigned int n)
return m;
}
-static unsigned long free_unpacked(struct unpacked *n)
+static size_t free_unpacked(struct unpacked *n)
{
- unsigned long freed_mem = sizeof_delta_index(n->index);
+ size_t freed_mem = sizeof_delta_index(n->index);
free_delta_index(n->index);
n->index = NULL;
if (n->data) {
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v2 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param to `size_t`
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (3 preceding siblings ...)
2026-08-05 16:14 ` [PATCH v2 04/12] pack-objects: widen `free_unpacked()` return " Johannes Schindelin via GitGitGadget
@ 2026-08-05 16:14 ` Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 06/12] delta: widen `create_delta()` and `diff_delta()` " Johannes Schindelin via GitGitGadget
` (7 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-08-05 16:14 UTC (permalink / raw)
To: git; +Cc: Patrick Steinhardt, Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
The pair must move together because `find_deltas()` passes `&mem_usage`
to `try_delta()`: widening either alone breaks the type match.
`mem_usage` accumulates per-object byte counts already computed in
`size_t` (`SIZE()` and `sizeof_delta_index()` reach here through
`free_unpacked()`, now `size_t`), and was the last 32-bit-on-Windows
narrowing point in the delta-window memory accounting chain. With this
commit, that chain uses `size_t` consistently except for
`sizeof_delta_index()`'s still-narrow return, whose value is bounded by
`create_delta_index()`'s entries cap.
`window_memory_limit` (config-driven via `git_config_ulong()`) stays
`unsigned long`: it is only compared against `mem_usage` and promotes.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
builtin/pack-objects.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 503ebbf091..96ecee393e 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -2804,7 +2804,7 @@ size_t oe_get_size_slow(struct packing_data *pack,
}
static int try_delta(struct unpacked *trg, struct unpacked *src,
- unsigned max_depth, unsigned long *mem_usage)
+ unsigned max_depth, size_t *mem_usage)
{
struct object_entry *trg_entry = trg->entry;
struct object_entry *src_entry = src->entry;
@@ -2991,7 +2991,7 @@ static void find_deltas(struct object_entry **list, unsigned *list_size,
{
uint32_t i, idx = 0, count = 0;
struct unpacked *array;
- unsigned long mem_usage = 0;
+ size_t mem_usage = 0;
CALLOC_ARRAY(array, window);
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v2 06/12] delta: widen `create_delta()` and `diff_delta()` to `size_t`
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (4 preceding siblings ...)
2026-08-05 16:14 ` [PATCH v2 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param " Johannes Schindelin via GitGitGadget
@ 2026-08-05 16:14 ` Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields " Johannes Schindelin via GitGitGadget
` (6 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-08-05 16:14 UTC (permalink / raw)
To: git; +Cc: Patrick Steinhardt, Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Last stop in the delta-encoding API widening for >4 GiB blobs on
Windows: with `create_delta_index()` done in the prior commit and
`create_delta()`/`diff_delta()` finished here, every byte count that
crosses delta.h is now `size_t`. The struct fields they store into have
been `size_t` since the diff-delta struct widening.
The API change must move with all callers in the same commit (the build
only passes when every `&delta_size` matches the new `size_t*`). Caller
updates are kept minimal:
* builtin/pack-objects.c `get_delta()` and `try_delta()`: widen only
the local `delta_size` variable; the surrounding unsigned-long
locals and their `cast_size_t_to_ulong()` shims are out of scope
here and will be cleaned up in their own commits.
* builtin/fast-import.c, diff.c, t/helper/test-pack-deltas.c:
keep the local unsigned-long delta size (each feeds a still-
unsigned-long downstream consumer: zlib's `avail_in`,
`deflate_it()`, the test helper's own `do_compress()`), and bridge
via a temporary `size_t` plus `cast_size_t_to_ulong()`. The new
casts are paid back in later topics that widen those consumers.
* t/helper/test-delta.c: widen the local outright (no downstream
consumer beyond the test's own `out_size`, which is already
`size_t`).
Note that GCC struggles a bit to figure out that `deltalen` is always
initialized before it is used; To help it along, we initialize it to 0.
This work-around will go away in a later patch series when `deltalen`
can be widened to `size_t`.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
builtin/fast-import.c | 6 ++++--
builtin/pack-objects.c | 6 ++++--
delta.h | 10 +++++-----
diff-delta.c | 4 ++--
diff.c | 4 +++-
t/helper/test-delta.c | 2 +-
t/helper/test-pack-deltas.c | 5 +++--
7 files changed, 22 insertions(+), 15 deletions(-)
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index aa656c5195..1c6e5366c2 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -962,7 +962,7 @@ static int store_object(
struct object_entry *e;
unsigned char hdr[96];
struct object_id oid;
- unsigned long hdrlen, deltalen;
+ unsigned long hdrlen, deltalen = 0;
struct git_hash_ctx c;
git_zstream s;
struct repo_config_values *cfg = repo_config_values(the_repository);
@@ -998,11 +998,13 @@ static int store_object(
if (last && last->data.len && last->data.buf && last->depth < max_depth
&& dat->len > the_hash_algo->rawsz) {
+ size_t deltalen_st;
delta_count_attempts_by_type[type]++;
delta = diff_delta(last->data.buf, last->data.len,
dat->buf, dat->len,
- &deltalen, dat->len - the_hash_algo->rawsz);
+ &deltalen_st, dat->len - the_hash_algo->rawsz);
+ deltalen = cast_size_t_to_ulong(deltalen_st);
} else
delta = NULL;
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 96ecee393e..08c6d294cc 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -353,7 +353,8 @@ static void index_commit_for_bitmap(struct commit *commit)
static void *get_delta(struct object_entry *entry)
{
- unsigned long size, base_size, delta_size;
+ unsigned long size, base_size;
+ size_t delta_size;
void *buf, *base_buf, *delta_buf;
enum object_type type;
size_t size_st = 0, base_size_st = 0;
@@ -2808,7 +2809,8 @@ static int try_delta(struct unpacked *trg, struct unpacked *src,
{
struct object_entry *trg_entry = trg->entry;
struct object_entry *src_entry = src->entry;
- unsigned long trg_size, src_size, delta_size, sizediff, max_size, sz;
+ unsigned long trg_size, src_size, sizediff, max_size, sz;
+ size_t delta_size;
unsigned ref_depth;
enum object_type type;
void *delta_buf;
diff --git a/delta.h b/delta.h
index 12075c54c5..42a211905d 100644
--- a/delta.h
+++ b/delta.h
@@ -42,8 +42,8 @@ size_t sizeof_delta_index(struct delta_index *index);
*/
void *
create_delta(const struct delta_index *index,
- const void *buf, unsigned long bufsize,
- unsigned long *delta_size, unsigned long max_delta_size);
+ const void *buf, size_t bufsize,
+ size_t *delta_size, size_t max_delta_size);
/*
* diff_delta: create a delta from source buffer to target buffer
@@ -54,9 +54,9 @@ create_delta(const struct delta_index *index,
* updated with its size. The returned buffer must be freed by the caller.
*/
static inline void *
-diff_delta(const void *src_buf, unsigned long src_bufsize,
- const void *trg_buf, unsigned long trg_bufsize,
- unsigned long *delta_size, unsigned long max_delta_size)
+diff_delta(const void *src_buf, size_t src_bufsize,
+ const void *trg_buf, size_t trg_bufsize,
+ size_t *delta_size, size_t max_delta_size)
{
struct delta_index *index = create_delta_index(src_buf, src_bufsize);
if (index) {
diff --git a/diff-delta.c b/diff-delta.c
index bcc331af3e..7cbedeb507 100644
--- a/diff-delta.c
+++ b/diff-delta.c
@@ -318,8 +318,8 @@ size_t sizeof_delta_index(struct delta_index *index)
void *
create_delta(const struct delta_index *index,
- const void *trg_buf, unsigned long trg_size,
- unsigned long *delta_size, unsigned long max_size)
+ const void *trg_buf, size_t trg_size,
+ size_t *delta_size, size_t max_size)
{
unsigned int i, val;
off_t outpos, moff;
diff --git a/diff.c b/diff.c
index 2a9d0d8687..69eb2f76a4 100644
--- a/diff.c
+++ b/diff.c
@@ -3647,9 +3647,11 @@ static void emit_binary_diff_body(struct diff_options *o,
delta = NULL;
deflated = deflate_it(two->ptr, two->size, &deflate_size);
if (one->size && two->size) {
+ size_t delta_size_st = 0;
delta = diff_delta(one->ptr, one->size,
two->ptr, two->size,
- &delta_size, deflate_size);
+ &delta_size_st, deflate_size);
+ delta_size = cast_size_t_to_ulong(delta_size_st);
if (delta) {
void *to_free = delta;
orig_size = delta_size;
diff --git a/t/helper/test-delta.c b/t/helper/test-delta.c
index 8223a60229..d807afef75 100644
--- a/t/helper/test-delta.c
+++ b/t/helper/test-delta.c
@@ -32,7 +32,7 @@ int cmd__delta(int argc, const char **argv)
die_errno("unable to read '%s'", argv[3]);
if (argv[1][1] == 'd') {
- unsigned long delta_size;
+ size_t delta_size;
out_buf = diff_delta(from.buf, from.len,
data.buf, data.len,
&delta_size, 0);
diff --git a/t/helper/test-pack-deltas.c b/t/helper/test-pack-deltas.c
index 840797cf0d..5e0f726842 100644
--- a/t/helper/test-pack-deltas.c
+++ b/t/helper/test-pack-deltas.c
@@ -49,7 +49,7 @@ static void write_ref_delta(struct hashfile *f,
{
unsigned char header[MAX_PACK_OBJECT_HEADER];
unsigned long delta_size, compressed_size, hdrlen;
- size_t size, base_size;
+ size_t size, base_size, delta_size_st = 0;
enum object_type type;
void *base_buf, *delta_buf;
void *buf = odb_read_object(the_repository->objects,
@@ -65,7 +65,8 @@ static void write_ref_delta(struct hashfile *f,
die("unable to read %s", oid_to_hex(base));
delta_buf = diff_delta(base_buf, base_size,
- buf, size, &delta_size, 0);
+ buf, size, &delta_size_st, 0);
+ delta_size = cast_size_t_to_ulong(delta_size_st);
compressed_size = do_compress(&delta_buf, delta_size);
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v2 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields to `size_t`
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (5 preceding siblings ...)
2026-08-05 16:14 ` [PATCH v2 06/12] delta: widen `create_delta()` and `diff_delta()` " Johannes Schindelin via GitGitGadget
@ 2026-08-05 16:14 ` Johannes Schindelin via GitGitGadget
2026-08-07 21:41 ` Junio C Hamano
2026-08-07 22:06 ` Junio C Hamano
2026-08-05 16:14 ` [PATCH v2 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local " Johannes Schindelin via GitGitGadget
` (5 subsequent siblings)
12 siblings, 2 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-08-05 16:14 UTC (permalink / raw)
To: git; +Cc: Patrick Steinhardt, Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Bundling the two widenings: four call sites pass `&stream.avail_in`
directly to `use_pack()`, and widening either type fencepost alone would
force a bridge variable at each. Doing both together is the simpler end
state and is the prerequisite for the `do_compress()` widening in the
next commit, which is what lets `write_no_reuse_object()` lose its last
`cast_size_t_to_ulong()` shim.
The unsigned-long locals widened at the other `use_pack()` callers
(avail / remaining / left) hold pack-window sizes bounded by
`core.packedGitWindowSize`, so the change is type consistency rather
than a new >4GB capability. `git_zstream.avail_in`/`avail_out` likewise
reach zlib's `uInt` fields only after `zlib_buf_cap()`'s 1 GiB cap, so
the wrapper already accepted `size_t`-shaped inputs in practice.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
builtin/pack-objects.c | 8 ++++----
git-zlib.h | 4 ++--
pack-check.c | 4 ++--
packfile.c | 4 ++--
packfile.h | 3 ++-
5 files changed, 12 insertions(+), 11 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 08c6d294cc..87aa8f44e7 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -488,7 +488,7 @@ static void copy_pack_data(struct hashfile *f,
off_t len)
{
unsigned char *in;
- unsigned long avail;
+ size_t avail;
while (len) {
in = use_pack(p, w_curs, offset, &avail);
@@ -2261,7 +2261,7 @@ static void check_object(struct object_entry *entry, uint32_t object_index)
struct object_id base_ref;
struct object_entry *base_entry;
unsigned long used, used_0;
- unsigned long avail;
+ size_t avail;
off_t ofs;
unsigned char *buf, c;
enum object_type type;
@@ -2773,8 +2773,8 @@ size_t oe_get_size_slow(struct packing_data *pack,
struct pack_window *w_curs;
unsigned char *buf;
enum object_type type;
- unsigned long used, avail;
- size_t size;
+ unsigned long used;
+ size_t avail, size;
if (e->type_ != OBJ_OFS_DELTA && e->type_ != OBJ_REF_DELTA) {
size_t sz;
diff --git a/git-zlib.h b/git-zlib.h
index 44380e8ad3..0b24b15bd0 100644
--- a/git-zlib.h
+++ b/git-zlib.h
@@ -5,8 +5,8 @@
typedef struct git_zstream {
struct z_stream_s z;
- unsigned long avail_in;
- unsigned long avail_out;
+ size_t avail_in;
+ size_t avail_out;
size_t total_in;
size_t total_out;
unsigned char *next_in;
diff --git a/pack-check.c b/pack-check.c
index 5adfb3f272..befb860472 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -34,7 +34,7 @@ int check_pack_crc(struct packed_git *p, struct pack_window **w_curs,
uint32_t data_crc = crc32(0, NULL, 0);
do {
- unsigned long avail;
+ size_t avail;
void *data = use_pack(p, w_curs, offset, &avail);
if (avail > len)
avail = len;
@@ -71,7 +71,7 @@ static int verify_packfile(struct repository *r,
r->hash_algo->init_fn(&ctx);
do {
- unsigned long remaining;
+ size_t remaining;
unsigned char *in = use_pack(p, w_curs, offset, &remaining);
offset += remaining;
if (!pack_sig_ofs)
diff --git a/packfile.c b/packfile.c
index 1d1b23b6cc..629fe46a6a 100644
--- a/packfile.c
+++ b/packfile.c
@@ -620,7 +620,7 @@ static int in_window(struct repository *r, struct pack_window *win,
unsigned char *use_pack(struct packed_git *p,
struct pack_window **w_cursor,
off_t offset,
- unsigned long *left)
+ size_t *left)
{
struct pack_window *win = *w_cursor;
@@ -960,7 +960,7 @@ int unpack_object_header(struct packed_git *p,
size_t *sizep)
{
unsigned char *base;
- unsigned long left;
+ size_t left;
unsigned long used;
enum object_type type;
diff --git a/packfile.h b/packfile.h
index 2329a69701..3cff8bdcb9 100644
--- a/packfile.h
+++ b/packfile.h
@@ -240,7 +240,8 @@ uint32_t get_pack_fanout(struct packed_git *p, uint32_t value);
struct object_database;
-unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);
+unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t,
+ size_t *);
void close_pack_windows(struct packed_git *);
void close_pack(struct packed_git *);
void unuse_pack(struct pack_window **);
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v2 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local to `size_t`
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (6 preceding siblings ...)
2026-08-05 16:14 ` [PATCH v2 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields " Johannes Schindelin via GitGitGadget
@ 2026-08-05 16:14 ` Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 09/12] diff: widen `deflate_it()`'s bound local from int " Johannes Schindelin via GitGitGadget
` (4 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-08-05 16:14 UTC (permalink / raw)
To: git; +Cc: Patrick Steinhardt, Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Prep for the upcoming `git_deflate_bound()` widening to `size_t`: the
local that catches its return needs to be `size_t` too, otherwise the
widening would introduce a silent Windows narrowing here. No semantic
effect with the current unsigned-long-returning `git_deflate_bound()`
(`size_t == unsigned long` on this caller's platforms today).
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
archive-zip.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/archive-zip.c b/archive-zip.c
index 97ea8d60d6..a487d4c041 100644
--- a/archive-zip.c
+++ b/archive-zip.c
@@ -206,7 +206,7 @@ static void *zlib_deflate_raw(void *data, unsigned long size,
unsigned long *compressed_size)
{
git_zstream stream;
- unsigned long maxsize;
+ size_t maxsize;
void *buffer;
int result;
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v2 09/12] diff: widen `deflate_it()`'s bound local from int to `size_t`
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (7 preceding siblings ...)
2026-08-05 16:14 ` [PATCH v2 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local " Johannes Schindelin via GitGitGadget
@ 2026-08-05 16:14 ` Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 10/12] http-push: widen `start_put()`'s size local from `ssize_t` " Johannes Schindelin via GitGitGadget
` (3 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-08-05 16:14 UTC (permalink / raw)
To: git; +Cc: Patrick Steinhardt, Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Fixes a pre-existing silent narrowing from `git_deflate_bound()`'s
`unsigned long` return into an `int` local: anything past 2 GiB has
always wrapped negative here and then been re-extended to `size_t`
inside `xmalloc()`. Also prep for the upcoming `git_deflate_bound()`
widening to `size_t`, which would extend the narrowing further if
`bound` stayed `int`.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
diff.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/diff.c b/diff.c
index 69eb2f76a4..c14f69719b 100644
--- a/diff.c
+++ b/diff.c
@@ -3609,7 +3609,7 @@ static unsigned char *deflate_it(char *data,
unsigned long size,
unsigned long *result_size)
{
- int bound;
+ size_t bound;
unsigned char *deflated;
git_zstream stream;
struct repo_config_values *cfg = repo_config_values(the_repository);
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v2 10/12] http-push: widen `start_put()`'s size local from `ssize_t` to `size_t`
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (8 preceding siblings ...)
2026-08-05 16:14 ` [PATCH v2 09/12] diff: widen `deflate_it()`'s bound local from int " Johannes Schindelin via GitGitGadget
@ 2026-08-05 16:14 ` Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local " Johannes Schindelin via GitGitGadget
` (2 subsequent siblings)
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-08-05 16:14 UTC (permalink / raw)
To: git; +Cc: Patrick Steinhardt, Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
The local is initialised from `git_deflate_bound()` (an unsigned upper
bound on the deflated output, never negative) and used in exactly three
places: the initialising assignment, `strbuf_grow(buf, size)` whose
parameter is already `size_t`, and `stream.avail_out` which became
`size_t` in the prior commit. There is no comparison against zero or a
negative value, no subtraction, no arithmetic that depends on
signedness, and no path that would assign a signed quantity to it.
The original `ssize_t` was the wrong type to begin with: a
`git_deflate_bound()` result above `SSIZE_MAX` would have wrapped
negative on assignment and then implicitly re-extended to a huge
`size_t` at `strbuf_grow()`/`stream.avail_out`, requesting an absurd
allocation. That is not a real-world concern for the object sizes
http-push pushes today, but it is also the reason the type needs to move
to `size_t` before `git_deflate_bound()` itself is widened.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
http-push.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/http-push.c b/http-push.c
index 3c23cbba27..2a07d14259 100644
--- a/http-push.c
+++ b/http-push.c
@@ -367,7 +367,7 @@ static void start_put(struct transfer_request *request)
void *unpacked;
size_t len;
int hdrlen;
- ssize_t size;
+ size_t size;
git_zstream stream;
struct repo_config_values *cfg = repo_config_values(the_repository);
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v2 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local to `size_t`
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (9 preceding siblings ...)
2026-08-05 16:14 ` [PATCH v2 10/12] http-push: widen `start_put()`'s size local from `ssize_t` " Johannes Schindelin via GitGitGadget
@ 2026-08-05 16:14 ` Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 12/12] git-zlib: widen `git_deflate_bound()` " Johannes Schindelin via GitGitGadget
2026-08-06 6:15 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Patrick Steinhardt
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-08-05 16:14 UTC (permalink / raw)
To: git; +Cc: Patrick Steinhardt, Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Prep for the upcoming `git_deflate_bound()` widening to `size_t`. The
local is only ever the return value of `git_deflate_bound()` and the
`xmalloc()`/`stream.avail_out` sizes derived from it; widening it has no
semantic effect today.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
t/helper/test-pack-deltas.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/t/helper/test-pack-deltas.c b/t/helper/test-pack-deltas.c
index 5e0f726842..959705feca 100644
--- a/t/helper/test-pack-deltas.c
+++ b/t/helper/test-pack-deltas.c
@@ -22,7 +22,7 @@ static unsigned long do_compress(void **pptr, unsigned long size)
{
git_zstream stream;
void *in, *out;
- unsigned long maxsize;
+ size_t maxsize;
git_deflate_init(&stream, 1);
maxsize = git_deflate_bound(&stream, size);
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v2 12/12] git-zlib: widen `git_deflate_bound()` to `size_t`
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (10 preceding siblings ...)
2026-08-05 16:14 ` [PATCH v2 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local " Johannes Schindelin via GitGitGadget
@ 2026-08-05 16:14 ` Johannes Schindelin via GitGitGadget
2026-08-06 6:15 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Patrick Steinhardt
12 siblings, 0 replies; 39+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2026-08-05 16:14 UTC (permalink / raw)
To: git; +Cc: Patrick Steinhardt, Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
All four `unsigned long`/`int`/`ssize_t` receivers across archive-zip,
diff, http-push and t/helper/test-pack-deltas were widened to `size_t`
in the prior commits, and remote-curl and fast-import were already
there. With every caller prepared, both the parameter and the return
type can now move without introducing any silent narrowing.
For inputs above zlib's `uLong` range (i.e. >4 GiB on platforms where
`uLong` is 32-bit, notably 64-bit Windows), defer to zlib's stored-block
formula (the same fallback it would itself use, see
https://github.com/madler/zlib/blob/v1.3.2/deflate.c#L832-L928 keeping
in mind that for large sizes, the `storelen` would be relevant, also
compare with https://github.com/madler/zlib/issues/549 for a fuller
story) plus the worst-case wrapper overhead. The existing path through
`deflateBound()` is unchanged for inputs that fit.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
git-zlib.c | 16 ++++++++++++++--
git-zlib.h | 2 +-
2 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/git-zlib.c b/git-zlib.c
index d21adb3bf5..ebbbcc6d1a 100644
--- a/git-zlib.c
+++ b/git-zlib.c
@@ -167,9 +167,21 @@ int git_inflate(git_zstream *strm, int flush)
return status;
}
-unsigned long git_deflate_bound(git_zstream *strm, unsigned long size)
+size_t git_deflate_bound(git_zstream *strm, size_t size)
{
- return deflateBound(&strm->z, size);
+#if SIZE_MAX > ULONG_MAX
+ if (size > maximum_unsigned_value_of_type(uLong))
+ /*
+ * deflateBound() takes uLong, which is 32-bit on
+ * Windows. For inputs above that range, return zlib's
+ * stored-block formula (the conservative path it would
+ * itself use for an unknown stream state) plus the
+ * worst-case wrapper overhead.
+ */
+ return size + (size >> 5) + (size >> 7) + (size >> 11)
+ + 7 + 18;
+#endif
+ return deflateBound(&strm->z, (uLong)size);
}
void git_deflate_init(git_zstream *strm, int level)
diff --git a/git-zlib.h b/git-zlib.h
index 0b24b15bd0..9248d11ca9 100644
--- a/git-zlib.h
+++ b/git-zlib.h
@@ -25,6 +25,6 @@ void git_deflate_end(git_zstream *);
int git_deflate_abort(git_zstream *);
int git_deflate_end_gently(git_zstream *);
int git_deflate(git_zstream *, int flush);
-unsigned long git_deflate_bound(git_zstream *, unsigned long);
+size_t git_deflate_bound(git_zstream *, size_t);
#endif /* GIT_ZLIB_H */
--
gitgitgadget
^ permalink raw reply related [flat|nested] 39+ messages in thread
* Re: [PATCH v2 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t`
2026-08-05 16:14 ` [PATCH v2 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
@ 2026-08-06 6:15 ` Patrick Steinhardt
0 siblings, 0 replies; 39+ messages in thread
From: Patrick Steinhardt @ 2026-08-06 6:15 UTC (permalink / raw)
To: Johannes Schindelin via GitGitGadget; +Cc: git, Johannes Schindelin
On Wed, Aug 05, 2026 at 04:14:28PM +0000, Johannes Schindelin via GitGitGadget wrote:
> diff --git a/delta.h b/delta.h
> index eb5c6d2fdb..ab0279168c 100644
> --- a/delta.h
> +++ b/delta.h
> @@ -28,7 +28,7 @@ void free_delta_index(struct delta_index *index);
> *
> * Given pointer must be what create_delta_index() returned, or NULL.
> */
> -unsigned long sizeof_delta_index(struct delta_index *index);
> +size_t sizeof_delta_index(struct delta_index *index);
>
> /*
> * create_delta: create a delta from given index for the given buffer
Okay. At this point in time there's still at least one caller that
assigns the result of `sizeof_delta_index()` to an `unsigned long`. But
at the end of the series all callers assign to a `size_t`.
Patrick
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v2 00/12] Next size_t stop: pack-objects/delta
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
` (11 preceding siblings ...)
2026-08-05 16:14 ` [PATCH v2 12/12] git-zlib: widen `git_deflate_bound()` " Johannes Schindelin via GitGitGadget
@ 2026-08-06 6:15 ` Patrick Steinhardt
2026-08-06 17:30 ` Junio C Hamano
12 siblings, 1 reply; 39+ messages in thread
From: Patrick Steinhardt @ 2026-08-06 6:15 UTC (permalink / raw)
To: Johannes Schindelin via GitGitGadget; +Cc: git, Johannes Schindelin
On Wed, Aug 05, 2026 at 04:14:27PM +0000, Johannes Schindelin via GitGitGadget wrote:
> Changes since v1:
>
> * The return value of sizeof_delta_index() is now included in the unsigned
> long -> size_t work.
> * To assign correct values to the now-widened max_delta_cache_size, a new
> pair of helpers are introduced and used: git_parse_size_t() and
> git_config_size_t()
> * There are now two references regarding the provenance of the
> deflateBound() formula in the corresponding commit message.
This addresses all of the comments I had. Thanks!
Patrick
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v2 00/12] Next size_t stop: pack-objects/delta
2026-08-06 6:15 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Patrick Steinhardt
@ 2026-08-06 17:30 ` Junio C Hamano
0 siblings, 0 replies; 39+ messages in thread
From: Junio C Hamano @ 2026-08-06 17:30 UTC (permalink / raw)
To: Patrick Steinhardt
Cc: Johannes Schindelin via GitGitGadget, git, Johannes Schindelin
Patrick Steinhardt <ps@pks.im> writes:
> On Wed, Aug 05, 2026 at 04:14:27PM +0000, Johannes Schindelin via GitGitGadget wrote:
>> Changes since v1:
>>
>> * The return value of sizeof_delta_index() is now included in the unsigned
>> long -> size_t work.
>> * To assign correct values to the now-widened max_delta_cache_size, a new
>> pair of helpers are introduced and used: git_parse_size_t() and
>> git_config_size_t()
>> * There are now two references regarding the provenance of the
>> deflateBound() formula in the corresponding commit message.
>
> This addresses all of the comments I had. Thanks!
>
> Patrick
Thanks, both. Shall we mark the topic for 'next'?
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v2 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields to `size_t`
2026-08-05 16:14 ` [PATCH v2 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields " Johannes Schindelin via GitGitGadget
@ 2026-08-07 21:41 ` Junio C Hamano
2026-08-07 22:06 ` Junio C Hamano
1 sibling, 0 replies; 39+ messages in thread
From: Junio C Hamano @ 2026-08-07 21:41 UTC (permalink / raw)
To: Johannes Schindelin via GitGitGadget
Cc: git, Patrick Steinhardt, Johannes Schindelin
"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:
> diff --git a/git-zlib.h b/git-zlib.h
> index 44380e8ad3..0b24b15bd0 100644
> --- a/git-zlib.h
> +++ b/git-zlib.h
> @@ -5,8 +5,8 @@
>
> typedef struct git_zstream {
> struct z_stream_s z;
> - unsigned long avail_in;
> - unsigned long avail_out;
> + size_t avail_in;
> + size_t avail_out;
> size_t total_in;
> size_t total_out;
> unsigned char *next_in;
We have these size_t which means we can use a buffer larger than 4GB
where size_t is larger than 32-bit ulong. But these are sizes of a
single contiguous buffer, so I think that is why the log message
mentioned that this is more of type consistency than being able to
handle larger data (I do not think people feed >4GB contiguious
buffer in one go in practice).
- zlib_buf_cap() is still "unsigned long", and zlib_pre_call()
feeds these potentially wider values to it. Is it possible that
we trigger truncation before the avail_in/avail_out is compared
with ZLIB_BUF_MAX in the zlib_buf_cap() function?
- unpack_object_header_buffer() still takes "unsigned long" length;
builtin/pack-objects.c:oe_get_size_slow() passes size_t avail to
unpack_object_header_buffer(). This comes from use_pack(), so it
is a relatively small value stored in wider size_t but I am unsure
if your static checker would not flag for potential truncation?
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v2 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields to `size_t`
2026-08-05 16:14 ` [PATCH v2 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields " Johannes Schindelin via GitGitGadget
2026-08-07 21:41 ` Junio C Hamano
@ 2026-08-07 22:06 ` Junio C Hamano
1 sibling, 0 replies; 39+ messages in thread
From: Junio C Hamano @ 2026-08-07 22:06 UTC (permalink / raw)
To: Johannes Schindelin via GitGitGadget
Cc: git, Patrick Steinhardt, Johannes Schindelin
[jc: Sorry, I hit <SEND> before I was ready]
"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:
> diff --git a/git-zlib.h b/git-zlib.h
> index 44380e8ad3..0b24b15bd0 100644
> --- a/git-zlib.h
> +++ b/git-zlib.h
> @@ -5,8 +5,8 @@
>
> typedef struct git_zstream {
> struct z_stream_s z;
> - unsigned long avail_in;
> - unsigned long avail_out;
> + size_t avail_in;
> + size_t avail_out;
> size_t total_in;
> size_t total_out;
> unsigned char *next_in;
We use 'size_t', which means we can use a buffer larger than 4 GB
on systems where 'size_t' is wider than a 32-bit 'unsigned long'.
But these represent the size of a single contiguous buffer, so
I think that is why the log message mentioned that this is more
about type consistency than being able to handle larger data, as
I do not think anyone would reasonably feed a contiguous buffer
larger than 4 GB in one go in practice. For that reason, two
details stood out to me:
- zlib_buf_cap() still returns 'unsigned long', and
zlib_pre_call() feeds these potentially wider values to it.
Is it possible that we trigger truncation before 'avail_in'
or 'avail_out' is compared with 'ZLIB_BUF_MAX' in
zlib_buf_cap()?
- unpack_object_header_buffer() still takes an 'unsigned long'
length, while oe_get_size_slow() in 'builtin/pack-objects.c'
passes a 'size_t' 'avail' to it. This comes from use_pack(),
so it is a relatively small value stored in a wider 'size_t',
but I am unsure whether your static checker would flag this for
potential truncation.
They are probably harmless in practice, but they are still a bit
concerning from the standpoint of type consistency.
^ permalink raw reply [flat|nested] 39+ messages in thread
end of thread, other threads:[~2026-08-07 22:06 UTC | newest]
Thread overview: 39+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-09 16:49 [PATCH 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
2026-08-05 9:22 ` Patrick Steinhardt
2026-08-05 13:51 ` Johannes Schindelin
2026-07-09 16:49 ` [PATCH 02/12] delta: widen `create_delta_index()` parameter " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 03/12] pack-objects: widen delta-cache accounting " Johannes Schindelin via GitGitGadget
2026-08-05 9:23 ` Patrick Steinhardt
2026-08-05 13:52 ` Johannes Schindelin
2026-07-09 16:49 ` [PATCH 04/12] pack-objects: widen `free_unpacked()` return " Johannes Schindelin via GitGitGadget
2026-08-05 9:23 ` Patrick Steinhardt
2026-07-09 16:49 ` [PATCH 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 06/12] delta: widen `create_delta()` and `diff_delta()` " Johannes Schindelin via GitGitGadget
2026-08-05 9:23 ` Patrick Steinhardt
2026-07-09 16:49 ` [PATCH 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 09/12] diff: widen `deflate_it()`'s bound local from int " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 10/12] http-push: widen `start_put()`'s size local from `ssize_t` " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local " Johannes Schindelin via GitGitGadget
2026-07-09 16:49 ` [PATCH 12/12] git-zlib: widen `git_deflate_bound()` " Johannes Schindelin via GitGitGadget
2026-08-05 9:23 ` Patrick Steinhardt
2026-08-05 13:58 ` Johannes Schindelin
2026-08-05 16:14 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 01/12] diff-delta: widen `struct delta_index`' size fields to `size_t` Johannes Schindelin via GitGitGadget
2026-08-06 6:15 ` Patrick Steinhardt
2026-08-05 16:14 ` [PATCH v2 02/12] delta: widen `create_delta_index()` parameter " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 03/12] pack-objects: widen delta-cache accounting " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 04/12] pack-objects: widen `free_unpacked()` return " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 05/12] pack-objects: widen `mem_usage` and `try_delta()`'s out-param " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 06/12] delta: widen `create_delta()` and `diff_delta()` " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields " Johannes Schindelin via GitGitGadget
2026-08-07 21:41 ` Junio C Hamano
2026-08-07 22:06 ` Junio C Hamano
2026-08-05 16:14 ` [PATCH v2 08/12] archive-zip: widen `zlib_deflate_raw()`'s maxsize local " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 09/12] diff: widen `deflate_it()`'s bound local from int " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 10/12] http-push: widen `start_put()`'s size local from `ssize_t` " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 11/12] t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local " Johannes Schindelin via GitGitGadget
2026-08-05 16:14 ` [PATCH v2 12/12] git-zlib: widen `git_deflate_bound()` " Johannes Schindelin via GitGitGadget
2026-08-06 6:15 ` [PATCH v2 00/12] Next size_t stop: pack-objects/delta Patrick Steinhardt
2026-08-06 17:30 ` Junio C Hamano
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).