* [PATCH 1/6] object-file: check size of inflated loose object against its header
2026-10-30 10:28 [PATCH 0/6] loose object size checks and a few size truncations Elia Pinto
@ 2026-10-30 10:28 ` Elia Pinto
2026-10-30 10:28 ` [PATCH 2/6] object-file: use size_t for loose object sizes Elia Pinto
` (5 subsequent siblings)
6 siblings, 0 replies; 18+ messages in thread
From: Elia Pinto @ 2026-10-30 10:28 UTC (permalink / raw)
To: git
Cc: Johannes Schindelin, Elia Pinto, Jeff King, Junio C Hamano,
Patrick Steinhardt, Ævar Arnfjörð Bjarmason
unpack_loose_rest() allocates a buffer of the size given in the object
header and inflates the rest of the stream into it, but it never checks
that the stream actually produced that many bytes. If the compressed
data ends early we return success with the tail of the buffer never
written, and callers like odb_read_object() hand it out as object
content. For example, "git cat-file commit" on a loose object whose
header says "commit 200" but which contains only 44 bytes prints 156
bytes of uninitialized heap memory.
The opposite case slips through, too: when the whole object fits into
the buffer filled by unpack_loose_header(), we copy only "size" bytes
out of it, the stream has already ended, and the extra data is silently
dropped.
The packfile reader in unpack_compressed_entry() and index-pack already
compare total_out with the expected size. Do the same here and report
the object as corrupt if they differ. Today fsck notices the first case
only by accident, because it hashes the uninitialized bytes and gets a
mismatch.
Teach loose_obj() in lib-loose.sh to write a bogus size, so that the
tests can create such objects.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
---
object-file.c | 15 +++++++++++----
t/lib-loose.sh | 5 ++++-
t/t1060-object-corruption.sh | 28 ++++++++++++++++++++++++++++
3 files changed, 43 insertions(+), 5 deletions(-)
diff --git a/object-file.c b/object-file.c
index c049bf13f7..6d39138511 100644
--- a/object-file.c
+++ b/object-file.c
@@ -212,14 +212,14 @@ void *unpack_loose_rest(git_zstream *stream,
void *buffer, unsigned long size,
const struct object_id *oid)
{
- size_t bytes = strlen(buffer) + 1, n;
+ size_t hdrlen = strlen(buffer) + 1, bytes, n;
unsigned char *buf = xmallocz(size);
int status = Z_OK;
- n = stream->total_out - bytes;
+ n = stream->total_out - hdrlen;
if (n > size)
n = size;
- memcpy(buf, (char *) buffer + bytes, n);
+ memcpy(buf, (char *) buffer + hdrlen, n);
bytes = n;
if (bytes <= size) {
/*
@@ -244,7 +244,14 @@ void *unpack_loose_rest(git_zstream *stream,
}
}
- if (status != Z_STREAM_END) {
+ /*
+ * Make sure that the stream produced exactly as many bytes as the
+ * header claims. Otherwise we would hand back a buffer whose tail
+ * was never written (if the stream is short), or silently drop data
+ * (if the stream is long but ended before we noticed).
+ */
+ if (status != Z_STREAM_END ||
+ stream->total_out - hdrlen != size) {
error(_("corrupt loose object '%s'"), oid_to_hex(oid));
FREE_AND_NULL(buf);
} else if (stream->avail_in) {
diff --git a/t/lib-loose.sh b/t/lib-loose.sh
index 3613631eaf..807fca6f05 100644
--- a/t/lib-loose.sh
+++ b/t/lib-loose.sh
@@ -5,9 +5,12 @@
#
# oid=$(echo foo | loose_obj .git/objects blob)
#
+# If $3 is given, it is written as the size in the object header instead
+# of the real size of the contents.
+#
loose_obj () {
cat >tmp_loose.content &&
- size=$(wc -c <tmp_loose.content) &&
+ size=${3:-$(wc -c <tmp_loose.content)} &&
{
# Do not quote $size here; we want the shell
# to strip whitespace that "wc" adds on some platforms.
diff --git a/t/t1060-object-corruption.sh b/t/t1060-object-corruption.sh
index d2ef468b45..bd3d1adf5d 100755
--- a/t/t1060-object-corruption.sh
+++ b/t/t1060-object-corruption.sh
@@ -3,6 +3,7 @@
test_description='see how we handle various forms of corruption'
. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-loose.sh
# convert "1234abcd" to ".git/objects/12/34abcd"
obj_to_file() {
@@ -163,4 +164,31 @@ test_expect_success 'corrupted loose commit can be read from alternate' '
test_grep "inflate: data stream error" err
'
+test_expect_success 'setup loose objects with bogus header size' '
+ git init bogus-size &&
+ (
+ cd bogus-size &&
+ printf "short" |
+ loose_obj .git/objects commit 200 >../oid-too-big &&
+ printf "much longer" |
+ loose_obj .git/objects commit 3 >../oid-too-small
+ )
+'
+
+test_expect_success 'reading loose object shorter than its header fails' '
+ test_must_fail git -C bogus-size cat-file commit $(cat oid-too-big) 2>err &&
+ test_grep "corrupt loose object" err
+'
+
+test_expect_success 'reading loose object longer than its header fails' '
+ test_must_fail git -C bogus-size cat-file commit $(cat oid-too-small) 2>err &&
+ test_grep "corrupt loose object" err
+'
+
+test_expect_success 'fsck reports loose objects with bogus header size' '
+ test_must_fail git -C bogus-size fsck 2>err &&
+ test_grep "corrupt loose object .$(cat oid-too-big)" err &&
+ test_grep "corrupt loose object .$(cat oid-too-small)" err
+'
+
test_done
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 2/6] object-file: use size_t for loose object sizes
2026-10-30 10:28 [PATCH 0/6] loose object size checks and a few size truncations Elia Pinto
2026-10-30 10:28 ` [PATCH 1/6] object-file: check size of inflated loose object against its header Elia Pinto
@ 2026-10-30 10:28 ` Elia Pinto
2026-10-30 10:28 ` [PATCH 3/6] odb/source-loose: check object size when streaming loose objects Elia Pinto
` (4 subsequent siblings)
6 siblings, 0 replies; 18+ messages in thread
From: Elia Pinto @ 2026-10-30 10:28 UTC (permalink / raw)
To: git
Cc: Johannes Schindelin, Elia Pinto, Patrick Steinhardt, Jonathan Tan,
Junio C Hamano
Since c6a4629e32 (odb: use size_t for object_info.sizep and the size
APIs, 2026-06-15), parse_loose_header() reports the object size as a
size_t. But unpack_loose_rest() still takes an "unsigned long", which
is only 32 bits on Windows. A header claiming 4GiB + 5 bytes thus
makes us allocate a 6-byte buffer, the size check added by the previous
commit is happy because the stream really has 5 bytes, and the caller
gets that buffer back together with *oi->sizep set to 4GiB + 5. Any
user of the object then reads far beyond the end of the buffer;
read_loose_object() already does so when it hashes the contents.
Use size_t for the size in unpack_loose_rest() and check_stream_oid(),
and while at it for the size of the mapped file in map_fd(),
odb_source_loose_map_object() and unpack_loose_header(), so that the
whole loose object reading path agrees on the type.
There is no test, as the reader allocates a buffer of the size claimed
in the header before it can notice that the stream is short, so a test
would need 4GiB of memory. The problem can be reproduced on Windows
with the loose_obj() helper from the previous commit:
printf short |
loose_obj .git/objects commit $((4 * 1024 * 1024 * 1024 + 5))
followed by "git cat-file commit" on the resulting object ID.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
---
object-file.c | 14 +++++++-------
object-file.h | 6 +++---
odb/source-loose.c | 8 ++++----
3 files changed, 14 insertions(+), 14 deletions(-)
diff --git a/object-file.c b/object-file.c
index 6d39138511..0fc240b1e8 100644
--- a/object-file.c
+++ b/object-file.c
@@ -155,7 +155,7 @@ int stream_object_signature(struct repository *r,
* Map and close the given loose object fd. The path argument is used for
* error reporting.
*/
-static void *map_fd(int fd, const char *path, unsigned long *size)
+static void *map_fd(int fd, const char *path, size_t *size)
{
void *map = NULL;
struct stat st;
@@ -176,9 +176,9 @@ static void *map_fd(int fd, const char *path, unsigned long *size)
enum unpack_loose_header_result unpack_loose_header(git_zstream *stream,
unsigned char *map,
- unsigned long mapsize,
+ size_t mapsize,
void *buffer,
- unsigned long bufsiz)
+ size_t bufsiz)
{
int status;
@@ -209,7 +209,7 @@ enum unpack_loose_header_result unpack_loose_header(git_zstream *stream,
}
void *unpack_loose_rest(git_zstream *stream,
- void *buffer, unsigned long size,
+ void *buffer, size_t size,
const struct object_id *oid)
{
size_t hdrlen = strlen(buffer) + 1, bytes, n;
@@ -1135,7 +1135,7 @@ int for_each_loose_file_in_source(struct odb_source *source,
static int check_stream_oid(git_zstream *stream,
const char *hdr,
- unsigned long size,
+ size_t size,
const char *path,
const struct object_id *expected_oid,
const struct git_hash_algo *algop)
@@ -1143,7 +1143,7 @@ static int check_stream_oid(git_zstream *stream,
struct git_hash_ctx c;
struct object_id real_oid;
unsigned char buf[4096];
- unsigned long total_read;
+ size_t total_read;
int status = Z_OK;
git_hash_init(&c, algop);
@@ -1203,7 +1203,7 @@ int read_loose_object(struct repository *repo,
int ret = -1;
int fd;
void *map = NULL;
- unsigned long mapsize;
+ size_t mapsize;
git_zstream stream;
char hdr[MAX_HEADER_LEN];
size_t *size = oi->sizep;
diff --git a/object-file.h b/object-file.h
index f44758c4f8..a8da9dcd22 100644
--- a/object-file.h
+++ b/object-file.h
@@ -157,11 +157,11 @@ enum unpack_loose_header_result {
*/
enum unpack_loose_header_result unpack_loose_header(git_zstream *stream,
unsigned char *map,
- unsigned long mapsize,
+ size_t mapsize,
void *buffer,
- unsigned long bufsiz);
+ size_t bufsiz);
void *unpack_loose_rest(git_zstream *stream,
- void *buffer, unsigned long size,
+ void *buffer, size_t size,
const struct object_id *oid);
int parse_loose_header(const char *hdr, struct object_info *oi);
diff --git a/odb/source-loose.c b/odb/source-loose.c
index 50cb388c35..d77e93e7c4 100644
--- a/odb/source-loose.c
+++ b/odb/source-loose.c
@@ -73,7 +73,7 @@ static int read_object_info_from_path(struct odb_source_loose *loose,
{
int ret;
int fd;
- unsigned long mapsize;
+ size_t mapsize;
void *map = NULL;
git_zstream stream, *stream_to_end = NULL;
char hdr[MAX_HEADER_LEN];
@@ -250,7 +250,7 @@ static int open_loose_object(struct odb_source_loose *loose,
static void *odb_source_loose_map_object(struct odb_source_loose *loose,
const struct object_id *oid,
- unsigned long *size)
+ size_t *size)
{
const char *p;
int fd = open_loose_object(loose, oid, &p);
@@ -285,7 +285,7 @@ struct odb_loose_read_stream {
ODB_LOOSE_READ_STREAM_ERROR,
} z_state;
void *mapped;
- unsigned long mapsize;
+ size_t mapsize;
char hdr[32];
int hdr_avail;
int hdr_used;
@@ -356,7 +356,7 @@ static int odb_source_loose_read_object_stream(struct odb_stream **out,
struct odb_source_loose *loose = odb_source_loose_downcast(source);
struct object_info oi = OBJECT_INFO_INIT;
struct odb_loose_read_stream *st;
- unsigned long mapsize;
+ size_t mapsize;
void *mapped;
mapped = odb_source_loose_map_object(loose, oid, &mapsize);
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 3/6] odb/source-loose: check object size when streaming loose objects
2026-10-30 10:28 [PATCH 0/6] loose object size checks and a few size truncations Elia Pinto
2026-10-30 10:28 ` [PATCH 1/6] object-file: check size of inflated loose object against its header Elia Pinto
2026-10-30 10:28 ` [PATCH 2/6] object-file: use size_t for loose object sizes Elia Pinto
@ 2026-10-30 10:28 ` Elia Pinto
2026-10-30 10:28 ` [PATCH 4/6] cat-file: use size_t for the length passed to batch_write() Elia Pinto
` (3 subsequent siblings)
6 siblings, 0 replies; 18+ messages in thread
From: Elia Pinto @ 2026-10-30 10:28 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Elia Pinto, Junio C Hamano,
Patrick Steinhardt
read_istream_loose() inflates a loose object until zlib reports the end
of the stream, but never compares the number of bytes it produced with
the size given in the object header. A corrupt object whose stream is
shorter than its header claims is thus streamed out without any error,
e.g. "git cat-file blob" exits with 0 after writing fewer bytes than
"git cat-file -s" reports. The same happens if the stream is longer than
the header claims.
Keep track of how many bytes we handed out and report the object as
corrupt if that does not match the size from the header once the stream
ends.
The tests create such objects with the loose_obj() helper, which can
already write a bogus size into the header.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
---
odb/source-loose.c | 14 ++++++++++++++
t/t1006-cat-file.sh | 18 ++++++++++++++++++
2 files changed, 32 insertions(+)
diff --git a/odb/source-loose.c b/odb/source-loose.c
index d77e93e7c4..2233714ac4 100644
--- a/odb/source-loose.c
+++ b/odb/source-loose.c
@@ -289,6 +289,8 @@ struct odb_loose_read_stream {
char hdr[32];
int hdr_avail;
int hdr_used;
+ size_t total_read;
+ struct object_id oid;
};
static ssize_t read_istream_loose(struct odb_stream *_st, char *buf, size_t sz)
@@ -326,6 +328,16 @@ static ssize_t read_istream_loose(struct odb_stream *_st, char *buf, size_t sz)
if (status == Z_STREAM_END) {
git_inflate_end(&st->z);
+ /*
+ * The object must be exactly as large as its
+ * header claims.
+ */
+ if (st->total_read + total_read != st->base.size) {
+ error(_("corrupt loose object '%s'"),
+ oid_to_hex(&st->oid));
+ st->z_state = ODB_LOOSE_READ_STREAM_ERROR;
+ return -1;
+ }
st->z_state = ODB_LOOSE_READ_STREAM_DONE;
break;
}
@@ -335,6 +347,7 @@ static ssize_t read_istream_loose(struct odb_stream *_st, char *buf, size_t sz)
return -1;
}
}
+ st->total_read += total_read;
return total_read;
}
@@ -388,6 +401,7 @@ static int odb_source_loose_read_object_stream(struct odb_stream **out,
st->mapped = mapped;
st->mapsize = mapsize;
+ oidcpy(&st->oid, oid);
st->hdr_used = strlen(st->hdr) + 1;
st->hdr_avail = st->z.total_out;
st->z_state = ODB_LOOSE_READ_STREAM_INUSE;
diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh
index 72fc8acc61..31a76c1b5d 100755
--- a/t/t1006-cat-file.sh
+++ b/t/t1006-cat-file.sh
@@ -1481,4 +1481,22 @@ test_objects_filter "object:type=commit"
test_objects_filter "object:type=tag"
test_objects_filter "object:type=tree"
+test_expect_success 'setup blobs with bogus size in header' '
+ git init bogus-size &&
+ echo_without_newline short |
+ loose_obj bogus-size/.git/objects blob 100 >oid-too-big &&
+ echo_without_newline "much longer" |
+ loose_obj bogus-size/.git/objects blob 3 >oid-too-small
+'
+
+test_expect_success 'streaming blob shorter than its header fails' '
+ test_must_fail git -C bogus-size cat-file blob $(cat oid-too-big) >out 2>err &&
+ test_grep "corrupt loose object" err
+'
+
+test_expect_success 'streaming blob longer than its header fails' '
+ test_must_fail git -C bogus-size cat-file blob $(cat oid-too-small) >out 2>err &&
+ test_grep "corrupt loose object" err
+'
+
test_done
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 4/6] cat-file: use size_t for the length passed to batch_write()
2026-10-30 10:28 [PATCH 0/6] loose object size checks and a few size truncations Elia Pinto
` (2 preceding siblings ...)
2026-10-30 10:28 ` [PATCH 3/6] odb/source-loose: check object size when streaming loose objects Elia Pinto
@ 2026-10-30 10:28 ` Elia Pinto
2026-10-30 10:28 ` [PATCH 5/6] notes: avoid truncating the size of concatenated notes Elia Pinto
` (2 subsequent siblings)
6 siblings, 0 replies; 18+ messages in thread
From: Elia Pinto @ 2026-10-30 10:28 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Elia Pinto, Junio C Hamano,
René Scharfe
batch_write() takes the length of the data as an "int", but its callers
pass the size of whole objects as returned by odb_read_object() or
filter_object(), which is a size_t. For an object between 2GiB and
4GiB the length wraps to a negative value, which fwrite() and
write_or_die() then see as an enormous size_t. We happily write the
object and then keep going past the end of the buffer, sending whatever
memory follows to stdout, until we hit an unmapped page and die with
"write error: Bad address". Objects of 4GiB and more are truncated
instead, which desynchronizes the --batch output.
This affects all platforms, not only those with a 32-bit "long". It can
be triggered with "--batch" on any non-blob object, or on blobs with
--filters or --textconv, as blobs are otherwise streamed. For example:
blob=$(test-tool genzeros $((3 * 1024 * 1024 * 1024)) |
git hash-object -w --stdin) &&
echo "$blob big" |
git cat-file --batch="%(objectname) %(rest)" --filters >out
fails with "write error: Bad address"; with stdout going to /dev/null,
strace shows write() being called on addresses far beyond the end of the
3GiB buffer. There is no test, as it would need several GiB of memory
and disk.
Both fwrite() and write_or_die() take a size_t already, so just change
the parameter type.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
---
builtin/cat-file.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index 8870a210ec..c846fe3116 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -425,7 +425,7 @@ static void expand_format(struct strbuf *sb, const char *start,
}
}
-static void batch_write(struct batch_options *opt, const void *data, int len)
+static void batch_write(struct batch_options *opt, const void *data, size_t len)
{
if (opt->buffer_output) {
if (fwrite(data, 1, len, stdout) != len)
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 5/6] notes: avoid truncating the size of concatenated notes
2026-10-30 10:28 [PATCH 0/6] loose object size checks and a few size truncations Elia Pinto
` (3 preceding siblings ...)
2026-10-30 10:28 ` [PATCH 4/6] cat-file: use size_t for the length passed to batch_write() Elia Pinto
@ 2026-10-30 10:28 ` Elia Pinto
2026-10-30 10:28 ` [PATCH 6/6] diff: use size_t in count_trailing_blank() Elia Pinto
2026-11-17 11:38 ` [PATCH v2 0/5] loose object size checks and a few size truncations Elia Pinto
6 siblings, 0 replies; 18+ messages in thread
From: Elia Pinto @ 2026-10-30 10:28 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Elia Pinto, Patrick Steinhardt,
Junio C Hamano
combine_notes_concatenate() reads both notes with odb_read_object(),
which returns their sizes as size_t, but computes the size of the
combined buffer in an "unsigned long". On platforms where that type is
32 bits wide, like Windows, two notes adding up to 4GiB or more give a
small buf_len: we allocate that many bytes and then memcpy() the full
contents of both notes into the buffer, writing far past its end.
The notes can come from a fetched notes ref. Besides notes merges with
the "union" strategy and notes.rewriteMode=concatenate (the default),
the function is also used when loading a notes tree that contains more
than one note for the same object, e.g. at different fanout levels, so
merely displaying such notes is enough to trigger it.
Compute the size as a size_t with st_add3(), and go through
cast_size_t_to_ulong() when passing it to odb_write_object(), whose
length parameter is still an "unsigned long". That makes us die on such
platforms instead of writing a truncated blob, until the write API
learns about size_t, too.
No test, as triggering this needs a platform with a 32-bit "long" and
notes larger than 4GiB.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
---
notes.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/notes.c b/notes.c
index ec9c2cb150..05da28b8d4 100644
--- a/notes.c
+++ b/notes.c
@@ -811,8 +811,7 @@ int combine_notes_concatenate(struct object_id *cur_oid,
const struct object_id *new_oid)
{
char *cur_msg = NULL, *new_msg = NULL, *buf;
- unsigned long buf_len;
- size_t cur_len, new_len;
+ size_t cur_len, new_len, buf_len;
enum object_type cur_type, new_type;
int ret;
@@ -839,7 +838,7 @@ int combine_notes_concatenate(struct object_id *cur_oid,
cur_len--;
/* concatenate cur_msg and new_msg into buf */
- buf_len = cur_len + 2 + new_len;
+ buf_len = st_add3(cur_len, 2, new_len);
buf = (char *) xmalloc(buf_len);
memcpy(buf, cur_msg, cur_len);
buf[cur_len] = '\n';
@@ -850,7 +849,7 @@ int combine_notes_concatenate(struct object_id *cur_oid,
/* create a new blob object from buf */
ret = odb_write_object(the_repository->objects, buf,
- buf_len, OBJ_BLOB, cur_oid);
+ cast_size_t_to_ulong(buf_len), OBJ_BLOB, cur_oid);
free(buf);
return ret;
}
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 6/6] diff: use size_t in count_trailing_blank()
2026-10-30 10:28 [PATCH 0/6] loose object size checks and a few size truncations Elia Pinto
` (4 preceding siblings ...)
2026-10-30 10:28 ` [PATCH 5/6] notes: avoid truncating the size of concatenated notes Elia Pinto
@ 2026-10-30 10:28 ` Elia Pinto
2026-11-17 11:38 ` [PATCH v2 0/5] loose object size checks and a few size truncations Elia Pinto
6 siblings, 0 replies; 18+ messages in thread
From: Elia Pinto @ 2026-10-30 10:28 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Elia Pinto, Junio C Hamano, Jeff King
Since 3c7a31204f (xdiff: use size_t for buffer sizes, 2026-09-30),
mmfile_t stores its size as a size_t, but count_trailing_blank() still
copies it into a "long". On platforms where "long" is 32 bits wide,
like Windows, a buffer between 2GiB and 4GiB gives a negative size, and
"ptr += size - 1" points about 2GiB before the start of the buffer,
which we then dereference.
check_blank_at_eof() calls this function whenever the whitespace rule
includes blank-at-eof, which is the default, and it does so before
xdi_diff() rejects inputs larger than MAX_XDIFF_SIZE. Large files are
normally treated as binary and never get here, but a "diff" attribute
or a textconv driver makes us diff them as text. A repository can ship
such an attribute in its .gitattributes, so "git show" or "git log -p"
on a commit adding a 3GiB file can be made to crash.
Use size_t; the rest of the function works unchanged with it.
No test, as triggering this needs a platform with a 32-bit "long" and
a blob larger than 2GiB.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
---
diff.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/diff.c b/diff.c
index b4ac17f8ef..e42e53d43e 100644
--- a/diff.c
+++ b/diff.c
@@ -711,7 +711,7 @@ static unsigned long diff_filespec_size(struct repository *r,
static int count_trailing_blank(mmfile_t *mf)
{
char *ptr = mf->ptr;
- long size = mf->size;
+ size_t size = mf->size;
int cnt = 0;
if (!size)
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH v2 0/5] loose object size checks and a few size truncations
2026-10-30 10:28 [PATCH 0/6] loose object size checks and a few size truncations Elia Pinto
` (5 preceding siblings ...)
2026-10-30 10:28 ` [PATCH 6/6] diff: use size_t in count_trailing_blank() Elia Pinto
@ 2026-11-17 11:38 ` Elia Pinto
2026-11-17 11:38 ` [PATCH v2 1/5] object-file: check size of inflated loose object against its header Elia Pinto
` (4 more replies)
6 siblings, 5 replies; 18+ messages in thread
From: Elia Pinto @ 2026-11-17 11:38 UTC (permalink / raw)
To: git
Cc: Johannes Schindelin, Elia Pinto, Junio C Hamano, Toon Claes,
Patrick Steinhardt, Jeff King, René Scharfe, Jonathan Tan,
Ævar Arnfjörð Bjarmason
Changes since v1:
- v1's 6/6 ("diff: use size_t in count_trailing_blank()") is dropped.
The same line is now changed by 2/4 of v2 of my series "widen a
few sizes and counts that are 32 bits everywhere" [1], which also
widens the counter of count_trailing_blank().
- Rebased onto master at e1ec09b23c (The 22nd batch), as v1 no longer
applied: f495b0e7b9 (odb/source-files: reload alternates on retry,
2026-10-28) also adds a test at the end of t1006, where 3/5 adds its
tests. The code of the five patches is unchanged.
- The messages of 2/5 and 5/5 now say that on 64-bit Windows "long"
has 32 bits while size_t has 64, which is what truncates the sizes.
There were no comments on v1.
These patches come from the same search for truncated sizes on LLP64
(64-bit Windows, where "long" is 32 bits but size_t is 64 bits) as the
"use size_t for a few more object and buffer sizes" series [2]. Its
cover letter explains the method (a MinGW cross build and clang-tidy
with -Wshorten-64-to-32). For these I followed each value back to where
it comes from and forward to where it is used, because they can lead to
out-of-bounds memory accesses. For that reason I sent all of them but
3/5 to the git-security list first; they do not consider them security
issues, so here they are. While checking the loose object reader I also
found two missing size checks that have nothing to do with LLP64 (1/5
and 3/5).
1/5 A loose object whose zlib stream is shorter than the size in its
header is accepted, and the uninitialized tail of the buffer is
handed out as object content (all platforms; e.g. "git cat-file
commit" prints heap memory). A longer stream is silently cut.
2/5 On Windows the size from the loose object header is truncated, so
the caller gets a buffer much smaller than the size it is told,
and reads beyond its end.
3/5 The same check as 1/5 for the streaming reader of loose objects,
which today streams out a short or long object without an error.
4/5 "git cat-file --batch" writes memory past the end of objects
between 2GiB and 4GiB (all platforms; non-blob objects, or blobs
with --filters/--textconv).
5/5 On Windows, concatenating notes that add up to 4GiB or more
overflows a heap buffer.
1/5 and 3/5 come with tests, which are cheap, and fail without the
fix. The others would need objects of several GiB (and 2/5 and 5/5 a
32-bit "long"), so their commit messages describe how to reproduce
them instead. I have not run anything on Windows: the Windows-only
issues are based on reading the code and on the cross-compiled build.
5/5 passes the combined size to odb_write_object() through
cast_size_t_to_ulong(), as the length parameter is still an "unsigned
long" on master. With 1/4 of [2], which makes it a size_t, the cast
becomes unnecessary but harmless; the two series do not conflict.
The series is based on master at e1ec09b23c (The 22nd batch).
[1] https://lore.kernel.org/git/cover.1794909894.git.gitter.spiros@gmail.com/
[2] https://lore.kernel.org/git/cover.1793439137.git.gitter.spiros@gmail.com/
Elia Pinto (5):
object-file: check size of inflated loose object against its header
object-file: use size_t for loose object sizes
odb/source-loose: check object size when streaming loose objects
cat-file: use size_t for the length passed to batch_write()
notes: avoid truncating the size of concatenated notes
builtin/cat-file.c | 2 +-
notes.c | 7 +++----
object-file.c | 29 ++++++++++++++++++-----------
object-file.h | 6 +++---
odb/source-loose.c | 22 ++++++++++++++++++----
t/lib-loose.sh | 5 ++++-
t/t1006-cat-file.sh | 18 ++++++++++++++++++
t/t1060-object-corruption.sh | 28 ++++++++++++++++++++++++++++
8 files changed, 93 insertions(+), 24 deletions(-)
Range-diff against v1:
1: 6c2efb87fb = 1: edb5f34498 object-file: check size of inflated loose object against its header
2: ee4439354a ! 2: eca5d3ddb8 object-file: use size_t for loose object sizes
@@ Commit message
Since c6a4629e32 (odb: use size_t for object_info.sizep and the size
APIs, 2026-06-15), parse_loose_header() reports the object size as a
- size_t. But unpack_loose_rest() still takes an "unsigned long", which
- is only 32 bits on Windows. A header claiming 4GiB + 5 bytes thus
- makes us allocate a 6-byte buffer, the size check added by the previous
- commit is happy because the stream really has 5 bytes, and the caller
- gets that buffer back together with *oi->sizep set to 4GiB + 5. Any
- user of the object then reads far beyond the end of the buffer;
- read_loose_object() already does so when it hashes the contents.
+ size_t, but unpack_loose_rest() still takes it as an "unsigned long".
+ On 64-bit Windows, where "long" has 32 bits while size_t has 64, the
+ size is truncated on the way. A header claiming 4GiB + 5 bytes thus
+ makes us allocate a 6-byte buffer, the size check added by the
+ previous commit is happy because the stream really has 5 bytes, and
+ the caller gets that buffer back together with *oi->sizep set to
+ 4GiB + 5. Any user of the object then reads far beyond the end of the
+ buffer; read_loose_object() already does so when it hashes the
+ contents.
Use size_t for the size in unpack_loose_rest() and check_stream_oid(),
and while at it for the size of the mapped file in map_fd(),
3: dee492c3d8 ! 3: a8572aa836 odb/source-loose: check object size when streaming loose objects
@@ odb/source-loose.c: static int odb_source_loose_read_object_stream(struct odb_st
st->z_state = ODB_LOOSE_READ_STREAM_INUSE;
## t/t1006-cat-file.sh ##
-@@ t/t1006-cat-file.sh: test_objects_filter "object:type=commit"
- test_objects_filter "object:type=tag"
- test_objects_filter "object:type=tree"
+@@ t/t1006-cat-file.sh: test_expect_success PIPE 'concurrently written alternates are loaded' '
+ )
+ '
+test_expect_success 'setup blobs with bogus size in header' '
+ git init bogus-size &&
4: b2083632a1 = 4: 5b61a1b089 cat-file: use size_t for the length passed to batch_write()
5: 21f5a4e009 ! 5: 04f320d1de notes: avoid truncating the size of concatenated notes
@@ Commit message
combine_notes_concatenate() reads both notes with odb_read_object(),
which returns their sizes as size_t, but computes the size of the
- combined buffer in an "unsigned long". On platforms where that type is
- 32 bits wide, like Windows, two notes adding up to 4GiB or more give a
- small buf_len: we allocate that many bytes and then memcpy() the full
- contents of both notes into the buffer, writing far past its end.
+ combined buffer in an "unsigned long". On 64-bit Windows, where "long"
+ has 32 bits while size_t has 64, two notes adding up to 4GiB or more
+ give a small buf_len: we allocate that many bytes and then memcpy()
+ the full contents of both notes into the buffer, writing far past its
+ end.
The notes can come from a fetched notes ref. Besides notes merges with
the "union" strategy and notes.rewriteMode=concatenate (the default),
6: 79f1781abd < -: ---------- diff: use size_t in count_trailing_blank()
base-commit: e1ec09b23c8e0deb21a0b95ac5996422277395d4
--
2.53.0
^ permalink raw reply [flat|nested] 18+ messages in thread* [PATCH v2 1/5] object-file: check size of inflated loose object against its header
2026-11-17 11:38 ` [PATCH v2 0/5] loose object size checks and a few size truncations Elia Pinto
@ 2026-11-17 11:38 ` Elia Pinto
2026-11-17 19:33 ` Junio C Hamano
2026-11-17 11:38 ` [PATCH v2 2/5] object-file: use size_t for loose object sizes Elia Pinto
` (3 subsequent siblings)
4 siblings, 1 reply; 18+ messages in thread
From: Elia Pinto @ 2026-11-17 11:38 UTC (permalink / raw)
To: git
Cc: Johannes Schindelin, Elia Pinto,
Ævar Arnfjörð Bjarmason, Patrick Steinhardt,
Jeff King, Junio C Hamano
unpack_loose_rest() allocates a buffer of the size given in the object
header and inflates the rest of the stream into it, but it never checks
that the stream actually produced that many bytes. If the compressed
data ends early we return success with the tail of the buffer never
written, and callers like odb_read_object() hand it out as object
content. For example, "git cat-file commit" on a loose object whose
header says "commit 200" but which contains only 44 bytes prints 156
bytes of uninitialized heap memory.
The opposite case slips through, too: when the whole object fits into
the buffer filled by unpack_loose_header(), we copy only "size" bytes
out of it, the stream has already ended, and the extra data is silently
dropped.
The packfile reader in unpack_compressed_entry() and index-pack already
compare total_out with the expected size. Do the same here and report
the object as corrupt if they differ. Today fsck notices the first case
only by accident, because it hashes the uninitialized bytes and gets a
mismatch.
Teach loose_obj() in lib-loose.sh to write a bogus size, so that the
tests can create such objects.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
---
object-file.c | 15 +++++++++++----
t/lib-loose.sh | 5 ++++-
t/t1060-object-corruption.sh | 28 ++++++++++++++++++++++++++++
3 files changed, 43 insertions(+), 5 deletions(-)
diff --git a/object-file.c b/object-file.c
index e04ce0bc11..5d03ae94c7 100644
--- a/object-file.c
+++ b/object-file.c
@@ -212,14 +212,14 @@ void *unpack_loose_rest(git_zstream *stream,
void *buffer, unsigned long size,
const struct object_id *oid)
{
- size_t bytes = strlen(buffer) + 1, n;
+ size_t hdrlen = strlen(buffer) + 1, bytes, n;
unsigned char *buf = xmallocz(size);
int status = Z_OK;
- n = stream->total_out - bytes;
+ n = stream->total_out - hdrlen;
if (n > size)
n = size;
- memcpy(buf, (char *) buffer + bytes, n);
+ memcpy(buf, (char *) buffer + hdrlen, n);
bytes = n;
if (bytes <= size) {
/*
@@ -244,7 +244,14 @@ void *unpack_loose_rest(git_zstream *stream,
}
}
- if (status != Z_STREAM_END) {
+ /*
+ * Make sure that the stream produced exactly as many bytes as the
+ * header claims. Otherwise we would hand back a buffer whose tail
+ * was never written (if the stream is short), or silently drop data
+ * (if the stream is long but ended before we noticed).
+ */
+ if (status != Z_STREAM_END ||
+ stream->total_out - hdrlen != size) {
error(_("corrupt loose object '%s'"), oid_to_hex(oid));
FREE_AND_NULL(buf);
} else if (stream->avail_in) {
diff --git a/t/lib-loose.sh b/t/lib-loose.sh
index 3613631eaf..807fca6f05 100644
--- a/t/lib-loose.sh
+++ b/t/lib-loose.sh
@@ -5,9 +5,12 @@
#
# oid=$(echo foo | loose_obj .git/objects blob)
#
+# If $3 is given, it is written as the size in the object header instead
+# of the real size of the contents.
+#
loose_obj () {
cat >tmp_loose.content &&
- size=$(wc -c <tmp_loose.content) &&
+ size=${3:-$(wc -c <tmp_loose.content)} &&
{
# Do not quote $size here; we want the shell
# to strip whitespace that "wc" adds on some platforms.
diff --git a/t/t1060-object-corruption.sh b/t/t1060-object-corruption.sh
index d2ef468b45..bd3d1adf5d 100755
--- a/t/t1060-object-corruption.sh
+++ b/t/t1060-object-corruption.sh
@@ -3,6 +3,7 @@
test_description='see how we handle various forms of corruption'
. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-loose.sh
# convert "1234abcd" to ".git/objects/12/34abcd"
obj_to_file() {
@@ -163,4 +164,31 @@ test_expect_success 'corrupted loose commit can be read from alternate' '
test_grep "inflate: data stream error" err
'
+test_expect_success 'setup loose objects with bogus header size' '
+ git init bogus-size &&
+ (
+ cd bogus-size &&
+ printf "short" |
+ loose_obj .git/objects commit 200 >../oid-too-big &&
+ printf "much longer" |
+ loose_obj .git/objects commit 3 >../oid-too-small
+ )
+'
+
+test_expect_success 'reading loose object shorter than its header fails' '
+ test_must_fail git -C bogus-size cat-file commit $(cat oid-too-big) 2>err &&
+ test_grep "corrupt loose object" err
+'
+
+test_expect_success 'reading loose object longer than its header fails' '
+ test_must_fail git -C bogus-size cat-file commit $(cat oid-too-small) 2>err &&
+ test_grep "corrupt loose object" err
+'
+
+test_expect_success 'fsck reports loose objects with bogus header size' '
+ test_must_fail git -C bogus-size fsck 2>err &&
+ test_grep "corrupt loose object .$(cat oid-too-big)" err &&
+ test_grep "corrupt loose object .$(cat oid-too-small)" err
+'
+
test_done
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH v2 1/5] object-file: check size of inflated loose object against its header
2026-11-17 11:38 ` [PATCH v2 1/5] object-file: check size of inflated loose object against its header Elia Pinto
@ 2026-11-17 19:33 ` Junio C Hamano
0 siblings, 0 replies; 18+ messages in thread
From: Junio C Hamano @ 2026-11-17 19:33 UTC (permalink / raw)
To: Elia Pinto
Cc: git, Johannes Schindelin, Ævar Arnfjörð Bjarmason,
Patrick Steinhardt, Jeff King
Elia Pinto <gitter.spiros@gmail.com> writes:
> unpack_loose_rest() allocates a buffer of the size given in the object
> header and inflates the rest of the stream into it, but it never checks
> that the stream actually produced that many bytes. If the compressed
> data ends early we return success with the tail of the buffer never
> written, and callers like odb_read_object() hand it out as object
> content. For example, "git cat-file commit" on a loose object whose
> header says "commit 200" but which contains only 44 bytes prints 156
> bytes of uninitialized heap memory.
Interesting.
We only inflate to read up to the first MAX_HEADER_LEN bytes, and
that is sufficient to read "<type> <len>\0". unpack_loose_rest()
first prepares a buffer that is <len> bytes big, copies the
remainder of the header buffer to it, and then goes into a loop to
fill the remainder of the buffer with the remainder of the zlib
stream inflated. The condition to terminate the loop is for
git_inflate() to return something other than Z_OK, and we are
protected from the case where the deflated payload inflates beyond
the prepared buffer (i.e., the malicious or broken object file claims
it is smaller than the inflated data, attempting to cause a buffer
overrun) because we would stop the loop with Z_BUF_ERROR, which
signals that we prepared a buffer that is too small to receive the
payload. But git_inflate() may return Z_OK without filling all of
the buffer we prepared (i.e., the malicious or broken object file
claims with <len> that it is larger than it actually is), and we did
not check for this condition.
> The opposite case slips through, too: when the whole object fits into
> the buffer filled by unpack_loose_header(), we copy only "size" bytes
> out of it, the stream has already ended, and the extra data is silently
> dropped.
In other words, when the object file is inflated into a buffer that
is MAX_HEADER_LEN bytes long, "<type> <len>\0" may be 10 bytes long;
MAX_HEADER_LEN is 32 bytes, so there can be up to 22 bytes of
payload already inflated in the same header buffer. If "<len>" says
that the payload is only 10 bytes, even though it actually inflates
to 15 bytes, we happily copy the 10 bytes to the final buffer,
ignoring the remaining 5 bytes without even noticing.
> The packfile reader in unpack_compressed_entry() and index-pack already
> compare total_out with the expected size. Do the same here and report
> the object as corrupt if they differ. Today fsck notices the first case
> only by accident, because it hashes the uninitialized bytes and gets a
> mismatch.
;-)
> Teach loose_obj() in lib-loose.sh to write a bogus size, so that the
> tests can create such objects.
"lib-loose.sh" -> "t/lib-loose.sh" for clarity.
Nicely spotted.
> diff --git a/object-file.c b/object-file.c
> index e04ce0bc11..5d03ae94c7 100644
> --- a/object-file.c
> +++ b/object-file.c
> @@ -212,14 +212,14 @@ void *unpack_loose_rest(git_zstream *stream,
> void *buffer, unsigned long size,
> const struct object_id *oid)
> {
> - size_t bytes = strlen(buffer) + 1, n;
> + size_t hdrlen = strlen(buffer) + 1, bytes, n;
Earlier, "bytes" here was initialized so that buffer[bytes] points
at the first byte beyond the terminating NUL in "<type> <len>\0".
The parse_loose_header() function, which our caller has called to
find "size", guarantees that the terminating NUL exists, so it is
safe to call strlen().
Now we have a dedicated variable hdrlen that gives the length of the
header, including the NUL at its end.
> unsigned char *buf = xmallocz(size);
We allocate <len> bytes (in size) for buf[] for the payload.
> int status = Z_OK;
>
> - n = stream->total_out - bytes;
> + n = stream->total_out - hdrlen;
> if (n > size)
> n = size;
The arithmetic is the same here as before. n gives us the number of
bytes from the real payload we already have when we inflated the
header. If that is larger than size (i.e., <len>, which the object
header claimed to be the size of the payload), we limit n to that
size.
> - memcpy(buf, (char *) buffer + bytes, n);
> + memcpy(buf, (char *) buffer + hdrlen, n);
And we copy the first part of the payload data out of the tail of
the header buffer to the beginning of buf[].
> bytes = n;
Here, as before, bytes keeps track of the number of bytes of the
real payload added to buf[].
> if (bytes <= size) {
> /*
> @@ -244,7 +244,14 @@ void *unpack_loose_rest(git_zstream *stream,
> }
> }
Before this point, we have an "if" block that reads more data from
the zlib stream after the initial part read by unpack_loose_header()
into buf[] until it fills up or until the zlib stream finishes,
whichever comes first.
Thanks to the condition of the "if" block covering the bytes==size
case, we would have made sure we called git_inflate(Z_FINISH) even
when we had all the payload bytes inflated into buffer[] by the
caller's call to unpack_loose_header(). So we can rely on status
being what we got from git_inflate(). And we are not going to bust
the end of buf[] because stream->avail_out keeps track of the
remaining space in buf[].
> - if (status != Z_STREAM_END) {
> + /*
> + * Make sure that the stream produced exactly as many bytes as the
> + * header claims. Otherwise we would hand back a buffer whose tail
> + * was never written (if the stream is short), or silently drop data
> + * (if the stream is long but ended before we noticed).
> + */
> + if (status != Z_STREAM_END ||
> + stream->total_out - hdrlen != size) {
So a successful inflation of the payload should give us Z_STREAM_END
and total_out being the same as the sum of the payload <len> (aka
size) and the header length (aka hdrlen). I might have written the
latter as
stream->total_out != hdrlen + size
but that does not go beyond personal preference, I guess. What I am
saying is that I see this patch is doing exactly the right thing.
Looking good. Thanks.
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v2 2/5] object-file: use size_t for loose object sizes
2026-11-17 11:38 ` [PATCH v2 0/5] loose object size checks and a few size truncations Elia Pinto
2026-11-17 11:38 ` [PATCH v2 1/5] object-file: check size of inflated loose object against its header Elia Pinto
@ 2026-11-17 11:38 ` Elia Pinto
2026-11-17 22:34 ` Junio C Hamano
2026-11-17 11:38 ` [PATCH v2 3/5] odb/source-loose: check object size when streaming loose objects Elia Pinto
` (2 subsequent siblings)
4 siblings, 1 reply; 18+ messages in thread
From: Elia Pinto @ 2026-11-17 11:38 UTC (permalink / raw)
To: git
Cc: Johannes Schindelin, Elia Pinto, Junio C Hamano,
Patrick Steinhardt, Jonathan Tan
Since c6a4629e32 (odb: use size_t for object_info.sizep and the size
APIs, 2026-06-15), parse_loose_header() reports the object size as a
size_t, but unpack_loose_rest() still takes it as an "unsigned long".
On 64-bit Windows, where "long" has 32 bits while size_t has 64, the
size is truncated on the way. A header claiming 4GiB + 5 bytes thus
makes us allocate a 6-byte buffer, the size check added by the
previous commit is happy because the stream really has 5 bytes, and
the caller gets that buffer back together with *oi->sizep set to
4GiB + 5. Any user of the object then reads far beyond the end of the
buffer; read_loose_object() already does so when it hashes the
contents.
Use size_t for the size in unpack_loose_rest() and check_stream_oid(),
and while at it for the size of the mapped file in map_fd(),
odb_source_loose_map_object() and unpack_loose_header(), so that the
whole loose object reading path agrees on the type.
There is no test, as the reader allocates a buffer of the size claimed
in the header before it can notice that the stream is short, so a test
would need 4GiB of memory. The problem can be reproduced on Windows
with the loose_obj() helper from the previous commit:
printf short |
loose_obj .git/objects commit $((4 * 1024 * 1024 * 1024 + 5))
followed by "git cat-file commit" on the resulting object ID.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
---
object-file.c | 14 +++++++-------
object-file.h | 6 +++---
odb/source-loose.c | 8 ++++----
3 files changed, 14 insertions(+), 14 deletions(-)
diff --git a/object-file.c b/object-file.c
index 5d03ae94c7..779167f789 100644
--- a/object-file.c
+++ b/object-file.c
@@ -155,7 +155,7 @@ int stream_object_signature(struct repository *r,
* Map and close the given loose object fd. The path argument is used for
* error reporting.
*/
-static void *map_fd(int fd, const char *path, unsigned long *size)
+static void *map_fd(int fd, const char *path, size_t *size)
{
void *map = NULL;
struct stat st;
@@ -176,9 +176,9 @@ static void *map_fd(int fd, const char *path, unsigned long *size)
enum unpack_loose_header_result unpack_loose_header(git_zstream *stream,
unsigned char *map,
- unsigned long mapsize,
+ size_t mapsize,
void *buffer,
- unsigned long bufsiz)
+ size_t bufsiz)
{
int status;
@@ -209,7 +209,7 @@ enum unpack_loose_header_result unpack_loose_header(git_zstream *stream,
}
void *unpack_loose_rest(git_zstream *stream,
- void *buffer, unsigned long size,
+ void *buffer, size_t size,
const struct object_id *oid)
{
size_t hdrlen = strlen(buffer) + 1, bytes, n;
@@ -1136,7 +1136,7 @@ int for_each_loose_file_in_source(struct odb_source *source,
static int check_stream_oid(git_zstream *stream,
const char *hdr,
- unsigned long size,
+ size_t size,
const char *path,
const struct object_id *expected_oid,
const struct git_hash_algo *algop)
@@ -1144,7 +1144,7 @@ static int check_stream_oid(git_zstream *stream,
struct git_hash_ctx c;
struct object_id real_oid;
unsigned char buf[4096];
- unsigned long total_read;
+ size_t total_read;
int status = Z_OK;
git_hash_init(&c, algop);
@@ -1204,7 +1204,7 @@ int read_loose_object(struct repository *repo,
int ret = -1;
int fd;
void *map = NULL;
- unsigned long mapsize;
+ size_t mapsize;
git_zstream stream;
char hdr[MAX_HEADER_LEN];
size_t *size = oi->sizep;
diff --git a/object-file.h b/object-file.h
index f44758c4f8..a8da9dcd22 100644
--- a/object-file.h
+++ b/object-file.h
@@ -157,11 +157,11 @@ enum unpack_loose_header_result {
*/
enum unpack_loose_header_result unpack_loose_header(git_zstream *stream,
unsigned char *map,
- unsigned long mapsize,
+ size_t mapsize,
void *buffer,
- unsigned long bufsiz);
+ size_t bufsiz);
void *unpack_loose_rest(git_zstream *stream,
- void *buffer, unsigned long size,
+ void *buffer, size_t size,
const struct object_id *oid);
int parse_loose_header(const char *hdr, struct object_info *oi);
diff --git a/odb/source-loose.c b/odb/source-loose.c
index 50cb388c35..d77e93e7c4 100644
--- a/odb/source-loose.c
+++ b/odb/source-loose.c
@@ -73,7 +73,7 @@ static int read_object_info_from_path(struct odb_source_loose *loose,
{
int ret;
int fd;
- unsigned long mapsize;
+ size_t mapsize;
void *map = NULL;
git_zstream stream, *stream_to_end = NULL;
char hdr[MAX_HEADER_LEN];
@@ -250,7 +250,7 @@ static int open_loose_object(struct odb_source_loose *loose,
static void *odb_source_loose_map_object(struct odb_source_loose *loose,
const struct object_id *oid,
- unsigned long *size)
+ size_t *size)
{
const char *p;
int fd = open_loose_object(loose, oid, &p);
@@ -285,7 +285,7 @@ struct odb_loose_read_stream {
ODB_LOOSE_READ_STREAM_ERROR,
} z_state;
void *mapped;
- unsigned long mapsize;
+ size_t mapsize;
char hdr[32];
int hdr_avail;
int hdr_used;
@@ -356,7 +356,7 @@ static int odb_source_loose_read_object_stream(struct odb_stream **out,
struct odb_source_loose *loose = odb_source_loose_downcast(source);
struct object_info oi = OBJECT_INFO_INIT;
struct odb_loose_read_stream *st;
- unsigned long mapsize;
+ size_t mapsize;
void *mapped;
mapped = odb_source_loose_map_object(loose, oid, &mapsize);
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH v2 2/5] object-file: use size_t for loose object sizes
2026-11-17 11:38 ` [PATCH v2 2/5] object-file: use size_t for loose object sizes Elia Pinto
@ 2026-11-17 22:34 ` Junio C Hamano
0 siblings, 0 replies; 18+ messages in thread
From: Junio C Hamano @ 2026-11-17 22:34 UTC (permalink / raw)
To: Elia Pinto; +Cc: git, Johannes Schindelin, Patrick Steinhardt, Jonathan Tan
Elia Pinto <gitter.spiros@gmail.com> writes:
> Since c6a4629e32 (odb: use size_t for object_info.sizep and the size
> APIs, 2026-06-15), parse_loose_header() reports the object size as a
> size_t, but unpack_loose_rest() still takes it as an "unsigned long".
> On 64-bit Windows, where "long" has 32 bits while size_t has 64, the
> size is truncated on the way. A header claiming 4GiB + 5 bytes thus
> makes us allocate a 6-byte buffer, the size check added by the
> previous commit is happy because the stream really has 5 bytes, and
> the caller gets that buffer back together with *oi->sizep set to
> 4GiB + 5. Any user of the object then reads far beyond the end of the
> buffer; read_loose_object() already does so when it hashes the
> contents.
>
> Use size_t for the size in unpack_loose_rest() and check_stream_oid(),
> and while at it for the size of the mapped file in map_fd(),
> odb_source_loose_map_object() and unpack_loose_header(), so that the
> whole loose object reading path agrees on the type.
Makes sense. And the changes are straightforward.
Move from ulong to size_t would help truncation on platforms with
32-bit ulong and 64-bit size_t, but does not addres any unsigned
arithmetic wrapping around. Are there places where we should
be using st_add() family of functions?
Thanks.
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v2 3/5] odb/source-loose: check object size when streaming loose objects
2026-11-17 11:38 ` [PATCH v2 0/5] loose object size checks and a few size truncations Elia Pinto
2026-11-17 11:38 ` [PATCH v2 1/5] object-file: check size of inflated loose object against its header Elia Pinto
2026-11-17 11:38 ` [PATCH v2 2/5] object-file: use size_t for loose object sizes Elia Pinto
@ 2026-11-17 11:38 ` Elia Pinto
2026-11-17 22:55 ` Junio C Hamano
2026-11-17 11:38 ` [PATCH v2 4/5] cat-file: use size_t for the length passed to batch_write() Elia Pinto
2026-11-17 11:38 ` [PATCH v2 5/5] notes: avoid truncating the size of concatenated notes Elia Pinto
4 siblings, 1 reply; 18+ messages in thread
From: Elia Pinto @ 2026-11-17 11:38 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Elia Pinto, Junio C Hamano,
Patrick Steinhardt
read_istream_loose() inflates a loose object until zlib reports the end
of the stream, but never compares the number of bytes it produced with
the size given in the object header. A corrupt object whose stream is
shorter than its header claims is thus streamed out without any error,
e.g. "git cat-file blob" exits with 0 after writing fewer bytes than
"git cat-file -s" reports. The same happens if the stream is longer than
the header claims.
Keep track of how many bytes we handed out and report the object as
corrupt if that does not match the size from the header once the stream
ends.
The tests create such objects with the loose_obj() helper, which can
already write a bogus size into the header.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
---
odb/source-loose.c | 14 ++++++++++++++
t/t1006-cat-file.sh | 18 ++++++++++++++++++
2 files changed, 32 insertions(+)
diff --git a/odb/source-loose.c b/odb/source-loose.c
index d77e93e7c4..2233714ac4 100644
--- a/odb/source-loose.c
+++ b/odb/source-loose.c
@@ -289,6 +289,8 @@ struct odb_loose_read_stream {
char hdr[32];
int hdr_avail;
int hdr_used;
+ size_t total_read;
+ struct object_id oid;
};
static ssize_t read_istream_loose(struct odb_stream *_st, char *buf, size_t sz)
@@ -326,6 +328,16 @@ static ssize_t read_istream_loose(struct odb_stream *_st, char *buf, size_t sz)
if (status == Z_STREAM_END) {
git_inflate_end(&st->z);
+ /*
+ * The object must be exactly as large as its
+ * header claims.
+ */
+ if (st->total_read + total_read != st->base.size) {
+ error(_("corrupt loose object '%s'"),
+ oid_to_hex(&st->oid));
+ st->z_state = ODB_LOOSE_READ_STREAM_ERROR;
+ return -1;
+ }
st->z_state = ODB_LOOSE_READ_STREAM_DONE;
break;
}
@@ -335,6 +347,7 @@ static ssize_t read_istream_loose(struct odb_stream *_st, char *buf, size_t sz)
return -1;
}
}
+ st->total_read += total_read;
return total_read;
}
@@ -388,6 +401,7 @@ static int odb_source_loose_read_object_stream(struct odb_stream **out,
st->mapped = mapped;
st->mapsize = mapsize;
+ oidcpy(&st->oid, oid);
st->hdr_used = strlen(st->hdr) + 1;
st->hdr_avail = st->z.total_out;
st->z_state = ODB_LOOSE_READ_STREAM_INUSE;
diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh
index f2b24dd278..622d734ad3 100755
--- a/t/t1006-cat-file.sh
+++ b/t/t1006-cat-file.sh
@@ -1513,4 +1513,22 @@ test_expect_success PIPE 'concurrently written alternates are loaded' '
)
'
+test_expect_success 'setup blobs with bogus size in header' '
+ git init bogus-size &&
+ echo_without_newline short |
+ loose_obj bogus-size/.git/objects blob 100 >oid-too-big &&
+ echo_without_newline "much longer" |
+ loose_obj bogus-size/.git/objects blob 3 >oid-too-small
+'
+
+test_expect_success 'streaming blob shorter than its header fails' '
+ test_must_fail git -C bogus-size cat-file blob $(cat oid-too-big) >out 2>err &&
+ test_grep "corrupt loose object" err
+'
+
+test_expect_success 'streaming blob longer than its header fails' '
+ test_must_fail git -C bogus-size cat-file blob $(cat oid-too-small) >out 2>err &&
+ test_grep "corrupt loose object" err
+'
+
test_done
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH v2 3/5] odb/source-loose: check object size when streaming loose objects
2026-11-17 11:38 ` [PATCH v2 3/5] odb/source-loose: check object size when streaming loose objects Elia Pinto
@ 2026-11-17 22:55 ` Junio C Hamano
0 siblings, 0 replies; 18+ messages in thread
From: Junio C Hamano @ 2026-11-17 22:55 UTC (permalink / raw)
To: Elia Pinto; +Cc: git, Johannes Schindelin, Patrick Steinhardt
Elia Pinto <gitter.spiros@gmail.com> writes:
> if (status == Z_STREAM_END) {
> git_inflate_end(&st->z);
> + /*
> + * The object must be exactly as large as its
> + * header claims.
> + */
> + if (st->total_read + total_read != st->base.size) {
> + error(_("corrupt loose object '%s'"),
> + oid_to_hex(&st->oid));
> + st->z_state = ODB_LOOSE_READ_STREAM_ERROR;
> + return -1;
> + }
Hmph, is this unconditionally noisy error() warranted? The other,
existing failure mode in this function below this "if" block, whose
"return -1" is visible as precontext for the next hunk, is a silent
error return. Yet, you are trying to set an appropriate error code
in st->z_state, which presumably is so that the caller can react to
it and give an appropriate error message up there, not down here,
no?
It feels very inconsistent. Either the existing silent "return -1"
we see below should also become noisy, or we make sure that some
caller up in the callchain reacts to st->z_state to report an error,
keeping this new code just as silent. My preference is the latter.
> st->z_state = ODB_LOOSE_READ_STREAM_DONE;
> break;
> }
> @@ -335,6 +347,7 @@ static ssize_t read_istream_loose(struct odb_stream *_st, char *buf, size_t sz)
> return -1;
> }
> }
> + st->total_read += total_read;
> return total_read;
> }
>
> @@ -388,6 +401,7 @@ static int odb_source_loose_read_object_stream(struct odb_stream **out,
>
> st->mapped = mapped;
> st->mapsize = mapsize;
> + oidcpy(&st->oid, oid);
And if we do choose to make the low-level helper silent (and return
error code via st->z_state), we do not need to add this oid member
to the structure or pay the overhead to memcpy here.
Thanks.
> diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh
> index f2b24dd278..622d734ad3 100755
> --- a/t/t1006-cat-file.sh
> +++ b/t/t1006-cat-file.sh
> @@ -1513,4 +1513,22 @@ test_expect_success PIPE 'concurrently written alternates are loaded' '
> )
> '
>
> +test_expect_success 'setup blobs with bogus size in header' '
> + git init bogus-size &&
> + echo_without_newline short |
> + loose_obj bogus-size/.git/objects blob 100 >oid-too-big &&
> + echo_without_newline "much longer" |
> + loose_obj bogus-size/.git/objects blob 3 >oid-too-small
> +'
> +
> +test_expect_success 'streaming blob shorter than its header fails' '
> + test_must_fail git -C bogus-size cat-file blob $(cat oid-too-big) >out 2>err &&
> + test_grep "corrupt loose object" err
> +'
> +
> +test_expect_success 'streaming blob longer than its header fails' '
> + test_must_fail git -C bogus-size cat-file blob $(cat oid-too-small) >out 2>err &&
> + test_grep "corrupt loose object" err
> +'
> +
> test_done
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v2 4/5] cat-file: use size_t for the length passed to batch_write()
2026-11-17 11:38 ` [PATCH v2 0/5] loose object size checks and a few size truncations Elia Pinto
` (2 preceding siblings ...)
2026-11-17 11:38 ` [PATCH v2 3/5] odb/source-loose: check object size when streaming loose objects Elia Pinto
@ 2026-11-17 11:38 ` Elia Pinto
2026-11-17 22:59 ` Junio C Hamano
2026-11-17 11:38 ` [PATCH v2 5/5] notes: avoid truncating the size of concatenated notes Elia Pinto
4 siblings, 1 reply; 18+ messages in thread
From: Elia Pinto @ 2026-11-17 11:38 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Elia Pinto, Junio C Hamano,
René Scharfe
batch_write() takes the length of the data as an "int", but its callers
pass the size of whole objects as returned by odb_read_object() or
filter_object(), which is a size_t. For an object between 2GiB and
4GiB the length wraps to a negative value, which fwrite() and
write_or_die() then see as an enormous size_t. We happily write the
object and then keep going past the end of the buffer, sending whatever
memory follows to stdout, until we hit an unmapped page and die with
"write error: Bad address". Objects of 4GiB and more are truncated
instead, which desynchronizes the --batch output.
This affects all platforms, not only those with a 32-bit "long". It can
be triggered with "--batch" on any non-blob object, or on blobs with
--filters or --textconv, as blobs are otherwise streamed. For example:
blob=$(test-tool genzeros $((3 * 1024 * 1024 * 1024)) |
git hash-object -w --stdin) &&
echo "$blob big" |
git cat-file --batch="%(objectname) %(rest)" --filters >out
fails with "write error: Bad address"; with stdout going to /dev/null,
strace shows write() being called on addresses far beyond the end of the
3GiB buffer. There is no test, as it would need several GiB of memory
and disk.
Both fwrite() and write_or_die() take a size_t already, so just change
the parameter type.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
---
builtin/cat-file.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index 8870a210ec..c846fe3116 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -425,7 +425,7 @@ static void expand_format(struct strbuf *sb, const char *start,
}
}
-static void batch_write(struct batch_options *opt, const void *data, int len)
+static void batch_write(struct batch_options *opt, const void *data, size_t len)
{
if (opt->buffer_output) {
if (fwrite(data, 1, len, stdout) != len)
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH v2 4/5] cat-file: use size_t for the length passed to batch_write()
2026-11-17 11:38 ` [PATCH v2 4/5] cat-file: use size_t for the length passed to batch_write() Elia Pinto
@ 2026-11-17 22:59 ` Junio C Hamano
0 siblings, 0 replies; 18+ messages in thread
From: Junio C Hamano @ 2026-11-17 22:59 UTC (permalink / raw)
To: Elia Pinto; +Cc: git, Johannes Schindelin, René Scharfe
Elia Pinto <gitter.spiros@gmail.com> writes:
> batch_write() takes the length of the data as an "int", but its callers
> pass the size of whole objects as returned by odb_read_object() or
> filter_object(), which is a size_t.
> ...
> Both fwrite() and write_or_die() take a size_t already, so just change
> the parameter type.
Well analyzed but somehow I find the proposed log message a tad too
verbose. I am almost tempted to say that the above 5-6 lines
suffice.
> -static void batch_write(struct batch_options *opt, const void *data, int len)
> +static void batch_write(struct batch_options *opt, const void *data, size_t len)
> {
> if (opt->buffer_output) {
> if (fwrite(data, 1, len, stdout) != len)
The change is trivially correct. Thanks.
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v2 5/5] notes: avoid truncating the size of concatenated notes
2026-11-17 11:38 ` [PATCH v2 0/5] loose object size checks and a few size truncations Elia Pinto
` (3 preceding siblings ...)
2026-11-17 11:38 ` [PATCH v2 4/5] cat-file: use size_t for the length passed to batch_write() Elia Pinto
@ 2026-11-17 11:38 ` Elia Pinto
2026-11-17 23:01 ` Junio C Hamano
4 siblings, 1 reply; 18+ messages in thread
From: Elia Pinto @ 2026-11-17 11:38 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Elia Pinto, Junio C Hamano,
Patrick Steinhardt
combine_notes_concatenate() reads both notes with odb_read_object(),
which returns their sizes as size_t, but computes the size of the
combined buffer in an "unsigned long". On 64-bit Windows, where "long"
has 32 bits while size_t has 64, two notes adding up to 4GiB or more
give a small buf_len: we allocate that many bytes and then memcpy()
the full contents of both notes into the buffer, writing far past its
end.
The notes can come from a fetched notes ref. Besides notes merges with
the "union" strategy and notes.rewriteMode=concatenate (the default),
the function is also used when loading a notes tree that contains more
than one note for the same object, e.g. at different fanout levels, so
merely displaying such notes is enough to trigger it.
Compute the size as a size_t with st_add3(), and go through
cast_size_t_to_ulong() when passing it to odb_write_object(), whose
length parameter is still an "unsigned long". That makes us die on such
platforms instead of writing a truncated blob, until the write API
learns about size_t, too.
No test, as triggering this needs a platform with a 32-bit "long" and
notes larger than 4GiB.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
---
notes.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/notes.c b/notes.c
index ec9c2cb150..05da28b8d4 100644
--- a/notes.c
+++ b/notes.c
@@ -811,8 +811,7 @@ int combine_notes_concatenate(struct object_id *cur_oid,
const struct object_id *new_oid)
{
char *cur_msg = NULL, *new_msg = NULL, *buf;
- unsigned long buf_len;
- size_t cur_len, new_len;
+ size_t cur_len, new_len, buf_len;
enum object_type cur_type, new_type;
int ret;
@@ -839,7 +838,7 @@ int combine_notes_concatenate(struct object_id *cur_oid,
cur_len--;
/* concatenate cur_msg and new_msg into buf */
- buf_len = cur_len + 2 + new_len;
+ buf_len = st_add3(cur_len, 2, new_len);
buf = (char *) xmalloc(buf_len);
memcpy(buf, cur_msg, cur_len);
buf[cur_len] = '\n';
@@ -850,7 +849,7 @@ int combine_notes_concatenate(struct object_id *cur_oid,
/* create a new blob object from buf */
ret = odb_write_object(the_repository->objects, buf,
- buf_len, OBJ_BLOB, cur_oid);
+ cast_size_t_to_ulong(buf_len), OBJ_BLOB, cur_oid);
free(buf);
return ret;
}
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread