From: Elia Pinto <gitter.spiros@gmail.com>
To: git@vger.kernel.org
Cc: "Johannes Schindelin" <Johannes.Schindelin@gmx.de>,
"Elia Pinto" <gitter.spiros@gmail.com>,
"Junio C Hamano" <gitster@pobox.com>,
"Toon Claes" <toon@iotcl.com>, "Patrick Steinhardt" <ps@pks.im>,
"Jeff King" <peff@peff.net>, "René Scharfe" <l.s.r@web.de>,
"Jonathan Tan" <jonathantanmy@fastmail.com>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH v2 0/5] loose object size checks and a few size truncations
Date: Tue, 17 Nov 2026 12:38:53 +0100 [thread overview]
Message-ID: <cover.1794914210.git.gitter.spiros@gmail.com> (raw)
In-Reply-To: <cover.1793355390.git.gitter.spiros@gmail.com>
Changes since v1:
- v1's 6/6 ("diff: use size_t in count_trailing_blank()") is dropped.
The same line is now changed by 2/4 of v2 of my series "widen a
few sizes and counts that are 32 bits everywhere" [1], which also
widens the counter of count_trailing_blank().
- Rebased onto master at e1ec09b23c (The 22nd batch), as v1 no longer
applied: f495b0e7b9 (odb/source-files: reload alternates on retry,
2026-10-28) also adds a test at the end of t1006, where 3/5 adds its
tests. The code of the five patches is unchanged.
- The messages of 2/5 and 5/5 now say that on 64-bit Windows "long"
has 32 bits while size_t has 64, which is what truncates the sizes.
There were no comments on v1.
These patches come from the same search for truncated sizes on LLP64
(64-bit Windows, where "long" is 32 bits but size_t is 64 bits) as the
"use size_t for a few more object and buffer sizes" series [2]. Its
cover letter explains the method (a MinGW cross build and clang-tidy
with -Wshorten-64-to-32). For these I followed each value back to where
it comes from and forward to where it is used, because they can lead to
out-of-bounds memory accesses. For that reason I sent all of them but
3/5 to the git-security list first; they do not consider them security
issues, so here they are. While checking the loose object reader I also
found two missing size checks that have nothing to do with LLP64 (1/5
and 3/5).
1/5 A loose object whose zlib stream is shorter than the size in its
header is accepted, and the uninitialized tail of the buffer is
handed out as object content (all platforms; e.g. "git cat-file
commit" prints heap memory). A longer stream is silently cut.
2/5 On Windows the size from the loose object header is truncated, so
the caller gets a buffer much smaller than the size it is told,
and reads beyond its end.
3/5 The same check as 1/5 for the streaming reader of loose objects,
which today streams out a short or long object without an error.
4/5 "git cat-file --batch" writes memory past the end of objects
between 2GiB and 4GiB (all platforms; non-blob objects, or blobs
with --filters/--textconv).
5/5 On Windows, concatenating notes that add up to 4GiB or more
overflows a heap buffer.
1/5 and 3/5 come with tests, which are cheap, and fail without the
fix. The others would need objects of several GiB (and 2/5 and 5/5 a
32-bit "long"), so their commit messages describe how to reproduce
them instead. I have not run anything on Windows: the Windows-only
issues are based on reading the code and on the cross-compiled build.
5/5 passes the combined size to odb_write_object() through
cast_size_t_to_ulong(), as the length parameter is still an "unsigned
long" on master. With 1/4 of [2], which makes it a size_t, the cast
becomes unnecessary but harmless; the two series do not conflict.
The series is based on master at e1ec09b23c (The 22nd batch).
[1] https://lore.kernel.org/git/cover.1794909894.git.gitter.spiros@gmail.com/
[2] https://lore.kernel.org/git/cover.1793439137.git.gitter.spiros@gmail.com/
Elia Pinto (5):
object-file: check size of inflated loose object against its header
object-file: use size_t for loose object sizes
odb/source-loose: check object size when streaming loose objects
cat-file: use size_t for the length passed to batch_write()
notes: avoid truncating the size of concatenated notes
builtin/cat-file.c | 2 +-
notes.c | 7 +++----
object-file.c | 29 ++++++++++++++++++-----------
object-file.h | 6 +++---
odb/source-loose.c | 22 ++++++++++++++++++----
t/lib-loose.sh | 5 ++++-
t/t1006-cat-file.sh | 18 ++++++++++++++++++
t/t1060-object-corruption.sh | 28 ++++++++++++++++++++++++++++
8 files changed, 93 insertions(+), 24 deletions(-)
Range-diff against v1:
1: 6c2efb87fb = 1: edb5f34498 object-file: check size of inflated loose object against its header
2: ee4439354a ! 2: eca5d3ddb8 object-file: use size_t for loose object sizes
@@ Commit message
Since c6a4629e32 (odb: use size_t for object_info.sizep and the size
APIs, 2026-06-15), parse_loose_header() reports the object size as a
- size_t. But unpack_loose_rest() still takes an "unsigned long", which
- is only 32 bits on Windows. A header claiming 4GiB + 5 bytes thus
- makes us allocate a 6-byte buffer, the size check added by the previous
- commit is happy because the stream really has 5 bytes, and the caller
- gets that buffer back together with *oi->sizep set to 4GiB + 5. Any
- user of the object then reads far beyond the end of the buffer;
- read_loose_object() already does so when it hashes the contents.
+ size_t, but unpack_loose_rest() still takes it as an "unsigned long".
+ On 64-bit Windows, where "long" has 32 bits while size_t has 64, the
+ size is truncated on the way. A header claiming 4GiB + 5 bytes thus
+ makes us allocate a 6-byte buffer, the size check added by the
+ previous commit is happy because the stream really has 5 bytes, and
+ the caller gets that buffer back together with *oi->sizep set to
+ 4GiB + 5. Any user of the object then reads far beyond the end of the
+ buffer; read_loose_object() already does so when it hashes the
+ contents.
Use size_t for the size in unpack_loose_rest() and check_stream_oid(),
and while at it for the size of the mapped file in map_fd(),
3: dee492c3d8 ! 3: a8572aa836 odb/source-loose: check object size when streaming loose objects
@@ odb/source-loose.c: static int odb_source_loose_read_object_stream(struct odb_st
st->z_state = ODB_LOOSE_READ_STREAM_INUSE;
## t/t1006-cat-file.sh ##
-@@ t/t1006-cat-file.sh: test_objects_filter "object:type=commit"
- test_objects_filter "object:type=tag"
- test_objects_filter "object:type=tree"
+@@ t/t1006-cat-file.sh: test_expect_success PIPE 'concurrently written alternates are loaded' '
+ )
+ '
+test_expect_success 'setup blobs with bogus size in header' '
+ git init bogus-size &&
4: b2083632a1 = 4: 5b61a1b089 cat-file: use size_t for the length passed to batch_write()
5: 21f5a4e009 ! 5: 04f320d1de notes: avoid truncating the size of concatenated notes
@@ Commit message
combine_notes_concatenate() reads both notes with odb_read_object(),
which returns their sizes as size_t, but computes the size of the
- combined buffer in an "unsigned long". On platforms where that type is
- 32 bits wide, like Windows, two notes adding up to 4GiB or more give a
- small buf_len: we allocate that many bytes and then memcpy() the full
- contents of both notes into the buffer, writing far past its end.
+ combined buffer in an "unsigned long". On 64-bit Windows, where "long"
+ has 32 bits while size_t has 64, two notes adding up to 4GiB or more
+ give a small buf_len: we allocate that many bytes and then memcpy()
+ the full contents of both notes into the buffer, writing far past its
+ end.
The notes can come from a fetched notes ref. Besides notes merges with
the "union" strategy and notes.rewriteMode=concatenate (the default),
6: 79f1781abd < -: ---------- diff: use size_t in count_trailing_blank()
base-commit: e1ec09b23c8e0deb21a0b95ac5996422277395d4
--
2.53.0
next prev parent reply other threads:[~2026-11-17 11:39 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-30 10:28 [PATCH 0/6] loose object size checks and a few size truncations Elia Pinto
2026-10-30 10:28 ` [PATCH 1/6] object-file: check size of inflated loose object against its header Elia Pinto
2026-10-30 10:28 ` [PATCH 2/6] object-file: use size_t for loose object sizes Elia Pinto
2026-10-30 10:28 ` [PATCH 3/6] odb/source-loose: check object size when streaming loose objects Elia Pinto
2026-10-30 10:28 ` [PATCH 4/6] cat-file: use size_t for the length passed to batch_write() Elia Pinto
2026-10-30 10:28 ` [PATCH 5/6] notes: avoid truncating the size of concatenated notes Elia Pinto
2026-10-30 10:28 ` [PATCH 6/6] diff: use size_t in count_trailing_blank() Elia Pinto
2026-11-17 11:38 ` Elia Pinto [this message]
2026-11-17 11:38 ` [PATCH v2 1/5] object-file: check size of inflated loose object against its header Elia Pinto
2026-11-17 19:33 ` Junio C Hamano
2026-11-17 11:38 ` [PATCH v2 2/5] object-file: use size_t for loose object sizes Elia Pinto
2026-11-17 22:34 ` Junio C Hamano
2026-11-17 11:38 ` [PATCH v2 3/5] odb/source-loose: check object size when streaming loose objects Elia Pinto
2026-11-17 22:55 ` Junio C Hamano
2026-11-17 11:38 ` [PATCH v2 4/5] cat-file: use size_t for the length passed to batch_write() Elia Pinto
2026-11-17 22:59 ` Junio C Hamano
2026-11-17 11:38 ` [PATCH v2 5/5] notes: avoid truncating the size of concatenated notes Elia Pinto
2026-11-17 23:01 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1794914210.git.gitter.spiros@gmail.com \
--to=gitter.spiros@gmail.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=avarab@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jonathantanmy@fastmail.com \
--cc=l.s.r@web.de \
--cc=peff@peff.net \
--cc=ps@pks.im \
--cc=toon@iotcl.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox