Git development
 help / color / mirror / Atom feed
From: Elia Pinto <gitter.spiros@gmail.com>
To: git@vger.kernel.org
Cc: "Johannes Schindelin" <Johannes.Schindelin@gmx.de>,
	"Elia Pinto" <gitter.spiros@gmail.com>,
	"Junio C Hamano" <gitster@pobox.com>,
	"Toon Claes" <toon@iotcl.com>, "Patrick Steinhardt" <ps@pks.im>,
	"Jeff King" <peff@peff.net>, "René Scharfe" <l.s.r@web.de>,
	"Jonathan Tan" <jonathantanmy@fastmail.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH v2 0/5] loose object size checks and a few size truncations
Date: Tue, 17 Nov 2026 12:38:53 +0100	[thread overview]
Message-ID: <cover.1794914210.git.gitter.spiros@gmail.com> (raw)
In-Reply-To: <cover.1793355390.git.gitter.spiros@gmail.com>

Changes since v1:

 - v1's 6/6 ("diff: use size_t in count_trailing_blank()") is dropped.
   The same line is now changed by 2/4 of v2 of my series "widen a
   few sizes and counts that are 32 bits everywhere" [1], which also
   widens the counter of count_trailing_blank().

 - Rebased onto master at e1ec09b23c (The 22nd batch), as v1 no longer
   applied: f495b0e7b9 (odb/source-files: reload alternates on retry,
   2026-10-28) also adds a test at the end of t1006, where 3/5 adds its
   tests. The code of the five patches is unchanged.

 - The messages of 2/5 and 5/5 now say that on 64-bit Windows "long"
   has 32 bits while size_t has 64, which is what truncates the sizes.

There were no comments on v1.

These patches come from the same search for truncated sizes on LLP64
(64-bit Windows, where "long" is 32 bits but size_t is 64 bits) as the
"use size_t for a few more object and buffer sizes" series [2]. Its
cover letter explains the method (a MinGW cross build and clang-tidy
with -Wshorten-64-to-32). For these I followed each value back to where
it comes from and forward to where it is used, because they can lead to
out-of-bounds memory accesses. For that reason I sent all of them but
3/5 to the git-security list first; they do not consider them security
issues, so here they are. While checking the loose object reader I also
found two missing size checks that have nothing to do with LLP64 (1/5
and 3/5).

 1/5 A loose object whose zlib stream is shorter than the size in its
     header is accepted, and the uninitialized tail of the buffer is
     handed out as object content (all platforms; e.g. "git cat-file
     commit" prints heap memory). A longer stream is silently cut.

 2/5 On Windows the size from the loose object header is truncated, so
     the caller gets a buffer much smaller than the size it is told,
     and reads beyond its end.

 3/5 The same check as 1/5 for the streaming reader of loose objects,
     which today streams out a short or long object without an error.

 4/5 "git cat-file --batch" writes memory past the end of objects
     between 2GiB and 4GiB (all platforms; non-blob objects, or blobs
     with --filters/--textconv).

 5/5 On Windows, concatenating notes that add up to 4GiB or more
     overflows a heap buffer.

1/5 and 3/5 come with tests, which are cheap, and fail without the
fix. The others would need objects of several GiB (and 2/5 and 5/5 a
32-bit "long"), so their commit messages describe how to reproduce
them instead. I have not run anything on Windows: the Windows-only
issues are based on reading the code and on the cross-compiled build.

5/5 passes the combined size to odb_write_object() through
cast_size_t_to_ulong(), as the length parameter is still an "unsigned
long" on master. With 1/4 of [2], which makes it a size_t, the cast
becomes unnecessary but harmless; the two series do not conflict.

The series is based on master at e1ec09b23c (The 22nd batch).

[1] https://lore.kernel.org/git/cover.1794909894.git.gitter.spiros@gmail.com/
[2] https://lore.kernel.org/git/cover.1793439137.git.gitter.spiros@gmail.com/

Elia Pinto (5):
  object-file: check size of inflated loose object against its header
  object-file: use size_t for loose object sizes
  odb/source-loose: check object size when streaming loose objects
  cat-file: use size_t for the length passed to batch_write()
  notes: avoid truncating the size of concatenated notes

 builtin/cat-file.c           |  2 +-
 notes.c                      |  7 +++----
 object-file.c                | 29 ++++++++++++++++++-----------
 object-file.h                |  6 +++---
 odb/source-loose.c           | 22 ++++++++++++++++++----
 t/lib-loose.sh               |  5 ++++-
 t/t1006-cat-file.sh          | 18 ++++++++++++++++++
 t/t1060-object-corruption.sh | 28 ++++++++++++++++++++++++++++
 8 files changed, 93 insertions(+), 24 deletions(-)

Range-diff against v1:
1:  6c2efb87fb = 1:  edb5f34498 object-file: check size of inflated loose object against its header
2:  ee4439354a ! 2:  eca5d3ddb8 object-file: use size_t for loose object sizes
    @@ Commit message
     
         Since c6a4629e32 (odb: use size_t for object_info.sizep and the size
         APIs, 2026-06-15), parse_loose_header() reports the object size as a
    -    size_t. But unpack_loose_rest() still takes an "unsigned long", which
    -    is only 32 bits on Windows. A header claiming 4GiB + 5 bytes thus
    -    makes us allocate a 6-byte buffer, the size check added by the previous
    -    commit is happy because the stream really has 5 bytes, and the caller
    -    gets that buffer back together with *oi->sizep set to 4GiB + 5. Any
    -    user of the object then reads far beyond the end of the buffer;
    -    read_loose_object() already does so when it hashes the contents.
    +    size_t, but unpack_loose_rest() still takes it as an "unsigned long".
    +    On 64-bit Windows, where "long" has 32 bits while size_t has 64, the
    +    size is truncated on the way. A header claiming 4GiB + 5 bytes thus
    +    makes us allocate a 6-byte buffer, the size check added by the
    +    previous commit is happy because the stream really has 5 bytes, and
    +    the caller gets that buffer back together with *oi->sizep set to
    +    4GiB + 5. Any user of the object then reads far beyond the end of the
    +    buffer; read_loose_object() already does so when it hashes the
    +    contents.
     
         Use size_t for the size in unpack_loose_rest() and check_stream_oid(),
         and while at it for the size of the mapped file in map_fd(),
3:  dee492c3d8 ! 3:  a8572aa836 odb/source-loose: check object size when streaming loose objects
    @@ odb/source-loose.c: static int odb_source_loose_read_object_stream(struct odb_st
      	st->z_state = ODB_LOOSE_READ_STREAM_INUSE;
     
      ## t/t1006-cat-file.sh ##
    -@@ t/t1006-cat-file.sh: test_objects_filter "object:type=commit"
    - test_objects_filter "object:type=tag"
    - test_objects_filter "object:type=tree"
    +@@ t/t1006-cat-file.sh: test_expect_success PIPE 'concurrently written alternates are loaded' '
    + 	)
    + '
      
     +test_expect_success 'setup blobs with bogus size in header' '
     +	git init bogus-size &&
4:  b2083632a1 = 4:  5b61a1b089 cat-file: use size_t for the length passed to batch_write()
5:  21f5a4e009 ! 5:  04f320d1de notes: avoid truncating the size of concatenated notes
    @@ Commit message
     
         combine_notes_concatenate() reads both notes with odb_read_object(),
         which returns their sizes as size_t, but computes the size of the
    -    combined buffer in an "unsigned long". On platforms where that type is
    -    32 bits wide, like Windows, two notes adding up to 4GiB or more give a
    -    small buf_len: we allocate that many bytes and then memcpy() the full
    -    contents of both notes into the buffer, writing far past its end.
    +    combined buffer in an "unsigned long". On 64-bit Windows, where "long"
    +    has 32 bits while size_t has 64, two notes adding up to 4GiB or more
    +    give a small buf_len: we allocate that many bytes and then memcpy()
    +    the full contents of both notes into the buffer, writing far past its
    +    end.
     
         The notes can come from a fetched notes ref. Besides notes merges with
         the "union" strategy and notes.rewriteMode=concatenate (the default),
6:  79f1781abd < -:  ---------- diff: use size_t in count_trailing_blank()

base-commit: e1ec09b23c8e0deb21a0b95ac5996422277395d4
-- 
2.53.0


  parent reply	other threads:[~2026-11-17 11:39 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-30 10:28 [PATCH 0/6] loose object size checks and a few size truncations Elia Pinto
2026-10-30 10:28 ` [PATCH 1/6] object-file: check size of inflated loose object against its header Elia Pinto
2026-10-30 10:28 ` [PATCH 2/6] object-file: use size_t for loose object sizes Elia Pinto
2026-10-30 10:28 ` [PATCH 3/6] odb/source-loose: check object size when streaming loose objects Elia Pinto
2026-10-30 10:28 ` [PATCH 4/6] cat-file: use size_t for the length passed to batch_write() Elia Pinto
2026-10-30 10:28 ` [PATCH 5/6] notes: avoid truncating the size of concatenated notes Elia Pinto
2026-10-30 10:28 ` [PATCH 6/6] diff: use size_t in count_trailing_blank() Elia Pinto
2026-11-17 11:38 ` Elia Pinto [this message]
2026-11-17 11:38   ` [PATCH v2 1/5] object-file: check size of inflated loose object against its header Elia Pinto
2026-11-17 19:33     ` Junio C Hamano
2026-11-17 11:38   ` [PATCH v2 2/5] object-file: use size_t for loose object sizes Elia Pinto
2026-11-17 22:34     ` Junio C Hamano
2026-11-17 11:38   ` [PATCH v2 3/5] odb/source-loose: check object size when streaming loose objects Elia Pinto
2026-11-17 22:55     ` Junio C Hamano
2026-11-17 11:38   ` [PATCH v2 4/5] cat-file: use size_t for the length passed to batch_write() Elia Pinto
2026-11-17 22:59     ` Junio C Hamano
2026-11-17 11:38   ` [PATCH v2 5/5] notes: avoid truncating the size of concatenated notes Elia Pinto
2026-11-17 23:01     ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1794914210.git.gitter.spiros@gmail.com \
    --to=gitter.spiros@gmail.com \
    --cc=Johannes.Schindelin@gmx.de \
    --cc=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jonathantanmy@fastmail.com \
    --cc=l.s.r@web.de \
    --cc=peff@peff.net \
    --cc=ps@pks.im \
    --cc=toon@iotcl.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox