All of lore.kernel.org
 help / color / mirror / Atom feed
From: Elia Pinto <gitter.spiros@gmail.com>
To: git@vger.kernel.org
Cc: "Johannes Schindelin" <Johannes.Schindelin@gmx.de>,
	"Elia Pinto" <gitter.spiros@gmail.com>,
	"Junio C Hamano" <gitster@pobox.com>,
	"Toon Claes" <toon@iotcl.com>, "Patrick Steinhardt" <ps@pks.im>,
	"Jeff King" <peff@peff.net>, "René Scharfe" <l.s.r@web.de>,
	"Jonathan Tan" <jonathantanmy@fastmail.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH v2 0/5] loose object size checks and a few size truncations
Date: Tue, 17 Nov 2026 12:38:53 +0100	[thread overview]
Message-ID: <cover.1794914210.git.gitter.spiros@gmail.com> (raw)
In-Reply-To: <cover.1793355390.git.gitter.spiros@gmail.com>

Changes since v1:

 - v1's 6/6 ("diff: use size_t in count_trailing_blank()") is dropped.
   The same line is now changed by 2/4 of v2 of my series "widen a
   few sizes and counts that are 32 bits everywhere" [1], which also
   widens the counter of count_trailing_blank().

 - Rebased onto master at e1ec09b23c (The 22nd batch), as v1 no longer
   applied: f495b0e7b9 (odb/source-files: reload alternates on retry,
   2026-10-28) also adds a test at the end of t1006, where 3/5 adds its
   tests. The code of the five patches is unchanged.

 - The messages of 2/5 and 5/5 now say that on 64-bit Windows "long"
   has 32 bits while size_t has 64, which is what truncates the sizes.

There were no comments on v1.

These patches come from the same search for truncated sizes on LLP64
(64-bit Windows, where "long" is 32 bits but size_t is 64 bits) as the
"use size_t for a few more object and buffer sizes" series [2]. Its
cover letter explains the method (a MinGW cross build and clang-tidy
with -Wshorten-64-to-32). For these I followed each value back to where
it comes from and forward to where it is used, because they can lead to
out-of-bounds memory accesses. For that reason I sent all of them but
3/5 to the git-security list first; they do not consider them security
issues, so here they are. While checking the loose object reader I also
found two missing size checks that have nothing to do with LLP64 (1/5
and 3/5).

 1/5 A loose object whose zlib stream is shorter than the size in its
     header is accepted, and the uninitialized tail of the buffer is
     handed out as object content (all platforms; e.g. "git cat-file
     commit" prints heap memory). A longer stream is silently cut.

 2/5 On Windows the size from the loose object header is truncated, so
     the caller gets a buffer much smaller than the size it is told,
     and reads beyond its end.

 3/5 The same check as 1/5 for the streaming reader of loose objects,
     which today streams out a short or long object without an error.

 4/5 "git cat-file --batch" writes memory past the end of objects
     between 2GiB and 4GiB (all platforms; non-blob objects, or blobs
     with --filters/--textconv).

 5/5 On Windows, concatenating notes that add up to 4GiB or more
     overflows a heap buffer.

1/5 and 3/5 come with tests, which are cheap, and fail without the
fix. The others would need objects of several GiB (and 2/5 and 5/5 a
32-bit "long"), so their commit messages describe how to reproduce
them instead. I have not run anything on Windows: the Windows-only
issues are based on reading the code and on the cross-compiled build.

5/5 passes the combined size to odb_write_object() through
cast_size_t_to_ulong(), as the length parameter is still an "unsigned
long" on master. With 1/4 of [2], which makes it a size_t, the cast
becomes unnecessary but harmless; the two series do not conflict.

The series is based on master at e1ec09b23c (The 22nd batch).

[1] https://lore.kernel.org/git/cover.1794909894.git.gitter.spiros@gmail.com/
[2] https://lore.kernel.org/git/cover.1793439137.git.gitter.spiros@gmail.com/

Elia Pinto (5):
  object-file: check size of inflated loose object against its header
  object-file: use size_t for loose object sizes
  odb/source-loose: check object size when streaming loose objects
  cat-file: use size_t for the length passed to batch_write()
  notes: avoid truncating the size of concatenated notes

 builtin/cat-file.c           |  2 +-
 notes.c                      |  7 +++----
 object-file.c                | 29 ++++++++++++++++++-----------
 object-file.h                |  6 +++---
 odb/source-loose.c           | 22 ++++++++++++++++++----
 t/lib-loose.sh               |  5 ++++-
 t/t1006-cat-file.sh          | 18 ++++++++++++++++++
 t/t1060-object-corruption.sh | 28 ++++++++++++++++++++++++++++
 8 files changed, 93 insertions(+), 24 deletions(-)

Range-diff against v1:
1:  6c2efb87fb = 1:  edb5f34498 object-file: check size of inflated loose object against its header
2:  ee4439354a ! 2:  eca5d3ddb8 object-file: use size_t for loose object sizes
    @@ Commit message
     
         Since c6a4629e32 (odb: use size_t for object_info.sizep and the size
         APIs, 2026-06-15), parse_loose_header() reports the object size as a
    -    size_t. But unpack_loose_rest() still takes an "unsigned long", which
    -    is only 32 bits on Windows. A header claiming 4GiB + 5 bytes thus
    -    makes us allocate a 6-byte buffer, the size check added by the previous
    -    commit is happy because the stream really has 5 bytes, and the caller
    -    gets that buffer back together with *oi->sizep set to 4GiB + 5. Any
    -    user of the object then reads far beyond the end of the buffer;
    -    read_loose_object() already does so when it hashes the contents.
    +    size_t, but unpack_loose_rest() still takes it as an "unsigned long".
    +    On 64-bit Windows, where "long" has 32 bits while size_t has 64, the
    +    size is truncated on the way. A header claiming 4GiB + 5 bytes thus
    +    makes us allocate a 6-byte buffer, the size check added by the
    +    previous commit is happy because the stream really has 5 bytes, and
    +    the caller gets that buffer back together with *oi->sizep set to
    +    4GiB + 5. Any user of the object then reads far beyond the end of the
    +    buffer; read_loose_object() already does so when it hashes the
    +    contents.
     
         Use size_t for the size in unpack_loose_rest() and check_stream_oid(),
         and while at it for the size of the mapped file in map_fd(),
3:  dee492c3d8 ! 3:  a8572aa836 odb/source-loose: check object size when streaming loose objects
    @@ odb/source-loose.c: static int odb_source_loose_read_object_stream(struct odb_st
      	st->z_state = ODB_LOOSE_READ_STREAM_INUSE;
     
      ## t/t1006-cat-file.sh ##
    -@@ t/t1006-cat-file.sh: test_objects_filter "object:type=commit"
    - test_objects_filter "object:type=tag"
    - test_objects_filter "object:type=tree"
    +@@ t/t1006-cat-file.sh: test_expect_success PIPE 'concurrently written alternates are loaded' '
    + 	)
    + '
      
     +test_expect_success 'setup blobs with bogus size in header' '
     +	git init bogus-size &&
4:  b2083632a1 = 4:  5b61a1b089 cat-file: use size_t for the length passed to batch_write()
5:  21f5a4e009 ! 5:  04f320d1de notes: avoid truncating the size of concatenated notes
    @@ Commit message
     
         combine_notes_concatenate() reads both notes with odb_read_object(),
         which returns their sizes as size_t, but computes the size of the
    -    combined buffer in an "unsigned long". On platforms where that type is
    -    32 bits wide, like Windows, two notes adding up to 4GiB or more give a
    -    small buf_len: we allocate that many bytes and then memcpy() the full
    -    contents of both notes into the buffer, writing far past its end.
    +    combined buffer in an "unsigned long". On 64-bit Windows, where "long"
    +    has 32 bits while size_t has 64, two notes adding up to 4GiB or more
    +    give a small buf_len: we allocate that many bytes and then memcpy()
    +    the full contents of both notes into the buffer, writing far past its
    +    end.
     
         The notes can come from a fetched notes ref. Besides notes merges with
         the "union" strategy and notes.rewriteMode=concatenate (the default),
6:  79f1781abd < -:  ---------- diff: use size_t in count_trailing_blank()

base-commit: e1ec09b23c8e0deb21a0b95ac5996422277395d4
-- 
2.53.0


  parent reply	other threads:[~2026-11-17 11:39 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-30 10:28 [PATCH 0/6] loose object size checks and a few size truncations Elia Pinto
2026-10-30 10:28 ` [PATCH 1/6] object-file: check size of inflated loose object against its header Elia Pinto
2026-10-30 10:28 ` [PATCH 2/6] object-file: use size_t for loose object sizes Elia Pinto
2026-10-30 10:28 ` [PATCH 3/6] odb/source-loose: check object size when streaming loose objects Elia Pinto
2026-10-30 10:28 ` [PATCH 4/6] cat-file: use size_t for the length passed to batch_write() Elia Pinto
2026-10-30 10:28 ` [PATCH 5/6] notes: avoid truncating the size of concatenated notes Elia Pinto
2026-10-30 10:28 ` [PATCH 6/6] diff: use size_t in count_trailing_blank() Elia Pinto
2026-11-17 11:38 ` Elia Pinto [this message]
2026-11-17 11:38   ` [PATCH v2 1/5] object-file: check size of inflated loose object against its header Elia Pinto
2026-11-17 19:33     ` Junio C Hamano
2026-11-17 11:38   ` [PATCH v2 2/5] object-file: use size_t for loose object sizes Elia Pinto
2026-11-17 22:34     ` Junio C Hamano
2026-11-17 11:38   ` [PATCH v2 3/5] odb/source-loose: check object size when streaming loose objects Elia Pinto
2026-11-17 22:55     ` Junio C Hamano
2026-11-17 11:38   ` [PATCH v2 4/5] cat-file: use size_t for the length passed to batch_write() Elia Pinto
2026-11-17 22:59     ` Junio C Hamano
2026-11-17 11:38   ` [PATCH v2 5/5] notes: avoid truncating the size of concatenated notes Elia Pinto
2026-11-17 23:01     ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1794914210.git.gitter.spiros@gmail.com \
    --to=gitter.spiros@gmail.com \
    --cc=Johannes.Schindelin@gmx.de \
    --cc=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jonathantanmy@fastmail.com \
    --cc=l.s.r@web.de \
    --cc=peff@peff.net \
    --cc=ps@pks.im \
    --cc=toon@iotcl.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.