Git development
 help / color / mirror / Atom feed
* [PATCH 0/3] Allow merges with results over 2 GiB on 64-bit Windows
@ 2026-11-17 17:43 Christian Couder
  2026-11-17 17:43 ` [PATCH 1/3] notes-merge: use size_t in write_buf_to_worktree() Christian Couder
                   ` (2 more replies)
  0 siblings, 3 replies; 4+ messages in thread
From: Christian Couder @ 2026-11-17 17:43 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Patrick Steinhardt, Kaartic Sivaraam, Jeff King,
	Johannes Schindelin, Christian Couder

Two recent series improved how merge results are sized:

  - `jk/xdiff-size_t` (merged by 81db32ddce) replaced `mmbuffer_t`
    with `mmfile_t`, whose size is a `size_t` instead of a `long`.

  - `kn/safer-mem-arithmetic-in-xdiff` (merged by 1c93cc03ea) computes
    the merge buffer size as a `size_t` instead of an `int`, with
    overflow checks.

As the latter targets the `maint` branches, which don't have
`jk/xdiff-size_t`, it also makes `xdl_do_merge()` fail when the result
is bigger than `LONG_MAX`, so that it cannot be silently truncated
when stored into mmbuffer_t's `long` size on LLP64 platforms like
64-bit Windows.

On `master`, that check is not needed for `mmfile_t` anymore, but as
Kaartic pointed out when that series was discussed, merge results were
still passed on to a few functions taking an `unsigned long`. This
series converts those to `size_t`, and then removes the check.

Patch 1/3 also fixes an existing bug: since c6a4629e32 (odb: use
size_t for object_info.sizep and the size APIs, 2026-06-15), a
delete/modify conflict in `git notes merge` on a note of 4 GiB or more
silently writes a truncated note on LLP64 platforms. That patch could
be applied on its own.

## Overview of the patches

  - Patch 1/3 changes the size argument of `write_buf_to_worktree()`
    in "notes-merge.c" from `unsigned long` to `size_t`.

  - Patch 2/3 uses `size_t` for the length argument of
    `odb_write_object()`, `odb_write_object_ext()`, the
    `odb_source_write_object()` wrapper and `write_loose_object()`,
    and for the size stored in `struct inmemory_object`, so that
    object lengths are not truncated before reaching the already
    `size_t` backends.

  - Patch 3/3 removes the `size > LONG_MAX` check in `xdl_do_merge()`,
    lets the `t4300` tests that expect a successful big merge run on
    LLP64 platforms, and removes the test that expected a failure
    there.

## Testing

The code paths changed here only behave differently where `long` is 32
bits and `size_t` is 64 bits, so Linux and macOS CI cannot exercise
them. It would be very nice if someone with a 64-bit Windows setup
could run `t4300` with `GIT_TEST_LONG=1`. Beware that it needs enough
memory and disk for a merge result just over 4 GiB.

## CI

Some test scripts didn't want to start on MacOS but otherwise all the
tests passed, see:

https://github.com/chriscool/git/actions/runs/44118577071


Christian Couder (3):
  notes-merge: use size_t in write_buf_to_worktree()
  odb: widen odb_write_object() length to size_t
  xdiff: drop LONG_MAX check on merge result size

 notes-merge.c         |  2 +-
 odb.c                 |  2 +-
 odb.h                 |  4 ++--
 odb/source-inmemory.c |  2 +-
 odb/source-loose.c    |  2 +-
 odb/source.h          |  2 +-
 t/t4300-merge-tree.sh | 13 ++-----------
 xdiff/xmerge.c        |  4 ----
 8 files changed, 9 insertions(+), 22 deletions(-)


base-commit: e1ec09b23c8e0deb21a0b95ac5996422277395d4
-- 
2.56.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 1/3] notes-merge: use size_t in write_buf_to_worktree()
  2026-11-17 17:43 [PATCH 0/3] Allow merges with results over 2 GiB on 64-bit Windows Christian Couder
@ 2026-11-17 17:43 ` Christian Couder
  2026-11-17 17:43 ` [PATCH 2/3] odb: widen odb_write_object() length to size_t Christian Couder
  2026-11-17 17:43 ` [PATCH 3/3] xdiff: drop LONG_MAX check on merge result size Christian Couder
  2 siblings, 0 replies; 4+ messages in thread
From: Christian Couder @ 2026-11-17 17:43 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Patrick Steinhardt, Kaartic Sivaraam, Jeff King,
	Johannes Schindelin, Christian Couder

The write_buf_to_worktree() function in "notes-merge.c" has an
`unsigned long` size argument, while its two callers from that same
file:

1. write_note_to_worktree(), and
2. ll_merge_in_worktree(),

both pass a `size_t` to that size argument.

Since c6a4629e32 (odb: use size_t for object_info.sizep and the size
APIs, 2026-06-15), the first caller reads the note into a `size_t` size
before passing it on to write_buf_to_worktree(). On LLP64 platforms
like 64-bit Windows, where `unsigned long` is 32 bits, a delete/modify
conflict on a note of 4 GiB or more therefore silently writes a
truncated note into NOTES_MERGE_WORKTREE.

The second caller passes the size of the ll_merge() result, which is a
`size_t` since 81db32ddce (Merge branch 'jk/xdiff-size_t', 2026-10-15).
That size cannot be truncated as since 1c93cc03ea (Merge branch
'kn/safer-mem-arithmetic-in-xdiff', 2026-10-16), xdl_do_merge() errors
out if it is greater than LONG_MAX.

But we want to get rid of that `size > LONG_MAX` check in a following
commit.

To fix the truncation issue with the first caller and to prepare for
getting rid of the check, let's convert the size argument in
write_buf_to_worktree() to a `size_t`.

Signed-off-by: Christian Couder <christian.couder@gmail.com>
---
 notes-merge.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/notes-merge.c b/notes-merge.c
index d361e70a48..4b010bcfc0 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -310,7 +310,7 @@ static void check_notes_merge_worktree(struct notes_merge_options *o)
 }
 
 static void write_buf_to_worktree(const struct object_id *obj,
-				  const char *buf, unsigned long size)
+				  const char *buf, size_t size)
 {
 	int fd;
 	char *path = repo_git_path(the_repository, NOTES_MERGE_WORKTREE "/%s", oid_to_hex(obj));
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* [PATCH 2/3] odb: widen odb_write_object() length to size_t
  2026-11-17 17:43 [PATCH 0/3] Allow merges with results over 2 GiB on 64-bit Windows Christian Couder
  2026-11-17 17:43 ` [PATCH 1/3] notes-merge: use size_t in write_buf_to_worktree() Christian Couder
@ 2026-11-17 17:43 ` Christian Couder
  2026-11-17 17:43 ` [PATCH 3/3] xdiff: drop LONG_MAX check on merge result size Christian Couder
  2 siblings, 0 replies; 4+ messages in thread
From: Christian Couder @ 2026-11-17 17:43 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Patrick Steinhardt, Kaartic Sivaraam, Jeff King,
	Johannes Schindelin, Christian Couder

odb_write_object() and odb_write_object_ext() use an `unsigned long`
for their object length argument, while almost everything they pass it
to already uses `size_t`: hash_object_file(), format_object_header(),
convert_object_file(), the `write_object` callback of `struct
odb_source`, and `git_zstream`.

Many callers also pass a `size_t`, often a `strbuf.len`. On LLP64
platforms like 64-bit Windows, where `unsigned long` is 32 bits, such
lengths may silently be truncated.

Since 81db32ddce (Merge branch 'jk/xdiff-size_t', 2026-10-15), the size
of an ll_merge() result is also passed as a `size_t` to
odb_write_object(). It cannot be truncated as since 1c93cc03ea (Merge
branch 'kn/safer-mem-arithmetic-in-xdiff', 2026-10-16), that size
argument cannot be greater than LONG_MAX because xdl_do_merge() errors
out if `size > LONG_MAX`.

But we want to get rid of that `size > LONG_MAX` check in a following
commit.

To fix the possible truncation issues with many callers and to prepare
for getting rid of the check, let's use `size_t` for:

- the length argument of odb_write_object(), odb_write_object_ext(),
  the odb_source_write_object() wrapper, and write_loose_object(),

- the size stored in `struct inmemory_object`, which is later
  copied back into `size_t` fields (`object_info.sizep` and
  `odb_stream.size`).

Signed-off-by: Christian Couder <christian.couder@gmail.com>
---
 odb.c                 | 2 +-
 odb.h                 | 4 ++--
 odb/source-inmemory.c | 2 +-
 odb/source-loose.c    | 2 +-
 odb/source.h          | 2 +-
 5 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/odb.c b/odb.c
index de5be285d3..56a7910b03 100644
--- a/odb.c
+++ b/odb.c
@@ -702,7 +702,7 @@ void odb_assert_oid_type(struct object_database *odb,
 }
 
 int odb_write_object_ext(struct object_database *odb,
-			 const void *buf, unsigned long len,
+			 const void *buf, size_t len,
 			 enum object_type type,
 			 struct object_id *oid,
 			 const struct object_id *compat_oid_in,
diff --git a/odb.h b/odb.h
index 857637f980..a09748bdc2 100644
--- a/odb.h
+++ b/odb.h
@@ -634,14 +634,14 @@ enum odb_write_object_flags {
  * Returns 0 on success, a negative error code otherwise.
  */
 int odb_write_object_ext(struct object_database *odb,
-			 const void *buf, unsigned long len,
+			 const void *buf, size_t len,
 			 enum object_type type,
 			 struct object_id *oid,
 			 const struct object_id *compat_oid,
 			 enum odb_write_object_flags flags);
 
 static inline int odb_write_object(struct object_database *odb,
-				   const void *buf, unsigned long len,
+				   const void *buf, size_t len,
 				   enum object_type type,
 				   struct object_id *oid)
 {
diff --git a/odb/source-inmemory.c b/odb/source-inmemory.c
index 75cf29fa9d..c13cffc805 100644
--- a/odb/source-inmemory.c
+++ b/odb/source-inmemory.c
@@ -10,7 +10,7 @@
 struct inmemory_object {
 	enum object_type type;
 	const void *buf;
-	unsigned long size;
+	size_t size;
 };
 
 static const struct inmemory_object *find_cached_object(struct odb_source_inmemory *source,
diff --git a/odb/source-loose.c b/odb/source-loose.c
index 50cb388c35..60801e70e8 100644
--- a/odb/source-loose.c
+++ b/odb/source-loose.c
@@ -758,7 +758,7 @@ static int end_loose_object_common(struct odb_source_loose *loose,
 
 static int write_loose_object(struct odb_source_loose *loose,
 			      const struct object_id *oid, char *hdr,
-			      int hdrlen, const void *buf, unsigned long len,
+			      int hdrlen, const void *buf, size_t len,
 			      const time_t *mtime, unsigned flags)
 {
 	int fd, ret;
diff --git a/odb/source.h b/odb/source.h
index 7b56f51b82..8daf5df31c 100644
--- a/odb/source.h
+++ b/odb/source.h
@@ -454,7 +454,7 @@ static inline int odb_source_freshen_object(struct odb_source *source,
  * object ID and the compatibility object ID, if non-NULL.
  */
 static inline int odb_source_write_object(struct odb_source *source,
-					  const void *buf, unsigned long len,
+					  const void *buf, size_t len,
 					  enum object_type type,
 					  const struct object_id *oid,
 					  const struct object_id *compat_oid,
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* [PATCH 3/3] xdiff: drop LONG_MAX check on merge result size
  2026-11-17 17:43 [PATCH 0/3] Allow merges with results over 2 GiB on 64-bit Windows Christian Couder
  2026-11-17 17:43 ` [PATCH 1/3] notes-merge: use size_t in write_buf_to_worktree() Christian Couder
  2026-11-17 17:43 ` [PATCH 2/3] odb: widen odb_write_object() length to size_t Christian Couder
@ 2026-11-17 17:43 ` Christian Couder
  2 siblings, 0 replies; 4+ messages in thread
From: Christian Couder @ 2026-11-17 17:43 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Patrick Steinhardt, Kaartic Sivaraam, Jeff King,
	Johannes Schindelin, Christian Couder

Since 288266c11d (xdiff: use size_t for merge buffer size computations,
2026-10-16), a `size > LONG_MAX` check in xdl_do_merge() makes sure the
size isn't truncated on LLP64 platforms like 64-bit Windows.

This check was useful because that `size` was assigned to mmbuffer_t's
size which was a `long`. Since 81db32ddce (Merge branch
'jk/xdiff-size_t', 2026-10-15) though, `mmbuffer_t` has been replaced
with `mmfile_t`, which uses `size_t` for its `size` member.

That size was still passed to odb_write_object() and
write_buf_to_worktree() that accepted `unsigned long` arguments, but
previous commits have changed the types of those arguments to `size_t`.
So the `size > LONG_MAX` check in xdl_do_merge() is not preventing any
truncation anymore.

Let's then get rid of this `size > LONG_MAX` check and allow sizes as
big as `size_t` allows, so that merges with results over 2 GiB no
longer fail on 64-bit Windows.

This also allows us to clean up related tests in "t4300-merge-tree.sh".

Note that overflows in the size computation itself, for example when
`size_t` is 32 bits, are still caught by the st_add*() calls added by
b1313872ce (xdiff: check for overflow when computing merge buffer size,
2026-10-16).

Signed-off-by: Christian Couder <christian.couder@gmail.com>
---
 t/t4300-merge-tree.sh | 13 ++-----------
 xdiff/xmerge.c        |  4 ----
 2 files changed, 2 insertions(+), 15 deletions(-)

diff --git a/t/t4300-merge-tree.sh b/t/t4300-merge-tree.sh
index 41386d6025..54bf6c0327 100755
--- a/t/t4300-merge-tree.sh
+++ b/t/t4300-merge-tree.sh
@@ -430,15 +430,13 @@ test_expect_success EXPENSIVE 'setup: base commit with attributes' '
 	git commit -a -m side2
 '
 
-# We have LONG_IS_64BIT as a prerequisite since we guard against downcasts
-# from 64-bit size_t to 32-bit long (for example on Windows)
-test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,LONG_IS_64BIT 'no buffer overflow in merge-tree --write-tree' '
+test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'no buffer overflow in merge-tree --write-tree' '
 	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
 	tree=$(head -n 1 out) &&
 	git cat-file -e "$tree^{tree}"
 '
 
-test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,LONG_IS_64BIT 'conflicted paths are listed without buffer overflow' '
+test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'conflicted paths are listed without buffer overflow' '
 	test_expect_code 1 git merge-tree --write-tree --name-only \
 		side1 side2 >out &&
 	sed -n "2,\$p" out | sed "/^\$/,\$d" | sort -u >actual &&
@@ -449,13 +447,6 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,LONG_IS_64BIT 'conflicted paths ar
 	test_cmp expect actual
 '
 
-# Ensure we properly identify a downcast from 64-bit size_t to
-# 32-bit long on Windows
-test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT 'merge-tree dies when merge buffer size does not fit long' '
-	test_must_fail git merge-tree --write-tree side1 side2 2>err &&
-	test_grep "failure to merge" err
-'
-
 test_expect_success EXPENSIVE,!SIZE_T_IS_64BIT 'merge-tree dies when merge buffer size overflows size_t' '
 	test_must_fail git merge-tree --write-tree side1 side2 2>err &&
 	test_grep "size_t overflow" err
diff --git a/xdiff/xmerge.c b/xdiff/xmerge.c
index 5a78b88fad..2ef9c049e5 100644
--- a/xdiff/xmerge.c
+++ b/xdiff/xmerge.c
@@ -669,10 +669,6 @@ static int xdl_do_merge(xdfenv_t *xe1, xdchange_t *xscr1,
 						    ancestor_name,
 						    favor, changes, NULL, style,
 						    marker_size);
-		if (size > LONG_MAX) {
-			xdl_cleanup_merge(changes);
-			return -1;
-		}
 		result->ptr = xdl_malloc(size);
 		if (!result->ptr) {
 			xdl_cleanup_merge(changes);
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2026-11-17 17:44 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-17 17:43 [PATCH 0/3] Allow merges with results over 2 GiB on 64-bit Windows Christian Couder
2026-11-17 17:43 ` [PATCH 1/3] notes-merge: use size_t in write_buf_to_worktree() Christian Couder
2026-11-17 17:43 ` [PATCH 2/3] odb: widen odb_write_object() length to size_t Christian Couder
2026-11-17 17:43 ` [PATCH 3/3] xdiff: drop LONG_MAX check on merge result size Christian Couder

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox