* [PATCH 0/3] Allow merges with results over 2 GiB on 64-bit Windows
@ 2026-11-17 17:43 Christian Couder
2026-11-17 17:43 ` [PATCH 1/3] notes-merge: use size_t in write_buf_to_worktree() Christian Couder
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Christian Couder @ 2026-11-17 17:43 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Patrick Steinhardt, Kaartic Sivaraam, Jeff King,
Johannes Schindelin, Christian Couder
Two recent series improved how merge results are sized:
- `jk/xdiff-size_t` (merged by 81db32ddce) replaced `mmbuffer_t`
with `mmfile_t`, whose size is a `size_t` instead of a `long`.
- `kn/safer-mem-arithmetic-in-xdiff` (merged by 1c93cc03ea) computes
the merge buffer size as a `size_t` instead of an `int`, with
overflow checks.
As the latter targets the `maint` branches, which don't have
`jk/xdiff-size_t`, it also makes `xdl_do_merge()` fail when the result
is bigger than `LONG_MAX`, so that it cannot be silently truncated
when stored into mmbuffer_t's `long` size on LLP64 platforms like
64-bit Windows.
On `master`, that check is not needed for `mmfile_t` anymore, but as
Kaartic pointed out when that series was discussed, merge results were
still passed on to a few functions taking an `unsigned long`. This
series converts those to `size_t`, and then removes the check.
Patch 1/3 also fixes an existing bug: since c6a4629e32 (odb: use
size_t for object_info.sizep and the size APIs, 2026-06-15), a
delete/modify conflict in `git notes merge` on a note of 4 GiB or more
silently writes a truncated note on LLP64 platforms. That patch could
be applied on its own.
## Overview of the patches
- Patch 1/3 changes the size argument of `write_buf_to_worktree()`
in "notes-merge.c" from `unsigned long` to `size_t`.
- Patch 2/3 uses `size_t` for the length argument of
`odb_write_object()`, `odb_write_object_ext()`, the
`odb_source_write_object()` wrapper and `write_loose_object()`,
and for the size stored in `struct inmemory_object`, so that
object lengths are not truncated before reaching the already
`size_t` backends.
- Patch 3/3 removes the `size > LONG_MAX` check in `xdl_do_merge()`,
lets the `t4300` tests that expect a successful big merge run on
LLP64 platforms, and removes the test that expected a failure
there.
## Testing
The code paths changed here only behave differently where `long` is 32
bits and `size_t` is 64 bits, so Linux and macOS CI cannot exercise
them. It would be very nice if someone with a 64-bit Windows setup
could run `t4300` with `GIT_TEST_LONG=1`. Beware that it needs enough
memory and disk for a merge result just over 4 GiB.
## CI
Some test scripts didn't want to start on MacOS but otherwise all the
tests passed, see:
https://github.com/chriscool/git/actions/runs/44118577071
Christian Couder (3):
notes-merge: use size_t in write_buf_to_worktree()
odb: widen odb_write_object() length to size_t
xdiff: drop LONG_MAX check on merge result size
notes-merge.c | 2 +-
odb.c | 2 +-
odb.h | 4 ++--
odb/source-inmemory.c | 2 +-
odb/source-loose.c | 2 +-
odb/source.h | 2 +-
t/t4300-merge-tree.sh | 13 ++-----------
xdiff/xmerge.c | 4 ----
8 files changed, 9 insertions(+), 22 deletions(-)
base-commit: e1ec09b23c8e0deb21a0b95ac5996422277395d4
--
2.56.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/3] notes-merge: use size_t in write_buf_to_worktree()
2026-11-17 17:43 [PATCH 0/3] Allow merges with results over 2 GiB on 64-bit Windows Christian Couder
@ 2026-11-17 17:43 ` Christian Couder
2026-11-18 8:25 ` Patrick Steinhardt
2026-11-17 17:43 ` [PATCH 2/3] odb: widen odb_write_object() length to size_t Christian Couder
2026-11-17 17:43 ` [PATCH 3/3] xdiff: drop LONG_MAX check on merge result size Christian Couder
2 siblings, 1 reply; 6+ messages in thread
From: Christian Couder @ 2026-11-17 17:43 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Patrick Steinhardt, Kaartic Sivaraam, Jeff King,
Johannes Schindelin, Christian Couder
The write_buf_to_worktree() function in "notes-merge.c" has an
`unsigned long` size argument, while its two callers from that same
file:
1. write_note_to_worktree(), and
2. ll_merge_in_worktree(),
both pass a `size_t` to that size argument.
Since c6a4629e32 (odb: use size_t for object_info.sizep and the size
APIs, 2026-06-15), the first caller reads the note into a `size_t` size
before passing it on to write_buf_to_worktree(). On LLP64 platforms
like 64-bit Windows, where `unsigned long` is 32 bits, a delete/modify
conflict on a note of 4 GiB or more therefore silently writes a
truncated note into NOTES_MERGE_WORKTREE.
The second caller passes the size of the ll_merge() result, which is a
`size_t` since 81db32ddce (Merge branch 'jk/xdiff-size_t', 2026-10-15).
That size cannot be truncated as since 1c93cc03ea (Merge branch
'kn/safer-mem-arithmetic-in-xdiff', 2026-10-16), xdl_do_merge() errors
out if it is greater than LONG_MAX.
But we want to get rid of that `size > LONG_MAX` check in a following
commit.
To fix the truncation issue with the first caller and to prepare for
getting rid of the check, let's convert the size argument in
write_buf_to_worktree() to a `size_t`.
Signed-off-by: Christian Couder <christian.couder@gmail.com>
---
notes-merge.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/notes-merge.c b/notes-merge.c
index d361e70a48..4b010bcfc0 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -310,7 +310,7 @@ static void check_notes_merge_worktree(struct notes_merge_options *o)
}
static void write_buf_to_worktree(const struct object_id *obj,
- const char *buf, unsigned long size)
+ const char *buf, size_t size)
{
int fd;
char *path = repo_git_path(the_repository, NOTES_MERGE_WORKTREE "/%s", oid_to_hex(obj));
--
2.56.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/3] odb: widen odb_write_object() length to size_t
2026-11-17 17:43 [PATCH 0/3] Allow merges with results over 2 GiB on 64-bit Windows Christian Couder
2026-11-17 17:43 ` [PATCH 1/3] notes-merge: use size_t in write_buf_to_worktree() Christian Couder
@ 2026-11-17 17:43 ` Christian Couder
2026-11-18 8:25 ` Patrick Steinhardt
2026-11-17 17:43 ` [PATCH 3/3] xdiff: drop LONG_MAX check on merge result size Christian Couder
2 siblings, 1 reply; 6+ messages in thread
From: Christian Couder @ 2026-11-17 17:43 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Patrick Steinhardt, Kaartic Sivaraam, Jeff King,
Johannes Schindelin, Christian Couder
odb_write_object() and odb_write_object_ext() use an `unsigned long`
for their object length argument, while almost everything they pass it
to already uses `size_t`: hash_object_file(), format_object_header(),
convert_object_file(), the `write_object` callback of `struct
odb_source`, and `git_zstream`.
Many callers also pass a `size_t`, often a `strbuf.len`. On LLP64
platforms like 64-bit Windows, where `unsigned long` is 32 bits, such
lengths may silently be truncated.
Since 81db32ddce (Merge branch 'jk/xdiff-size_t', 2026-10-15), the size
of an ll_merge() result is also passed as a `size_t` to
odb_write_object(). It cannot be truncated as since 1c93cc03ea (Merge
branch 'kn/safer-mem-arithmetic-in-xdiff', 2026-10-16), that size
argument cannot be greater than LONG_MAX because xdl_do_merge() errors
out if `size > LONG_MAX`.
But we want to get rid of that `size > LONG_MAX` check in a following
commit.
To fix the possible truncation issues with many callers and to prepare
for getting rid of the check, let's use `size_t` for:
- the length argument of odb_write_object(), odb_write_object_ext(),
the odb_source_write_object() wrapper, and write_loose_object(),
- the size stored in `struct inmemory_object`, which is later
copied back into `size_t` fields (`object_info.sizep` and
`odb_stream.size`).
Signed-off-by: Christian Couder <christian.couder@gmail.com>
---
odb.c | 2 +-
odb.h | 4 ++--
odb/source-inmemory.c | 2 +-
odb/source-loose.c | 2 +-
odb/source.h | 2 +-
5 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/odb.c b/odb.c
index de5be285d3..56a7910b03 100644
--- a/odb.c
+++ b/odb.c
@@ -702,7 +702,7 @@ void odb_assert_oid_type(struct object_database *odb,
}
int odb_write_object_ext(struct object_database *odb,
- const void *buf, unsigned long len,
+ const void *buf, size_t len,
enum object_type type,
struct object_id *oid,
const struct object_id *compat_oid_in,
diff --git a/odb.h b/odb.h
index 857637f980..a09748bdc2 100644
--- a/odb.h
+++ b/odb.h
@@ -634,14 +634,14 @@ enum odb_write_object_flags {
* Returns 0 on success, a negative error code otherwise.
*/
int odb_write_object_ext(struct object_database *odb,
- const void *buf, unsigned long len,
+ const void *buf, size_t len,
enum object_type type,
struct object_id *oid,
const struct object_id *compat_oid,
enum odb_write_object_flags flags);
static inline int odb_write_object(struct object_database *odb,
- const void *buf, unsigned long len,
+ const void *buf, size_t len,
enum object_type type,
struct object_id *oid)
{
diff --git a/odb/source-inmemory.c b/odb/source-inmemory.c
index 75cf29fa9d..c13cffc805 100644
--- a/odb/source-inmemory.c
+++ b/odb/source-inmemory.c
@@ -10,7 +10,7 @@
struct inmemory_object {
enum object_type type;
const void *buf;
- unsigned long size;
+ size_t size;
};
static const struct inmemory_object *find_cached_object(struct odb_source_inmemory *source,
diff --git a/odb/source-loose.c b/odb/source-loose.c
index 50cb388c35..60801e70e8 100644
--- a/odb/source-loose.c
+++ b/odb/source-loose.c
@@ -758,7 +758,7 @@ static int end_loose_object_common(struct odb_source_loose *loose,
static int write_loose_object(struct odb_source_loose *loose,
const struct object_id *oid, char *hdr,
- int hdrlen, const void *buf, unsigned long len,
+ int hdrlen, const void *buf, size_t len,
const time_t *mtime, unsigned flags)
{
int fd, ret;
diff --git a/odb/source.h b/odb/source.h
index 7b56f51b82..8daf5df31c 100644
--- a/odb/source.h
+++ b/odb/source.h
@@ -454,7 +454,7 @@ static inline int odb_source_freshen_object(struct odb_source *source,
* object ID and the compatibility object ID, if non-NULL.
*/
static inline int odb_source_write_object(struct odb_source *source,
- const void *buf, unsigned long len,
+ const void *buf, size_t len,
enum object_type type,
const struct object_id *oid,
const struct object_id *compat_oid,
--
2.56.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 3/3] xdiff: drop LONG_MAX check on merge result size
2026-11-17 17:43 [PATCH 0/3] Allow merges with results over 2 GiB on 64-bit Windows Christian Couder
2026-11-17 17:43 ` [PATCH 1/3] notes-merge: use size_t in write_buf_to_worktree() Christian Couder
2026-11-17 17:43 ` [PATCH 2/3] odb: widen odb_write_object() length to size_t Christian Couder
@ 2026-11-17 17:43 ` Christian Couder
2 siblings, 0 replies; 6+ messages in thread
From: Christian Couder @ 2026-11-17 17:43 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Patrick Steinhardt, Kaartic Sivaraam, Jeff King,
Johannes Schindelin, Christian Couder
Since 288266c11d (xdiff: use size_t for merge buffer size computations,
2026-10-16), a `size > LONG_MAX` check in xdl_do_merge() makes sure the
size isn't truncated on LLP64 platforms like 64-bit Windows.
This check was useful because that `size` was assigned to mmbuffer_t's
size which was a `long`. Since 81db32ddce (Merge branch
'jk/xdiff-size_t', 2026-10-15) though, `mmbuffer_t` has been replaced
with `mmfile_t`, which uses `size_t` for its `size` member.
That size was still passed to odb_write_object() and
write_buf_to_worktree() that accepted `unsigned long` arguments, but
previous commits have changed the types of those arguments to `size_t`.
So the `size > LONG_MAX` check in xdl_do_merge() is not preventing any
truncation anymore.
Let's then get rid of this `size > LONG_MAX` check and allow sizes as
big as `size_t` allows, so that merges with results over 2 GiB no
longer fail on 64-bit Windows.
This also allows us to clean up related tests in "t4300-merge-tree.sh".
Note that overflows in the size computation itself, for example when
`size_t` is 32 bits, are still caught by the st_add*() calls added by
b1313872ce (xdiff: check for overflow when computing merge buffer size,
2026-10-16).
Signed-off-by: Christian Couder <christian.couder@gmail.com>
---
t/t4300-merge-tree.sh | 13 ++-----------
xdiff/xmerge.c | 4 ----
2 files changed, 2 insertions(+), 15 deletions(-)
diff --git a/t/t4300-merge-tree.sh b/t/t4300-merge-tree.sh
index 41386d6025..54bf6c0327 100755
--- a/t/t4300-merge-tree.sh
+++ b/t/t4300-merge-tree.sh
@@ -430,15 +430,13 @@ test_expect_success EXPENSIVE 'setup: base commit with attributes' '
git commit -a -m side2
'
-# We have LONG_IS_64BIT as a prerequisite since we guard against downcasts
-# from 64-bit size_t to 32-bit long (for example on Windows)
-test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,LONG_IS_64BIT 'no buffer overflow in merge-tree --write-tree' '
+test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'no buffer overflow in merge-tree --write-tree' '
test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
tree=$(head -n 1 out) &&
git cat-file -e "$tree^{tree}"
'
-test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,LONG_IS_64BIT 'conflicted paths are listed without buffer overflow' '
+test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'conflicted paths are listed without buffer overflow' '
test_expect_code 1 git merge-tree --write-tree --name-only \
side1 side2 >out &&
sed -n "2,\$p" out | sed "/^\$/,\$d" | sort -u >actual &&
@@ -449,13 +447,6 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,LONG_IS_64BIT 'conflicted paths ar
test_cmp expect actual
'
-# Ensure we properly identify a downcast from 64-bit size_t to
-# 32-bit long on Windows
-test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT 'merge-tree dies when merge buffer size does not fit long' '
- test_must_fail git merge-tree --write-tree side1 side2 2>err &&
- test_grep "failure to merge" err
-'
-
test_expect_success EXPENSIVE,!SIZE_T_IS_64BIT 'merge-tree dies when merge buffer size overflows size_t' '
test_must_fail git merge-tree --write-tree side1 side2 2>err &&
test_grep "size_t overflow" err
diff --git a/xdiff/xmerge.c b/xdiff/xmerge.c
index 5a78b88fad..2ef9c049e5 100644
--- a/xdiff/xmerge.c
+++ b/xdiff/xmerge.c
@@ -669,10 +669,6 @@ static int xdl_do_merge(xdfenv_t *xe1, xdchange_t *xscr1,
ancestor_name,
favor, changes, NULL, style,
marker_size);
- if (size > LONG_MAX) {
- xdl_cleanup_merge(changes);
- return -1;
- }
result->ptr = xdl_malloc(size);
if (!result->ptr) {
xdl_cleanup_merge(changes);
--
2.56.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 1/3] notes-merge: use size_t in write_buf_to_worktree()
2026-11-17 17:43 ` [PATCH 1/3] notes-merge: use size_t in write_buf_to_worktree() Christian Couder
@ 2026-11-18 8:25 ` Patrick Steinhardt
0 siblings, 0 replies; 6+ messages in thread
From: Patrick Steinhardt @ 2026-11-18 8:25 UTC (permalink / raw)
To: Christian Couder
Cc: git, Junio C Hamano, Kaartic Sivaraam, Jeff King,
Johannes Schindelin
On Tue, Nov 17, 2026 at 06:43:55PM +0100, Christian Couder wrote:
> The write_buf_to_worktree() function in "notes-merge.c" has an
> `unsigned long` size argument, while its two callers from that same
> file:
>
> 1. write_note_to_worktree(), and
> 2. ll_merge_in_worktree(),
>
> both pass a `size_t` to that size argument.
>
> Since c6a4629e32 (odb: use size_t for object_info.sizep and the size
> APIs, 2026-06-15), the first caller reads the note into a `size_t` size
> before passing it on to write_buf_to_worktree(). On LLP64 platforms
> like 64-bit Windows, where `unsigned long` is 32 bits, a delete/modify
> conflict on a note of 4 GiB or more therefore silently writes a
> truncated note into NOTES_MERGE_WORKTREE.
Heh, this feels like nobody would ever hit such an issue. But fixing it
is the right thing regardless of that, of course.
[snip]
> To fix the truncation issue with the first caller and to prepare for
> getting rid of the check, let's convert the size argument in
> write_buf_to_worktree() to a `size_t`.
Makes sense. I've been lamenting about that before, but I really wish
that C wasn't so painful to work with in this area. All these implicit
casts are just so error-prone.
> diff --git a/notes-merge.c b/notes-merge.c
> index d361e70a48..4b010bcfc0 100644
> --- a/notes-merge.c
> +++ b/notes-merge.c
> @@ -310,7 +310,7 @@ static void check_notes_merge_worktree(struct notes_merge_options *o)
> }
>
> static void write_buf_to_worktree(const struct object_id *obj,
> - const char *buf, unsigned long size)
> + const char *buf, size_t size)
> {
> int fd;
> char *path = repo_git_path(the_repository, NOTES_MERGE_WORKTREE "/%s", oid_to_hex(obj));
Looks obviously correct to me. We pass the value to `write_in_full()`,
which already accepts a `size_t` as input parameter.
Patrick
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 2/3] odb: widen odb_write_object() length to size_t
2026-11-17 17:43 ` [PATCH 2/3] odb: widen odb_write_object() length to size_t Christian Couder
@ 2026-11-18 8:25 ` Patrick Steinhardt
0 siblings, 0 replies; 6+ messages in thread
From: Patrick Steinhardt @ 2026-11-18 8:25 UTC (permalink / raw)
To: Christian Couder
Cc: git, Junio C Hamano, Kaartic Sivaraam, Jeff King,
Johannes Schindelin
On Tue, Nov 17, 2026 at 06:43:56PM +0100, Christian Couder wrote:
> odb_write_object() and odb_write_object_ext() use an `unsigned long`
> for their object length argument, while almost everything they pass it
> to already uses `size_t`: hash_object_file(), format_object_header(),
> convert_object_file(), the `write_object` callback of `struct
> odb_source`, and `git_zstream`.
Huh, I thought that we already had patches recently that converted this
function to use `size_t`... oh, yes, this is overlapping with Elia's
patch series [1]. I think it would make sense to rebase your patch
series on top of his patches.
[1]: <20261021101640.329990-1-gitter.spiros@gmail.com>
[snip]
> To fix the possible truncation issues with many callers and to prepare
> for getting rid of the check, let's use `size_t` for:
>
> - the length argument of odb_write_object(), odb_write_object_ext(),
> the odb_source_write_object() wrapper, and write_loose_object(),
>
> - the size stored in `struct inmemory_object`, which is later
> copied back into `size_t` fields (`object_info.sizep` and
> `odb_stream.size`).
Yeah, makes sense.
> diff --git a/odb.c b/odb.c
> index de5be285d3..56a7910b03 100644
> --- a/odb.c
> +++ b/odb.c
> @@ -702,7 +702,7 @@ void odb_assert_oid_type(struct object_database *odb,
> }
>
> int odb_write_object_ext(struct object_database *odb,
> - const void *buf, unsigned long len,
> + const void *buf, size_t len,
> enum object_type type,
> struct object_id *oid,
> const struct object_id *compat_oid_in,
Right. All functions that we pass `len` to already accept a `size_t`,
except for `odb_source_write_object()`. But you also adapt that function
at the same time, so we're good.
> diff --git a/odb/source.h b/odb/source.h
> index 7b56f51b82..8daf5df31c 100644
> --- a/odb/source.h
> +++ b/odb/source.h
> @@ -454,7 +454,7 @@ static inline int odb_source_freshen_object(struct odb_source *source,
> * object ID and the compatibility object ID, if non-NULL.
> */
> static inline int odb_source_write_object(struct odb_source *source,
> - const void *buf, unsigned long len,
> + const void *buf, size_t len,
> enum object_type type,
> const struct object_id *oid,
> const struct object_id *compat_oid,
And the callback functions fo the backend already use a `size_t`, so
those don't need to be changed.
Patrick
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-11-18 8:25 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-17 17:43 [PATCH 0/3] Allow merges with results over 2 GiB on 64-bit Windows Christian Couder
2026-11-17 17:43 ` [PATCH 1/3] notes-merge: use size_t in write_buf_to_worktree() Christian Couder
2026-11-18 8:25 ` Patrick Steinhardt
2026-11-17 17:43 ` [PATCH 2/3] odb: widen odb_write_object() length to size_t Christian Couder
2026-11-18 8:25 ` Patrick Steinhardt
2026-11-17 17:43 ` [PATCH 3/3] xdiff: drop LONG_MAX check on merge result size Christian Couder
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox