From: Junio C Hamano <gitster@pobox.com>
To: Elia Pinto <gitter.spiros@gmail.com>
Cc: git@vger.kernel.org,
Johannes Schindelin <Johannes.Schindelin@gmx.de>,
"brian m. carlson" <sandals@crustytoothpaste.net>,
Neeraj Singh <neerajsi@microsoft.com>,
Patrick Steinhardt <ps@pks.im>
Subject: Re: [PATCH v2 1/4] csum-file: use size_t for the count passed to hashwrite()
Date: Tue, 17 Nov 2026 09:52:57 -0800 [thread overview]
Message-ID: <xmqqeccjfipy.fsf@gitster.g> (raw)
In-Reply-To: <07baad46b153b255b6e70f2597f1bf41b9c48dfd.1794909894.git.gitter.spiros@gmail.com> (Elia Pinto's message of "Tue, 17 Nov 2026 11:37:53 +0100")
Elia Pinto <gitter.spiros@gmail.com> writes:
> hashwrite() takes the byte count as "uint32_t" on all platforms. When
> pack-objects writes the entire compressed buffer of an object in a
> single call, which can exceed 4GiB if the object is in memory, poorly
> compressible and core.bigFileThreshold is raised accordingly, it writes
> only "count mod 2^32" bytes, corrupting the pack.
> ...
> diff --git a/csum-file.h b/csum-file.h
> index 6ed74d1637..f1c8a9df13 100644
> --- a/csum-file.h
> +++ b/csum-file.h
> @@ -75,7 +75,7 @@ void free_hashfile(struct hashfile *f);
> * Finalize the hashfile by flushing data to disk and free'ing it.
> */
> int finalize_hashfile(struct hashfile *, unsigned char *, enum fsync_component, unsigned int);
> -void hashwrite(struct hashfile *, const void *, uint32_t);
> +void hashwrite(struct hashfile *, const void *, size_t);
> void hashflush(struct hashfile *f);
> void crc32_begin(struct hashfile *);
> uint32_t crc32_end(struct hashfile *);
src/csum_file.rs still thinks that hashwrite() takes u32, though.
If a caller of hashwrite() has a contiguous region of memory that
exceeds 4GiB in length, shouldn't it be looping to call hashwrite()
in 4GiB (or less) chunks, without changing the API contract, I would
have to wonder.
pub mod c {
use std::os::raw::{c_char, c_int, c_void};
extern "C" {
pub fn hashfd(algop: *const c_void, fd: i32, name: *const c_char) -> *mut c_void;
pub fn hashwrite(f: *mut c_void, data: *const c_void, len: u32);
pub fn hashflush(f: *mut c_void);
pub fn finalize_hashfile(
f: *mut c_void,
data: *mut u8,
component: u32,
flags: u32,
) -> c_int;
}
}
next prev parent reply other threads:[~2026-11-17 17:52 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-01 15:36 [PATCH 0/3] use size_t for a few sizes that are 32 bits everywhere Elia Pinto
2026-11-01 15:36 ` [PATCH 1/3] csum-file: use size_t for the count passed to hashwrite() Elia Pinto
2026-11-01 15:36 ` [PATCH 2/3] diff: use size_t for the size passed to count_lines() Elia Pinto
2026-11-04 13:58 ` Phillip Wood
2026-11-04 15:45 ` Phillip Wood
2026-11-01 15:36 ` [PATCH 3/3] diffcore-delta: use size_t for the size of the buffer to hash Elia Pinto
2026-11-17 10:37 ` [PATCH v2 0/4] widen a few sizes and counts that are 32 bits everywhere Elia Pinto
2026-11-17 10:37 ` [PATCH v2 1/4] csum-file: use size_t for the count passed to hashwrite() Elia Pinto
2026-11-17 17:52 ` Junio C Hamano [this message]
2026-11-17 23:50 ` brian m. carlson
2026-11-17 10:37 ` [PATCH v2 2/4] diff: use size_t for sizes and line counts of complete rewrites Elia Pinto
2026-11-17 10:37 ` [PATCH v2 3/4] diffcore-delta: use size_t for the size of the buffer to hash Elia Pinto
2026-11-17 23:21 ` Junio C Hamano
2026-11-17 10:37 ` [PATCH v2 4/4] diff: use uintmax_t for diffstat totals and graph scaling Elia Pinto
2026-11-17 16:36 ` Torsten Bögershausen
2026-11-17 23:13 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqeccjfipy.fsf@gitster.g \
--to=gitster@pobox.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=git@vger.kernel.org \
--cc=gitter.spiros@gmail.com \
--cc=neerajsi@microsoft.com \
--cc=ps@pks.im \
--cc=sandals@crustytoothpaste.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.