From: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
michelbach94@gmail.com,
"Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
Subject: [PATCH v2 6/7] pack-objects: do not truncate result in-pack object size on 32-bit systems
Date: Wed, 13 Jul 2016 17:44:03 +0200 [thread overview]
Message-ID: <20160713154404.22909-7-pclouds@gmail.com> (raw)
In-Reply-To: <20160713154404.22909-1-pclouds@gmail.com>
A typical diff will not show what's going on and you need to see full
functions. The core code is like this, at the end of of write_one()
e->idx.offset = *offset;
size = write_object(f, e, *offset);
if (!size) {
e->idx.offset = recursing;
return WRITE_ONE_BREAK;
}
written_list[nr_written++] = &e->idx;
/* make sure off_t is sufficiently large not to wrap */
if (signed_add_overflows(*offset, size))
die("pack too large for current definition of off_t");
*offset += size;
Here we can see that the in-pack object size is returned by
write_object (or indirectly by write_reuse_object). And it's used to
calculate object offsets, which end up in the pack index file,
generated at the end.
If "size" overflows (on 32-bit sytems, unsigned long is 32-bit while
off_t can be 64-bit), we got wrong offsets and produce incorrect .idx
file, which may make it look like the .pack file is corrupted.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
builtin/pack-objects.c | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index a3a98c5..ac7a3a5 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -341,8 +341,8 @@ static unsigned long write_no_reuse_object(struct sha1file *f, struct object_ent
}
/* Return 0 if we will bust the pack-size limit */
-static unsigned long write_reuse_object(struct sha1file *f, struct object_entry *entry,
- unsigned long limit, int usable_delta)
+static off_t write_reuse_object(struct sha1file *f, struct object_entry *entry,
+ unsigned long limit, int usable_delta)
{
struct packed_git *p = entry->in_pack;
struct pack_window *w_curs = NULL;
@@ -415,11 +415,12 @@ static unsigned long write_reuse_object(struct sha1file *f, struct object_entry
}
/* Return 0 if we will bust the pack-size limit */
-static unsigned long write_object(struct sha1file *f,
- struct object_entry *entry,
- off_t write_offset)
+static off_t write_object(struct sha1file *f,
+ struct object_entry *entry,
+ off_t write_offset)
{
- unsigned long limit, len;
+ unsigned long limit;
+ off_t len;
int usable_delta, to_reuse;
if (!pack_to_stdout)
@@ -491,7 +492,7 @@ static enum write_one_status write_one(struct sha1file *f,
struct object_entry *e,
off_t *offset)
{
- unsigned long size;
+ off_t size;
int recursing;
/*
--
2.9.1.564.gb2f7278
next prev parent reply other threads:[~2016-07-13 15:45 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-24 22:38 [bug] Reliably Reproducible Bad Packing of Objects Christoph Michelbach
2016-07-02 9:10 ` Duy Nguyen
2016-07-02 14:35 ` Duy Nguyen
2016-07-05 17:05 ` [PATCH 0/5] Number truncation with 4+ GB files on 32-bit systems Nguyễn Thái Ngọc Duy
2016-07-05 17:05 ` [PATCH 1/5] pack-objects: pass length to check_pack_crc() without truncation Nguyễn Thái Ngọc Duy
2016-07-12 17:16 ` Junio C Hamano
2016-07-05 17:05 ` [PATCH 2/5] sha1_file.c: use type off_t* for object_info->disk_sizep Nguyễn Thái Ngọc Duy
2016-07-12 17:20 ` Junio C Hamano
2016-07-12 19:26 ` Junio C Hamano
2016-07-05 17:05 ` [PATCH 3/5] index-pack: correct "len" type in unpack_data() Nguyễn Thái Ngọc Duy
2016-07-05 20:25 ` Johannes Sixt
2016-07-06 15:25 ` Duy Nguyen
2016-07-06 16:04 ` Junio C Hamano
2016-07-06 16:08 ` Junio C Hamano
2016-07-05 17:05 ` [PATCH 4/5] index-pack: report correct bad object offsets even if they are large Nguyễn Thái Ngọc Duy
2016-07-12 17:24 ` Junio C Hamano
2016-07-12 19:27 ` Junio C Hamano
2016-07-05 17:05 ` [PATCH 5/5] index-pack: correct "offset" type in unpack_entry_data() Nguyễn Thái Ngọc Duy
2016-07-05 18:11 ` [PATCH 0/5] Number truncation with 4+ GB files on 32-bit systems Christoph Michelbach
[not found] ` <1467756891.4798.1.camel@gmail.com>
[not found] ` <CACsJy8BDQbanGsf=3z3K-OuH0++EuqQFEB22udXJT+WZnFKSBg@mail.gmail.com>
2016-07-06 18:02 ` Christoph Michelbach
2016-07-06 18:54 ` Duy Nguyen
2016-07-10 10:41 ` Duy Nguyen
2016-07-10 10:42 ` [PATCH 6/5] pack-objects: do not truncate result in-pack object size " Nguyễn Thái Ngọc Duy
2016-07-10 10:45 ` [PATCH 7/5] fsck: use streaming interface for large blobs in pack Nguyễn Thái Ngọc Duy
2016-07-12 18:39 ` Junio C Hamano
2016-07-12 19:06 ` Junio C Hamano
2016-07-12 17:07 ` [PATCH 0/5] Number truncation with 4+ GB files on 32-bit systems Junio C Hamano
2016-07-12 18:48 ` Junio C Hamano
2016-07-12 20:38 ` Junio C Hamano
2016-07-13 6:01 ` Duy Nguyen
2016-07-13 15:43 ` [PATCH v2 0/7] " Nguyễn Thái Ngọc Duy
2016-07-13 15:43 ` [PATCH v2 1/7] pack-objects: pass length to check_pack_crc() without truncation Nguyễn Thái Ngọc Duy
2016-07-13 15:43 ` [PATCH v2 2/7] sha1_file.c: use type off_t* for object_info->disk_sizep Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` [PATCH v2 3/7] index-pack: correct "len" type in unpack_data() Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` [PATCH v2 4/7] index-pack: report correct bad object offsets even if they are large Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` [PATCH v2 5/7] index-pack: correct "offset" type in unpack_entry_data() Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` Nguyễn Thái Ngọc Duy [this message]
2016-07-13 15:44 ` [PATCH v2 7/7] fsck: use streaming interface for large blobs in pack Nguyễn Thái Ngọc Duy
2016-07-13 16:16 ` [PATCH v2 0/7] Number truncation with 4+ GB files on 32-bit systems Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160713154404.22909-7-pclouds@gmail.com \
--to=pclouds@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=michelbach94@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.