All of lore.kernel.org
 help / color / mirror / Atom feed
From: Taylor Blau <me@ttaylorr.com>
To: Phillip Wood <phillip.wood@dunelm.org.uk>
Cc: git@vger.kernel.org, Derrick Stolee <stolee@gmail.com>,
	Phillip Wood <phillip.wood123@gmail.com>
Subject: Re: [PATCH 2/4] midx repack: avoid potential integer overflow on 64 bit systems
Date: Tue, 20 May 2025 13:59:36 -0400	[thread overview]
Message-ID: <aCzDCEK7OhCYyAZa@nand.local> (raw)
In-Reply-To: <9f07da4fe71d21b14226d8f0132cd3c8600fba13.1747753388.git.phillip.wood@dunelm.org.uk>

On Tue, May 20, 2025 at 04:04:25PM +0100, Phillip Wood wrote:
> From: Phillip Wood <phillip.wood@dunelm.org.uk>
>
> On a 64 bit system the calculation
>
>     p->pack_size * pack_info[i].referenced_objects
>
> could overflow. If a pack file contains 2^28 objects with an average
> compressed size of 1KB then the pack size will be 2^38B. If all of the
> objects are referenced by the multi-pack index the sum above will
> overflow. Avoid this by using shifted integer arithmetic and changing
> the order of the calculation so that the pack size is divided by the
> total number of objects in the pack before multiplying by the number of
> objects referenced by the multi-pack index. Using a shift of 14 bits
> should give reasonable accuracy while avoiding overflow for pack sizes
> less that 1PB.

Ahhh, this renders some of comments on the previous patch moot. I think
that this is a not-unreasonable concern to be addressing even on modern
64-bit systems, since I have definitely encountered packs that have on
the order of ~2^28 objects in them.

I like this approach quite a bit, thanks!

Thanks,
Taylor

  reply	other threads:[~2025-05-20 17:59 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-05-20 15:04 [PATCH 0/4] midx repack: fix overflow on 32 bit systems Phillip Wood
2025-05-20 15:04 ` [PATCH 1/4] midx repack: avoid integer " Phillip Wood
2025-05-20 17:54   ` Taylor Blau
2025-05-21 15:19     ` Phillip Wood
2025-05-23  0:34       ` Taylor Blau
2025-05-21 13:10   ` D. Ben Knoble
2025-05-21 15:01     ` Junio C Hamano
2025-05-21 15:20     ` Phillip Wood
2025-05-20 15:04 ` [PATCH 2/4] midx repack: avoid potential integer overflow on 64 " Phillip Wood
2025-05-20 17:59   ` Taylor Blau [this message]
2025-05-21 15:20     ` Phillip Wood
2025-05-20 15:04 ` [PATCH 3/4] midx: avoid negative array index Phillip Wood
2025-05-20 17:58   ` Taylor Blau
2025-05-20 15:04 ` [PATCH 4/4] midx docs: clarify tie breaking Phillip Wood
2025-05-20 18:07   ` Taylor Blau
2025-05-21 15:20     ` Phillip Wood
2025-05-21 13:14   ` D. Ben Knoble
2025-05-22 15:55 ` [PATCH v2 0/4] midx repack: fix overflow on 32 bit systems Phillip Wood
2025-05-22 15:55   ` [PATCH v2 1/4] midx repack: avoid integer " Phillip Wood
2025-05-22 15:55   ` [PATCH v2 2/4] midx repack: avoid potential integer overflow on 64 " Phillip Wood
2025-05-22 15:55   ` [PATCH v2 3/4] midx: avoid negative array index Phillip Wood
2025-05-22 15:55   ` [PATCH v2 4/4] midx docs: clarify tie breaking Phillip Wood
2025-05-23  0:36   ` [PATCH v2 0/4] midx repack: fix overflow on 32 bit systems Taylor Blau
2025-05-27  8:26     ` Phillip Wood
2025-05-27 15:42       ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aCzDCEK7OhCYyAZa@nand.local \
    --to=me@ttaylorr.com \
    --cc=git@vger.kernel.org \
    --cc=phillip.wood123@gmail.com \
    --cc=phillip.wood@dunelm.org.uk \
    --cc=stolee@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.