Git development
 help / color / mirror / Atom feed
From: Taylor Blau <me@ttaylorr.com>
To: Derrick Stolee <stolee@gmail.com>
Cc: git@vger.kernel.org, Junio C Hamano <gitster@pobox.com>,
	Jeff King <peff@peff.net>, Elijah Newren <newren@gmail.com>
Subject: Re: [PATCH v2 2/4] pack-objects: support reachability bitmaps with `--path-walk`
Date: Fri, 19 Jun 2026 10:16:04 -0400	[thread overview]
Message-ID: <ajVPJGXuhugDcT+A@nand.local> (raw)
In-Reply-To: <6e4a8764-3c56-42c8-a87e-40a94c6c34e9@gmail.com>

On Fri, Jun 12, 2026 at 09:03:41AM -0400, Derrick Stolee wrote:
> On 6/2/2026 6:21 PM, Taylor Blau wrote:
>
> > As a result, we can see significantly reduced pack sizes from p5311
> > before this commit:
>
> I mentioned this before, but the pack _sizes_ aren't changing in this
> example. We are computing them more quickly, though.

Thanks for pointing this out. The paragraph following the perf output
below correctly explains the results ("We get the same size of output
pack, but [...]"), but this one is obviously wrong.

> Since we are testing --path-walk on both sides, the change across this
> commit is that we are using the bitmaps for the "counting objects" phase
> and then potentially using the --path-walk algorithm to construct the
> packfile.

I'm not sure I agree here. Because we are using bitmaps, we're relying
on pack-reuse to construct the output pack, not --path-walk. I mentioned
in git-pack-objects(1), but the combination of seeing "--path-walk" and
"--use-bitmap-index" together only means that we will use a path-walk
traversal as fallback if we can't get an answer by relying on bitmaps.

> And I wonder if the test setup creates a situation where we are always
> reusing deltas from the underlying packfile, so the --path-walk algorithm
> isn't doing anything to help with delta compression at this point and the
> difference in this patch is that we are replacing the object reachability
> calculation entirely with bitmaps.
>
> I suppose what I'm really worried about is that I'm hoping to see some
> evidence from a large-scale test that demonstrates that the two algorithms
> are working in tandem in a non-trivial way. I haven't seen it yet, but I
> also don't have evidence that they _aren't_ working together.

Your thinking is correct here that the test setup intentionally creates
a situation where we are reusing objects/deltas verbatim from the
bitmapped pack.

I'm not sure what "working in tandem" means here. At read time, the two
options mutually exclude one another, meaning we'll use bitmaps if we
have them, or do a path-walk traversal otherwise (or if the bitmaps we
have are somehow insufficient to perform the traversal).

The goal of this patch is not to demonstrate that the two work together
at the same time, but rather that we can write a pack using --path-walk,
and generate reachability bitmaps simultaneously.

Let me know if you have more thoughts on what "working together in a
non-trivial" way would look like here. If there are ways to improve the
compatibility of these two features in a way that yields better
performance via either smaller packs, faster generation, or both, I'm
all ears :-).

Thanks,
Taylor

  reply	other threads:[~2026-06-19 14:16 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-05-27 23:18 [PATCH 0/3] pack-objects: support bitmaps and delta-islands with `--path-walk` Taylor Blau
2026-05-27 23:18 ` [PATCH 1/3] pack-objects: support reachability bitmaps " Taylor Blau
2026-05-27 23:18 ` [PATCH 2/3] pack-objects: extract `record_tree_depth()` helper Taylor Blau
2026-05-27 23:18 ` [PATCH 3/3] pack-objects: support `--delta-islands` with `--path-walk` Taylor Blau
2026-05-28 15:28 ` [PATCH 0/3] pack-objects: support bitmaps and delta-islands " Derrick Stolee
2026-05-29 17:26   ` Derrick Stolee
2026-05-29 20:07     ` Taylor Blau
2026-05-29 21:28       ` Derrick Stolee
2026-05-29 22:20         ` Taylor Blau
2026-06-02 22:21 ` [PATCH v2 0/4] " Taylor Blau
2026-06-02 22:21   ` [PATCH v2 1/4] t/perf: drop p5311's lookup-table permutation Taylor Blau
2026-06-02 22:21   ` [PATCH v2 2/4] pack-objects: support reachability bitmaps with `--path-walk` Taylor Blau
2026-06-12 13:03     ` Derrick Stolee
2026-06-19 14:16       ` Taylor Blau [this message]
2026-06-19 14:36         ` Derrick Stolee
2026-06-19 14:46           ` Taylor Blau
2026-06-12 13:24     ` Derrick Stolee
2026-06-19 14:28       ` Taylor Blau
2026-06-19 14:40         ` Derrick Stolee
2026-06-19 14:52           ` Taylor Blau
2026-06-19 15:33             ` Derrick Stolee
2026-06-15 20:57     ` Junio C Hamano
2026-06-19 14:08       ` Taylor Blau
2026-06-02 22:21   ` [PATCH v2 3/4] pack-objects: extract `record_tree_depth()` helper Taylor Blau
2026-06-02 22:21   ` [PATCH v2 4/4] pack-objects: support `--delta-islands` with `--path-walk` Taylor Blau

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ajVPJGXuhugDcT+A@nand.local \
    --to=me@ttaylorr.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=newren@gmail.com \
    --cc=peff@peff.net \
    --cc=stolee@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox