From: Taylor Blau <me@ttaylorr.com>
To: git@vger.kernel.org
Cc: Junio C Hamano <gitster@pobox.com>,
Derrick Stolee <stolee@gmail.com>, Jeff King <peff@peff.net>,
Elijah Newren <newren@gmail.com>
Subject: [RFC PATCH 5/7] pack-objects: support reachability bitmaps with `--path-walk`
Date: Sun, 3 May 2026 20:11:29 -0400 [thread overview]
Message-ID: <f50f8df01a9f216d5b4388b2fe4ff58077b574f3.1777853408.git.me@ttaylorr.com> (raw)
In-Reply-To: <cover.1777853408.git.me@ttaylorr.com>
When 'pack-objects' is invoked with '--path-walk', it prevents us from
using reachability bitmaps.
This behavior dates back to 70664d2865c (pack-objects: add --path-walk
option, 2025-05-16), which included a comment in the relevant portion of
the command-line arguments handling that read as follows:
/*
* We must disable the bitmaps because we are removing
* the --objects / --objects-edge[-aggressive] options.
*/
In fb2c309b7d3 (pack-objects: pass --objects with --path-walk,
2026-05-02), we adjusted this behavior to also pass "--objects", but
still disable use of reachability bitmaps.
Fortunately, disabling reachability bitmaps is not strictly necessary.
Consider a couple of pack-objects use-cases: one during repacking, when
we would ordinarily generate reachability bitmaps, and another for
serving fetches and clones, when we would ordinarily read existing
bitmaps:
- When attempting to generate reachability bitmaps, we would fail to do
so since path-walk reveals objects through the
`add_objects_by_path()` callback rather than, e.g., `show_commit()`,
so the bitmap selector's `index_commit_for_bitmap()` was never called
for any commit.
The selection routine then had no candidates and bitmap writing was
effectively a no-op.
- On the bitmap-reading side, an invocation like "git pack-objects
--use-bitmap-index --path-walk" never even tried to consult a bitmap,
even when one was sitting on disk that could have answered the
request.
Neither restriction is required. They are discussed in turn:
- For bitmap-writing, all we need is for `index_commit_for_bitmap()` to
see each commit that path-walk visits. The path-walk callback already
groups commits into a single batch keyed by `OBJ_COMMIT`, so invoking
`index_commit_for_bitmap()` from there gives the bitmap selection
routine the same input it would have gotten from `show_commit()` in
the regular traversal.
The candidate set of commits is identical, though the ordering
differs. Bitmap selection is sensitive to commit ordering, but
commits are visited in the same order as we see them from
`get_revision()` so bitmap selection should be identical with or
without `--path-walk`.
- For bitmap-reading, all we need is for `revs->tree_objects` (and so
on for blobs and tags) to be set, otherwise bitmap traversal would
only emit commit objects.
In commit fb2c309b7d3, those flags are set via passing "--objects",
so bitmap traversal under "--path-walk" packs everything just like
any other "--use-bitmap-index" invocation.
If an existing reachability bitmap is unable to satisfy the request (no
bitmap on disk, haves not in the bitmapped pack, etc.) we fall through
to path-walk's own enumeration, just as the regular traversal falls back
when a bitmap is unavailable.
In other words: "--path-walk --use-bitmap-index" uses reachability
bitmaps when available, and otherwise enumerates via path-walk.
The regression in t5310 deserves a word about pack-reuse. With
pack-reuse enabled (the default), the output pack copies whole regions
of the existing bitmapped pack before `traverse_bitmap_commit_list()`
even runs, so a naive test would happily pass even if, say,
`revs->blob_objects` is set to 0. We test both with and without
pack-reuse enabled. When pack-reuse is disabled, pack-objects must
enumerate the resulting bitmap without copying any existing on-disk in
the rev_info setup that pack-reuse would otherwise paper over.
Update Documentation/git-pack-objects.adoc to drop the "ignored"
claim about --use-bitmap-index in favor of describing the new
fallback chain.
Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
Documentation/git-pack-objects.adoc | 6 +++--
builtin/pack-objects.c | 10 +++++++-
t/t5310-pack-bitmaps.sh | 36 +++++++++++++++++++++++++++++
3 files changed, 49 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc
index 6c7bbff5be5..60e594c7bc4 100644
--- a/Documentation/git-pack-objects.adoc
+++ b/Documentation/git-pack-objects.adoc
@@ -406,8 +406,10 @@ Incompatible with `--delta-islands`. Path-walk supports
the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,
`sparse:oid=<blob>`, `tree:0`, `object:type=<type>`, and `combine:`
over any of those. Other filter forms fall back to the regular object
-traversal. The `--use-bitmap-index` option will be ignored in the
-presence of `--path-walk`.
+traversal. When `--use-bitmap-index` is specified with `--path-walk`, a
+successful bitmap traversal is used for object enumeration, with
+path-walk remaining as the fallback traversal when the bitmap cannot
+satisfy the request.
DELTA ISLANDS
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index ba00d8148ab..1a5f1afd32e 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -4732,6 +4732,15 @@ static int add_objects_by_path(const char *path,
continue;
add_object_entry(oid, type, path, exclude);
+
+ if (type == OBJ_COMMIT && write_bitmap_index) {
+ struct commit *commit;
+
+ commit = lookup_commit(the_repository, oid);
+ if (!commit)
+ die(_("could not find commit %s"), oid_to_hex(oid));
+ index_commit_for_bitmap(commit);
+ }
}
oe_end = to_pack.nr_objects;
@@ -5193,7 +5202,6 @@ int cmd_pack_objects(int argc,
if (path_walk) {
strvec_push(&rp, "--boundary");
strvec_push(&rp, "--objects");
- use_bitmap_index = 0;
} else if (thin) {
use_internal_rev_list = 1;
strvec_push(&rp, shallow
diff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh
index f693cb56691..69c5da1580a 100755
--- a/t/t5310-pack-bitmaps.sh
+++ b/t/t5310-pack-bitmaps.sh
@@ -577,6 +577,42 @@ test_bitmap_cases
sane_unset GIT_TEST_PACK_USE_BITMAP_BOUNDARY_TRAVERSAL
+test_expect_success 'path-walk repack can write and use bitmap indexes' '
+ test_when_finished "rm -rf path-walk-bitmap" &&
+ git init path-walk-bitmap &&
+ (
+ cd path-walk-bitmap &&
+ test_commit first &&
+ test_commit second &&
+ test_commit third &&
+
+ git repack -a -d -b --path-walk &&
+ git rev-list --test-bitmap --use-bitmap-index HEAD &&
+
+ git rev-parse HEAD >in &&
+
+ git rev-list --objects --no-object-names HEAD >expect.raw &&
+ sort expect.raw >expect &&
+
+ for reuse in true false
+ do
+ : >trace.txt &&
+
+ GIT_TRACE2_EVENT="$(pwd)/trace.txt" \
+ git -c pack.allowPackReuse=$reuse pack-objects \
+ --stdout --revs --path-walk --use-bitmap-index \
+ <in >out.pack &&
+ grep "\"category\":\"bitmap\",\"key\":\"bitmap/hits\"" trace.txt &&
+
+ git index-pack out.pack &&
+
+ list_packed_objects out.idx >actual.raw &&
+ sort actual.raw >actual &&
+ test_cmp expect actual || return 1
+ done
+ )
+'
+
test_expect_success 'incremental repack fails when bitmaps are requested' '
test_commit more-1 &&
test_must_fail git repack -d 2>err &&
--
2.54.0.4.g6aa0d38a4ec
next prev parent reply other threads:[~2026-05-04 0:11 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-05-04 0:11 [RFC PATCH 0/7] pack-bitmap: resolve various `--path-walk` incompatibilities Taylor Blau
2026-05-04 0:11 ` [RFC PATCH 1/7] pack-objects: update `--path-walk`'s existing incompatibilities Taylor Blau
2026-05-04 12:22 ` Derrick Stolee
2026-05-04 0:11 ` [RFC PATCH 2/7] path-walk: support `tree:0` filter Taylor Blau
2026-05-04 12:30 ` Derrick Stolee
2026-05-04 21:55 ` Kristoffer Haugsbakk
2026-05-04 0:11 ` [RFC PATCH 3/7] path-walk: support `object:type` filter Taylor Blau
2026-05-04 12:32 ` Derrick Stolee
2026-05-04 0:11 ` [RFC PATCH 4/7] path-walk: support `combine` filter Taylor Blau
2026-05-04 0:11 ` Taylor Blau [this message]
2026-05-04 0:11 ` [RFC PATCH 6/7] pack-objects: extract `record_tree_depth()` helper Taylor Blau
2026-05-04 0:11 ` [RFC PATCH 7/7] pack-objects: support `--delta-islands` with `--path-walk` Taylor Blau
2026-05-04 12:13 ` [RFC PATCH 0/7] pack-bitmap: resolve various `--path-walk` incompatibilities Derrick Stolee
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f50f8df01a9f216d5b4388b2fe4ff58077b574f3.1777853408.git.me@ttaylorr.com \
--to=me@ttaylorr.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=newren@gmail.com \
--cc=peff@peff.net \
--cc=stolee@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox