* [PATCH v3 11/12] path-walk: support `object:type` filter
From: Taylor Blau via GitGitGadget @ 2026-05-11 18:13 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Taylor Blau, Derrick Stolee, Taylor Blau
In-Reply-To: <pull.2101.v3.git.1778523189.gitgitgadget@gmail.com>
From: Taylor Blau <me@ttaylorr.com>
The `object:type` filter accepts only objects of a single type; it is
the second member of the object-info-only filter family that bitmap
traversal already supports.
Like `blob:none` and `tree:0`, it can be evaluated with nothing more
than the object's type, which is exactly the granularity path-walk's
existing info->{commits,trees,blobs,tags} flags already control.
Map `LOFC_OBJECT_TYPE` in `prepare_filters()` by AND-ing each flag
against the filtered type. A single `object:type=X` filter
applied to the default info (all flags = 1) leaves `info->X = 1` and
all the others 0, which is what we want.
Using an AND rather than straight assignment prepares us for a
subsequent change to implement combined object filters.
The path-walk machinery is mostly already wired for the per-type
distinction:
- `walk_path()` calls `path_fn` for a batch only when the corresponding
`info->X` flag is set, so unwanted types are silently not reported.
- `add_tree_entries()` skips tree entries of type `OBJ_BLOB` when
`info->blobs` is unset, so we don't even allocate paths for them.
- The commit-walk loop short-circuits the root-tree fetch when
`!info->trees && !info->blobs`, so commit-only filters don't descend
into trees at all.
But there are a couple of side effects of the "trees off, blobs on" case
that need fixing:
1. 'setup_pending_objects()' previously skipped pending trees as soon
as `info->trees` was zero. For 'object:type=blob' the call site
needs those pending trees: a lightweight tag pointing to a tree, or
an annotated tag whose peeled target is a tree, can both reach
blobs that are otherwise unreachable from any commit's root tree.
Loosen the gate to "if (!info->trees && !info->blobs) continue" and
similarly retrieve the root_tree_list whenever either trees or
blobs are wanted.
2. The revision machinery's `handle_commit()` drops pending trees when
`revs->tree_objects` is zero (see the 'OBJ_TREE' handler in
revision.c), so by the time path-walk sees the pending list
after `prepare_revision_walk()` the tree-bearing pendings would
already be gone. Fix this by setting
revs->tree_objects = info->trees || info->blobs
so pending trees survive `prepare_revision_walk()` whenever we
need to walk into them. Path-walk still resets tree_objects to
zero immediately after `prepare_revision_walk()` returns, so the
rev-walk itself never enumerates trees redundantly with
path-walk's own descent.
Add coverage in t6601 for each of the four `object:type` values. The
'object:type=blob' test in particular asserts that file2 and child/file
(both reachable only through tag-pointed trees) show up in the output,
exercising the pending-tree fix.
Update Documentation/git-pack-objects.adoc to add object:type to
the list of supported --filter forms.
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
Documentation/git-pack-objects.adoc | 2 +-
path-walk.c | 13 ++++-
path-walk.h | 6 +++
t/t6601-path-walk.sh | 84 +++++++++++++++++++++++++++++
4 files changed, 103 insertions(+), 2 deletions(-)
diff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc
index 4ebda0fa9e..e96d2c00b3 100644
--- a/Documentation/git-pack-objects.adoc
+++ b/Documentation/git-pack-objects.adoc
@@ -405,7 +405,7 @@ will be automatically changed to version `1`.
Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
ignored in the presence of `--path-walk`. Whe `--path-walk` option
supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,
-`tree:0`, and `sparse:<oid>`.
+`tree:0`, `object:type=<type>`, and `sparse:<oid>`.
DELTA ISLANDS
diff --git a/path-walk.c b/path-walk.c
index f6f95da550..839505839b 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -381,7 +381,7 @@ static int walk_path(struct path_walk_context *ctx,
ret = ctx->info->path_fn(path, &filtered, list->type,
ctx->info->path_fn_data);
oid_array_clear(&filtered);
- } else if (path_is_for_direct_objects(path) ||
+ } else if ((!ctx->info->strict_types && path_is_for_direct_objects(path)) ||
(list->type == OBJ_TREE && ctx->info->trees) ||
(list->type == OBJ_BLOB && ctx->info->blobs) ||
(list->type == OBJ_TAG && ctx->info->tags)) {
@@ -610,6 +610,17 @@ static int prepare_filters(struct path_walk_info *info,
}
return 1;
+ case LOFC_OBJECT_TYPE:
+ if (info) {
+ info->commits &= options->object_type == OBJ_COMMIT;
+ info->tags &= options->object_type == OBJ_TAG;
+ info->trees &= options->object_type == OBJ_TREE;
+ info->blobs &= options->object_type == OBJ_BLOB;
+ info->strict_types = 1;
+ list_objects_filter_release(options);
+ }
+ return 1;
+
case LOFC_SPARSE_OID:
if (info) {
struct object_id sparse_oid;
diff --git a/path-walk.h b/path-walk.h
index 7e57ae5f65..a2652b2d46 100644
--- a/path-walk.h
+++ b/path-walk.h
@@ -47,6 +47,12 @@ struct path_walk_info {
int blobs;
int tags;
+ /**
+ * If 'strict_types' is 0, then direct object requests will no longer
+ * override the object type restrictions.
+ */
+ int strict_types;
+
/**
* If non-zero, specifies a maximum blob size. Blobs with a
* size equal to or greater than this limit will not be
diff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh
index deaeab9a3e..0515aaddbf 100755
--- a/t/t6601-path-walk.sh
+++ b/t/t6601-path-walk.sh
@@ -645,6 +645,90 @@ test_expect_success 'tree:1 filter is rejected' '
test_grep "tree:1 filter not supported by the path-walk API" err
'
+test_expect_success 'all, object:type=commit filter' '
+ test-tool path-walk --filter=object:type=commit -- --all >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ blobs:0
+ commits:4
+ tags:0
+ trees:0
+ EOF
+
+ test_cmp_sorted expect out
+'
+
+test_expect_success 'all, object:type=tag filter' '
+ test-tool path-walk --filter=object:type=tag -- --all >out &&
+
+ cat >expect <<-EOF &&
+ 0:tag:/tags:$(git rev-parse refs/tags/first)
+ 0:tag:/tags:$(git rev-parse refs/tags/second.1)
+ 0:tag:/tags:$(git rev-parse refs/tags/second.2)
+ 0:tag:/tags:$(git rev-parse refs/tags/third)
+ 0:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 0:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 0:tag:/tags:$(git rev-parse refs/tags/blob-tag)
+ blobs:0
+ commits:0
+ tags:7
+ trees:0
+ EOF
+
+ test_cmp_sorted expect out
+'
+
+test_expect_success 'all, object:type=tree filter' '
+ test-tool path-walk --filter=object:type=tree -- --all >out &&
+
+ cat >expect <<-EOF &&
+ 0:tree::$(git rev-parse topic^{tree})
+ 0:tree::$(git rev-parse base^{tree})
+ 0:tree::$(git rev-parse base~1^{tree})
+ 0:tree::$(git rev-parse base~2^{tree})
+ 0:tree::$(git rev-parse refs/tags/tree-tag^{})
+ 0:tree::$(git rev-parse refs/tags/tree-tag2^{})
+ 1:tree:a/:$(git rev-parse base:a)
+ 2:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
+ 3:tree:left/:$(git rev-parse base:left)
+ 3:tree:left/:$(git rev-parse base~2:left)
+ 4:tree:right/:$(git rev-parse topic:right)
+ 4:tree:right/:$(git rev-parse base~1:right)
+ 4:tree:right/:$(git rev-parse base~2:right)
+ blobs:0
+ commits:0
+ tags:0
+ trees:13
+ EOF
+
+ test_cmp_sorted expect out
+'
+
+test_expect_success 'all, object:type=blob filter' '
+ test-tool path-walk --filter=object:type=blob -- --all >out &&
+
+ cat >expect <<-EOF &&
+ 0:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
+ 0:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
+ 1:blob:a:$(git rev-parse base~2:a)
+ 2:blob:left/b:$(git rev-parse base:left/b)
+ 2:blob:left/b:$(git rev-parse base~2:left/b)
+ 3:blob:right/c:$(git rev-parse base~2:right/c)
+ 3:blob:right/c:$(git rev-parse topic:right/c)
+ 4:blob:right/d:$(git rev-parse base~1:right/d)
+ blobs:8
+ commits:0
+ tags:0
+ trees:0
+ EOF
+
+ test_cmp_sorted expect out
+'
+
test_expect_success 'setup sparse filter blob' '
# Cone-mode patterns: include root, exclude all dirs, include left/
cat >patterns <<-\EOF &&
--
gitgitgadget
^ permalink raw reply related
* [PATCH v3 10/12] path-walk: support `tree:0` filter
From: Taylor Blau via GitGitGadget @ 2026-05-11 18:13 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Taylor Blau, Derrick Stolee, Taylor Blau
In-Reply-To: <pull.2101.v3.git.1778523189.gitgitgadget@gmail.com>
From: Taylor Blau <me@ttaylorr.com>
The `tree:0` object filter omits all trees and blobs from the result,
keeping only commits and tags. Consequently, this filter type should
has a fairly straightforward integration with path-walk, as the decision
to include an object depends only on its type and does not depend on any
path-sensitive state.
Mapping it onto `path_walk_info` is direct: set `info->trees = 0` and
`info->blobs = 0` in `prepare_filters()` when the `LOFC_TREE_DEPTH`
choice is requested with depth zero. The existing code already plumbs
those flags through the rest of the walk:
- 'walk_objects_by_path()' sets `revs->blob_objects = info->blobs` and
`revs->tree_objects = info->trees` before `prepare_revision_walk()`,
so the revision walk doesn't try to enumerate trees or blobs itself.
- The commit-walk loop short-circuits the root-tree fetch with
"if (!info->trees && !info->blobs) continue;", so we never even
look up the root tree, let alone descend into it.
- `setup_pending_objects()` skips pending trees and blobs based on
the same flags.
This means the path-walk doesn't allocate or expand any tree structures
at all under `tree:0`, which matches the intended behavior of the
filter.
Non-zero tree-depth filters are not supported. Those depend on the depth
at which a tree is visited, which is a path-walk concept the filter
machinery doesn't currently share with the path-walk API. Reject them in
`prepare_filters()` with a helpful error and let pack-objects fall back
to the regular traversal, the same way it already does for unsupported
filters.
Add coverage in t6601 for both `--all` and a single-branch case to
confirm that no trees or blobs are emitted, and a separate test that
`tree:1` is rejected with the expected error message. Place the new
tests before "setup sparse filter blob" so they run on the original set
of refs, before the orphan branch that the sparse-tree tests create.
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
Documentation/git-pack-objects.adoc | 4 +--
path-walk.c | 42 +++++++++++++++++++++++--
t/t6601-path-walk.sh | 49 +++++++++++++++++++++++++++++
3 files changed, 90 insertions(+), 5 deletions(-)
diff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc
index f0887ab5b9..4ebda0fa9e 100644
--- a/Documentation/git-pack-objects.adoc
+++ b/Documentation/git-pack-objects.adoc
@@ -404,8 +404,8 @@ will be automatically changed to version `1`.
+
Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
ignored in the presence of `--path-walk`. Whe `--path-walk` option
-supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`, and
-`sparse:<oid>`.
+supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,
+`tree:0`, and `sparse:<oid>`.
DELTA ISLANDS
diff --git a/path-walk.c b/path-walk.c
index 1172385402..f6f95da550 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -389,8 +389,8 @@ static int walk_path(struct path_walk_context *ctx,
ctx->info->path_fn_data);
}
- /* Expand data for children. */
- if (list->type == OBJ_TREE) {
+ /* Expand data for children, unless this is a direct-object path. */
+ if (list->type == OBJ_TREE && !path_is_for_direct_objects(path)) {
for (size_t i = 0; i < list->oids.nr; i++) {
ret |= add_tree_entries(ctx,
path,
@@ -441,11 +441,14 @@ static int setup_pending_objects(struct path_walk_info *info,
{
struct type_and_oid_list *tags = NULL;
struct type_and_oid_list *tagged_blobs = NULL;
+ struct type_and_oid_list *tagged_trees = NULL;
struct type_and_oid_list *root_tree_list = NULL;
if (info->tags)
CALLOC_ARRAY(tags, 1);
CALLOC_ARRAY(tagged_blobs, 1);
+ if (!info->trees)
+ CALLOC_ARRAY(tagged_trees, 1);
root_tree_list = strmap_get(&ctx->paths_to_lists, root_path);
/*
@@ -490,7 +493,15 @@ static int setup_pending_objects(struct path_walk_info *info,
switch (obj->type) {
case OBJ_TREE:
- if (pending->path) {
+ if (tagged_trees) {
+ /*
+ * Trees are disabled but pending trees
+ * should still be emitted. Collect them
+ * into a "/tagged-trees" list that
+ * bypasses the object type filter.
+ */
+ oid_array_append(&tagged_trees->oids, &obj->oid);
+ } else if (pending->path) {
char *path = *pending->path ? xstrfmt("%s/", pending->path)
: xstrdup("");
add_path_to_list(ctx, path, OBJ_TREE, &obj->oid, 1);
@@ -534,6 +545,18 @@ static int setup_pending_objects(struct path_walk_info *info,
free(tagged_blobs);
}
}
+ if (tagged_trees) {
+ if (tagged_trees->oids.nr) {
+ const char *tagged_tree_path = "/tagged-trees";
+ tagged_trees->type = OBJ_TREE;
+ tagged_trees->maybe_interesting = 1;
+ strmap_put(&ctx->paths_to_lists, tagged_tree_path, tagged_trees);
+ push_to_stack(ctx, tagged_tree_path);
+ } else {
+ oid_array_clear(&tagged_trees->oids);
+ free(tagged_trees);
+ }
+ }
if (tags) {
if (tags->oids.nr) {
const char *tag_path = "/tags";
@@ -574,6 +597,19 @@ static int prepare_filters(struct path_walk_info *info,
}
return 1;
+ case LOFC_TREE_DEPTH:
+ if (options->tree_exclude_depth) {
+ error(_("tree:%lu filter not supported by the path-walk API"),
+ options->tree_exclude_depth);
+ return 0;
+ }
+ if (info) {
+ info->trees = 0;
+ info->blobs = 0;
+ list_objects_filter_release(options);
+ }
+ return 1;
+
case LOFC_SPARSE_OID:
if (info) {
struct object_id sparse_oid;
diff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh
index ac294867a5..deaeab9a3e 100755
--- a/t/t6601-path-walk.sh
+++ b/t/t6601-path-walk.sh
@@ -596,6 +596,55 @@ test_expect_success 'all, blob:limit=3 filter' '
test_cmp_sorted expect out
'
+test_expect_success 'all, tree:0 filter' '
+ test-tool path-walk --filter=tree:0 -- --all >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tag:/tags:$(git rev-parse refs/tags/first)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.1)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.2)
+ 1:tag:/tags:$(git rev-parse refs/tags/third)
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
+ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
+ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
+ 3:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag^{tree})
+ 3:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag2)
+ blobs:2
+ commits:4
+ tags:7
+ trees:2
+ EOF
+
+ test_cmp_sorted expect out
+'
+
+test_expect_success 'topic only, tree:0 filter' '
+ test-tool path-walk --filter=tree:0 -- topic >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ blobs:0
+ commits:3
+ tags:0
+ trees:0
+ EOF
+
+ test_cmp_sorted expect out
+'
+
+test_expect_success 'tree:1 filter is rejected' '
+ test_must_fail test-tool path-walk --filter=tree:1 -- --all 2>err &&
+ test_grep "tree:1 filter not supported by the path-walk API" err
+'
+
test_expect_success 'setup sparse filter blob' '
# Cone-mode patterns: include root, exclude all dirs, include left/
cat >patterns <<-\EOF &&
--
gitgitgadget
^ permalink raw reply related
* [PATCH v3 09/12] pack-objects: support sparse:oid filter with path-walk
From: Derrick Stolee via GitGitGadget @ 2026-05-11 18:13 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Taylor Blau, Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.v3.git.1778523189.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
The --filter=sparse:<oid> option to 'git pack-objects' allows focusing
an object set to a sparse-checkout definition. This reduces the set of
matching blobs while retaining all reachable trees. No server currently
supports fetching with this filter because it is expensive to compute
and reachability bitmaps do not help without a significant effort to
extend the bitmap feature to store bitmaps for each supported sparse-
checkout definition.
Without focusing on serving fetches and clones with these filters, there
are still benefits that could be realized by making this faster. With
the sparse index, it's more realistic now than ever to be able to
operate a local clone that was bootstrapped by a packfile created with
a sparse filter, because the missing trees are not needed to move a
sparse-checkout from one commit to another or to view the history of any
path in scope. Such clones could perhaps be bootstrapped by partial
bundles.
Previously, constructing these sparse packs has been incredibly
computationally inefficient. The revision walk that explores which
objects are in scope spends a lot of time checking each object to see if
it matches the sparse-checkout patterns, causing quadratic behavior
(number of objects times number of sparse-checkout patterns). This
improves somewhat when using cone-mode sparse-checkout patterns that can
use hashtables and prefix matches to determine containment. However, the
check per object is still too expensive for most cases.
This is where the path-walk feature comes in. We can proceed as normal
by placing objects in bins by path and _then_ check a group of objects
all at once. Since sparse:<oid> only restricts blobs, the path-walk must
include all reachable trees while using the cone-mode patterns to skip
blobs at paths outside the sparse scope. This establishes a baseline for
a potential future "treesparse:<oid>" filter that would also restrict
trees, but introducing such a new filter is deferred to a later change.
The implementation here is focused around loading the sparse-checkout
patterns from the provided object ID and checking that the patterns are
indeed cone-mode patterns. We can then load the correct pattern list
into the path walk context and use the logic that already exists from
bff45557675 (backfill: add --sparse option, 2025-02-03), though that
feature loads sparse-checkout patterns from the worktree's local
settings and also restricts tree objects. We use a combination of errors
and warnings to signal problems during this load. The difference is that
errors are likely fatal for the non-path-walk version while the warnings
are probably just implementation details for the path-walk version and
the 'git pack-objects' command can fall back to the revision walk
version.
Now that the SEEN flag is deferred until after pattern checks (from the
previous commit), handle the case where a tree with a shared OID appears
at both an out-of-cone and in-cone path. When trees are not being pruned
(pl_sparse_trees == 0), the path-walk re-walks the tree at the in-cone
path so that in-cone blobs within it are discovered. The new tests in
t5317 and t6601 demonstrate this behavior and would fail without these
changes.
The performance test p5315 shows the impact of this change when using
sparse filters:
Test HEAD~1 HEAD
----------------------------------------------------------------------
5315.10: repack (sparse:oid) 77.98 77.47 -0.7%
5315.11: repack size (sparse:oid) 187.5M 187.4M -0.0%
5315.12: repack (sparse:oid, --path-walk) 77.91 31.41 -59.7%
5315.13: repack size (sparse:oid, --path-walk) 187.5M 161.1M -14.1%
These performance tests were run on the Git repository. The --path-walk
feature shows meaningful space savings (14% smaller for sparse packs)
and dramatic time savings (60% faster) by leveraging the path-walk's
ability to skip blobs outside the sparse scope.
Co-authored-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Taylor Blaue <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
Documentation/git-backfill.adoc | 4 +
Documentation/git-pack-objects.adoc | 3 +-
builtin/pack-objects.c | 16 ++-
path-walk.c | 81 ++++++++++++++-
t/t5317-pack-objects-filter-objects.sh | 125 +++++++++++++++++++++++
t/t6601-path-walk.sh | 131 +++++++++++++++++++++++++
6 files changed, 350 insertions(+), 10 deletions(-)
diff --git a/Documentation/git-backfill.adoc b/Documentation/git-backfill.adoc
index c0a3b80615..82d6a1969d 100644
--- a/Documentation/git-backfill.adoc
+++ b/Documentation/git-backfill.adoc
@@ -80,6 +80,10 @@ OPTIONS
+
You may also use commit-limiting options understood by
linkgit:git-rev-list[1] such as `--first-parent`, `--since`, or pathspecs.
++
+Most `--filter=<spec>` options don't work with the purpose of
+`git backfill`, but the `sparse:<oid>` filter is integrated to provide a
+focused set of paths to download, distinct from the `--sparse` option.
SEE ALSO
--------
diff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc
index c193917731..f0887ab5b9 100644
--- a/Documentation/git-pack-objects.adoc
+++ b/Documentation/git-pack-objects.adoc
@@ -404,7 +404,8 @@ will be automatically changed to version `1`.
+
Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
ignored in the presence of `--path-walk`. Whe `--path-walk` option
-supports the `--filter=<spec>` forms `blob:none` and `blob:limit=<n>`.
+supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`, and
+`sparse:<oid>`.
DELTA ISLANDS
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index bc9fb5b457..b783dc62bc 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -4754,7 +4754,7 @@ static int add_objects_by_path(const char *path,
return 0;
}
-static void get_object_list_path_walk(struct rev_info *revs)
+static int get_object_list_path_walk(struct rev_info *revs)
{
struct path_walk_info info = PATH_WALK_INFO_INIT;
unsigned int processed = 0;
@@ -4777,8 +4777,9 @@ static void get_object_list_path_walk(struct rev_info *revs)
result = walk_objects_by_path(&info);
trace2_region_leave("pack-objects", "path-walk", revs->repo);
- if (result)
- die(_("failed to pack objects via path-walk"));
+ path_walk_info_clear(&info);
+
+ return result;
}
static void get_object_list(struct rev_info *revs, struct strvec *argv)
@@ -4841,8 +4842,13 @@ static void get_object_list(struct rev_info *revs, struct strvec *argv)
fn_show_object = show_object;
if (path_walk) {
- get_object_list_path_walk(revs);
- } else {
+ if (get_object_list_path_walk(revs)) {
+ warning(_("failed to pack objects via path-walk"));
+ path_walk = 0;
+ }
+ }
+
+ if (!path_walk) {
if (prepare_revision_walk(revs))
die(_("revision walk setup failed"));
mark_edges_uninteresting(revs, show_edge, sparse);
diff --git a/path-walk.c b/path-walk.c
index 21cc40c392..1172385402 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -10,6 +10,7 @@
#include "hex.h"
#include "list-objects.h"
#include "list-objects-filter-options.h"
+#include "object-name.h"
#include "odb.h"
#include "object.h"
#include "oid-array.h"
@@ -180,10 +181,6 @@ static int add_tree_entries(struct path_walk_context *ctx,
return -1;
}
- /* Skip this object if already seen. */
- if (o->flags & SEEN)
- continue;
-
strbuf_setlen(&path, base_len);
strbuf_add(&path, entry.path, entry.pathlen);
@@ -194,6 +191,40 @@ static int add_tree_entries(struct path_walk_context *ctx,
if (type == OBJ_TREE)
strbuf_addch(&path, '/');
+ if (o->flags & SEEN) {
+ /*
+ * A tree with a shared OID may appear at multiple
+ * paths. Even though we already added this tree to
+ * the output at some other path, we still need to
+ * walk into it at this in-cone path to discover
+ * blobs that were not found at the earlier
+ * out-of-cone path.
+ *
+ * Only do this for paths not yet in our map, to
+ * avoid duplicate entries when the same tree OID
+ * appears at the same path across multiple commits.
+ */
+ if (type == OBJ_TREE && ctx->info->pl &&
+ ctx->info->pl->use_cone_patterns &&
+ !ctx->info->pl_sparse_trees &&
+ !strmap_contains(&ctx->paths_to_lists, path.buf)) {
+ int dtype;
+ enum pattern_match_result m;
+ m = path_matches_pattern_list(path.buf, path.len,
+ path.buf + base_len,
+ &dtype,
+ ctx->info->pl,
+ ctx->repo->index);
+ if (m != NOT_MATCHED) {
+ add_path_to_list(ctx, path.buf, type,
+ &entry.oid,
+ !(o->flags & UNINTERESTING));
+ push_to_stack(ctx, path.buf);
+ }
+ }
+ continue;
+ }
+
if (ctx->info->pl) {
int dtype;
enum pattern_match_result match;
@@ -543,6 +574,48 @@ static int prepare_filters(struct path_walk_info *info,
}
return 1;
+ case LOFC_SPARSE_OID:
+ if (info) {
+ struct object_id sparse_oid;
+ struct repository *repo = info->revs->repo;
+
+ if (info->pl) {
+ warning(_("sparse filter cannot be combined with existing sparse patterns"));
+ return 0;
+ }
+
+ if (repo_get_oid_with_flags(repo,
+ options->sparse_oid_name,
+ &sparse_oid,
+ GET_OID_BLOB)) {
+ error(_("unable to access sparse blob in '%s'"),
+ options->sparse_oid_name);
+ return 0;
+ }
+
+ CALLOC_ARRAY(info->pl, 1);
+ info->pl->use_cone_patterns = 1;
+
+ if (add_patterns_from_blob_to_list(&sparse_oid, "", 0,
+ info->pl) < 0) {
+ clear_pattern_list(info->pl);
+ FREE_AND_NULL(info->pl);
+ error(_("unable to parse sparse filter data in '%s'"),
+ oid_to_hex(&sparse_oid));
+ return 0;
+ }
+
+ if (!info->pl->use_cone_patterns) {
+ clear_pattern_list(info->pl);
+ FREE_AND_NULL(info->pl);
+ warning(_("sparse filter is not cone-mode compatible"));
+ return 0;
+ }
+
+ list_objects_filter_release(options);
+ }
+ return 1;
+
default:
error(_("object filter '%s' not supported by the path-walk API"),
list_objects_filter_spec(options));
diff --git a/t/t5317-pack-objects-filter-objects.sh b/t/t5317-pack-objects-filter-objects.sh
index 501d715b9a..dddb79ba62 100755
--- a/t/t5317-pack-objects-filter-objects.sh
+++ b/t/t5317-pack-objects-filter-objects.sh
@@ -478,4 +478,129 @@ test_expect_success 'verify pack-objects w/ --missing=allow-any' '
EOF
'
+# Test that --path-walk produces the same object set as standard traversal
+# when using sparse:oid filters with cone-mode patterns.
+#
+# The sparse:oid filter restricts only blobs, not trees. Both standard
+# and path-walk should produce identical sets of blobs, commits, and trees.
+
+test_expect_success 'setup pw_sparse for path-walk comparison' '
+ git init pw_sparse &&
+ mkdir -p pw_sparse/inc/sub pw_sparse/exc/sub &&
+
+ for n in 1 2
+ do
+ echo "inc $n" >pw_sparse/inc/file$n &&
+ echo "inc sub $n" >pw_sparse/inc/sub/file$n &&
+ echo "exc $n" >pw_sparse/exc/file$n &&
+ echo "exc sub $n" >pw_sparse/exc/sub/file$n &&
+ echo "root $n" >pw_sparse/root$n || return 1
+ done &&
+
+ git -C pw_sparse add . &&
+ git -C pw_sparse commit -m "first" &&
+
+ echo "inc 1 modified" >pw_sparse/inc/file1 &&
+ echo "exc 1 modified" >pw_sparse/exc/file1 &&
+ echo "root 1 modified" >pw_sparse/root1 &&
+ git -C pw_sparse add . &&
+ git -C pw_sparse commit -m "second" &&
+
+ # Cone-mode sparse pattern: include root + inc/
+ printf "/*\n!/*/\n/inc/\n" |
+ git -C pw_sparse hash-object -w --stdin >sparse_oid
+'
+
+test_expect_success 'sparse:oid with --path-walk produces same blobs' '
+ oid=$(cat sparse_oid) &&
+
+ git -C pw_sparse pack-objects --revs --stdout \
+ --filter=sparse:oid=$oid >standard.pack <<-EOF &&
+ HEAD
+ EOF
+ git -C pw_sparse index-pack ../standard.pack &&
+ git -C pw_sparse verify-pack -v ../standard.pack >standard_verify &&
+
+ git -C pw_sparse pack-objects --revs --stdout \
+ --path-walk --filter=sparse:oid=$oid >pathwalk.pack <<-EOF &&
+ HEAD
+ EOF
+ git -C pw_sparse index-pack ../pathwalk.pack &&
+ git -C pw_sparse verify-pack -v ../pathwalk.pack >pathwalk_verify &&
+
+ # Blobs must match exactly
+ grep -E "^[0-9a-f]{40} blob" standard_verify |
+ awk "{print \$1}" | sort >standard_blobs &&
+ grep -E "^[0-9a-f]{40} blob" pathwalk_verify |
+ awk "{print \$1}" | sort >pathwalk_blobs &&
+ test_cmp standard_blobs pathwalk_blobs &&
+
+ # Commits must match exactly
+ grep -E "^[0-9a-f]{40} commit" standard_verify |
+ awk "{print \$1}" | sort >standard_commits &&
+ grep -E "^[0-9a-f]{40} commit" pathwalk_verify |
+ awk "{print \$1}" | sort >pathwalk_commits &&
+ test_cmp standard_commits pathwalk_commits
+'
+
+test_expect_success 'sparse:oid with --path-walk includes all trees' '
+ # The sparse:oid filter restricts only blobs, not trees.
+ # Both standard and path-walk should include the same trees.
+ grep -E "^[0-9a-f]{40} tree" standard_verify |
+ awk "{print \$1}" | sort >standard_trees &&
+ grep -E "^[0-9a-f]{40} tree" pathwalk_verify |
+ awk "{print \$1}" | sort >pathwalk_trees &&
+
+ test_cmp standard_trees pathwalk_trees
+'
+
+# Test the edge case where the same tree/blob OID appears at both an
+# in-cone and out-of-cone path. When sibling directories have identical
+# contents, they share a tree OID. The path-walk defers marking objects
+# SEEN until after checking sparse patterns, so an object at an out-of-cone
+# path can still be discovered at an in-cone path.
+
+test_expect_success 'setup pw_shared for shared OID across cone boundary' '
+ git init pw_shared &&
+ mkdir pw_shared/aaa pw_shared/zzz &&
+ echo "shared content" >pw_shared/aaa/file &&
+ echo "shared content" >pw_shared/zzz/file &&
+ echo "root file" >pw_shared/rootfile &&
+ git -C pw_shared add . &&
+ git -C pw_shared commit -m "aaa and zzz share tree OID" &&
+
+ # Verify they share a tree OID
+ aaa_tree=$(git -C pw_shared rev-parse HEAD:aaa) &&
+ zzz_tree=$(git -C pw_shared rev-parse HEAD:zzz) &&
+ test "$aaa_tree" = "$zzz_tree" &&
+
+ # Cone pattern: include root + zzz/ (not aaa/)
+ printf "/*\n!/*/\n/zzz/\n" |
+ git -C pw_shared hash-object -w --stdin >shared_sparse_oid
+'
+
+test_expect_success 'shared tree OID: --path-walk blobs match standard' '
+ oid=$(cat shared_sparse_oid) &&
+
+ git -C pw_shared pack-objects --revs --stdout \
+ --filter=sparse:oid=$oid >shared_std.pack <<-EOF &&
+ HEAD
+ EOF
+ git -C pw_shared index-pack ../shared_std.pack &&
+ git -C pw_shared verify-pack -v ../shared_std.pack >shared_std_verify &&
+
+ git -C pw_shared pack-objects --revs --stdout \
+ --path-walk --filter=sparse:oid=$oid >shared_pw.pack <<-EOF &&
+ HEAD
+ EOF
+ git -C pw_shared index-pack ../shared_pw.pack &&
+ git -C pw_shared verify-pack -v ../shared_pw.pack >shared_pw_verify &&
+
+ grep -E "^[0-9a-f]{40} blob" shared_std_verify |
+ awk "{print \$1}" | sort >shared_std_blobs &&
+ grep -E "^[0-9a-f]{40} blob" shared_pw_verify |
+ awk "{print \$1}" | sort >shared_pw_blobs &&
+ test_cmp shared_std_blobs shared_pw_blobs
+'
+
test_done
diff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh
index 02ad83dfb0..ac294867a5 100755
--- a/t/t6601-path-walk.sh
+++ b/t/t6601-path-walk.sh
@@ -596,4 +596,135 @@ test_expect_success 'all, blob:limit=3 filter' '
test_cmp_sorted expect out
'
+test_expect_success 'setup sparse filter blob' '
+ # Cone-mode patterns: include root, exclude all dirs, include left/
+ cat >patterns <<-\EOF &&
+ /*
+ !/*/
+ /left/
+ EOF
+ sparse_oid=$(git hash-object -w -t blob patterns)
+'
+
+test_expect_success 'all, sparse:oid filter' '
+ test-tool path-walk --filter=sparse:oid=$sparse_oid -- --all >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tag:/tags:$(git rev-parse refs/tags/first)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.1)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.2)
+ 1:tag:/tags:$(git rev-parse refs/tags/third)
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
+ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
+ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
+ 3:tree::$(git rev-parse topic^{tree})
+ 3:tree::$(git rev-parse base^{tree})
+ 3:tree::$(git rev-parse base~1^{tree})
+ 3:tree::$(git rev-parse base~2^{tree})
+ 3:tree::$(git rev-parse refs/tags/tree-tag^{})
+ 3:tree::$(git rev-parse refs/tags/tree-tag2^{})
+ 4:blob:a:$(git rev-parse base~2:a)
+ 5:blob:file2:$(git rev-parse refs/tags/tree-tag2^{}:file2)
+ 6:tree:a/:$(git rev-parse base:a)
+ 7:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
+ 8:tree:left/:$(git rev-parse base:left)
+ 8:tree:left/:$(git rev-parse base~2:left)
+ 9:blob:left/b:$(git rev-parse base~2:left/b)
+ 9:blob:left/b:$(git rev-parse base:left/b)
+ 10:tree:right/:$(git rev-parse topic:right)
+ 10:tree:right/:$(git rev-parse base~1:right)
+ 10:tree:right/:$(git rev-parse base~2:right)
+ blobs:6
+ commits:4
+ tags:7
+ trees:13
+ EOF
+
+ test_cmp_sorted expect out
+'
+
+test_expect_success 'topic only, sparse:oid filter' '
+ test-tool path-walk --filter=sparse:oid=$sparse_oid -- topic >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tree::$(git rev-parse topic^{tree})
+ 1:tree::$(git rev-parse base~1^{tree})
+ 1:tree::$(git rev-parse base~2^{tree})
+ 2:blob:a:$(git rev-parse base~2:a)
+ 3:tree:left/:$(git rev-parse base~2:left)
+ 4:blob:left/b:$(git rev-parse base~2:left/b)
+ 5:tree:right/:$(git rev-parse topic:right)
+ 5:tree:right/:$(git rev-parse base~1:right)
+ 5:tree:right/:$(git rev-parse base~2:right)
+ blobs:2
+ commits:3
+ tags:0
+ trees:7
+ EOF
+
+ test_cmp_sorted expect out
+'
+
+# Demonstrate the SEEN flag ordering issue: when the same tree/blob OID
+# appears at two sibling paths where one is in-cone and the other is
+# out-of-cone, the path-walk must still discover blobs at the in-cone
+# path even when the shared tree OID was first encountered out-of-cone.
+# Since sparse:oid includes all trees, the out-of-cone tree (aaa/) is
+# walked first, and its blob is skipped. The path-walk then re-walks
+# the same tree OID at the in-cone path (zzz/) to find the blob there.
+
+test_expect_success 'setup shared tree OID across cone boundary' '
+ git checkout --orphan shared-tree &&
+ git rm -rf . &&
+ mkdir aaa zzz &&
+ echo "shared content" >aaa/file &&
+ echo "shared content" >zzz/file &&
+ echo "root file" >rootfile &&
+ git add aaa zzz rootfile &&
+ git commit -m "aaa and zzz have same tree OID" &&
+
+ # Verify they really share a tree OID
+ aaa_tree=$(git rev-parse HEAD:aaa) &&
+ zzz_tree=$(git rev-parse HEAD:zzz) &&
+ test "$aaa_tree" = "$zzz_tree" &&
+
+ # Cone pattern: include root + zzz/ (not aaa/)
+ cat >shared-patterns <<-\EOF &&
+ /*
+ !/*/
+ /zzz/
+ EOF
+ shared_sparse_oid=$(git hash-object -w -t blob shared-patterns)
+'
+
+test_expect_success 'sparse:oid with shared tree OID across cone boundary' '
+ test-tool path-walk \
+ --filter=sparse:oid=$shared_sparse_oid \
+ -- shared-tree >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse shared-tree)
+ 1:tree::$(git rev-parse shared-tree^{tree})
+ 2:blob:rootfile:$(git rev-parse shared-tree:rootfile)
+ 3:tree:aaa/:$(git rev-parse shared-tree:aaa)
+ 4:tree:zzz/:$(git rev-parse shared-tree:zzz)
+ 5:blob:zzz/file:$(git rev-parse shared-tree:zzz/file)
+ blobs:2
+ commits:1
+ tags:0
+ trees:3
+ EOF
+
+ test_cmp_sorted expect out
+'
+
test_done
--
gitgitgadget
^ permalink raw reply related
* [PATCH v3 08/12] path-walk: add pl_sparse_trees to control tree pruning
From: Derrick Stolee via GitGitGadget @ 2026-05-11 18:13 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Taylor Blau, Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.v3.git.1778523189.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
The path-walk API prunes trees and blobs when a sparse-checkout pattern
list is provided, which is the correct behavior for 'git backfill
--sparse' since it only needs to fill in objects at paths within the
sparse cone.
However, a future change will use the path-walk API with a sparse:<oid>
filter that restricts only blobs while retaining all reachable trees.
To support both behaviors, add a 'pl_sparse_trees' flag to
path_walk_info. When set (as in 'git backfill --sparse' and the
--stdin-pl test helper mode), the sparse patterns prune both trees and
blobs. When unset, only blobs are filtered and all trees are walked and
reported.
Additionally, move the SEEN flag assignment in add_tree_entries() to
after the sparse pattern and pathspec checks. Previously, SEEN was set
immediately upon discovering an object, before checking whether its path
matched the sparse patterns. When the same object ID appeared at
multiple paths (e.g. sibling directories with identical contents), the
first path to be visited would mark the object as SEEN. If that path was
outside the sparse cone, the object would be skipped there but also
never discovered at its in-cone path.
By deferring the SEEN flag until after the checks pass, objects that are
skipped due to sparse filtering remain discoverable at other paths where
they may be in scope.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
builtin/backfill.c | 1 +
path-walk.c | 5 +++--
path-walk.h | 6 ++++++
t/helper/test-path-walk.c | 6 +++++-
t/t6601-path-walk.sh | 37 +++++++++++++++++++++++++++++++++++++
5 files changed, 52 insertions(+), 3 deletions(-)
diff --git a/builtin/backfill.c b/builtin/backfill.c
index 5254a42711..e71e0f4742 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -109,6 +109,7 @@ static int do_backfill(struct backfill_context *ctx)
if (ctx->sparse) {
CALLOC_ARRAY(info.pl, 1);
+ info.pl_sparse_trees = 1;
if (get_sparse_checkout_patterns(info.pl)) {
path_walk_info_clear(&info);
return error(_("problem loading sparse-checkout"));
diff --git a/path-walk.c b/path-walk.c
index 16fdfd7c5a..21cc40c392 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -183,7 +183,6 @@ static int add_tree_entries(struct path_walk_context *ctx,
/* Skip this object if already seen. */
if (o->flags & SEEN)
continue;
- o->flags |= SEEN;
strbuf_setlen(&path, base_len);
strbuf_add(&path, entry.path, entry.pathlen);
@@ -204,7 +203,8 @@ static int add_tree_entries(struct path_walk_context *ctx,
ctx->repo->index);
if (ctx->info->pl->use_cone_patterns &&
- match == NOT_MATCHED)
+ match == NOT_MATCHED &&
+ (type == OBJ_BLOB || ctx->info->pl_sparse_trees))
continue;
else if (!ctx->info->pl->use_cone_patterns &&
type == OBJ_BLOB &&
@@ -239,6 +239,7 @@ static int add_tree_entries(struct path_walk_context *ctx,
continue;
}
+ o->flags |= SEEN;
add_path_to_list(ctx, path.buf, type, &entry.oid,
!(o->flags & UNINTERESTING));
diff --git a/path-walk.h b/path-walk.h
index 60ceb65433..7e57ae5f65 100644
--- a/path-walk.h
+++ b/path-walk.h
@@ -76,8 +76,14 @@ struct path_walk_info {
* of the cone. If not in cone mode, then all tree paths will be
* explored but the path_fn will only be called when the path matches
* the sparse-checkout patterns.
+ *
+ * When 'pl_sparse_trees' is zero, the sparse patterns only restrict
+ * blobs and all trees are included in the walk output. This matches
+ * the behavior of the sparse:oid object filter. When nonzero, trees
+ * are also pruned by the sparse patterns (as used by backfill).
*/
struct pattern_list *pl;
+ int pl_sparse_trees;
};
#define PATH_WALK_INFO_INIT { \
diff --git a/t/helper/test-path-walk.c b/t/helper/test-path-walk.c
index 88f86ae0dc..3f2b50a9aa 100644
--- a/t/helper/test-path-walk.c
+++ b/t/helper/test-path-walk.c
@@ -68,7 +68,7 @@ static int emit_block(const char *path, struct oid_array *oids,
int cmd__path_walk(int argc, const char **argv)
{
- int res, stdin_pl = 0;
+ int res, stdin_pl = 0, pl_sparse_trees = -1;
struct rev_info revs = REV_INFO_INIT;
struct path_walk_info info = PATH_WALK_INFO_INIT;
struct path_walk_test_data data = { 0 };
@@ -89,6 +89,8 @@ int cmd__path_walk(int argc, const char **argv)
N_("toggle aggressive edge walk")),
OPT_BOOL(0, "stdin-pl", &stdin_pl,
N_("read a pattern list over stdin")),
+ OPT_BOOL(0, "pl-sparse-trees", &pl_sparse_trees,
+ N_("toggle pruning of trees by sparse patterns")),
OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
OPT_END(),
};
@@ -116,6 +118,8 @@ int cmd__path_walk(int argc, const char **argv)
if (stdin_pl) {
struct strbuf in = STRBUF_INIT;
CALLOC_ARRAY(info.pl, 1);
+ info.pl_sparse_trees = (pl_sparse_trees >= 0) ?
+ pl_sparse_trees : 1;
info.pl->use_cone_patterns = 1;
diff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh
index 45f366d738..02ad83dfb0 100755
--- a/t/t6601-path-walk.sh
+++ b/t/t6601-path-walk.sh
@@ -206,6 +206,43 @@ test_expect_success 'base & topic, sparse' '
test_cmp_sorted expect out
'
+test_expect_success 'base & topic, sparse, no tree pruning' '
+ cat >patterns <<-EOF &&
+ /*
+ !/*/
+ /left/
+ EOF
+
+ test-tool path-walk --stdin-pl --no-pl-sparse-trees \
+ -- base topic <patterns >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tree::$(git rev-parse topic^{tree})
+ 1:tree::$(git rev-parse base^{tree})
+ 1:tree::$(git rev-parse base~1^{tree})
+ 1:tree::$(git rev-parse base~2^{tree})
+ 2:blob:a:$(git rev-parse base~2:a)
+ 3:tree:a/:$(git rev-parse base:a)
+ 4:tree:left/:$(git rev-parse base:left)
+ 4:tree:left/:$(git rev-parse base~2:left)
+ 5:blob:left/b:$(git rev-parse base~2:left/b)
+ 5:blob:left/b:$(git rev-parse base:left/b)
+ 6:tree:right/:$(git rev-parse topic:right)
+ 6:tree:right/:$(git rev-parse base~1:right)
+ 6:tree:right/:$(git rev-parse base~2:right)
+ blobs:3
+ commits:4
+ tags:0
+ trees:10
+ EOF
+
+ test_cmp_sorted expect out
+'
+
test_expect_success 'topic only' '
test-tool path-walk -- topic >out &&
--
gitgitgadget
^ permalink raw reply related
* [PATCH v3 07/12] path-walk: support blob size limit filter
From: Derrick Stolee via GitGitGadget @ 2026-05-11 18:13 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Taylor Blau, Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.v3.git.1778523189.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
Extend the path-walk API to handle the 'blob:limit=<size>' object
filter natively. This filter omits blobs whose size is equal to or
greater than the given limit, matching the semantics used by the
list-objects-filter machinery.
When revs->filter.choice is LOFC_BLOB_LIMIT, the prepare_filters()
method stores the limit value in info->blob_limit and clears the filter
from revs. If the limit is zero, this degenerates to blob:none (all
blobs excluded), so info->blobs is set to 0 instead.
During walk_path(), blob batches are filtered before being delivered to
the callback: each blob's size is checked via odb_read_object_info(),
and only blobs strictly smaller than the limit are included. Blobs whose
size cannot be determined (e.g. missing in a partial clone) are
conservatively included, matching the existing filter behavior. Empty
batches after filtering are skipped entirely.
The check for inclusion in the path batch looks a little strange at
first glance. We use odb_read_object_info() to read the object's size.
Based on all of the assumptions to this point, this _should_ return
OBJ_BLOB. Since we are focused on the size filter, we use a
short-circuited OR (||) to skip the size check if that method returns a
different object type.
Notice that this inspection of object sizes requires the content to be
present in the repository. The odb_read_object_info() call will download
a missing blob on-demand. This means that the use of the path-walk API
within 'git backfill' would not operate nicely with this filter type.
The intention of that command is to download missing blobs in batches.
Downloading objects one-by-one would go against the point. Update the
validation in 'git backfill' to add its own compatibility check on top
of path_walk_filter_compatible().
Add tests for blob:limit=0 (equivalent to blob:none) and blob:limit=3
(which exercises partial filtering within a batch where some blobs are
kept and others are excluded).
Co-authored-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
Documentation/git-pack-objects.adoc | 2 +-
builtin/backfill.c | 2 +
path-walk.c | 41 +++++++++++++--
path-walk.h | 7 +++
t/t5620-backfill.sh | 2 +-
t/t6601-path-walk.sh | 82 +++++++++++++++++++++++++++++
6 files changed, 130 insertions(+), 6 deletions(-)
diff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc
index 917045d5c3..c193917731 100644
--- a/Documentation/git-pack-objects.adoc
+++ b/Documentation/git-pack-objects.adoc
@@ -404,7 +404,7 @@ will be automatically changed to version `1`.
+
Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
ignored in the presence of `--path-walk`. Whe `--path-walk` option
-supports the `--filter=<spec>` form `blob:none`.
+supports the `--filter=<spec>` forms `blob:none` and `blob:limit=<n>`.
DELTA ISLANDS
diff --git a/builtin/backfill.c b/builtin/backfill.c
index b80f9ebe69..5254a42711 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -98,6 +98,8 @@ static void reject_unsupported_rev_list_options(struct rev_info *revs)
"--diff-merges");
if (!path_walk_filter_compatible(&revs->filter))
die(_("cannot backfill with these filter options"));
+ if (revs->filter.blob_limit_value)
+ die(_("cannot backfill with blob size limits"));
}
static int do_backfill(struct backfill_context *ctx)
diff --git a/path-walk.c b/path-walk.c
index cc587268e7..16fdfd7c5a 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -10,6 +10,7 @@
#include "hex.h"
#include "list-objects.h"
#include "list-objects-filter-options.h"
+#include "odb.h"
#include "object.h"
#include "oid-array.h"
#include "path.h"
@@ -326,13 +327,35 @@ static int walk_path(struct path_walk_context *ctx,
/*
* Evaluate function pointer on this data, if requested.
* Ignore object type filters for tagged objects (path starts
- * with `/`).
+ * with `/`), first for blobs and then other types.
*/
- if ((list->type == OBJ_TREE && (ctx->info->trees || path_is_for_direct_objects(path))) ||
- (list->type == OBJ_BLOB && (ctx->info->blobs || path_is_for_direct_objects(path))) ||
- (list->type == OBJ_TAG && ctx->info->tags))
+ if (list->type == OBJ_BLOB &&
+ ctx->info->blob_limit &&
+ !path_is_for_direct_objects(path)) {
+ struct oid_array filtered = OID_ARRAY_INIT;
+
+ for (size_t i = 0; i < list->oids.nr; i++) {
+ unsigned long size;
+
+ if (odb_read_object_info(ctx->repo->objects,
+ &list->oids.oid[i],
+ &size) != OBJ_BLOB ||
+ size < ctx->info->blob_limit)
+ oid_array_append(&filtered,
+ &list->oids.oid[i]);
+ }
+
+ if (filtered.nr)
+ ret = ctx->info->path_fn(path, &filtered, list->type,
+ ctx->info->path_fn_data);
+ oid_array_clear(&filtered);
+ } else if (path_is_for_direct_objects(path) ||
+ (list->type == OBJ_TREE && ctx->info->trees) ||
+ (list->type == OBJ_BLOB && ctx->info->blobs) ||
+ (list->type == OBJ_TAG && ctx->info->tags)) {
ret = ctx->info->path_fn(path, &list->oids, list->type,
ctx->info->path_fn_data);
+ }
/* Expand data for children. */
if (list->type == OBJ_TREE) {
@@ -509,6 +532,16 @@ static int prepare_filters(struct path_walk_info *info,
}
return 1;
+ case LOFC_BLOB_LIMIT:
+ if (info) {
+ if (!options->blob_limit_value)
+ info->blobs = 0;
+ else
+ info->blob_limit = options->blob_limit_value;
+ list_objects_filter_release(options);
+ }
+ return 1;
+
default:
error(_("object filter '%s' not supported by the path-walk API"),
list_objects_filter_spec(options));
diff --git a/path-walk.h b/path-walk.h
index a1736ecb2b..60ceb65433 100644
--- a/path-walk.h
+++ b/path-walk.h
@@ -47,6 +47,13 @@ struct path_walk_info {
int blobs;
int tags;
+ /**
+ * If non-zero, specifies a maximum blob size. Blobs with a
+ * size equal to or greater than this limit will not be
+ * emitted unless included in 'pending'.
+ */
+ unsigned long blob_limit;
+
/**
* When 'prune_all_uninteresting' is set and a path has all objects
* marked as UNINTERESTING, then the path-walk will not visit those
diff --git a/t/t5620-backfill.sh b/t/t5620-backfill.sh
index ede89f8c33..d2ea68e065 100755
--- a/t/t5620-backfill.sh
+++ b/t/t5620-backfill.sh
@@ -20,7 +20,7 @@ test_expect_success 'backfill rejects incompatible filter options' '
test_grep "cannot backfill with these filter options" err &&
test_must_fail git backfill --objects --filter=blob:limit=10m 2>err &&
- test_grep "cannot backfill with these filter options" err
+ test_grep "cannot backfill with blob size limits" err
'
# We create objects in the 'src' repo.
diff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh
index b0ee31ee2d..45f366d738 100755
--- a/t/t6601-path-walk.sh
+++ b/t/t6601-path-walk.sh
@@ -477,4 +477,86 @@ test_expect_success 'topic only, blob:none filter' '
test_cmp_sorted expect out
'
+test_expect_success 'all, blob:limit=0 filter' '
+ test-tool path-walk --filter=blob:limit=0 -- --all >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tag:/tags:$(git rev-parse refs/tags/first)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.1)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.2)
+ 1:tag:/tags:$(git rev-parse refs/tags/third)
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
+ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
+ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
+ 3:tree::$(git rev-parse topic^{tree})
+ 3:tree::$(git rev-parse base^{tree})
+ 3:tree::$(git rev-parse base~1^{tree})
+ 3:tree::$(git rev-parse base~2^{tree})
+ 3:tree::$(git rev-parse refs/tags/tree-tag^{})
+ 3:tree::$(git rev-parse refs/tags/tree-tag2^{})
+ 4:tree:a/:$(git rev-parse base:a)
+ 5:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
+ 6:tree:left/:$(git rev-parse base:left)
+ 6:tree:left/:$(git rev-parse base~2:left)
+ 7:tree:right/:$(git rev-parse topic:right)
+ 7:tree:right/:$(git rev-parse base~1:right)
+ 7:tree:right/:$(git rev-parse base~2:right)
+ blobs:2
+ commits:4
+ tags:7
+ trees:13
+ EOF
+
+ test_cmp_sorted expect out
+'
+
+test_expect_success 'all, blob:limit=3 filter' '
+ test-tool path-walk --filter=blob:limit=3 -- --all >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tag:/tags:$(git rev-parse refs/tags/first)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.1)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.2)
+ 1:tag:/tags:$(git rev-parse refs/tags/third)
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
+ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
+ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
+ 3:tree::$(git rev-parse topic^{tree})
+ 3:tree::$(git rev-parse base^{tree})
+ 3:tree::$(git rev-parse base~1^{tree})
+ 3:tree::$(git rev-parse base~2^{tree})
+ 3:tree::$(git rev-parse refs/tags/tree-tag^{})
+ 3:tree::$(git rev-parse refs/tags/tree-tag2^{})
+ 4:blob:a:$(git rev-parse base~2:a)
+ 5:tree:a/:$(git rev-parse base:a)
+ 6:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
+ 7:tree:left/:$(git rev-parse base:left)
+ 7:tree:left/:$(git rev-parse base~2:left)
+ 8:blob:left/b:$(git rev-parse base~2:left/b)
+ 9:tree:right/:$(git rev-parse topic:right)
+ 9:tree:right/:$(git rev-parse base~1:right)
+ 9:tree:right/:$(git rev-parse base~2:right)
+ 10:blob:right/c:$(git rev-parse base~2:right/c)
+ 11:blob:right/d:$(git rev-parse base~1:right/d)
+ blobs:6
+ commits:4
+ tags:7
+ trees:13
+ EOF
+
+ test_cmp_sorted expect out
+'
+
test_done
--
gitgitgadget
^ permalink raw reply related
* [PATCH v3 06/12] backfill: die on incompatible filter options
From: Derrick Stolee via GitGitGadget @ 2026-05-11 18:13 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Taylor Blau, Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.v3.git.1778523189.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
The 'git backfill' command uses the path-walk API in a critical way: it
uses the objects output from the command to find the batches of missing
objects that should be requested from the server. Unlike 'git
pack-objects', we cannot fall back to another mechanism.
The previous change added the path_walk_filter_compatible() method that
we can reuse here. Use it during argument validation in cmd_backfill().
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
builtin/backfill.c | 5 ++---
t/t5620-backfill.sh | 8 ++++++++
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/builtin/backfill.c b/builtin/backfill.c
index 7ffab2ea74..b80f9ebe69 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -96,9 +96,8 @@ static void reject_unsupported_rev_list_options(struct rev_info *revs)
if (revs->explicit_diff_merges)
die(_("'%s' cannot be used with 'git backfill'"),
"--diff-merges");
- if (revs->filter.choice)
- die(_("'%s' cannot be used with 'git backfill'"),
- "--filter");
+ if (!path_walk_filter_compatible(&revs->filter))
+ die(_("cannot backfill with these filter options"));
}
static int do_backfill(struct backfill_context *ctx)
diff --git a/t/t5620-backfill.sh b/t/t5620-backfill.sh
index 94f35ce190..ede89f8c33 100755
--- a/t/t5620-backfill.sh
+++ b/t/t5620-backfill.sh
@@ -15,6 +15,14 @@ test_expect_success 'backfill rejects unexpected arguments' '
test_grep "unrecognized argument: --unexpected-arg" err
'
+test_expect_success 'backfill rejects incompatible filter options' '
+ test_must_fail git backfill --objects --filter=tree:1 2>err &&
+ test_grep "cannot backfill with these filter options" err &&
+
+ test_must_fail git backfill --objects --filter=blob:limit=10m 2>err &&
+ test_grep "cannot backfill with these filter options" err
+'
+
# We create objects in the 'src' repo.
test_expect_success 'setup repo for object creation' '
echo "{print \$1}" >print_1.awk &&
--
gitgitgadget
^ permalink raw reply related
* [PATCH v3 05/12] path-walk: support blobless filter
From: Derrick Stolee via GitGitGadget @ 2026-05-11 18:13 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Taylor Blau, Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.v3.git.1778523189.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
The 'git pack-objects' command can opt-in to using the path-walk API for
scanning the objects. Currently, this option is dynamically disabled if
combined with '--filter=<X>', even when using a simple filter such as
'blob:none' to signal a blobless packfile. This is a common scenario for
repos at scale, so is worth integrating.
Also, users can opt-in to the '--path-walk' option by default through
the pack.usePathWalk=true config option. When using that in a blobless
partial clone, the following warning can appear even though the user did
not specify either option directly:
warning: cannot use --filter with --path-walk
Teach the path-walk API to handle the 'blob:none' object filter
natively. When revs->filter.choice is LOFC_BLOB_NONE, the path-walk
sets info->blobs to 0 (skipping all blob objects) and clears the
filter from revs so that prepare_revision_walk() does not reject the
configuration.
This check is implemented in the static prepare_filters() method, which
will simultaneously check if the input filters are compatible and will
make the appropriate mutations to the path_walk_info and filters if the
path_walk_info is non-NULL. This allows us to use this logic both in the
API method path_walk_filter_compatible() for use in
builtin/pack-objects.c and as a prep step in walk_objects_by_path().
Update the test helper (test-path-walk) to accept --filter=<spec>
as a test-tool option (before '--'), applying it to revs after
setup_revisions() to avoid the --objects requirement check. We can also
revert recent GIT_TEST_PACK_PATH_WALK overrides in t5620.
Also switch test-path-walk from REV_INFO_INIT with manual repo
assignment to repo_init_revisions(), which properly initializes
the filter_spec strbuf needed for filter parsing.
Add tests for blob:none with --all and with a single branch.
The performance test p5315 shows the impact of this change when using
blobless filters:
Test HEAD~1 HEAD
---------------------------------------------------------------------
5315.6: repack (blob:none) 13.53 13.87 +2.5%
5315.7: repack size (blob:none) 137.7M 137.8M +0.1%
5315.8: repack (blob:none, --path-walk) 13.51 23.43 +73.4%
5315.9: repack size (blob:none, --path-walk) 137.7M 115.2M -16.3%
These performance tests were run on the Git repository. The --path-walk
feature shows meaningful space savings (16% smaller for blobless packs)
at the cost of increased computation time due to the two compression
passes. This data demonstrates that the feature is engaged and provides
real compression benefits when --no-reuse-delta forces fresh deltas.
Co-Authored-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
Documentation/git-pack-objects.adoc | 6 +--
builtin/pack-objects.c | 2 +-
path-walk.c | 30 ++++++++++++++
path-walk.h | 7 ++++
t/helper/test-path-walk.c | 11 ++++-
t/t5620-backfill.sh | 9 -----
t/t6601-path-walk.sh | 62 +++++++++++++++++++++++++++++
7 files changed, 113 insertions(+), 14 deletions(-)
diff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc
index b78175fbe1..917045d5c3 100644
--- a/Documentation/git-pack-objects.adoc
+++ b/Documentation/git-pack-objects.adoc
@@ -402,9 +402,9 @@ will be automatically changed to version `1`.
of filenames that cause collisions in Git's default name-hash
algorithm.
+
-Incompatible with `--delta-islands`, `--shallow`, or `--filter`. The
-`--use-bitmap-index` option will be ignored in the presence of
-`--path-walk.`
+Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
+ignored in the presence of `--path-walk`. Whe `--path-walk` option
+supports the `--filter=<spec>` form `blob:none`.
DELTA ISLANDS
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 4338962904..bc9fb5b457 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -5177,7 +5177,7 @@ int cmd_pack_objects(int argc,
if (path_walk) {
const char *option = NULL;
- if (filter_options.choice)
+ if (!path_walk_filter_compatible(&filter_options))
option = "--filter";
else if (use_delta_islands)
option = "--delta-islands";
diff --git a/path-walk.c b/path-walk.c
index 59a7670c5b..cc587268e7 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -9,6 +9,7 @@
#include "hashmap.h"
#include "hex.h"
#include "list-objects.h"
+#include "list-objects-filter-options.h"
#include "object.h"
#include "oid-array.h"
#include "path.h"
@@ -494,6 +495,32 @@ static int setup_pending_objects(struct path_walk_info *info,
return 0;
}
+static int prepare_filters(struct path_walk_info *info,
+ struct list_objects_filter_options *options)
+{
+ switch (options->choice) {
+ case LOFC_DISABLED:
+ return 1;
+
+ case LOFC_BLOB_NONE:
+ if (info) {
+ info->blobs = 0;
+ list_objects_filter_release(options);
+ }
+ return 1;
+
+ default:
+ error(_("object filter '%s' not supported by the path-walk API"),
+ list_objects_filter_spec(options));
+ return 0;
+ }
+}
+
+int path_walk_filter_compatible(struct list_objects_filter_options *options)
+{
+ return prepare_filters(NULL, options);
+}
+
/**
* Given the configuration of 'info', walk the commits based on 'info->revs' and
* call 'info->path_fn' on each discovered path.
@@ -521,6 +548,9 @@ int walk_objects_by_path(struct path_walk_info *info)
trace2_region_enter("path-walk", "commit-walk", info->revs->repo);
+ if (!prepare_filters(info, &info->revs->filter))
+ return -1;
+
CALLOC_ARRAY(commit_list, 1);
commit_list->type = OBJ_COMMIT;
diff --git a/path-walk.h b/path-walk.h
index 657eeda8ec..a1736ecb2b 100644
--- a/path-walk.h
+++ b/path-walk.h
@@ -90,3 +90,10 @@ void path_walk_info_clear(struct path_walk_info *info);
* Returns nonzero on an error.
*/
int walk_objects_by_path(struct path_walk_info *info);
+
+struct list_objects_filter_options;
+/**
+ * Given a set of options for filtering objects, return 1 if the options
+ * are compatible with the path-walk API and 0 otherwise.
+ */
+int path_walk_filter_compatible(struct list_objects_filter_options *options);
diff --git a/t/helper/test-path-walk.c b/t/helper/test-path-walk.c
index fe63002c2b..88f86ae0dc 100644
--- a/t/helper/test-path-walk.c
+++ b/t/helper/test-path-walk.c
@@ -4,6 +4,7 @@
#include "dir.h"
#include "environment.h"
#include "hex.h"
+#include "list-objects-filter-options.h"
#include "object-name.h"
#include "object.h"
#include "pretty.h"
@@ -71,6 +72,8 @@ int cmd__path_walk(int argc, const char **argv)
struct rev_info revs = REV_INFO_INIT;
struct path_walk_info info = PATH_WALK_INFO_INIT;
struct path_walk_test_data data = { 0 };
+ struct list_objects_filter_options filter_options =
+ LIST_OBJECTS_FILTER_INIT;
struct option options[] = {
OPT_BOOL(0, "blobs", &info.blobs,
N_("toggle inclusion of blob objects")),
@@ -86,11 +89,12 @@ int cmd__path_walk(int argc, const char **argv)
N_("toggle aggressive edge walk")),
OPT_BOOL(0, "stdin-pl", &stdin_pl,
N_("read a pattern list over stdin")),
+ OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
OPT_END(),
};
setup_git_directory();
- revs.repo = the_repository;
+ repo_init_revisions(the_repository, &revs, NULL);
argc = parse_options(argc, argv, NULL,
options, path_walk_usage,
@@ -101,6 +105,10 @@ int cmd__path_walk(int argc, const char **argv)
else
usage(path_walk_usage[0]);
+ /* Apply the filter after setup_revisions to avoid the --objects check. */
+ if (filter_options.choice)
+ list_objects_filter_copy(&revs.filter, &filter_options);
+
info.revs = &revs;
info.path_fn = emit_block;
info.path_fn_data = &data;
@@ -129,6 +137,7 @@ int cmd__path_walk(int argc, const char **argv)
free(info.pl);
}
+ list_objects_filter_release(&filter_options);
release_revisions(&revs);
return res;
}
diff --git a/t/t5620-backfill.sh b/t/t5620-backfill.sh
index e174290787..94f35ce190 100755
--- a/t/t5620-backfill.sh
+++ b/t/t5620-backfill.sh
@@ -298,9 +298,6 @@ test_expect_success 'backfill with prefix pathspec' '
git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
test_line_count = 48 missing &&
- # If we enable --path-walk here, we will get a warning overs stderr
- # due to incompatibilities with --filter.
- GIT_TEST_PACK_PATH_WALK=0 \
git -C backfill-path backfill HEAD -- d/f 2>err &&
test_must_be_empty err &&
@@ -318,9 +315,6 @@ test_expect_success 'backfill with multiple pathspecs' '
git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
test_line_count = 48 missing &&
- # If we enable --path-walk here, we will get a warning overs stderr
- # due to incompatibilities with --filter.
- GIT_TEST_PACK_PATH_WALK=0 \
git -C backfill-path backfill HEAD -- d/f a 2>err &&
test_must_be_empty err &&
@@ -338,9 +332,6 @@ test_expect_success 'backfill with wildcard pathspec' '
git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
test_line_count = 48 missing &&
- # If we enable --path-walk here, we will get a warning overs stderr
- # due to incompatibilities with --filter.
- GIT_TEST_PACK_PATH_WALK=0 \
git -C backfill-path backfill HEAD -- "d/file.*.txt" 2>err &&
test_must_be_empty err &&
diff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh
index 56bd1e3c5b..b0ee31ee2d 100755
--- a/t/t6601-path-walk.sh
+++ b/t/t6601-path-walk.sh
@@ -415,4 +415,66 @@ test_expect_success 'trees are reported exactly once' '
test_line_count = 1 out-filtered
'
+test_expect_success 'all, blob:none filter' '
+ test-tool path-walk --filter=blob:none -- --all >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tag:/tags:$(git rev-parse refs/tags/first)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.1)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.2)
+ 1:tag:/tags:$(git rev-parse refs/tags/third)
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
+ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
+ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
+ 3:tree::$(git rev-parse topic^{tree})
+ 3:tree::$(git rev-parse base^{tree})
+ 3:tree::$(git rev-parse base~1^{tree})
+ 3:tree::$(git rev-parse base~2^{tree})
+ 3:tree::$(git rev-parse refs/tags/tree-tag^{})
+ 3:tree::$(git rev-parse refs/tags/tree-tag2^{})
+ 4:tree:a/:$(git rev-parse base:a)
+ 5:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
+ 6:tree:left/:$(git rev-parse base:left)
+ 6:tree:left/:$(git rev-parse base~2:left)
+ 7:tree:right/:$(git rev-parse topic:right)
+ 7:tree:right/:$(git rev-parse base~1:right)
+ 7:tree:right/:$(git rev-parse base~2:right)
+ blobs:2
+ commits:4
+ tags:7
+ trees:13
+ EOF
+
+ test_cmp_sorted expect out
+'
+
+test_expect_success 'topic only, blob:none filter' '
+ test-tool path-walk --filter=blob:none -- topic >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tree::$(git rev-parse topic^{tree})
+ 1:tree::$(git rev-parse base~1^{tree})
+ 1:tree::$(git rev-parse base~2^{tree})
+ 2:tree:left/:$(git rev-parse base~2:left)
+ 3:tree:right/:$(git rev-parse topic:right)
+ 3:tree:right/:$(git rev-parse base~1:right)
+ 3:tree:right/:$(git rev-parse base~2:right)
+ blobs:0
+ commits:3
+ tags:0
+ trees:7
+ EOF
+
+ test_cmp_sorted expect out
+'
+
test_done
--
gitgitgadget
^ permalink raw reply related
* [PATCH v3 04/12] path-walk: always emit directly-requested objects
From: Derrick Stolee via GitGitGadget @ 2026-05-11 18:13 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Taylor Blau, Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.v3.git.1778523189.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
We are preparing to integrate the path-walk API with some --filter options
in 'git pack-objects', but there is a subtle issue that is revealed when
those are put together and the test suite is run with
GIT_TEST_PACK_PATH_WALK=1.
When a filter reduces the set of requested objects, this results in
filtering out directly-requested objects, such as in the download of needed
blobs in a blobless partial clone.
The root cause is that the scan of pending objects in the path-walk API
respects the filters set in the path_walk_info instead of overriding them
for pending objects.
We can tell that a path is part of the directly-referenced objects if its
path name starts with '/' (other paths, including root trees never have this
starting character). Create a path_is_for_direct_objects() to make this
meaning clear, especially as we add more references in the future as we
integrate the path-walk API with partial clone filter options.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
Documentation/technical/api-path-walk.adoc | 7 ++++
path-walk.c | 41 ++++++++++++++--------
path-walk.h | 5 +++
3 files changed, 38 insertions(+), 15 deletions(-)
diff --git a/Documentation/technical/api-path-walk.adoc b/Documentation/technical/api-path-walk.adoc
index a67de1b143..6e17b13d61 100644
--- a/Documentation/technical/api-path-walk.adoc
+++ b/Documentation/technical/api-path-walk.adoc
@@ -48,6 +48,13 @@ commits.
applications could disable some options to make it simpler to walk
the objects or to have fewer calls to `path_fn`.
+
+Note that objects directly requested as pending objects (such as targets
+of lightweight tags or other ref tips) are always emitted to `path_fn`,
+even when the corresponding type flag is disabled. Only objects
+discovered during the tree walk are subject to these type filters. This
+ensures that objects specifically requested through the revision input
+are never silently dropped.
++
While it is possible to walk only commits in this way, consumers would be
better off using the revision walk API instead.
diff --git a/path-walk.c b/path-walk.c
index 6e426af433..59a7670c5b 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -248,6 +248,16 @@ static int add_tree_entries(struct path_walk_context *ctx,
return 0;
}
+/*
+ * Paths starting with '/' (e.g., "/tags", "/tagged-blobs") hold objects that
+ * were directly requested by 'pending' objects rather than discovered during
+ * tree traversal.
+ */
+static int path_is_for_direct_objects(const char *path)
+{
+ return path[0] == '/';
+}
+
/*
* For each path in paths_to_explore, walk the trees another level
* and add any found blobs to the batch (but only if they exist and
@@ -306,14 +316,19 @@ static int walk_path(struct path_walk_context *ctx,
if (list->type == OBJ_BLOB &&
ctx->revs->prune_data.nr &&
+ !path_is_for_direct_objects(path) &&
!match_pathspec(ctx->repo->index, &ctx->revs->prune_data,
path, strlen(path), 0,
NULL, 0))
return 0;
- /* Evaluate function pointer on this data, if requested. */
- if ((list->type == OBJ_TREE && ctx->info->trees) ||
- (list->type == OBJ_BLOB && ctx->info->blobs) ||
+ /*
+ * Evaluate function pointer on this data, if requested.
+ * Ignore object type filters for tagged objects (path starts
+ * with `/`).
+ */
+ if ((list->type == OBJ_TREE && (ctx->info->trees || path_is_for_direct_objects(path))) ||
+ (list->type == OBJ_BLOB && (ctx->info->blobs || path_is_for_direct_objects(path))) ||
(list->type == OBJ_TAG && ctx->info->tags))
ret = ctx->info->path_fn(path, &list->oids, list->type,
ctx->info->path_fn_data);
@@ -374,10 +389,8 @@ static int setup_pending_objects(struct path_walk_info *info,
if (info->tags)
CALLOC_ARRAY(tags, 1);
- if (info->blobs)
- CALLOC_ARRAY(tagged_blobs, 1);
- if (info->trees)
- root_tree_list = strmap_get(&ctx->paths_to_lists, root_path);
+ CALLOC_ARRAY(tagged_blobs, 1);
+ root_tree_list = strmap_get(&ctx->paths_to_lists, root_path);
/*
* Pending objects include:
@@ -421,8 +434,6 @@ static int setup_pending_objects(struct path_walk_info *info,
switch (obj->type) {
case OBJ_TREE:
- if (!info->trees)
- continue;
if (pending->path) {
char *path = *pending->path ? xstrfmt("%s/", pending->path)
: xstrdup("");
@@ -435,8 +446,6 @@ static int setup_pending_objects(struct path_walk_info *info,
break;
case OBJ_BLOB:
- if (!info->blobs)
- continue;
if (pending->path)
add_path_to_list(ctx, pending->path, OBJ_BLOB, &obj->oid, 1);
else
@@ -532,15 +541,17 @@ int walk_objects_by_path(struct path_walk_info *info)
push_to_stack(&ctx, root_path);
/*
- * Set these values before preparing the walk to catch
- * lightweight tags pointing to non-commits and indexed objects.
+ * Ensure that prepare_revision_walk() keeps all pending objects
+ * even through an object type filter.
*/
- info->revs->blob_objects = info->blobs;
- info->revs->tree_objects = info->trees;
+ info->revs->blob_objects = info->revs->tree_objects = 1;
if (prepare_revision_walk(info->revs))
die(_("failed to setup revision walk"));
+ info->revs->blob_objects = info->blobs;
+ info->revs->tree_objects = info->trees;
+
/*
* Walk trees to mark them as UNINTERESTING.
* This is particularly important when 'edge_aggressive' is set.
diff --git a/path-walk.h b/path-walk.h
index 5ef5a8440e..657eeda8ec 100644
--- a/path-walk.h
+++ b/path-walk.h
@@ -36,6 +36,11 @@ struct path_walk_info {
/**
* Initialize which object types the path_fn should be called on. This
* could also limit the walk to skip blobs if not set.
+ *
+ * Note: even when 'blobs' or 'trees' is disabled, objects that are
+ * directly requested as pending objects will still be emitted to
+ * path_fn. Only objects discovered during the tree walk are filtered by
+ * these flags.
*/
int commits;
int trees;
--
gitgitgadget
^ permalink raw reply related
* [PATCH v3 03/12] t/perf: add pack-objects filter and path-walk benchmark
From: Derrick Stolee via GitGitGadget @ 2026-05-11 18:13 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Taylor Blau, Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.v3.git.1778523189.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
Add p5315-pack-objects-filter.sh to measure the performance of
'git pack-objects --revs --all' under different filter and traversal
combinations:
* no filter (baseline)
* --filter=blob:none (blobless)
* --filter=sparse:oid=<oid> (cone-mode sparse)
Each filter scenario is tested both with and without --path-walk,
producing paired measurements that show the impact of the path-walk
traversal for each filter type as we integrate the --path-walk feature
with different --filter options. It currently has no integration so
falls back to the standard revision walk. Thus, there are no significant
differences in the current results other than a full repack (and even
then, the --path-walk feature is not incredibly different for the
default Git repository):
Test HEAD
-----------------------------------------------------
5315.2: repack (no filter) 27.91
5315.3: repack size (no filter) 250.7M
5315.4: repack (no filter, --path-walk) 34.92
5315.5: repack size (no filter, --path-walk) 220.0M
5315.6: repack (blob:none) 13.63
5315.7: repack size (blob:none) 137.6M
5315.8: repack (blob:none, --path-walk) 13.48
5315.9: repack size (blob:none, --path-walk) 137.7M
5315.10: repack (sparse:oid) 72.67
5315.11: repack size (sparse:oid) 187.4M
5315.12: repack (sparse:oid, --path-walk) 72.47
5315.13: repack size (sparse:oid, --path-walk) 187.4M
The sparse filter definition is built automatically by sampling
depth-2 directories from the test repository, making the test work
on any repo passed via GIT_PERF_LARGE_REPO. For repos that lack
depth-2 directories, a single top-level directory is used; for flat
repos, the sparse tests are skipped via prerequisite.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
t/perf/p5315-pack-objects-filter.sh | 129 ++++++++++++++++++++++++++++
1 file changed, 129 insertions(+)
create mode 100755 t/perf/p5315-pack-objects-filter.sh
diff --git a/t/perf/p5315-pack-objects-filter.sh b/t/perf/p5315-pack-objects-filter.sh
new file mode 100755
index 0000000000..b009039c89
--- /dev/null
+++ b/t/perf/p5315-pack-objects-filter.sh
@@ -0,0 +1,129 @@
+#!/bin/sh
+
+test_description='Tests pack-objects performance with filters and --path-walk'
+. ./perf-lib.sh
+
+test_perf_large_repo
+
+test_expect_success 'setup filter inputs' '
+ # Sample a few depth-2 directories from the test repo to build
+ # a cone-mode sparse-checkout definition. The sampling picks
+ # directories at evenly-spaced positions so the choice is stable
+ # and scales to repos of any shape.
+
+ git ls-tree -d --name-only HEAD >top-dirs &&
+ top_nr=$(wc -l <top-dirs) &&
+
+ >depth2-dirs &&
+ while read tdir
+ do
+ git ls-tree -d --name-only "HEAD:$tdir" 2>/dev/null |
+ sed "s|^|$tdir/|" >>depth2-dirs || return 1
+ done <top-dirs &&
+
+ d2_nr=$(wc -l <depth2-dirs) &&
+
+ if test "$d2_nr" -ge 2
+ then
+ # Pick two directories from evenly-spaced positions.
+ first=$(sed -n "1p" depth2-dirs) &&
+ mid=$(sed -n "$((d2_nr / 2 + 1))p" depth2-dirs) &&
+
+ p1=$(dirname "$first") &&
+ p2=$(dirname "$mid") &&
+
+ # Build cone-mode sparse-checkout patterns.
+ {
+ echo "/*" &&
+ echo "!/*/" &&
+ echo "/$p1/" &&
+ echo "!/$p1/*/" &&
+ if test "$p1" != "$p2"
+ then
+ echo "/$p2/" &&
+ echo "!/$p2/*/"
+ fi &&
+ echo "/$first/" &&
+ if test "$first" != "$mid"
+ then
+ echo "/$mid/"
+ fi
+ } >sparse-patterns &&
+
+ git hash-object -w sparse-patterns >sparse-oid &&
+ echo "Sparse cone: $first $mid" &&
+ cat sparse-patterns &&
+ test_set_prereq SPARSE_OID
+ elif test "$top_nr" -ge 1
+ then
+ # Fallback: use a single top-level directory.
+ first=$(sed -n "1p" top-dirs) &&
+ {
+ echo "/*" &&
+ echo "!/*/" &&
+ echo "/$first/"
+ } >sparse-patterns &&
+
+ git hash-object -w sparse-patterns >sparse-oid &&
+ echo "Sparse cone: $first" &&
+ cat sparse-patterns &&
+ test_set_prereq SPARSE_OID
+ fi
+'
+
+test_perf 'repack (no filter)' '
+ git pack-objects --stdout --no-reuse-delta --revs --all </dev/null >pk
+'
+
+test_size 'repack size (no filter)' '
+ test_file_size pk
+'
+
+test_perf 'repack (no filter, --path-walk)' '
+ git pack-objects --stdout --no-reuse-delta --revs --all --path-walk </dev/null >pk
+'
+
+test_size 'repack size (no filter, --path-walk)' '
+ test_file_size pk
+'
+
+test_perf 'repack (blob:none)' '
+ git pack-objects --stdout --no-reuse-delta --revs --all --filter=blob:none </dev/null >pk
+'
+
+test_size 'repack size (blob:none)' '
+ test_file_size pk
+'
+
+test_perf 'repack (blob:none, --path-walk)' '
+ git pack-objects --stdout --no-reuse-delta --revs --all --path-walk \
+ --filter=blob:none </dev/null >pk
+'
+
+test_size 'repack size (blob:none, --path-walk)' '
+ test_file_size pk
+'
+
+test_perf 'repack (sparse:oid)' \
+ --prereq SPARSE_OID '
+ git pack-objects --stdout --no-reuse-delta --revs --all \
+ --filter=sparse:oid=$(cat sparse-oid) </dev/null >pk
+'
+
+test_size 'repack size (sparse:oid)' \
+ --prereq SPARSE_OID '
+ test_file_size pk
+'
+
+test_perf 'repack (sparse:oid, --path-walk)' \
+ --prereq SPARSE_OID '
+ git pack-objects --stdout --no-reuse-delta --revs --all --path-walk \
+ --filter=sparse:oid=$(cat sparse-oid) </dev/null >pk
+'
+
+test_size 'repack size (sparse:oid, --path-walk)' \
+ --prereq SPARSE_OID '
+ test_file_size pk
+'
+
+test_done
--
gitgitgadget
^ permalink raw reply related
* [PATCH v3 02/12] pack-objects: pass --objects with --path-walk
From: Derrick Stolee via GitGitGadget @ 2026-05-11 18:12 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Taylor Blau, Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.v3.git.1778523189.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
When 'git pack-objects' has the --path-walk option enabled, it uses a
different set of revision walk parameters than normal. For once,
--objects was previously assumed by the path-walk API and was not needed
to be added. We also needed --boundary to allow discovering
UNINTERESTING objects to use as delta bases.
We will be updating the path-walk API soon to work with some filter
options. However, the revision machinery will trigger a fatal error:
fatal: object filtering requires --objects
The fix is easy: add the --objects option as an argument. This has no
effect on the path-walk API but does simplify the revision option
parsing for the objects filter.
We can remove the comment about "removing" the options because they were
never removed and instead not added. We still need to disable using
bitmaps.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
builtin/pack-objects.c | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index dd2480a73d..4338962904 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -5190,10 +5190,7 @@ int cmd_pack_objects(int argc,
}
if (path_walk) {
strvec_push(&rp, "--boundary");
- /*
- * We must disable the bitmaps because we are removing
- * the --objects / --objects-edge[-aggressive] options.
- */
+ strvec_push(&rp, "--objects");
use_bitmap_index = 0;
} else if (thin) {
use_internal_rev_list = 1;
--
gitgitgadget
^ permalink raw reply related
* [PATCH v3 01/12] t5620: make test work with path-walk var
From: Derrick Stolee via GitGitGadget @ 2026-05-11 18:12 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Taylor Blau, Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.v3.git.1778523189.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
The GIT_TEST_PACK_PATH_WALK test variable allows enabling the
--path-walk option to 'git pack-objects' by default. This sometimes
engages the warning that --path-walk is incompatible with the --filter
option. These tests in t5620 fail due to this warning over stderr in
this case. Disable this variable for this moment until these options
work together.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
t/t5620-backfill.sh | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/t/t5620-backfill.sh b/t/t5620-backfill.sh
index 94f35ce190..e174290787 100755
--- a/t/t5620-backfill.sh
+++ b/t/t5620-backfill.sh
@@ -298,6 +298,9 @@ test_expect_success 'backfill with prefix pathspec' '
git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
test_line_count = 48 missing &&
+ # If we enable --path-walk here, we will get a warning overs stderr
+ # due to incompatibilities with --filter.
+ GIT_TEST_PACK_PATH_WALK=0 \
git -C backfill-path backfill HEAD -- d/f 2>err &&
test_must_be_empty err &&
@@ -315,6 +318,9 @@ test_expect_success 'backfill with multiple pathspecs' '
git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
test_line_count = 48 missing &&
+ # If we enable --path-walk here, we will get a warning overs stderr
+ # due to incompatibilities with --filter.
+ GIT_TEST_PACK_PATH_WALK=0 \
git -C backfill-path backfill HEAD -- d/f a 2>err &&
test_must_be_empty err &&
@@ -332,6 +338,9 @@ test_expect_success 'backfill with wildcard pathspec' '
git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
test_line_count = 48 missing &&
+ # If we enable --path-walk here, we will get a warning overs stderr
+ # due to incompatibilities with --filter.
+ GIT_TEST_PACK_PATH_WALK=0 \
git -C backfill-path backfill HEAD -- "d/file.*.txt" 2>err &&
test_must_be_empty err &&
--
gitgitgadget
^ permalink raw reply related
* [PATCH v3 00/12] pack-objects: integrate --path-walk and some --filter options
From: Derrick Stolee via GitGitGadget @ 2026-05-11 18:12 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Taylor Blau, Derrick Stolee
In-Reply-To: <pull.2101.v2.git.1777926079.gitgitgadget@gmail.com>
NOTE: This series is based on en/backfill-fixes-and-edges.
The 'git pack-objects' command has a '--path-walk' option that uses the
path-walk API instead of a typical revision walk to group objects into
chunks by path name instead of relying solely on name-hashes to group
similar files together. (It also does a second compression pass looking for
better deltas after the first pass that is focused within chunks per path.)
The '--path-walk' feature was not previously integrated with the '--filter'
feature, so a warning would appear and disable the path-walk API when a
filter is given. This patch series integrates these together in the
following ways:
* --filter=blob:none updates the path-walk API options to skip blobs.
* --filter=blob:limit=<size> adds a scan to a list of blob objects to
remove objects that are too large.
* --filter=sparse:<oid> adds a scan to the chunks to validate that the
paths match the sparse-checkout patterns.
In particular, this last check is significantly faster than the previous
algorithm because it can check all objects at a given path simultaneously
instead of checking all sparse-checkout patterns for each discovered blob
object.
A subtlety must be added here, in that we must change how we mark an object
as "seen" during the path-walk. We may need to add an object to multiple
paths and only mark it as "seen" if it indeed matched the sparse-checkout
patterns as the path is accepted for emitting to the callback. This adds a
new filter that the "seen" objects must also be removed from later chunks to
avoid sending the same object as grouped to multiple chunks.
There's also a subtle detail here in that the path-walk API also prunes tree
paths based on cone-mode sparse-checkouts, to enable 'git backfill --sparse'
operating quickly for small sparse-checkout scopes. But the
--filter=sparse:<oid> feature doesn't prune trees!
As a future step, I do plan to recommend that we add a treesparse:<oid>
setting that does allow us to trim the tree set by cone-mode sparse
patterns. At the time that partial clone filters were being created, cone
mode sparse-checkout didn't exist and neither did the sparse index. Those
features together make a smaller tree set possible, assuming the user never
needs to change their scope. This would be a significant change so it is not
implemented here, though the git pack-objects integration would be quick
after this series completes.
Neither the sparse:<oid> or hypothetical treesparse:<oid> options are or
should necessarily be supported by Git servers. It's too expensive to
compute dynamically and it doesn't work well with reachability bitmaps. What
becomes possible with this change is that it becomes reasonably fast to
construct bundles with these filters that can bootstrap a working
environment with the full history of all files within a given
sparse-checkout scope.
Performance Results
===================
Since the '--path-walk' option is ignored in today's Git version when a
'--filter' is added, the performance matches the behavior without
'--path-walk'. For the tables below, you can compare the rows against each
other (time and then packfile size) for the mode without and then with
'--path-walk' as a representation of "before" and "after". (These tables are
repeated in the commit messages as new implementations improve specific
rows.)
I chose a number of open source repositories of various sizes and shapes:
git/git
=======
Test HEAD
-------------------------------------------------------------------
5315.2: repack (no filter) 27.73
5315.3: repack size (no filter) 250.6M
5315.4: repack (no filter, --path-walk) 35.19
5315.5: repack size (no filter, --path-walk) 220.1M
5315.6: repack (blob:none) 13.42
5315.7: repack size (blob:none) 137.6M
5315.8: repack (blob:none, --path-walk) 20.98
5315.9: repack size (blob:none, --path-walk) 115.2M
5315.10: repack (sparse:oid) 72.53
5315.11: repack size (sparse:oid) 187.5M
5315.12: repack (sparse:oid, --path-walk) 29.00
5315.13: repack size (sparse:oid, --path-walk) 161.0M
nodejs/node
===========
Test HEAD
--------------------------------------------------------------------
5315.2: repack (no filter) 75.53
5315.3: repack size (no filter) 0.9G
5315.4: repack (no filter, --path-walk) 80.54
5315.5: repack size (no filter, --path-walk) 885.7M
5315.6: repack (blob:none) 12.65
5315.7: repack size (blob:none) 148.6M
5315.8: repack (blob:none, --path-walk) 17.60
5315.9: repack size (blob:none, --path-walk) 134.6M
5315.10: repack (sparse:oid) 518.84
5315.11: repack size (sparse:oid) 153.4M
5315.12: repack (sparse:oid, --path-walk) 27.99
5315.13: repack size (sparse:oid, --path-walk) 139.4M
microsoft/fluentui
==================
Test HEAD
--------------------------------------------------------------------
5315.2: repack (no filter) 146.77
5315.3: repack size (no filter) 562.1M
5315.4: repack (no filter, --path-walk) 72.82
5315.5: repack size (no filter, --path-walk) 172.6M
5315.6: repack (blob:none) 4.84
5315.7: repack size (blob:none) 62.7M
5315.8: repack (blob:none, --path-walk) 5.19
5315.9: repack size (blob:none, --path-walk) 59.9M
5315.10: repack (sparse:oid) 59.95
5315.11: repack size (sparse:oid) 85.6M
5315.12: repack (sparse:oid, --path-walk) 15.16
5315.13: repack size (sparse:oid, --path-walk) 72.5M
microsoftdocs/azure-devops-docs
===============================
Test HEAD
---------------------------------------------------------------------
5315.2: repack (no filter) 4.41
5315.3: repack size (no filter) 1.6G
5315.4: repack (no filter, --path-walk) 6.00
5315.5: repack size (no filter, --path-walk) 1.6G
5315.6: repack (blob:none) 1.35
5315.7: repack size (blob:none) 60.0M
5315.8: repack (blob:none, --path-walk) 1.23
5315.9: repack size (blob:none, --path-walk) 60.0M
5315.10: repack (sparse:oid) 138.24
5315.11: repack size (sparse:oid) 84.4M
5315.12: repack (sparse:oid, --path-walk) 1.86
5315.13: repack size (sparse:oid, --path-walk) 84.4M
Performance conclusions
=======================
As seen in earlier series around the '--path-walk' feature, the space
savings can be valuable but is not always guaranteed. When the space savings
doesn't happen, then the time spent is generally slower because of the
two-pass mechanism. The microsoftdocs/azure-devops-docs repo demonstrates
this case quite clearly.
However, even in these cases the 'sparse:<oid>' filters are much faster
because of the ability to check an entire set of objects against the
sparse-checkout patterns only once.
Thanks, -Stolee
UPDATES IN V2
=============
* Rebased onto en/backfill-fixes-and-edges to properly integrate with the
incompatible rev-list options logic in that series.
* Updated documentation as behavior changes. Credit to Taylor Blau for
presenting these suggestions in his RFC [2].
* Added three patches of Taylor's to extend more filter options.
UPDATES IN V3
=============
Upon realizing that the tests were not passing with
GIT_TEST_PACK_PATH_WALK=1, I spent a lot of time reworking each patch to
pass all tests with that variable enabled. This led to a lot of meaningful
changes:
* A new patch updates t5620-backfill.sh because they are currently failing
due to a check for quiet stderr checks. These changes are reverted later
when the filters are integrated so the warning stops being written.
* I move the logic for the path-walk API emitting "directly requested"
objects (non-commits in the 'pending' list). This is substantial enough
to be its own patch.
* The filtering logic is pulled entirely within the path-walk API instead
of needing integration within builtin/pack-objects.c.
* The tree:0 filter had a lot to be desired when fetching missing objects,
so is substantially updated.
* The object-type filter requires a change to the typical direct-request
behavior, including a new 'strict_types' member that prevents ever
allowing objects against type.
* The combine filter needed better logic around multiple blob size limits,
to take the smaller of the two.
* The t6601-path-walk.sh script has many test updates to better reflect the
new behavior, as required by the other partial clone tests under
GIT_TEST_PACK_PATH_WALK=1.
* Doc updates for 'form' to 'forms' when multiple forms are supported.
I've also updated Taylor's bitmap-related patches into three commits on top
of this series (see [3]).
P.S. I've CC'd the folks who were on the original path-walk feature thread
[1]
[1]
https://lore.kernel.org/git/pull.1819.git.1741571455.gitgitgadget@gmail.com/
[2] https://lore.kernel.org/git/cover.1777853408.git.me@ttaylorr.com/
[3]
https://github.com/derrickstolee/git/compare/path-walk-filters...derrickstolee:git:path-walk-bitmaps
Derrick Stolee (9):
t5620: make test work with path-walk var
pack-objects: pass --objects with --path-walk
t/perf: add pack-objects filter and path-walk benchmark
path-walk: always emit directly-requested objects
path-walk: support blobless filter
backfill: die on incompatible filter options
path-walk: support blob size limit filter
path-walk: add pl_sparse_trees to control tree pruning
pack-objects: support sparse:oid filter with path-walk
Taylor Blau (3):
path-walk: support `tree:0` filter
path-walk: support `object:type` filter
path-walk: support `combine` filter
Documentation/git-backfill.adoc | 4 +
Documentation/git-pack-objects.adoc | 8 +-
Documentation/technical/api-path-walk.adoc | 7 +
builtin/backfill.c | 8 +-
builtin/pack-objects.c | 23 +-
path-walk.c | 260 ++++++++++-
path-walk.h | 31 ++
t/helper/test-path-walk.c | 17 +-
t/perf/p5315-pack-objects-filter.sh | 129 ++++++
t/t5317-pack-objects-filter-objects.sh | 125 +++++
t/t5620-backfill.sh | 8 +
t/t6601-path-walk.sh | 516 +++++++++++++++++++++
12 files changed, 1093 insertions(+), 43 deletions(-)
create mode 100755 t/perf/p5315-pack-objects-filter.sh
base-commit: a1ad4a0fca14cdeb55ab9fb065551b15cafa8a4f
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2101%2Fderrickstolee%2Fpath-walk-filters-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2101/derrickstolee/path-walk-filters-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/2101
Range-diff vs v2:
-: ---------- > 1: 0840110116 t5620: make test work with path-walk var
1: b361680338 = 2: 5cc6383380 pack-objects: pass --objects with --path-walk
2: 9ea81aa726 = 3: 77329cf8f4 t/perf: add pack-objects filter and path-walk benchmark
-: ---------- > 4: 50933cccf5 path-walk: always emit directly-requested objects
3: ed4d277a2c ! 5: b2deb7f819 path-walk: support blobless filter
@@ Commit message
Update the test helper (test-path-walk) to accept --filter=<spec>
as a test-tool option (before '--'), applying it to revs after
- setup_revisions() to avoid the --objects requirement check.
+ setup_revisions() to avoid the --objects requirement check. We can also
+ revert recent GIT_TEST_PACK_PATH_WALK overrides in t5620.
Also switch test-path-walk from REV_INFO_INIT with manual repo
assignment to repo_init_revisions(), which properly initializes
@@ t/helper/test-path-walk.c: int cmd__path_walk(int argc, const char **argv)
return res;
}
+ ## t/t5620-backfill.sh ##
+@@ t/t5620-backfill.sh: test_expect_success 'backfill with prefix pathspec' '
+ git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
+ test_line_count = 48 missing &&
+
+- # If we enable --path-walk here, we will get a warning overs stderr
+- # due to incompatibilities with --filter.
+- GIT_TEST_PACK_PATH_WALK=0 \
+ git -C backfill-path backfill HEAD -- d/f 2>err &&
+ test_must_be_empty err &&
+
+@@ t/t5620-backfill.sh: test_expect_success 'backfill with multiple pathspecs' '
+ git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
+ test_line_count = 48 missing &&
+
+- # If we enable --path-walk here, we will get a warning overs stderr
+- # due to incompatibilities with --filter.
+- GIT_TEST_PACK_PATH_WALK=0 \
+ git -C backfill-path backfill HEAD -- d/f a 2>err &&
+ test_must_be_empty err &&
+
+@@ t/t5620-backfill.sh: test_expect_success 'backfill with wildcard pathspec' '
+ git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
+ test_line_count = 48 missing &&
+
+- # If we enable --path-walk here, we will get a warning overs stderr
+- # due to incompatibilities with --filter.
+- GIT_TEST_PACK_PATH_WALK=0 \
+ git -C backfill-path backfill HEAD -- "d/file.*.txt" 2>err &&
+ test_must_be_empty err &&
+
+
## t/t6601-path-walk.sh ##
@@ t/t6601-path-walk.sh: test_expect_success 'trees are reported exactly once' '
test_line_count = 1 out-filtered
@@ t/t6601-path-walk.sh: test_expect_success 'trees are reported exactly once' '
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
-+ 2:tree::$(git rev-parse topic^{tree})
-+ 2:tree::$(git rev-parse base^{tree})
-+ 2:tree::$(git rev-parse base~1^{tree})
-+ 2:tree::$(git rev-parse base~2^{tree})
-+ 2:tree::$(git rev-parse refs/tags/tree-tag^{})
-+ 2:tree::$(git rev-parse refs/tags/tree-tag2^{})
-+ 3:tree:a/:$(git rev-parse base:a)
-+ 4:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
-+ 5:tree:left/:$(git rev-parse base:left)
-+ 5:tree:left/:$(git rev-parse base~2:left)
-+ 6:tree:right/:$(git rev-parse topic:right)
-+ 6:tree:right/:$(git rev-parse base~1:right)
-+ 6:tree:right/:$(git rev-parse base~2:right)
-+ blobs:0
++ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
++ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
++ 3:tree::$(git rev-parse topic^{tree})
++ 3:tree::$(git rev-parse base^{tree})
++ 3:tree::$(git rev-parse base~1^{tree})
++ 3:tree::$(git rev-parse base~2^{tree})
++ 3:tree::$(git rev-parse refs/tags/tree-tag^{})
++ 3:tree::$(git rev-parse refs/tags/tree-tag2^{})
++ 4:tree:a/:$(git rev-parse base:a)
++ 5:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
++ 6:tree:left/:$(git rev-parse base:left)
++ 6:tree:left/:$(git rev-parse base~2:left)
++ 7:tree:right/:$(git rev-parse topic:right)
++ 7:tree:right/:$(git rev-parse base~1:right)
++ 7:tree:right/:$(git rev-parse base~2:right)
++ blobs:2
+ commits:4
+ tags:7
+ trees:13
4: da4c19af42 = 6: da191e29e6 backfill: die on incompatible filter options
5: d309345fec ! 7: a1ab70405d path-walk: support blob size limit filter
@@ Documentation/git-pack-objects.adoc: will be automatically changed to version `1
Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
ignored in the presence of `--path-walk`. Whe `--path-walk` option
-supports the `--filter=<spec>` form `blob:none`.
-+supports the `--filter=<spec>` form `blob:none` and `blob:limit=<n>`.
++supports the `--filter=<spec>` forms `blob:none` and `blob:limit=<n>`.
DELTA ISLANDS
@@ path-walk.c
#include "oid-array.h"
#include "path.h"
@@ path-walk.c: static int walk_path(struct path_walk_context *ctx,
- /* Evaluate function pointer on this data, if requested. */
- if ((list->type == OBJ_TREE && ctx->info->trees) ||
- (list->type == OBJ_BLOB && ctx->info->blobs) ||
+ /*
+ * Evaluate function pointer on this data, if requested.
+ * Ignore object type filters for tagged objects (path starts
+- * with `/`).
++ * with `/`), first for blobs and then other types.
+ */
+- if ((list->type == OBJ_TREE && (ctx->info->trees || path_is_for_direct_objects(path))) ||
+- (list->type == OBJ_BLOB && (ctx->info->blobs || path_is_for_direct_objects(path))) ||
- (list->type == OBJ_TAG && ctx->info->tags))
-- ret = ctx->info->path_fn(path, &list->oids, list->type,
-- ctx->info->path_fn_data);
-+ (list->type == OBJ_TAG && ctx->info->tags)) {
-+ struct oid_array *oids = &list->oids;
++ if (list->type == OBJ_BLOB &&
++ ctx->info->blob_limit &&
++ !path_is_for_direct_objects(path)) {
+ struct oid_array filtered = OID_ARRAY_INIT;
+
-+ if (list->type == OBJ_BLOB && ctx->info->blob_limit) {
-+ for (size_t i = 0; i < list->oids.nr; i++) {
-+ unsigned long size;
++ for (size_t i = 0; i < list->oids.nr; i++) {
++ unsigned long size;
+
-+ if (odb_read_object_info(ctx->repo->objects,
-+ &list->oids.oid[i],
-+ &size) != OBJ_BLOB ||
-+ size < ctx->info->blob_limit)
-+ oid_array_append(&filtered,
-+ &list->oids.oid[i]);
-+ }
-+ oids = &filtered;
++ if (odb_read_object_info(ctx->repo->objects,
++ &list->oids.oid[i],
++ &size) != OBJ_BLOB ||
++ size < ctx->info->blob_limit)
++ oid_array_append(&filtered,
++ &list->oids.oid[i]);
+ }
+
-+ if (oids->nr)
-+ ret = ctx->info->path_fn(path, oids, list->type,
++ if (filtered.nr)
++ ret = ctx->info->path_fn(path, &filtered, list->type,
+ ctx->info->path_fn_data);
+ oid_array_clear(&filtered);
++ } else if (path_is_for_direct_objects(path) ||
++ (list->type == OBJ_TREE && ctx->info->trees) ||
++ (list->type == OBJ_BLOB && ctx->info->blobs) ||
++ (list->type == OBJ_TAG && ctx->info->tags)) {
+ ret = ctx->info->path_fn(path, &list->oids, list->type,
+ ctx->info->path_fn_data);
+ }
/* Expand data for children. */
@@ path-walk.c: static int prepare_filters(struct path_walk_info *info,
+ case LOFC_BLOB_LIMIT:
+ if (info) {
-+ if (!options->blob_limit_value) {
++ if (!options->blob_limit_value)
+ info->blobs = 0;
-+ } else {
++ else
+ info->blob_limit = options->blob_limit_value;
-+ }
+ list_objects_filter_release(options);
+ }
+ return 1;
@@ path-walk.h: struct path_walk_info {
+ /**
+ * If non-zero, specifies a maximum blob size. Blobs with a
-+ * size equal to or greater than this limit will be omitted
-+ * from the walk. Blobs smaller than the limit (or blobs
-+ * whose size cannot be determined) are still visited.
++ * size equal to or greater than this limit will not be
++ * emitted unless included in 'pending'.
+ */
+ unsigned long blob_limit;
+
@@ t/t6601-path-walk.sh: test_expect_success 'topic only, blob:none filter' '
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
-+ 2:tree::$(git rev-parse topic^{tree})
-+ 2:tree::$(git rev-parse base^{tree})
-+ 2:tree::$(git rev-parse base~1^{tree})
-+ 2:tree::$(git rev-parse base~2^{tree})
-+ 2:tree::$(git rev-parse refs/tags/tree-tag^{})
-+ 2:tree::$(git rev-parse refs/tags/tree-tag2^{})
-+ 3:tree:a/:$(git rev-parse base:a)
-+ 4:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
-+ 5:tree:left/:$(git rev-parse base:left)
-+ 5:tree:left/:$(git rev-parse base~2:left)
-+ 6:tree:right/:$(git rev-parse topic:right)
-+ 6:tree:right/:$(git rev-parse base~1:right)
-+ 6:tree:right/:$(git rev-parse base~2:right)
-+ blobs:0
++ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
++ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
++ 3:tree::$(git rev-parse topic^{tree})
++ 3:tree::$(git rev-parse base^{tree})
++ 3:tree::$(git rev-parse base~1^{tree})
++ 3:tree::$(git rev-parse base~2^{tree})
++ 3:tree::$(git rev-parse refs/tags/tree-tag^{})
++ 3:tree::$(git rev-parse refs/tags/tree-tag2^{})
++ 4:tree:a/:$(git rev-parse base:a)
++ 5:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
++ 6:tree:left/:$(git rev-parse base:left)
++ 6:tree:left/:$(git rev-parse base~2:left)
++ 7:tree:right/:$(git rev-parse topic:right)
++ 7:tree:right/:$(git rev-parse base~1:right)
++ 7:tree:right/:$(git rev-parse base~2:right)
++ blobs:2
+ commits:4
+ tags:7
+ trees:13
@@ t/t6601-path-walk.sh: test_expect_success 'topic only, blob:none filter' '
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
-+ 2:tree::$(git rev-parse topic^{tree})
-+ 2:tree::$(git rev-parse base^{tree})
-+ 2:tree::$(git rev-parse base~1^{tree})
-+ 2:tree::$(git rev-parse base~2^{tree})
-+ 2:tree::$(git rev-parse refs/tags/tree-tag^{})
-+ 2:tree::$(git rev-parse refs/tags/tree-tag2^{})
-+ 3:blob:a:$(git rev-parse base~2:a)
-+ 4:tree:a/:$(git rev-parse base:a)
-+ 5:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
-+ 6:tree:left/:$(git rev-parse base:left)
-+ 6:tree:left/:$(git rev-parse base~2:left)
-+ 7:blob:left/b:$(git rev-parse base~2:left/b)
-+ 8:tree:right/:$(git rev-parse topic:right)
-+ 8:tree:right/:$(git rev-parse base~1:right)
-+ 8:tree:right/:$(git rev-parse base~2:right)
-+ 9:blob:right/c:$(git rev-parse base~2:right/c)
-+ 10:blob:right/d:$(git rev-parse base~1:right/d)
-+ blobs:4
++ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
++ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
++ 3:tree::$(git rev-parse topic^{tree})
++ 3:tree::$(git rev-parse base^{tree})
++ 3:tree::$(git rev-parse base~1^{tree})
++ 3:tree::$(git rev-parse base~2^{tree})
++ 3:tree::$(git rev-parse refs/tags/tree-tag^{})
++ 3:tree::$(git rev-parse refs/tags/tree-tag2^{})
++ 4:blob:a:$(git rev-parse base~2:a)
++ 5:tree:a/:$(git rev-parse base:a)
++ 6:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
++ 7:tree:left/:$(git rev-parse base:left)
++ 7:tree:left/:$(git rev-parse base~2:left)
++ 8:blob:left/b:$(git rev-parse base~2:left/b)
++ 9:tree:right/:$(git rev-parse topic:right)
++ 9:tree:right/:$(git rev-parse base~1:right)
++ 9:tree:right/:$(git rev-parse base~2:right)
++ 10:blob:right/c:$(git rev-parse base~2:right/c)
++ 11:blob:right/d:$(git rev-parse base~1:right/d)
++ blobs:6
+ commits:4
+ tags:7
+ trees:13
6: 55ed02772b = 8: 2360a5be81 path-walk: add pl_sparse_trees to control tree pruning
7: dcb1a29d3a ! 9: d9f5a98830 pack-objects: support sparse:oid filter with path-walk
@@ Documentation/git-pack-objects.adoc: will be automatically changed to version `1
+
Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
ignored in the presence of `--path-walk`. Whe `--path-walk` option
--supports the `--filter=<spec>` form `blob:none` and `blob:limit=<n>`.
-+supports the `--filter=<spec>` form `blob:none`, `blob:limit=<n>`, and
+-supports the `--filter=<spec>` forms `blob:none` and `blob:limit=<n>`.
++supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`, and
+`sparse:<oid>`.
DELTA ISLANDS
## builtin/pack-objects.c ##
+@@ builtin/pack-objects.c: static int add_objects_by_path(const char *path,
+ return 0;
+ }
+
+-static void get_object_list_path_walk(struct rev_info *revs)
++static int get_object_list_path_walk(struct rev_info *revs)
+ {
+ struct path_walk_info info = PATH_WALK_INFO_INIT;
+ unsigned int processed = 0;
@@ builtin/pack-objects.c: static void get_object_list_path_walk(struct rev_info *revs)
result = walk_objects_by_path(&info);
trace2_region_leave("pack-objects", "path-walk", revs->repo);
+- if (result)
+- die(_("failed to pack objects via path-walk"));
+ path_walk_info_clear(&info);
+
- if (result)
- die(_("failed to pack objects via path-walk"));
++ return result;
}
+
+ static void get_object_list(struct rev_info *revs, struct strvec *argv)
+@@ builtin/pack-objects.c: static void get_object_list(struct rev_info *revs, struct strvec *argv)
+ fn_show_object = show_object;
+
+ if (path_walk) {
+- get_object_list_path_walk(revs);
+- } else {
++ if (get_object_list_path_walk(revs)) {
++ warning(_("failed to pack objects via path-walk"));
++ path_walk = 0;
++ }
++ }
++
++ if (!path_walk) {
+ if (prepare_revision_walk(revs))
+ die(_("revision walk setup failed"));
+ mark_edges_uninteresting(revs, show_edge, sparse);
## path-walk.c ##
@@
8: 772cdcaeed ! 10: c9efff0ea8 path-walk: support `tree:0` filter
@@ Documentation/git-pack-objects.adoc: will be automatically changed to version `1
+
Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
ignored in the presence of `--path-walk`. Whe `--path-walk` option
--supports the `--filter=<spec>` form `blob:none`, `blob:limit=<n>`, and
+-supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`, and
-`sparse:<oid>`.
-+supports the `--filter=<spec>` form `blob:none`, `blob:limit=<n>`,
++supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,
+`tree:0`, and `sparse:<oid>`.
DELTA ISLANDS
## path-walk.c ##
+@@ path-walk.c: static int walk_path(struct path_walk_context *ctx,
+ ctx->info->path_fn_data);
+ }
+
+- /* Expand data for children. */
+- if (list->type == OBJ_TREE) {
++ /* Expand data for children, unless this is a direct-object path. */
++ if (list->type == OBJ_TREE && !path_is_for_direct_objects(path)) {
+ for (size_t i = 0; i < list->oids.nr; i++) {
+ ret |= add_tree_entries(ctx,
+ path,
+@@ path-walk.c: static int setup_pending_objects(struct path_walk_info *info,
+ {
+ struct type_and_oid_list *tags = NULL;
+ struct type_and_oid_list *tagged_blobs = NULL;
++ struct type_and_oid_list *tagged_trees = NULL;
+ struct type_and_oid_list *root_tree_list = NULL;
+
+ if (info->tags)
+ CALLOC_ARRAY(tags, 1);
+ CALLOC_ARRAY(tagged_blobs, 1);
++ if (!info->trees)
++ CALLOC_ARRAY(tagged_trees, 1);
+ root_tree_list = strmap_get(&ctx->paths_to_lists, root_path);
+
+ /*
+@@ path-walk.c: static int setup_pending_objects(struct path_walk_info *info,
+
+ switch (obj->type) {
+ case OBJ_TREE:
+- if (pending->path) {
++ if (tagged_trees) {
++ /*
++ * Trees are disabled but pending trees
++ * should still be emitted. Collect them
++ * into a "/tagged-trees" list that
++ * bypasses the object type filter.
++ */
++ oid_array_append(&tagged_trees->oids, &obj->oid);
++ } else if (pending->path) {
+ char *path = *pending->path ? xstrfmt("%s/", pending->path)
+ : xstrdup("");
+ add_path_to_list(ctx, path, OBJ_TREE, &obj->oid, 1);
+@@ path-walk.c: static int setup_pending_objects(struct path_walk_info *info,
+ free(tagged_blobs);
+ }
+ }
++ if (tagged_trees) {
++ if (tagged_trees->oids.nr) {
++ const char *tagged_tree_path = "/tagged-trees";
++ tagged_trees->type = OBJ_TREE;
++ tagged_trees->maybe_interesting = 1;
++ strmap_put(&ctx->paths_to_lists, tagged_tree_path, tagged_trees);
++ push_to_stack(ctx, tagged_tree_path);
++ } else {
++ oid_array_clear(&tagged_trees->oids);
++ free(tagged_trees);
++ }
++ }
+ if (tags) {
+ if (tags->oids.nr) {
+ const char *tag_path = "/tags";
@@ path-walk.c: static int prepare_filters(struct path_walk_info *info,
}
return 1;
@@ t/t6601-path-walk.sh: test_expect_success 'all, blob:limit=3 filter' '
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
-+ blobs:0
++ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
++ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
++ 3:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag^{tree})
++ 3:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag2)
++ blobs:2
+ commits:4
+ tags:7
-+ trees:0
++ trees:2
+ EOF
+
+ test_cmp_sorted expect out
9: c557f3c59d ! 11: b221ea4712 path-walk: support `object:type` filter
@@ Documentation/git-pack-objects.adoc
@@ Documentation/git-pack-objects.adoc: will be automatically changed to version `1`.
Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
ignored in the presence of `--path-walk`. Whe `--path-walk` option
- supports the `--filter=<spec>` form `blob:none`, `blob:limit=<n>`,
+ supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,
-`tree:0`, and `sparse:<oid>`.
+`tree:0`, `object:type=<type>`, and `sparse:<oid>`.
@@ Documentation/git-pack-objects.adoc: will be automatically changed to version `1
DELTA ISLANDS
## path-walk.c ##
-@@ path-walk.c: static int setup_pending_objects(struct path_walk_info *info,
- CALLOC_ARRAY(tags, 1);
- if (info->blobs)
- CALLOC_ARRAY(tagged_blobs, 1);
-- if (info->trees)
-+ if (info->trees || info->blobs)
- root_tree_list = strmap_get(&ctx->paths_to_lists, root_path);
-
- /*
-@@ path-walk.c: static int setup_pending_objects(struct path_walk_info *info,
-
- switch (obj->type) {
- case OBJ_TREE:
-- if (!info->trees)
-+ if (!info->trees && !info->blobs)
- continue;
- if (pending->path) {
- char *path = *pending->path ? xstrfmt("%s/", pending->path)
+@@ path-walk.c: static int walk_path(struct path_walk_context *ctx,
+ ret = ctx->info->path_fn(path, &filtered, list->type,
+ ctx->info->path_fn_data);
+ oid_array_clear(&filtered);
+- } else if (path_is_for_direct_objects(path) ||
++ } else if ((!ctx->info->strict_types && path_is_for_direct_objects(path)) ||
+ (list->type == OBJ_TREE && ctx->info->trees) ||
+ (list->type == OBJ_BLOB && ctx->info->blobs) ||
+ (list->type == OBJ_TAG && ctx->info->tags)) {
@@ path-walk.c: static int prepare_filters(struct path_walk_info *info,
}
return 1;
@@ path-walk.c: static int prepare_filters(struct path_walk_info *info,
+ info->tags &= options->object_type == OBJ_TAG;
+ info->trees &= options->object_type == OBJ_TREE;
+ info->blobs &= options->object_type == OBJ_BLOB;
++ info->strict_types = 1;
+ list_objects_filter_release(options);
+ }
+ return 1;
@@ path-walk.c: static int prepare_filters(struct path_walk_info *info,
case LOFC_SPARSE_OID:
if (info) {
struct object_id sparse_oid;
-@@ path-walk.c: int walk_objects_by_path(struct path_walk_info *info)
- /*
- * Set these values before preparing the walk to catch
- * lightweight tags pointing to non-commits and indexed objects.
-+ *
-+ * Keep tree_objects set whenever blobs are wanted: blobs may
-+ * be reachable through trees that show up as pending objects
-+ * (e.g., via lightweight tags pointing to trees, or annotated
-+ * tags whose peeled target is a tree). Without tree_objects,
-+ * prepare_revision_walk() would discard those pending trees
-+ * and we would never descend into them.
- */
- info->revs->blob_objects = info->blobs;
-- info->revs->tree_objects = info->trees;
-+ info->revs->tree_objects = info->trees || info->blobs;
+
+ ## path-walk.h ##
+@@ path-walk.h: struct path_walk_info {
+ int blobs;
+ int tags;
- if (prepare_revision_walk(info->revs))
- die(_("failed to setup revision walk"));
++ /**
++ * If 'strict_types' is 0, then direct object requests will no longer
++ * override the object type restrictions.
++ */
++ int strict_types;
++
+ /**
+ * If non-zero, specifies a maximum blob size. Blobs with a
+ * size equal to or greater than this limit will not be
## t/t6601-path-walk.sh ##
@@ t/t6601-path-walk.sh: test_expect_success 'tree:1 filter is rejected' '
@@ t/t6601-path-walk.sh: test_expect_success 'tree:1 filter is rejected' '
+ 0:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
+ 0:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
+ 1:blob:a:$(git rev-parse base~2:a)
-+ 2:blob:file2:$(git rev-parse refs/tags/tree-tag2^{}:file2)
-+ 3:blob:child/file:$(git rev-parse refs/tags/tree-tag:child/file)
-+ 4:blob:left/b:$(git rev-parse base:left/b)
-+ 4:blob:left/b:$(git rev-parse base~2:left/b)
-+ 5:blob:right/c:$(git rev-parse base~2:right/c)
-+ 5:blob:right/c:$(git rev-parse topic:right/c)
-+ 6:blob:right/d:$(git rev-parse base~1:right/d)
-+ blobs:10
++ 2:blob:left/b:$(git rev-parse base:left/b)
++ 2:blob:left/b:$(git rev-parse base~2:left/b)
++ 3:blob:right/c:$(git rev-parse base~2:right/c)
++ 3:blob:right/c:$(git rev-parse topic:right/c)
++ 4:blob:right/d:$(git rev-parse base~1:right/d)
++ blobs:8
+ commits:0
+ tags:0
+ trees:0
10: 5423273edf ! 12: 16bd3c3161 path-walk: support `combine` filter
@@ Commit message
the standalone `sparse:oid` behavior.
Implementation-wise, the existing `prepare_filters()` called
- `list_objects_filter_release()` inside each case branch. That works fine
- for top-level filters, but `combine` filters need to recurse over its
- child filters without releasing each one in turn (since the parent's
- release iterates the sub array). Split `prepare_filters()` into a
- recursive helper that performs only the mutation, plus a thin wrapper
- that calls the helper and then releases the top-level filter once.
+ `list_objects_filter_release()` inside each case branch. That works fine for
+ top-level filters, but `combine` filters need to recurse over its child
+ filters without releasing each one in turn (since the parent's release
+ iterates the sub array). Split `prepare_filters()` into a recursive helper
+ that performs only the mutation, plus a thin wrapper that calls the helper
+ and then releases the top-level filter once.
The `LOFC_COMBINE` case in the helper just walks `sub_nr` and recurses;
child filters are released by the wrapper's single
- `list_objects_filter_release()` call on the parent (which itself
- recursively releases each sub-filter, the same way it always has).
+ `list_objects_filter_release()` call on the parent (which itself recursively
+ releases each sub-filter, the same way it always has).
If any sub-filter is unsupported (e.g. "tree:1", "sparse:<path>", or a
not-yet-supported choice), the recursion bubbles a failure up and the
@@ Documentation/git-pack-objects.adoc
@@ Documentation/git-pack-objects.adoc: will be automatically changed to version `1`.
Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
ignored in the presence of `--path-walk`. Whe `--path-walk` option
- supports the `--filter=<spec>` form `blob:none`, `blob:limit=<n>`,
+ supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,
-`tree:0`, `object:type=<type>`, and `sparse:<oid>`.
+`tree:0`, `object:type=<type>`, and `sparse:<oid>`. These supported filter
+types can be combined with the `combine:<spec>+<spec>` form.
@@ path-walk.c: static int setup_pending_objects(struct path_walk_info *info,
{
switch (options->choice) {
case LOFC_DISABLED:
- return 1;
-
- case LOFC_BLOB_NONE:
-- if (info) {
-+ if (info)
- info->blobs = 0;
-- list_objects_filter_release(options);
-- }
- return 1;
-
- case LOFC_BLOB_LIMIT:
+@@ path-walk.c: static int prepare_filters(struct path_walk_info *info,
if (info) {
- if (!options->blob_limit_value) {
+ if (!options->blob_limit_value)
info->blobs = 0;
-- } else {
-+ } else if (!info->blob_limit ||
-+ options->blob_limit_value < info->blob_limit) {
+- else
++ else if (!info->blob_limit ||
++ info->blob_limit > options->blob_limit_value)
info->blob_limit = options->blob_limit_value;
- }
-- list_objects_filter_release(options);
+ list_objects_filter_release(options);
}
- return 1;
-
@@ path-walk.c: static int prepare_filters(struct path_walk_info *info,
if (info) {
info->trees = 0;
@@ path-walk.c: static int prepare_filters(struct path_walk_info *info,
}
return 1;
-@@ path-walk.c: static int prepare_filters(struct path_walk_info *info,
- info->tags &= options->object_type == OBJ_TAG;
- info->trees &= options->object_type == OBJ_TREE;
- info->blobs &= options->object_type == OBJ_BLOB;
-- list_objects_filter_release(options);
- }
- return 1;
-
@@ path-walk.c: static int prepare_filters(struct path_walk_info *info,
warning(_("sparse filter is not cone-mode compatible"));
return 0;
@@ t/t6601-path-walk.sh: test_expect_success 'all, object:type=blob filter' '
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
-+ blobs:0
++ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
++ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
++ 3:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag^{tree})
++ 3:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag2)
++ blobs:2
+ commits:4
+ tags:7
-+ trees:0
++ trees:2
+ EOF
+
+ test_cmp_sorted expect out
@@ t/t6601-path-walk.sh: test_expect_success 'all, object:type=blob filter' '
+ -- --all >out &&
+
+ cat >expect <<-EOF &&
-+ 0:blob:a:$(git rev-parse base~2:a)
-+ 1:blob:left/b:$(git rev-parse base~2:left/b)
-+ 2:blob:right/c:$(git rev-parse base~2:right/c)
-+ 3:blob:right/d:$(git rev-parse base~1:right/d)
-+ blobs:4
++ 0:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
++ 0:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
++ 1:blob:a:$(git rev-parse base~2:a)
++ 2:blob:left/b:$(git rev-parse base~2:left/b)
++ 3:blob:right/c:$(git rev-parse base~2:right/c)
++ 4:blob:right/d:$(git rev-parse base~1:right/d)
++ blobs:6
+ commits:0
+ tags:0
+ trees:0
--
gitgitgadget
^ permalink raw reply
* Re: [PATCH v3 3/7] odb: update `struct odb_write_stream` read() callback
From: Jeff King @ 2026-05-11 17:58 UTC (permalink / raw)
To: Justin Tobler; +Cc: git, ps, gitster
In-Reply-To: <20260402213220.2651523-4-jltobler@gmail.com>
On Thu, Apr 02, 2026 at 04:32:16PM -0500, Justin Tobler wrote:
> @@ -1098,9 +1099,16 @@ int odb_source_loose_write_stream(struct odb_source *source,
> unsigned char *in0 = stream.next_in;
>
> if (!stream.avail_in && !in_stream->is_finished) {
> - const void *in = in_stream->read(in_stream, &stream.avail_in);
> - stream.next_in = (void *)in;
> - in0 = (unsigned char *)in;
> + ssize_t read_len = odb_write_stream_read(in_stream, buf,
> + sizeof(buf));
> + if (read_len < 0) {
> + err = -1;
> + goto cleanup;
> + }
> +
> + stream.avail_in = read_len;
> + stream.next_in = buf;
> + in0 = buf;
If we hit this "goto cleanup", we'll leak the "fd" descriptor opened
earlier. We either need to close(fd) here, or do so in the cleanup
handler (but that means consistently setting fd to a sentinel value
after we close it, which we do not currently do).
Noticed by Coverity (I guess this series just hit "jch", since it's
"new" as of today's run).
-Peff
^ permalink raw reply
* Re: [PATCH v2 2/3] builtin/log: prefetch necessary blobs for `git cherry`
From: Elijah Newren @ 2026-05-11 17:45 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Derrick Stolee, Elijah Newren via GitGitGadget, git
In-Reply-To: <xmqqtsseu09t.fsf@gitster.g>
On Sun, May 10, 2026 at 7:51 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Derrick Stolee <stolee@gmail.com> writes:
>
> > Ultimately, _this_ patch cares about a diff. Could we compute a
> > "diff prep" computation using the core diff library instead of
> > inventing a second queue of results for diffing?
> >
> > Patch 3 cares about a "scan prep" which cares about loading all
> > blobs for a given tree with respect to a pathspec. This is very
> > similar to what a checkout would do, though it ultimately uses
> > a form of diff to find out what change should be applied to the
> > working directory. Perhaps 'git archive' is a better matching
> > example.
> >
> > I don't mean to make your series more complicated. I value what
> > you're doing and can see how your current attention can be used
> > to make further improvements later. By implementing things in a
> > common location, then we can have later integrations add to the
> > confidence in the feature through tests covering each user-facing
> > use.
> >
> > I'm not sure if it makes sense to attempt to create a universal
> > library method that would be used by builtin/log.c _and_ diff.c,
> > at least not right now. I'm most interested in having this logic
> > be more reusable in the future without needing to move code
> > across files.
>
> The points raised in the message I am responding here, together with
> the ones in <31763514-2602-4d8e-ac25-70590f090947@gmail.com>, remain
> unanswered.
>
> Should I still keep these patches in my tree, hoping that responses
> may come some day? I will mark the topic as "Expeting review
> responses" in the draft "What's cooking" report I work from for now,
> but it has been quite a while since we looked at the patches, so...?
Sorry for not responding sooner. There have been a number of
incidents at work (including a big problematic one the day Derrick
sent his email), and I was pulled into both firefighting and
remediation duties which have sucked up all my time. I owe responses
to Derrick, Patrick, Johannes, Phillip, and Taylor on a variety of
topics.
For this particular series, maybe mark as expecting a re-roll, since
Stolee suggested adding a test on patch 3?
^ permalink raw reply
* [PATCH v5 5/5] format-rev: introduce builtin for on-demand pretty formatting
From: kristofferhaugsbakk @ 2026-05-11 15:45 UTC (permalink / raw)
To: Junio C Hamano
Cc: Kristoffer Haugsbakk, git, ben.knoble, Phillip Wood, Ramsay Jones
In-Reply-To: <V5_CV_format-rev.6c9@msgid.xyz>
From: Kristoffer Haugsbakk <code@khaugsbakk.name>
Introduce a new builtin for pretty formatting one revision expression
per line or commit object names found in running text.
Sometimes you want to format commits. Most of the time you’re
walking the graph, e.g. getting a range of commits like
`master..topic`. That’s a job for git-log(1).
But there are times when you want to format commits that you encounter
on demand:
• Full hashes in running text that you might want to pretty-print
• git-last-modified(1) outputs full hashes that you can do the same
with
• git-cherry(1) has `-v` for commit subject, but maybe you want
something else?
But now you can’t use git-log(1), git-show(1), or git-rev-list(1):
• You can’t feed commits piecemeal to these commands, one input
for one output; they block until standard in is closed
• You can’t feed a list of possibly duplicate commits, like the output
of git-last-modified(1); they effectively deduplicate the output
Beyond these two points there’s also the input massage problem: you
cannot feed mixed input (revisions mixed with arbitrary text).
One might hope that git-cat-file(1) can save us. But it doesn’t
support pretty formats.
But there is one command that already both handles revisions as
arguments, revisions on standard input, and even revisions mixed in
with arbitrary text. Namely git-name-rev(1): the command for outputting
symbolic names for commits.
We made some room in `builtin/name-rev.c` two commits ago. Let’s
now add this new git-format-rev(1) command. Taking inspiration from
git-name-rev(1), there are two modes:
• revs: like git-name-rev(1) in argv mode, but one revision per line
on standard in
• text: like git-name-rev(1) with `--annotate-stdin`
***
We need to add this command to the exception list in
`t/t1517-outside-repo.sh` because it uses “EXPERIMENTAL!”
in the usage line.
Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Helped-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
---
Notes (series):
v5:
• (All doc changes)
• Fix definition list mistake
• No blank line between the two modes
• Also replace `::` delimiters with `;;`. This is not strictly
needed since the definition list is inside an open block. But
it is consistent with all other definition list in definition
lists I’ve seen. (The command option section is a long
definition list.)
• Rewrite part that I was unhappy about regarding
`--stdin-mode=text` and `-z`:
https://lore.kernel.org/git/c04d9cf9-e6a9-4e12-8025-9baededfdafc@app.fastmail.com/
• Replace `abcdef012...` with just “some object name”. The object
name is arbitrary and my AsciiDoc output seems to do some weird
things for `...` inside backticks for HTML.
• Replace “this problem is just another” with “this problem
*is solved* with...”
***
v4:
• Squash in “SQUASH???” commits from Junio’s branch:
• Fix -Werror=maybe-uninitialized at builtin/name-rev.c:893:25
• Add this new builtin to Meson build file as well
• Flush every output line/record with `maybe_flush_or_die(...)` (Phillip)
• Introduce `-z` and other NUL output options (Phillip)
• Tests:
• More tests: “name lookup failures” (This would have caught my
uninitialized mistake from v3)
• More tests: for NUL/LF termination
• Fix stale setup command which uses “name-rev” in the name
• Use `setup:` prefix for both (now two) setup tests
• Drop some `:/fifth` expressions in favor just `fifth` (the tag from
`test_commit`)
• Flesh out documentation
• --stdin-mode=revs: Replace “Could not get [sha1][object name]” since the
user could be using SHA256
• This error string was originally stolen from name-rev
• Remember to add this command to `.gitignore` this time
• Correct doc to `git [show][log] --no-walk -1` (next to `git show`)
• Plus some minor things!
v3:
• And don’t forget to document --notes this time
https://lore.kernel.org/git/CALnO6CB5WOTp_e7Kv3CrEbQ+3XE-gDxNVHf7qATBEbyKWfxpLg@mail.gmail.com/
.gitignore | 1 +
Documentation/git-format-rev.adoc | 215 ++++++++++++++++++++++++++++
Documentation/meson.build | 1 +
Makefile | 1 +
builtin.h | 1 +
builtin/name-rev.c | 223 ++++++++++++++++++++++++++++++
command-list.txt | 1 +
git.c | 1 +
t/t1517-outside-repo.sh | 3 +-
t/t6120-describe.sh | 194 ++++++++++++++++++++++++++
10 files changed, 640 insertions(+), 1 deletion(-)
create mode 100644 Documentation/git-format-rev.adoc
diff --git a/.gitignore b/.gitignore
index 24635cf2d6f..e406d3741cd 100644
--- a/.gitignore
+++ b/.gitignore
@@ -71,6 +71,7 @@
/git-for-each-ref
/git-for-each-repo
/git-format-patch
+/git-format-rev
/git-fsck
/git-fsck-objects
/git-fsmonitor--daemon
diff --git a/Documentation/git-format-rev.adoc b/Documentation/git-format-rev.adoc
new file mode 100644
index 00000000000..c40d52e9f6d
--- /dev/null
+++ b/Documentation/git-format-rev.adoc
@@ -0,0 +1,215 @@
+git-format-rev(1)
+=================
+
+NAME
+----
+git-format-rev - EXPERIMENTAL: Pretty format revisions on demand
+
+
+SYNOPSIS
+--------
+[synopsis]
+(EXPERIMENTAL!) git format-rev --stdin-mode=<mode> --format=<pretty> [--[no-]notes=<ref>] [-z] [--[no-]null-output] [--[no-]null-input]
+
+DESCRIPTION
+-----------
+
+Pretty format revisions from standard input.
+
+THIS COMMAND IS EXPERIMENTAL. THE BEHAVIOR MAY CHANGE.
+
+OPTIONS
+-------
+
+`--stdin-mode=<mode>`::
+ How to interpret standard input data:
++
+--
+`revs`;; Each line or record (see the <<io,INPUT AND OUTPUT FORMATS>>
+ section) is interpreted as a commit. Any kind of revision
+ expression can be used (see linkgit:gitrevisions[7]). Annotated
+ tags are peeled (see linkgit:gitglossary[7]).
++
+The argument `rev` is also accepted.
+
+`text`;; Formats all commit object names found in freeform text. These
+ must the full object names, i.e. abbreviated hexidecimal object
+ names will not be interpreted.
++
+Anything that is parsed as an object name but that is not found to be a
+commit object name is left alone (echoed).
+--
+
+`--format=<pretty>`::
+ Pretty format string.
+
+`--notes=<ref>`::
+`--no-notes`::
+ Custom notes ref. Notes are displayed when using the `%N`
+ atom. See linkgit:git-notes[1].
+
+`-z`::
+`--null`::
+ Use _NUL_ character to terminate both input and output instead
+ of newline. This option cannot be negated.
++
+This is useful if both the input and output could contain newlines or if
+the input to this command also uses _NUL_ character termination; see the
+<<io,INPUT AND OUTPUT FORMATS>> section below.
++
+The mode `--stdin-mode=text` can have use for this option when it needs
+to process input like for example `git last-modified -z`; see the
+<<examples,EXAMPLES>> section below.
+
+`--null-output`::
+`--no-null-output`::
+ Use _NUL_ character to terminate output instead of newline. The
+ default is `--no-null-output`.
++
+This is useful if the output could contain newlines, for example if the
+`%n` (newline) atom is used.
+
+`--null-input`::
+`--no-null-input`::
+ Use _NUL_ character to terminate input instead of newline. The
+ default is `--no-null-input`.
++
+This is useful if the input revision expressions could contain newlines.
+
+[[io]]
+INPUT AND OUTPUT FORMAT
+-----------------------
+
+The command uses newlines for both input and output termination by
+default. See the `-z`, `--null-output`, and `--null-input` options for
+using _NUL_ character as the terminator.
+
+The mode `--stdin-mode=revs` outputs one formatted commit followed by
+the terminator. This could either be called a _line_ or a _record_ in
+case "line" is too suggestive of newline termination.
+
+Note that this means that the terminator character (newline or _NUL_)
+acts as a _terminator_, not a _separator_. In other words, the final
+line or record is also terminated by the terminator character.
+
+The mode `--stdin-mode=text` replaces each object name with the
+formatted commit, i.e. the format `%s` would transform some commit
+object name to `<subject>` without any termination. Like this:
+
+----
+Did we not fix this in "<subject>"?
+----
+
+It is safe to interactively read and write from this command since each
+record is immediately flushed.
+
+[[examples]]
+EXAMPLES
+--------
+
+The command linkgit:git-last-modified[1] shows the commit that each file
+was last modified in.
+
+----
+$ git last-modified -- README.md Makefile
+7798034171030be0909c56377a4e0e10e6d2df93 Makefile
+c50fbb2dd225e7e82abba4380423ae105089f4d7 README.md
+----
+
+We can pipe the result to this command in order to replace the object
+name with the commit author.
+
+----
+$ git last-modified -- README.md Makefile |
+ git format-rev --stdin-mode=text --format=%an
+Junio C Hamano Makefile
+Todd Zullinger README.md
+----
+
+Another example is _formatting commits in commit messages_. Given this commit message:
+
+----
+Fix off-by-one error
+
+Fix off-by-one error introduced in
+e83c5163316f89bfbde7d9ab23ca2e25604af290.
+
+We thought we fixed this in 5569bf9bbedd63a00780fc5c110e0cfab3aa97b9 but
+that only covered 1/3 of the faulty cases.
+----
+
+We can format the commits and use par(1) to reflow the text, say in a
+`commit-msg` hook:
+
+----
+$ git config set hook.reference-commits.event commit-msg
+$ git config set hook.reference-commits.command reference-commits
+$ cat $(which reference-commits)
+#/bin/sh
+
+msg="$1"
+rewritten=$(mktemp)
+git format-rev --stdin-mode=text --format=reference <"$msg" |
+ par >"$rewritten"
+mv "$rewritten" "$msg"
+----
+
+Which will produce something like this:
+
+----
+Fix off-by-one error
+
+Fix off-by-one error introduced in e83c5163316 (Implement better memory
+allocator, 2005-04-07).
+
+We thought we fixed this in 5569bf9bbed (Fix memory allocator,
+2005-06-22) but that only covered 1/3 of the faulty cases.
+----
+
+DISCUSSION
+----------
+
+This command lets you format any number of revisions in any order
+through one command invocation. Consider the
+linkgit:git-last-modified[1] case from the <<examples,EXAMPLES>> section
+above:
+
+1. There might be hundreds of files
+2. Commits can be repeated, i.e. two or more files were last modified in
+ the same commit
+
+Two widely-used commands which pretty formats commits are
+linkgit:git-log[1] and linkgit:git-show[1]. It turns out that they are
+not a good fit for the above use case.
+
+- The output of linkgit:git-last-modified[1] would have to be processed
+ in stages since you need to transform the first column separately and
+ then link the author to the filename. But this is surmountable.
+- You can feed each commit to `git show` or `git log --no-walk -1`. But
+ that means that you need to create a process for each line.
+- Let’s say that you want to use one process, not one per line. So you
+ want to feed all the commits to the command. Now you face the problem
+ that you have to feed all the commits to the commands before you get
+ any output (this is also the case for the `--stdin` modes). In other
+ words, you cannot loop through each line, get the author for the
+ commit, and output the author and the filename. You need to feed all
+ the commits, get back all the output, and match the output with the
+ filename.
+- But the next problem is that commands will deduplicate the input and
+ only output one commit one single time only. Thus you cannot make the
+ output order match the input order, since a commit could have been
+ repeated in the original input.
+
+In short, it is straightforward to use these two commands if you use one
+process per line. It is much more work if you just want to use one
+process, but still doable. In contrast, this problem is solved with just
+another shell pipeline with this command.
+
+SEE ALSO
+--------
+linkgit:git-name-rev[1],
+linkgit:git-log[1].
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Documentation/meson.build b/Documentation/meson.build
index d6365b888bb..58e7c6a0b8a 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -55,6 +55,7 @@ manpages = {
'git-for-each-ref.adoc' : 1,
'git-for-each-repo.adoc' : 1,
'git-format-patch.adoc' : 1,
+ 'git-format-rev.adoc' : 1,
'git-fsck-objects.adoc' : 1,
'git-fsck.adoc' : 1,
'git-fsmonitor--daemon.adoc' : 1,
diff --git a/Makefile b/Makefile
index 15b1ded1a0b..cbaf91fd846 100644
--- a/Makefile
+++ b/Makefile
@@ -895,6 +895,7 @@ BUILT_INS += $(patsubst builtin/%.o,git-%$X,$(BUILTIN_OBJS))
BUILT_INS += git-cherry$X
BUILT_INS += git-cherry-pick$X
BUILT_INS += git-format-patch$X
+BUILT_INS += git-format-rev$X
BUILT_INS += git-fsck-objects$X
BUILT_INS += git-init$X
BUILT_INS += git-maintenance$X
diff --git a/builtin.h b/builtin.h
index 235c51f30e5..63813c90125 100644
--- a/builtin.h
+++ b/builtin.h
@@ -189,6 +189,7 @@ int cmd_fmt_merge_msg(int argc, const char **argv, const char *prefix, struct re
int cmd_for_each_ref(int argc, const char **argv, const char *prefix, struct repository *repo);
int cmd_for_each_repo(int argc, const char **argv, const char *prefix, struct repository *repo);
int cmd_format_patch(int argc, const char **argv, const char *prefix, struct repository *repo);
+int cmd_format_rev(int argc, const char **argv, const char *prefix, struct repository *repo);
int cmd_fsck(int argc, const char **argv, const char *prefix, struct repository *repo);
int cmd_fsmonitor__daemon(int argc, const char **argv, const char *prefix, struct repository *repo);
int cmd_gc(int argc, const char **argv, const char *prefix, struct repository *repo);
diff --git a/builtin/name-rev.c b/builtin/name-rev.c
index 475efb0b82b..5494b0424b3 100644
--- a/builtin/name-rev.c
+++ b/builtin/name-rev.c
@@ -18,6 +18,10 @@
#include "commit-graph.h"
#include "wildmatch.h"
#include "mem-pool.h"
+#include "pretty.h"
+#include "revision.h"
+#include "notes.h"
+#include "write-or-die.h"
/*
* One day. See the 'name a rev shortly after epoch' test in t6120 when
@@ -272,14 +276,26 @@ struct name_ref_data {
struct string_list exclude_filters;
};
+struct pretty_format {
+ struct pretty_print_context ctx;
+ struct userformat_want want;
+};
+
enum command_type {
NAME_REV = 1,
+ FORMAT_REV = 2,
+};
+
+enum stdin_mode {
+ TEXT = 1,
+ REVS = 2,
};
struct command {
enum command_type type;
union {
int name_only;
+ struct pretty_format *pretty_format;
} u;
};
@@ -290,6 +306,13 @@ static void init_name_rev_command(struct command *cmd,
cmd->u.name_only = name_only;
}
+static void init_format_rev_command(struct command *cmd,
+ struct pretty_format *pretty_format)
+{
+ cmd->type = FORMAT_REV;
+ cmd->u.pretty_format = pretty_format;
+}
+
static struct tip_table {
struct tip_table_entry {
struct object_id oid;
@@ -495,6 +518,27 @@ static const char *get_rev_name(const struct object *o, struct strbuf *buf)
}
}
+static const char *get_format_rev(const struct commit *c,
+ struct pretty_format *format_ctx,
+ struct strbuf *buf)
+{
+ strbuf_reset(buf);
+
+ if (format_ctx->want.notes) {
+ struct strbuf notebuf = STRBUF_INIT;
+
+ format_display_notes(&c->object.oid, ¬ebuf,
+ get_log_output_encoding(),
+ format_ctx->ctx.fmt == CMIT_FMT_USERFORMAT);
+ format_ctx->ctx.notes_message = strbuf_detach(¬ebuf, NULL);
+ }
+
+ pretty_print_commit(&format_ctx->ctx, c, buf);
+ FREE_AND_NULL(format_ctx->ctx.notes_message);
+
+ return buf->buf;
+}
+
static void show_name(const struct object *obj,
const char *caller_name,
int always, int allow_undefined, int name_only)
@@ -564,6 +608,18 @@ static void name_rev_line(char *p, struct command *cmd)
else
printf("%.*s (%s)", p_len, p_start, name);
break;
+ case FORMAT_REV:
+ if (!oid_ret)
+ o = parse_object(the_repository, &oid);
+ if (o && o->type == OBJ_COMMIT)
+ name = get_format_rev((const struct commit *)o,
+ cmd->u.pretty_format,
+ &buf);
+ if (name)
+ printf("%.*s%s", p_len - hexsz, p_start, name);
+ else
+ printf("%.*s", p_len, p_start);
+ break;
default:
BUG("uncovered case: %d", cmd->type);
}
@@ -717,3 +773,170 @@ int cmd_name_rev(int argc,
object_array_clear(&revs);
return 0;
}
+
+struct format_nul_data {
+ bool nul_input;
+ bool nul_output;
+};
+
+static int format_nul_cb(const struct option *option,
+ const char *arg,
+ int unset)
+{
+ struct format_nul_data *data = option->value;
+ data->nul_input = 1;
+ data->nul_output = 1;
+ BUG_ON_OPT_NEG(unset);
+ BUG_ON_OPT_ARG(arg);
+ return 0;
+}
+
+static enum stdin_mode parse_stdin_mode(const char *stdin_mode)
+{
+ if (!strcmp(stdin_mode, "text"))
+ return TEXT;
+ else if (!strcmp(stdin_mode, "revs") ||
+ !strcmp(stdin_mode, "rev"))
+ return REVS;
+ else
+ die(_("'%s' needs to be either text, revs, or rev"),
+ "--stdin-mode");
+}
+
+static char const *const format_rev_usage[] = {
+ N_("(EXPERIMENTAL!) git format-rev --stdin-mode=<mode> "
+ "--format=<pretty> [--[no-]notes=<ref>] "
+ "[-z] [--[no-]null-output] [--[no-]null-input]"),
+ NULL
+};
+
+int cmd_format_rev(int argc,
+ const char **argv,
+ const char *prefix,
+ struct repository *repo UNUSED)
+{
+ const char *format = NULL;
+ enum stdin_mode stdin_mode;
+ const char *stdin_mode_arg = NULL;
+ struct format_nul_data nul_data = { 0, 0 };
+ char output_terminator;
+ strbuf_getline_fn getline_fn;
+ struct display_notes_opt format_notes_opt;
+ struct rev_info format_rev = REV_INFO_INIT;
+ struct pretty_format format_pp = { 0 };
+ struct string_list notes = STRING_LIST_INIT_NODUP;
+ struct strbuf scratch_buf = STRBUF_INIT;
+ struct command cmd;
+ struct option opts[] = {
+ OPT_STRING(0, "format", &format, N_("format"),
+ N_("pretty format to use")),
+ OPT_STRING(0, "stdin-mode", &stdin_mode_arg, N_("stdin-mode"),
+ N_("how revs are processed")),
+ OPT_STRING_LIST(0, "notes", ¬es, N_("notes"),
+ N_("display notes for pretty format")),
+ OPT_CALLBACK_F('z', "null", &nul_data, N_("z"),
+ N_("Use NUL for input and output termination"),
+ PARSE_OPT_NOARG | PARSE_OPT_NONEG, format_nul_cb),
+ OPT_BOOL(0, "null-input", &nul_data.nul_input,
+ N_("Use NUL for input termination")),
+ OPT_BOOL(0, "null-output", &nul_data.nul_output,
+ N_("Use NUL for output termination")),
+ OPT_END(),
+ };
+
+ argc = parse_options(argc, argv, prefix, opts, format_rev_usage, 0);
+
+ if (argc > 0) {
+ error(_("too many arguments"));
+ usage_with_options(format_rev_usage, opts);
+ }
+
+ if (!format)
+ die(_("'%s' is required"), "--format");
+ if (!stdin_mode_arg)
+ die(_("'%s' is required"), "--stdin-mode");
+
+ getline_fn = nul_data.nul_input ? strbuf_getline_nul : strbuf_getline_lf;
+ output_terminator = nul_data.nul_output ? '\0' : '\n';
+
+ init_display_notes(&format_notes_opt);
+ stdin_mode = parse_stdin_mode(stdin_mode_arg);
+
+ get_commit_format(format, &format_rev);
+ format_pp.ctx.rev = &format_rev;
+ format_pp.ctx.fmt = format_rev.commit_format;
+ format_pp.ctx.abbrev = format_rev.abbrev;
+ format_pp.ctx.date_mode_explicit = format_rev.date_mode_explicit;
+ format_pp.ctx.date_mode = format_rev.date_mode;
+ format_pp.ctx.color = GIT_COLOR_AUTO;
+
+ userformat_find_requirements(format,
+ &format_pp.want);
+ if (format_pp.want.notes) {
+ int ignore_show_notes = 0;
+ struct string_list_item *n;
+
+ for_each_string_list_item(n, ¬es)
+ enable_ref_display_notes(&format_notes_opt,
+ &ignore_show_notes,
+ n->string);
+ load_display_notes(&format_notes_opt);
+ }
+
+ init_format_rev_command(&cmd, &format_pp);
+
+ switch (stdin_mode) {
+ case TEXT:
+ while (getline_fn(&scratch_buf, stdin) != EOF) {
+ name_rev_line(scratch_buf.buf, &cmd);
+ /*
+ * We do not pass on the terminator to name_rev_line,
+ * unlike name-rev.
+ */
+ printf("%c", output_terminator);
+ maybe_flush_or_die(stdout, "stdout");
+ }
+ break;
+ case REVS:
+ while (getline_fn(&scratch_buf, stdin) != EOF) {
+ struct object_id oid;
+ struct object *object;
+ struct object *peeled;
+
+ if (repo_get_oid(the_repository, scratch_buf.buf, &oid)) {
+ fprintf(stderr, "Could not get object name for %s. Skipping.\n",
+ scratch_buf.buf);
+ continue;
+ }
+
+ object = parse_object(the_repository, &oid);
+ if (!object) {
+ fprintf(stderr, "Could not get object for %s. Skipping.\n",
+ scratch_buf.buf);
+ continue;
+ }
+
+ peeled = deref_tag(the_repository, object, scratch_buf.buf, 0);
+ if (!peeled || peeled->type != OBJ_COMMIT) {
+ fprintf(stderr,
+ "Could not get commit for %s. Skipping.\n",
+ scratch_buf.buf);
+ continue;
+ }
+
+ get_format_rev((struct commit *)peeled,
+ &format_pp, &scratch_buf);
+ printf("%s%c", scratch_buf.buf, output_terminator);
+ maybe_flush_or_die(stdout, "stdout");
+ strbuf_release(&scratch_buf);
+ }
+ break;
+ default:
+ BUG("uncovered case: %d", stdin_mode);
+ }
+
+ strbuf_release(&scratch_buf);
+ string_list_clear(¬es, 0);
+ release_display_notes(&format_notes_opt);
+ return 0;
+}
diff --git a/command-list.txt b/command-list.txt
index f9005cf4597..df729872dca 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -108,6 +108,7 @@ git-fmt-merge-msg purehelpers
git-for-each-ref plumbinginterrogators
git-for-each-repo plumbinginterrogators
git-format-patch mainporcelain
+git-format-rev plumbinginterrogators
git-fsck ancillaryinterrogators complete
git-gc mainporcelain
git-get-tar-commit-id plumbinginterrogators
diff --git a/git.c b/git.c
index 2b212e6675d..af5b0422b00 100644
--- a/git.c
+++ b/git.c
@@ -578,6 +578,7 @@ static struct cmd_struct commands[] = {
{ "for-each-ref", cmd_for_each_ref, RUN_SETUP },
{ "for-each-repo", cmd_for_each_repo, RUN_SETUP_GENTLY },
{ "format-patch", cmd_format_patch, RUN_SETUP },
+ { "format-rev", cmd_format_rev, RUN_SETUP },
{ "fsck", cmd_fsck, RUN_SETUP },
{ "fsck-objects", cmd_fsck, RUN_SETUP },
{ "fsmonitor--daemon", cmd_fsmonitor__daemon, RUN_SETUP },
diff --git a/t/t1517-outside-repo.sh b/t/t1517-outside-repo.sh
index c824c1a25cf..360a9323343 100755
--- a/t/t1517-outside-repo.sh
+++ b/t/t1517-outside-repo.sh
@@ -114,7 +114,8 @@ do
archimport | citool | credential-netrc | credential-libsecret | \
credential-osxkeychain | cvsexportcommit | cvsimport | cvsserver | \
daemon | \
- difftool--helper | filter-branch | fsck-objects | get-tar-commit-id | \
+ difftool--helper | filter-branch | format-rev | fsck-objects | \
+ get-tar-commit-id | \
gui | gui--askpass | \
http-backend | http-fetch | http-push | init-db | \
merge-octopus | merge-one-file | merge-resolve | mergetool | \
diff --git a/t/t6120-describe.sh b/t/t6120-describe.sh
index 62789f76381..8ee3d2c37d0 100755
--- a/t/t6120-describe.sh
+++ b/t/t6120-describe.sh
@@ -801,4 +801,198 @@ test_expect_success 'do not be fooled by invalid describe format ' '
test_must_fail git cat-file -t "refs/tags/super-invalid/./../...../ ~^:/?*[////\\\\\\&}/busted.lock-42-g"$(cat out)
'
+test_expect_success 'setup: format-rev' '
+ mkdir repo-format &&
+ git -C repo-format init &&
+ test_commit -C repo-format first &&
+ test_commit -C repo-format second &&
+ test_commit -C repo-format third &&
+ test_commit -C repo-format fourth &&
+ test_commit -C repo-format fifth &&
+ test_commit -C repo-format sixth &&
+ test_commit -C repo-format seventh &&
+ test_commit -C repo-format eighth
+'
+
+test_expect_success 'format-rev --stdin-mode=revs' '
+ cat >expect <<-\EOF &&
+ eighth
+ seventh
+ fifth
+ EOF
+ git -C repo-format format-rev --stdin-mode=revs \
+ --format=%s >actual <<-\EOF &&
+ HEAD
+ HEAD~
+ HEAD~3
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'format-rev --stdin-mode=text from rev-list same as log' '
+ git -C repo-format log --format=reference >expect &&
+ test_file_not_empty expect &&
+ git -C repo-format rev-list HEAD >list &&
+ git -C repo-format format-rev --stdin-mode=text \
+ --format=reference <list >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'format-rev --stdin-mode=text with running text and tree oid' '
+ cmit_oid=$(git -C repo-format rev-parse fifth) &&
+ reference=$(git -C repo-format log -n1 --format=reference fifth) &&
+ tree=$(git -C repo-format rev-parse HEAD^{tree}) &&
+ cat >expect <<-EOF &&
+ We thought we fixed this in ${reference}.
+ But look at this tree: ${tree}.
+ EOF
+ git -C repo-format format-rev --stdin-mode=text --format=reference \
+ >actual <<-EOF &&
+ We thought we fixed this in ${cmit_oid}.
+ But look at this tree: ${tree}.
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'format-rev with %N (note)' '
+ test_when_finished "git -C repo-format notes remove" &&
+ git -C repo-format notes add -m"Make a note" &&
+ printf "Make a note\n\n\n" >expect &&
+ git -C repo-format format-rev --stdin-mode=revs \
+ --format="tformat:%N" \
+ >actual <<-\EOF &&
+ HEAD
+ HEAD~
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'format-rev --notes<ref> (custom notes ref)' '
+ # One custom notes ref
+ test_when_finished "git -C repo-format notes remove" &&
+ test_when_finished "git -C repo-format notes --ref=word remove" &&
+ git -C repo-format notes add -m"default" &&
+ git -C repo-format notes --ref=word add -m"custom" &&
+ printf "custom\n\n" >expect &&
+ git -C repo-format format-rev --stdin-mode=revs \
+ --format="tformat:%N" \
+ --notes=word \
+ >actual <<-\EOF &&
+ HEAD
+ EOF
+ test_cmp expect actual &&
+ # Glob all
+ printf "default\ncustom\n\n" >expect &&
+ git -C repo-format format-rev --stdin-mode=revs \
+ --format="tformat:%N" \
+ --notes=* >actual <<-\EOF &&
+ HEAD
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'format-rev --stdin-mode=revs on annotated tag peels to commit' '
+ test_when_finished "git -C repo-format tag -d version" &&
+ git -C repo-format tag -a -m"new version" version &&
+ cat >expect <<-\EOF &&
+ eighth
+ EOF
+ git -C repo-format format-rev --stdin-mode=revs \
+ --format=%s \
+ >actual <<-\EOF &&
+ version
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'format-rev --stdin-mode=revs lookup failures' '
+ test_when_finished "git -C repo-format tag -d tag-to-tree" &&
+ invalid_syntax=not-valid &&
+ non_existing_oid=${EMPTY_BLOB} &&
+ tree=$(git -C repo-format rev-parse eighth^{tree}) &&
+ git -C repo-format tag -a -mmessage tag-to-tree "$tree" &&
+ tag_to_tree=$(git -C repo-format rev-parse tag-to-tree) &&
+ cat >expect <<-EOF &&
+ Could not get object name for ${invalid_syntax}. Skipping.
+ Could not get object for ${non_existing_oid}. Skipping.
+ Could not get commit for ${tree}. Skipping.
+ Could not get commit for ${tag_to_tree}. Skipping.
+ EOF
+ git -C repo-format format-rev --stdin-mode=revs \
+ --format=%s \
+ 2>actual >out <<-EOF &&
+ ${invalid_syntax}
+ ${non_existing_oid}
+ ${tree}
+ ${tag_to_tree}
+ EOF
+ test_line_count = 0 out &&
+ test_cmp expect actual
+'
+
+
+test_expect_success 'format-rev -z --stdin-mode=text with object name lookup failures' '
+ printf "%s\0" "$(git -C repo-format rev-parse HEAD)" >input &&
+ printf "%s\0" "$(git -C repo-format rev-parse HEAD^{tree})" >>input &&
+ printf "%s\0" "$EMPTY_BLOB" >>input &&
+ printf "%s\0" "$(git -C repo-format log --format=%s -1)" >expect &&
+ printf "%s\0" "$(git -C repo-format rev-parse HEAD^{tree})" >>expect &&
+ printf "%s\0" "$EMPTY_BLOB" >>expect &&
+ git -C repo-format format-rev --stdin-mode=text \
+ --format=%s -z <input >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'setup: format-rev input and output separators' '
+ git -C repo-format rev-list HEAD >input-lf &&
+ git -C repo-format rev-list -z HEAD >input-nul &&
+ git -C repo-format log --format=%s >output-lf &&
+ git -C repo-format log -z --format=%s >output-nul &&
+ echo revs >stdin-modes &&
+ echo text >>stdin-modes
+'
+
+while read mode
+do
+ test_expect_success "format-rev -z --stdin-mode=$mode" '
+ cat output-nul >expect &&
+ git -C repo-format format-rev --stdin-mode="$mode" \
+ --format=%s -z <input-nul >actual &&
+ test_cmp expect actual
+ '
+
+ test_expect_success "format-rev -z --no-null-input --no-null-output --stdin-mode=$mode" '
+ cat output-lf >expect &&
+ git -C repo-format format-rev --stdin-mode="$mode" \
+ --format=%s -z --no-null-input --no-null-output \
+ <input-lf >actual &&
+ test_cmp expect actual
+ '
+
+ test_expect_success "format-rev ---null-input --stdin-mode=$mode" '
+ cat output-lf >expect &&
+ git -C repo-format format-rev --stdin-mode="$mode" \
+ --format=%s --null-input \
+ <input-nul >actual &&
+ test_cmp expect actual
+ '
+
+ test_expect_success "format-rev --null-output --stdin-mode=$mode" '
+ cat output-nul >expect &&
+ git -C repo-format format-rev --stdin-mode="$mode" \
+ --format=%s --null-output \
+ <input-lf >actual &&
+ test_cmp expect actual
+ '
+
+ test_expect_success "format-rev -z --stdin-mode=$mode with multi-line output" '
+ format="%s%n%aI" &&
+ git -C repo-format log -z --format="$format" \
+ >expect &&
+ git -C repo-format format-rev --stdin-mode="$mode" \
+ --format="$format" -z <input-nul >actual &&
+ test_cmp expect actual
+ '
+done <stdin-modes
+
test_done
--
2.54.0.13.g9c7419e39f8
^ permalink raw reply related
* [PATCH v5 4/5] name-rev: make dedicated --annotate-stdin --name-only test
From: kristofferhaugsbakk @ 2026-05-11 15:45 UTC (permalink / raw)
To: Junio C Hamano
Cc: Kristoffer Haugsbakk, git, ben.knoble, Phillip Wood, Ramsay Jones
In-Reply-To: <V5_CV_format-rev.6c9@msgid.xyz>
From: Kristoffer Haugsbakk <code@khaugsbakk.name>
The previous commit split the `--name-only` handling:
1. `--annotate-stdin`: uses the new `struct command`
2. The rest: uses `struct name_ref_data`
But there is no dedicated test for the option combination in (1). That
means that the following tests will fail if you neglect to set
`command.u.name_only` properly:
name-rev --annotate-stdin works with commitGraph
name-rev --annotate-stdin works with non-monotonic timestamps
even though it has nothing to do with what these tests are supposed
to test.
Let’s add another regression test now that it is relevant.
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
---
t/t6120-describe.sh | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/t/t6120-describe.sh b/t/t6120-describe.sh
index 2c70cc561ad..62789f76381 100755
--- a/t/t6120-describe.sh
+++ b/t/t6120-describe.sh
@@ -298,6 +298,20 @@ test_expect_success 'name-rev --annotate-stdin' '
test_cmp expect actual
'
+test_expect_success 'name-rev --annotate-stdin --name-only' '
+ >expect.unsorted &&
+ for rev in $(git rev-list --all)
+ do
+ name=$(git name-rev --name-only $rev) &&
+ echo "$name" >>expect.unsorted || return 1
+ done &&
+ sort <expect.unsorted >expect &&
+ git name-rev --annotate-stdin --name-only \
+ <list >actual.unsorted &&
+ sort <actual.unsorted >actual &&
+ test_cmp expect actual
+'
+
test_expect_success 'name-rev --stdin deprecated' '
git rev-list --all >list &&
if ! test_have_prereq WITH_BREAKING_CHANGES
--
2.54.0.13.g9c7419e39f8
^ permalink raw reply related
* [PATCH v5 3/5] name-rev: factor code for sharing with a new command
From: kristofferhaugsbakk @ 2026-05-11 15:45 UTC (permalink / raw)
To: Junio C Hamano
Cc: Kristoffer Haugsbakk, git, ben.knoble, Phillip Wood, Ramsay Jones
In-Reply-To: <V5_CV_format-rev.6c9@msgid.xyz>
From: Kristoffer Haugsbakk <code@khaugsbakk.name>
We are about to introduce a new command git-format-rev(1) to this
file. Let’s factor some code so that we can share it with the new
command.
We want to be able to format commits found in freeform text, and
git-name-rev(1) already has a function for that but for symbolic
names. Let’s use a tagged union for the command-specific payload.
No functional changes.
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
---
Notes (series):
v4:
• Pull out `*(p + 1)` instead of doing it in every `case` (Phillip)
• Go back to using `continue` instead of `goto` (Phillip)
builtin/name-rev.c | 53 ++++++++++++++++++++++++++++++++++------------
1 file changed, 40 insertions(+), 13 deletions(-)
diff --git a/builtin/name-rev.c b/builtin/name-rev.c
index 6357eaa76d0..475efb0b82b 100644
--- a/builtin/name-rev.c
+++ b/builtin/name-rev.c
@@ -272,6 +272,24 @@ struct name_ref_data {
struct string_list exclude_filters;
};
+enum command_type {
+ NAME_REV = 1,
+};
+
+struct command {
+ enum command_type type;
+ union {
+ int name_only;
+ } u;
+};
+
+static void init_name_rev_command(struct command *cmd,
+ int name_only)
+{
+ cmd->type = NAME_REV;
+ cmd->u.name_only = name_only;
+}
+
static struct tip_table {
struct tip_table_entry {
struct object_id oid;
@@ -507,7 +525,7 @@ static char const * const name_rev_usage[] = {
NULL
};
-static void name_rev_line(char *p, struct name_ref_data *data)
+static void name_rev_line(char *p, struct command *cmd)
{
struct strbuf buf = STRBUF_INIT;
int counter = 0;
@@ -524,25 +542,32 @@ static void name_rev_line(char *p, struct name_ref_data *data)
const char *name = NULL;
char c = *(p + 1);
int p_len = p - p_start + 1;
+ struct object *o = NULL;
+ int oid_ret = 1;
counter = 0;
*(p + 1) = 0;
- if (!repo_get_oid(the_repository, p - (hexsz - 1), &oid)) {
- struct object *o =
- lookup_object(the_repository, &oid);
+ oid_ret = repo_get_oid(the_repository, p - (hexsz - 1), &oid);
+ *(p + 1) = c;
+
+ switch (cmd->type) {
+ case NAME_REV:
+ if (!oid_ret)
+ o = lookup_object(the_repository, &oid);
if (o)
name = get_rev_name(o, &buf);
+ if (!name)
+ continue;
+ if (cmd->u.name_only)
+ printf("%.*s%s", p_len - hexsz, p_start, name);
+ else
+ printf("%.*s (%s)", p_len, p_start, name);
+ break;
+ default:
+ BUG("uncovered case: %d", cmd->type);
}
- *(p + 1) = c;
-
- if (!name)
- continue;
- if (data->name_only)
- printf("%.*s%s", p_len - hexsz, p_start, name);
- else
- printf("%.*s (%s)", p_len, p_start, name);
p_start = p + 1;
}
}
@@ -567,6 +592,7 @@ int cmd_name_rev(int argc,
#endif
int all = 0, annotate_stdin = 0, allow_undefined = 1, always = 0, peel_tag = 0;
struct name_ref_data data = { 0, 0, STRING_LIST_INIT_NODUP, STRING_LIST_INIT_NODUP };
+ struct command cmd;
struct option opts[] = {
OPT_BOOL(0, "name-only", &data.name_only, N_("print only ref-based names (no object names)")),
OPT_BOOL(0, "tags", &data.tags_only, N_("only use tags to name the commits")),
@@ -596,6 +622,7 @@ int cmd_name_rev(int argc,
init_commit_rev_name(&rev_names);
repo_config(the_repository, git_default_config, NULL);
argc = parse_options(argc, argv, prefix, opts, name_rev_usage, 0);
+ init_name_rev_command(&cmd, data.name_only);
#ifndef WITH_BREAKING_CHANGES
if (transform_stdin) {
@@ -663,7 +690,7 @@ int cmd_name_rev(int argc,
while (strbuf_getline(&sb, stdin) != EOF) {
strbuf_addch(&sb, '\n');
- name_rev_line(sb.buf, &data);
+ name_rev_line(sb.buf, &cmd);
}
strbuf_release(&sb);
} else if (all) {
--
2.54.0.13.g9c7419e39f8
^ permalink raw reply related
* [PATCH v5 2/5] name-rev: run clang-format before factoring code
From: kristofferhaugsbakk @ 2026-05-11 15:45 UTC (permalink / raw)
To: Junio C Hamano
Cc: Kristoffer Haugsbakk, git, ben.knoble, Phillip Wood, Ramsay Jones
In-Reply-To: <V5_CV_format-rev.6c9@msgid.xyz>
From: Kristoffer Haugsbakk <code@khaugsbakk.name>
We are about to move code around to prepare for adding a new
command. Let’s deal with clang-format changes first in the affected
areas.
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
---
builtin/name-rev.c | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/builtin/name-rev.c b/builtin/name-rev.c
index 171e7bd0e98..6357eaa76d0 100644
--- a/builtin/name-rev.c
+++ b/builtin/name-rev.c
@@ -519,22 +519,22 @@ static void name_rev_line(char *p, struct name_ref_data *data)
if (!ishex(*p)) {
counter = 0;
} else if (++counter == hexsz &&
- !ishex(*(p+1))) {
+ !ishex(*(p + 1))) {
struct object_id oid;
const char *name = NULL;
- char c = *(p+1);
+ char c = *(p + 1);
int p_len = p - p_start + 1;
counter = 0;
- *(p+1) = 0;
+ *(p + 1) = 0;
if (!repo_get_oid(the_repository, p - (hexsz - 1), &oid)) {
struct object *o =
lookup_object(the_repository, &oid);
if (o)
name = get_rev_name(o, &buf);
}
- *(p+1) = c;
+ *(p + 1) = c;
if (!name)
continue;
@@ -571,9 +571,9 @@ int cmd_name_rev(int argc,
OPT_BOOL(0, "name-only", &data.name_only, N_("print only ref-based names (no object names)")),
OPT_BOOL(0, "tags", &data.tags_only, N_("only use tags to name the commits")),
OPT_STRING_LIST(0, "refs", &data.ref_filters, N_("pattern"),
- N_("only use refs matching <pattern>")),
+ N_("only use refs matching <pattern>")),
OPT_STRING_LIST(0, "exclude", &data.exclude_filters, N_("pattern"),
- N_("ignore refs matching <pattern>")),
+ N_("ignore refs matching <pattern>")),
OPT_GROUP(""),
OPT_BOOL(0, "all", &all, N_("list all commits reachable from all refs")),
#ifndef WITH_BREAKING_CHANGES
@@ -585,10 +585,10 @@ int cmd_name_rev(int argc,
#endif /* WITH_BREAKING_CHANGES */
OPT_BOOL(0, "annotate-stdin", &annotate_stdin, N_("annotate text from stdin")),
OPT_BOOL(0, "undefined", &allow_undefined, N_("allow to print `undefined` names (default)")),
- OPT_BOOL(0, "always", &always,
- N_("show abbreviated commit object as fallback")),
+ OPT_BOOL(0, "always", &always,
+ N_("show abbreviated commit object as fallback")),
OPT_HIDDEN_BOOL(0, "peel-tag", &peel_tag,
- N_("dereference tags in the input (internal use)")),
+ N_("dereference tags in the input (internal use)")),
OPT_END(),
};
--
2.54.0.13.g9c7419e39f8
^ permalink raw reply related
* [PATCH v5 1/5] name-rev: wrap both blocks in braces
From: kristofferhaugsbakk @ 2026-05-11 15:45 UTC (permalink / raw)
To: Junio C Hamano
Cc: Kristoffer Haugsbakk, git, ben.knoble, Phillip Wood, Ramsay Jones
In-Reply-To: <V5_CV_format-rev.6c9@msgid.xyz>
From: Kristoffer Haugsbakk <code@khaugsbakk.name>
See `CodingGuidelines`:
- When there are multiple arms to a conditional and some of them
require braces, enclose even a single line block in braces for
consistency. [...]
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
---
Notes (series):
v2:
Fix stray formatting of `(p+1)`
builtin/name-rev.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/builtin/name-rev.c b/builtin/name-rev.c
index 6188cf98ce0..171e7bd0e98 100644
--- a/builtin/name-rev.c
+++ b/builtin/name-rev.c
@@ -466,9 +466,9 @@ static const char *get_rev_name(const struct object *o, struct strbuf *buf)
if (!n)
return NULL;
- if (!n->generation)
+ if (!n->generation) {
return n->tip_name;
- else {
+ } else {
strbuf_reset(buf);
strbuf_addstr(buf, n->tip_name);
strbuf_strip_suffix(buf, "^0");
@@ -516,9 +516,9 @@ static void name_rev_line(char *p, struct name_ref_data *data)
for (p_start = p; *p; p++) {
#define ishex(x) (isdigit((x)) || ((x) >= 'a' && (x) <= 'f'))
- if (!ishex(*p))
+ if (!ishex(*p)) {
counter = 0;
- else if (++counter == hexsz &&
+ } else if (++counter == hexsz &&
!ishex(*(p+1))) {
struct object_id oid;
const char *name = NULL;
--
2.54.0.13.g9c7419e39f8
^ permalink raw reply related
* [PATCH v5 0/5] format-rev: introduce builtin for on-demand pretty formatting
From: kristofferhaugsbakk @ 2026-05-11 15:45 UTC (permalink / raw)
To: Junio C Hamano
Cc: Kristoffer Haugsbakk, git, ben.knoble, Phillip Wood, Ramsay Jones
In-Reply-To: <V4_CV_format-rev.6aa@msgid.xyz>
From: Kristoffer Haugsbakk <code@khaugsbakk.name>
(Subject from v2: name-rev: learn --format=<pretty>)
Topic name (applied): kh/name-rev-custom-format
Topic summary: Introduce a new builtin for pretty formatting either (1) one
revision expression per line or (2) commit object names found in running
text.
See the last patch for the motivation. In short there isn’t anything that I
have found that lets you format however many commits you want through one
process (so looping over `git show` is excepted).
The other patches prepare for this change.
§ Changes in v5
Notes from patch 5/5, all doc changes:
• Fix definition list mistake
• No blank line between the two modes
• Also replace `::` delimiters with `;;`. This is not strictly
needed since the definition list is inside an open block. But
it is consistent with all other definition list in definition
lists I’ve seen. (The command option section is a long
definition list.)
• Rewrite part that I was unhappy about regarding
`--stdin-mode=text` and `-z`:
https://lore.kernel.org/git/c04d9cf9-e6a9-4e12-8025-9baededfdafc@app.fastmail.com/
• Replace `abcdef012...` with just “some object name”. The object
name is arbitrary and my AsciiDoc output seems to do some weird
things for `...` inside backticks for HTML.
• Replace “this problem is just another” with “this problem
*is solved* with...”
§ Link to v4
https://lore.kernel.org/git/V4_CV_format-rev.6aa@msgid.xyz/
[1/5] name-rev: wrap both blocks in braces
[2/5] name-rev: run clang-format before factoring code
[3/5] name-rev: factor code for sharing with a new command
[4/5] name-rev: make dedicated --annotate-stdin --name-only test
[5/5] format-rev: introduce builtin for on-demand pretty formatting
.gitignore | 1 +
Documentation/git-format-rev.adoc | 215 +++++++++++++++++++++
Documentation/meson.build | 1 +
Makefile | 1 +
builtin.h | 1 +
builtin/name-rev.c | 300 +++++++++++++++++++++++++++---
command-list.txt | 1 +
git.c | 1 +
t/t1517-outside-repo.sh | 3 +-
t/t6120-describe.sh | 208 +++++++++++++++++++++
10 files changed, 706 insertions(+), 26 deletions(-)
create mode 100644 Documentation/git-format-rev.adoc
Interdiff against v4:
diff --git a/Documentation/git-format-rev.adoc b/Documentation/git-format-rev.adoc
index 436980012bc..c40d52e9f6d 100644
--- a/Documentation/git-format-rev.adoc
+++ b/Documentation/git-format-rev.adoc
@@ -25,13 +25,14 @@ OPTIONS
How to interpret standard input data:
+
--
-`revs`:: Each line or record (see the <<io,INPUT AND OUTPUT FORMATS>>
+`revs`;; Each line or record (see the <<io,INPUT AND OUTPUT FORMATS>>
section) is interpreted as a commit. Any kind of revision
expression can be used (see linkgit:gitrevisions[7]). Annotated
tags are peeled (see linkgit:gitglossary[7]).
+
The argument `rev` is also accepted.
-`text`:: Formats all commit object names found in freeform text. These
+
+`text`;; Formats all commit object names found in freeform text. These
must the full object names, i.e. abbreviated hexidecimal object
names will not be interpreted.
+
@@ -53,8 +54,12 @@ commit object name is left alone (echoed).
of newline. This option cannot be negated.
+
This is useful if both the input and output could contain newlines or if
-the input could contain _NUL_ characters; see the <<io,INPUT AND OUTPUT
-FORMATS>> section.
+the input to this command also uses _NUL_ character termination; see the
+<<io,INPUT AND OUTPUT FORMATS>> section below.
++
+The mode `--stdin-mode=text` can have use for this option when it needs
+to process input like for example `git last-modified -z`; see the
+<<examples,EXAMPLES>> section below.
`--null-output`::
`--no-null-output`::
@@ -70,8 +75,6 @@ This is useful if the output could contain newlines, for example if the
default is `--no-null-input`.
+
This is useful if the input revision expressions could contain newlines.
-It is also useful if the input could contain _NUL_ characters; see the
-<<io,INPUT AND OUTPUT FORMATS>> section.
[[io]]
INPUT AND OUTPUT FORMAT
@@ -90,16 +93,13 @@ acts as a _terminator_, not a _separator_. In other words, the final
line or record is also terminated by the terminator character.
The mode `--stdin-mode=text` replaces each object name with the
-formatted commit, i.e. the format `%s` would transform the object name
-`abcdef012...` to `<subject>` without any termination. Like this:
+formatted commit, i.e. the format `%s` would transform some commit
+object name to `<subject>` without any termination. Like this:
----
Did we not fix this in "<subject>"?
----
-Regarding input in this mode: using `-z` or `--null-input` makes sure
-that _NUL_ characters in the input are passed through correctly.
-
It is safe to interactively read and write from this command since each
record is immediately flushed.
@@ -202,8 +202,8 @@ not a good fit for the above use case.
In short, it is straightforward to use these two commands if you use one
process per line. It is much more work if you just want to use one
-process, but still doable. In contrast, this problem is just another
-shell pipeline with this command.
+process, but still doable. In contrast, this problem is solved with just
+another shell pipeline with this command.
SEE ALSO
--------
Range-diff against v4:
1: 9cb5cfd1ec3 = 1: 9cb5cfd1ec3 name-rev: wrap both blocks in braces
2: 14900271321 = 2: 14900271321 name-rev: run clang-format before factoring code
3: 724ec022894 = 3: 724ec022894 name-rev: factor code for sharing with a new command
4: 382efc3ddb8 = 4: 382efc3ddb8 name-rev: make dedicated --annotate-stdin --name-only test
5: 049a45e32bc ! 5: 425eb16728c format-rev: introduce builtin for on-demand pretty formatting
@@ Documentation/git-format-rev.adoc (new)
+ How to interpret standard input data:
++
+--
-+`revs`:: Each line or record (see the <<io,INPUT AND OUTPUT FORMATS>>
++`revs`;; Each line or record (see the <<io,INPUT AND OUTPUT FORMATS>>
+ section) is interpreted as a commit. Any kind of revision
+ expression can be used (see linkgit:gitrevisions[7]). Annotated
+ tags are peeled (see linkgit:gitglossary[7]).
++
+The argument `rev` is also accepted.
-+`text`:: Formats all commit object names found in freeform text. These
++
++`text`;; Formats all commit object names found in freeform text. These
+ must the full object names, i.e. abbreviated hexidecimal object
+ names will not be interpreted.
++
@@ Documentation/git-format-rev.adoc (new)
+ of newline. This option cannot be negated.
++
+This is useful if both the input and output could contain newlines or if
-+the input could contain _NUL_ characters; see the <<io,INPUT AND OUTPUT
-+FORMATS>> section.
++the input to this command also uses _NUL_ character termination; see the
++<<io,INPUT AND OUTPUT FORMATS>> section below.
+++
++The mode `--stdin-mode=text` can have use for this option when it needs
++to process input like for example `git last-modified -z`; see the
++<<examples,EXAMPLES>> section below.
+
+`--null-output`::
+`--no-null-output`::
@@ Documentation/git-format-rev.adoc (new)
+ default is `--no-null-input`.
++
+This is useful if the input revision expressions could contain newlines.
-+It is also useful if the input could contain _NUL_ characters; see the
-+<<io,INPUT AND OUTPUT FORMATS>> section.
+
+[[io]]
+INPUT AND OUTPUT FORMAT
@@ Documentation/git-format-rev.adoc (new)
+line or record is also terminated by the terminator character.
+
+The mode `--stdin-mode=text` replaces each object name with the
-+formatted commit, i.e. the format `%s` would transform the object name
-+`abcdef012...` to `<subject>` without any termination. Like this:
++formatted commit, i.e. the format `%s` would transform some commit
++object name to `<subject>` without any termination. Like this:
+
+----
+Did we not fix this in "<subject>"?
+----
+
-+Regarding input in this mode: using `-z` or `--null-input` makes sure
-+that _NUL_ characters in the input are passed through correctly.
-+
+It is safe to interactively read and write from this command since each
+record is immediately flushed.
+
@@ Documentation/git-format-rev.adoc (new)
+
+In short, it is straightforward to use these two commands if you use one
+process per line. It is much more work if you just want to use one
-+process, but still doable. In contrast, this problem is just another
-+shell pipeline with this command.
++process, but still doable. In contrast, this problem is solved with just
++another shell pipeline with this command.
+
+SEE ALSO
+--------
base-commit: 67006b9db8b772423ad0706029286096307d2567
--
2.54.0.13.g9c7419e39f8
^ permalink raw reply related
* Re: Noômen: Gifted Invite Enclosed (Github Community Partnership)
From: Sarah J @ 2026-05-11 15:38 UTC (permalink / raw)
To: git
In-Reply-To: <019e0d05-2060-7b52-8a61-cce41dbc2e52@devthusiastcraft.com>
Noômen, just checking if you saw my previous email.
Your GitHub profile was selected for a free lifetime membership to Devthusiast (normally $1,800/year).
If you want to join the newsletter, just reply “yes.”
Otherwise we’ll pass the spot to another engineer in 7 days.
All the best,
Team @ Devthusiast
On Sat, May 9, 2026 1:55 PM, Sarah J <sarah.j@devthusiastcraft.com>
[sarah.j@devthusiastcraft.com]> wrote:
> Hey Noômen
>
> Annually, we pick engineers from Github, and your GitHub profile https://github.com/bnhassin was selected this year.
>
> You're officially invited to a lifetime membership to devthusiast, our email newsletter for tech founders that love to tinker. And because we selected your profile, it's completely free for you.
>
> Some of what you can expect to find in our daily newsletter:
>
> - Latest in AI: Latest AI news from our inside sources at OpenAI, Anthropic and Google
> - VC Radar: The latest tech funding news, before they come out on Tech Crunch
> - AI Wars, Model Power Rankings: Today’s leaderboard of the top AI models
> - Tinker of the Week: One useful open-source tool that is flying under the radar
>
> Please respond with “yes” to acknowledge receipt of this message, or we will have to choose a different profile. Once confirmed you will get your first newsletter edition!
>
> Welcome,
> Team @ Devthusiast
>
^ permalink raw reply
* Re: [PATCH v2 00/10] pack-objects: integrate --path-walk and some --filter options
From: Derrick Stolee @ 2026-05-11 13:58 UTC (permalink / raw)
To: Junio C Hamano
Cc: Taylor Blau, Derrick Stolee via GitGitGadget, git,
christian.couder, johannes.schindelin, johncai86, karthik.188,
kristofferhaugsbakk, newren, peff, ps
In-Reply-To: <xmqq8q9qtzly.fsf@gitster.g>
On 5/10/2026 11:05 PM, Junio C Hamano wrote:
> Derrick Stolee <stolee@gmail.com> writes:
>
>> On 5/5/2026 3:01 PM, Taylor Blau wrote:
>>> On Tue, May 05, 2026 at 12:18:28PM -0400, Derrick Stolee wrote:
>>> ...
>>> I think this is a consequence of us not sending directly-referenced
>>> blobs with `--filter=blob:none` when running the filters through
>>> `--path-walk`. Something like:
>>> ...
>> Thanks for this suggestion. I got pulled away from my investigation, so
>> wasn't to this point yet.
>>> fixes t5310 for me. I haven't looked into any of the other failures yet
>>> since you mentioned that you're looking into them, but let me know if
>>> you want to tag-team any of these.
>>>
>>> (As a related side-note, I noticed that GIT_TEST_PACK_PATH_WALK=1 is not
>>> currently in the TEST-vars CI build. I'm not sure if there are
>>> historical reasons for leaving it out, but if not I think it would be
>>> worthwhile to add it.)
>> I think the initial idea was that the feature was too niche to add it to
>> the CI builds right away. Your series is going to make it a lot more
>> important, so adding this to CI builds may be valuable.
>
> Should I expect a new [v3] iteration anytime soon? Not getting
> antsy, but just going through the "What's cooking" report to see if
> there are things I should immediately advance.
I hope to have a new version today. It was a more substantial change
getting things working correctly and in the right points in time. Lots
of rebases and rewrites have been happening.
Thanks,
-Stolee
^ permalink raw reply
* [PATCH v8] checkout: extend --track with a "fetch" mode to refresh start-point
From: Harald Nordgren via GitGitGadget @ 2026-05-11 13:47 UTC (permalink / raw)
To: git
Cc: Ramsay Jones, D. Ben Knoble, Kristoffer Haugsbakk, Marc Branchaud,
Phillip Wood, Harald Nordgren, Harald Nordgren
In-Reply-To: <pull.2281.v7.git.git.1778280727849.gitgitgadget@gmail.com>
From: Harald Nordgren <haraldnordgren@gmail.com>
If you want to fork your topic branch from the very latest of the
tip of a branch your remote has, you would do:
git fetch origin some-branch
git checkout -b new_branch --track origin/some-branch
Extend the "--track" option of "git checkout" and allow users to
write
git checkout -b new_branch --track=fetch origin/some-branch
to (1) fetch 'some-branch' from the remote 'origin', updating the
remote-tracking branch 'origin/some-branch', (2) arrange subsequent
'git pull' on 'new_branch' to interact with 'origin/some-branch' and
(3) fork 'new_branch' from it.
In the value of the '--track' option, 'fetch' can be combined with
the existing 'direct' (default) and 'inherit' modes via a
comma-separated list. Examples:
git checkout -b new_branch --track=fetch,inherit some_local_branch
git switch -c new_branch --track=fetch origin/some-branch
When "fetch" is requested and <start-point> is in <remote>/<branch>
form, run "git fetch <remote> <branch>" before resolving the ref, so
that other remote-tracking branches are left untouched. If
<start-point> is a bare remote name like "origin" (which resolves to
that remote's default branch), "git fetch <remote>" is run instead,
since the target branch is not known up front. Abort the checkout if
the fetch fails.
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
---
checkout: --track=fetch
Check the fallback ref using arg directly instead of reconstructing it.
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2281%2FHaraldNordgren%2Fcheckout-fetch-start-point-v8
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2281/HaraldNordgren/checkout-fetch-start-point-v8
Pull-Request: https://github.com/git/git/pull/2281
Range-diff vs v7:
1: de375d55f1 ! 1: 61c2199fd5 checkout: extend --track with a "fetch" mode to refresh start-point
@@ builtin/checkout.c
#include "revision.h"
+#include "run-command.h"
#include "setup.h"
-+#include "strvec.h"
+ #include "strvec.h"
#include "submodule.h"
- #include "symlinks.h"
- #include "trace2.h"
@@ builtin/checkout.c: struct checkout_opts {
int count_checkout_paths;
int overlay_mode;
@@ builtin/checkout.c: struct branch_info {
+ if (resolve_fetch_target(arg, &remote_name, &src_ref))
+ return;
+
-+ if (src_ref) {
-+ const char *short_src = src_ref;
++ {
+ struct object_id oid;
+
-+ skip_prefix(short_src, "refs/heads/", &short_src);
-+ strbuf_addf(&dst_ref, "refs/remotes/%s/%s", remote_name, short_src);
++ if (strchr(arg, '/'))
++ strbuf_addf(&dst_ref, "refs/remotes/%s", arg);
++ else
++ strbuf_addf(&dst_ref, "refs/remotes/%s/HEAD", arg);
+ if (!refs_read_ref(get_main_ref_store(the_repository),
+ dst_ref.buf, &oid))
+ have_existing_ref = 1;
Documentation/git-checkout.adoc | 13 ++-
Documentation/git-switch.adoc | 13 ++-
builtin/checkout.c | 168 +++++++++++++++++++++++++++++++-
t/t7201-co.sh | 132 +++++++++++++++++++++++++
4 files changed, 319 insertions(+), 7 deletions(-)
diff --git a/Documentation/git-checkout.adoc b/Documentation/git-checkout.adoc
index 43ccf47cf6..28f17f427e 100644
--- a/Documentation/git-checkout.adoc
+++ b/Documentation/git-checkout.adoc
@@ -158,11 +158,22 @@ of it").
resets _<branch>_ to the start point instead of failing.
`-t`::
-`--track[=(direct|inherit)]`::
+`--track[=(direct|inherit|fetch)[,...]]`::
When creating a new branch, set up "upstream" configuration. See
`--track` in linkgit:git-branch[1] for details. As a convenience,
--track without -b implies branch creation.
+
+The argument is a comma-separated list. `direct` (the default) and
+`inherit` select the tracking mode and are mutually exclusive. Adding
+`fetch` requests that the remote be fetched before _<start-point>_ is
+resolved, so the new branch starts from a fresh tip: when
+_<start-point>_ is in _<remote>/<branch>_ form, only that branch is
+updated; when _<start-point>_ is a bare remote name (e.g. `origin`),
+only the remote's default branch is updated. If the fetch fails and the
+corresponding remote-tracking ref already exists, a warning is printed
+and the checkout proceeds from the existing tip; otherwise the checkout
+is aborted.
++
If no `-b` option is given, the name of the new branch will be
derived from the remote-tracking branch, by looking at the local part of
the refspec configured for the corresponding remote, and then stripping
diff --git a/Documentation/git-switch.adoc b/Documentation/git-switch.adoc
index 87707e9265..3f54cf39e9 100644
--- a/Documentation/git-switch.adoc
+++ b/Documentation/git-switch.adoc
@@ -154,11 +154,22 @@ should result in deletion of the path).
attached to a terminal, regardless of `--quiet`.
`-t`::
-`--track[ (direct|inherit)]`::
+`--track[=(direct|inherit|fetch)[,...]]`::
When creating a new branch, set up "upstream" configuration.
`-c` is implied. See `--track` in linkgit:git-branch[1] for
details.
+
+The argument is a comma-separated list. `direct` (the default) and
+`inherit` select the tracking mode and are mutually exclusive. Adding
+`fetch` requests that the remote be fetched before _<start-point>_ is
+resolved, so the new branch starts from a fresh tip: when
+_<start-point>_ is in _<remote>/<branch>_ form, only that branch is
+updated; when _<start-point>_ is a bare remote name (e.g. `origin`),
+only the remote's default branch is updated. If the fetch fails and the
+corresponding remote-tracking ref already exists, a warning is printed
+and the switch proceeds from the existing tip; otherwise the switch is
+aborted.
++
If no `-c` option is given, the name of the new branch will be derived
from the remote-tracking branch, by looking at the local part of the
refspec configured for the corresponding remote, and then stripping
diff --git a/builtin/checkout.c b/builtin/checkout.c
index ac0186a33e..157242bc9f 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -26,10 +26,12 @@
#include "preload-index.h"
#include "read-cache.h"
#include "refs.h"
+#include "refspec.h"
#include "remote.h"
#include "repo-settings.h"
#include "resolve-undo.h"
#include "revision.h"
+#include "run-command.h"
#include "setup.h"
#include "strvec.h"
#include "submodule.h"
@@ -62,6 +64,7 @@ struct checkout_opts {
int count_checkout_paths;
int overlay_mode;
int dwim_new_local_branch;
+ int fetch;
int discard_changes;
int accept_ref;
int accept_pathspec;
@@ -113,6 +116,157 @@ struct branch_info {
char *checkout;
};
+static int resolve_fetch_target(const char *arg, char **remote_out,
+ char **src_ref_out)
+{
+ const char *slash;
+ char *remote_name;
+ struct remote *remote;
+ struct refspec_item query = { 0 };
+ struct strbuf dst = STRBUF_INIT;
+ const char *rest;
+
+ *remote_out = NULL;
+ *src_ref_out = NULL;
+
+ if (!arg || !*arg)
+ return -1;
+
+ slash = strchr(arg, '/');
+ if (slash == arg)
+ return -1;
+ remote_name = slash ? xstrndup(arg, slash - arg) : xstrdup(arg);
+
+ remote = remote_get(remote_name);
+ if (!remote || !remote_is_configured(remote, 1)) {
+ free(remote_name);
+ return -1;
+ }
+
+ rest = (slash && slash[1]) ? slash + 1 : NULL;
+ if (!rest) {
+ struct object_id oid;
+ const char *head_target;
+ const char *short_target;
+
+ strbuf_addf(&dst, "refs/remotes/%s/HEAD", remote_name);
+ head_target = refs_resolve_ref_unsafe(get_main_ref_store(the_repository),
+ dst.buf,
+ RESOLVE_REF_READING |
+ RESOLVE_REF_NO_RECURSE,
+ &oid, NULL);
+ strbuf_reset(&dst);
+ if (head_target &&
+ skip_prefix(head_target, "refs/remotes/", &short_target) &&
+ skip_prefix(short_target, remote_name, &short_target) &&
+ *short_target == '/')
+ rest = short_target + 1;
+ }
+
+ if (rest) {
+ strbuf_addf(&dst, "refs/remotes/%s/%s", remote_name, rest);
+ query.dst = dst.buf;
+ if (!remote_find_tracking(remote, &query) && query.src) {
+ *src_ref_out = xstrdup(query.src);
+ free(query.src);
+ } else {
+ *src_ref_out = xstrdup(rest);
+ }
+ }
+
+ strbuf_release(&dst);
+ *remote_out = remote_name;
+ return 0;
+}
+
+static void fetch_remote_for_start_point(const char *arg)
+{
+ char *remote_name = NULL;
+ char *src_ref = NULL;
+ struct child_process cmd = CHILD_PROCESS_INIT;
+ struct strbuf dst_ref = STRBUF_INIT;
+ int have_existing_ref = 0;
+
+ if (resolve_fetch_target(arg, &remote_name, &src_ref))
+ return;
+
+ {
+ struct object_id oid;
+
+ if (strchr(arg, '/'))
+ strbuf_addf(&dst_ref, "refs/remotes/%s", arg);
+ else
+ strbuf_addf(&dst_ref, "refs/remotes/%s/HEAD", arg);
+ if (!refs_read_ref(get_main_ref_store(the_repository),
+ dst_ref.buf, &oid))
+ have_existing_ref = 1;
+ }
+
+ strvec_pushl(&cmd.args, "fetch", remote_name, NULL);
+ if (src_ref)
+ strvec_push(&cmd.args, src_ref);
+ cmd.git_cmd = 1;
+ if (run_command(&cmd)) {
+ if (have_existing_ref)
+ warning(_("failed to fetch start-point '%s'; "
+ "using existing '%s'"),
+ arg, dst_ref.buf);
+ else
+ die(_("failed to fetch start-point '%s'"), arg);
+ }
+
+ free(remote_name);
+ free(src_ref);
+ strbuf_release(&dst_ref);
+}
+
+static int parse_opt_checkout_track(const struct option *opt,
+ const char *arg, int unset)
+{
+ struct checkout_opts *opts = opt->value;
+ struct string_list tokens = STRING_LIST_INIT_DUP;
+ struct string_list_item *item;
+ int saw_direct = 0, saw_inherit = 0;
+ int ret = 0;
+
+ opts->fetch = 0;
+
+ if (unset) {
+ opts->track = BRANCH_TRACK_NEVER;
+ return 0;
+ }
+
+ opts->track = BRANCH_TRACK_EXPLICIT;
+ if (!arg)
+ return 0;
+
+ string_list_split(&tokens, arg, ",", -1);
+ for_each_string_list_item(item, &tokens) {
+ if (!strcmp(item->string, "fetch")) {
+ opts->fetch = 1;
+ } else if (!strcmp(item->string, "direct")) {
+ saw_direct = 1;
+ opts->track = BRANCH_TRACK_EXPLICIT;
+ } else if (!strcmp(item->string, "inherit")) {
+ saw_inherit = 1;
+ opts->track = BRANCH_TRACK_INHERIT;
+ } else {
+ ret = error(_("option `%s' expects \"%s\", \"%s\", "
+ "or \"%s\""),
+ "--track", "direct", "inherit", "fetch");
+ goto out;
+ }
+ }
+
+ if (saw_direct && saw_inherit)
+ ret = error(_("option `%s' cannot combine \"%s\" and \"%s\""),
+ "--track", "direct", "inherit");
+
+out:
+ string_list_clear(&tokens, 0);
+ return ret;
+}
+
static void branch_info_release(struct branch_info *info)
{
free(info->name);
@@ -1244,7 +1398,6 @@ static int git_checkout_config(const char *var, const char *value,
opts->dwim_new_local_branch = git_config_bool(var, value);
return 0;
}
-
if (starts_with(var, "submodule."))
return git_default_submodule_config(var, value, NULL);
@@ -1741,10 +1894,10 @@ static struct option *add_common_switch_branch_options(
{
struct option options[] = {
OPT_BOOL('d', "detach", &opts->force_detach, N_("detach HEAD at named commit")),
- OPT_CALLBACK_F('t', "track", &opts->track, "(direct|inherit)",
+ OPT_CALLBACK_F('t', "track", opts, "(direct|inherit|fetch)[,...]",
N_("set branch tracking configuration"),
PARSE_OPT_OPTARG,
- parse_opt_tracking_mode),
+ parse_opt_checkout_track),
OPT__FORCE(&opts->force, N_("force checkout (throw away local modifications)"),
PARSE_OPT_NOCOMPLETE),
OPT_STRING(0, "orphan", &opts->new_orphan_branch, N_("new-branch"), N_("new unborn branch")),
@@ -1949,8 +2102,13 @@ static int checkout_main(int argc, const char **argv, const char *prefix,
opts->dwim_new_local_branch &&
opts->track == BRANCH_TRACK_UNSPECIFIED &&
!opts->new_branch;
- int n = parse_branchname_arg(argc, argv, dwim_ok, which_command,
- &new_branch_info, opts, &rev);
+ int n;
+
+ if (opts->fetch)
+ fetch_remote_for_start_point(argv[0]);
+
+ n = parse_branchname_arg(argc, argv, dwim_ok, which_command,
+ &new_branch_info, opts, &rev);
argv += n;
argc -= n;
} else if (!opts->accept_ref && opts->from_treeish) {
diff --git a/t/t7201-co.sh b/t/t7201-co.sh
index 9bcf7c0b40..19ac6a1a2e 100755
--- a/t/t7201-co.sh
+++ b/t/t7201-co.sh
@@ -801,4 +801,136 @@ test_expect_success 'tracking info copied with autoSetupMerge=inherit' '
test_cmp_config "" --default "" branch.main2.merge
'
+test_expect_success 'setup upstream for --track=fetch tests' '
+ git checkout main &&
+ git init fetch_upstream &&
+ test_commit -C fetch_upstream u_main &&
+ git remote add fetch_upstream fetch_upstream &&
+ git fetch fetch_upstream &&
+ git -C fetch_upstream checkout -b fetch_new &&
+ test_commit -C fetch_upstream u_new
+'
+
+test_expect_success 'checkout --track=fetch -b picks up branch created upstream after clone' '
+ git checkout main &&
+ test_must_fail git rev-parse --verify refs/remotes/fetch_upstream/fetch_new &&
+ git checkout --track=fetch -b local_new fetch_upstream/fetch_new &&
+ test_cmp_rev refs/remotes/fetch_upstream/fetch_new HEAD &&
+ test_cmp_config fetch_upstream branch.local_new.remote &&
+ test_cmp_config refs/heads/fetch_new branch.local_new.merge
+'
+
+test_expect_success 'checkout --track=fetch <remote>/<branch> leaves other tracking branches untouched' '
+ git checkout main &&
+ git -C fetch_upstream checkout -b fetch_target &&
+ test_commit -C fetch_upstream u_target_pre &&
+ git -C fetch_upstream checkout -b fetch_other &&
+ test_commit -C fetch_upstream u_other_pre &&
+ git fetch fetch_upstream &&
+ other_before=$(git rev-parse refs/remotes/fetch_upstream/fetch_other) &&
+ git -C fetch_upstream checkout fetch_target &&
+ test_commit -C fetch_upstream u_target_post &&
+ git -C fetch_upstream checkout fetch_other &&
+ test_commit -C fetch_upstream u_other_post &&
+ git checkout --track=fetch -b local_target fetch_upstream/fetch_target &&
+ test_cmp_rev refs/remotes/fetch_upstream/fetch_target HEAD &&
+ test "$(git rev-parse refs/remotes/fetch_upstream/fetch_other)" = "$other_before"
+'
+
+test_expect_success 'checkout --track=fetch with bare remote name fetches only <remote>/HEAD target' '
+ git checkout main &&
+ git -C fetch_upstream checkout main &&
+ git remote set-head fetch_upstream main &&
+ git -C fetch_upstream checkout -b fetch_unrelated &&
+ test_commit -C fetch_upstream u_unrelated_pre &&
+ git fetch fetch_upstream fetch_unrelated &&
+ unrelated_before=$(git rev-parse refs/remotes/fetch_upstream/fetch_unrelated) &&
+ git -C fetch_upstream checkout main &&
+ test_commit -C fetch_upstream u_main_post &&
+ git -C fetch_upstream checkout fetch_unrelated &&
+ test_commit -C fetch_upstream u_unrelated_post &&
+ git checkout --track=fetch -b local_from_remote fetch_upstream &&
+ test_cmp_rev refs/remotes/fetch_upstream/main HEAD &&
+ test "$(git rev-parse refs/remotes/fetch_upstream/fetch_unrelated)" = "$unrelated_before"
+'
+
+test_expect_success 'checkout --track=fetch aborts and does not create branch when no existing ref' '
+ git checkout main &&
+ test_might_fail git branch -D bogus &&
+ test_must_fail git checkout --track=fetch -b bogus fetch_upstream/does_not_exist &&
+ test_must_fail git rev-parse --verify refs/heads/bogus
+'
+
+test_expect_success 'checkout --track=fetch warns and proceeds when fetch fails but ref exists' '
+ git checkout main &&
+ git -C fetch_upstream checkout -b fetch_offline &&
+ test_commit -C fetch_upstream u_offline &&
+ git fetch fetch_upstream fetch_offline &&
+ saved_url=$(git config remote.fetch_upstream.url) &&
+ test_when_finished "git config remote.fetch_upstream.url \"$saved_url\"" &&
+ git config remote.fetch_upstream.url ./does-not-exist &&
+ git checkout --track=fetch -b local_offline fetch_upstream/fetch_offline 2>err &&
+ test_grep "failed to fetch" err &&
+ test_cmp_rev refs/remotes/fetch_upstream/fetch_offline HEAD
+'
+
+test_expect_success 'checkout --track=fetch resolves through configured fetch refspec' '
+ git checkout main &&
+ git -C fetch_upstream checkout -b fetch_refspec &&
+ test_commit -C fetch_upstream u_refspec &&
+ git fetch fetch_upstream fetch_refspec &&
+ git remote add fetch_custom ./fetch_upstream &&
+ test_when_finished "git remote remove fetch_custom" &&
+ git config --replace-all remote.fetch_custom.fetch \
+ "+refs/heads/*:refs/remotes/custom-ns/*" &&
+ git fetch fetch_custom &&
+ test_commit -C fetch_upstream u_refspec_post &&
+ git checkout --track=fetch -b local_refspec custom-ns/fetch_refspec &&
+ test_cmp_rev refs/remotes/custom-ns/fetch_refspec HEAD
+'
+
+test_expect_success 'checkout --track=inherit,direct is rejected' '
+ test_must_fail git checkout --track=inherit,direct -b bad fetch_upstream/fetch_new 2>err &&
+ test_grep "cannot combine" err
+'
+
+test_expect_success 'checkout --track=fetch then --track=direct drops fetch (last-one-wins)' '
+ git checkout main &&
+ git -C fetch_upstream checkout -b fetch_lastwin &&
+ test_commit -C fetch_upstream u_lastwin &&
+ test_must_fail git rev-parse --verify refs/remotes/fetch_upstream/fetch_lastwin &&
+ test_must_fail git checkout --track=fetch --track=direct \
+ -b local_lastwin fetch_upstream/fetch_lastwin &&
+ test_must_fail git rev-parse --verify refs/remotes/fetch_upstream/fetch_lastwin
+'
+
+test_expect_success 'checkout --track=fetch,inherit fetches and inherits' '
+ git checkout main &&
+ git -C fetch_upstream checkout -b fetch_inherit &&
+ test_commit -C fetch_upstream u_inherit &&
+ git fetch fetch_upstream fetch_inherit &&
+ git checkout -b base_inherit fetch_upstream/fetch_inherit &&
+ test_commit -C fetch_upstream u_inherit2 &&
+ git checkout main &&
+ git checkout --track=fetch,inherit -b local_inherit base_inherit &&
+ test_cmp_rev refs/remotes/fetch_upstream/fetch_inherit HEAD &&
+ test_cmp_config fetch_upstream branch.local_inherit.remote &&
+ test_cmp_config refs/heads/fetch_inherit branch.local_inherit.merge
+'
+
+test_expect_success 'checkout --track=bogus reports an error' '
+ git checkout main &&
+ test_must_fail git checkout --track=bogus -b bogus_branch fetch_upstream/fetch_new 2>err &&
+ test_grep "expects" err
+'
+
+test_expect_success 'switch --track=fetch -c picks up branch created upstream after clone' '
+ git checkout main &&
+ git -C fetch_upstream checkout -b fetch_switch &&
+ test_commit -C fetch_upstream u_switch &&
+ test_must_fail git rev-parse --verify refs/remotes/fetch_upstream/fetch_switch &&
+ git switch --track=fetch -c local_switch fetch_upstream/fetch_switch &&
+ test_cmp_rev refs/remotes/fetch_upstream/fetch_switch HEAD
+'
+
test_done
base-commit: 7760f83b59750c27df653c5c46d0f80e44cfe02c
--
gitgitgadget
^ permalink raw reply related
* Re: [PATCH v4 5/5] format-rev: introduce builtin for on-demand pretty formatting
From: Kristoffer Haugsbakk @ 2026-05-11 13:25 UTC (permalink / raw)
To: Kristoffer Haugsbakk, git
Cc: D. Ben Knoble, Phillip Wood, Ramsay Jones, Junio C Hamano
In-Reply-To: <V4_format-rev_new_builtin.6af@msgid.xyz>
On Thu, May 7, 2026, at 21:34, kristofferhaugsbakk@fastmail.com wrote:
> From: Kristoffer Haugsbakk <code@khaugsbakk.name>
>
> Introduce a new builtin for pretty formatting one revision expression
> per line or commit object names found in running text.
>
>[snip]
>
> diff --git a/Documentation/git-format-rev.adoc
>[snip]
> +OPTIONS
> +-------
> +
> +`--stdin-mode=<mode>`::
> + How to interpret standard input data:
> ++
> +--
> +`revs`:: Each line or record (see the <<io,INPUT AND OUTPUT FORMATS>>
> + section) is interpreted as a commit. Any kind of revision
> + expression can be used (see linkgit:gitrevisions[7]). Annotated
> + tags are peeled (see linkgit:gitglossary[7]).
> ++
> +The argument `rev` is also accepted.
> +`text`:: Formats all commit object names found in freeform text. These
> + must the full object names, i.e. abbreviated hexidecimal object
> + names will not be interpreted.
> ++
> +Anything that is parsed as an object name but that is not found to be a
> +commit object name is left alone (echoed).
> +--
This nested definition list is malformed. I’ll need to fix that.
> +
> +`--format=<pretty>`::
> + Pretty format string.
>[snip]
^ permalink raw reply
* kh/name-rev-custom-format
From: Kristoffer Haugsbakk @ 2026-05-11 13:22 UTC (permalink / raw)
To: Junio C Hamano, git
In-Reply-To: <xmqq4iketzh0.fsf@gitster.g>
On Mon, May 11, 2026, at 05:08, Junio C Hamano wrote:
> * kh/name-rev-custom-format (2026-05-07) 5 commits
> - format-rev: introduce builtin for on-demand pretty formatting
> - name-rev: make dedicated --annotate-stdin --name-only test
> - name-rev: factor code for sharing with a new command
> - name-rev: run clang-format before factoring code
> - name-rev: wrap both blocks in braces
>
> A new builtin "git format-rev" is introduced for pretty formatting
> one revision expression per line or commit object names found in
> running text.
>
> Will merge to 'next'.
> source: <V4_CV_format-rev.6aa@msgid.xyz>
I’m very glad that it ready for `next`. But... please hold off on
merging it to `next` for the next version. I managed to make an
AsciiDoc mistake.
Sorry about that.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox