From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Cc: Toon Claes <toon@iotcl.com>,
Karthik Nayak <karthik.188@gmail.com>,
Taylor Blau <me@ttaylorr.com>,
Junio C Hamano <gitster@pobox.com>
Subject: [PATCH v3 04/11] builtin/cat-file: support "blob:none" objects filter
Date: Wed, 02 Apr 2025 13:13:39 +0200 [thread overview]
Message-ID: <20250402-pks-cat-file-object-type-filter-v3-4-4da6bb21871c@pks.im> (raw)
In-Reply-To: <20250402-pks-cat-file-object-type-filter-v3-0-4da6bb21871c@pks.im>
Implement support for the "blob:none" filter in git-cat-file(1), which
causes us to omit all blobs.
Note that this new filter requires us to read the object type via
`oid_object_info_extended()` in `batch_object_write()`. But as we try to
optimize away reading objects from the database the `data->info.typep`
pointer may not be set. We thus have to adapt the logic to conditionally
set the pointer in cases where the filter is given.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
Documentation/git-cat-file.adoc | 4 +++-
builtin/cat-file.c | 15 ++++++++++++-
t/t1006-cat-file.sh | 47 +++++++++++++++++++++++++++++++++++++++--
3 files changed, 62 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-cat-file.adoc b/Documentation/git-cat-file.adoc
index da92eed1170..afcdb0a4738 100644
--- a/Documentation/git-cat-file.adoc
+++ b/Documentation/git-cat-file.adoc
@@ -88,7 +88,9 @@ OPTIONS
been explicitly requested via any of the batch modes that read objects
via standard input (`--batch`, `--batch-check`) will be reported as
"filtered". Excluded objects in `--batch-all-objects` mode will not be
- printed at all. No filters are supported yet.
+ printed at all. The '<filter-spec>' may be one of the following:
++
+The form '--filter=blob:none' omits all blobs.
--path=<path>::
For use with `--textconv` or `--filters`, to allow specifying an object
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index 0e2176c4491..bcceb646f85 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -482,7 +482,8 @@ static void batch_object_write(const char *obj_name,
if (!data->skip_object_info) {
int ret;
- if (use_mailmap)
+ if (use_mailmap ||
+ opt->objects_filter.choice == LOFC_BLOB_NONE)
data->info.typep = &data->type;
if (pack)
@@ -500,6 +501,14 @@ static void batch_object_write(const char *obj_name,
switch (opt->objects_filter.choice) {
case LOFC_DISABLED:
break;
+ case LOFC_BLOB_NONE:
+ if (data->type == OBJ_BLOB) {
+ if (!opt->all_objects)
+ report_object_status(opt, obj_name,
+ &data->oid, "excluded");
+ return;
+ }
+ break;
default:
BUG("unsupported objects filter");
}
@@ -1039,6 +1048,10 @@ int cmd_cat_file(int argc,
switch (batch.objects_filter.choice) {
case LOFC_DISABLED:
break;
+ case LOFC_BLOB_NONE:
+ if (!batch.enabled)
+ usage(_("objects filter only supported in batch mode"));
+ break;
default:
usagef(_("objects filter not supported: '%s'"),
list_object_filter_config_name(batch.objects_filter.choice));
diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh
index 9ce4eda6e68..7404c135b1e 100755
--- a/t/t1006-cat-file.sh
+++ b/t/t1006-cat-file.sh
@@ -1354,7 +1354,22 @@ test_expect_success PERL '--batch-command info is unbuffered by default' '
'
test_expect_success 'setup for objects filter' '
- git init repo
+ git init repo &&
+ (
+ # Seed the repository with three different sets of objects:
+ #
+ # - The first set is fully packed and has a bitmap.
+ # - The second set is packed, but has no bitmap.
+ # - The third set is loose.
+ #
+ # This ensures that we cover all these types as expected.
+ cd repo &&
+ test_commit first &&
+ git repack -Adb &&
+ test_commit second &&
+ git repack -d &&
+ test_commit third
+ )
'
test_expect_success 'objects filter with unknown option' '
@@ -1365,7 +1380,7 @@ test_expect_success 'objects filter with unknown option' '
test_cmp expect err
'
-for option in blob:none blob:limit=1 object:type=tag sparse:oid=1234 tree:1 sparse:path=x
+for option in blob:limit=1 object:type=tag sparse:oid=1234 tree:1 sparse:path=x
do
test_expect_success "objects filter with unsupported option $option" '
case "$option" in
@@ -1393,4 +1408,32 @@ test_expect_success 'objects filter: disabled' '
test_cmp expect.sorted actual.sorted
'
+test_objects_filter () {
+ filter="$1"
+
+ test_expect_success "objects filter: $filter" '
+ git -C repo cat-file --batch-check="%(objectname)" --batch-all-objects --filter="$filter" >actual &&
+ sort actual >actual.sorted &&
+ git -C repo rev-list --objects --no-object-names --all --filter="$filter" --filter-provided-objects >expect &&
+ sort expect >expect.sorted &&
+ test_cmp expect.sorted actual.sorted
+ '
+
+ test_expect_success "objects filter prints excluded objects: $filter" '
+ # Find all objects that would be excluded by the current filter.
+ git -C repo rev-list --objects --no-object-names --all >all &&
+ git -C repo rev-list --objects --no-object-names --all --filter="$filter" --filter-provided-objects >filtered &&
+ sort all >all.sorted &&
+ sort filtered >filtered.sorted &&
+ comm -23 all.sorted filtered.sorted >expected.excluded &&
+ test_line_count -gt 0 expected.excluded &&
+
+ git -C repo cat-file --batch-check="%(objectname)" --filter="$filter" <expected.excluded >actual &&
+ awk "/excluded/{ print \$1 }" actual | sort >actual.excluded &&
+ test_cmp expected.excluded actual.excluded
+ '
+}
+
+test_objects_filter "blob:none"
+
test_done
--
2.49.0.604.gff1f9ca942.dirty
next prev parent reply other threads:[~2025-04-02 11:13 UTC|newest]
Thread overview: 72+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-02-21 7:47 [PATCH 0/9] builtin/cat-file: allow filtering objects in batch mode Patrick Steinhardt
2025-02-21 7:47 ` [PATCH 1/9] builtin/cat-file: rename variable that tracks usage Patrick Steinhardt
2025-02-21 7:47 ` [PATCH 2/9] builtin/cat-file: wire up an option to filter objects Patrick Steinhardt
2025-02-26 15:20 ` Toon Claes
2025-02-28 10:51 ` Patrick Steinhardt
2025-02-28 17:44 ` Junio C Hamano
2025-03-03 10:40 ` Patrick Steinhardt
2025-02-27 11:20 ` Karthik Nayak
2025-02-21 7:47 ` [PATCH 3/9] builtin/cat-file: support "blob:none" objects filter Patrick Steinhardt
2025-02-26 15:22 ` Toon Claes
2025-02-27 11:26 ` Karthik Nayak
2025-02-21 7:47 ` [PATCH 4/9] builtin/cat-file: support "blob:limit=" " Patrick Steinhardt
2025-02-21 7:47 ` [PATCH 5/9] builtin/cat-file: support "object:type=" " Patrick Steinhardt
2025-02-26 15:23 ` Toon Claes
2025-02-28 10:51 ` Patrick Steinhardt
2025-02-21 7:47 ` [PATCH 6/9] pack-bitmap: expose function to iterate over bitmapped objects Patrick Steinhardt
2025-02-24 18:05 ` Junio C Hamano
2025-02-25 6:59 ` Patrick Steinhardt
2025-02-25 16:59 ` Junio C Hamano
2025-02-27 23:26 ` Taylor Blau
2025-02-28 10:54 ` Patrick Steinhardt
2025-02-27 23:23 ` Taylor Blau
2025-02-27 23:32 ` Junio C Hamano
2025-02-27 23:39 ` Taylor Blau
2025-02-21 7:47 ` [PATCH 7/9] pack-bitmap: introduce function to check whether a pack is bitmapped Patrick Steinhardt
2025-02-27 23:33 ` Taylor Blau
2025-02-21 7:47 ` [PATCH 8/9] builtin/cat-file: deduplicate logic to iterate over all objects Patrick Steinhardt
2025-02-21 7:47 ` [PATCH 9/9] builtin/cat-file: use bitmaps to efficiently filter by object type Patrick Steinhardt
2025-02-27 11:38 ` Karthik Nayak
2025-02-27 23:48 ` Taylor Blau
2025-03-27 9:43 ` [PATCH v2 00/10] builtin/cat-file: allow filtering objects in batch mode Patrick Steinhardt
2025-03-27 9:43 ` [PATCH v2 01/10] builtin/cat-file: rename variable that tracks usage Patrick Steinhardt
2025-04-01 9:51 ` Karthik Nayak
2025-04-02 11:13 ` Patrick Steinhardt
2025-04-07 20:25 ` Junio C Hamano
2025-03-27 9:43 ` [PATCH v2 02/10] builtin/cat-file: wire up an option to filter objects Patrick Steinhardt
2025-04-01 11:45 ` Toon Claes
2025-04-02 11:13 ` Patrick Steinhardt
2025-04-01 12:05 ` Karthik Nayak
2025-04-02 11:13 ` Patrick Steinhardt
2025-03-27 9:43 ` [PATCH v2 03/10] builtin/cat-file: support "blob:none" objects filter Patrick Steinhardt
2025-04-01 12:22 ` Karthik Nayak
2025-04-01 12:31 ` Karthik Nayak
2025-04-02 11:13 ` Patrick Steinhardt
2025-03-27 9:43 ` [PATCH v2 04/10] builtin/cat-file: support "blob:limit=" " Patrick Steinhardt
2025-03-27 9:44 ` [PATCH v2 05/10] builtin/cat-file: support "object:type=" " Patrick Steinhardt
2025-03-27 9:44 ` [PATCH v2 06/10] pack-bitmap: allow passing payloads to `show_reachable_fn()` Patrick Steinhardt
2025-04-01 12:17 ` Toon Claes
2025-04-02 11:13 ` Patrick Steinhardt
2025-03-27 9:44 ` [PATCH v2 07/10] pack-bitmap: add function to iterate over filtered bitmapped objects Patrick Steinhardt
2025-03-27 9:44 ` [PATCH v2 08/10] pack-bitmap: introduce function to check whether a pack is bitmapped Patrick Steinhardt
2025-04-01 11:46 ` Toon Claes
2025-04-02 11:13 ` Patrick Steinhardt
2025-03-27 9:44 ` [PATCH v2 09/10] builtin/cat-file: deduplicate logic to iterate over all objects Patrick Steinhardt
2025-04-01 12:13 ` Toon Claes
2025-04-02 11:13 ` Patrick Steinhardt
2025-04-03 18:24 ` Toon Claes
2025-03-27 9:44 ` [PATCH v2 10/10] builtin/cat-file: use bitmaps to efficiently filter by object type Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 00/11] builtin/cat-file: allow filtering objects in batch mode Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 01/11] builtin/cat-file: rename variable that tracks usage Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 02/11] builtin/cat-file: introduce function to report object status Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 03/11] builtin/cat-file: wire up an option to filter objects Patrick Steinhardt
2025-04-02 11:13 ` Patrick Steinhardt [this message]
2025-04-02 11:13 ` [PATCH v3 05/11] builtin/cat-file: support "blob:limit=" objects filter Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 06/11] builtin/cat-file: support "object:type=" " Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 07/11] pack-bitmap: allow passing payloads to `show_reachable_fn()` Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 08/11] pack-bitmap: add function to iterate over filtered bitmapped objects Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 09/11] pack-bitmap: introduce function to check whether a pack is bitmapped Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 10/11] builtin/cat-file: deduplicate logic to iterate over all objects Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 11/11] builtin/cat-file: use bitmaps to efficiently filter by object type Patrick Steinhardt
2025-04-03 8:17 ` [PATCH v3 00/11] builtin/cat-file: allow filtering objects in batch mode Karthik Nayak
2025-04-08 0:32 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250402-pks-cat-file-object-type-filter-v3-4-4da6bb21871c@pks.im \
--to=ps@pks.im \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=karthik.188@gmail.com \
--cc=me@ttaylorr.com \
--cc=toon@iotcl.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).