From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Cc: Toon Claes <toon@iotcl.com>,
Karthik Nayak <karthik.188@gmail.com>,
Taylor Blau <me@ttaylorr.com>,
Junio C Hamano <gitster@pobox.com>
Subject: [PATCH v2 07/10] pack-bitmap: add function to iterate over filtered bitmapped objects
Date: Thu, 27 Mar 2025 10:44:02 +0100 [thread overview]
Message-ID: <20250327-pks-cat-file-object-type-filter-v2-7-4bbc7085d7c5@pks.im> (raw)
In-Reply-To: <20250327-pks-cat-file-object-type-filter-v2-0-4bbc7085d7c5@pks.im>
Introduce a function that allows the caller to iterate over all
bitmapped objects that match a given filter. This mechanism will be used
in a subsequent commit to optimize object filters in git-cat-file(1).
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
pack-bitmap.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++------
pack-bitmap.h | 12 ++++++++++++
2 files changed, 65 insertions(+), 6 deletions(-)
diff --git a/pack-bitmap.c b/pack-bitmap.c
index d192fb87da9..6adb8aaa1c2 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -1662,6 +1662,7 @@ static void init_type_iterator(struct ewah_or_iterator *it,
static void show_objects_for_type(
struct bitmap_index *bitmap_git,
+ struct bitmap *objects,
enum object_type object_type,
show_reachable_fn show_reach,
void *payload)
@@ -1672,8 +1673,6 @@ static void show_objects_for_type(
struct ewah_or_iterator it;
eword_t filter;
- struct bitmap *objects = bitmap_git->result;
-
init_type_iterator(&it, bitmap_git, object_type);
for (i = 0; i < objects->word_alloc &&
@@ -2025,6 +2024,50 @@ static void filter_packed_objects_from_bitmap(struct bitmap_index *bitmap_git,
}
}
+int for_each_bitmapped_object(struct bitmap_index *bitmap_git,
+ struct list_objects_filter_options *filter,
+ show_reachable_fn show_reach,
+ void *payload)
+{
+ struct bitmap *filtered_bitmap = NULL;
+ uint32_t objects_nr;
+ size_t full_word_count;
+ int ret;
+
+ if (!can_filter_bitmap(filter)) {
+ ret = -1;
+ goto out;
+ }
+
+ objects_nr = bitmap_num_objects(bitmap_git);
+ full_word_count = objects_nr / BITS_IN_EWORD;
+
+ /* We start from the all-1 bitmap and then filter down from there. */
+ filtered_bitmap = bitmap_word_alloc(full_word_count + !!(objects_nr % BITS_IN_EWORD));
+ memset(filtered_bitmap->words, 0xff, full_word_count * sizeof(*filtered_bitmap->words));
+ for (size_t i = full_word_count * BITS_IN_EWORD; i < objects_nr; i++)
+ bitmap_set(filtered_bitmap, i);
+
+ if (filter_bitmap(bitmap_git, NULL, filtered_bitmap, filter) < 0) {
+ ret = -1;
+ goto out;
+ }
+
+ show_objects_for_type(bitmap_git, filtered_bitmap,
+ OBJ_COMMIT, show_reach, payload);
+ show_objects_for_type(bitmap_git, filtered_bitmap,
+ OBJ_TREE, show_reach, payload);
+ show_objects_for_type(bitmap_git, filtered_bitmap,
+ OBJ_BLOB, show_reach, payload);
+ show_objects_for_type(bitmap_git, filtered_bitmap,
+ OBJ_TAG, show_reach, payload);
+
+ ret = 0;
+out:
+ bitmap_free(filtered_bitmap);
+ return ret;
+}
+
struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs,
int filter_provided_objects)
{
@@ -2519,13 +2562,17 @@ void traverse_bitmap_commit_list(struct bitmap_index *bitmap_git,
{
assert(bitmap_git->result);
- show_objects_for_type(bitmap_git, OBJ_COMMIT, show_reachable, NULL);
+ show_objects_for_type(bitmap_git, bitmap_git->result,
+ OBJ_COMMIT, show_reachable, NULL);
if (revs->tree_objects)
- show_objects_for_type(bitmap_git, OBJ_TREE, show_reachable, NULL);
+ show_objects_for_type(bitmap_git, bitmap_git->result,
+ OBJ_TREE, show_reachable, NULL);
if (revs->blob_objects)
- show_objects_for_type(bitmap_git, OBJ_BLOB, show_reachable, NULL);
+ show_objects_for_type(bitmap_git, bitmap_git->result,
+ OBJ_BLOB, show_reachable, NULL);
if (revs->tag_objects)
- show_objects_for_type(bitmap_git, OBJ_TAG, show_reachable, NULL);
+ show_objects_for_type(bitmap_git, bitmap_git->result,
+ OBJ_TAG, show_reachable, NULL);
show_extended_objects(bitmap_git, revs, show_reachable);
}
diff --git a/pack-bitmap.h b/pack-bitmap.h
index de6bf534fef..079bae32466 100644
--- a/pack-bitmap.h
+++ b/pack-bitmap.h
@@ -79,6 +79,18 @@ int test_bitmap_pseudo_merges(struct repository *r);
int test_bitmap_pseudo_merge_commits(struct repository *r, uint32_t n);
int test_bitmap_pseudo_merge_objects(struct repository *r, uint32_t n);
+struct list_objects_filter_options;
+
+/*
+ * Filter bitmapped objects and iterate through all resulting objects,
+ * executing `show_reach` for each of them. Returns `-1` in case the filter is
+ * not supported, `0` otherwise.
+ */
+int for_each_bitmapped_object(struct bitmap_index *bitmap_git,
+ struct list_objects_filter_options *filter,
+ show_reachable_fn show_reach,
+ void *payload);
+
#define GIT_TEST_PACK_USE_BITMAP_BOUNDARY_TRAVERSAL \
"GIT_TEST_PACK_USE_BITMAP_BOUNDARY_TRAVERSAL"
--
2.49.0.472.ge94155a9ec.dirty
next prev parent reply other threads:[~2025-03-27 9:44 UTC|newest]
Thread overview: 72+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-02-21 7:47 [PATCH 0/9] builtin/cat-file: allow filtering objects in batch mode Patrick Steinhardt
2025-02-21 7:47 ` [PATCH 1/9] builtin/cat-file: rename variable that tracks usage Patrick Steinhardt
2025-02-21 7:47 ` [PATCH 2/9] builtin/cat-file: wire up an option to filter objects Patrick Steinhardt
2025-02-26 15:20 ` Toon Claes
2025-02-28 10:51 ` Patrick Steinhardt
2025-02-28 17:44 ` Junio C Hamano
2025-03-03 10:40 ` Patrick Steinhardt
2025-02-27 11:20 ` Karthik Nayak
2025-02-21 7:47 ` [PATCH 3/9] builtin/cat-file: support "blob:none" objects filter Patrick Steinhardt
2025-02-26 15:22 ` Toon Claes
2025-02-27 11:26 ` Karthik Nayak
2025-02-21 7:47 ` [PATCH 4/9] builtin/cat-file: support "blob:limit=" " Patrick Steinhardt
2025-02-21 7:47 ` [PATCH 5/9] builtin/cat-file: support "object:type=" " Patrick Steinhardt
2025-02-26 15:23 ` Toon Claes
2025-02-28 10:51 ` Patrick Steinhardt
2025-02-21 7:47 ` [PATCH 6/9] pack-bitmap: expose function to iterate over bitmapped objects Patrick Steinhardt
2025-02-24 18:05 ` Junio C Hamano
2025-02-25 6:59 ` Patrick Steinhardt
2025-02-25 16:59 ` Junio C Hamano
2025-02-27 23:26 ` Taylor Blau
2025-02-28 10:54 ` Patrick Steinhardt
2025-02-27 23:23 ` Taylor Blau
2025-02-27 23:32 ` Junio C Hamano
2025-02-27 23:39 ` Taylor Blau
2025-02-21 7:47 ` [PATCH 7/9] pack-bitmap: introduce function to check whether a pack is bitmapped Patrick Steinhardt
2025-02-27 23:33 ` Taylor Blau
2025-02-21 7:47 ` [PATCH 8/9] builtin/cat-file: deduplicate logic to iterate over all objects Patrick Steinhardt
2025-02-21 7:47 ` [PATCH 9/9] builtin/cat-file: use bitmaps to efficiently filter by object type Patrick Steinhardt
2025-02-27 11:38 ` Karthik Nayak
2025-02-27 23:48 ` Taylor Blau
2025-03-27 9:43 ` [PATCH v2 00/10] builtin/cat-file: allow filtering objects in batch mode Patrick Steinhardt
2025-03-27 9:43 ` [PATCH v2 01/10] builtin/cat-file: rename variable that tracks usage Patrick Steinhardt
2025-04-01 9:51 ` Karthik Nayak
2025-04-02 11:13 ` Patrick Steinhardt
2025-04-07 20:25 ` Junio C Hamano
2025-03-27 9:43 ` [PATCH v2 02/10] builtin/cat-file: wire up an option to filter objects Patrick Steinhardt
2025-04-01 11:45 ` Toon Claes
2025-04-02 11:13 ` Patrick Steinhardt
2025-04-01 12:05 ` Karthik Nayak
2025-04-02 11:13 ` Patrick Steinhardt
2025-03-27 9:43 ` [PATCH v2 03/10] builtin/cat-file: support "blob:none" objects filter Patrick Steinhardt
2025-04-01 12:22 ` Karthik Nayak
2025-04-01 12:31 ` Karthik Nayak
2025-04-02 11:13 ` Patrick Steinhardt
2025-03-27 9:43 ` [PATCH v2 04/10] builtin/cat-file: support "blob:limit=" " Patrick Steinhardt
2025-03-27 9:44 ` [PATCH v2 05/10] builtin/cat-file: support "object:type=" " Patrick Steinhardt
2025-03-27 9:44 ` [PATCH v2 06/10] pack-bitmap: allow passing payloads to `show_reachable_fn()` Patrick Steinhardt
2025-04-01 12:17 ` Toon Claes
2025-04-02 11:13 ` Patrick Steinhardt
2025-03-27 9:44 ` Patrick Steinhardt [this message]
2025-03-27 9:44 ` [PATCH v2 08/10] pack-bitmap: introduce function to check whether a pack is bitmapped Patrick Steinhardt
2025-04-01 11:46 ` Toon Claes
2025-04-02 11:13 ` Patrick Steinhardt
2025-03-27 9:44 ` [PATCH v2 09/10] builtin/cat-file: deduplicate logic to iterate over all objects Patrick Steinhardt
2025-04-01 12:13 ` Toon Claes
2025-04-02 11:13 ` Patrick Steinhardt
2025-04-03 18:24 ` Toon Claes
2025-03-27 9:44 ` [PATCH v2 10/10] builtin/cat-file: use bitmaps to efficiently filter by object type Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 00/11] builtin/cat-file: allow filtering objects in batch mode Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 01/11] builtin/cat-file: rename variable that tracks usage Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 02/11] builtin/cat-file: introduce function to report object status Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 03/11] builtin/cat-file: wire up an option to filter objects Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 04/11] builtin/cat-file: support "blob:none" objects filter Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 05/11] builtin/cat-file: support "blob:limit=" " Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 06/11] builtin/cat-file: support "object:type=" " Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 07/11] pack-bitmap: allow passing payloads to `show_reachable_fn()` Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 08/11] pack-bitmap: add function to iterate over filtered bitmapped objects Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 09/11] pack-bitmap: introduce function to check whether a pack is bitmapped Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 10/11] builtin/cat-file: deduplicate logic to iterate over all objects Patrick Steinhardt
2025-04-02 11:13 ` [PATCH v3 11/11] builtin/cat-file: use bitmaps to efficiently filter by object type Patrick Steinhardt
2025-04-03 8:17 ` [PATCH v3 00/11] builtin/cat-file: allow filtering objects in batch mode Karthik Nayak
2025-04-08 0:32 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250327-pks-cat-file-object-type-filter-v2-7-4bbc7085d7c5@pks.im \
--to=ps@pks.im \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=karthik.188@gmail.com \
--cc=me@ttaylorr.com \
--cc=toon@iotcl.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).