From: Jonathan Tan <jonathantanmy@google.com>
To: git@vger.kernel.org
Cc: "Jonathan Tan" <jonathantanmy@google.com>,
"Junio C Hamano" <gitster@pobox.com>,
"SZEDER Gábor" <szeder.dev@gmail.com>,
"Taylor Blau" <me@ttaylorr.com>
Subject: [PATCH v2 00/15] bloom: changed-path Bloom filters v2
Date: Wed, 30 Aug 2023 09:43:40 -0700 [thread overview]
Message-ID: <cover.1693413637.git.jonathantanmy@google.com> (raw)
In-Reply-To: <cover.1692654233.git.me@ttaylorr.com>
Here's an updated patch set containing updates to the tests, making them
run on non-root commits to ensure that we exercise the Bloom filters.
Thanks to SZEDER Gábor for spotting this.
Jonathan Tan (4):
gitformat-commit-graph: describe version 2 of BDAT
t4216: test changed path filters with high bit paths
repo-settings: introduce commitgraph.changedPathsVersion
commit-graph: new filter ver. that fixes murmur3
Taylor Blau (11):
t/helper/test-read-graph.c: extract `dump_graph_info()`
bloom.h: make `load_bloom_filter_from_graph()` public
t/helper/test-read-graph: implement `bloom-filters` mode
bloom: annotate filters with hash version
bloom: prepare to discard incompatible Bloom filters
t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()`
commit-graph.c: unconditionally load Bloom filters
commit-graph: drop unnecessary `graph_read_bloom_data_context`
object.h: fix mis-aligned flag bits table
commit-graph: reuse existing Bloom filters where possible
bloom: introduce `deinit_bloom_filters()`
Documentation/config/commitgraph.txt | 26 ++-
Documentation/gitformat-commit-graph.txt | 9 +-
bloom.c | 208 +++++++++++++++++++++--
bloom.h | 38 ++++-
commit-graph.c | 36 +++-
object.h | 3 +-
oss-fuzz/fuzz-commit-graph.c | 2 +-
repo-settings.c | 6 +-
repository.h | 2 +-
t/helper/test-bloom.c | 9 +-
t/helper/test-read-graph.c | 65 +++++--
t/t0095-bloom.sh | 8 +
t/t4216-log-bloom.sh | 202 +++++++++++++++++++++-
13 files changed, 564 insertions(+), 50 deletions(-)
Range-diff against v1:
1: dcfc987741 = 1: dcfc987741 gitformat-commit-graph: describe version 2 of BDAT
2: 0c56f2a9e9 = 2: 0c56f2a9e9 t/helper/test-read-graph.c: extract `dump_graph_info()`
3: 8405b845e5 = 3: 8405b845e5 bloom.h: make `load_bloom_filter_from_graph()` public
4: 3a25f90c15 = 4: 3a25f90c15 t/helper/test-read-graph: implement `bloom-filters` mode
5: e300d338e1 ! 5: 7858010665 t4216: test changed path filters with high bit paths
@@ t/t4216-log-bloom.sh: test_expect_success 'Bloom generation backfills empty comm
+ true
+'
+
++test_expect_success 'setup make another commit' '
++ # "git log" does not use Bloom filters for root commits - see how, in
++ # revision.c, rev_compare_tree() (the only code path that eventually calls
++ # get_bloom_filter()) is only called by try_to_simplify_commit() when the commit
++ # has one parent. Therefore, make another commit so that we perform the tests on
++ # a non-root commit.
++ test_commit -C highbit1 anotherc1 "another$CENT"
++'
++
+test_expect_success 'version 1 changed-path used when version 1 requested' '
+ (
+ cd highbit1 &&
-+ test_bloom_filters_used "-- $CENT"
++ test_bloom_filters_used "-- another$CENT"
+ )
+'
+
6: 6bc665e1d3 = 6: 94ad289dbb repo-settings: introduce commitgraph.changedPathsVersion
7: 7ef3b2bbbd ! 7: 44d3163125 commit-graph: new filter ver. that fixes murmur3
@@ t/t4216-log-bloom.sh: test_expect_success 'version 1 changed-path used when vers
+ (
+ cd highbit1 &&
+ git config --add commitgraph.changedPathsVersion 2 &&
-+ test_bloom_filters_not_used "-- $CENT"
++ test_bloom_filters_not_used "-- another$CENT"
+ )
+'
+
@@ t/t4216-log-bloom.sh: test_expect_success 'version 1 changed-path used when vers
+ (
+ cd highbit1 &&
+ git config --add commitgraph.changedPathsVersion -1 &&
-+ test_bloom_filters_used "-- $CENT"
++ test_bloom_filters_used "-- another$CENT"
+ )
+'
+
@@ t/t4216-log-bloom.sh: test_expect_success 'version 1 changed-path used when vers
+ )
+'
+
++test_expect_success 'setup make another commit' '
++ # "git log" does not use Bloom filters for root commits - see how, in
++ # revision.c, rev_compare_tree() (the only code path that eventually calls
++ # get_bloom_filter()) is only called by try_to_simplify_commit() when the commit
++ # has one parent. Therefore, make another commit so that we perform the tests on
++ # a non-root commit.
++ test_commit -C highbit2 anotherc2 "another$CENT"
++'
++
+test_expect_success 'version 2 changed-path used when version 2 requested' '
+ (
+ cd highbit2 &&
-+ test_bloom_filters_used "-- $CENT"
++ test_bloom_filters_used "-- another$CENT"
+ )
+'
+
@@ t/t4216-log-bloom.sh: test_expect_success 'version 1 changed-path used when vers
+ (
+ cd highbit2 &&
+ git config --add commitgraph.changedPathsVersion 1 &&
-+ test_bloom_filters_not_used "-- $CENT"
++ test_bloom_filters_not_used "-- another$CENT"
+ )
+'
+
@@ t/t4216-log-bloom.sh: test_expect_success 'version 1 changed-path used when vers
+ (
+ cd highbit2 &&
+ git config --add commitgraph.changedPathsVersion -1 &&
-+ test_bloom_filters_used "-- $CENT"
++ test_bloom_filters_used "-- another$CENT"
+ )
+'
+
8: 302caee39d = 8: a5bf23a7d3 bloom: annotate filters with hash version
9: d2b0726266 = 9: 3de6cd8460 bloom: prepare to discard incompatible Bloom filters
10: 9a9992220f = 10: ef04389a0e t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()`
11: 607945ab05 = 11: e0c1c1ccec commit-graph.c: unconditionally load Bloom filters
12: e397d83895 = 12: 4d57f51854 commit-graph: drop unnecessary `graph_read_bloom_data_context`
13: bd3ad6b6c0 = 13: a3b4e7ef59 object.h: fix mis-aligned flag bits table
14: 2996f0fdb6 = 14: 05357f9533 commit-graph: reuse existing Bloom filters where possible
15: a3b5b22db0 = 15: 58a1d90e6d bloom: introduce `deinit_bloom_filters()`
--
2.42.0.rc2.253.gd59a3bf2b4-goog
next prev parent reply other threads:[~2023-08-30 18:29 UTC|newest]
Thread overview: 76+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-21 21:43 [PATCH 00/15] bloom: changed-path Bloom filters v2 Taylor Blau
2023-08-21 21:43 ` [PATCH 01/15] gitformat-commit-graph: describe version 2 of BDAT Taylor Blau
2023-08-21 21:44 ` [PATCH 02/15] t/helper/test-read-graph.c: extract `dump_graph_info()` Taylor Blau
2023-08-21 21:44 ` [PATCH 03/15] bloom.h: make `load_bloom_filter_from_graph()` public Taylor Blau
2023-08-21 21:44 ` [PATCH 04/15] t/helper/test-read-graph: implement `bloom-filters` mode Taylor Blau
2023-08-21 21:44 ` [PATCH 05/15] t4216: test changed path filters with high bit paths Taylor Blau
2023-08-21 21:44 ` [PATCH 06/15] repo-settings: introduce commitgraph.changedPathsVersion Taylor Blau
2023-08-21 21:44 ` [PATCH 07/15] commit-graph: new filter ver. that fixes murmur3 Taylor Blau
2023-08-26 15:06 ` SZEDER Gábor
2023-08-29 16:31 ` Jonathan Tan
2023-08-30 20:02 ` SZEDER Gábor
2023-09-01 20:56 ` Jonathan Tan
2023-09-25 23:03 ` Taylor Blau
2023-10-08 14:35 ` SZEDER Gábor
2023-10-09 18:17 ` Taylor Blau
2023-10-09 19:31 ` Taylor Blau
2023-10-09 19:52 ` Junio C Hamano
2023-10-10 20:34 ` Taylor Blau
2023-08-21 21:44 ` [PATCH 08/15] bloom: annotate filters with hash version Taylor Blau
2023-08-21 21:44 ` [PATCH 09/15] bloom: prepare to discard incompatible Bloom filters Taylor Blau
2023-08-21 21:44 ` [PATCH 10/15] t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()` Taylor Blau
2023-08-21 21:44 ` [PATCH 11/15] commit-graph.c: unconditionally load Bloom filters Taylor Blau
2023-08-21 21:44 ` [PATCH 12/15] commit-graph: drop unnecessary `graph_read_bloom_data_context` Taylor Blau
2023-08-21 21:44 ` [PATCH 13/15] object.h: fix mis-aligned flag bits table Taylor Blau
2023-08-21 21:44 ` [PATCH 14/15] commit-graph: reuse existing Bloom filters where possible Taylor Blau
2023-08-21 21:44 ` [PATCH 15/15] bloom: introduce `deinit_bloom_filters()` Taylor Blau
2023-08-24 22:22 ` [PATCH 00/15] bloom: changed-path Bloom filters v2 Jonathan Tan
2023-08-25 17:06 ` Jonathan Tan
2023-08-29 22:18 ` Jonathan Tan
2023-08-29 23:16 ` Junio C Hamano
2023-08-30 16:43 ` Jonathan Tan [this message]
2023-08-30 16:43 ` [PATCH v2 01/15] gitformat-commit-graph: describe version 2 of BDAT Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 02/15] t/helper/test-read-graph.c: extract `dump_graph_info()` Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 03/15] bloom.h: make `load_bloom_filter_from_graph()` public Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 04/15] t/helper/test-read-graph: implement `bloom-filters` mode Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 05/15] t4216: test changed path filters with high bit paths Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 06/15] repo-settings: introduce commitgraph.changedPathsVersion Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 07/15] commit-graph: new filter ver. that fixes murmur3 Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 08/15] bloom: annotate filters with hash version Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 09/15] bloom: prepare to discard incompatible Bloom filters Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 10/15] t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()` Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 11/15] commit-graph.c: unconditionally load Bloom filters Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 12/15] commit-graph: drop unnecessary `graph_read_bloom_data_context` Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 13/15] object.h: fix mis-aligned flag bits table Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 14/15] commit-graph: reuse existing Bloom filters where possible Jonathan Tan
2023-08-30 16:43 ` [PATCH v2 15/15] bloom: introduce `deinit_bloom_filters()` Jonathan Tan
2023-08-30 19:38 ` [PATCH v2 00/15] bloom: changed-path Bloom filters v2 Junio C Hamano
2023-10-10 20:33 ` [PATCH v3 00/17] bloom: changed-path Bloom filters v2 (& sundries) Taylor Blau
2023-10-10 20:33 ` [PATCH v3 01/17] t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()` Taylor Blau
2023-10-10 20:33 ` [PATCH v3 02/17] revision.c: consult Bloom filters for root commits Taylor Blau
2023-10-10 20:33 ` [PATCH v3 03/17] commit-graph: ensure Bloom filters are read with consistent settings Taylor Blau
2023-10-17 8:45 ` Patrick Steinhardt
2023-10-10 20:33 ` [PATCH v3 04/17] gitformat-commit-graph: describe version 2 of BDAT Taylor Blau
2023-10-10 20:33 ` [PATCH v3 05/17] t/helper/test-read-graph.c: extract `dump_graph_info()` Taylor Blau
2023-10-17 8:45 ` Patrick Steinhardt
2023-10-18 17:37 ` Taylor Blau
2023-10-18 23:56 ` Junio C Hamano
2023-10-10 20:33 ` [PATCH v3 06/17] bloom.h: make `load_bloom_filter_from_graph()` public Taylor Blau
2023-10-10 20:33 ` [PATCH v3 07/17] t/helper/test-read-graph: implement `bloom-filters` mode Taylor Blau
2023-10-10 20:33 ` [PATCH v3 08/17] t4216: test changed path filters with high bit paths Taylor Blau
2023-10-17 8:45 ` Patrick Steinhardt
2023-10-18 17:41 ` Taylor Blau
2023-10-10 20:33 ` [PATCH v3 09/17] repo-settings: introduce commitgraph.changedPathsVersion Taylor Blau
2023-10-10 20:33 ` [PATCH v3 10/17] commit-graph: new filter ver. that fixes murmur3 Taylor Blau
2023-10-17 8:45 ` Patrick Steinhardt
2023-10-18 17:46 ` Taylor Blau
2023-10-10 20:33 ` [PATCH v3 11/17] bloom: annotate filters with hash version Taylor Blau
2023-10-10 20:33 ` [PATCH v3 12/17] bloom: prepare to discard incompatible Bloom filters Taylor Blau
2023-10-10 20:33 ` [PATCH v3 13/17] commit-graph.c: unconditionally load " Taylor Blau
2023-10-17 8:45 ` Patrick Steinhardt
2023-10-10 20:34 ` [PATCH v3 14/17] commit-graph: drop unnecessary `graph_read_bloom_data_context` Taylor Blau
2023-10-10 20:34 ` [PATCH v3 15/17] object.h: fix mis-aligned flag bits table Taylor Blau
2023-10-10 20:34 ` [PATCH v3 16/17] commit-graph: reuse existing Bloom filters where possible Taylor Blau
2023-10-10 20:34 ` [PATCH v3 17/17] bloom: introduce `deinit_bloom_filters()` Taylor Blau
2023-10-17 8:45 ` [PATCH v3 00/17] bloom: changed-path Bloom filters v2 (& sundries) Patrick Steinhardt
2023-10-18 17:47 ` Taylor Blau
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1693413637.git.jonathantanmy@google.com \
--to=jonathantanmy@google.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=me@ttaylorr.com \
--cc=szeder.dev@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).