From: "SZEDER Gábor" <szeder.dev@gmail.com>
To: Jeff King <peff@peff.net>
Cc: "Derrick Stolee" <stolee@gmail.com>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
"Stefan Beller" <sbeller@google.com>, git <git@vger.kernel.org>,
"Duy Nguyen" <pclouds@gmail.com>
Subject: Re: We should add a "git gc --auto" after "git clone" due to commit graph
Date: Mon, 8 Oct 2018 18:41:41 +0200 [thread overview]
Message-ID: <20181008164141.GZ23446@szeder.dev> (raw)
In-Reply-To: <20181003191805.GB16666@sigill.intra.peff.net>
On Wed, Oct 03, 2018 at 03:18:05PM -0400, Jeff King wrote:
> I'm still excited about the prospect of a bloom filter for paths which
> each commit touches. I think that's the next big frontier in getting
> things like "git log -- path" to a reasonable run-time.
There is certainly potential there. With a (very) rough PoC
experiment, a 8MB bloom filter, and a carefully choosen path I can
achieve a nice, almost 25x speedup:
$ time git rev-list --count HEAD -- t/valgrind/valgrind.sh
6
real 0m1.563s
user 0m1.519s
sys 0m0.045s
$ time GIT_USE_POC_BLOOM_FILTER=y ~/src/git/git rev-list --count HEAD -- t/valgrind/valgrind.sh
6
real 0m0.063s
user 0m0.043s
sys 0m0.020s
bloom filter total queries: 16269 definitely not: 16195 maybe: 74 false positives: 64 fp ratio: 0.003934
But I'm afraid it will take a while until I get around to turn it into
something presentable...
next prev parent reply other threads:[~2018-10-08 16:41 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-03 13:23 We should add a "git gc --auto" after "git clone" due to commit graph Ævar Arnfjörð Bjarmason
2018-10-03 13:36 ` SZEDER Gábor
2018-10-03 13:42 ` Derrick Stolee
2018-10-03 14:18 ` Ævar Arnfjörð Bjarmason
2018-10-03 14:01 ` Ævar Arnfjörð Bjarmason
2018-10-03 14:17 ` SZEDER Gábor
2018-10-03 14:22 ` Ævar Arnfjörð Bjarmason
2018-10-03 14:53 ` SZEDER Gábor
2018-10-03 15:19 ` Ævar Arnfjörð Bjarmason
2018-10-03 16:59 ` SZEDER Gábor
2018-10-05 6:09 ` Junio C Hamano
2018-10-10 22:07 ` SZEDER Gábor
2018-10-10 23:01 ` Ævar Arnfjörð Bjarmason
2018-10-03 19:08 ` Stefan Beller
2018-10-03 19:21 ` Jeff King
2018-10-03 20:35 ` Ævar Arnfjörð Bjarmason
2018-10-03 17:47 ` Stefan Beller
2018-10-03 18:47 ` Ævar Arnfjörð Bjarmason
2018-10-03 18:51 ` Jeff King
2018-10-03 18:59 ` Derrick Stolee
2018-10-03 19:18 ` Jeff King
2018-10-08 16:41 ` SZEDER Gábor [this message]
2018-10-08 16:57 ` Derrick Stolee
2018-10-08 18:10 ` SZEDER Gábor
2018-10-08 18:29 ` Derrick Stolee
2018-10-09 3:08 ` Jeff King
2018-10-09 13:48 ` Bloom Filters (was Re: We should add a "git gc --auto" after "git clone" due to commit graph) Derrick Stolee
2018-10-09 18:45 ` Ævar Arnfjörð Bjarmason
2018-10-09 18:46 ` Jeff King
2018-10-09 19:03 ` Derrick Stolee
2018-10-09 21:14 ` Jeff King
2018-10-09 23:12 ` Bloom Filters Jeff King
2018-10-09 23:13 ` [PoC -- do not apply 1/3] initial tree-bitmap proof of concept Jeff King
2018-10-09 23:14 ` [PoC -- do not apply 2/3] test-tree-bitmap: add "dump" mode Jeff King
2018-10-10 0:48 ` Junio C Hamano
2018-10-11 3:13 ` Jeff King
2018-10-09 23:14 ` [PoC -- do not apply 3/3] test-tree-bitmap: replace ewah with custom rle encoding Jeff King
2018-10-10 0:58 ` Junio C Hamano
2018-10-11 3:20 ` Jeff King
2018-10-11 12:33 ` Bloom Filters Derrick Stolee
2018-10-11 13:43 ` Jeff King
2018-10-09 21:30 ` We should add a "git gc --auto" after "git clone" due to commit graph SZEDER Gábor
2018-10-09 19:34 ` [PATCH 0/4] Bloom filter experiment SZEDER Gábor
2018-10-09 19:34 ` [PATCH 1/4] Add a (very) barebones Bloom filter implementation SZEDER Gábor
2018-10-09 19:34 ` [PATCH 2/4] commit-graph: write a Bloom filter containing changed paths for each commit SZEDER Gábor
2018-10-09 21:06 ` Jeff King
2018-10-09 21:37 ` SZEDER Gábor
2018-10-09 19:34 ` [PATCH 3/4] revision.c: use the Bloom filter to speed up path-limited revision walks SZEDER Gábor
2018-10-09 19:34 ` [PATCH 4/4] revision.c: add GIT_TRACE_BLOOM_FILTER for a bit of statistics SZEDER Gábor
2018-10-09 19:47 ` [PATCH 0/4] Bloom filter experiment Derrick Stolee
2018-10-11 1:21 ` [PATCH 0/2] Per-commit filter proof of concept Jonathan Tan
2018-10-11 1:21 ` [PATCH 1/2] One filter per commit Jonathan Tan
2018-10-11 12:49 ` Derrick Stolee
2018-10-11 19:11 ` [PATCH] Per-commit and per-parent filters for 2 parents Jonathan Tan
2018-10-11 1:21 ` [PATCH 2/2] Only make bloom filter for first parent Jonathan Tan
2018-10-11 7:37 ` [PATCH 0/2] Per-commit filter proof of concept Ævar Arnfjörð Bjarmason
2018-10-15 14:39 ` [PATCH 0/4] Bloom filter experiment Derrick Stolee
2018-10-16 4:45 ` Junio C Hamano
2018-10-16 11:13 ` Derrick Stolee
2018-10-16 12:57 ` Ævar Arnfjörð Bjarmason
2018-10-16 13:03 ` Derrick Stolee
2018-10-18 2:00 ` Junio C Hamano
2018-10-16 23:41 ` Jonathan Tan
2018-10-08 23:02 ` We should add a "git gc --auto" after "git clone" due to commit graph Junio C Hamano
2018-10-03 14:32 ` Duy Nguyen
2018-10-03 16:45 ` Duy Nguyen
2018-10-04 21:42 ` [RFC PATCH] " Ævar Arnfjörð Bjarmason
2018-10-05 12:05 ` Derrick Stolee
2018-10-05 13:05 ` Ævar Arnfjörð Bjarmason
2018-10-05 13:45 ` Derrick Stolee
2018-10-05 14:04 ` Ævar Arnfjörð Bjarmason
2018-10-05 19:21 ` Jeff King
2018-10-05 19:41 ` Derrick Stolee
2018-10-05 19:47 ` Jeff King
2018-10-05 20:00 ` Derrick Stolee
2018-10-05 20:02 ` Jeff King
2018-10-05 20:01 ` Ævar Arnfjörð Bjarmason
2018-10-05 20:09 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181008164141.GZ23446@szeder.dev \
--to=szeder.dev@gmail.com \
--cc=avarab@gmail.com \
--cc=git@vger.kernel.org \
--cc=pclouds@gmail.com \
--cc=peff@peff.net \
--cc=sbeller@google.com \
--cc=stolee@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.