From: Junio C Hamano <gitster@pobox.com>
To: Jeff King <peff@peff.net>
Cc: Taylor Blau <me@ttaylorr.com>, git@vger.kernel.org
Subject: Re: [PATCH 4/4] t5326: test propagating hashcache values
Date: Mon, 13 Sep 2021 22:49:22 -0700 [thread overview]
Message-ID: <xmqqilz34sgt.fsf@gitster.g> (raw)
In-Reply-To: <YUAxxPcd7RHC2AH+@coredump.intra.peff.net> (Jeff King's message of "Tue, 14 Sep 2021 01:23:16 -0400")
Jeff King <peff@peff.net> writes:
> On Mon, Sep 13, 2021 at 07:05:32PM -0700, Junio C Hamano wrote:
>
>> Taylor Blau <me@ttaylorr.com> writes:
>>
>> > Alas, there they are. They are basically no different than having the
>> > name-hash for single pack bitmaps, it's just now we don't throw them
>> > away when generating a MIDX bitmap from a state where the repository
>> > already has a single-pack bitmap.
>>
>> I actually wasn't expecting any CPU/time difference.
>
> I was, for the same reason we saw an improvement there in ae4f07fbcc
> (pack-bitmap: implement optional name_hash cache, 2013-12-21): without a
> name-hash, we try a bunch of fruitless deltas before we find a decent
> one.
Nice. We learn new things every once in a while ;-)
>> I hope that we are talking about the same name-hash, which is used
>> to sort the blobs so that when pack-objects try to find a good delta
>> base, the blobs from the same path will sit close to each other and
>> hopefully fit in the pack window.
>
> Yes, exactly. We spend less time finding the good ones if the likely
> candidates are close together. We may _also_ find better ones overall,
> depending on the number of candidates and the window size.
It is a pleasant surprise that in a real history like linux.git we
can even find good delta base without the name hash (unless we are
using insanely wide window size, that is). The objects in such a
case will be sorted solely by size, larger to smaller, and we need
to find a good delta base within that window. It may not be as
horrible as fast-import (which only tries to delta against the
previous single object), but with ~70k paths in a single revision
with history that is ~1m deep, I was pessimistic to see a size-only
sort to drop even a pair of blobs from the same path within the
default window size of 10. But it seems that such a pessimism is
unwarranted, which is a good news ;-).
next prev parent reply other threads:[~2021-09-14 5:49 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-07 21:17 [PATCH 0/4] pack-bitmap: permute existing namehash values Taylor Blau
2021-09-07 21:17 ` [PATCH 1/4] t/helper/test-bitmap.c: add 'dump-hashes' mode Taylor Blau
2021-09-08 1:37 ` Ævar Arnfjörð Bjarmason
2021-09-08 2:24 ` Taylor Blau
2021-09-07 21:17 ` [PATCH 2/4] pack-bitmap.c: propagate namehash values from existing bitmaps Taylor Blau
2021-09-07 21:18 ` [PATCH 3/4] midx.c: respect 'pack.writeBitmapHashcache' when writing bitmaps Taylor Blau
2021-09-08 1:40 ` Ævar Arnfjörð Bjarmason
2021-09-08 2:28 ` Taylor Blau
2021-09-09 8:18 ` Ævar Arnfjörð Bjarmason
2021-09-09 9:34 ` Ævar Arnfjörð Bjarmason
2021-09-09 14:55 ` Taylor Blau
2021-09-09 15:50 ` Ævar Arnfjörð Bjarmason
2021-09-09 16:23 ` Taylor Blau
2021-09-09 14:47 ` Taylor Blau
2021-09-13 0:38 ` Junio C Hamano
2021-09-14 1:15 ` Taylor Blau
2021-09-07 21:18 ` [PATCH 4/4] t5326: test propagating hashcache values Taylor Blau
2021-09-08 1:46 ` Ævar Arnfjörð Bjarmason
2021-09-08 2:30 ` Taylor Blau
2021-09-17 8:56 ` Ævar Arnfjörð Bjarmason
2021-09-17 17:32 ` Taylor Blau
2021-09-17 19:22 ` Ævar Arnfjörð Bjarmason
2021-09-13 0:46 ` Junio C Hamano
2021-09-14 1:12 ` Taylor Blau
2021-09-14 2:05 ` Junio C Hamano
2021-09-14 5:11 ` Taylor Blau
2021-09-14 5:17 ` Taylor Blau
2021-09-14 5:27 ` Jeff King
2021-09-14 5:31 ` Taylor Blau
2021-09-14 5:23 ` Jeff King
2021-09-14 5:49 ` Junio C Hamano [this message]
2021-09-14 22:05 ` [PATCH v2 0/7] pack-bitmap: permute existing namehash values Taylor Blau
2021-09-14 22:06 ` [PATCH v2 1/7] t/helper/test-bitmap.c: add 'dump-hashes' mode Taylor Blau
2021-09-14 22:06 ` [PATCH v2 2/7] pack-bitmap.c: propagate namehash values from existing bitmaps Taylor Blau
2021-09-14 22:06 ` [PATCH v2 3/7] midx.c: respect 'pack.writeBitmapHashcache' when writing bitmaps Taylor Blau
2021-09-14 22:06 ` [PATCH v2 4/7] p5326: create missing 'perf-tag' tag Taylor Blau
2021-09-16 22:36 ` Jeff King
2021-09-17 4:14 ` Taylor Blau
2021-09-14 22:06 ` [PATCH v2 5/7] p5326: don't set core.multiPackIndex unnecessarily Taylor Blau
2021-09-16 22:38 ` Jeff King
2021-09-14 22:06 ` [PATCH v2 6/7] p5326: generate pack bitmaps before writing the MIDX bitmap Taylor Blau
2021-09-16 22:45 ` Jeff King
2021-09-17 4:20 ` Taylor Blau
2021-09-14 22:06 ` [PATCH v2 7/7] t5326: test propagating hashcache values Taylor Blau
2021-09-16 22:49 ` Jeff King
2021-09-16 22:52 ` [PATCH v2 0/7] pack-bitmap: permute existing namehash values Jeff King
2021-09-17 21:21 ` [PATCH v3 " Taylor Blau
2021-09-17 21:21 ` [PATCH v3 1/7] t/helper/test-bitmap.c: add 'dump-hashes' mode Taylor Blau
2021-09-17 21:21 ` [PATCH v3 2/7] pack-bitmap.c: propagate namehash values from existing bitmaps Taylor Blau
2021-09-17 21:21 ` [PATCH v3 3/7] midx.c: respect 'pack.writeBitmapHashcache' when writing bitmaps Taylor Blau
2021-09-17 21:21 ` [PATCH v3 4/7] p5326: create missing 'perf-tag' tag Taylor Blau
2021-09-17 21:21 ` [PATCH v3 5/7] p5326: don't set core.multiPackIndex unnecessarily Taylor Blau
2021-09-17 21:21 ` [PATCH v3 6/7] p5326: generate pack bitmaps before writing the MIDX bitmap Taylor Blau
2021-09-17 21:21 ` [PATCH v3 7/7] t5326: test propagating hashcache values Taylor Blau
2021-09-17 22:12 ` [PATCH v3 0/7] pack-bitmap: permute existing namehash values Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqilz34sgt.fsf@gitster.g \
--to=gitster@pobox.com \
--cc=git@vger.kernel.org \
--cc=me@ttaylorr.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.