All of lore.kernel.org
 help / color / mirror / Atom feed
From: Taylor Blau <me@ttaylorr.com>
To: Jeff King <peff@peff.net>
Cc: git@vger.kernel.org, Elijah Newren <newren@gmail.com>,
	Junio C Hamano <gitster@pobox.com>,
	Patrick Steinhardt <ps@pks.im>
Subject: Re: [PATCH 00/11] pack-bitmap: convert offset to ref deltas where possible
Date: Tue, 19 Nov 2024 18:34:12 -0500	[thread overview]
Message-ID: <Zz0gdHrwC4CTAtZn@nand.local> (raw)
In-Reply-To: <Zz0aaPdHiFyoRkKg@nand.local>

On Tue, Nov 19, 2024 at 06:08:24PM -0500, Taylor Blau wrote:
> Then when running the same command, we get results that are quite
> encouraging. The runtime jumps to 24.213 seconds, which is ~9.73 seconds
> slower than the average of the baseline measurements. But it takes
> ~10.418 seconds on my machine to compute the forward index. So it's
> really around 688ms *faster* than the baseline, despite doing a little
> more work.

Sorry, there is a much quicker way to generate the forward index at
runtime, which is the following:

--- 8< ---
diff --git a/midx.c b/midx.c
index 67e0d64004..1023bfa51a 100644
--- a/midx.c
+++ b/midx.c
@@ -989,3 +989,17 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag

 	return verify_midx_error;
 }
+
+void midx_populate_forward_index(struct multi_pack_index *m)
+{
+	uint32_t i;
+
+	ALLOC_ARRAY(m->forward_idx, m->num_objects);
+
+	trace2_region_enter("midx", "populate_forward_index", the_repository);
+
+	for (i = 0; i < m->num_objects; i++)
+		m->forward_idx[pack_pos_to_midx(m, i)] = i;
+
+	trace2_region_leave("midx", "populate_forward_index", the_repository);
+}
--- >8 ---

I don't know if this makes as much of a difference for regular
multi-pack reuse, since there we're only making calls to
midx_pair_to_pack_pos(), which already uses midx_pack_order_cmp(), so
there isn't any additional translation to do.

Thanks,
Taylor

  reply	other threads:[~2024-11-19 23:34 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-10-09 20:30 [PATCH 00/11] pack-bitmap: convert offset to ref deltas where possible Taylor Blau
2024-10-09 20:31 ` [PATCH 01/11] pack-bitmap.c: do not pass `pack_pos` to `try_partial_reuse()` Taylor Blau
2024-10-09 20:31 ` [PATCH 02/11] pack-bitmap.c: avoid unnecessary `offset_to_pack_pos()` Taylor Blau
2024-10-09 20:31 ` [PATCH 03/11] pack-bitmap.c: delay calling 'offset_to_pack_pos()' Taylor Blau
2024-10-09 20:31 ` [PATCH 04/11] pack-bitmap.c: compare `base_offset` to `delta_obj_offset` Taylor Blau
2024-10-09 20:31 ` [PATCH 05/11] pack-bitmap.c: extract `find_base_bitmap_pos()` Taylor Blau
2024-10-09 20:31 ` [PATCH 06/11] pack-bitmap: drop `from_midx` field from `bitmapped_pack` Taylor Blau
2024-10-09 20:31 ` [PATCH 07/11] write_reused_pack_one(): translate bit positions directly Taylor Blau
2024-10-11  8:16   ` Jeff King
2024-11-04 20:36     ` Taylor Blau
2024-10-09 20:31 ` [PATCH 08/11] t5332: enable OFS_DELTAs via test_pack_objects_reused Taylor Blau
2024-10-11  8:19   ` Jeff King
2024-11-04 20:50     ` Taylor Blau
2024-10-09 20:31 ` [PATCH 09/11] pack-bitmap: enable cross-pack delta reuse Taylor Blau
2024-10-11  8:31   ` Jeff King
2024-11-04 21:00     ` Taylor Blau
2024-10-09 20:31 ` [PATCH 10/11] pack-bitmap.c: record whether the result was filtered Taylor Blau
2024-10-11  8:35   ` Jeff King
2024-11-04 21:01     ` Taylor Blau
2024-10-09 20:31 ` [PATCH 11/11] pack-bitmap: enable reusing deltas with base objects in 'haves' bitmap Taylor Blau
2024-10-10 16:46 ` [PATCH 00/11] pack-bitmap: convert offset to ref deltas where possible Junio C Hamano
2024-10-10 17:07   ` Taylor Blau
2024-10-10 20:20     ` Junio C Hamano
2024-10-10 20:32       ` Taylor Blau
2024-10-11  7:54       ` Jeff King
2024-10-11  8:01         ` Jeff King
2024-10-11 16:23           ` Junio C Hamano
2024-10-11  8:38 ` Jeff King
2024-11-19 23:08   ` Taylor Blau
2024-11-19 23:34     ` Taylor Blau [this message]
2024-12-18 12:57       ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Zz0gdHrwC4CTAtZn@nand.local \
    --to=me@ttaylorr.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=newren@gmail.com \
    --cc=peff@peff.net \
    --cc=ps@pks.im \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.