From: Taylor Blau <me@ttaylorr.com>
To: git@vger.kernel.org
Cc: Jeff King <peff@peff.net>, Elijah Newren <newren@gmail.com>,
Junio C Hamano <gitster@pobox.com>
Subject: [PATCH v2 03/19] midx: teach `nth_midxed_pack_int_id()` about incremental MIDXs
Date: Wed, 17 Jul 2024 17:12:04 -0400 [thread overview]
Message-ID: <f449a72877de97d89e31a56724b4f65be2f33f20.1721250704.git.me@ttaylorr.com> (raw)
In-Reply-To: <cover.1721250704.git.me@ttaylorr.com>
The function `nth_midxed_pack_int_id()` takes in a object position in
MIDX lexicographic order and returns an identifier of the pack from
which that object was selected in the MIDX.
Currently, the given object position is an index into the lexicographic
order of objects in a single MIDX. Change this position to instead refer
into the concatenated lexicographic order of all MIDXs in a MIDX chain.
This has two visible effects within the implementation of
`prepare_midx_pack()`:
- First, the given position is now an index into the concatenated
lexicographic order of all MIDXs in the order in which they appear
in the MIDX chain.
- Second the pack ID returned from this function is now also in the
concatenated order of packs among all layers of the MIDX chain in
the same order that they appear in the MIDX chain.
To do this, introduce the first of two general purpose helpers, this one
being `midx_for_object()`. `midx_for_object()` takes a double pointer to
a `struct multi_pack_index` as well as an object `pos` in terms of the
entire MIDX chain[^1].
The function chases down the '->base_midx' field until it finds the MIDX
layer within the chain that contains the given object. It then:
- modifies the double pointer to point to the containing MIDX, instead
of the tip of the chain, and
- returns the MIDX-local position[^2] at which the given object can be
found.
Use this function within `nth_midxed_pack_int_id()` so that the `pos` it
expects is now relative to the entire MIDX chain, and that it returns
the appropriate pack position for that object.
[^1]: As a reminder, this means that the object is identified among the
objects contained in all layers of the incremental MIDX chain, not any
particular layer. For example, consider MIDX chain with two individual
MIDXs, one with 4 objects and another with 3 objects. If the MIDX with
4 objects appears earlier in the chain, then asking for pack "6" would
return the second object in the MIDX with 3 objects.
[^2]: Building on the previous example, asking for object 6 in a MIDX
chain with (4, 3) objects, respectively, this would set the double
pointer to point at the MIDX containing three objects, and would
return an index to the second object within that MIDX.
Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
midx.c | 23 +++++++++++++++++++++--
1 file changed, 21 insertions(+), 2 deletions(-)
diff --git a/midx.c b/midx.c
index 3992b05465..39d358da20 100644
--- a/midx.c
+++ b/midx.c
@@ -242,6 +242,23 @@ void close_midx(struct multi_pack_index *m)
free(m);
}
+static uint32_t midx_for_object(struct multi_pack_index **_m, uint32_t pos)
+{
+ struct multi_pack_index *m = *_m;
+ while (m && pos < m->num_objects_in_base)
+ m = m->base_midx;
+
+ if (!m)
+ BUG("NULL multi-pack-index for object position: %"PRIu32, pos);
+
+ if (pos >= m->num_objects + m->num_objects_in_base)
+ die(_("invalid MIDX object position, MIDX is likely corrupt"));
+
+ *_m = m;
+
+ return pos - m->num_objects_in_base;
+}
+
int prepare_midx_pack(struct repository *r, struct multi_pack_index *m, uint32_t pack_int_id)
{
struct strbuf pack_name = STRBUF_INIT;
@@ -334,8 +351,10 @@ off_t nth_midxed_offset(struct multi_pack_index *m, uint32_t pos)
uint32_t nth_midxed_pack_int_id(struct multi_pack_index *m, uint32_t pos)
{
- return get_be32(m->chunk_object_offsets +
- (off_t)pos * MIDX_CHUNK_OFFSET_WIDTH);
+ pos = midx_for_object(&m, pos);
+
+ return m->num_packs_in_base + get_be32(m->chunk_object_offsets +
+ (off_t)pos * MIDX_CHUNK_OFFSET_WIDTH);
}
int fill_midx_entry(struct repository *r,
--
2.46.0.rc0.94.g9b2aff57b3
next prev parent reply other threads:[~2024-07-17 21:12 UTC|newest]
Thread overview: 102+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-06-06 23:04 [PATCH 00/19] midx: incremental multi-pack indexes, part one Taylor Blau
2024-06-06 23:04 ` [PATCH 01/19] Documentation: describe incremental MIDX format Taylor Blau
2024-06-06 23:04 ` [PATCH 02/19] midx: add new fields for incremental MIDX chains Taylor Blau
2024-06-06 23:04 ` [PATCH 03/19] midx: teach `nth_midxed_pack_int_id()` about incremental MIDXs Taylor Blau
2024-06-06 23:04 ` [PATCH 04/19] midx: teach `prepare_midx_pack()` " Taylor Blau
2024-06-06 23:04 ` [PATCH 05/19] midx: teach `nth_midxed_object_oid()` " Taylor Blau
2024-06-06 23:04 ` [PATCH 06/19] midx: teach `nth_bitmapped_pack()` " Taylor Blau
2024-06-06 23:04 ` [PATCH 07/19] midx: introduce `bsearch_one_midx()` Taylor Blau
2024-06-06 23:04 ` [PATCH 08/19] midx: teach `bsearch_midx()` about incremental MIDXs Taylor Blau
2024-06-06 23:04 ` [PATCH 09/19] midx: teach `nth_midxed_offset()` " Taylor Blau
2024-06-06 23:04 ` [PATCH 10/19] midx: teach `fill_midx_entry()` " Taylor Blau
2024-06-06 23:04 ` [PATCH 11/19] midx: remove unused `midx_locate_pack()` Taylor Blau
2024-06-06 23:05 ` [PATCH 12/19] midx: teach `midx_contains_pack()` about incremental MIDXs Taylor Blau
2024-06-06 23:05 ` [PATCH 13/19] midx: teach `midx_preferred_pack()` " Taylor Blau
2024-06-06 23:05 ` [PATCH 14/19] midx: teach `midx_fanout_add_midx_fanout()` " Taylor Blau
2024-06-06 23:05 ` [PATCH 15/19] midx: support reading incremental MIDX chains Taylor Blau
2024-06-06 23:05 ` [PATCH 16/19] midx: implement verification support for incremental MIDXs Taylor Blau
2024-06-06 23:05 ` [PATCH 17/19] t: retire 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP' Taylor Blau
2024-06-06 23:05 ` [PATCH 18/19] t/t5313-pack-bounds-checks.sh: prepare for sub-directories Taylor Blau
2024-06-06 23:05 ` [PATCH 19/19] midx: implement support for writing incremental MIDX chains Taylor Blau
2024-06-06 23:06 ` [PATCH 00/19] midx: incremental multi-pack indexes, part one Taylor Blau
2024-06-07 18:33 ` Junio C Hamano
2024-06-07 20:29 ` Taylor Blau
2024-06-07 17:55 ` Junio C Hamano
2024-06-07 20:31 ` Taylor Blau
2024-06-25 23:21 ` Junio C Hamano
2024-06-26 0:44 ` Elijah Newren
2024-07-17 21:11 ` [PATCH v2 " Taylor Blau
2024-07-17 21:11 ` [PATCH v2 01/19] Documentation: describe incremental MIDX format Taylor Blau
2024-08-01 9:19 ` Jeff King
2024-08-01 18:52 ` Taylor Blau
2024-07-17 21:12 ` [PATCH v2 02/19] midx: add new fields for incremental MIDX chains Taylor Blau
2024-08-01 9:21 ` Jeff King
2024-08-01 18:54 ` Taylor Blau
2024-07-17 21:12 ` Taylor Blau [this message]
2024-08-01 9:30 ` [PATCH v2 03/19] midx: teach `nth_midxed_pack_int_id()` about incremental MIDXs Jeff King
2024-08-01 18:57 ` Taylor Blau
2024-07-17 21:12 ` [PATCH v2 04/19] midx: teach `prepare_midx_pack()` " Taylor Blau
2024-08-01 9:35 ` Jeff King
2024-08-01 19:00 ` Taylor Blau
2024-07-17 21:12 ` [PATCH v2 05/19] midx: teach `nth_midxed_object_oid()` " Taylor Blau
2024-08-01 9:38 ` Jeff King
2024-08-01 19:03 ` Taylor Blau
2024-07-17 21:12 ` [PATCH v2 06/19] midx: teach `nth_bitmapped_pack()` " Taylor Blau
2024-08-01 9:39 ` Jeff King
2024-08-01 19:07 ` Taylor Blau
2024-07-17 21:12 ` [PATCH v2 07/19] midx: introduce `bsearch_one_midx()` Taylor Blau
2024-08-01 10:06 ` Jeff King
2024-08-01 19:54 ` Taylor Blau
2024-07-17 21:12 ` [PATCH v2 08/19] midx: teach `bsearch_midx()` about incremental MIDXs Taylor Blau
2024-08-01 10:07 ` Jeff King
2024-07-17 21:12 ` [PATCH v2 09/19] midx: teach `nth_midxed_offset()` " Taylor Blau
2024-08-01 10:08 ` Jeff King
2024-07-17 21:12 ` [PATCH v2 10/19] midx: teach `fill_midx_entry()` " Taylor Blau
2024-08-01 10:12 ` Jeff King
2024-08-01 20:01 ` Taylor Blau
2024-07-17 21:12 ` [PATCH v2 11/19] midx: remove unused `midx_locate_pack()` Taylor Blau
2024-08-01 10:14 ` Jeff King
2024-08-01 20:01 ` Taylor Blau
2024-07-17 21:12 ` [PATCH v2 12/19] midx: teach `midx_contains_pack()` about incremental MIDXs Taylor Blau
2024-08-01 10:17 ` Jeff King
2024-07-17 21:12 ` [PATCH v2 13/19] midx: teach `midx_preferred_pack()` " Taylor Blau
2024-08-01 10:25 ` Jeff King
2024-08-01 20:05 ` Taylor Blau
2024-07-17 21:12 ` [PATCH v2 14/19] midx: teach `midx_fanout_add_midx_fanout()` " Taylor Blau
2024-08-01 10:29 ` Jeff King
2024-08-01 20:09 ` Taylor Blau
2024-07-17 21:12 ` [PATCH v2 15/19] midx: support reading incremental MIDX chains Taylor Blau
2024-08-01 10:40 ` Jeff King
2024-08-01 20:35 ` Taylor Blau
2024-07-17 21:12 ` [PATCH v2 16/19] midx: implement verification support for incremental MIDXs Taylor Blau
2024-08-01 10:41 ` Jeff King
2024-07-17 21:12 ` [PATCH v2 17/19] t: retire 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP' Taylor Blau
2024-08-01 10:46 ` Jeff King
2024-08-01 20:36 ` Taylor Blau
2024-07-17 21:12 ` [PATCH v2 18/19] t/t5313-pack-bounds-checks.sh: prepare for sub-directories Taylor Blau
2024-07-17 21:12 ` [PATCH v2 19/19] midx: implement support for writing incremental MIDX chains Taylor Blau
2024-08-01 11:07 ` Jeff King
2024-08-01 20:39 ` Taylor Blau
2024-08-01 11:14 ` [PATCH v2 00/19] midx: incremental multi-pack indexes, part one Jeff King
2024-08-01 20:41 ` Taylor Blau
2024-08-06 15:36 ` [PATCH v3 " Taylor Blau
2024-08-06 15:36 ` [PATCH v3 01/19] Documentation: describe incremental MIDX format Taylor Blau
2024-08-06 15:36 ` [PATCH v3 02/19] midx: add new fields for incremental MIDX chains Taylor Blau
2024-08-06 15:37 ` [PATCH v3 03/19] midx: teach `nth_midxed_pack_int_id()` about incremental MIDXs Taylor Blau
2024-08-06 15:37 ` [PATCH v3 04/19] midx: teach `prepare_midx_pack()` " Taylor Blau
2024-08-06 15:37 ` [PATCH v3 05/19] midx: teach `nth_midxed_object_oid()` " Taylor Blau
2024-08-06 15:37 ` [PATCH v3 06/19] midx: teach `nth_bitmapped_pack()` " Taylor Blau
2024-08-06 15:37 ` [PATCH v3 07/19] midx: introduce `bsearch_one_midx()` Taylor Blau
2024-08-06 15:37 ` [PATCH v3 08/19] midx: teach `bsearch_midx()` about incremental MIDXs Taylor Blau
2024-08-06 15:37 ` [PATCH v3 09/19] midx: teach `nth_midxed_offset()` " Taylor Blau
2024-08-06 15:37 ` [PATCH v3 10/19] midx: teach `fill_midx_entry()` " Taylor Blau
2024-08-06 15:37 ` [PATCH v3 11/19] midx: remove unused `midx_locate_pack()` Taylor Blau
2024-08-06 15:37 ` [PATCH v3 12/19] midx: teach `midx_contains_pack()` about incremental MIDXs Taylor Blau
2024-08-06 15:37 ` [PATCH v3 13/19] midx: teach `midx_preferred_pack()` " Taylor Blau
2024-08-06 15:37 ` [PATCH v3 14/19] midx: teach `midx_fanout_add_midx_fanout()` " Taylor Blau
2024-08-06 15:37 ` [PATCH v3 15/19] midx: support reading incremental MIDX chains Taylor Blau
2024-08-06 15:37 ` [PATCH v3 16/19] midx: implement verification support for incremental MIDXs Taylor Blau
2024-08-06 15:38 ` [PATCH v3 17/19] t: retire 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP' Taylor Blau
2024-08-06 15:38 ` [PATCH v3 18/19] t/t5313-pack-bounds-checks.sh: prepare for sub-directories Taylor Blau
2024-08-06 15:38 ` [PATCH v3 19/19] midx: implement support for writing incremental MIDX chains Taylor Blau
2024-08-12 14:27 ` [PATCH v3 00/19] midx: incremental multi-pack indexes, part one Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f449a72877de97d89e31a56724b4f65be2f33f20.1721250704.git.me@ttaylorr.com \
--to=me@ttaylorr.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=newren@gmail.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).