From: Taylor Blau <me@ttaylorr.com>
To: Patrick Steinhardt <ps@pks.im>
Cc: git@vger.kernel.org, Toon Claes <toon@iotcl.com>,
Derrick Stolee <stolee@gmail.com>
Subject: Re: [PATCH v2 6/9] midx: load multi-pack indices via their source
Date: Thu, 7 Aug 2025 18:25:22 -0400 [thread overview]
Message-ID: <aJUn0qeliNQ/nnWr@nand.local> (raw)
In-Reply-To: <20250807-b4-pks-midx-deduplicate-source-info-v2-6-bcffb8fc119c@pks.im>
On Thu, Aug 07, 2025 at 10:09:56AM +0200, Patrick Steinhardt wrote:
> To load a multi-pack index the caller is expected to pass both the
> repository and the object directory where the multi-pack index is
> located. While this works, this layout has a couple of downsides:
>
> - We need to pass in information reduntant with the owning source,
> namely its object directory and whether the source is local or not.
>
> - We don't have access to the source when loading the multi-pack
> index. If we had that access, we could store a pointer to the owning
> source in the MIDX and thus deduplicate some information.
>
> - Multi-pack indices are inherently specific to the object source and
> its format. With the goal of pluggable object backends in mind we
> will eventually want the backends to own the logic of reading and
> writing multi-pack indices. Making the logic work on top of object
> sources is a step into that direction.
>
> Refactor loading of multi-pack indices accordingly.
>
> This surfaces one small problem though: git-multi-pack-index(1) and our
> MIDX test helper both know to read and write multi-pack-indices located
> in a different object directory. This issue is addressed by adding the
> user-provided object directory as an in-memory alternate.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
> builtin/multi-pack-index.c | 18 ++++++++++++--
> midx.c | 57 ++++++++++++++++++++-------------------------
> midx.h | 6 ++---
> t/helper/test-read-midx.c | 25 ++++++++++++--------
> t/t5319-multi-pack-index.sh | 8 +++----
> 5 files changed, 62 insertions(+), 52 deletions(-)
>
> diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
> index aa25b06f9d..e4a9305af3 100644
> --- a/builtin/multi-pack-index.c
> +++ b/builtin/multi-pack-index.c
> @@ -64,12 +64,20 @@ static int parse_object_dir(const struct option *opt, const char *arg,
> char **value = opt->value;
> free(*value);
> if (unset)
> - *value = xstrdup(repo_get_object_directory(the_repository));
> + *value = xstrdup(the_repository->objects->sources->path);
> else
> *value = real_pathdup(arg, 1);
> return 0;
> }
>
> +static struct odb_source *handle_object_dir_option(struct repository *repo)
> +{
> + struct odb_source *source = odb_find_source(repo->objects, opts.object_dir);
> + if (!source)
> + source = odb_add_to_alternates_memory(repo->objects, opts.object_dir);
> + return source;
> +}
> +
Hmm... I admit that I am pretty unfamiliar with the --object-dir option
here and only have a vague recollection of why it was added in the first
place.
I am not sure one way or another if adding the user-provided directory
as an in-memory alternate is the right thing to do here.
Stolee (CC'd) would you mind sharing your thoughts on this?
Thanks,
Taylor
next prev parent reply other threads:[~2025-08-07 22:25 UTC|newest]
Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-07-29 14:12 [PATCH 0/8] midx: stop deduplicating info redundant with their sources Patrick Steinhardt
2025-07-29 14:12 ` [PATCH 1/8] odb: store locality in object database sources Patrick Steinhardt
2025-08-06 16:39 ` Toon Claes
2025-08-07 6:15 ` Patrick Steinhardt
2025-08-07 8:12 ` Karthik Nayak
2025-07-29 14:12 ` [PATCH 2/8] odb: allow `odb_find_source()` to fail Patrick Steinhardt
2025-07-29 14:12 ` [PATCH 3/8] odb: return newly created in-memory sources Patrick Steinhardt
2025-08-06 16:40 ` Toon Claes
2025-08-07 6:16 ` Patrick Steinhardt
2025-08-07 8:21 ` Karthik Nayak
2025-07-29 14:12 ` [PATCH 4/8] midx: drop redundant `struct repository` parameter Patrick Steinhardt
2025-07-29 14:12 ` [PATCH 5/8] midx: load multi-pack indices via their source Patrick Steinhardt
2025-08-07 8:49 ` Karthik Nayak
2025-08-07 8:51 ` Karthik Nayak
2025-07-29 14:12 ` [PATCH 6/8] midx: write " Patrick Steinhardt
2025-08-07 8:55 ` Karthik Nayak
2025-07-29 14:12 ` [PATCH 7/8] midx: stop duplicating info redundant with its owning source Patrick Steinhardt
2025-07-29 14:12 ` [PATCH 8/8] midx: compute paths via their source Patrick Steinhardt
2025-07-29 18:33 ` [PATCH 0/8] midx: stop deduplicating info redundant with their sources Junio C Hamano
2025-07-30 5:21 ` Patrick Steinhardt
2025-08-07 8:09 ` [PATCH v2 0/9] midx: stop duplicating " Patrick Steinhardt
2025-08-07 8:09 ` [PATCH v2 1/9] odb: store locality in object database sources Patrick Steinhardt
2025-08-07 22:10 ` Taylor Blau
2025-08-07 8:09 ` [PATCH v2 2/9] odb: allow `odb_find_source()` to fail Patrick Steinhardt
2025-08-07 22:12 ` Taylor Blau
2025-08-11 11:56 ` Patrick Steinhardt
2025-08-07 8:09 ` [PATCH v2 3/9] odb: return newly created in-memory sources Patrick Steinhardt
2025-08-07 22:16 ` Taylor Blau
2025-08-11 11:56 ` Patrick Steinhardt
2025-08-07 8:09 ` [PATCH v2 4/9] odb: simplify calling `link_alt_odb_entry()` Patrick Steinhardt
2025-08-07 22:21 ` Taylor Blau
2025-08-07 8:09 ` [PATCH v2 5/9] midx: drop redundant `struct repository` parameter Patrick Steinhardt
2025-08-07 8:09 ` [PATCH v2 6/9] midx: load multi-pack indices via their source Patrick Steinhardt
2025-08-07 22:25 ` Taylor Blau [this message]
2025-08-07 8:09 ` [PATCH v2 7/9] midx: write " Patrick Steinhardt
2025-08-07 22:25 ` Taylor Blau
2025-08-07 8:09 ` [PATCH v2 8/9] midx: stop duplicating info redundant with its owning source Patrick Steinhardt
2025-08-07 8:09 ` [PATCH v2 9/9] midx: compute paths via their source Patrick Steinhardt
2025-08-07 22:27 ` [PATCH v2 0/9] midx: stop duplicating info redundant with their sources Taylor Blau
2025-08-11 11:56 ` Patrick Steinhardt
2025-08-07 8:58 ` [PATCH 0/8] midx: stop deduplicating " Karthik Nayak
2025-08-11 13:46 ` [PATCH v3 00/10] midx: stop duplicating " Patrick Steinhardt
2025-08-11 13:46 ` [PATCH v3 01/10] odb: store locality in object database sources Patrick Steinhardt
2025-08-11 13:46 ` [PATCH v3 02/10] odb: allow `odb_find_source()` to fail Patrick Steinhardt
2025-08-11 13:46 ` [PATCH v3 03/10] odb: consistently use "dir" to refer to alternate's directory Patrick Steinhardt
2025-08-11 13:46 ` [PATCH v3 04/10] odb: return newly created in-memory sources Patrick Steinhardt
2025-08-11 13:46 ` [PATCH v3 05/10] odb: simplify calling `link_alt_odb_entry()` Patrick Steinhardt
2025-08-11 13:46 ` [PATCH v3 06/10] midx: drop redundant `struct repository` parameter Patrick Steinhardt
2025-08-11 13:46 ` [PATCH v3 07/10] midx: load multi-pack indices via their source Patrick Steinhardt
2025-08-11 13:46 ` [PATCH v3 08/10] midx: write " Patrick Steinhardt
2025-08-11 13:46 ` [PATCH v3 09/10] midx: stop duplicating info redundant with its owning source Patrick Steinhardt
2025-08-11 13:46 ` [PATCH v3 10/10] midx: compute paths via their source Patrick Steinhardt
2025-08-28 22:46 ` [PATCH v3 00/10] midx: stop duplicating info redundant with their sources Junio C Hamano
2025-08-29 0:34 ` Taylor Blau
2025-08-30 13:39 ` Derrick Stolee
2025-09-02 6:36 ` Patrick Steinhardt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aJUn0qeliNQ/nnWr@nand.local \
--to=me@ttaylorr.com \
--cc=git@vger.kernel.org \
--cc=ps@pks.im \
--cc=stolee@gmail.com \
--cc=toon@iotcl.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).