git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Taylor Blau <me@ttaylorr.com>
To: Patrick Steinhardt <ps@pks.im>
Cc: git@vger.kernel.org, Toon Claes <toon@iotcl.com>,
	Derrick Stolee <stolee@gmail.com>
Subject: Re: [PATCH v2 6/9] midx: load multi-pack indices via their source
Date: Thu, 7 Aug 2025 18:25:22 -0400	[thread overview]
Message-ID: <aJUn0qeliNQ/nnWr@nand.local> (raw)
In-Reply-To: <20250807-b4-pks-midx-deduplicate-source-info-v2-6-bcffb8fc119c@pks.im>

On Thu, Aug 07, 2025 at 10:09:56AM +0200, Patrick Steinhardt wrote:
> To load a multi-pack index the caller is expected to pass both the
> repository and the object directory where the multi-pack index is
> located. While this works, this layout has a couple of downsides:
>
>   - We need to pass in information reduntant with the owning source,
>     namely its object directory and whether the source is local or not.
>
>   - We don't have access to the source when loading the multi-pack
>     index. If we had that access, we could store a pointer to the owning
>     source in the MIDX and thus deduplicate some information.
>
>   - Multi-pack indices are inherently specific to the object source and
>     its format. With the goal of pluggable object backends in mind we
>     will eventually want the backends to own the logic of reading and
>     writing multi-pack indices. Making the logic work on top of object
>     sources is a step into that direction.
>
> Refactor loading of multi-pack indices accordingly.
>
> This surfaces one small problem though: git-multi-pack-index(1) and our
> MIDX test helper both know to read and write multi-pack-indices located
> in a different object directory. This issue is addressed by adding the
> user-provided object directory as an in-memory alternate.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  builtin/multi-pack-index.c  | 18 ++++++++++++--
>  midx.c                      | 57 ++++++++++++++++++++-------------------------
>  midx.h                      |  6 ++---
>  t/helper/test-read-midx.c   | 25 ++++++++++++--------
>  t/t5319-multi-pack-index.sh |  8 +++----
>  5 files changed, 62 insertions(+), 52 deletions(-)
>
> diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
> index aa25b06f9d..e4a9305af3 100644
> --- a/builtin/multi-pack-index.c
> +++ b/builtin/multi-pack-index.c
> @@ -64,12 +64,20 @@ static int parse_object_dir(const struct option *opt, const char *arg,
>  	char **value = opt->value;
>  	free(*value);
>  	if (unset)
> -		*value = xstrdup(repo_get_object_directory(the_repository));
> +		*value = xstrdup(the_repository->objects->sources->path);
>  	else
>  		*value = real_pathdup(arg, 1);
>  	return 0;
>  }
>
> +static struct odb_source *handle_object_dir_option(struct repository *repo)
> +{
> +	struct odb_source *source = odb_find_source(repo->objects, opts.object_dir);
> +	if (!source)
> +		source = odb_add_to_alternates_memory(repo->objects, opts.object_dir);
> +	return source;
> +}
> +

Hmm... I admit that I am pretty unfamiliar with the --object-dir option
here and only have a vague recollection of why it was added in the first
place.

I am not sure one way or another if adding the user-provided directory
as an in-memory alternate is the right thing to do here.

Stolee (CC'd) would you mind sharing your thoughts on this?

Thanks,
Taylor

  reply	other threads:[~2025-08-07 22:25 UTC|newest]

Thread overview: 56+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-07-29 14:12 [PATCH 0/8] midx: stop deduplicating info redundant with their sources Patrick Steinhardt
2025-07-29 14:12 ` [PATCH 1/8] odb: store locality in object database sources Patrick Steinhardt
2025-08-06 16:39   ` Toon Claes
2025-08-07  6:15     ` Patrick Steinhardt
2025-08-07  8:12   ` Karthik Nayak
2025-07-29 14:12 ` [PATCH 2/8] odb: allow `odb_find_source()` to fail Patrick Steinhardt
2025-07-29 14:12 ` [PATCH 3/8] odb: return newly created in-memory sources Patrick Steinhardt
2025-08-06 16:40   ` Toon Claes
2025-08-07  6:16     ` Patrick Steinhardt
2025-08-07  8:21   ` Karthik Nayak
2025-07-29 14:12 ` [PATCH 4/8] midx: drop redundant `struct repository` parameter Patrick Steinhardt
2025-07-29 14:12 ` [PATCH 5/8] midx: load multi-pack indices via their source Patrick Steinhardt
2025-08-07  8:49   ` Karthik Nayak
2025-08-07  8:51     ` Karthik Nayak
2025-07-29 14:12 ` [PATCH 6/8] midx: write " Patrick Steinhardt
2025-08-07  8:55   ` Karthik Nayak
2025-07-29 14:12 ` [PATCH 7/8] midx: stop duplicating info redundant with its owning source Patrick Steinhardt
2025-07-29 14:12 ` [PATCH 8/8] midx: compute paths via their source Patrick Steinhardt
2025-07-29 18:33 ` [PATCH 0/8] midx: stop deduplicating info redundant with their sources Junio C Hamano
2025-07-30  5:21   ` Patrick Steinhardt
2025-08-07  8:09 ` [PATCH v2 0/9] midx: stop duplicating " Patrick Steinhardt
2025-08-07  8:09   ` [PATCH v2 1/9] odb: store locality in object database sources Patrick Steinhardt
2025-08-07 22:10     ` Taylor Blau
2025-08-07  8:09   ` [PATCH v2 2/9] odb: allow `odb_find_source()` to fail Patrick Steinhardt
2025-08-07 22:12     ` Taylor Blau
2025-08-11 11:56       ` Patrick Steinhardt
2025-08-07  8:09   ` [PATCH v2 3/9] odb: return newly created in-memory sources Patrick Steinhardt
2025-08-07 22:16     ` Taylor Blau
2025-08-11 11:56       ` Patrick Steinhardt
2025-08-07  8:09   ` [PATCH v2 4/9] odb: simplify calling `link_alt_odb_entry()` Patrick Steinhardt
2025-08-07 22:21     ` Taylor Blau
2025-08-07  8:09   ` [PATCH v2 5/9] midx: drop redundant `struct repository` parameter Patrick Steinhardt
2025-08-07  8:09   ` [PATCH v2 6/9] midx: load multi-pack indices via their source Patrick Steinhardt
2025-08-07 22:25     ` Taylor Blau [this message]
2025-08-07  8:09   ` [PATCH v2 7/9] midx: write " Patrick Steinhardt
2025-08-07 22:25     ` Taylor Blau
2025-08-07  8:09   ` [PATCH v2 8/9] midx: stop duplicating info redundant with its owning source Patrick Steinhardt
2025-08-07  8:09   ` [PATCH v2 9/9] midx: compute paths via their source Patrick Steinhardt
2025-08-07 22:27   ` [PATCH v2 0/9] midx: stop duplicating info redundant with their sources Taylor Blau
2025-08-11 11:56     ` Patrick Steinhardt
2025-08-07  8:58 ` [PATCH 0/8] midx: stop deduplicating " Karthik Nayak
2025-08-11 13:46 ` [PATCH v3 00/10] midx: stop duplicating " Patrick Steinhardt
2025-08-11 13:46   ` [PATCH v3 01/10] odb: store locality in object database sources Patrick Steinhardt
2025-08-11 13:46   ` [PATCH v3 02/10] odb: allow `odb_find_source()` to fail Patrick Steinhardt
2025-08-11 13:46   ` [PATCH v3 03/10] odb: consistently use "dir" to refer to alternate's directory Patrick Steinhardt
2025-08-11 13:46   ` [PATCH v3 04/10] odb: return newly created in-memory sources Patrick Steinhardt
2025-08-11 13:46   ` [PATCH v3 05/10] odb: simplify calling `link_alt_odb_entry()` Patrick Steinhardt
2025-08-11 13:46   ` [PATCH v3 06/10] midx: drop redundant `struct repository` parameter Patrick Steinhardt
2025-08-11 13:46   ` [PATCH v3 07/10] midx: load multi-pack indices via their source Patrick Steinhardt
2025-08-11 13:46   ` [PATCH v3 08/10] midx: write " Patrick Steinhardt
2025-08-11 13:46   ` [PATCH v3 09/10] midx: stop duplicating info redundant with its owning source Patrick Steinhardt
2025-08-11 13:46   ` [PATCH v3 10/10] midx: compute paths via their source Patrick Steinhardt
2025-08-28 22:46   ` [PATCH v3 00/10] midx: stop duplicating info redundant with their sources Junio C Hamano
2025-08-29  0:34     ` Taylor Blau
2025-08-30 13:39       ` Derrick Stolee
2025-09-02  6:36         ` Patrick Steinhardt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aJUn0qeliNQ/nnWr@nand.local \
    --to=me@ttaylorr.com \
    --cc=git@vger.kernel.org \
    --cc=ps@pks.im \
    --cc=stolee@gmail.com \
    --cc=toon@iotcl.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).