git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Taylor Blau <me@ttaylorr.com>
To: git@vger.kernel.org
Cc: Elijah Newren <newren@gmail.com>, Jeff King <peff@peff.net>,
	Junio C Hamano <gitster@pobox.com>
Subject: [PATCH v6 0/9] repack: avoid MIDX'ing cruft pack(s) where possible
Date: Mon, 23 Jun 2025 18:32:03 -0400	[thread overview]
Message-ID: <cover.1750717921.git.me@ttaylorr.com> (raw)
In-Reply-To: <cover.1744413969.git.me@ttaylorr.com>

Here is an additional reroll of my series to create MIDXs that do not
include a repository's cruft pack(s).

Nearly everything is identical between this version and the previous
(v5), with two exceptions:

 - Adjusted where to split a long line in show_object_pack_hint().

 - Fixed a test failure with GIT_TEST_MULTI_PACK_INDEX_WRITE_INCREMENTAL

Thanks for Junio and Peff (respectively) for pointing out each of the
above. As usual, a range-diff is attached for convenience.

Thanks in advance for any review :-).

Taylor Blau (9):
  pack-objects: use standard option incompatibility functions
  pack-objects: limit scope in 'add_object_entry_from_pack()'
  pack-objects: factor out handling '--stdin-packs'
  pack-objects: declare 'rev_info' for '--stdin-packs' earlier
  pack-objects: perform name-hash traversal for unpacked objects
  pack-objects: fix typo in 'show_object_pack_hint()'
  pack-objects: swap 'show_{object,commit}_pack_hint'
  pack-objects: introduce '--stdin-packs=follow'
  repack: exclude cruft pack(s) from the MIDX where possible

 Documentation/config/repack.adoc    |   7 +
 Documentation/git-pack-objects.adoc |  10 +-
 builtin/pack-objects.c              | 193 ++++++++++++++++++----------
 builtin/repack.c                    | 187 ++++++++++++++++++++++++---
 t/t5331-pack-objects-stdin.sh       | 122 +++++++++++++++++-
 t/t7704-repack-cruft.sh             | 145 +++++++++++++++++++++
 6 files changed, 573 insertions(+), 91 deletions(-)

Range-diff against v5:
 1:  19fab7a35c =  1:  8e7b2dacc7 pack-objects: use standard option incompatibility functions
 2:  6f2d3f17a4 =  2:  86fb36d317 pack-objects: limit scope in 'add_object_entry_from_pack()'
 3:  c06f5b264a =  3:  19e8c789e9 pack-objects: factor out handling '--stdin-packs'
 4:  40d7d87cb1 =  4:  c9f874eb94 pack-objects: declare 'rev_info' for '--stdin-packs' earlier
 5:  5e2599436c =  5:  6b0149a32d pack-objects: perform name-hash traversal for unpacked objects
 6:  3a5c3f63d8 =  6:  f31dd00a98 pack-objects: fix typo in 'show_object_pack_hint()'
 7:  796e8743f8 =  7:  5d15055985 pack-objects: swap 'show_{object,commit}_pack_hint'
 8:  8830775beb !  8:  3699c25337 pack-objects: introduce '--stdin-packs=follow'
    @@ builtin/pack-objects.c: static int add_object_entry_from_pack(const struct objec
     -		return;
     +	enum stdin_packs_mode mode = *(enum stdin_packs_mode *)data;
     +	if (mode == STDIN_PACKS_MODE_FOLLOW) {
    -+		if (object->type == OBJ_BLOB && !has_object(the_repository,
    -+							    &object->oid, 0))
    ++		if (object->type == OBJ_BLOB &&
    ++		    !has_object(the_repository, &object->oid, 0))
     +			return;
     +		add_object_entry(&object->oid, object->type, name, 0);
     +	} else {
 9:  8f505179cc !  9:  f519777059 repack: exclude cruft pack(s) from the MIDX where possible
    @@ builtin/repack.c: int cmd_repack(int argc,
      	string_list_sort(&names);
      
     +	if (get_local_multi_pack_index(the_repository)) {
    -+		uint32_t i;
     +		struct multi_pack_index *m =
     +			get_local_multi_pack_index(the_repository);
     +
    -+		ALLOC_ARRAY(midx_pack_names, m->num_packs);
    -+		for (i = 0; i < m->num_packs; i++)
    -+			midx_pack_names[midx_pack_names_nr++] = xstrdup(m->pack_names[i]);
    ++		ALLOC_ARRAY(midx_pack_names,
    ++			    m->num_packs + m->num_packs_in_base);
    ++
    ++		for (; m; m = m->base_midx)
    ++			for (uint32_t i = 0; i < m->num_packs; i++)
    ++				midx_pack_names[midx_pack_names_nr++] =
    ++					xstrdup(m->pack_names[i]);
     +	}
     +
      	close_object_store(the_repository->objects);

base-commit: f9aa0eedb37eb94d9d3711ef0d565fd7cb3b6148
-- 
2.50.0.61.g1981e40f2d

  parent reply	other threads:[~2025-06-23 22:32 UTC|newest]

Thread overview: 105+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-04-11 23:26 [RFC PATCH 0/8] repack: avoid MIDX'ing cruft pack(s) where possible Taylor Blau
2025-04-11 23:26 ` [RFC PATCH 1/8] pack-objects: use standard option incompatibility functions Taylor Blau
2025-04-11 23:26 ` [RFC PATCH 2/8] pack-objects: limit scope in 'add_object_entry_from_pack()' Taylor Blau
2025-04-11 23:26 ` [RFC PATCH 3/8] pack-objects: factor out handling '--stdin-packs' Taylor Blau
2025-04-11 23:26 ` [RFC PATCH 4/8] pack-objects: declare 'rev_info' for '--stdin-packs' earlier Taylor Blau
2025-04-11 23:26 ` [RFC PATCH 5/8] pack-objects: perform name-hash traversal for unpacked objects Taylor Blau
2025-04-11 23:26 ` [RFC PATCH 6/8] pack-objects: introduce '--stdin-packs=follow' Taylor Blau
2025-04-11 23:26 ` [RFC PATCH 7/8] repack: keep track of existing MIDX'd packs Taylor Blau
2025-04-11 23:26 ` [RFC PATCH 8/8] repack: exclude cruft pack(s) from the MIDX where possible Taylor Blau
2025-04-14 20:06 ` [PATCH v2 0/8] repack: avoid MIDX'ing cruft pack(s) " Taylor Blau
2025-04-14 20:06   ` [PATCH v2 1/8] pack-objects: use standard option incompatibility functions Taylor Blau
2025-04-14 20:41     ` Junio C Hamano
2025-04-15 19:32       ` Taylor Blau
2025-04-15 19:48         ` Junio C Hamano
2025-04-15 22:27           ` Taylor Blau
2025-04-14 20:06   ` [PATCH v2 2/8] object-store-ll.h: add note about designated initializers Taylor Blau
2025-04-14 21:07     ` Junio C Hamano
2025-04-15 19:51       ` Taylor Blau
2025-04-15  2:57     ` Elijah Newren
2025-04-15 19:47       ` Taylor Blau
2025-04-14 20:06   ` [PATCH v2 3/8] pack-objects: limit scope in 'add_object_entry_from_pack()' Taylor Blau
2025-04-15  3:10     ` Elijah Newren
2025-04-14 20:06   ` [PATCH v2 4/8] pack-objects: factor out handling '--stdin-packs' Taylor Blau
2025-04-14 20:06   ` [PATCH v2 5/8] pack-objects: declare 'rev_info' for '--stdin-packs' earlier Taylor Blau
2025-04-14 20:06   ` [PATCH v2 6/8] pack-objects: perform name-hash traversal for unpacked objects Taylor Blau
2025-04-15  3:10     ` Elijah Newren
2025-04-15 19:57       ` Taylor Blau
2025-04-14 20:06   ` [PATCH v2 7/8] pack-objects: introduce '--stdin-packs=follow' Taylor Blau
2025-04-15  3:11     ` Elijah Newren
2025-04-15 20:45       ` Taylor Blau
2025-04-16  5:26         ` Elijah Newren
2025-04-14 20:06   ` [PATCH v2 8/8] repack: exclude cruft pack(s) from the MIDX where possible Taylor Blau
2025-04-15  3:11     ` Elijah Newren
2025-04-15 20:51       ` Taylor Blau
2025-04-15  2:57   ` [PATCH v2 0/8] repack: avoid MIDX'ing cruft pack(s) " Elijah Newren
2025-04-15 22:05     ` Taylor Blau
2025-04-15 22:46 ` [PATCH v3 0/9] " Taylor Blau
2025-04-15 22:46   ` [PATCH v3 1/9] pack-objects: use standard option incompatibility functions Taylor Blau
2025-04-15 22:46   ` [PATCH v3 2/9] pack-objects: limit scope in 'add_object_entry_from_pack()' Taylor Blau
2025-04-16  0:58     ` Junio C Hamano
2025-04-16 22:07       ` Taylor Blau
2025-04-16  5:31     ` Elijah Newren
2025-04-16 22:07       ` Taylor Blau
2025-04-15 22:46   ` [PATCH v3 3/9] pack-objects: factor out handling '--stdin-packs' Taylor Blau
2025-04-16  0:59     ` Junio C Hamano
2025-04-15 22:46   ` [PATCH v3 4/9] pack-objects: declare 'rev_info' for '--stdin-packs' earlier Taylor Blau
2025-04-15 22:47   ` [PATCH v3 5/9] pack-objects: perform name-hash traversal for unpacked objects Taylor Blau
2025-04-16  9:21     ` Junio C Hamano
2025-04-15 22:47   ` [PATCH v3 6/9] pack-objects: fix typo in 'show_object_pack_hint()' Taylor Blau
2025-04-16  5:36     ` Elijah Newren
2025-04-15 22:47   ` [PATCH v3 7/9] pack-objects: swap 'show_{object,commit}_pack_hint' Taylor Blau
2025-04-15 22:47   ` [PATCH v3 8/9] pack-objects: introduce '--stdin-packs=follow' Taylor Blau
2025-04-15 22:47   ` [PATCH v3 9/9] repack: exclude cruft pack(s) from the MIDX where possible Taylor Blau
2025-04-16  5:56     ` Elijah Newren
2025-04-16 22:16       ` Taylor Blau
2025-05-13  3:34         ` Elijah Newren
2025-05-28 23:20 ` [PATCH v4 0/9] repack: avoid MIDX'ing cruft pack(s) " Taylor Blau
2025-05-28 23:20   ` [PATCH v4 1/9] pack-objects: use standard option incompatibility functions Taylor Blau
2025-05-28 23:20   ` [PATCH v4 2/9] pack-objects: limit scope in 'add_object_entry_from_pack()' Taylor Blau
2025-05-28 23:20   ` [PATCH v4 3/9] pack-objects: factor out handling '--stdin-packs' Taylor Blau
2025-05-28 23:20   ` [PATCH v4 4/9] pack-objects: declare 'rev_info' for '--stdin-packs' earlier Taylor Blau
2025-05-28 23:20   ` [PATCH v4 5/9] pack-objects: perform name-hash traversal for unpacked objects Taylor Blau
2025-05-28 23:20   ` [PATCH v4 6/9] pack-objects: fix typo in 'show_object_pack_hint()' Taylor Blau
2025-05-28 23:20   ` [PATCH v4 7/9] pack-objects: swap 'show_{object,commit}_pack_hint' Taylor Blau
2025-05-28 23:20   ` [PATCH v4 8/9] pack-objects: introduce '--stdin-packs=follow' Taylor Blau
2025-05-28 23:20   ` [PATCH v4 9/9] repack: exclude cruft pack(s) from the MIDX where possible Taylor Blau
2025-06-19 11:33     ` Carlo Marcelo Arenas Belón
2025-06-19 13:08     ` [PATCH] fixup! " Carlo Marcelo Arenas Belón
2025-06-19 17:07       ` Junio C Hamano
2025-06-19 23:26         ` Taylor Blau
2025-05-29  0:07   ` [PATCH v4 0/9] repack: avoid MIDX'ing cruft pack(s) " Taylor Blau
2025-05-29  0:15     ` Elijah Newren
2025-06-19 23:30 ` [PATCH v5 " Taylor Blau
2025-06-19 23:30   ` [PATCH v5 1/9] pack-objects: use standard option incompatibility functions Taylor Blau
2025-06-19 23:30   ` [PATCH v5 2/9] pack-objects: limit scope in 'add_object_entry_from_pack()' Taylor Blau
2025-06-19 23:30   ` [PATCH v5 3/9] pack-objects: factor out handling '--stdin-packs' Taylor Blau
2025-06-19 23:30   ` [PATCH v5 4/9] pack-objects: declare 'rev_info' for '--stdin-packs' earlier Taylor Blau
2025-06-19 23:30   ` [PATCH v5 5/9] pack-objects: perform name-hash traversal for unpacked objects Taylor Blau
2025-06-19 23:30   ` [PATCH v5 6/9] pack-objects: fix typo in 'show_object_pack_hint()' Taylor Blau
2025-06-19 23:30   ` [PATCH v5 7/9] pack-objects: swap 'show_{object,commit}_pack_hint' Taylor Blau
2025-06-19 23:30   ` [PATCH v5 8/9] pack-objects: introduce '--stdin-packs=follow' Taylor Blau
2025-06-20 15:27     ` Junio C Hamano
2025-06-19 23:30   ` [PATCH v5 9/9] repack: exclude cruft pack(s) from the MIDX where possible Taylor Blau
2025-06-21  4:35     ` Jeff King
2025-06-23 18:47       ` Taylor Blau
2025-06-24 10:54         ` Jeff King
2025-06-24 16:05           ` Taylor Blau
2025-06-23 22:32 ` Taylor Blau [this message]
2025-06-23 22:32   ` [PATCH v6 1/9] pack-objects: use standard option incompatibility functions Taylor Blau
2025-06-24 15:52     ` Junio C Hamano
2025-06-24 16:06       ` Taylor Blau
2025-06-23 22:32   ` [PATCH v6 2/9] pack-objects: limit scope in 'add_object_entry_from_pack()' Taylor Blau
2025-06-23 22:49     ` Junio C Hamano
2025-06-23 22:32   ` [PATCH v6 3/9] pack-objects: factor out handling '--stdin-packs' Taylor Blau
2025-06-23 22:32   ` [PATCH v6 4/9] pack-objects: declare 'rev_info' for '--stdin-packs' earlier Taylor Blau
2025-06-23 22:59     ` Junio C Hamano
2025-06-23 22:32   ` [PATCH v6 5/9] pack-objects: perform name-hash traversal for unpacked objects Taylor Blau
2025-06-23 23:08     ` Junio C Hamano
2025-06-24 16:08       ` Taylor Blau
2025-06-23 22:32   ` [PATCH v6 6/9] pack-objects: fix typo in 'show_object_pack_hint()' Taylor Blau
2025-06-23 22:32   ` [PATCH v6 7/9] pack-objects: swap 'show_{object,commit}_pack_hint' Taylor Blau
2025-06-23 22:32   ` [PATCH v6 8/9] pack-objects: introduce '--stdin-packs=follow' Taylor Blau
2025-06-23 23:35     ` Junio C Hamano
2025-06-24 16:10       ` Taylor Blau
2025-06-23 22:32   ` [PATCH v6 9/9] repack: exclude cruft pack(s) from the MIDX where possible Taylor Blau

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1750717921.git.me@ttaylorr.com \
    --to=me@ttaylorr.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=newren@gmail.com \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).