Git development
 help / color / mirror / Atom feed
From: Jeff King <peff@peff.net>
To: Taylor Blau <ttaylorr@openai.com>
Cc: git@vger.kernel.org, Junio C Hamano <gitster@pobox.com>,
	Ted Nyman <tnyman@openai.com>, Elijah Newren <newren@github.com>
Subject: Re: [PATCH 4/4] repack: retain cruft packs in MIDXs containing kept packs
Date: Wed, 30 Sep 2026 16:53:11 -0400	[thread overview]
Message-ID: <20260930205311.GC747209@coredump.intra.peff.net> (raw)
In-Reply-To: <e942c256334e4de31ec0a1cb2d5f8c7465d8696f.1790731662.git.me@ttaylorr.com>

On Tue, Sep 29, 2026 at 08:28:58PM -0500, Taylor Blau wrote:

> When performing a geometric repack with 'repack.midxMustContainCruft'
> set to "false", Git uses '--stdin-packs=follow' to copy (once-cruft)
> objects needed for reachability closure out of cruft packs. .keep packs
> do not need to participate in that walk, though they *are* included in
> the resulting MIDX.
> 
> A .keep pack can contain a commit that reaches an object whose only copy
> is in a cruft pack. When there is no previous MIDX and the repack writes
> a new pack, neither `midx_has_unknown_packs()` nor the `!names.nr`
> fallback require that cruft pack to be included. If the kept commit (or
> a descendant of it) is selected for bitmap coverage, the bitmap writer
> fails because the MIDX does not contain all of its reachable objects.
> 
> Include cruft packs whenever the MIDX contains kept packs. This also
> retains cruft when the kept packs happen to have full closure, or when
> '--pack-kept-objects' lets the repack walk them. It avoids having to
> establish their closure before deciding which packs the MIDX needs.

OK. This makes sense to me, but two questions:

  1. Is this going to kick in racily because of the .keep that we
     temporarily install during pushes? That could cause unexpected
     performance changes in a big repo when the midx sometimes has to
     randomly include cruft packs.

  2. I'd have thought that the solution would be to treat .keep packs
     like other included follow-packs: traverse them in the usual way.
     But maybe there are good reasons we didn't do that in the first
     place.

-Peff

  reply	other threads:[~2026-09-30 20:53 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-30  1:28 [PATCH 0/4] repack: various corner cases for cruft-less MIDXs Taylor Blau
2026-09-30  1:28 ` [PATCH 1/4] pack-objects: introduce `stdin_packs_context` struct Taylor Blau
2026-09-30 17:42   ` Junio C Hamano
2026-10-01  3:13     ` Taylor Blau
2026-09-30  1:28 ` [PATCH 2/4] pack-objects: ensure tree/tag closure with '--stdin-packs=follow' Taylor Blau
2026-09-30 17:51   ` Junio C Hamano
2026-09-30 18:16   ` Derrick Stolee
2026-10-01  3:14     ` Taylor Blau
2026-10-01 23:22     ` Elijah Newren
2026-10-02  0:51       ` Taylor Blau
2026-10-02 23:02         ` Jeff King
2026-09-30 20:31   ` Jeff King
2026-10-01  3:18     ` Taylor Blau
2026-09-30  1:28 ` [PATCH 3/4] repack: retain cruft packs in MIDXs after incremental repacks Taylor Blau
2026-09-30 20:45   ` Jeff King
2026-10-01  3:21     ` Taylor Blau
2026-09-30  1:28 ` [PATCH 4/4] repack: retain cruft packs in MIDXs containing kept packs Taylor Blau
2026-09-30 20:53   ` Jeff King [this message]
2026-10-01  3:35     ` Taylor Blau
2026-09-30 20:55 ` [PATCH 0/4] repack: various corner cases for cruft-less MIDXs Jeff King
2026-10-01  3:37   ` Taylor Blau
2026-10-01  4:11 ` [PATCH v2 0/8] " Taylor Blau
2026-10-01  4:11   ` [PATCH v2 1/8] pack-objects: introduce `stdin_packs_context` struct Taylor Blau
2026-10-01  4:11   ` [PATCH v2 2/8] pack-objects: ensure tree/tag closure with '--stdin-packs=follow' Taylor Blau
2026-10-02 23:13     ` Jeff King
2026-10-03  0:55       ` Taylor Blau
2026-10-03  1:06         ` Jeff King
2026-10-01  4:11   ` [PATCH v2 3/8] repack: retain cruft packs in MIDXs after incremental repacks Taylor Blau
2026-10-01  4:11   ` [PATCH v2 4/8] repack: use a sorted list for explicitly kept packs Taylor Blau
2026-10-02 23:16     ` Jeff King
2026-10-01  4:11   ` [PATCH v2 5/8] repack: follow kept packs when omitting cruft from the MIDX Taylor Blau
2026-10-02 23:25     ` Jeff King
2026-10-03  0:50       ` Taylor Blau
2026-10-01  4:11   ` [PATCH v2 6/8] repack: track the preferred pack explicitly in MIDX write steps Taylor Blau
2026-10-02 23:28     ` Jeff King
2026-10-03  1:00       ` Taylor Blau
2026-10-03  1:07         ` Jeff King
2026-10-01  4:12   ` [PATCH v2 7/8] repack: defer allocating the append plan's write step Taylor Blau
2026-10-01  4:12   ` [PATCH v2 8/8] repack: include required packs in incremental MIDX writes Taylor Blau
2026-10-02 23:41     ` Jeff King
2026-10-03  1:01       ` Taylor Blau

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260930205311.GC747209@coredump.intra.peff.net \
    --to=peff@peff.net \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=newren@github.com \
    --cc=tnyman@openai.com \
    --cc=ttaylorr@openai.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox