Git development
 help / color / mirror / Atom feed
From: Patrick Steinhardt <ps@pks.im>
To: Jeff King <peff@peff.net>
Cc: Elijah Newren via GitGitGadget <gitgitgadget@gmail.com>,
	git@vger.kernel.org, Elijah Newren <newren@gmail.com>
Subject: Re: [PATCH 2/2] packfile: recover when a multi-pack-index names a removed pack
Date: Mon, 24 Aug 2026 07:40:22 +0200	[thread overview]
Message-ID: <aovZRjcIbAUqswFT@pks.im> (raw)
In-Reply-To: <20260824045529.GB142844@coredump.intra.peff.net>

On Mon, Aug 24, 2026 at 12:55:29AM -0400, Jeff King wrote:
> On Tue, Aug 18, 2026 at 10:34:06PM +0000, Elijah Newren via GitGitGadget wrote:
> 
> > Teach find_pack_entry() to recover.  After the normal multi-pack-index
> > lookup and the regular pack fallback both miss, check whether the object
> > is nonetheless present in a covered multi-pack-index (bsearch_midx()).
> > If it is, its recorded owner must have become unavailable, so scan that
> > index's packs directly for a surviving copy.  The bsearch gate keeps
> > genuine misses (i.e. objects absent from the index) on the fast path, and
> > because the recovery lives in find_pack_entry() itself it also fixes the
> > OBJECT_INFO_QUICK callers that never reprepare.
> 
> You don't even have to pay the bsearch() again. We'd already have looked
> in the midx earlier in the function. We just need to distinguish three
> cases:
> 
>   1. it was not in the midx (or there is no midx)
> 
>   2. it was in the midx but we could not load it (pack invalid, or
>      object in the bad_objects list)
> 
>   3. it was in the midx and is available
> 
> In fill_midx_entry() we return a boolean that lumps cases 1+2 together,
> versus case 3. It could return a tri-state that would let us distinguish
> all three. And then your fallback would kick in only for case 2 (case 3
> already returned with success, and case 1 means the midx does not even
> mention the object).
> 
> This is all assuming the fallback is worth pursuing. I'm still puzzled
> why this specific case would matter when we have the same (already
> solved) problem of reading a regular .idx whose .pack has gone away.

I've tried to clarify in a parallel message already, but the issue is
that we skip over any packfiles that covered by a MIDX when doing the
lookup. So any secondary packfiles that contain the object would be
completely ignored, and that's why we don't find the object there.

But this mail here suggests an alternative fix: instead of re-scanning
all packfiles like the patch proposes, wouldn't the proper fix be to not
ignore _all_ MIDX'd packs, but only the pack that _should_ have
contained the object?

Ultimately though, this would be equivalent to turning the function's
return value into a tri-state as suggested by Peff here. The only case
where the issue can occur is in case (2), and in that case we should not
skip MIDX'd packs at all as the MIDX'd pack that should've contained the
pack does not exist anyway.

Patrick

  reply	other threads:[~2026-08-24  5:40 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-18 22:34 [PATCH 0/2] Objects treated as missing despite being present, due to race with geometric repacking Elijah Newren via GitGitGadget
2026-08-18 22:34 ` [PATCH 1/2] replay: fail gracefully when a merge input is unreadable Elijah Newren via GitGitGadget
2026-08-19 18:09   ` Junio C Hamano
2026-08-21  1:44     ` Elijah Newren
2026-08-21  3:37       ` Junio C Hamano
2026-08-18 22:34 ` [PATCH 2/2] packfile: recover when a multi-pack-index names a removed pack Elijah Newren via GitGitGadget
2026-08-19 18:21   ` Junio C Hamano
2026-08-20  7:54   ` Patrick Steinhardt
2026-08-21  1:36     ` Elijah Newren
2026-08-24  4:48       ` Jeff King
2026-08-24  5:13         ` Patrick Steinhardt
2026-08-24  6:55           ` Jeff King
2026-08-24  7:06             ` Jeff King
2026-08-24  7:23               ` Jeff King
2026-08-25  7:38               ` Elijah Newren
2026-08-24  4:55   ` Jeff King
2026-08-24  5:40     ` Patrick Steinhardt [this message]
2026-08-24  7:03       ` Jeff King
2026-08-25  7:19     ` Elijah Newren
2026-08-24 14:45   ` Derrick Stolee
2026-08-25  7:38     ` Elijah Newren
2026-08-24 14:46   ` Derrick Stolee
2026-08-25 19:00 ` [PATCH v2 0/4] Objects treated as missing despite being present, due to race with geometric repacking Elijah Newren via GitGitGadget
2026-08-25 19:00   ` [PATCH v2 1/4] replay: fail gracefully when a merge input is unreadable Elijah Newren via GitGitGadget
2026-08-25 19:00   ` [PATCH v2 2/4] mktree: plug per-tree leak in --batch mode Elijah Newren via GitGitGadget
2026-08-25 19:00   ` [PATCH v2 3/4] packfile: recover object lookups racing a concurrent repack Elijah Newren via GitGitGadget
2026-08-25 19:00   ` [PATCH v2 4/4] packfile: recover when a multi-pack-index names a removed pack Elijah Newren via GitGitGadget

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aovZRjcIbAUqswFT@pks.im \
    --to=ps@pks.im \
    --cc=git@vger.kernel.org \
    --cc=gitgitgadget@gmail.com \
    --cc=newren@gmail.com \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox