Re: MIDX woes, was Re: [ANNOUNCE] Git v2.54.0-rc2

[Jeff King <peff@peff.net>]

On Wed, Apr 15, 2026 at 02:04:44PM -0700, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >>  * Further work on incremental repacking using MIDX/bitmap
> >
> > I just noticed that a fetch with v2.54.0-rc2 into an existing repository
> > rendered it unusable for Git v2.53.0:
> >
> >   fatal: multi-pack-index version 2 not recognized
> >
> > Is it possible that v2.54.0-rc2 forcefully uses a MIDX version that has
> > _just_ been introduced?
> >
> > If so, that might have been a premature bump of the default MIDX version,
> > if even the preceding Git version fails to handle that MIDX version. It is
> > guaranteed to cause substantial problems in setups where e.g. libgit2 or
> > JGit is expected to be used interchangeably with Git. It also causes
> > problems when users have to downgrade Git, or use several Git versions
> > side by side (e.g. using GitHub Desktop, which bundles its own version of
> > Git).
> 
> Is b2ec8e90 (midx: do not require packs to be sorted in
> lexicographic order, 2026-02-24), which says
> 
>     This change produces MIDXs which may not be correctly read with external
>     tools or older versions of Git. Though older versions of Git know how to
>     gracefully degrade and ignore any MIDX(s) they consider corrupt,
>     external tools may not be as robust. To avoid unintentionally breaking
>     any such tools, guard this change behind a version bump in the MIDX's
>     on-disk format.
>     
> relevant?  The version bump seems to be doing more harm to "older
> versions of Git" that "know how to gracefully degrade" by not
> allowing them to degrade.

Eek. The midx _should_ be an optional component, so returning an error
during load would then just look in the regular pack .idx files. But
there are a few die() calls in the loading function. :(

Something like the patch below on top of v2.53.0 gets the desired effect
(we print "version 2 not recognized to stderr" and then everything works
anyway). But of course we can't go back in time now to fix it (and
earlier versions).

I wonder how libgit2 and JGit handle this:

  - Looking in the source, JGit will throw an exception, which is
    presumably caught and handled OK (at least seems to using the "jgit"
    CLI I have handy). So that's good.

  - libgit2 seems to "return midx_error()" if the signature or version
    isn't matched. I don't have an easy tool built against it, but
    looking at the code I think it would quietly fall back to using the
    actual packs.

So it really is just our old versions that are the problem.

I think removing the .midx file (and optionally regenerating with the
old version) would be the appropriate workaround, but I wonder how hard
it would be to go back to generating v1 midx files by default. I know v2
is a building block for more advanced features, but for those who are
not using those features yet it is a strict regression.

-Peff

---
diff --git a/midx.c b/midx.c
index a75ea99a0d..79ce6a1e3b 100644
--- a/midx.c
+++ b/midx.c
@@ -134,22 +134,26 @@ static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *sou
 
 	CALLOC_ARRAY(m, 1);
 	m->data = midx_map;
 	m->data_len = midx_size;
 	m->source = source;
 
 	m->signature = get_be32(m->data);
-	if (m->signature != MIDX_SIGNATURE)
-		die(_("multi-pack-index signature 0x%08x does not match signature 0x%08x"),
+	if (m->signature != MIDX_SIGNATURE) {
+		error(_("multi-pack-index signature 0x%08x does not match signature 0x%08x"),
 		      m->signature, MIDX_SIGNATURE);
+		goto cleanup_fail;
+	}
 
 	m->version = m->data[MIDX_BYTE_FILE_VERSION];
-	if (m->version != MIDX_VERSION)
-		die(_("multi-pack-index version %d not recognized"),
+	if (m->version != MIDX_VERSION) {
+		error(_("multi-pack-index version %d not recognized"),
 		      m->version);
+		goto cleanup_fail;
+	}
 
 	hash_version = m->data[MIDX_BYTE_HASH_VERSION];
 	if (hash_version != oid_version(r->hash_algo)) {
 		error(_("multi-pack-index hash version %u does not match version %u"),
 		      hash_version, oid_version(r->hash_algo));
 		goto cleanup_fail;
 	}