Re: [PATCH 06/16] midx: support custom `--base` for incremental MIDX writes

[Taylor Blau <me@ttaylorr.com>]

On Tue, Apr 07, 2026 at 01:57:53AM -0400, Jeff King wrote:
> On Sun, Mar 29, 2026 at 05:41:06PM -0400, Taylor Blau wrote:
>
> > Introduce a new `--base` option for both subcommands to specify the
> > checksum of the MIDX layer to use as the base. The given checksum must
> > refer to a valid layer in the MIDX chain that is an ancestor of the
> > topmost layer being written or compacted.
>
> OK, that makes sense.
>
> > The special value "none" is accepted to produce a root layer with no
> > parent. This will be needed when the incremental repacking machinery
> > determines that the bottommost layers of the chain should be replaced.
>
> My gut feeling on reading is that a chain based on "none" is really just
> a non-incremental midx, so we could just write that. But:
>
>   1. I don't recall off-hand how expensive it is to convert a
>      non-incremental to an incremental one. I.e., can we just rename the
>      file and write out a chain link and it becomes incremental?
>
>   2. Even if it is cheap to convert, I can believe it makes geometric
>      repacking easier to think about if we just always think about it as
>      an incremental midx.
>
> And perhaps (3), I am just wrong that they are the same thing. ;) I
> haven't thought it through all that carefully.

In this case we are truly writing an incremental MIDX layer without any
parent. So it's logically similar from a non-incremental MIDX in the
sense that there is only a single layer, but it is still an incremental
MIDX nonetheless.

We have to write these as incremental layers, since we can't replace the
main $GIT_DIR/objects/pack/multi-pack-index, so that takes (1) off of
the table.

> > If no `--base` is given, behavior is unchanged: `compact` uses "from's"
> > immediate parent in the chain, and `write` appends to the existing tip.
> >
> > For the `write` subcommand, `--base` requires `--checksum-only`. A plain
> > `write --incremental` appends a new layer to the live chain tip with no
> > mechanism to atomically replace it; overriding the base would produce a
> > layer that does not extend the tip, breaking chain invariants. With
> > `--checksum-only` the chain is left unmodified and the caller is
> > responsible for assembling a valid chain.
>
> OK, makes sense. If we are using --base we are doing something clever
> and weird, and the chain tip should be left untouched. I guess "midx
> write" would never delete an incremental (it is always just adding one
> more to the chain), so we don't have to worry about that. Presumably
> we'll eventually write out the new chain tip from the caller, which will
> then have to remember to delete the now-orphaned chain elements. But
> that's kind of the point of the new geometric approach, so it seems like
> things are falling into place.

Yup, that is driven at the 'repack' layer and will appear later on in
this series.

Thanks,
Taylor