Re: [PATCH 06/16] midx: support custom `--base` for incremental MIDX writes

[Jeff King <peff@peff.net>]

On Sun, Mar 29, 2026 at 05:41:06PM -0400, Taylor Blau wrote:

> Introduce a new `--base` option for both subcommands to specify the
> checksum of the MIDX layer to use as the base. The given checksum must
> refer to a valid layer in the MIDX chain that is an ancestor of the
> topmost layer being written or compacted.

OK, that makes sense.

> The special value "none" is accepted to produce a root layer with no
> parent. This will be needed when the incremental repacking machinery
> determines that the bottommost layers of the chain should be replaced.

My gut feeling on reading is that a chain based on "none" is really just
a non-incremental midx, so we could just write that. But:

  1. I don't recall off-hand how expensive it is to convert a
     non-incremental to an incremental one. I.e., can we just rename the
     file and write out a chain link and it becomes incremental?

  2. Even if it is cheap to convert, I can believe it makes geometric
     repacking easier to think about if we just always think about it as
     an incremental midx.

And perhaps (3), I am just wrong that they are the same thing. ;) I
haven't thought it through all that carefully.

> If no `--base` is given, behavior is unchanged: `compact` uses "from's"
> immediate parent in the chain, and `write` appends to the existing tip.
> 
> For the `write` subcommand, `--base` requires `--checksum-only`. A plain
> `write --incremental` appends a new layer to the live chain tip with no
> mechanism to atomically replace it; overriding the base would produce a
> layer that does not extend the tip, breaking chain invariants. With
> `--checksum-only` the chain is left unmodified and the caller is
> responsible for assembling a valid chain.

OK, makes sense. If we are using --base we are doing something clever
and weird, and the chain tip should be left untouched. I guess "midx
write" would never delete an incremental (it is always just adding one
more to the chain), so we don't have to worry about that. Presumably
we'll eventually write out the new chain tip from the caller, which will
then have to remember to delete the now-orphaned chain elements. But
that's kind of the point of the new geometric approach, so it seems like
things are falling into place.

> For `compact`, no such restriction applies. The compaction operation
> atomically replaces the compacted range in the chain file, so writing
> the result on top of any valid ancestor preserves chain invariants.

OK, so there's an implicit base..tip compaction here (as opposed to
without --base, an implicit root..trip compaction?). Makes sense.

> @@ -83,6 +85,13 @@ marker).
>  		and packs not present in an existing MIDX layer.
>  		Migrates non-incremental MIDXs to incremental ones when
>  		necessary.
> +
> +	--base=<checksum>::
> +		Specify the checksum of an existing MIDX layer to use
> +		as the base when writing a new incremental layer.
> +		The special value `none` indicates that the new layer
> +		should have no base (i.e., it becomes a root layer).
> +		Requires `--checksum-only`.

Not really a critique of this patch, but I'd guess that a lot of these
options are getting esoteric enough that it is hard to understand 

> [...]

The patch looked as advertised, and I didn't find anything in it to
comment on.

-Peff