Re: Commit graph chains with no corresponding files?

[Bryan Turner <bturner@atlassian.com>]

On Thu, Feb 25, 2021 at 6:20 AM Derrick Stolee <stolee@gmail.com> wrote:
>
> On 2/24/2021 11:54 PM, Bryan Turner wrote:
> > On Mon, Jun 29, 2020 at 6:51 PM Derrick Stolee <stolee@gmail.com> wrote:
> >>
> >> On 6/29/2020 6:07 PM, Jonathan Tan wrote:
> >>> At $DAYJOB, a few people have reported "warning: unable to find all
> >>> commit-graph files" warnings. Their commit-graph-chain files have a few
> >>> lines, but they only have one commit graph file with very few commits. I
> >>> suspected something happening during fetch, because (as far as I know) a
> >>> fetch may cause an incremental commit graph to be written, but I ran a
> >>> fetch on a large repository myself and didn't run into this problem.
> >>>
> >>> Has anyone ran into this problem before, and know how to reproduce?
> >
> > I don't have any specific reproduction steps, but we've just run into
> > our first case of this on Git 2.29. I ended up kicking off a full `git
> > commit-graph write` to fix it. That displayed the same warning, but
> > commands run after it no longer do. Prior to writing the new graph, I
> > had this:
> > $ ls
> > commit-graph-chain  graph-88f5fe6e0c659e3742e556982263813d528ead81.graph
>
> The contents of the 'commit-graph-chain' file are critical to diagnosing
> the problems here. Likely it had multiple lines.
>
> > Afterward, the `objects/info/commits-graphs` directory still exists
> > but is empty, and there is now an `objects/commit-graph` that didn't
> > exist before. `git commit-graph verify` seems happy with the state of
> > things.
>
> Yes, a full rewrite without "--split" will get you to this state.
>
> >> The incremental commit-graph code deletes any commit-graph files
> >> that do not appear in the chain. I believe this is done by comparing
> >> the contents of the ".git/objects/info/commit-graphs/" directory to
> >> the contents of the chain file.
> >>
> >> These appear to be case-sensitive, full-path comparisons.
> >>
> >> It is _possible_ that something like a case switch or a symlink
> >> could be causing a problem here. That's where I would look on
> >> the affected systems.
> >
> > Are commit graphs potentially problematic in repositories that are
> > borrowing objects from other repositories via alternates?
>
> This was definitely part of the design, with the intention of
> working with a common base in the alternate. However, if the
> alternate collapses layers, then the repo that is borrowing
> from that alternate may have a broken chain.

Thanks for the analysis, Derrick. This seems like a likely culprit for
how the repository got into this state, because it is a fork (of a
fork) and does use a series of alternates.

>
> It is likely a better setup to have the alternate keep a
> commit-graph file and leave the dependent repos clear of a
> commit-graph. _Or_ the dependent repos should use a full
> commit-graph instead of a chain.

Skipping writing the commit graph in forks seems like a reasonable
place for us to start, given the way it currently works, but always
writing full graphs may be another option. If the fork is able to
borrow the commit graph from its origin across the alternate, though,
then that implies there may not be a lot of value in writing commit
graphs in the forks (since they're likely to share the majority of
their refs with their origin).

>
> If you have a better idea for how to make this work, then there
> is room for improvement.
>
> For example, if we ensure during the commit-graph write that
> all layers of the chain are within our local repo, then these
> dependency issues go away without breaking any old Git versions
> that are reading the data.

Naively, this was the way I assumed it already worked--which is why I
was writing commit graphs in forks in the first place.

>
> > Have there
> > been important changes to commit graphs since 2.29?
>
> Not in the area of commit-graph chains.

Thanks again!
-b

>
> Thanks,
> -Stolee