Re: Possible bug: identical lines added/removed in git diff

[Stefan Beller <sbeller@google.com>]

On Thu, Aug 30, 2018 at 12:20 PM Jeff King <peff@peff.net> wrote:
>
> On Thu, Aug 30, 2018 at 12:16:22PM -0700, Stefan Beller wrote:
>
> > On Wed, Aug 29, 2018 at 7:54 PM Jeff King <peff@peff.net> wrote:
> > >
> > > On Wed, Aug 29, 2018 at 10:10:25PM -0400, Gabriel Holodak wrote:
> > >
> > > > > Could you cut down to a real minimal reproduction, i.e. just these 20
> > > > > lines or so?
> > > >
> > > > I'm working on getting down to a minimal reproduction, a few lines at
> > > > a time. One thing that seems strange: as I've removed lines, there are
> > > > a bunch of lines that don't matter. Then I'll find some lines that, if
> > > > removed, completely fix the issue. But the ordering for these
> > > > apparently important lines doesn't matter. They just have to be
> > > > somewhere in the file to cause the duplicated diffs.
> > > >
> > > > I'll upload again when I've figured out all the unimportant lines to remove.
> > >
> > > Yeah, I reproduced based on your initial post, but noticed that when I
> > > cut it down the problem went away.
> >
> > Oh, I had to look further down than I did initially. Now I can reproduce it
> > from the initial data as well.
> >
> > Note that it goes away with --minimal.
>
> That's interesting. I did wonder if this was in fact a bug, or simply
> that Myers does not promise to find the absolute minimal diff. I'm
> _still_ not sure, especially because the minimization is so obvious in
> this case (literally the first "-" and the first "+" line of a
> contiguous hunk are identical).

The `Myers` (our default) diff algorithm is really the Myers algorithm +
a heuristic that cuts off the long tail when it is very costly to compute
the minimal diff.

The `minimal` diff is the true Myers algorithm and I'd vouch for its
correctness and being the minimal number of lines in the diff output.

The Myers is implemented before
https://github.com/git/git/blob/master/xdiff/xdiffi.c#L135
and the heuristics is after that line.

> > I have a patch cooking (which was sent out as
> > https://public-inbox.org/git/20180810221857.87399-1-sbeller@google.com/)
> >
> > and one of the weaknesses in that patch is the lack of explanation on
> > when the heuristic is applied as I have not fully understood it yet.
>
> I'm not sure I understand it either. But at least knowing that --minimal
> changes the output gives a lead for investigation (I don't really have
> time to dig into it in the next few days, though).

An interesting (to me) approach for digging into that would include
finding these examples at scale, which I presented in
https://public-inbox.org/git/20180810001010.58870-1-sbeller@google.com/
but I guess reading the code would work just as fine.

Thanks,
Stefan