commit gone after merge - how to debug?

commit gone after merge - how to debug?

[Igor Lautar]

Hi,

This looks really weird and I cannot explain why it occurs.

Setup is as follows:
 - origin
 - mirror
 - local clone

Reference repository is origin from where builds are done etc.
Parallel to that, we keep a mirror that is synced manually
(fetch/merge/push).

I do this from my local clone (which is mostly just tracking origin
and mirror, no local branches).

What happened is that after a merge of mirror/master into local
master, a commit (that also exists on origin/master) is lost.

Lost as in:
pre-merge:
git log <file modified by commit>
 - commit shown in history
git merge mirror/master
 - no conflicts
git log <file modified by commit>
 - commit NOT shown in file history any more and file does not have this change

Doing git log shows commit as being present in repository history. One
interesting point is that one of the parents is previous merge commit
of same branches.

Unfortunately, I cannot open up repository for public access, but
would appreciate any pointers how to debug this.

git fsck finds some dangling blobs/commits, but no other
warnings/errors, I can clone repo just fine, everything seams in
order.

How can I debug what the merge is doing?

git version 1.7.12.1 on mac:
Darwin 12.2.0 Darwin Kernel Version 12.2.0: Sat Aug 25 00:48:52 PDT
2012; root:xnu-2050.18.24~1/RELEASE_X86_64 x86_64

Regards,
Igor

PS. please keep me in CC, I'm not on list

Re: commit gone after merge - how to debug?

[Tomas Carnecky]

On Mon, 26 Nov 2012 14:06:09 +0100, Igor Lautar <igor.lautar@gmail.com> wrote:
> git log <file modified by commit>
>  - commit NOT shown in file history any more and file does not have this change

does `git log --full-history <file modified by commit>` show the commit?

Re: commit gone after merge - how to debug?

[Matthieu Moy]

Tomas Carnecky <tomas.carnecky@gmail.com> writes:

> On Mon, 26 Nov 2012 14:06:09 +0100, Igor Lautar <igor.lautar@gmail.com> wrote:
>> git log <file modified by commit>
>>  - commit NOT shown in file history any more and file does not have this change
>
> does `git log --full-history <file modified by commit>` show the commit?

The other related question being: does reading the section "History
Simplification" in "man git-log" help? ;-)

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: commit gone after merge - how to debug?

[Igor Lautar]

On Mon, Nov 26, 2012 at 2:23 PM, Matthieu Moy
<Matthieu.Moy@grenoble-inp.fr> wrote:
> The other related question being: does reading the section "History
> Simplification" in "man git-log" help? ;-)

Somewhat, but it does not explain why the file no longer has that
change. I can understand omitting history if end result is the same,
but here it shouldn't be - I cannot find a commit that reversed that
change, so the change should still be in after the merge?

The file in question was not modified on mirror, nor was modified on
origin after that change.

Re: commit gone after merge - how to debug?

[Matthieu Moy]

Igor Lautar <igor.lautar@gmail.com> writes:

> On Mon, Nov 26, 2012 at 2:23 PM, Matthieu Moy
> <Matthieu.Moy@grenoble-inp.fr> wrote:
>> The other related question being: does reading the section "History
>> Simplification" in "man git-log" help? ;-)
>
> Somewhat, but it does not explain why the file no longer has that
> change.

It still has, but it's not shown by "git log <file>", probably because
one of the parent of the merge commit introduces no change for this
file, so one side of the merge is not needed to explain you how you went
from the origin of time to the last commit.

Try this:

commit=<sha1 of your merge commit>
# Show diff with first parent:
git diff "$commit" "$commit"^1
# Show diff with second parent:
git diff "$commit" "$commit"^2

> I can understand omitting history if end result is the same, but here
> it shouldn't be - I cannot find a commit that reversed that change, so
> the change should still be in after the merge?

revert is not the only situation that can lead to history
simplification. I'm no expert in the domain, but I think if you did the
same change in two branches, the merge will be candidate for history
simplification.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: commit gone after merge - how to debug?

[Igor Lautar]

On Mon, Nov 26, 2012 at 2:38 PM, Matthieu Moy
<Matthieu.Moy@grenoble-inp.fr> wrote:
> Igor Lautar <igor.lautar@gmail.com> writes:
>> Somewhat, but it does not explain why the file no longer has that
>> change.
>
> It still has, but it's not shown by "git log <file>", probably because
> one of the parent of the merge commit introduces no change for this
> file, so one side of the merge is not needed to explain you how you went
> from the origin of time to the last commit.

No, the change is not there. See below.

> Try this:
>
> commit=<sha1 of your merge commit>
> # Show diff with first parent:
> git diff "$commit" "$commit"^1
> # Show diff with second parent:
> git diff "$commit" "$commit"^2

Yes, change is shown in commit^2, but actual file after merge does not have it.

I've double and triple checked, it is just not there. In the end, I've
cherry-picked the same commit after the merge and change is applied.
If change would be there after the merge, cherry-pick would not have
anything to do (whole commit is a one line change in single file).

So its not that the history is hidden, the change *is* missing after the merge.

Is there anything else I can try to figure out why its missing (other
than actually debugging git code/scripts)? Like debug output for each
change being considered/merged in?

Regards,
Igor

Re: commit gone after merge - how to debug?

[Matthieu Moy]

Igor Lautar <igor.lautar@gmail.com> writes:

> Yes, change is shown in commit^2, but actual file after merge does not have it.

Your initial message was about the output of "git log". Do you mean that
the file, on the filesystem, does not have the line introduced by the
commit?

If so, check the content registered in the repository too:

git show <merge-commit>:<file-name>

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: commit gone after merge - how to debug?

[Igor Lautar]

On Mon, Nov 26, 2012 at 3:03 PM, Matthieu Moy
<Matthieu.Moy@grenoble-inp.fr> wrote:
> Your initial message was about the output of "git log". Do you mean that
> the file, on the filesystem, does not have the line introduced by the
> commit?

Yes, sorry if I was not clear enough.

> If so, check the content registered in the repository too:
>
> git show <merge-commit>:<file-name>

Content shown is identical to the one in working copy, ie., it is
missing one line change.

git annotate <file> <merge commit>
 - shows that particular line as if it has originated from when the
file was originally added to repo.
git annotate <file> <merge commit>^2
 - shows line as being modified by a commit done after file was added
- ie., state I would expect after a merge

Re: commit gone after merge - how to debug?

[Matthieu Moy]

Igor Lautar <igor.lautar@gmail.com> writes:

> git annotate <file> <merge commit>^2
>  - shows line as being modified by a commit done after file was added
> - ie., state I would expect after a merge

What about "git annotate <file> <merge-commit>^1"?

Was the merge completely automatic, or were there any conflict?

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: commit gone after merge - how to debug?

[Igor Lautar]

On Mon, Nov 26, 2012 at 3:19 PM, Matthieu Moy
<Matthieu.Moy@grenoble-inp.fr> wrote:
> What about "git annotate <file> <merge-commit>^1"?

No change, line version goes back to when file was added.

> Was the merge completely automatic, or were there any conflict?

No conflicts at all. In fact, that particular file was not touched by
one side of merge, only by another. It seems like git ignored the
change, but still recorded history (shown only with --full-history).

Re: commit gone after merge - how to debug?

[Matthieu Moy]

[ Jumping back in time ]

Igor Lautar <igor.lautar@gmail.com> writes:

>> Try this:
>>
>> commit=<sha1 of your merge commit>
>> # Show diff with first parent:
>> git diff "$commit" "$commit"^1
>> # Show diff with second parent:
>> git diff "$commit" "$commit"^2
>
> Yes, change is shown in commit^2, but actual file after merge does not have it.

My commands had the wrong order. It should have been git diff
"$commit"^2 "$commit". So, it showed the reverse of the modification
introduced by the commit. If you see your change here, it means the
change was reverted by the merge.

My understanding of the situation up to now is:

M
|\
A C
|/
B

($commit = M, $commit^1 = A and $commit^2 = B)

Your file had a content (say, "old") at revision B. It changed content
(say, to "new") at revision C, and at some point. A did not change it so
it had the content "old". Then you merged, expected the merge commit M
to get content "new", and actually got "old".

So, your history looks like:

M (old)
| \
|  `---.
|       \
A (old)  C (new)
|       /
|  .---'
| /
|/
B (old)

and "git diff C M" shows the diff between new and old.

Something went wrong during the merge, I guess it used an ancestor (B
above) that had "new" as content. I don't see how this happened (rather
clearly, your history is more complex than my example above), but
"GIT_MERGE_VERBOSITY=5 git merge" will show you which common ancestor
was used, it may help.

What's possible is that someone had already merged the branch containing
"new", got conflicts, and resolved it in favor of "old" somewhere in the
history of your master branch.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: commit gone after merge - how to debug?

[Igor Lautar]

On Mon, Nov 26, 2012 at 3:50 PM, Matthieu Moy
<Matthieu.Moy@grenoble-inp.fr> wrote:
> What's possible is that someone had already merged the branch containing
> "new", got conflicts, and resolved it in favor of "old" somewhere in the
> history of your master branch.

This is exactly what happened. I've actually found a merge of origin
to mirror which reversed the change some time back and was
subsequently merged back to origin later on. Most probably human error
during merge.

Interestingly, this was my first thought as well, but I've must have
overlooked that particular merge the first time.

Anyhow, it sorted now, many thanks for your help,
Igor

Re: commit gone after merge - how to debug?

[Igor Lautar]

On Mon, Nov 26, 2012 at 2:10 PM, Tomas Carnecky
<tomas.carnecky@gmail.com> wrote:
> On Mon, 26 Nov 2012 14:06:09 +0100, Igor Lautar <igor.lautar@gmail.com> wrote:
>> git log <file modified by commit>
>>  - commit NOT shown in file history any more and file does not have this change
>
> does `git log --full-history <file modified by commit>` show the commit?

Indeed it does.

Did the merge with verbosity set to 5. It says the commit I'm merging
in is virtual (probably as it is a merge commit in itself?).

Why would commit be left behind after merge? What kind of history
triggers this scenario?
Just trying to understand reasoning as its counter-intuitive to what I
know now. This may affect our workflow (ie., change it so we avoid it
happening).