baffled again

baffled again

[tony.luck]

So I have another anomaly in my GIT tree.  A patch to
back out a bogus change to arch/ia64/hp/sim/boot/bootloader.c
in my release branch at commit

 62d75f3753647656323b0365faa43fc1a8f7be97

appears to have been lost when I merged the release branch to
the test branch at commit

 0c3e091838f02c537ccab3b6e8180091080f7df2

So now this file still has this:

/* SSC_WAIT_COMPLETION appears to want this large alignment.  gcc < 4
* seems to give it by default, however gcc > 4 is smarter and may
* not.
*/
struct disk_stat {
	int fd;
	unsigned count;
} __attribute__ ((aligned (16)));

in the test branch, when I think the comment and __attribute__
should have been backout out.

-Tony

Tree is at rsync://rsync.kernel.org/pub/scm/linux/kernel/git/aegl/linux-2.6.git

Re: baffled again

[Tony Luck]

I'm at home, and too lazy to log in to work to look at my tree.  But I
have a theory
as to what went wrong for me.

At the start I had a file, same contents in test and release branch.

I applied a patch to release, and pulled to test.  So the contents are still
the same, both with the patch applied.

Next, I was given a better patch (the first one just masked the real problem
and happened to make the symptoms go away). This patch touches a
completely different file.  So I applied a patch to revert the change
in release,
and the new patch.

Now ... when I try to merge release into test, my guess is that GIT is
looking at the common ancestor before I touched anything.  So when
it compares the current state of this file it sees that I have the bad patch
in the test tree, and the release tree has the "original" version (which has
had the patch applied and reverted ... so the contents are back at the
original state).

So GIT decides that the test branch has had a patch, and the release
branch hasn't ... and so it merges by keeping the version in test.

Plausible?

-Tony

Re: baffled again

[Linus Torvalds]

On Tue, 23 Aug 2005, Tony Luck wrote:
> 
> So GIT decides that the test branch has had a patch, and the release
> branch hasn't ... and so it merges by keeping the version in test.
> 
> Plausible?

Very. Sounds like what happened.

		Linus

Re: baffled again

[Junio C Hamano]

tony.luck@intel.com writes:

> So I have another anomaly in my GIT tree.  A patch to
> back out a bogus change to arch/ia64/hp/sim/boot/bootloader.c
> in my release branch at commit
>
>  62d75f3753647656323b0365faa43fc1a8f7be97
>
> appears to have been lost when I merged the release branch to
> the test branch at commit
>
>  0c3e091838f02c537ccab3b6e8180091080f7df2

    : siamese; git cat-file commit 0c3e091838f02c537ccab3b6e8180091080f7df2 
    tree 61a407356d1e897e0badea552ce69e657cab6108
    parent 7ffacc1a2527c219b834fe226a7a55dc67ca3637
    parent a4cce10492358b33d33bb43f98284c80482037e8
    author Tony Luck <tony.luck@intel.com> 1124808655 -0700
    committer Tony Luck <tony.luck@intel.com> 1124808655 -0700

    Pull release into test branch

So I pulled 7ffacc and a4cce1 from your repository and started
digging from there.  7ffacc was the head of "test" branch back
then, and a4cce1 was the head of "release" branch.  I checked
out 7ffacc in the repository and pulled a4cce1 into it, using
the GIT with the "optimum merge-base" patch.

    : siamese; git pull . aegl-release
    Packing 0 objects
    Unpacking 0 objects

    * committish: a4cce10492358b33d33bb43f98284c80482037e8	refs/heads/aegl-release from .
    Trying to find the optimum merge base.
    Trying to merge a4cce10492358b33d33bb43f98284c80482037e8 into 7ffacc1a2527c219b834fe226a7a55dc67ca3637 using c1ffb910f7a4e1e79d462bb359067d97ad1a8a25.
    Simple merge failed, trying Automatic merge
    Auto-merging arch/ia64/sn/kernel/io_init.c.
    Committed merge db376974c0aebb9e99e5cd0bce21088c6a9d927c
     arch/ia64/hp/sim/boot/boot_head.S |    2 +-
     1 files changed, 1 insertions(+), 1 deletions(-)

It is using c1ffb9 as the merge base.  The problematic path
in the three trees involved are:

: siamese; git ls-tree -r aegl-test-7ffacc1a | grep arch/ia64/hp/sim/boot/bootloader.c
100644 blob a7bed60b69f9e8de9a49944e22d03fb388ae93c7	arch/ia64/hp/sim/boot/bootloader.c
: siamese; git ls-tree -r aegl-release-a4cce1 | grep arch/ia64/hp/sim/boot/bootloader.c
100644 blob 51a7b7b4dd0e7c5720683a40637cdb79a31ec4c4	arch/ia64/hp/sim/boot/bootloader.c
: siamese; git ls-tree -r aegl-c1ffb9 | grep arch/ia64/hp/sim/boot/bootloader.c
100644 blob 51a7b7b4dd0e7c5720683a40637cdb79a31ec4c4	arch/ia64/hp/sim/boot/bootloader.c

So the file did not change between the merge base and release,
and test had the change.  merge-cache picked the one in the test
release.  Your guess in the other message hits the mark.

I wonder what _other_ candidates these two commits have in
common and what would have happened if they were used as the
base instead?

    : siamese; git merge-base -a aegl-test-7ffacc1a aegl-release-a4cce1
    f6fdd7d9c273bb2a20ab467cb57067494f932fa3
    3a931d4cca1b6dabe1085cc04e909575df9219ae
    c1ffb910f7a4e1e79d462bb359067d97ad1a8a25

You can check what variant of the file each of these commits
contain.  What is happening is:

* the problematic patch 4aec0f is one before 3a931d.  Among the
  three merge-base candidates, only 3a931d contains teh wrongly
  patched version.

* the problematic change 4aec0f patch introduces is part of test
  branch, because it was pulled via release.

* the tip of release being merged into test has this patch
  reverted, and the file is exactly the same as before 4aec0f
  patch.

So three-way trivial merge algorithm says, "hey, the file did
not change between common ancestor and release but it is
different in test, so the one in the test branch must be the
merge result."

This does not have much to do with which common ancestor
merge-base chooses.  Sorry, I am not sure what is the right way
to resolve this offhand.

Re: baffled again

[Daniel Barkalow]

On Wed, 24 Aug 2005, Junio C Hamano wrote:

> tony.luck@intel.com writes:
>
> > So I have another anomaly in my GIT tree.  A patch to
> > back out a bogus change to arch/ia64/hp/sim/boot/bootloader.c
> > in my release branch at commit
> >
> >  62d75f3753647656323b0365faa43fc1a8f7be97
> >
> > appears to have been lost when I merged the release branch to
> > the test branch at commit
> >
> >  0c3e091838f02c537ccab3b6e8180091080f7df2
>
>     : siamese; git cat-file commit 0c3e091838f02c537ccab3b6e8180091080f7df2
>     tree 61a407356d1e897e0badea552ce69e657cab6108
>     parent 7ffacc1a2527c219b834fe226a7a55dc67ca3637
>     parent a4cce10492358b33d33bb43f98284c80482037e8
>     author Tony Luck <tony.luck@intel.com> 1124808655 -0700
>     committer Tony Luck <tony.luck@intel.com> 1124808655 -0700
>
>     Pull release into test branch
>
> So I pulled 7ffacc and a4cce1 from your repository and started
> digging from there.  7ffacc was the head of "test" branch back
> then, and a4cce1 was the head of "release" branch.  I checked
> out 7ffacc in the repository and pulled a4cce1 into it, using
> the GIT with the "optimum merge-base" patch.
>
>     : siamese; git pull . aegl-release
>     Packing 0 objects
>     Unpacking 0 objects
>
>     * committish: a4cce10492358b33d33bb43f98284c80482037e8	refs/heads/aegl-release from .
>     Trying to find the optimum merge base.
>     Trying to merge a4cce10492358b33d33bb43f98284c80482037e8 into 7ffacc1a2527c219b834fe226a7a55dc67ca3637 using c1ffb910f7a4e1e79d462bb359067d97ad1a8a25.
>     Simple merge failed, trying Automatic merge
>     Auto-merging arch/ia64/sn/kernel/io_init.c.
>     Committed merge db376974c0aebb9e99e5cd0bce21088c6a9d927c
>      arch/ia64/hp/sim/boot/boot_head.S |    2 +-
>      1 files changed, 1 insertions(+), 1 deletions(-)
>
> It is using c1ffb9 as the merge base.  The problematic path
> in the three trees involved are:
>
> : siamese; git ls-tree -r aegl-test-7ffacc1a | grep arch/ia64/hp/sim/boot/bootloader.c
> 100644 blob a7bed60b69f9e8de9a49944e22d03fb388ae93c7	arch/ia64/hp/sim/boot/bootloader.c
> : siamese; git ls-tree -r aegl-release-a4cce1 | grep arch/ia64/hp/sim/boot/bootloader.c
> 100644 blob 51a7b7b4dd0e7c5720683a40637cdb79a31ec4c4	arch/ia64/hp/sim/boot/bootloader.c
> : siamese; git ls-tree -r aegl-c1ffb9 | grep arch/ia64/hp/sim/boot/bootloader.c
> 100644 blob 51a7b7b4dd0e7c5720683a40637cdb79a31ec4c4	arch/ia64/hp/sim/boot/bootloader.c
>
> So the file did not change between the merge base and release,
> and test had the change.  merge-cache picked the one in the test
> release.  Your guess in the other message hits the mark.
>
> I wonder what _other_ candidates these two commits have in
> common and what would have happened if they were used as the
> base instead?
>
>     : siamese; git merge-base -a aegl-test-7ffacc1a aegl-release-a4cce1
>     f6fdd7d9c273bb2a20ab467cb57067494f932fa3
>     3a931d4cca1b6dabe1085cc04e909575df9219ae
>     c1ffb910f7a4e1e79d462bb359067d97ad1a8a25
>
> You can check what variant of the file each of these commits
> contain.
>
> What is happening is:
>
> * the problematic patch 4aec0f is one before 3a931d.  Among the
>   three merge-base candidates, only 3a931d contains teh wrongly
>   patched version.
>
> * the problematic change 4aec0f patch introduces is part of test
>   branch, because it was pulled via release.
>
> * the tip of release being merged into test has this patch
>   reverted, and the file is exactly the same as before 4aec0f
>   patch.
>
> So three-way trivial merge algorithm says, "hey, the file did
> not change between common ancestor and release but it is
> different in test, so the one in the test branch must be the
> merge result."
>
> This does not have much to do with which common ancestor
> merge-base chooses.  Sorry, I am not sure what is the right way
> to resolve this offhand.

If it picks 3a931d4cca1b6dabe1085cc04e909575df9219ae, it will determine
that the file didn't change between that and test, and is different in
release, so the one in release must be right. I believe that the hint that
something is going on is that different common ancestors give
different trivial merges (as opposed to some giving failure and some
giving the same result), and resolving it probably involves identifying
that that paths from f6f... and c1f... to release don't keep the same blob
through the middle, despite having the same ends.

	-Daniel
*This .sig left intentionally blank*

Re: baffled again

[Linus Torvalds]

On Wed, 24 Aug 2005, Junio C Hamano wrote:
> 
> This does not have much to do with which common ancestor
> merge-base chooses.  Sorry, I am not sure what is the right way
> to resolve this offhand.

I think git did the "right thing", it just happened to be the thing that
Tony didn't want. Which makes it the "wrong thing", of course, but from a
purely technical standpoint, I don't think there's anything really wrong
with the merge. 

Basically, he had two branches, A and B, and both contained the same patch
(but _not_ the same commit). One undid it, the other did not.  There's no
real way to say which one is "correct", and both cases clearly merge
perfectly, so both outcomes "patch applied" and "patch reverted" are
really equally valid.

Now, if the shared patch hadn't been a patch, but a shared _commit_, then 
the thing would have been unambiguous - the shared commit would have been 
the merge point, and the revert would have clearly undone that shared 
commit.

What does this all mean? It just means that merging doesn't necessarily
even _have_ "one right answer". Automatic merges can be dangerous. The git
"global three-way" merge (global because it bases it's original state on
_global_ history, rather than local one) is about as safe as it gets (*), 
but even it can have these ambigious cases that it resolves automatically, 
and not the way you wanted it to.

		Linus

(*) "safe as it gets" of course also means "potentially really annoying",
since it tends to require manual fixups for any even possibly half-way
ambiguous case.

Re: baffled again

[Linus Torvalds]

On Wed, 24 Aug 2005, Linus Torvalds wrote:
> 
> Basically, he had two branches, A and B, and both contained the same patch
> (but _not_ the same commit). One undid it, the other did not.  There's no
> real way to say which one is "correct", and both cases clearly merge
> perfectly, so both outcomes "patch applied" and "patch reverted" are
> really equally valid.

In fact, the case that git selected ("patch applied"), is not only the one
that is very fundamentally the one git will always select in this kind of
situation - in some respects is actually the nicer choice of the two.

While it may cause problems (ie the revert was the right thing to do),
it's at least the state that is less likely to be "lost". Having a revert
disappear is likely better than having a real change disappear. The
reaction to the reverted code showing up again is likely "damn, won't that
bug ever go away, I fixed it once already" - but at least people will see 
that it's fair: "it was applied twice, so let's revert it twice".

In contrast, the reaction to a patch going away is likely just very
confusing: you have two people applying it, but only one reverting it will
revert both, while the first person who applied it may never have realized 
it got reverted.

		Linus

Re: baffled again

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> In fact, the case that git selected ("patch applied"), is not only the one
> that is very fundamentally the one git will always select in this kind of
> situation - in some respects is actually the nicer choice of the two.

While I appreciate the excuse for not taking immediate and hasty
action, I have two problems with your analysis.

 * I am not yet convinced that it is _not_ by accident that git
   ended up choosing the nicer choice of the two.

 * Even if it does always choose the nicer choice of the two,
   Tony was lucky (no pun intended).  Rather, we were lucky that
   Tony was observant.  A careless merger may well have easily
   missed this mismerge (from the human point of view).

Re: baffled again

[Tony Luck]

>  * Even if it does always choose the nicer choice of the two,
>    Tony was lucky (no pun intended).  Rather, we were lucky that
>    Tony was observant.  A careless merger may well have easily
>    missed this mismerge (from the human point of view).

Actually I can't take credit here. This was a case of the "many-eyes" of
open source working at its finest ... someone e-mailed me and told me
that I should have backed out the old patch before applying the new one.
While typing the e-mail to say that I already had in the release branch,
I found the problem that it had been "lost" in the merge into the test branch.

But this is a good reminder that merging is not a precise science, and
there is more than one plausible merge in many situations ... and while
GIT will pick the one that you want far more often than not, there is
the possibility that it will surprise you.  Maybe there should be a note
to this effect in the tutorial.  Git is not magic, nor is it imbued with
DWIM technology.

-Tony

Re: baffled again

[Horst von Brand]

Tony Luck <tony.luck@gmail.com> wrote:
> >  * Even if it does always choose the nicer choice of the two,
> >    Tony was lucky (no pun intended).  Rather, we were lucky that
> >    Tony was observant.  A careless merger may well have easily
> >    missed this mismerge (from the human point of view).

> Actually I can't take credit here. This was a case of the "many-eyes" of
> open source working at its finest ... someone e-mailed me and told me
> that I should have backed out the old patch before applying the new one.
> While typing the e-mail to say that I already had in the release branch,
> I found the problem that it had been "lost" in the merge into the test
> branch.

> But this is a good reminder that merging is not a precise science, and
> there is more than one plausible merge in many situations ... and while
> GIT will pick the one that you want far more often than not, there is
> the possibility that it will surprise you.  Maybe there should be a note
> to this effect in the tutorial.  Git is not magic, nor is it imbued with
> DWIM technology.

I have to disagree. If in some corner case it can do the wrong thing, no
amount of "I told you so in the docu!" will save the day. People /will/
overlook it, or be bitten when they have all but forgotten about it.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: baffled again

[Daniel Barkalow]

On Wed, 24 Aug 2005, Linus Torvalds wrote:

> Now, if the shared patch hadn't been a patch, but a shared _commit_, then
> the thing would have been unambiguous - the shared commit would have been
> the merge point, and the revert would have clearly undone that shared
> commit.

Actually, it was a shared commit
(4aec0fb12267718c750475f3404337ad13caa8f5), which was (an ancestor of) a
candidate merge point, but wasn't the one selected. Since a different one
was chosen, it looked to the 3-way merge like a shared patch (since it
ignores the untaken parent in the merges in the history).

This should be fixable, but it'll require more cleverness in read-tree.

	-Daniel
*This .sig left intentionally blank*

RE: baffled again

[Luck, Tony]

>I think git did the "right thing", it just happened to be the thing that
>Tony didn't want. Which makes it the "wrong thing", of course, but from a
>purely technical standpoint, I don't think there's anything really wrong
>with the merge. 

On the plus side ... at least it wasn't a dumb user error this time [unless
you count merging the incorrect patch in the first place, and then having
to revert it :-) ].

Could GIT be smarter here?  Perhaps it could pick a few likely
ancestors and run the merge with each ... and then give some
warnings if there are files that come out differently?

-Tony