Why can't I use git-bisect to find the first *good* commit?

Why can't I use git-bisect to find the first *good* commit?

[Ævar Arnfjörð Bjarmason]

Something was broken a 100 revisions ago, has now been fixed, but I
want to find when it was fixed.

I'd expect this to work:

    $ git bisect start
    $ git bisect good
    $ git bisect bad HEAD~100
    Some good revs are not ancestor of the bad rev.
    git bisect cannot work properly in this case.
    Maybe you mistake good and bad revs?

But instead I have to do:

    $ git bisect start
    $ git bisect bad
    $ git bisect good HEAD~100

And then proceed to mark good revisions as bad, and bad revisions as
good.

That works, but it's very confusing.

Why can't bisect just do the right thing here and accept that your
more recent revesion is the good one, and the old one is the bad one?

Re: Why can't I use git-bisect to find the first *good* commit?

[Andreas Ericsson]

On 03/28/2011 11:32 AM, Ævar Arnfjörð Bjarmason wrote:
> Something was broken a 100 revisions ago, has now been fixed, but I
> want to find when it was fixed.
> 
> I'd expect this to work:
> 
>      $ git bisect start
>      $ git bisect good
>      $ git bisect bad HEAD~100
>      Some good revs are not ancestor of the bad rev.
>      git bisect cannot work properly in this case.
>      Maybe you mistake good and bad revs?
> 
> But instead I have to do:
> 
>      $ git bisect start
>      $ git bisect bad
>      $ git bisect good HEAD~100
> 
> And then proceed to mark good revisions as bad, and bad revisions as
> good.
> 
> That works, but it's very confusing.
> 
> Why can't bisect just do the right thing here and accept that your
> more recent revesion is the good one, and the old one is the bad one?

It's due to the fact that bisect is, in 99% of the cases, used to
locate the commit that introduced a bug and the implementation details
naturally gear towards that scenario, with fixed names that do the
Right Thing(tm) in 99% of all cases.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.

Re: Why can't I use git-bisect to find the first *good* commit?

[code.sculptor]

That's a really good point! Perhaps we should have a --invert flag?
Sent from my BlackBerry device on the Rogers Wireless Network

-----Original Message-----
From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Sender: git-owner@vger.kernel.org
Date: Mon, 28 Mar 2011 11:32:21 
To: Git Mailing List<git@vger.kernel.org>
Subject: Why can't I use git-bisect to find the first *good* commit?

Something was broken a 100 revisions ago, has now been fixed, but I
want to find when it was fixed.

I'd expect this to work:

    $ git bisect start
    $ git bisect good
    $ git bisect bad HEAD~100
    Some good revs are not ancestor of the bad rev.
    git bisect cannot work properly in this case.
    Maybe you mistake good and bad revs?

But instead I have to do:

    $ git bisect start
    $ git bisect bad
    $ git bisect good HEAD~100

And then proceed to mark good revisions as bad, and bad revisions as
good.

That works, but it's very confusing.

Why can't bisect just do the right thing here and accept that your
more recent revesion is the good one, and the old one is the bad one?
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Why can't I use git-bisect to find the first *good* commit?

[Matthieu Moy]

code.sculptor@gmail.com writes:

> That's a really good point! Perhaps we should have a --invert flag?

IIRC, bzr uses "yes" and "no" instead of "good" and "bad" for this
reason: "yes/no the code works" or "yes/no the code is broken" depending
on what you're looking for.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: Why can't I use git-bisect to find the first *good* commit?

[Vincent van Ravesteijn]

> Why can't bisect just do the right thing here and accept that your
> more recent revesion is the good one, and the old one is the bad one?

There was a recent discussion about this:

http://article.gmane.org/gmane.comp.version-control.git/165433

Vincent

Re: Why can't I use git-bisect to find the first *good* commit?

[Christian Couder]

On Mon, Mar 28, 2011 at 2:39 PM, Vincent van Ravesteijn <vfr@lyx.org> wrote:
>
>> Why can't bisect just do the right thing here and accept that your
>> more recent revesion is the good one, and the old one is the bad one?
>
> There was a recent discussion about this:
>
> http://article.gmane.org/gmane.comp.version-control.git/165433

The recent discussion about this topic is rather this one:

http://thread.gmane.org/gmane.comp.version-control.git/165141

and it refers to this one that started with Dscho's patch:

http://thread.gmane.org/gmane.comp.version-control.git/86063

Thanks,
Christian.

Re: Why can't I use git-bisect to find the first *good* commit?

[Andrew Garber]

> I'd expect this to work:
>
>    $ git bisect start
>    $ git bisect good
>    $ git bisect bad HEAD~100

So would I. I think the behavior of git bisect should be changed.
Right now, it's trying to find the first bad commit. Instead, it
should be trying to find the first commit where the code's good/bad
state *changed*. IOW, it should be able to handle both of the
following cases:

good <--- oldest
good
good
bad <--- the commit we want bisect to find
bad
bad <--- newest

bad <--- oldest
bad
bad
good <--- the commit we want bisect to find
good
good <--- newest

It shouldn't matter which end we start on, so long as one end gets
marks good, and the other end gets marked bad.

Andrew

Re: Why can't I use git-bisect to find the first *good* commit?

[Johannes Sixt]

Am 3/28/2011 16:29, schrieb Andrew Garber:
>> I'd expect this to work:
>>
>>    $ git bisect start
>>    $ git bisect good
>>    $ git bisect bad HEAD~100
> 
> So would I. I think the behavior of git bisect should be changed.
> Right now, it's trying to find the first bad commit. Instead, it
> should be trying to find the first commit where the code's good/bad
> state *changed*. IOW, it should be able to handle both of the
> following cases:
> 
> good <--- oldest
> good
> good
> bad <--- the commit we want bisect to find
> bad
> bad <--- newest
> 
> bad <--- oldest
> bad
> bad
> good <--- the commit we want bisect to find
> good
> good <--- newest
> 
> It shouldn't matter which end we start on, so long as one end gets
> marks good, and the other end gets marked bad.

Define "end" and "other end"! It's not that trivial.

      o--o--o--B
     /
  --o--o--o--o--G

When I have this history and I mark B as bad and G as good, will I now
find the first bad or the first good commit?

-- Hannes

Re: Why can't I use git-bisect to find the first *good* commit?

[Andrew Garber]

On Mon, Mar 28, 2011 at 10:40 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:
> Define "end" and "other end"! It's not that trivial.
>
>      o--o--o--B
>     /
>  --o--o--o--o--G
>
> When I have this history and I mark B as bad and G as good, will I now
> find the first bad or the first good commit?
>
> -- Hannes
>

That kind of situation shouldn't occur: IMO, bisect should only deal
with a single branch (the current branch).

Re: Why can't I use git-bisect to find the first *good* commit?

[Matthieu Moy]

Andrew Garber <andrew@andrewgarber.com> writes:

> On Mon, Mar 28, 2011 at 10:40 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:
>
>>      o--o--o--B
>>     /
>>  --o--o--o--o--G
>>
>> When I have this history and I mark B as bad and G as good, will I now
>> find the first bad or the first good commit?
>
> That kind of situation shouldn't occur: IMO, bisect should only deal
> with a single branch (the current branch).

Why?

It's not uncommon in real life to face the "it works in branch foo but
not in branch bar, where did it break?" problem. And one expects a great
tool such as Git to be able to answer it.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: Why can't I use git-bisect to find the first *good* commit?

[Andrew Garber]

If branch bar is broken, do a bisect on branch bar. The fact that
branch foo works in inconsequential.

On Mon, Mar 28, 2011 at 1:33 PM, Matthieu Moy
<Matthieu.Moy@grenoble-inp.fr> wrote:
> Andrew Garber <andrew@andrewgarber.com> writes:
>
>> On Mon, Mar 28, 2011 at 10:40 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:
>>
>>>      o--o--o--B
>>>     /
>>>  --o--o--o--o--G
>>>
>>> When I have this history and I mark B as bad and G as good, will I now
>>> find the first bad or the first good commit?
>>
>> That kind of situation shouldn't occur: IMO, bisect should only deal
>> with a single branch (the current branch).
>
> Why?
>
> It's not uncommon in real life to face the "it works in branch foo but
> not in branch bar, where did it break?" problem. And one expects a great
> tool such as Git to be able to answer it.
>
> --
> Matthieu Moy
> http://www-verimag.imag.fr/~moy/
>

Re: Why can't I use git-bisect to find the first *good* commit?

[Matthieu Moy]

[ please do not top-post ]

Andrew Garber <andrew@andrewgarber.com> writes:

> If branch bar is broken, do a bisect on branch bar. The fact that
> branch foo works in inconsequential.

Then which commit do you specify as "good"?

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: Why can't I use git-bisect to find the first *good* commit?

[Andrew Garber]

On Mon, Mar 28, 2011 at 1:55 PM, Matthieu Moy
<Matthieu.Moy@grenoble-inp.fr> wrote:

> Then which commit do you specify as "good"?

Any ancestral commit *on the same branch* which is know to be working.
Isn't the whole point of git bisect is to do binary search through
time? It only makes sense to me to use it on a single branch at a
time. Perhaps you could give a concrete example of where you could use
it for multiple branches simultaneously?

Re: Why can't I use git-bisect to find the first *good* commit?

[Matthieu Moy]

Andrew Garber <andrew@andrewgarber.com> writes:

> On Mon, Mar 28, 2011 at 1:55 PM, Matthieu Moy
> <Matthieu.Moy@grenoble-inp.fr> wrote:
>
>> Then which commit do you specify as "good"?
>
> Any ancestral commit *on the same branch* which is know to be working.

What is the point is finding manually a commit *on the same branch* when
the tool can do that for you? You don't know how old the breakage is, so
finding the first good commit will take some time. Knowing that the
other branch is good gives you a hint that the common ancestor between
branches should be good, so a good start would be to find the common
ancestor.

But again, why would you insist in doing that manually?

> Isn't the whole point of git bisect is to do binary search through
> time?

No. Bisect does a search through a DAG. And that is the whole point of
bisect: doing a binary search through time is something you could do
manually. That would be less convenient, but still workable. git bisect
is far more clever, and does something you could hardly do manually, or
at least not without getting headaches.

> Perhaps you could give a concrete example of where you could use it
> for multiple branches simultaneously?

Well, see my previous email.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: Why can't I use git-bisect to find the first *good* commit?

[demerphq]

On 28 March 2011 20:23, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:
> Andrew Garber <andrew@andrewgarber.com> writes:
>
>> On Mon, Mar 28, 2011 at 1:55 PM, Matthieu Moy
>> <Matthieu.Moy@grenoble-inp.fr> wrote:
>>
>>> Then which commit do you specify as "good"?
>>
>> Any ancestral commit *on the same branch* which is know to be working.
>
> What is the point is finding manually a commit *on the same branch* when
> the tool can do that for you? You don't know how old the breakage is, so
> finding the first good commit will take some time. Knowing that the
> other branch is good gives you a hint that the common ancestor between
> branches should be good, so a good start would be to find the common
> ancestor.

This doesn't make a lot of sense to me. It is just as likely NOT to be useful.

It could just as easily have been fixed in the other branch. So
knowing its good wont tell you where it was broken.

This started off with:

      o--o--o--B
     /
  --o--o--o--o--G

So lets say that the reality of each node looks like this:

      B--B--B--B*
     /
  --B--B--B--G--G*

How does knowing that G* is good help us find what broke B* again?

Your description matches the case of something like this:

      B--B--B--B*
     /
  --G--G--G--G--G*

But what about something like this:

      Bx--B--B--B*
     /
  --Gz--By--B--Gx--G*

How does knowing that G* is good help you to find that Bx broke the
code in the B* branch again?

Presumably 'By' broke the G* branch which was then fixed by Gx and
none of this information helps you at all identify that Bx broke the
B* branch.

Whereas a plain binary search on the B* branch would eventually find
that Bx was responsible.

>> Perhaps you could give a concrete example of where you could use it
>> for multiple branches simultaneously?
>
> Well, see my previous email.

Where you said "It's not uncommon in real life to face the "it works
in branch foo but
not in branch bar, where did it break?" problem. And one expects a great
tool such as Git to be able to answer it."

Seems to me that this is trying to cram two questions into one:

A) where did branch foo diverge from branch bar and
B) which commit between that ancestor and bar did things break.

Of course im probably missing something important here. Id like to
know what it is tho. :-)

cheers,
Yves

-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: Why can't I use git-bisect to find the first *good* commit?

[Andrew Garber]

>> What is the point is finding manually a commit *on the same branch* when
>> the tool can do that for you?

> Seems to me that this is trying to cram two questions into one:
>
> A) where did branch foo diverge from branch bar and
> B) which commit between that ancestor and bar did things break.

To find the answer to A, I generally just do this (using an alias):

git log --graph --oneline --all

It takes at most a couple of seconds... hardly what I'd call a manual process.

Re: Why can't I use git-bisect to find the first *good* commit?

[Matthieu Moy]

Andrew Garber <andrew@andrewgarber.com> writes:

>>> What is the point is finding manually a commit *on the same branch* when
>>> the tool can do that for you?
>
>> Seems to me that this is trying to cram two questions into one:
>>
>> A) where did branch foo diverge from branch bar and
>> B) which commit between that ancestor and bar did things break.

No. What I'm saying is that if you insist in not using bisect, you'll
probably have to answer these two questions separately.

> To find the answer to A, I generally just do this (using an alias):
>
> git log --graph --oneline --all
>
> It takes at most a couple of seconds... hardly what I'd call a manual
> process.

Suppose you have a bug in git.git that you see in pu, but not in next.
Try finding the common ancestor with your command, and see how long it
takes.

Yes, you'll be able to do it, but you still didn't tell us what was
wrong with

git bisect start
git bisect good origin/next
git bisect bad origin/pu
...

which is _way_ faster. And my example took git.git which isn't a very
large project, so real-life examples could be much worse.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: Why can't I use git-bisect to find the first *good* commit?

[Andrew Garber]

> Suppose you have a bug in git.git that you see in pu, but not in next.
> Try finding the common ancestor with your command, and see how long it
> takes.

Fair enough.

> Yes, you'll be able to do it, but you still didn't tell us what was
> wrong with
>
> git bisect start
> git bisect good origin/next
> git bisect bad origin/pu
> ...
>
> which is _way_ faster. And my example took git.git which isn't a very
> large project, so real-life examples could be much worse.

But what about demerphq's example? (see below)

>      Bx--B--B--B*
>     /
>  --Gz--By--B--Gx--G*
>
> How does knowing that G* is good help you to find that Bx broke the
> code in the B* branch again?
>
> Presumably 'By' broke the G* branch which was then fixed by Gx and
> none of this information helps you at all identify that Bx broke the
> B* branch.
>
> Whereas a plain binary search on the B* branch would eventually find
> that Bx was responsible.

Re: Why can't I use git-bisect to find the first *good* commit?

[Jeff King]

On Mon, Mar 28, 2011 at 04:12:49PM -0400, Andrew Garber wrote:

> But what about demerphq's example? (see below)
> 
> >      Bx--B--B--B*
> >     /
> >  --Gz--By--B--Gx--G*
> >
> > How does knowing that G* is good help you to find that Bx broke the
> > code in the B* branch again?
> >
> > Presumably 'By' broke the G* branch which was then fixed by Gx and
> > none of this information helps you at all identify that Bx broke the
> > B* branch.
> >
> > Whereas a plain binary search on the B* branch would eventually find
> > that Bx was responsible.

If you feed bisect a history where the bug flips off and on between good
and bad commits, you aren't necessarily going to get the answer you
want. But that has nothing to do with the history shape; it is a problem
in a linear history like this, too:

  --G--Bx--B--G--G--By--B

That being said, it is more likely to happen with cherry-picking, which
is more likely to happen with multiple branches. In git.git, we tend to
favor creating (or backporting) bugfix commits close to the bug itself,
and then merging the same commit up.

-Peff

Re: Why can't I use git-bisect to find the first *good* commit?

[Jeff King]

On Mon, Mar 28, 2011 at 04:25:21PM -0400, Jeff King wrote:

> On Mon, Mar 28, 2011 at 04:12:49PM -0400, Andrew Garber wrote:
> 
> > But what about demerphq's example? (see below)
> > 
> > >      Bx--B--B--B*
> > >     /
> > >  --Gz--By--B--Gx--G*
> > >
> > > How does knowing that G* is good help you to find that Bx broke the
> > > code in the B* branch again?
> > >
> > > Presumably 'By' broke the G* branch which was then fixed by Gx and
> > > none of this information helps you at all identify that Bx broke the
> > > B* branch.
> > >
> > > Whereas a plain binary search on the B* branch would eventually find
> > > that Bx was responsible.
> 
> If you feed bisect a history where the bug flips off and on between good
> and bad commits, you aren't necessarily going to get the answer you
> want. But that has nothing to do with the history shape; it is a problem
> in a linear history like this, too:
> 
>   --G--Bx--B--G--G--By--B

Actually, scratch what I said. I misread his graph. The fix has not yet
been cherry-picked, it just exists on the other branch. So there is no
flipping. But as Matthieu explained in another response, there is still
value in bisecting that graph.

-Peff

Re: Why can't I use git-bisect to find the first *good* commit?

[Matthieu Moy]

Andrew Garber <andrew@andrewgarber.com> writes:

> But what about demerphq's example? (see below)
>
>>      Bx--B--B--B*
>>     /
>>  --Gz--By--B--Gx--G*
>>
>> How does knowing that G* is good help you to find that Bx broke the
>> code in the B* branch again?

If all you want is to know which commit introduced the bug, it doesn't.
But usually, what you're really looking for is an explanation and a fix
for the bug. Let's see what git bisect tells us then:

$ git bisect start
$ git bisect good <good-branch>
$ git bisect bad <bad-branch>
Bisecting: a merge base must be tested
[f1fac16fb39dbe421b5cc4bcb945433495c794e1] ...
$ git bisect bad
The merge base f1fac16fb39dbe421b5cc4bcb945433495c794e1 is bad.
This means the bug has been fixed between f1fac16fb39dbe421b5cc4bcb945433495c794e1
 and [089840ef9f8b97ddc9e28fa152c65115fb0b649a].

This doesn't tell you where the bug was introduced, but gives you
something which is usually even more valuable: where to find the fix.
Then, you have the choice between merging the good branch into the bad
one (like merging a maintainance branch into a dev branch), or to bisect
again to find the actual fix in good-branch.

If you really wanted to find the first bad commit, you've spent one
iteration sub-optimally and can start a new bisect with the merge base
as the base commit.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: Why can't I use git-bisect to find the first *good* commit?

[Andreas Ericsson]

On 03/28/2011 08:57 PM, demerphq wrote:
> On 28 March 2011 20:23, Matthieu Moy<Matthieu.Moy@grenoble-inp.fr>  wrote:
>> Andrew Garber<andrew@andrewgarber.com>  writes:
>>
>>> On Mon, Mar 28, 2011 at 1:55 PM, Matthieu Moy
>>> <Matthieu.Moy@grenoble-inp.fr>  wrote:
>>>
>>>> Then which commit do you specify as "good"?
>>>
>>> Any ancestral commit *on the same branch* which is know to be working.
>>
>> What is the point is finding manually a commit *on the same branch* when
>> the tool can do that for you? You don't know how old the breakage is, so
>> finding the first good commit will take some time. Knowing that the
>> other branch is good gives you a hint that the common ancestor between
>> branches should be good, so a good start would be to find the common
>> ancestor.
> 
> This doesn't make a lot of sense to me. It is just as likely NOT to be useful.
> 
> It could just as easily have been fixed in the other branch. So
> knowing its good wont tell you where it was broken.
> 
> This started off with:
> 
>        o--o--o--B
>       /
>    --o--o--o--o--G
> 
> So lets say that the reality of each node looks like this:
> 
>        B--B--B--B*
>       /
>    --B--B--B--G--G*
> 
> How does knowing that G* is good help us find what broke B* again?
> 

Because if G* is good and B* is bad, git-bisect knows to look for
the merge-base (the leftmost lower B in your graph) to know that
it's not looking at uninteresting commits.

> Your description matches the case of something like this:
> 
>        B--B--B--B*
>       /
>    --G--G--G--G--G*
> 
> But what about something like this:
> 
>        Bx--B--B--B*
>       /
>    --Gz--By--B--Gx--G*
> 
> How does knowing that G* is good help you to find that Bx broke the
> code in the B* branch again?
> 

Because it helps locate Gz as the common ancestor between G* and B*.

To bisect a problem with B*, you'd mark B* as bad and, assuming Gz is
a stable release from which the feature-branch B has arisen, you'd
mark Gz as good. Then bisect will quite quickly find that Bx is the
culprit.

Or, if G* works well but B* does not and you can't be arsed to locate
the common ancestor yourself, you mark G* as good and B* as bad and then
git-bisect will automatically reset the "good" mark from G* to Gz. This
ofcourse doesn't work when the two points in doesn't share a common
ancestor, although that should be a rare occasion indeed and won't
make much sense.

> Presumably 'By' broke the G* branch which was then fixed by Gx and
> none of this information helps you at all identify that Bx broke the
> B* branch.
> 

You're right. For that, you'd need two different bisects. If the two
breakages are identical (an unlikely situation, but still), you'd end
up with git-bisect finding a single commit when marking GB* (the merge
of G* and B*) as bad and Gz as good. Which one is hard to tell though.
But then again, git-bisect has never claimed to be able to locate all
bugs at once. It can just help you locate where a particular bug was
introduced.

There are some cases where it will fail quite gruesomely though. When
one bug is covered by another and fixing the first bug uncovers the
one you're bisecting for. Even then it will still cut down considerably
on time spent doing implemetation analysis (aka "staring at code").

> Whereas a plain binary search on the B* branch would eventually find
> that Bx was responsible.
> 

True, but a binary search would mean about a bazillion git-bisect runs
on large and complex history (such as the kernel, or git itself) when
the bug is located in an already merged feature-branch, but noone knows
which one.

>>> Perhaps you could give a concrete example of where you could use it
>>> for multiple branches simultaneously?
>>
>> Well, see my previous email.
> 
> Where you said "It's not uncommon in real life to face the "it works
> in branch foo but
> not in branch bar, where did it break?" problem. And one expects a great
> tool such as Git to be able to answer it."
> 
> Seems to me that this is trying to cram two questions into one:
> 
> A) where did branch foo diverge from branch bar and
> B) which commit between that ancestor and bar did things break.
> 

The only question git-bisect tries to answer is "which is the first
bad commit". If you want the implementation details, I suggest you
read the code, but naturally it knows how to walk the DAG it's
inspecting and naturally it tries to make life easier for the human
it inevitable serves.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.

Re: Why can't I use git-bisect to find the first *good* commit?

[Michael Witten]

On Mon, Mar 28, 2011 at 09:32, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> But instead I have to do:
>
>    $ git bisect start
>    $ git bisect bad
>    $ git bisect good HEAD~100
>
> And then proceed to mark good revisions as bad, and bad revisions as
> good.
>
> That works, but it's very confusing.
>

To me, at least, that seems perfectly reasonable.

In particular, the problem seems to be your own thinking:

> And then proceed to mark good revisions as bad, and bad revisions as
> good.

Don't think of it like that.