git bisect on multiple cores

git bisect on multiple cores

[A B]

git bisect is really cool. I use it for the first time.
Just a thought, if you have a multiple core computer, can't you make
git build new versions in the background while testing the previuos
version? Alright, if you build 2 versions, one of them will never be
tested, but you will perhaps save some time by letting it build in the
background?

Just a thought...

Re: git bisect on multiple cores

[David Symonds]

On Tue, Apr 8, 2008 at 8:58 PM, A B <gentosaker@gmail.com> wrote:

> git bisect is really cool. I use it for the first time.
>  Just a thought, if you have a multiple core computer, can't you make
>  git build new versions in the background while testing the previuos
>  version? Alright, if you build 2 versions, one of them will never be
>  tested, but you will perhaps save some time by letting it build in the
>  background?

Git is used to track more than just source code that can be "built",
and git bisect can be used for more than just tracking down bugs.

I'm not convinced the considerable extra complexity would be
worthwhile. You'd have to have git bisect do checkouts to new
temporary directories, track them and clean them up.

It might be interesting if you approached it as a tri-section or a
general N-section where you try to divide the interval into N parts
and concurrently test N-1 commits. But really, do you find git bisect
all that slow in practice? You probably have a reasonable guess as to
where a regression has come in, and so even 1000 revisions needs at
most 10 bisections to find the culprit.


Dave.

Re: git bisect on multiple cores

[Junio C Hamano]

"David Symonds" <dsymonds@gmail.com> writes:

> Git is used to track more than just source code that can be "built",
> and git bisect can be used for more than just tracking down bugs.
>
> I'm not convinced the considerable extra complexity would be
> worthwhile. You'd have to have git bisect do checkouts to new
> temporary directories, track them and clean them up.

I personally think "git bisect" Porcelain does a bit too much.  For
example, it always checks the revision out, but some non-build checks can
be done without having a checkout.

The core idea of bisect, which is really about how to effectively use "git
rev-list --bisect", is quite simple.  You start by one bad and zero or
more "good" ones, where the "bad" one is a descendant of the good ones,
and hunt for a _single_ change that changed a "good" state into a "bad"
one.

This last point is important.  The only thing "bisect" can find is a
single flip where all of its ancestors are "good" and where all of its
descendants are "bad".  Think of a bad gene introduced by a mutation at a
particular commit (i.e. "generation") and that contaminates all of its
children and descendants forever.

If you visualize the commit ancestry graph growing from left to right like
we usually draw in our documentation and e-mails, and if you paint
known-to-be-good ones blue, known-to-be-bad ones red, and unknown ones
yellow, you will get a picture not unlike the ones shown on pp.111-115 of 
http://userweb.kernel.org/~junio/200607-ols.pdf

The underlying "git rev-list --bisect" takes a set of "good" commits and a
single "bad" commit, computes the set of commits that are unknown
("yellow"), and gives one of them that lies halfway from "good" ones
("blue") and the "bad" one ("red").

The way "git bisect" operates is:

 (0) Prime the process by getting a single "bad" and zero or more "good";
     switch to "bisect" branch.

 (1) Ask "rev-list --bisect" the midpoint; check out that revision for you
     to test.

 (2) It's your turn to give more information to "git bisect".

 (2-a) If you say "good", it is added to the set of "good"; go back to (1)

 (2-b) If you say "bad", it is set to "bad" (because of the way bisection
       works, this is always an ancestor of the previous "bad", and
       because the only thing we do is to find a single flip, keeping a
       single "bad" that is an ancestor of all the commits previously
       known-to-be-bad is sufficient); go back to (1)

One thing to note is that in (2-a) or (2-b), you do not necessarily have
to say the commit the command gave you in step (1) is good or bad.  If the
revision given by (1) is untestable, you can reset to another yellow one,
test that, and tell the command "This is good/bad".

So one way to speed up your bisection process would be:

 * Have multiple work trees (e.g. contrib/workdir/git-new-workdir);

 * Run bisect in one repository;

 * In step (1) of each round, look at gitk output and pick another commit
   that is distant from the one you are going to test.  In another work
   tree, check that one out and test it in parallel.

 * You can feed the good/bad information you obtained from the test you
   run in the neighbouring work tree, in addition to what you learn in
   your main tree, with "git bisect good $it" or "git bisect bad $it".

> It might be interesting if you approached it as a tri-section or a
> general N-section where you try to divide the interval into N parts
> and concurrently test N-1 commits. But really, do you find git bisect
> all that slow in practice? You probably have a reasonable guess as to
> where a regression has come in, and so even 1000 revisions needs at
> most 10 bisections to find the culprit.

I think adding N-section to "rev-list --bisect" would generally be an
interesting thing to do.  For one thing, it would allow you to automate
the step to "pick another commit that is distant from the one you are
going to test" in the above sequence.

Re: git bisect on multiple cores

[Christian Couder]

Le mardi 8 avril 2008, A B a écrit :
> git bisect is really cool. I use it for the first time.
> Just a thought, if you have a multiple core computer, can't you make
> git build new versions in the background while testing the previuos
> version? Alright, if you build 2 versions, one of them will never be
> tested, but you will perhaps save some time by letting it build in the
> background?

Yes, you can do that.

If you have cloned your repository twice (or more), then you can bisect 
compile and test at the same time in your 3 repositories (or more if you 
really want).

For example if bisecting in one repo asks you to test revision X, then you 
can bisect and then build (and even maybe start testing) in another repo 
assuming revision X is good, and in yet in another one assuming revision X 
is bad.

(In the repo where you assume X is good you use:

"git bisect start CURRENT-BAD X"

and in the third one, where you assume X is bad, you use:

"git bisect start X CURRENT-GOOD")

When you know that X was good then you can kill the build or test processes 
and "git bisect reset" in the repository where you assumed wrongly X was 
bad. You can then assume something else with "git bisect start B G" and 
build and test in this repo again.

In the repo that told you X was good, then you need only to use "git bisect 
good" or "git bisect bad" without building and testing to assume something 
about the revision that should be built and tested.

Regards,
Christian.

Re: git bisect on multiple cores

[Christian Couder]

Le mercredi 9 avril 2008, Christian Couder a écrit :
> Le mardi 8 avril 2008, A B a écrit :
> > git bisect is really cool. I use it for the first time.
> > Just a thought, if you have a multiple core computer, can't you make
> > git build new versions in the background while testing the previuos
> > version? Alright, if you build 2 versions, one of them will never be
> > tested, but you will perhaps save some time by letting it build in the
> > background?
>
> Yes, you can do that.
>
> If you have cloned your repository twice (or more), then you can bisect
> compile and test at the same time in your 3 repositories (or more if you
> really want).
>
> For example if bisecting in one repo asks you to test revision X, then
> you can bisect and then build (and even maybe start testing) in another
> repo assuming revision X is good, and in yet in another one assuming
> revision X is bad.
>
> (In the repo where you assume X is good you use:
>
> "git bisect start CURRENT-BAD X"
>
> and in the third one, where you assume X is bad, you use:
>
> "git bisect start X CURRENT-GOOD")

In fact, you have to give all the good revisions you have, not just the last 
one. So it should be 

"git bisect start CURRENT-BAD X GOOD1 GOOD2 ..."

or

"git bisect start X GOOD1 GOOD2 ..."

instead of the above.

> When you know that X was good then you can kill the build or test
> processes and "git bisect reset" in the repository where you assumed
> wrongly X was bad. You can then assume something else with "git bisect
> start B G" and build and test in this repo again.

Above also you must give all the good ones, so it is:

"git bisect start B G1 G2..."

> In the repo that told you X was good, then you need only to use "git
> bisect good" or "git bisect bad" without building and testing to assume
> something about the revision that should be built and tested.

Regards,
Christian.

Re: git bisect on multiple cores

[Karl Hasselström]

On 2008-04-08 12:58:47 +0200, A B wrote:

> git bisect is really cool. I use it for the first time. Just a
> thought, if you have a multiple core computer, can't you make git
> build new versions in the background while testing the previuos
> version? Alright, if you build 2 versions, one of them will never be
> tested, but you will perhaps save some time by letting it build in
> the background?

Adding parallelism to a binary search scales very badly -- I'd say
about logarithmically, but I haven't thought hard about it. If it's
possible to use the extra cores to speed up the build+test cycle,
that's vastly preferable.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: git bisect on multiple cores

[Matthieu Moy]

Karl Hasselström <kha@treskal.com> writes:

> On 2008-04-08 12:58:47 +0200, A B wrote:
>
>> git bisect is really cool. I use it for the first time. Just a
>> thought, if you have a multiple core computer, can't you make git
>> build new versions in the background while testing the previuos
>> version? Alright, if you build 2 versions, one of them will never be
>> tested, but you will perhaps save some time by letting it build in
>> the background?
>
> Adding parallelism to a binary search scales very badly -- I'd say
> about logarithmically, but I haven't thought hard about it. If it's
> possible to use the extra cores to speed up the build+test cycle,
> that's vastly preferable.

Probably logarithmically with the number of cores. But for reasonable
machines, this number is relatively low, so the log is not so costly.
For a binary search, using just 2 cores, you can try the next in the
list in case of a "git bisect good" for example, and if the hypothesis
is true, you've just gained a factor 2 (assuming it happens 50% of
times, that should be a 50% speedup). Similarly, you should get a
factor 2 with 3 cores.

And as said before, you can also try trisection or N-section. But no,
I won't claim this is simple to implement ;-).

-- 
Matthieu

Re: git bisect on multiple cores

[Karl Hasselström]

On 2008-04-09 10:07:25 +0200, Matthieu Moy wrote:

> Karl Hasselström <kha@treskal.com> writes:
>
> > Adding parallelism to a binary search scales very badly -- I'd say
> > about logarithmically, but I haven't thought hard about it. If
> > it's possible to use the extra cores to speed up the build+test
> > cycle, that's vastly preferable.
>
> Probably logarithmically with the number of cores. But for
> reasonable machines, this number is relatively low, so the log is
> not so costly. For a binary search, using just 2 cores, you can try
> the next in the list in case of a "git bisect good" for example, and
> if the hypothesis is true, you've just gained a factor 2 (assuming
> it happens 50% of times, that should be a 50% speedup). Similarly,
> you should get a factor 2 with 3 cores.

Yeah. But to get a factor 3, you need 7 cores; and for 4, you need 15.
It goes downhill from there. If your build+test cycle is
parallelizable at all, I don't think you'll find those numbers hard to
beat.

(There's also the fact that testing several revisions at once assumes
that the whole build+test cycle is automated, or at least most of it.
Otherwise you need more people as well as more cores.)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle