Re: script to pre-build all commits in a git-bisect, and use them during testing

[Junio C Hamano <gitster@pobox.com>]

Britton Kerin <britton.kerin@gmail.com> writes:

> Problem: it's annoying to have to build each commit when
> git-bisect'ing for something that isn't being tested automatically (so
> no git bisect run).
>
> Solution: https://github.com/bkerin/git-batch-bisect

So, because "git bisect run" would not be usable for whatever reason
to run a "build and then test" from the command line, you would
build all revisions using

    git batch-bisect runinall 'autoreconf --install && ./configure && make'

before starting to test any revision, and then after building
everything, you would do:

    git batch-bisect runincurrent ./test_program
    git batch-bisect good
    git batch-bisect runincurrent ./test_program
    git batch-bisect bad
    ...

where ./test_program cannot signal if it is good or bad mechanically
(e.g., by exiting with non-zero status for failure) but the testers
need to judge the good/bad manually before they can say "git
batch-bisect good/bad", I presume.

It's an interesting workaround for a test program that cannot be
automated.  You are willing to waste CPU cycles and diskspace to
build upfront and hold the build products for all N revisions in the
range, where with bisection you would have to build far fewer
revisions (which is the point of bisection), to optimize for
developer/tester latency, which tends to be more expensive.  The
trade-off may make good sense.