script to pre-build all commits in a git-bisect, and use them during testing

script to pre-build all commits in a git-bisect, and use them during testing

[Britton Kerin]

Problem: it's annoying to have to build each commit when
git-bisect'ing for something that isn't being tested automatically (so
no git bisect run).

Solution: https://github.com/bkerin/git-batch-bisect

It has an interface almost exactly like native git-bisect so it's
super easy to learn but it handles a couple additional commands to
create worktrees for all the commits, parallel build, and test in
them.

From the README:

     git batch-bisect start bad_commit good_commit
     git batch-bisect runinall 'autoreconf --install && ./configure && make'
     git batch-bisect runincurrent ./test_program
     git batch-bisect good
     git batch-bisect runincurrent ./test_program
     git batch-bisect bad
     # etc.

I'd be grateful for any feedback on this script.  I don't actually
bisect all that often, so it's only tested on one real-world project
so far.  I thought it might live in contrib someday if it's useful to
anyone else.

Britton

Re: script to pre-build all commits in a git-bisect, and use them during testing

[Junio C Hamano]

Britton Kerin <britton.kerin@gmail.com> writes:

> Problem: it's annoying to have to build each commit when
> git-bisect'ing for something that isn't being tested automatically (so
> no git bisect run).
>
> Solution: https://github.com/bkerin/git-batch-bisect

So, because "git bisect run" would not be usable for whatever reason
to run a "build and then test" from the command line, you would
build all revisions using

    git batch-bisect runinall 'autoreconf --install && ./configure && make'

before starting to test any revision, and then after building
everything, you would do:

    git batch-bisect runincurrent ./test_program
    git batch-bisect good
    git batch-bisect runincurrent ./test_program
    git batch-bisect bad
    ...

where ./test_program cannot signal if it is good or bad mechanically
(e.g., by exiting with non-zero status for failure) but the testers
need to judge the good/bad manually before they can say "git
batch-bisect good/bad", I presume.

It's an interesting workaround for a test program that cannot be
automated.  You are willing to waste CPU cycles and diskspace to
build upfront and hold the build products for all N revisions in the
range, where with bisection you would have to build far fewer
revisions (which is the point of bisection), to optimize for
developer/tester latency, which tends to be more expensive.  The
trade-off may make good sense.