Re: RFC/Pull Request: Refs db backend

[David Turner <dturner@twopensource.com>]

On Tue, 2015-06-23 at 17:23 +0700, Duy Nguyen wrote:
> On Tue, Jun 23, 2015 at 7:50 AM, David Turner
<dturner@twopensource.com> wrote:
> > To test this backend's correctness, I hacked test-lib.sh and
> > test-lib-functions.sh to run all tests under the refs backend.
> 
> Now we have two. split-index also benefits from running through full
> test suite like this. I propose we make "make test" run the test suite
> twice. The first run is with default configuration, no split index, no
> fancy ref backend. The second run enables split-index and switches to
> new backend, running through all test cases. In future we can also
> enable packv4 in this second run. There won't be a third run.
> 
> When the second ref backend comes, we can switch between the two
> backends using a random number generator where we control both
> algorithm and seed, so that when a test fails, the user can give us
> their seed and we can re-run with the same configuration.

I'm not in love with this idea, because it makes it hard to do
exhaustive testing efficiently.  I would rather have make test run
through all tests under all combinations -- or at least all relevant
tests.  We could perhaps mark tests with a list of features that they
exercise, so that we don't have to run e.g. t8xxx with alternate refs
backends.  

> Dozens of tests use manual ref/reflog reading/writing, or create
submodules
> > without passing --refs-backend-type to git init.  If those tests are
> > changed to use the update-ref machinery or test-refs-be-db (or, in
the
> > case of packed-refs, corrupt refs, and dumb fetch tests, are
skipped),
> > the only remaining failing tests are the git-new-workdir tests and
the
> > gitweb tests.
> 
> I haven't read the series, but I guess you should also add a few tests
> to run on the first run, so new code is exercised a bit even if people
> skip the second run.

I did this already, yes.