Re: [PATCH] travis-ci: run previously failed tests first, then slowest to fastest

[Junio C Hamano <gitster@pobox.com>]

Junio C Hamano <gitster@pobox.com> writes:

> This is cute, but isn't it useful even outside Travis's context?  I
> am not suggesting to touch anything other than .travis.yml file in
> this patch, but if I wanted to get the benefit from the idea in this
> patch when I run my tests manually, I can just tell prove to use the
> cached states, no?

It seems that exporting something like

    GIT_PROVE_OPTS="--timer --state=slow,save -j8" 

when running "make DEFAULT_TEST_TARGET=prove test" does give me the
same benefit by leaving the stats from the previous run in t/.prove
when making the test scheduling decisions.

One thing I noticed but didn't dig further to fix was that this
"prove --state" business did not seem to work well together with

    make T="...list of tests..." test

that limits the set of tests to perform.  For example:

    $ rm -f t/.prove
    $ make -j4 GIT_PROVE_OPTS="--timer --state=slow,save -j8" \
	   T="$( cd t && echo t0???-*.sh)" \
           DEFAULT_TEST_TARGET=prove test

runs all test in 0xxx series and populates t/.prove with them.  And
immediately after that, with t/.prove still there:

    $ make -j4 GIT_PROVE_OPTS="--timer --state=slow,save -j8" \
         DEFAULT_TEST_TARGET=prove \
         T=t0000-basic.sh test

does not limit the test to only 0000, but ends up running all the
others recorded in t/.prove file, it seems.

I would imagine that this would not affect your use case negatively,
as it is unlikely that your automated tests are skipping different
set of tests in each run.