Re: [PATCH 2/2] test-lib: GIT_TEST_ONLY to run only specific tests

[Ilya Bobyr <ilya.bobir@gmail.com>]

On 3/3/2014 4:08 PM, Junio C Hamano wrote:
> Ilya Bobyr <ilya.bobir@gmail.com> writes:
>
>> It might be that we are looking at different use cases, as you are
>> talking about whole test suits.
> I do not think so.

Good :)
I am trying to understand the use cases.  And make sure we are talking 
about the same ones.
I am not sure what are the use cases for GIT_SKIP_TESTS.

I think that while in it really nice when an interface allows to do new 
things, the main use case (or use cases) should be as easy and obvious 
in the first place.
If the target is the TDD use case I described, then it appears that a 
user needs to do double negation.  That is what concerns me.

> I do not see anything prevents you from saying
>
> 	GIT_SKIP_TESTS='t0000 !t0000.1 !t0000.4'
>
> to specify test-pieces in individual tests so that you can run the
> setup step (step .1) and the specific test (step .4) without running
> two tests in between.

While it could be done, it looks less obvious than this:

     GIT_TEST_ONLY='1 4' ./t0001-init.sh

What if we do what you proposed, but with GIT_RUN_TESTS?
If we want to have one interface, maybe, building on top of a "negation" 
(GIT_SKIP_TESTS) is not very good.
At the same time, GIT_TEST_ONLY is also too specific for a generic 
interface.
I could add GIT_RUN_TESTS and allow it to have all of the features, thus 
making GIT_SKIP_TESTS a working but deprecated tool.

Running specific tests:
     GIT_RUN_TESTS='t0000'
     GIT_RUN_TESTS='t0000.1 t0002'
     GIT_RUN_TESTS='1 3 7'

Negating some tests:
     GIT_RUN_TESTS='!t0000'
     GIT_RUN_TESTS='!t0000.1'
     GIT_RUN_TESTS='!1 !3 !7'

The above would work exactly as you described but with one less level of 
negation.  Default is everything, unless at least one positive pattern 
is given.

Later on range specification could added:
     GIT_RUN_TESTS='<11'

At least for now that would cover most use cases that I can think of 
that look reasonable.