Re: [PATCH v2 0/3] Make the Git tests emit TAP format

[Andreas Ericsson <ae@op5.se>]

On 06/15/2010 05:17 PM, Ævar Arnfjörð Bjarmason wrote:
> On Mon, Jun 14, 2010 at 22:10, Ævar Arnfjörð Bjarmason<avarab@gmail.com>  wrote:
>> On Mon, Jun 14, 2010 at 21:49, Jakub Narebski<jnareb@gmail.com>  wrote:
>>> Ævar Arnfjörð Bjarmason<avarab@gmail.com>  writes:
>>>
>>>> All this series does is slightly adjust the raw text output of our
>>>> tests so that it conforms. to the TAP standard, i.e. instead of this:
>>>>
>>>>     $ ./t0005-signals.sh
>>>>     *   ok 1: sigchain works
>>>>     * passed all 1 test(s)
>>>>
>>>> We get this:
>>>>
>>>>     $ ./t0005-signals.sh
>>>>     ok 1 - sigchain works
>>>>     # passed all 1 test(s)
>>>>     1..1
>>>>
>>>> Changing the output format like this gives us the ability to run the
>>>> Git tests with any TAP tool (like prove(1)) at no extra cost. Every
>>>> other existing way of running the tests continues to work, it's just
>>>> easier for machines to read the output now.
>>
>> I'll re-submit a version of the patch with a better commit message
>> which addresses all of the below.
>>
>>> This doesn't tell us if the result of running test suite with
>>> '--verbose' and/or with '--debug' changes, and if changes how?  Is it
>>> compatibile with TAP format so that TAP parsers understand it?
>>
>> It just changes in that the lines that previously said e.g. "*   ok 1:
>> sigchain works" now say "ok 1 - sigchain works".
>>
>> TAP parsers still understand it, since ignoring unknown garbage is
>> part of the TAP standard.
> 
> Actually it doesn't work for all the tests. The following crops up on
> prove -j 10 ./t[0-9]*.sh :: --verbose:
> 
>      ./t1007-hash-object.sh                             (Wstat: 0
> Tests: 19 Failed: 0)
>        Parse errors: Tests out of sequence.  Found (12) but expected (11)
>                      Tests out of sequence.  Found (13) but expected (12)
>                      Tests out of sequence.  Found (14) but expected (13)
>                      Tests out of sequence.  Found (15) but expected (14)
>                      Tests out of sequence.  Found (16) but expected (15)


Why are you using a plan at all? I absolutely detest using planned tap
series, and last I heard was that the majority of TAP-fanatics actually
agreed that using a plan to run tests was an extraordinarily lousy idea,
since it makes it harder to add tests properly.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.