Re: [PATCH 1/1] perl-tests: convert to ptest

["Björn Stenberg" <bjst@enea.com>]

Jesse Zhang wrote:
> How verbose should the output be? The TEST script actually does its own
> analyzing and summarizing of the test results. Do we want the summary
> only or all the details? Do you want the detailed output converted to
> ptest format too?
> 
> For example,
> 
>     # ./TEST 
>     t/base/cond....................................................ok
>     t/base/if......................................................ok
>     t/base/lex.....................................................ok
>     t/base/num.....................................................ok

I would say this higher-level form is OK. Just sed it into:

PASS: t/base/cond
PASS: t/base/if
PASS: t/base/lex
PASS: t/base/num

> The output from the actual test case conforms to the TAP format (not
> sure about the top-level TEST). If you have an analysis tool, why not
> support one more widely used format? I can't see much value in forcing
> one format everywhere.

There is no supplied analysis tool for ptest. Ptest is defined as producing a specific output format, to simplify parsing and analysis of the results however and wherever you want to do it.

The alternative would be to say that a specific analysis tool must be used which understands every different way of reporting results. And after going through a number of packages I can assure you are a lot of different ways to do that, some quite creative. It would mean that every package maintainer would also have to add the result format of his package into that analysis tool, likely creating a huge mess.

I prefer the current way of putting the responsibility on each package to produce a standard output format.

-- 
Björn