Re: Discussion: Package testing

["Björn Stenberg" <bjst@enea.com>]

Fredrik Hugosson wrote:
> I think you both might be missing some cases here. The problem is
> that 'make check' does not have a standardized meaning.

Even if there was a standardized meaning to "make check", there are lots of packages that use completely differents systems for their building and testing. Hence there is no generic solution, we have to adapt each individual package manually.

> 1) Function test level with testing run on target, package cross-compiled
> 2) Function test level with testing run on host, package cross-compiled
> 3) Function test level with testing run on host, package compiled for host
> 4) Unit test level with testing run on target, package cross-compiled
> 5) Unit test level with testing run on host, package compiled for host

Our proposal is focused on testing the target binaries on the target, in an attempt to limit complexity. Testing -native and -nativesdk packages is therefore outside this scope.

> Would be very nice for functional tests, but it will not be so
> simple for unit testing. For unit tests I belive you need some build
> target too, which leads back to the comment above.

Yes, this proposal only works for tests that run on a single machine. But from what I have seen so far, that covers the vast majority of included package tests. Do you have examples to the contrary?

There will certainly be room and need for other catergories of tests, which is one reason I've used the specific "ptest" moniker for this. This is not intended to be a covers-all-bases test solution. It's just one part.

-- 
Björn