Re: [PATCHv3 1/2] scripts/oe-selftest: Allow to run tests on random/all architecture

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Thu, 2015-12-17 at 16:38 +0200, Daniel Istrate wrote:
> Add an option for random arch into oe-selftest:
> --arch [random/all]
> 1. random: will set a random MACHINE for each test
> 2. all: will run tests for all architectures
> 
> Custom arch sets only weak default values (??=) for MACHINE in
> local.conf.
> This let test cases that require a specific MACHINE to be able to
> override it, using (?= or =).
> 
> e.g.:
> oe-selftest --run-tests signing --arch random -->
> will run all tests switching MACHINE randomly for each test
> 
> oe-selftest --run-tests signing --arch all -->
> for each arch will run all tests
> 
> oe-selftest --run-all-tests --arch random
> 
> Also update oeqa/selftest/base.py to accomodate this feature.
> 
> Fix for [YOCTO #5880].

This is actually incredibly confusing since "arch" means something
quite different in the OE sense and this is really "machine". I
therefore have concerns about this option name.

I also strongly dislike hardcoded lists of machines. That *cannot* go
in as it stands I'm afraid.

Cheers,

Richard