Re: [OE-core] [PATCH v3 0/3] ptest-runner 2.0

["Aníbal Limón" <anibal.limon@linux.intel.com>]

Hi Todor,

Comments below,
	alimon


On 01/05/2016 06:13 PM, Tudor Florea wrote:
> Hi Anibal,
> Please see my comments inline.
> Regards,
>   Tudor.
> 
> On 04/01/2016 20:29, Aníbal Limón wrote:
>> The whole set is sent to two ML for reviewing purposes the first
>> two patches are for oe-core and the last one for poky.
>>
>> The first two commits (oe-core) adds a new ptest-runner written in
>> python the sh one is conserved to use in tiny systems.
>>
>> What's new in ptest runner:
>>
>> - Monitor/timeout stdout, stderr of the test suite to avoid block
>> indefinetly.
> This is definitely something useful.
>> - Add option for change ptest root directory. - Add option for list
>> available tests. - Add option for only run certain tests.
>>
>> The last commit (meta-yocto) sets ptest-runner to 1.0 in poky-tiny
>> systems due to python dependency.
> I think adding python dependency is a shift in a wrong direction for
> for ptest-runner. The assumption that most embedded devices (tiny or
> not) have python might not be correct.
> The alternative of using the old version of ptest-runner only
> complicate things.
> I do think this kind of work is really useful but this should be done
> into a testing framework that runs outside of the DUT.

The original problem is the current sh ptest-runner blocks indefinitely
when a package ptest blocks (we don't control this), now python-ptest is
blocking the ptest-runner, see [1]. The way to solve this is adding
timeout in some place.

I know that bash support timeout option but this option works over the
whole process making us to define timeouts too big or per machine/device
also tiny systems mainly don't use bash. That's the reason for i decide
to write python version of ptest-runner that applies a timeout over the
output of the program give us better control/granularity over the
running ptest.


If you now other way to solve the problem please tell us.


[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=8021


>>
>> The changes are available in the git repository at:
>>
>> git://git.yoctoproject.org/poky-contrib alimon/ptest-runner 
>> http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=alimon/ptest-runner
>>
>>  Aníbal Limón (3): ptest-runner: Add version 2.0 re-implementation
>> in python. ptest-runner: Add a recipe for install ptest-runner
>> 2.0. conf/distro/poky-tiny.conf: Add default install of
>> ptest-runner 1.0
>>
>> meta-yocto/conf/distro/poky-tiny.conf              |   4 + 
>> .../ptest-runner/files/ptest-runner_2.0.py         | 162
>> +++++++++++++++++++++ .../ptest-runner/ptest-runner_2.0.bb
>> |  27 ++++ 3 files changed, 193 insertions(+) create mode 100755
>> meta/recipes-support/ptest-runner/files/ptest-runner_2.0.py create
>> mode 100644 meta/recipes-support/ptest-runner/ptest-runner_2.0.bb
>>