Re: [OE-core] [PATCH] oeqa/ptest: print a warning if ptests failed

["Richard Purdie" <richard.purdie@linuxfoundation.org>]

On Sat, 2020-12-19 at 18:29 +0100, Alexander Kanavin wrote:
> This allows spotting ptest regressions without having hard ptest failures
> (for that full ptest stability should be achieved).
> 
> Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
> ---
>  meta/lib/oeqa/runtime/cases/ptest.py | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/lib/oeqa/runtime/cases/ptest.py b/meta/lib/oeqa/runtime/cases/ptest.py
> index a9572c81f0..0800f3c27f 100644
> --- a/meta/lib/oeqa/runtime/cases/ptest.py
> +++ b/meta/lib/oeqa/runtime/cases/ptest.py
> @@ -108,4 +108,5 @@ class PtestRunnerTest(OERuntimeTestCase):
>              failmsg = failmsg + "Failed ptests:\n%s" % pprint.pformat(failed_tests)
>  
>          if failmsg:
> +            self.logger.warning("There were failing ptests.")
>              self.fail(failmsg)

Have you looked at this on the arm worker? That seems to have four
failing tests over three recipes:

{'libinput': ['libinput-test-suite'],
 'strace': ['qual_fault-syscall.test', 'qual_fault.test'],
 'valgrind': ['drd/tests/std_list']}

https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1355

Not sure if these always fail or are intermittent?

Cheers,

Richard