Re: [OE-core] [PATCH v2 2/9] testimage.bbclass: detect slirp from TEST_RUNQEMUPARAMS

[Mikko Rapeli <mikko.rapeli@linaro.org>]

Hi,

On Wed, Aug 23, 2023 at 11:57:50AM +0200, Alexander Kanavin wrote:
> On Wed, 23 Aug 2023 at 11:48, Mikko Rapeli <builder@linaro.org> wrote:
> 
> > Cannot run ptests without @expectedFailure as ptests are expected to fail
> > QMP released QEMU at 08/23/23 10:26:03 and took 0.13 seconds from connect
> > Cannot run ptests without @expectedFailure as ptests are expected to fail
> > QMP connected to QEMU at 08/23/23 10:26:04 and took 0.60 seconds
> > QMP released QEMU at 08/23/23 10:26:04 and took 0.13 seconds from connect
> > Cannot run ptests without @expectedFailure as ptests are expected to fail
> > RESULTS:
> > RESULTS - parselogs.ParseLogsTest.test_parselogs: PASSED (4.30s)
> > RESULTS - ping.PingTest.test_ping: PASSED (0.04s)
> > RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: PASSED (1.55s)
> > RESULTS - ssh.SSHTest.test_ssh: PASSED (1.01s)
> > RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s)
> > SUMMARY:
> > core-image-ptest-libtry-tiny-perl () - Ran 5 tests in 7.208s
> > core-image-ptest-libtry-tiny-perl - OK - All required tests passed (successes=3, skipped=1, failures=0, errors=0)
> >
> > The ptest execution seems to be skipped for all images.
> 
> This is actually correct and ptest did run. We have a soft and hard
> fail options for ptests, and it defaults to soft fail
> (PTEST_EXPECT_FAILURE = "1" in core-image-ptest), i.e. warnings on the
> autobuilder.

OK, that is good to know. In this case with local.conf additions for "slirp":

IMAGE_CLASSES += "testimage"
#IMAGE_FEATURES += "ssh-server-dropbear"
TEST_RUNQEMUPARAMS += "slirp"
# update kernel to latest available in poky
PREFERRED_VERSION_linux-yocto = ""
SANITY_TESTED_DISTROS = ""

The tests passed with one warning, which I'll try to check:

WARNING: core-image-ptest-openssh-1.0-r0 do_testimage: There were failing ptests.
Traceback (most recent call last):
  File "/home/builder/src/poky/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
    return func(*args, **kwargs)
  File "/home/builder/src/poky/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
    return func(*args, **kwargs)
  File "/home/builder/src/poky/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
    return func(*args, **kwargs)
  File "/home/builder/src/poky/meta/lib/oeqa/runtime/cases/ptest.py", line 27, in test_ptestrunner_expectfail
    self.do_ptestrunner()
  File "/home/builder/src/poky/meta/lib/oeqa/runtime/cases/ptest.py", line 120, in do_ptestrunner
    self.fail(failmsg)
AssertionError: 
Failed ptests:
{'openssh': ['banner']}
...
SUMMARY:
core-image-ptest-lttng-tools () - Ran 5 tests in 2289.072s
core-image-ptest-lttng-tools - OK - All required tests passed (successes=3, skipped=1, failures=0, errors=0)
NOTE: recipe core-image-ptest-lttng-tools-1.0-r0: task do_testimage: Succeeded
NOTE: Running noexec task 1405 of 1405 (/home/builder/src/poky/meta/recipes-core/images/core-image-ptest-all.bb:do_testimage)
NOTE: Tasks Summary: Attempted 1405 tasks of which 1304 didn't need to be rerun and all succeeded.

Summary: There was 1 WARNING message.

Cheers,

-Mikko