[PATCH 1/6] oeqa runtime ping.py: make run_network_serialdebug() specific to qemu

[PATCH 1/6] oeqa runtime ping.py: make run_network_serialdebug() specific to qemu

[Mikko Rapeli]

Only qemu runner supports self.targets.runner. ssh runner does not
define this and on error path this function causes another exception.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
---
 meta/lib/oeqa/runtime/cases/ping.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/cases/ping.py b/meta/lib/oeqa/runtime/cases/ping.py
index efb91d4cc9da..515d376602bd 100644
--- a/meta/lib/oeqa/runtime/cases/ping.py
+++ b/meta/lib/oeqa/runtime/cases/ping.py
@@ -10,6 +10,7 @@ from time import sleep
 from oeqa.runtime.case import OERuntimeTestCase, run_network_serialdebug
 from oeqa.core.decorator.oetimeout import OETimeout
 from oeqa.core.exception import OEQATimeoutError
+from oeqa.core.target.qemu import OEQemuTarget
 
 class PingTest(OERuntimeTestCase):
 
@@ -36,7 +37,8 @@ class PingTest(OERuntimeTestCase):
                     count = 0
                     sleep(1)
         except OEQATimeoutError:
-            run_network_serialdebug(self.target.runner)
+            if isinstance(self.target, OEQemuTarget):
+                run_network_serialdebug(self.target.runner)
             self.fail("Ping timeout error for address %s, count %s, output: %s" % (self.target.ip, count, output))
         msg = ('Expected 5 consecutive, got %d.\n'
                'ping output is:\n%s' % (count,output))
-- 
2.43.0

[PATCH 2/6] oeqa runtime ssh.py: make run_network_serialdebug() specific to qemu

[Mikko Rapeli]

ssh and serial port runner targets don't support separate
serial port communication channels.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
---
 meta/lib/oeqa/runtime/cases/ssh.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/cases/ssh.py b/meta/lib/oeqa/runtime/cases/ssh.py
index b632a29a0178..7fdad564a2d7 100644
--- a/meta/lib/oeqa/runtime/cases/ssh.py
+++ b/meta/lib/oeqa/runtime/cases/ssh.py
@@ -10,6 +10,7 @@ import signal
 from oeqa.runtime.case import OERuntimeTestCase, run_network_serialdebug
 from oeqa.core.decorator.depends import OETestDepends
 from oeqa.runtime.decorator.package import OEHasPackage
+from oeqa.core.target.qemu import OEQemuTarget
 
 class SSHTest(OERuntimeTestCase):
 
@@ -32,7 +33,8 @@ class SSHTest(OERuntimeTestCase):
               time.sleep(5)
               continue
           else:
-              run_network_serialdebug(self.target.runner)
+              if isinstance(self.target, OEQemuTarget):
+                  run_network_serialdebug(self.target.runner)
               self.fail("uname failed with \"%s\" (exit code %s)" % (output, status))
         if status != 0:
             self.fail("ssh failed with \"%s\" (exit code %s)" % (output, status))
-- 
2.43.0

[PATCH 3/6] oeqa runtime apt.py: check errors and fix quoting

[Mikko Rapeli]

Check that shell commands succeeded and fix quoting
warning:

core-image-minimal/lib/oeqa/runtime/cases/apt.py:44: SyntaxWarning: invalid escape sequence '\['
  self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's|\[trusted=yes\] http://bogus_ip:bogus_port|%s|g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
---
 meta/lib/oeqa/runtime/cases/apt.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/runtime/cases/apt.py b/meta/lib/oeqa/runtime/cases/apt.py
index 8000645843aa..b7cda0095394 100644
--- a/meta/lib/oeqa/runtime/cases/apt.py
+++ b/meta/lib/oeqa/runtime/cases/apt.py
@@ -36,12 +36,12 @@ class AptRepoTest(AptTest):
     def setup_source_config_for_package_install(self):
         apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
         apt_get_sourceslist_dir = '/etc/apt/'
-        self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s/all ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
+        self.target.run('cd %s && echo "deb [ allow-insecure=yes ] %s/all ./" > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
 
     def setup_source_config_for_package_install_signed(self):
         apt_get_source_server = 'http://%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
         apt_get_sourceslist_dir = '/etc/apt/'
-        self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's|\[trusted=yes\] http://bogus_ip:bogus_port|%s|g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
+        self.target.run("cd %s && cp sources.list sources.list.bak && sed -i 's|[trusted=yes] http://bogus_ip:bogus_port|%s|g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
 
     def cleanup_source_config_for_package_install(self):
         apt_get_sourceslist_dir = '/etc/apt/'
@@ -49,12 +49,12 @@ class AptRepoTest(AptTest):
 
     def cleanup_source_config_for_package_install_signed(self):
         apt_get_sourceslist_dir = '/etc/apt/'
-        self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
+        self.target.run('cd %s && mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
 
     def setup_key(self):
         # the key is found on the target /etc/pki/packagefeed-gpg/
         # named PACKAGEFEED-GPG-KEY-poky-branch
-        self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
+        self.target.run('cd %s && apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
 
     @skipIfNotFeature('package-management',
                       'Test requires package-management to be in IMAGE_FEATURES')
-- 
2.43.0

[PATCH 4/6] oeqa context.py: use TEST_SUITES if set

[Mikko Rapeli]

If build target has set TEST_SUITES, then that should
be the default test modules to execute. Fixes testexport.bbclass
to run same tests as testimage.bbclass which already
uses TEST_SUITES.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
---
 meta/lib/oeqa/core/context.py | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/lib/oeqa/core/context.py b/meta/lib/oeqa/core/context.py
index 9313271f5865..a0eb16e71602 100644
--- a/meta/lib/oeqa/core/context.py
+++ b/meta/lib/oeqa/core/context.py
@@ -179,9 +179,14 @@ class OETestContextExecutor(object):
         else:
             self.tc_kwargs['init']['td'] = {}
 
+        # Run image specific TEST_SUITE like testimage.bbclass by default
+        test_suites = self.tc_kwargs['init']['td'].get("TEST_SUITES").split()
+
         if args.run_tests:
             self.tc_kwargs['load']['modules'] = args.run_tests
             self.tc_kwargs['load']['modules_required'] = args.run_tests
+        elif test_suites:
+            self.tc_kwargs['load']['modules'] = test_suites
         else:
             self.tc_kwargs['load']['modules'] = []
 
-- 
2.43.0

[PATCH 5/6] testexport.bbclass oe-test: capture tests and data from all layers

[Mikko Rapeli]

testexport.bbclass only copied files from core layer to
the testexport.tar.gz to run tests. Then it removed
all tests and files which were not specified in
TEST_SUITES variable.

Stop removing files to include for example parselogs.py
test data files which are machine and/or layer specific.
TEST_SUITES variable is now read from build time exported
data store when running tests.

Then copy oeqa runtime files from all layers so that tests and
test data like parselogs ignore files will be provided
to testexport.tar.gz.

Adapt oe-test script to find "lib" directories from
the new structure with layer specific paths.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
---
 meta/classes-recipe/testexport.bbclass | 43 +++++++++++++-------------
 scripts/oe-test                        | 12 ++++---
 2 files changed, 30 insertions(+), 25 deletions(-)

diff --git a/meta/classes-recipe/testexport.bbclass b/meta/classes-recipe/testexport.bbclass
index cc4088c71a13..843d777e3bb9 100644
--- a/meta/classes-recipe/testexport.bbclass
+++ b/meta/classes-recipe/testexport.bbclass
@@ -85,6 +85,7 @@ def copy_needed_files(d, tc):
 
     export_path = d.getVar('TEST_EXPORT_DIR')
     corebase_path = d.getVar('COREBASE')
+    bblayers = d.getVar('BBLAYERS').split()
 
     # Clean everything before starting
     oe.path.remove(export_path)
@@ -92,17 +93,11 @@ def copy_needed_files(d, tc):
 
     # The source of files to copy are relative to 'COREBASE' directory
     # The destination is relative to 'TEST_EXPORT_DIR'
-    # Because we are squashing the libraries, we need to remove
-    # the layer/script directory
-    files_to_copy = [ os.path.join('meta', 'lib', 'oeqa', 'core'),
-                      os.path.join('meta', 'lib', 'oeqa', 'runtime'),
-                      os.path.join('meta', 'lib', 'oeqa', 'files'),
-                      os.path.join('meta', 'lib', 'oeqa', 'utils'),
-                      os.path.join('scripts', 'oe-test'),
+    # core files/dirs first
+    core_files_to_copy = [ os.path.join('scripts', 'oe-test'),
                       os.path.join('scripts', 'lib', 'argparse_oe.py'),
                       os.path.join('scripts', 'lib', 'scriptutils.py'), ]
-
-    for f in files_to_copy:
+    for f in core_files_to_copy:
         src = os.path.join(corebase_path, f)
         dst = os.path.join(export_path, f.split('/', 1)[-1])
         if os.path.isdir(src):
@@ -110,18 +105,21 @@ def copy_needed_files(d, tc):
         else:
             shutil.copy2(src, dst)
 
-    # Remove cases and just copy the ones specified
-    cases_path = os.path.join(export_path, 'lib', 'oeqa', 'runtime', 'cases')
-    oe.path.remove(cases_path)
-    bb.utils.mkdirhier(cases_path)
-    test_paths = get_runtime_paths(d)
-    test_modules = d.getVar('TEST_SUITES').split()
-    tc.loadTests(test_paths, modules=test_modules)
-    for f in getSuiteCasesFiles(tc.suites):
-        shutil.copy2(f, cases_path)
-        json_file = _get_json_file(f)
-        if json_file:
-            shutil.copy2(json_file, cases_path)
+    # layer specific files/dirs
+    layer_files_to_copy = [ os.path.join('lib', 'oeqa', 'core'),
+                      os.path.join('lib', 'oeqa', 'runtime'),
+                      os.path.join('lib', 'oeqa', 'files'),
+                      os.path.join('lib', 'oeqa', 'utils'),]
+    for layer in bblayers:
+        meta = os.path.basename(layer)
+        for f in layer_files_to_copy:
+            src = os.path.join(layer, f)
+            dst = os.path.join(export_path, meta, f)
+            if os.path.exists(src):
+                if os.path.isdir(src):
+                    oe.path.copytree(src, dst)
+                else:
+                    shutil.copy2(src, dst)
 
     # Copy test data
     image_name = ("%s/%s" % (d.getVar('DEPLOY_DIR_IMAGE'),
@@ -142,6 +140,9 @@ def copy_needed_files(d, tc):
     testexport_create_tarball(d, "testexport.tar.gz", d.getVar("TEST_EXPORT_DIR"))
 
     # Copy packages needed for runtime testing
+    test_paths = get_runtime_paths(d)
+    test_modules = d.getVar('TEST_SUITES').split()
+    tc.loadTests(test_paths, modules=test_modules)
     package_extraction(d, tc.suites)
     test_pkg_dir = d.getVar("TEST_NEEDED_PACKAGES_DIR")
     if os.path.isdir(test_pkg_dir) and os.listdir(test_pkg_dir):
diff --git a/scripts/oe-test b/scripts/oe-test
index 55985b0b2453..efb83c3e7809 100755
--- a/scripts/oe-test
+++ b/scripts/oe-test
@@ -7,14 +7,18 @@
 # SPDX-License-Identifier: MIT
 #
 
-import os
-import sys
 import argparse
+import glob
 import logging
+import os
+import sys
 
 scripts_path = os.path.dirname(os.path.realpath(__file__))
-lib_path = scripts_path + '/lib'
-sys.path = sys.path + [lib_path]
+lib_path = os.path.join(scripts_path, 'lib')
+sys.path.append(lib_path)
+meta_lib_paths = glob.glob(scripts_path + '/*/lib', root_dir=scripts_path, recursive=True)
+for p in meta_lib_paths:
+    sys.path.append(p)
 import argparse_oe
 import scriptutils
 
-- 
2.43.0

[PATCH 6/6] oeqa parselogs: add parselogs-ignores-genericarm64.txt

[Mikko Rapeli]

With safe to ignore pinctl and initrd warnings from
AMD kv260 boot.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
---
 .../lib/oeqa/runtime/cases/parselogs-ignores-genericarm64.txt | 4 ++++
 1 file changed, 4 insertions(+)
 create mode 100644 meta-yocto-bsp/lib/oeqa/runtime/cases/parselogs-ignores-genericarm64.txt

diff --git a/meta-yocto-bsp/lib/oeqa/runtime/cases/parselogs-ignores-genericarm64.txt b/meta-yocto-bsp/lib/oeqa/runtime/cases/parselogs-ignores-genericarm64.txt
new file mode 100644
index 000000000000..1e8b1f4b7d1b
--- /dev/null
+++ b/meta-yocto-bsp/lib/oeqa/runtime/cases/parselogs-ignores-genericarm64.txt
@@ -0,0 +1,4 @@
+# safe to ignore minor firmware issue
+zynqmp-pinctrl firmware:zynqmp-firmware:pinctrl: failed to set: pin
+# safe to ignore initrd warning
+Can't lookup blockdev
-- 
2.43.0

Patchtest results for [PATCH 6/6] oeqa parselogs: add parselogs-ignores-genericarm64.txt

[patchtest]

[-- Attachment #1: Type: text/plain, Size: 3229 bytes --]

Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch /home/patchtest/share/mboxes/6-6-oeqa-parselogs-add-parselogs-ignores-genericarm64.txt.patch

FAIL: test target mailing list: Series sent to the wrong mailing list or some patches from the series correspond to different mailing lists (test_mbox.TestMbox.test_target_mailing_list)

PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)

SKIP: pretest pylint: No python related patches, skipping test (test_python_pylint.PyLint.pretest_pylint)
SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: No new CVE patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new CVE patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new CVE patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: No python related patches, skipping test (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!

Re: [OE-core] [PATCH 1/6] oeqa runtime ping.py: make run_network_serialdebug() specific to qemu

[Antonin Godard]

On Fri Jun 27, 2025 at 8:29 AM CEST, Mikko Rapeli via lists.openembedded.org wrote:
> Only qemu runner supports self.targets.runner. ssh runner does not
> define this and on error path this function causes another exception.
>
> Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> ---
>  meta/lib/oeqa/runtime/cases/ping.py | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/meta/lib/oeqa/runtime/cases/ping.py b/meta/lib/oeqa/runtime/cases/ping.py
> index efb91d4cc9da..515d376602bd 100644
> --- a/meta/lib/oeqa/runtime/cases/ping.py
> +++ b/meta/lib/oeqa/runtime/cases/ping.py
> @@ -10,6 +10,7 @@ from time import sleep
>  from oeqa.runtime.case import OERuntimeTestCase, run_network_serialdebug
>  from oeqa.core.decorator.oetimeout import OETimeout
>  from oeqa.core.exception import OEQATimeoutError
> +from oeqa.core.target.qemu import OEQemuTarget
>  
>  class PingTest(OERuntimeTestCase):
>  
> @@ -36,7 +37,8 @@ class PingTest(OERuntimeTestCase):
>                      count = 0
>                      sleep(1)
>          except OEQATimeoutError:
> -            run_network_serialdebug(self.target.runner)
> +            if isinstance(self.target, OEQemuTarget):
> +                run_network_serialdebug(self.target.runner)
>              self.fail("Ping timeout error for address %s, count %s, output: %s" % (self.target.ip, count, output))
>          msg = ('Expected 5 consecutive, got %d.\n'
>                 'ping output is:\n%s' % (count,output))

Hi,

I think this series is causing issues on the autobuilder:

Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/qemuarm-tc/build/scripts/oe-selftest", line 58, in <module>
    ret = main()
  File "/srv/pokybuild/yocto-worker/qemuarm-tc/build/scripts/oe-selftest", line 45, in main
    results = args.func(logger, args)
  File "/srv/pokybuild/yocto-worker/qemuarm-tc/build/meta/lib/oeqa/selftest/context.py", line 413, in run
    self._process_args(logger, args)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/qemuarm-tc/build/meta/lib/oeqa/selftest/context.py", line 260, in _process_args
    super(OESelftestTestContextExecutor, self)._process_args(logger, args)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/qemuarm-tc/build/meta/lib/oeqa/core/context.py", line 183, in _process_args
    test_suites = self.tc_kwargs['init']['td'].get("TEST_SUITES").split()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'split'

Can you have a look?
https://autobuilder.yoctoproject.org/valkyrie/#/builders/42/builds/1848

Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH 5/6] testexport.bbclass oe-test: capture tests and data from all layers

[Richard Purdie]

On Fri, 2025-06-27 at 09:30 +0300, Mikko Rapeli via lists.openembedded.org wrote:
> testexport.bbclass only copied files from core layer to
> the testexport.tar.gz to run tests. Then it removed
> all tests and files which were not specified in
> TEST_SUITES variable.
> 
> Stop removing files to include for example parselogs.py
> test data files which are machine and/or layer specific.
> TEST_SUITES variable is now read from build time exported
> data store when running tests.
> 
> Then copy oeqa runtime files from all layers so that tests and
> test data like parselogs ignore files will be provided
> to testexport.tar.gz.
> 
> Adapt oe-test script to find "lib" directories from
> the new structure with layer specific paths.
> 
> Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> ---
>  meta/classes-recipe/testexport.bbclass | 43 +++++++++++++-------------
>  scripts/oe-test                        | 12 ++++---
>  2 files changed, 30 insertions(+), 25 deletions(-)

The patch does things which aren't really spelt out in the commit
message. This change removes the target specific filtering, putting all
tests into the output and not just the ones applicable to the target in
question.

Once you do that, "testexport" may as well be a script rather than a
task as it is no longer target specific.

As I've said elsewhere, I'm not against doing it but the commit message
should be clear, and it does raise the question of whether we just jump
to that external script and remove this class?

Cheers,

Richard

Re: [OE-core] [PATCH 1/6] oeqa runtime ping.py: make run_network_serialdebug() specific to qemu

[Richard Purdie]

On Fri, 2025-06-27 at 09:29 +0300, Mikko Rapeli via lists.openembedded.org wrote:
> Only qemu runner supports self.targets.runner. ssh runner does not
> define this and on error path this function causes another exception.
> 
> Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> ---
>  meta/lib/oeqa/runtime/cases/ping.py | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/lib/oeqa/runtime/cases/ping.py b/meta/lib/oeqa/runtime/cases/ping.py
> index efb91d4cc9da..515d376602bd 100644
> --- a/meta/lib/oeqa/runtime/cases/ping.py
> +++ b/meta/lib/oeqa/runtime/cases/ping.py
> @@ -10,6 +10,7 @@ from time import sleep
>  from oeqa.runtime.case import OERuntimeTestCase, run_network_serialdebug
>  from oeqa.core.decorator.oetimeout import OETimeout
>  from oeqa.core.exception import OEQATimeoutError
> +from oeqa.core.target.qemu import OEQemuTarget
>  
>  class PingTest(OERuntimeTestCase):
>  
> @@ -36,7 +37,8 @@ class PingTest(OERuntimeTestCase):
>                      count = 0
>                      sleep(1)
>          except OEQATimeoutError:
> -            run_network_serialdebug(self.target.runner)
> +            if isinstance(self.target, OEQemuTarget):
> +                run_network_serialdebug(self.target.runner)
>              self.fail("Ping timeout error for address %s, count %s, output: %s" % (self.target.ip, count, output))
>          msg = ('Expected 5 consecutive, got %d.\n'
>                 'ping output is:\n%s' % (count,output))

Rather than make the tests target runner specific, should we put a
dummy function in the main class, then backends can support this if
they can/want to?

Cheers,

Richard

Re: [OE-core] [PATCH 5/6] testexport.bbclass oe-test: capture tests and data from all layers

[Mikko Rapeli]

Hi,

On Sun, Jun 29, 2025 at 08:05:35AM +0100, Richard Purdie wrote:
> On Fri, 2025-06-27 at 09:30 +0300, Mikko Rapeli via lists.openembedded.org wrote:
> > testexport.bbclass only copied files from core layer to
> > the testexport.tar.gz to run tests. Then it removed
> > all tests and files which were not specified in
> > TEST_SUITES variable.
> > 
> > Stop removing files to include for example parselogs.py
> > test data files which are machine and/or layer specific.
> > TEST_SUITES variable is now read from build time exported
> > data store when running tests.
> > 
> > Then copy oeqa runtime files from all layers so that tests and
> > test data like parselogs ignore files will be provided
> > to testexport.tar.gz.
> > 
> > Adapt oe-test script to find "lib" directories from
> > the new structure with layer specific paths.
> > 
> > Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> > ---
> > �meta/classes-recipe/testexport.bbclass | 43 +++++++++++++-------------
> > �scripts/oe-test����������������������� | 12 ++++---
> > �2 files changed, 30 insertions(+), 25 deletions(-)
> 
> The patch does things which aren't really spelt out in the commit
> message. This change removes the target specific filtering, putting all
> tests into the output and not just the ones applicable to the target in
> question.

Sorry, tried to describe this but failed. Yes, all tests are now exported,
also the ones not needed by the current image target because the TEST_SUITES
variable is read from exported data at test run time.
 
> Once you do that, "testexport" may as well be a script rather than a
> task as it is no longer target specific.

For the test python files and data, yes, but not for the data file
which actually describes what tests to run.

> As I've said elsewhere, I'm not against doing it but the commit message
> should be clear, and it does raise the question of whether we just jump
> to that external script and remove this class?

We still need to export the per-image manifest and testdata. If those
are exported elsewhere, then these all could be an external script to
create a tar ball to run the configured tests.

Cheers,

-Mikko