[PATCH 1/3] oeqa/sdk: run the SDK's own copy of a host tool

[PATCH 1/3] oeqa/sdk: run the SDK's own copy of a host tool

[Trevor Woerner]

ensure_host_package() decides whether a test can run by looking for a
package name in the SDK's host manifest, which establishes neither that
the package is present nor that the command the test runs is the SDK's
own. On an extensible SDK the manifest lists what the shipped sstate
could provide, so a case can pass the check and then run a command that
is not there, or one belonging to the build host.

Ask the SDK for the command instead. An eSDK ships sstate for more than
it installs, so when the command is missing devtool installs the recipe
and the lookup runs again. A command that resolves outside the SDK now
fails rather than reporting a result about the build host.

AI-Generated: codex/claude-opus 5 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
---
 meta/lib/oeqa/sdk/case.py | 96 ++++++++++++++++++++++++++++-----------
 1 file changed, 69 insertions(+), 27 deletions(-)

diff --git a/meta/lib/oeqa/sdk/case.py b/meta/lib/oeqa/sdk/case.py
index 03cfde88ff42..5f269161f4e1 100644
--- a/meta/lib/oeqa/sdk/case.py
+++ b/meta/lib/oeqa/sdk/case.py
@@ -18,38 +18,80 @@ class OESDKTestCase(OETestCase):
                 (self.tc.sdk_env, cmd), shell=True, executable="/bin/bash",
                 stderr=subprocess.STDOUT, universal_newlines=True)
 
-    def ensure_host_package(self, *packages, recipe=None):
+    def find_sdk_binary(self, binaries):
         """
-        Check that the host variation of one of the packages listed is available
-        in the SDK (nativesdk-foo for SDK, foo-native for eSDK). The package is
-        a list for the case where debian-renaming may have occured, and the
-        manifest could contain 'foo' or 'libfoo'.
+        (name, outside) for the first of binaries that resolves.
 
-        If testing an eSDK and the package is not found, then try to install the
-        specified recipe to install it from sstate.
+        name is the command to run when it came from the SDK, and outside
+        lists what resolved beyond it otherwise. Callers run the name
+        returned: an eSDK stages native recipes, which need not name a
+        command the way their nativesdk counterparts do. perl-native keeps
+        its interpreter at bindir/perl-native/perl, reached as nativeperl.
+        """
+        if isinstance(binaries, str):
+            binaries = (binaries,)
+        sdk_root = os.path.dirname(self.tc.sdk_env)
+
+        outside = []
+        for binary in binaries:
+            try:
+                path = self._run("command -v %s" % binary).strip()
+            except subprocess.CalledProcessError:
+                continue
+            if path.startswith(sdk_root + os.sep):
+                return binary, []
+            outside.append("%s is %s" % (binary, path))
+        return None, outside
+
+    def ensure_sdk_binary(self, binaries, packages=()):
         """
+        Return the first of binaries that resolves inside the SDK.
 
-        # In a SDK the manifest is correct. In an eSDK the manifest may be
-        # correct (type=full) or not include packages that exist in sstate but
-        # not installed yet (minimal) so we should try to install the recipe.
-        for package in packages:
-            if isinstance(self.tc, OESDKExtTestContext):
-                package = package + "-native"
-            else:
-                package = "nativesdk-" + package
+        Fail when one resolves outside it, because a result from the build
+        host describes the host and not the SDK. Skip when none resolves.
+        """
+        if isinstance(binaries, str):
+            binaries = (binaries,)
+        if isinstance(packages, str):
+            packages = (packages,)
+        name, outside = self.find_sdk_binary(binaries)
+        if name:
+            return name
+        need = ", ".join(packages) or ", ".join(binaries)
+        if outside:
+            self.fail("Test %s needs one of %s from the SDK, but found %s"
+                      % (self.id(), need, "; ".join(outside)))
+        raise unittest.SkipTest("Test %s needs one of %s: the SDK has no %s"
+                                % (self.id(), need, " or ".join(binaries)))
+
+    def ensure_host_package(self, *packages, recipe=None, binary=None):
+        """
+        Return the command the test should run, having confirmed it comes
+        from the SDK rather than the build host.
 
-            if self.tc.hasHostPackage(package):
-                break
-        else:
-            if isinstance(self.tc, OESDKExtTestContext):
-                recipe = (recipe or packages[0]) + "-native"
-                print("Trying to install %s..." % recipe)
-                try:
-                    self._run('devtool sdk-install %s' % recipe)
-                except subprocess.CalledProcessError:
-                    raise unittest.SkipTest("Test %s needs one of %s" % (self.id(), ", ".join(packages)))
-            else:
-                raise unittest.SkipTest("Test %s needs one of %s" % (self.id(), ", ".join(packages)))
+        An eSDK ships sstate for more than it installs, so a tool can be
+        available without being present. When the command is missing, ask
+        devtool to install the recipe and look again.
+
+        binary names the commands the packages provide, and is needed only
+        where a command is not named after its package: pkgconf provides
+        pkg-config, and an eSDK reaches perl as nativeperl.
+        """
+        binaries = binary or packages
+
+        name, _ = self.find_sdk_binary(binaries)
+        if name:
+            return name
+
+        if isinstance(self.tc, OESDKExtTestContext):
+            install = (recipe or packages[0]) + "-native"
+            print("Trying to install %s..." % install)
+            try:
+                self._run('devtool sdk-install %s' % install)
+            except subprocess.CalledProcessError:
+                pass
+
+        return self.ensure_sdk_binary(binaries, packages)
 
     def ensure_target_package(self, *packages, multilib=False, recipe=None):
         """

[PATCH 2/3] oeqa/sdk: name the commands the host tool cases need

[Trevor Woerner]

These cases identify a tool only by the package carrying it, so they run
whatever the build host provides. gtk3 shows this is not confined to
eSDKs: its glib-2.0-utils check passes on a standard SDK while
glib-compile-resources and glib-mkenums both resolve to /usr/bin.

perl needs two names because the recipe installs it twice:
nativesdk-perl creates ${bindir}/perl, and perl-native creates
${bindir}/perl-native/perl, kept off PATH so recipes running "env perl"
do not find it.

AI-Generated: codex/claude-opus 5 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
---
 meta/lib/oeqa/sdk/cases/gtk3.py    | 3 ++-
 meta/lib/oeqa/sdk/cases/maturin.py | 8 +++++---
 meta/lib/oeqa/sdk/cases/meson.py   | 2 +-
 meta/lib/oeqa/sdk/cases/perl.py    | 4 ++--
 meta/lib/oeqa/sdk/cases/python.py  | 2 +-
 meta/lib/oeqa/sdk/cases/rust.py    | 2 ++
 6 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/meta/lib/oeqa/sdk/cases/gtk3.py b/meta/lib/oeqa/sdk/cases/gtk3.py
index cdaf50ed3842..336df6491c77 100644
--- a/meta/lib/oeqa/sdk/cases/gtk3.py
+++ b/meta/lib/oeqa/sdk/cases/gtk3.py
@@ -18,7 +18,8 @@ class GTK3Test(MesonTestBase):
     def setUp(self):
         super().setUp()
         self.ensure_target_package("gtk+3", "libgtk-3.0", recipe="gtk+3")
-        self.ensure_host_package("glib-2.0-utils", "libglib-2.0-utils", recipe="glib-2.0")
+        self.ensure_host_package("glib-2.0-utils", "libglib-2.0-utils",
+                                 recipe="glib-2.0", binary="glib-compile-resources")
 
     """
     Test that autotools and GTK+ 3 compiles correctly.
diff --git a/meta/lib/oeqa/sdk/cases/maturin.py b/meta/lib/oeqa/sdk/cases/maturin.py
index e3e8edc7819c..1476e6f19a38 100644
--- a/meta/lib/oeqa/sdk/cases/maturin.py
+++ b/meta/lib/oeqa/sdk/cases/maturin.py
@@ -16,10 +16,12 @@ errors_have_output()
 
 class MaturinTest(OESDKTestCase):
     def setUp(self):
-        self.ensure_host_package("python3-maturin")
+        self.ensure_host_package("python3-maturin", binary="maturin")
+        self.python3 = self.ensure_host_package("python3-core", recipe="python3",
+                                                binary="python3")
 
     def test_maturin_list_python(self):
-        out = self._run(r"""python3 -c 'import sys; print(f"{sys.executable}\n{sys.version_info.major}.{sys.version_info.minor}")'""")
+        out = self._run(r"""%s -c 'import sys; print(f"{sys.executable}\n{sys.version_info.major}.{sys.version_info.minor}")'""" % self.python3)
         executable, version = out.splitlines()
 
         output = self._run("maturin list-python")
@@ -32,7 +34,7 @@ class MaturinTest(OESDKTestCase):
 class MaturinDevelopTest(OESDKTestCase):
     def setUp(self):
         machine = self.td.get("MACHINE")
-        self.ensure_host_package("python3-maturin")
+        self.ensure_host_package("python3-maturin", binary="maturin")
 
         if not (
             self.tc.hasHostPackage("packagegroup-rust-cross-canadian-%s" % machine)
diff --git a/meta/lib/oeqa/sdk/cases/meson.py b/meta/lib/oeqa/sdk/cases/meson.py
index 7540d8ba5539..04ab22c73cf1 100644
--- a/meta/lib/oeqa/sdk/cases/meson.py
+++ b/meta/lib/oeqa/sdk/cases/meson.py
@@ -25,7 +25,7 @@ class MesonTestBase(OESDKTestCase):
             self.skipTest(f"{self.id()} does not support eSDK (https://bugzilla.yoctoproject.org/show_bug.cgi?id=15854)")
 
         self.ensure_host_package("meson")
-        self.ensure_host_package("pkgconf")
+        self.ensure_host_package("pkgconf", binary="pkg-config")
 
     def build_meson(self, sourcedir, builddir, installdir=None, options=""):
         """
diff --git a/meta/lib/oeqa/sdk/cases/perl.py b/meta/lib/oeqa/sdk/cases/perl.py
index a72bd2461a6b..330d472f2013 100644
--- a/meta/lib/oeqa/sdk/cases/perl.py
+++ b/meta/lib/oeqa/sdk/cases/perl.py
@@ -11,9 +11,9 @@ errors_have_output()
 
 class PerlTest(OESDKTestCase):
     def setUp(self):
-        self.ensure_host_package("perl")
+        self.perl = self.ensure_host_package("perl", binary=("perl", "nativeperl"))
 
     def test_perl(self):
-        cmd = "perl -e '$_=\"Uryyb, jbeyq\"; tr/a-zA-Z/n-za-mN-ZA-M/;print'"
+        cmd = "%s -e '$_=\"Uryyb, jbeyq\"; tr/a-zA-Z/n-za-mN-ZA-M/;print'" % self.perl
         output = self._run(cmd)
         self.assertEqual(output, "Hello, world")
diff --git a/meta/lib/oeqa/sdk/cases/python.py b/meta/lib/oeqa/sdk/cases/python.py
index b990cd889ad9..cbc7fb69b2d1 100644
--- a/meta/lib/oeqa/sdk/cases/python.py
+++ b/meta/lib/oeqa/sdk/cases/python.py
@@ -11,7 +11,7 @@ errors_have_output()
 
 class Python3Test(OESDKTestCase):
     def setUp(self):
-        self.ensure_host_package("python3-core", recipe="python3")
+        self.ensure_host_package("python3-core", recipe="python3", binary="python3")
 
     def test_python3(self):
         cmd = "python3 -c \"import codecs; print(codecs.encode('Uryyb, jbeyq', 'rot13'))\""
diff --git a/meta/lib/oeqa/sdk/cases/rust.py b/meta/lib/oeqa/sdk/cases/rust.py
index 4b115bebf521..20e05089af1b 100644
--- a/meta/lib/oeqa/sdk/cases/rust.py
+++ b/meta/lib/oeqa/sdk/cases/rust.py
@@ -29,6 +29,7 @@ class RustCompileTest(OESDKTestCase):
         machine = self.td.get("MACHINE")
         if not self.tc.hasHostPackage("packagegroup-rust-cross-canadian-%s" % machine):
             raise unittest.SkipTest("RustCompileTest class: SDK doesn't contain a Rust cross-canadian toolchain")
+        self.ensure_sdk_binary("cargo")
 
     def test_cargo_build(self):
         self._run('cd %s/hello; cargo add zstd' % (self.tc.sdk_dir))
@@ -50,6 +51,7 @@ class RustHostCompileTest(OESDKTestCase):
         machine = self.td.get("MACHINE")
         if not self.tc.hasHostPackage("packagegroup-rust-cross-canadian-%s" % machine):
             raise unittest.SkipTest("RustCompileTest class: SDK doesn't contain a Rust cross-canadian toolchain")
+        self.ensure_sdk_binary("cargo")
 
     def test_cargo_build(self):
         sdksys = self.td.get("SDK_SYS")

[PATCH 3/3] oeqa/sdk: run the Go driver the SDK installs

[Trevor Woerner]

The Go cases confirm go-cross-canadian is in the SDK and then guard on
"which go", which cannot succeed: the recipe installs ${TARGET_PREFIX}go
and leaves the unprefixed binary in its own libdir, off PATH. So the
cases skip on an SDK that does contain Go. Ask for either name and
run what the SDK provides.

Running them for the first time shows the module cases also need
-buildvcs=false, as they build inside a git tree and Go fails stamping
VCS information into the binary.

AI-Generated: codex/claude-opus 5 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
---
 meta/lib/oeqa/sdk/cases/go.py | 24 ++++++++----------------
 1 file changed, 8 insertions(+), 16 deletions(-)

diff --git a/meta/lib/oeqa/sdk/cases/go.py b/meta/lib/oeqa/sdk/cases/go.py
index 693be89e1df7..dfbbc4723cbe 100644
--- a/meta/lib/oeqa/sdk/cases/go.py
+++ b/meta/lib/oeqa/sdk/cases/go.py
@@ -30,21 +30,17 @@ class GoCompileTest(OESDKTestCase):
         if not self.tc.hasHostPackage("go-cross-canadian-%s" % translated_target_arch):
             raise unittest.SkipTest("GoCompileTest class: SDK doesn't contain a Go cross-canadian toolchain")
 
-        # Additional runtime check for go command availability
-        try:
-            self._run('which go')
-        except Exception as e:
-            raise unittest.SkipTest("GoCompileTest class: go command not available: %s" % str(e))
+        self.go = self.ensure_sdk_binary(("go", "${TARGET_PREFIX}go"))
 
     def test_go_build(self):
         """Test Go build command (native compilation)"""
-        self._run('cd %s; go build -o test test.go' % self.tc.sdk_dir)
+        self._run('cd %s; %s build -o test test.go' % (self.tc.sdk_dir, self.go))
 
     def test_go_module(self):
         """Test Go module creation and building"""
         # Create a simple Go module
-        self._run('cd %s; go mod init hello-go' % self.tc.sdk_dir)
-        self._run('cd %s; go build -o hello-go' % self.tc.sdk_dir)
+        self._run('cd %s; %s mod init hello-go' % (self.tc.sdk_dir, self.go))
+        self._run('cd %s; %s build -buildvcs=false -o hello-go' % (self.tc.sdk_dir, self.go))
 
     @classmethod
     def tearDownClass(self):
@@ -68,11 +64,7 @@ class GoHostCompileTest(OESDKTestCase):
         if not self.tc.hasHostPackage("go-cross-canadian-%s" % translated_target_arch):
             raise unittest.SkipTest("GoHostCompileTest class: SDK doesn't contain a Go cross-canadian toolchain")
 
-        # Additional runtime check for go command availability
-        try:
-            self._run('which go')
-        except Exception as e:
-            raise unittest.SkipTest("GoHostCompileTest class: go command not available: %s" % str(e))
+        self.go = self.ensure_sdk_binary(("go", "${TARGET_PREFIX}go"))
 
     def _get_go_arch(self):
         """Get Go architecture from SDK_SYS"""
@@ -85,13 +77,13 @@ class GoHostCompileTest(OESDKTestCase):
     def test_go_cross_compile(self):
         """Test Go cross-compilation for target"""
         goarch = self._get_go_arch()
-        self._run('cd %s; GOOS=linux GOARCH=%s go build -o test-%s test.go' % (self.tc.sdk_dir, goarch, goarch))
+        self._run('cd %s; GOOS=linux GOARCH=%s %s build -o test-%s test.go' % (self.tc.sdk_dir, goarch, self.go, goarch))
 
     def test_go_module_cross_compile(self):
         """Test Go module cross-compilation"""
         goarch = self._get_go_arch()
-        self._run('cd %s; go mod init hello-go' % self.tc.sdk_dir)
-        self._run('cd %s; GOOS=linux GOARCH=%s go build -o hello-go-%s' % (self.tc.sdk_dir, goarch, goarch))
+        self._run('cd %s; %s mod init hello-go' % (self.tc.sdk_dir, self.go))
+        self._run('cd %s; GOOS=linux GOARCH=%s %s build -buildvcs=false -o hello-go-%s' % (self.tc.sdk_dir, goarch, self.go, goarch))
 
     @classmethod
     def tearDownClass(self):

Re: [OE-core] [PATCH 1/3] oeqa/sdk: run the SDK's own copy of a host tool

[Mathieu Dubois-Briand]

On Sun Aug 30, 2026 at 8:36 PM CEST, Trevor Woerner via lists.openembedded.org wrote:
> ensure_host_package() decides whether a test can run by looking for a
> package name in the SDK's host manifest, which establishes neither that
> the package is present nor that the command the test runs is the SDK's
> own. On an extensible SDK the manifest lists what the shipped sstate
> could provide, so a case can pass the check and then run a command that
> is not there, or one belonging to the build host.
>
> Ask the SDK for the command instead. An eSDK ships sstate for more than
> it installs, so when the command is missing devtool installs the recipe
> and the lookup runs again. A command that resolves outside the SDK now
> fails rather than reporting a result about the build host.
>
> AI-Generated: codex/claude-opus 5 (xhigh)
> Signed-off-by: Trevor Woerner <twoerner@gmail.com>
> ---

Hi Trevor,

Thanks for your patch.

It looks like this is breaking the
meta_ide.MetaIDE.test_meta_ide_can_run_sdk_tests test:

2026-08-31 12:29:54,708 - oe-selftest - INFO - meta_ide.MetaIDE.test_meta_ide_can_run_sdk_tests (subunit.RemotedTestCase)
2026-08-31 12:29:54,709 - oe-selftest - INFO -  ... FAIL
...
2026-08-31 12:29:54,709 - oe-selftest - INFO - 12: 3/4 9/21 (127.32s) (0 failed) (meta_ide.MetaIDE.test_meta_ide_can_run_sdk_tests)
2026-08-31 12:29:54,709 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/qemux86-tc/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/meta_ide.py", line 60, in test_meta_ide_can_run_sdk_tests
    bitbake('-c testsdk meta-ide-support')
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/qemux86-tc/build/layers/openembedded-core/meta/lib/oeqa/utils/commands.py", line 236, in bitbake
    return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options)
  File "/srv/pokybuild/yocto-worker/qemux86-tc/build/layers/openembedded-core/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
    raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'bitbake  -c testsdk meta-ide-support' returned non-zero exit status 1:
...

https://autobuilder.yoctoproject.org/valkyrie/#/builders/28/builds/4509
https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/4500
https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/4614

Can you have a look at the issue?

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH 1/3] oeqa/sdk: run the SDK's own copy of a host tool

[Richard Purdie]

On Sun, 2026-08-30 at 14:36 -0400, Trevor Woerner via lists.openembedded.org wrote:
> ensure_host_package() decides whether a test can run by looking for a
> package name in the SDK's host manifest, which establishes neither that
> the package is present nor that the command the test runs is the SDK's
> own. On an extensible SDK the manifest lists what the shipped sstate
> could provide, so a case can pass the check and then run a command that
> is not there, or one belonging to the build host.
> 
> Ask the SDK for the command instead. An eSDK ships sstate for more than
> it installs, so when the command is missing devtool installs the recipe
> and the lookup runs again. A command that resolves outside the SDK now
> fails rather than reporting a result about the build host.
> 
> AI-Generated: codex/claude-opus 5 (xhigh)
> Signed-off-by: Trevor Woerner <twoerner@gmail.com>
> ---
>  meta/lib/oeqa/sdk/case.py | 96 ++++++++++++++++++++++++++++-----------
>  1 file changed, 69 insertions(+), 27 deletions(-)

I'll be honest, this feels/looks wrong. In order to say why, I need to
spend a lot of time looking into the problem more, trying to work out
the right way to solve it and so on.

If I keep doing this, people will just keep feeding me patches like
this and expecting me to give that level of feedback which I simply
can't do, it isn't easy.

So no, this isn't right and no, I don't feel it appropriate to merge
it, sorry.

Cheers,

Richard

Re: [OE-core] [PATCH 1/3] oeqa/sdk: run the SDK's own copy of a host tool

[Trevor Woerner]

On Mon 2026-08-31 @ 08:44:50 PM, Richard Purdie wrote:
> On Sun, 2026-08-30 at 14:36 -0400, Trevor Woerner via lists.openembedded.org wrote:
> > ensure_host_package() decides whether a test can run by looking for a
> > package name in the SDK's host manifest, which establishes neither that
> > the package is present nor that the command the test runs is the SDK's
> > own. On an extensible SDK the manifest lists what the shipped sstate
> > could provide, so a case can pass the check and then run a command that
> > is not there, or one belonging to the build host.
> > 
> > Ask the SDK for the command instead. An eSDK ships sstate for more than
> > it installs, so when the command is missing devtool installs the recipe
> > and the lookup runs again. A command that resolves outside the SDK now
> > fails rather than reporting a result about the build host.
> > 
> > AI-Generated: codex/claude-opus 5 (xhigh)
> > Signed-off-by: Trevor Woerner <twoerner@gmail.com>
> > ---
> >  meta/lib/oeqa/sdk/case.py | 96 ++++++++++++++++++++++++++++-----------
> >  1 file changed, 69 insertions(+), 27 deletions(-)
> 
> I'll be honest, this feels/looks wrong. In order to say why, I need to
> spend a lot of time looking into the problem more, trying to work out
> the right way to solve it and so on.
> 
> If I keep doing this, people will just keep feeding me patches like
> this and expecting me to give that level of feedback which I simply
> can't do, it isn't easy.
> 
> So no, this isn't right and no, I don't feel it appropriate to merge
> it, sorry.

Understood, thank you for the feedback.

> Cheers,
> 
> Richard