[PATCH 1/4] recipetool/create: show more of the license path when it can't be identified

[PATCH 1/4] recipetool/create: show more of the license path when it can't be identified

[Ross Burton]

If there are multiple source trees in a project (incredibly common with
go-mod, for example) then the relative path of the LICENSE file from
the source tree could just be "LICENSE", which is not useful when there
are tens of files across the recipe with that name.

Show the parent directory name too, to clarify which file is unknown.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 scripts/lib/recipetool/create.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index ea2ef5be637..94d52d60772 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -1250,7 +1250,7 @@ def match_licenses(licfiles, srctree, d):
                 license = 'Unknown'
                 logger.info("Please add the following line for '%s' to a 'lib/recipetool/licenses.csv' " \
                     "and replace `Unknown` with the license:\n" \
-                    "%s,Unknown" % (os.path.relpath(licfile, srctree), md5value))
+                    "%s,Unknown" % (os.path.relpath(licfile, srctree + "/.."), md5value))
         if license:
             licenses.append((license, os.path.relpath(licfile, srctree), md5value))
 
-- 
2.43.0

[PATCH 2/4] scripts/scriptutils: silence warning about S not existing in emptysrc

[Ross Burton]

This function creates an emptysrc recipe, but S points to a directory
that doesn't exist and bitbake warns about this.

As it is under the temporary working directory which will be deleted
later, create it to silence the warning.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 scripts/lib/scriptutils.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/scriptutils.py b/scripts/lib/scriptutils.py
index 81f0b01fa53..32e749dbb1a 100644
--- a/scripts/lib/scriptutils.py
+++ b/scripts/lib/scriptutils.py
@@ -182,7 +182,10 @@ def fetch_url(tinfoil, srcuri, srcrev, destdir, logger, preserve_tmp=False, mirr
                 f.write('UNPACKDIR = "%s"\n' % destdir)
 
                 # Set S out of the way so it doesn't get created under the workdir
-                f.write('S = "%s"\n' % os.path.join(tmpdir, 'emptysrc'))
+                s_dir = os.path.join(tmpdir, 'emptysrc')
+                bb.utils.mkdirhier(s_dir)
+                f.write('S = "%s"\n' % s_dir)
+
                 if not mirrors:
                     # We do not need PREMIRRORS since we are almost certainly
                     # fetching new source rather than something that has already
-- 
2.43.0

[PATCH 3/4] lib/oeqa/subprocesstweak: clean up __str__()

[Ross Burton]

Call super().__str__ to get the bulk of the string representation, and
we don't need to guard on output/strerr existing as they always set.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 meta/lib/oeqa/utils/subprocesstweak.py | 13 ++++---------
 1 file changed, 4 insertions(+), 9 deletions(-)

diff --git a/meta/lib/oeqa/utils/subprocesstweak.py b/meta/lib/oeqa/utils/subprocesstweak.py
index 3e43ed547bd..1774513023b 100644
--- a/meta/lib/oeqa/utils/subprocesstweak.py
+++ b/meta/lib/oeqa/utils/subprocesstweak.py
@@ -8,16 +8,11 @@ import subprocess
 class OETestCalledProcessError(subprocess.CalledProcessError):
     def __str__(self):
         def strify(o):
-            if isinstance(o, bytes):
-                return o.decode("utf-8", errors="replace")
-            else:
-                return o
+            return o.decode("utf-8", errors="replace") if isinstance(o, bytes) else o
 
-        s = "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
-        if hasattr(self, "output") and self.output:
-            s = s + "\nStandard Output: " + strify(self.output)
-        if hasattr(self, "stderr") and self.stderr:
-            s = s + "\nStandard Error: " + strify(self.stderr)
+        s = super().__str__()
+        s = s + "\nStandard Output: " + strify(self.output)
+        s = s + "\nStandard Error: " + strify(self.stderr)
         return s
 
 def errors_have_output():
-- 
2.43.0

[PATCH 4/4] oeqa/selftest: add test case for oeqa.utils.subprocesstweak

[Ross Burton]

This class has a monkey-patched CalledProcessError instance that extends
the __str__ method. Add a test case to ensure that it behaves as
expected.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 meta/lib/oeqa/selftest/cases/liboe.py | 35 ++++++++++++++++++++++++++-
 1 file changed, 34 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/selftest/cases/liboe.py b/meta/lib/oeqa/selftest/cases/liboe.py
index d5ffffdcb41..4eab9441435 100644
--- a/meta/lib/oeqa/selftest/cases/liboe.py
+++ b/meta/lib/oeqa/selftest/cases/liboe.py
@@ -9,7 +9,7 @@ from oeqa.utils.commands import get_bb_var, get_bb_vars, bitbake, runCmd
 import oe.path
 import os
 
-class LibOE(OESelftestTestCase):
+class CopyTreeTests(OESelftestTestCase):
 
     @classmethod
     def setUpClass(cls):
@@ -102,3 +102,36 @@ class LibOE(OESelftestTestCase):
         self.assertEqual(dstcnt, len(testfiles), "Number of files in dst (%s) differs from number of files in src(%s)." % (dstcnt, srccnt))
 
         oe.path.remove(testloc)
+
+class SubprocessTests(OESelftestTestCase):
+
+    def test_subprocess_tweak(self):
+        """
+        Test that the string representation of
+        oeqa.utils.subprocesstweak.OETestCalledProcessError includes stdout and
+        stderr, as expected.
+        """
+        script = """
+#! /bin/sh
+echo Ivn fgqbhg | tr '[a-zA-Z]' '[n-za-mN-ZA-M]'
+echo Ivn fgqree | tr '[a-zA-Z]' '[n-za-mN-ZA-M]' >&2
+exit 42
+        """
+
+        import subprocess
+        import unittest.mock
+        from oeqa.utils.subprocesstweak import OETestCalledProcessError
+
+        with self.assertRaises(OETestCalledProcessError) as cm:
+            with unittest.mock.patch("subprocess.CalledProcessError", OETestCalledProcessError):
+                subprocess.run(["bash", "-"], input=script, text=True, capture_output=True, check=True)
+
+        e = cm.exception
+        self.assertEqual(e.returncode, 42)
+        self.assertEqual("Via stdout\n", e.stdout)
+        self.assertEqual("Via stderr\n", e.stderr)
+
+        string = str(e)
+        self.assertIn("exit status 42", string)
+        self.assertIn("Standard Output: Via stdout", string)
+        self.assertIn("Standard Error: Via stderr", string)
-- 
2.43.0

Re: [OE-core] [PATCH 4/4] oeqa/selftest: add test case for oeqa.utils.subprocesstweak

[Mathieu Dubois-Briand]

On Wed Jun 4, 2025 at 4:03 PM CEST, Ross Burton via lists.openembedded.org wrote:
> This class has a monkey-patched CalledProcessError instance that extends
> the __str__ method. Add a test case to ensure that it behaves as
> expected.
>
> Signed-off-by: Ross Burton <ross.burton@arm.com>
> ---

Hi Ross,

I believe we have an issue with this series on selftests:

2025-06-05 11:58:18,605 - oe-selftest - INFO - setUpClass (liboe.CopyTreeTests) [ (subunit.RemotedTestCase)
2025-06-05 11:58:18,606 - oe-selftest - INFO -  ... ERROR
...
2025-06-05 11:58:18,606 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/core/case.py", line 39, in _oeSetUpClass
    clss.setUpClassMethod()
  File "/srv/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/liboe.py", line 16, in setUpClass
    super(LibOE, cls).setUpClass()
          ^^^^^
NameError: name 'LibOE' is not defined

https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/1644
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/1703
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1853

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH 4/4] oeqa/selftest: add test case for oeqa.utils.subprocesstweak

[Ross Burton]

> On 5 Jun 2025, at 19:17, Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> wrote:
> I believe we have an issue with this series on selftests:
> 
> 2025-06-05 11:58:18,605 - oe-selftest - INFO - setUpClass (liboe.CopyTreeTests) [ (subunit.RemotedTestCase)
> 2025-06-05 11:58:18,606 - oe-selftest - INFO -  ... ERROR
> ...
> 2025-06-05 11:58:18,606 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
>  File "/srv/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/core/case.py", line 39, in _oeSetUpClass
>    clss.setUpClassMethod()
>  File "/srv/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/liboe.py", line 16, in setUpClass
>    super(LibOE, cls).setUpClass()
>          ^^^^^
> NameError: name 'LibOE' is not defined

Yes, the v2 fixed this.

Ross