[OE Core] [meta][PATCH 0/3] Create selftest suite for go-vendor.bbclass

[OE Core] [meta][PATCH 0/3] Create selftest suite for go-vendor.bbclass

[John Ripple]

This patch series adds a selftest suit for the go-vendor.bbclass and
fixes a couple of bugs in the go-vendor.bbclass and the selftest suit itself.

John Ripple (3):
  go-vendor.bbclass: Fix test file exclusion and license propagation
  oeqa/selftest: Add go-vendor.bbclass selftest suite
  oeqa/utils/metadata: Handle empty git repos in git_rev_info

 meta/classes/go-vendor.bbclass           |   6 +-
 meta/lib/oeqa/selftest/cases/govendor.py | 264 +++++++++++++++++++++++
 meta/lib/oeqa/utils/metadata.py          |  11 +-
 3 files changed, 274 insertions(+), 7 deletions(-)
 create mode 100644 meta/lib/oeqa/selftest/cases/govendor.py

[OE-core] [meta][PATCH 1/3] go-vendor.bbclass: Fix test file exclusion and license propagation

[John Ripple]

Two bugs in do_go_vendor:

1. The shutil.ignore_patterns glob was "*._test.go" (with an extra dot),
   which never matches standard Go test files (*_test.go). Remove the
   extra dot.

2. The license-propagation block used os.path.join(src, "LICENSE") as the
   copy destination, writing into the already-processed vendor.fetch source
   tree rather than into the vendor destination directory.  Switch to
   os.path.join(dst, "LICENSE") so the LICENSE lands in vendor/ where go
   tools and licence scanners expect it.  Also fix the guard condition from
   not os.path.exists(subdir) (checking an unrelated relative name) to
   not os.path.exists(subdirLicense) (checking the actual target).

Signed-off-by: John Ripple <john.ripple@keysight.com>
---
 meta/classes/go-vendor.bbclass | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/classes/go-vendor.bbclass b/meta/classes/go-vendor.bbclass
index e879d629a8..85a3c1b586 100644
--- a/meta/classes/go-vendor.bbclass
+++ b/meta/classes/go-vendor.bbclass
@@ -141,7 +141,7 @@ python do_go_vendor() {
         shutil.copytree(src, dst, symlinks=True, dirs_exist_ok=True, \
             ignore=shutil.ignore_patterns(".git", \
                                             "vendor", \
-                                            "*._test.go"))
+                                            "*_test.go"))
 
         # If the root directory has a LICENSE file but not the subdir
         # we copy the root license to the sub module since the license
@@ -149,9 +149,9 @@ python do_go_vendor() {
         # see https://go.dev/ref/mod#vcs-license
         if subdir:
             rootdirLicese = os.path.join(rootdir, "LICENSE")
-            subdirLicense = os.path.join(src, "LICENSE")
+            subdirLicense = os.path.join(dst, "LICENSE")
 
-            if not os.path.exists(subdir) and \
+            if not os.path.exists(subdirLicense) and \
                 os.path.exists(rootdirLicese):
                 shutil.copy2(rootdirLicese, subdirLicense)
 

[OE-core] [meta][PATCH 2/3] oeqa/selftest: Add go-vendor.bbclass selftest suite

[John Ripple]

Add govendor.py with two test classes that cover the behaviors fixed and
defined in go-vendor.bbclass:

GoVendorSrcUriTests (offline, ~10 s): unit tests for go_src_uri() parsed
via bb.tinfoil; verifies the basic URI shape, path/pathmajor/subdir/replaces
encoding, and that non-git VCS omits the git-specific nobranch/protocol flags.

GoVendorTaskTests: runs do_go_vendor once against the real recipetool-go-test
project and checks that external deps land in vendor/, *_test.go files are
excluded, pathmajor=/v5 falls back to the repo root when no v5/ subdir exists,
modules.txt is reproduced in vendor/, a local replacement (./is) appears as
a symlink, and the S/src/GO_IMPORT/vendor -> S/src/import/vendor symlink is
created.

Signed-off-by: John Ripple <john.ripple@keysight.com>
---
 meta/lib/oeqa/selftest/cases/govendor.py | 264 +++++++++++++++++++++++
 1 file changed, 264 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/govendor.py

diff --git a/meta/lib/oeqa/selftest/cases/govendor.py b/meta/lib/oeqa/selftest/cases/govendor.py
new file mode 100644
index 0000000000..a56cb93698
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/govendor.py
@@ -0,0 +1,264 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+# Comprehensive tests for go-vendor.bbclass:
+#   GoVendorSrcUriTests  -- unit tests for the go_src_uri() helper (offline, ~10 s)
+#   GoVendorTaskTests    -- integration tests using the real recipetool-go-test project
+#                           (network fetch of git.yoctoproject.org + github.com/godbus/dbus)
+
+import os
+import shutil
+import tempfile
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer
+
+# Module-level temp layer shared by both test classes
+templayerdir = None
+
+
+def setUpModule():
+    global templayerdir
+    templayerdir = tempfile.mkdtemp(prefix='govendorqa')
+    create_temp_layer(templayerdir, 'selftestgovendor')
+    runCmd('bitbake-layers add-layer %s' % templayerdir)
+
+
+def tearDownModule():
+    runCmd('bitbake-layers remove-layer %s' % templayerdir, ignore_status=True)
+    runCmd('rm -rf %s' % templayerdir)
+
+
+# ---------------------------------------------------------------------------
+# Class 1: go_src_uri() helper function
+# ---------------------------------------------------------------------------
+
+class GoVendorSrcUriTests(OESelftestTestCase):
+    """Tests for the go_src_uri() Python helper in go-vendor.bbclass.
+
+    Parses a single fixture recipe via bb.tinfoil (config_only=True) once per
+    class so the function runs inside the real BitBake class-loading machinery.
+    All tests are offline and complete in ~10 s total.
+    """
+
+    @classmethod
+    def setUpClass(cls):
+        import bb.tinfoil
+        super().setUpClass()
+
+        recipe_dir = os.path.join(templayerdir, 'recipes-test-src-uri',
+                                  'go-vendor-src-uri')
+        os.makedirs(recipe_dir, exist_ok=True)
+        recipe_path = os.path.join(recipe_dir, 'go-vendor-src-uri_1.0.bb')
+
+        with open(recipe_path, 'w') as f:
+            f.write(
+                'SUMMARY = "Selftest fixture: go_src_uri parameter variants"\n'
+                'LICENSE = "CLOSED"\n'
+                '\n'
+                'inherit go-vendor\n'
+                '\n'
+                'GO_IMPORT = "example.com/testpkg"\n'
+                '\n'
+                'GO_SRC_URI_BASIC     = "${@go_src_uri(\'github.com/foo/bar\', \'v1.0.0\')}"\n'
+                'GO_SRC_URI_PATH      = "${@go_src_uri(\'github.com/foo/bar\', \'v2.1.0\', path=\'github.com/foo/bar/v2\')}"\n'
+                'GO_SRC_URI_PATHMAJOR = "${@go_src_uri(\'github.com/foo/bar\', \'v2.0.0\', pathmajor=\'/v2\')}"\n'
+                'GO_SRC_URI_SUBDIR    = "${@go_src_uri(\'github.com/foo/bar\', \'v1.0.0\', subdir=\'sub/pkg\')}"\n'
+                'GO_SRC_URI_REPLACES  = "${@go_src_uri(\'github.com/baz/qux\', \'v1.0.0\', replaces=\'../local/baz\')}"\n'
+                'GO_SRC_URI_SVN       = "${@go_src_uri(\'example.com/svnmod\', \'r42\', vcs=\'svn\')}"\n'
+            )
+
+        with bb.tinfoil.Tinfoil() as tinfoil:
+            tinfoil.prepare(config_only=True, quiet=2)
+            rd = tinfoil.parse_recipe_file(recipe_path)
+            cls.bp = rd.getVar('BP')
+            cls.uris = {k: rd.getVar('GO_SRC_URI_' + k)
+                        for k in ('BASIC', 'PATH', 'PATHMAJOR', 'SUBDIR',
+                                  'REPLACES', 'SVN')}
+
+    @classmethod
+    def tearDownClass(cls):
+        runCmd('rm -rf %s/recipes-test-src-uri' % templayerdir,
+               ignore_status=True)
+        super().tearDownClass()
+
+    # --- individual test methods -------------------------------------------
+
+    def test_basic(self):
+        """destsuffix uses expanded BP"""
+        uri = self.uris['BASIC']
+        expected_ds = 'destsuffix=%s/src/import/vendor.fetch/github.com/foo/bar@v1.0.0' % self.bp
+        self.assertIn(expected_ds, uri)
+        self.assertIn('go_module_path=github.com/foo/bar', uri)
+        self.assertIn('nobranch=1', uri)
+        self.assertIn('protocol=https', uri)
+        self.assertIn('is_go_dependency=1', uri)
+
+    def test_path_override(self):
+        """Explicit path= sets go_module_path independently of the repo URL."""
+        uri = self.uris['PATH']
+        self.assertIn('go_module_path=github.com/foo/bar/v2', uri)
+        expected_ds = 'destsuffix=%s/src/import/vendor.fetch/github.com/foo/bar@v2.1.0' % self.bp
+        self.assertIn(expected_ds, uri)
+
+    def test_pathmajor(self):
+        """pathmajor is encoded as go_pathmajor in the URI."""
+        self.assertIn('go_pathmajor=/v2', self.uris['PATHMAJOR'])
+
+    def test_subdir(self):
+        """subdir is encoded as go_subdir in the URI."""
+        self.assertIn('go_subdir=sub/pkg', self.uris['SUBDIR'])
+
+    def test_replaces(self):
+        """replaces is encoded as go_module_replacement in the URI."""
+        self.assertIn('go_module_replacement=../local/baz', self.uris['REPLACES'])
+
+    def test_non_git_vcs(self):
+        """Non-git VCS omits the git-specific nobranch and protocol=https flags."""
+        uri = self.uris['SVN']
+        self.assertIn('svn://example.com/svnmod', uri)
+        self.assertNotIn('nobranch', uri)
+        self.assertNotIn('protocol=https', uri)
+        self.assertIn('is_go_dependency=1', uri)
+
+
+# ---------------------------------------------------------------------------
+# Class 2: do_go_vendor task integration — real project
+# ---------------------------------------------------------------------------
+
+class GoVendorTaskTests(OESelftestTestCase):
+    """Integration tests for do_go_vendor using the real recipetool-go-test project.
+
+    The recipe fetches git.yoctoproject.org/recipetool-go-test (commit
+    c3e213c) as its main source and github.com/godbus/dbus/v5 v5.1.0 as an
+    external vendor dependency.  The project also has a local module
+    replacement (github.com/matryer/is => ./is) that lives inside the main
+    source tree.
+
+    do_go_vendor runs ONCE in setUpClass; individual test methods inspect the
+    resulting vendor directory without re-running bitbake.
+
+    Verified behaviors:
+      - External dependency files are copied to vendor/
+      - *_test.go files are excluded from the vendor copy
+      - pathmajor fallback: when the /v5 subdir is absent, repo root is used
+      - modules.txt is reproduced inside vendor/
+      - Local module replacement creates a relative symlink in vendor/
+      - S/src/GO_IMPORT/vendor symlink is created pointing at S/src/import/vendor
+    """
+
+    # git.yoctoproject.org/recipetool-go-test main branch HEAD
+    SRCREV = 'c3e213c01b6c1406b430df03ef0d1ae77de5d2f7'
+    GO_IMPORT = 'git.yoctoproject.org/recipetool-go-test'
+    RECIPE_NAME = 'govendor-real-test'
+
+    @classmethod
+    def setUpClass(cls):
+        from oeqa.utils import ftools
+        super().setUpClass()
+
+        # overlayfs-user.bb in meta-selftest requires a machine-specific mount
+        # point that this test environment does not provide.  Mask it so that
+        # bitbake can parse the full recipe universe without halting.
+        ftools.append_file(cls.testinc_path,
+            'BBMASK += "meta-selftest/recipes-test/overlayfs-user"')
+
+        recipe_dir = os.path.join(templayerdir, 'recipes-test', cls.RECIPE_NAME)
+        files_dir = os.path.join(recipe_dir, 'files')
+        os.makedirs(files_dir, exist_ok=True)
+
+        # modules.txt for recipetool-go-test at SRCREV:
+        #   - github.com/godbus/dbus/v5 v5.1.0  (external, fetched via go_src_uri)
+        #   - github.com/matryer/is v1.4.1       (local replacement => ./is)
+        with open(os.path.join(files_dir, 'modules.txt'), 'w') as fh:
+            fh.write(
+                '# github.com/godbus/dbus/v5 v5.1.0\n'
+                '## explicit; go 1.18\n'
+                'github.com/godbus/dbus/v5\n'
+                '# github.com/matryer/is v1.4.1 => ./is\n'
+                '## explicit; go 1.14\n'
+                'github.com/matryer/is\n'
+            )
+
+        with open(os.path.join(recipe_dir, '%s_1.0.bb' % cls.RECIPE_NAME), 'w') as fh:
+            fh.write(
+                'SUMMARY = "go-vendor integration test: recipetool-go-test"\n'
+                'LICENSE = "MIT"\n'
+                'LIC_FILES_CHKSUM = "file://src/${GO_IMPORT}/LICENSE;md5=4e3933dd47afbf115e484d11385fb3bd"\n'
+                '\n'
+                'inherit go-vendor\n'
+                '\n'
+                'GO_IMPORT = "%s"\n'
+                '\n'
+                'SRC_URI = "\\\n'
+                '    git://${GO_IMPORT};name=recipetool-go-test;branch=main;protocol=https'
+                ';nobranch=1;destsuffix=${GO_SRCURI_DESTSUFFIX} \\\n'
+                '    ${@go_src_uri(\'github.com/godbus/dbus\', \'v5.1.0\', \\\n'
+                '        path=\'github.com/godbus/dbus/v5\', pathmajor=\'/v5\')} \\\n'
+                '    file://modules.txt \\\n'
+                '    "\n'
+                'SRCREV_FORMAT = "recipetool-go-test"\n'
+                'SRCREV_recipetool-go-test = "%s"\n'
+                'SRCREV_github.com.godbus.dbus.v5 = "e523abc905595cf17fb0001a7d77eaaddfaa216d"\n'
+                % (cls.GO_IMPORT, cls.SRCREV)
+            )
+
+        bitbake('%s -c go_vendor' % cls.RECIPE_NAME)
+        cls.s_dir = get_bb_var('S', cls.RECIPE_NAME)
+        cls.vendor_dir = os.path.join(cls.s_dir, 'src', 'import', 'vendor')
+
+    @classmethod
+    def tearDownClass(cls):
+        runCmd('rm -rf %s/recipes-test' % templayerdir, ignore_status=True)
+        super().tearDownClass()
+
+    # --- test methods (all inspect cls.vendor_dir, no bitbake) ---------------
+
+    def test_external_dep_copied_to_vendor(self):
+        """godbus/dbus/v5 source files are copied into vendor/."""
+        dbus_dir = os.path.join(self.vendor_dir, 'github.com', 'godbus', 'dbus', 'v5')
+        self.assertTrue(os.path.isdir(dbus_dir),
+            'vendor/github.com/godbus/dbus/v5/ not created')
+        self.assertTrue(os.path.isfile(os.path.join(dbus_dir, 'conn.go')),
+            'conn.go not found in vendor/github.com/godbus/dbus/v5/')
+
+    def test_test_files_excluded_from_vendor(self):
+        """*_test.go files from godbus/dbus are not copied into vendor/."""
+        dbus_dir = os.path.join(self.vendor_dir, 'github.com', 'godbus', 'dbus', 'v5')
+        self.assertFalse(
+            os.path.isfile(os.path.join(dbus_dir, 'conn_test.go')),
+            'conn_test.go should have been excluded from vendor by *_test.go pattern')
+
+    def test_pathmajor_falls_back_to_root(self):
+        """godbus/dbus v5.1.0 has no v5/ subdir; pathmajor=/v5 falls back to root."""
+        dbus_dir = os.path.join(self.vendor_dir, 'github.com', 'godbus', 'dbus', 'v5')
+        # auth.go is at the repo root of godbus/dbus, not inside a v5/ subdir.
+        self.assertTrue(os.path.isfile(os.path.join(dbus_dir, 'auth.go')),
+            'auth.go not found; pathmajor=/v5 fallback to repo root did not work')
+
+    def test_modules_txt_in_vendor(self):
+        """modules.txt is reproduced inside vendor/ and lists the external dep."""
+        modules_txt = os.path.join(self.vendor_dir, 'modules.txt')
+        self.assertTrue(os.path.isfile(modules_txt))
+        with open(modules_txt) as fh:
+            content = fh.read()
+        self.assertIn('github.com/godbus/dbus/v5 v5.1.0', content)
+
+    def test_local_replacement_symlink(self):
+        """go.mod local replacement (./is) creates a symlink in vendor/ to S/src/GO_IMPORT/is."""
+        symlink = os.path.join(self.vendor_dir, 'github.com', 'matryer', 'is')
+        self.assertTrue(os.path.islink(symlink),
+            'vendor/github.com/matryer/is should be a symlink for the ./is local replacement')
+        expected_target = os.path.join(self.s_dir, 'src', self.GO_IMPORT, 'is')
+        self.assertEqual(os.path.realpath(symlink), os.path.realpath(expected_target),
+            'Local replacement symlink does not point to S/src/GO_IMPORT/is')
+
+    def test_go_import_vendor_symlink(self):
+        """do_go_vendor creates S/src/GO_IMPORT/vendor -> S/src/import/vendor."""
+        link = os.path.join(self.s_dir, 'src', self.GO_IMPORT, 'vendor')
+        self.assertTrue(os.path.islink(link),
+            'S/src/GO_IMPORT/vendor symlink not created')
+        self.assertEqual(os.path.realpath(link), os.path.realpath(self.vendor_dir),
+            'S/src/GO_IMPORT/vendor does not point to S/src/import/vendor')

[OE-core] [meta][PATCH 3/3] oeqa/utils/metadata: Handle empty git repos in git_rev_info

[John Ripple]

git_rev_info() calls repo.head.commit without guarding against
ValueError, which gitpython raises when HEAD points to an unborn
branch (e.g. a freshly-initialised layer directory with no commits).
Wrap the commit-info block in try/except so metadata collection
continues rather than crashing with a non-zero exit code.

Signed-off-by: John Ripple <john.ripple@keysight.com>
---
 meta/lib/oeqa/utils/metadata.py | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/utils/metadata.py b/meta/lib/oeqa/utils/metadata.py
index b320df67e0..c8df17392d 100644
--- a/meta/lib/oeqa/utils/metadata.py
+++ b/meta/lib/oeqa/utils/metadata.py
@@ -85,12 +85,15 @@ def git_rev_info(path):
         repo = Repo(path, search_parent_directories=True)
     except (InvalidGitRepositoryError, NoSuchPathError):
         return info
-    info['commit'] = repo.head.commit.hexsha
-    info['commit_count'] = repo.head.commit.count()
-    info['commit_time'] = repo.head.commit.committed_date
+    try:
+        info['commit'] = repo.head.commit.hexsha
+        info['commit_count'] = repo.head.commit.count()
+        info['commit_time'] = repo.head.commit.committed_date
+    except (ValueError, TypeError):
+        pass
     try:
         info['branch'] = repo.active_branch.name
-    except TypeError:
+    except (TypeError, ValueError):
         info['branch'] = '(nobranch)'
     return info
 

Re: [OE-core] [meta][PATCH 2/3] oeqa/selftest: Add go-vendor.bbclass selftest suite

[Alexander Kanavin]

On Tue, 7 Jul 2026 at 23:15, John Ripple via lists.openembedded.org
<john.ripple=keysight.com@lists.openembedded.org> wrote:
> +def setUpModule():
> +    global templayerdir
> +    templayerdir = tempfile.mkdtemp(prefix='govendorqa')
> +    create_temp_layer(templayerdir, 'selftestgovendor')
> +    runCmd('bitbake-layers add-layer %s' % templayerdir)
> +
> +
> +def tearDownModule():
> +    runCmd('bitbake-layers remove-layer %s' % templayerdir, ignore_status=True)
> +    runCmd('rm -rf %s' % templayerdir)

There is no need to create a new layer, and place files into it in the
test. There's meta-selftest in oe-core, and it exists exactly for this
purpose. As far as I see they're all static files, and do not get
modified inside the test.

Alex

Re: [OE-core] [meta][PATCH 3/3] oeqa/utils/metadata: Handle empty git repos in git_rev_info

[Alexander Kanavin]

On Tue, 7 Jul 2026 at 23:15, John Ripple via lists.openembedded.org
<john.ripple=keysight.com@lists.openembedded.org> wrote:
> +    try:
> +        info['commit'] = repo.head.commit.hexsha
> +        info['commit_count'] = repo.head.commit.count()
> +        info['commit_time'] = repo.head.commit.committed_date
> +    except (ValueError, TypeError):
> +        pass

Suppressing generic exceptions like this to avoid a specific problem
is problematic, as it can quietly suppress different, real problems.
I'd say any layers handled by this function must have a commit, and if
they don't it's a user error. How did this come up?

Alex