[PATCH v2 0/3] Create selftest suite for go-vendor.bbclass

[PATCH v2 0/3] Create selftest suite for go-vendor.bbclass

[John Ripple]

This patch series adds a selftest suit for the go-vendor.bbclass and
fixes a couple of bugs in the go-vendor.bbclass.

John Ripple (3):
  go-vendor.bbclass: Fix test file exclusion and license propagation
  oeqa/selftest: Add go-vendor.bbclass selftest suite
  go-vendor.bbclass: Run vendor_unlink task under fakeroot

 .../recipes-test/govendor/files/modules.txt   |   6 +
 .../govendor/go-vendor-src-uri_1.0.bb         |  13 ++
 .../govendor/govendor-real-test_1.0.bb        |  20 +++
 meta/classes/go-vendor.bbclass                |   9 +-
 meta/lib/oeqa/selftest/cases/govendor.py      | 169 ++++++++++++++++++
 5 files changed, 213 insertions(+), 4 deletions(-)
 create mode 100644 meta-selftest/recipes-test/govendor/files/modules.txt
 create mode 100644 meta-selftest/recipes-test/govendor/go-vendor-src-uri_1.0.bb
 create mode 100644 meta-selftest/recipes-test/govendor/govendor-real-test_1.0.bb
 create mode 100644 meta/lib/oeqa/selftest/cases/govendor.py

[meta][PATCH v2 1/3] go-vendor.bbclass: Fix test file exclusion and license propagation

[John Ripple]

Two bugs in do_go_vendor:

1. The shutil.ignore_patterns glob was "*._test.go" (with an extra dot),
   which never matches standard Go test files (*_test.go). Remove the
   extra dot.

2. The license-propagation block used os.path.join(src, "LICENSE") as the
   copy destination, writing into the already-processed vendor.fetch source
   tree rather than into the vendor destination directory.  Switch to
   os.path.join(dst, "LICENSE") so the LICENSE lands in vendor/ where go
   tools and licence scanners expect it.  Also fix the guard condition from
   not os.path.exists(subdir) (checking an unrelated relative name) to
   not os.path.exists(subdirLicense) (checking the actual target).

Signed-off-by: John Ripple <john.ripple@keysight.com>
---
 meta/classes/go-vendor.bbclass | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/classes/go-vendor.bbclass b/meta/classes/go-vendor.bbclass
index e879d629a8..85a3c1b586 100644
--- a/meta/classes/go-vendor.bbclass
+++ b/meta/classes/go-vendor.bbclass
@@ -141,7 +141,7 @@ python do_go_vendor() {
         shutil.copytree(src, dst, symlinks=True, dirs_exist_ok=True, \
             ignore=shutil.ignore_patterns(".git", \
                                             "vendor", \
-                                            "*._test.go"))
+                                            "*_test.go"))
 
         # If the root directory has a LICENSE file but not the subdir
         # we copy the root license to the sub module since the license
@@ -149,9 +149,9 @@ python do_go_vendor() {
         # see https://go.dev/ref/mod#vcs-license
         if subdir:
             rootdirLicese = os.path.join(rootdir, "LICENSE")
-            subdirLicense = os.path.join(src, "LICENSE")
+            subdirLicense = os.path.join(dst, "LICENSE")
 
-            if not os.path.exists(subdir) and \
+            if not os.path.exists(subdirLicense) and \
                 os.path.exists(rootdirLicese):
                 shutil.copy2(rootdirLicese, subdirLicense)
 

[meta][meta-selftest][PATCH v2 2/3] oeqa/selftest: Add go-vendor.bbclass selftest suite

[John Ripple]

Add govendor.py with two test classes that cover the behaviors fixed and
defined in go-vendor.bbclass:

GoVendorSrcUriTests (offline, ~10 s): unit tests for go_src_uri() parsed
via bb.tinfoil; verifies the basic URI shape, path/pathmajor/subdir/replaces
encoding, and that non-git VCS omits the git-specific nobranch/protocol flags.

GoVendorTaskTests: runs do_go_vendor once against the real recipetool-go-test
project and checks that external deps land in vendor/, *_test.go files are
excluded, pathmajor=/v5 falls back to the repo root when no v5/ subdir exists,
modules.txt is reproduced in vendor/, a local replacement (./is) appears as
a symlink, and the S/src/GO_IMPORT/vendor -> S/src/import/vendor symlink is
created.

Signed-off-by: John Ripple <john.ripple@keysight.com>
---
 .../recipes-test/govendor/files/modules.txt   |   6 +
 .../govendor/go-vendor-src-uri_1.0.bb         |  13 ++
 .../govendor/govendor-real-test_1.0.bb        |  20 +++
 meta/lib/oeqa/selftest/cases/govendor.py      | 169 ++++++++++++++++++
 4 files changed, 208 insertions(+)
 create mode 100644 meta-selftest/recipes-test/govendor/files/modules.txt
 create mode 100644 meta-selftest/recipes-test/govendor/go-vendor-src-uri_1.0.bb
 create mode 100644 meta-selftest/recipes-test/govendor/govendor-real-test_1.0.bb
 create mode 100644 meta/lib/oeqa/selftest/cases/govendor.py

diff --git a/meta-selftest/recipes-test/govendor/files/modules.txt b/meta-selftest/recipes-test/govendor/files/modules.txt
new file mode 100644
index 0000000000..f958aa80af
--- /dev/null
+++ b/meta-selftest/recipes-test/govendor/files/modules.txt
@@ -0,0 +1,6 @@
+# github.com/godbus/dbus/v5 v5.1.0
+## explicit; go 1.18
+github.com/godbus/dbus/v5
+# github.com/matryer/is v1.4.1 => ./is
+## explicit; go 1.14
+github.com/matryer/is
diff --git a/meta-selftest/recipes-test/govendor/go-vendor-src-uri_1.0.bb b/meta-selftest/recipes-test/govendor/go-vendor-src-uri_1.0.bb
new file mode 100644
index 0000000000..9623989f83
--- /dev/null
+++ b/meta-selftest/recipes-test/govendor/go-vendor-src-uri_1.0.bb
@@ -0,0 +1,13 @@
+SUMMARY = "Selftest fixture: go_src_uri parameter variants"
+LICENSE = "CLOSED"
+
+inherit go-vendor
+
+GO_IMPORT = "example.com/testpkg"
+
+GO_SRC_URI_BASIC     = "${@go_src_uri('github.com/foo/bar', 'v1.0.0')}"
+GO_SRC_URI_PATH      = "${@go_src_uri('github.com/foo/bar', 'v2.1.0', path='github.com/foo/bar/v2')}"
+GO_SRC_URI_PATHMAJOR = "${@go_src_uri('github.com/foo/bar', 'v2.0.0', pathmajor='/v2')}"
+GO_SRC_URI_SUBDIR    = "${@go_src_uri('github.com/foo/bar', 'v1.0.0', subdir='sub/pkg')}"
+GO_SRC_URI_REPLACES  = "${@go_src_uri('github.com/baz/qux', 'v1.0.0', replaces='../local/baz')}"
+GO_SRC_URI_SVN       = "${@go_src_uri('example.com/svnmod', 'r42', vcs='svn')}"
diff --git a/meta-selftest/recipes-test/govendor/govendor-real-test_1.0.bb b/meta-selftest/recipes-test/govendor/govendor-real-test_1.0.bb
new file mode 100644
index 0000000000..a8741a872f
--- /dev/null
+++ b/meta-selftest/recipes-test/govendor/govendor-real-test_1.0.bb
@@ -0,0 +1,20 @@
+SUMMARY = "go-vendor integration test: recipetool-go-test"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://src/${GO_IMPORT}/LICENSE;md5=4e3933dd47afbf115e484d11385fb3bd"
+
+inherit go-vendor
+
+GO_IMPORT = "git.yoctoproject.org/recipetool-go-test"
+
+# modules.txt for recipetool-go-test at SRCREV:
+#   - github.com/godbus/dbus/v5 v5.1.0  (external, fetched via go_src_uri)
+#   - github.com/matryer/is v1.4.1       (local replacement => ./is)
+SRC_URI = "\
+    git://${GO_IMPORT};name=recipetool-go-test;branch=main;protocol=https;nobranch=1;destsuffix=${GO_SRCURI_DESTSUFFIX} \
+    ${@go_src_uri('github.com/godbus/dbus', 'v5.1.0', path='github.com/godbus/dbus/v5', pathmajor='/v5')} \
+    file://modules.txt \
+    "
+SRCREV_FORMAT = "recipetool-go-test"
+# git.yoctoproject.org/recipetool-go-test main branch HEAD
+SRCREV_recipetool-go-test = "c3e213c01b6c1406b430df03ef0d1ae77de5d2f7"
+SRCREV_github.com.godbus.dbus.v5 = "e523abc905595cf17fb0001a7d77eaaddfaa216d"
diff --git a/meta/lib/oeqa/selftest/cases/govendor.py b/meta/lib/oeqa/selftest/cases/govendor.py
new file mode 100644
index 0000000000..65ca8a1145
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/govendor.py
@@ -0,0 +1,169 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+# Comprehensive tests for go-vendor.bbclass:
+#   GoVendorSrcUriTests  -- unit tests for the go_src_uri() helper (offline, ~10 s)
+#   GoVendorTaskTests    -- integration tests using the real recipetool-go-test project
+#                           (network fetch of git.yoctoproject.org + github.com/godbus/dbus)
+
+import os
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_var
+
+# ---------------------------------------------------------------------------
+# Class 1: go_src_uri() helper function
+# ---------------------------------------------------------------------------
+
+class GoVendorSrcUriTests(OESelftestTestCase):
+    """Tests for the go_src_uri() Python helper in go-vendor.bbclass.
+
+    Parses the go-vendor-src-uri fixture recipe (meta-selftest/recipes-test/
+    govendor/go-vendor-src-uri_1.0.bb) via bb.tinfoil (config_only=True) once
+    per class so the function runs inside the real BitBake class-loading
+    machinery.  All tests are offline and complete in ~10 s total.
+    """
+
+    @classmethod
+    def setUpClass(cls):
+        import bb.tinfoil
+        super().setUpClass()
+
+        recipe_path = os.path.join(cls.testlayer_path, 'recipes-test',
+                                    'govendor', 'go-vendor-src-uri_1.0.bb')
+
+        with bb.tinfoil.Tinfoil() as tinfoil:
+            tinfoil.prepare(config_only=True, quiet=2)
+            rd = tinfoil.parse_recipe_file(recipe_path)
+            cls.bp = rd.getVar('BP')
+            cls.uris = {k: rd.getVar('GO_SRC_URI_' + k)
+                        for k in ('BASIC', 'PATH', 'PATHMAJOR', 'SUBDIR',
+                                  'REPLACES', 'SVN')}
+
+    # --- individual test methods -------------------------------------------
+
+    def test_basic(self):
+        """destsuffix uses expanded BP"""
+        uri = self.uris['BASIC']
+        expected_ds = 'destsuffix=%s/src/import/vendor.fetch/github.com/foo/bar@v1.0.0' % self.bp
+        self.assertIn(expected_ds, uri)
+        self.assertIn('go_module_path=github.com/foo/bar', uri)
+        self.assertIn('nobranch=1', uri)
+        self.assertIn('protocol=https', uri)
+        self.assertIn('is_go_dependency=1', uri)
+
+    def test_path_override(self):
+        """Explicit path= sets go_module_path independently of the repo URL."""
+        uri = self.uris['PATH']
+        self.assertIn('go_module_path=github.com/foo/bar/v2', uri)
+        expected_ds = 'destsuffix=%s/src/import/vendor.fetch/github.com/foo/bar@v2.1.0' % self.bp
+        self.assertIn(expected_ds, uri)
+
+    def test_pathmajor(self):
+        """pathmajor is encoded as go_pathmajor in the URI."""
+        self.assertIn('go_pathmajor=/v2', self.uris['PATHMAJOR'])
+
+    def test_subdir(self):
+        """subdir is encoded as go_subdir in the URI."""
+        self.assertIn('go_subdir=sub/pkg', self.uris['SUBDIR'])
+
+    def test_replaces(self):
+        """replaces is encoded as go_module_replacement in the URI."""
+        self.assertIn('go_module_replacement=../local/baz', self.uris['REPLACES'])
+
+    def test_non_git_vcs(self):
+        """Non-git VCS omits the git-specific nobranch and protocol=https flags."""
+        uri = self.uris['SVN']
+        self.assertIn('svn://example.com/svnmod', uri)
+        self.assertNotIn('nobranch', uri)
+        self.assertNotIn('protocol=https', uri)
+        self.assertIn('is_go_dependency=1', uri)
+
+
+# ---------------------------------------------------------------------------
+# Class 2: do_go_vendor task integration — real project
+# ---------------------------------------------------------------------------
+
+class GoVendorTaskTests(OESelftestTestCase):
+    """Integration tests for do_go_vendor using the real recipetool-go-test project.
+
+    The govendor-real-test recipe (meta-selftest/recipes-test/govendor/
+    govendor-real-test_1.0.bb) fetches git.yoctoproject.org/recipetool-go-test
+    (commit c3e213c) as its main source and github.com/godbus/dbus/v5 v5.1.0
+    as an external vendor dependency.  The project also has a local module
+    replacement (github.com/matryer/is => ./is) that lives inside the main
+    source tree.
+
+    do_go_vendor runs ONCE in setUpClass; individual test methods inspect the
+    resulting vendor directory without re-running bitbake.
+
+    Verified behaviors:
+      - External dependency files are copied to vendor/
+      - *_test.go files are excluded from the vendor copy
+      - pathmajor fallback: when the /v5 subdir is absent, repo root is used
+      - modules.txt is reproduced inside vendor/
+      - Local module replacement creates a relative symlink in vendor/
+      - S/src/GO_IMPORT/vendor symlink is created pointing at S/src/import/vendor
+    """
+
+    GO_IMPORT = 'git.yoctoproject.org/recipetool-go-test'
+    RECIPE_NAME = 'govendor-real-test'
+
+    @classmethod
+    def setUpClass(cls):
+        super().setUpClass()
+
+        bitbake('%s -c go_vendor' % cls.RECIPE_NAME)
+        cls.s_dir = get_bb_var('S', cls.RECIPE_NAME)
+        cls.vendor_dir = os.path.join(cls.s_dir, 'src', 'import', 'vendor')
+
+    # --- test methods (all inspect cls.vendor_dir, no bitbake) ---------------
+
+    def test_external_dep_copied_to_vendor(self):
+        """godbus/dbus/v5 source files are copied into vendor/."""
+        dbus_dir = os.path.join(self.vendor_dir, 'github.com', 'godbus', 'dbus', 'v5')
+        self.assertTrue(os.path.isdir(dbus_dir),
+            'vendor/github.com/godbus/dbus/v5/ not created')
+        self.assertTrue(os.path.isfile(os.path.join(dbus_dir, 'conn.go')),
+            'conn.go not found in vendor/github.com/godbus/dbus/v5/')
+
+    def test_test_files_excluded_from_vendor(self):
+        """*_test.go files from godbus/dbus are not copied into vendor/."""
+        dbus_dir = os.path.join(self.vendor_dir, 'github.com', 'godbus', 'dbus', 'v5')
+        self.assertFalse(
+            os.path.isfile(os.path.join(dbus_dir, 'conn_test.go')),
+            'conn_test.go should have been excluded from vendor by *_test.go pattern')
+
+    def test_pathmajor_falls_back_to_root(self):
+        """godbus/dbus v5.1.0 has no v5/ subdir; pathmajor=/v5 falls back to root."""
+        dbus_dir = os.path.join(self.vendor_dir, 'github.com', 'godbus', 'dbus', 'v5')
+        # auth.go is at the repo root of godbus/dbus, not inside a v5/ subdir.
+        self.assertTrue(os.path.isfile(os.path.join(dbus_dir, 'auth.go')),
+            'auth.go not found; pathmajor=/v5 fallback to repo root did not work')
+
+    def test_modules_txt_in_vendor(self):
+        """modules.txt is reproduced inside vendor/ and lists the external dep."""
+        modules_txt = os.path.join(self.vendor_dir, 'modules.txt')
+        self.assertTrue(os.path.isfile(modules_txt))
+        with open(modules_txt) as fh:
+            content = fh.read()
+        self.assertIn('github.com/godbus/dbus/v5 v5.1.0', content)
+
+    def test_local_replacement_symlink(self):
+        """go.mod local replacement (./is) creates a symlink in vendor/ to S/src/GO_IMPORT/is."""
+        symlink = os.path.join(self.vendor_dir, 'github.com', 'matryer', 'is')
+        self.assertTrue(os.path.islink(symlink),
+            'vendor/github.com/matryer/is should be a symlink for the ./is local replacement')
+        expected_target = os.path.join(self.s_dir, 'src', self.GO_IMPORT, 'is')
+        self.assertEqual(os.path.realpath(symlink), os.path.realpath(expected_target),
+            'Local replacement symlink does not point to S/src/GO_IMPORT/is')
+
+    def test_go_import_vendor_symlink(self):
+        """do_go_vendor creates S/src/GO_IMPORT/vendor -> S/src/import/vendor."""
+        link = os.path.join(self.s_dir, 'src', self.GO_IMPORT, 'vendor')
+        self.assertTrue(os.path.islink(link),
+            'S/src/GO_IMPORT/vendor symlink not created')
+        self.assertEqual(os.path.realpath(link), os.path.realpath(self.vendor_dir),
+            'S/src/GO_IMPORT/vendor does not point to S/src/import/vendor')

[meta][PATCH v2 3/3] go-vendor.bbclass: Run vendor_unlink task under fakeroot

[John Ripple]

do_install and do_package both run under fakeroot (pseudo), which keeps
pseudo's ownership/inode database in sync with what is actually on disk
under D. do_vendor_unlink runs between them but was never marked
fakeroot, so its os.unlink() of the vendor symlink happened outside of
pseudo's view: the file disappeared from disk but pseudo's database
still held a stale record for that path/inode.

The next fakeroot task to touch that path (do_package) then finds the
on-disk state and pseudo's database disagreeing, and pseudo aborts with
"path mismatch" / "inode mismatch" errors, failing do_package.

Mark do_vendor_unlink fakeroot, matching do_install and do_package, so
its filesystem operations are tracked by pseudo consistently.

Signed-off-by: John Ripple <john.ripple@keysight.com>
---
 meta/classes/go-vendor.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/go-vendor.bbclass b/meta/classes/go-vendor.bbclass
index 85a3c1b586..5bb4e8e876 100644
--- a/meta/classes/go-vendor.bbclass
+++ b/meta/classes/go-vendor.bbclass
@@ -40,13 +40,14 @@ def go_src_uri(repo, version, path=None, subdir=None, \
 
     return src_uri
 
-python do_vendor_unlink() {
+fakeroot python do_vendor_unlink() {
     go_import = d.getVar('GO_IMPORT')
     linkname = os.path.join(d.getVar('D') + d.getVar('libdir'), 'go', 'src', go_import, 'vendor')
     if os.path.islink(linkname):
         os.unlink(linkname)
 }
 
+do_vendor_unlink[depends] += "virtual/fakeroot-native:do_populate_sysroot"
 addtask vendor_unlink before do_package after do_install
 
 python do_go_vendor() {

Re: [OE-core] [meta][meta-selftest][PATCH v2 2/3] oeqa/selftest: Add go-vendor.bbclass selftest suite

[Mathieu Dubois-Briand]

On Wed Jul 15, 2026 at 5:20 PM CEST, John Ripple via lists.openembedded.org wrote:
> Add govendor.py with two test classes that cover the behaviors fixed and
> defined in go-vendor.bbclass:
>
> GoVendorSrcUriTests (offline, ~10 s): unit tests for go_src_uri() parsed
> via bb.tinfoil; verifies the basic URI shape, path/pathmajor/subdir/replaces
> encoding, and that non-git VCS omits the git-specific nobranch/protocol flags.
>
> GoVendorTaskTests: runs do_go_vendor once against the real recipetool-go-test
> project and checks that external deps land in vendor/, *_test.go files are
> excluded, pathmajor=/v5 falls back to the repo root when no v5/ subdir exists,
> modules.txt is reproduced in vendor/, a local replacement (./is) appears as
> a symlink, and the S/src/GO_IMPORT/vendor -> S/src/import/vendor symlink is
> created.
>
> Signed-off-by: John Ripple <john.ripple@keysight.com>
> ---

Hi John,

Thanks for your patch.

We get a single failure with this patch, during the
reproducible.ReproducibleTests.test_reproducible_builds selftest:

https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/4392

Can you have a look at the issue?

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com