[PATCH 1/6 v2] license/license_image: Fix license file layout to avoid overlapping files

[PATCH 1/6 v2] license/license_image: Fix license file layout to avoid overlapping files

[Richard Purdie]

Currently DEPLOY_DIR/licenses is added to SSTATE_ALLOW_OVERLAP_FILES. This
leads to bugs since when one MACHINE_ARCH recipes is cleaned, it removes the
files for another which then results in later build failures as license files
disappear.

The solution is to include SSTAGE_PKGARCH in the path names to the license files.
That does mean a search has to be used to find the correct license files for a
given PN but that can be done via SSTATE_ARCHS.

The implication for other tools is the layout has changed so tools will need to
adapt to the new paths. The benefit is no more strange build failures such as from
patterns like:

MACHINE=qemux86-64 bitbake core-image-minimal
MACHINE=genericx86-64 bitbake core-image-minimal
MACHINE=qemux86-64 bitbake linux-yocto -c clean
MACHINE=genericx86-64 bitbake core-image-minimal -C rootfs

[YOCTO #14123]

For anyone finding this commit, I'd question how much people should be relying on
this code for tooling and suggest the SPDX manifests should be the preferred data
format going forward anyway.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes-global/license.bbclass       |  2 +-
 meta/classes-global/sstate.bbclass        |  2 -
 meta/classes-recipe/license_image.bbclass | 46 +++++++++++++++++------
 3 files changed, 35 insertions(+), 15 deletions(-)

v2 - use reversed order of SSTATE_ARCHS

diff --git a/meta/classes-global/license.bbclass b/meta/classes-global/license.bbclass
index 23625f0104f..b2e0d3fabaf 100644
--- a/meta/classes-global/license.bbclass
+++ b/meta/classes-global/license.bbclass
@@ -29,7 +29,7 @@ python do_populate_lic() {
     lic_files_paths = find_license_files(d)
 
     # The base directory we wrangle licenses to
-    destdir = os.path.join(d.getVar('LICSSTATEDIR'), d.getVar('PN'))
+    destdir = os.path.join(d.getVar('LICSSTATEDIR'), d.getVar('SSTATE_PKGARCH'), d.getVar('PN'))
     copy_license_files(lic_files_paths, destdir)
     info = get_recipe_info(d)
     with open(os.path.join(destdir, "recipeinfo"), "w") as f:
diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass
index c50198449c2..706c2ae9388 100644
--- a/meta/classes-global/sstate.bbclass
+++ b/meta/classes-global/sstate.bbclass
@@ -55,8 +55,6 @@ PV[vardepvalue] = "${PV}"
 SSTATE_EXTRAPATH[vardepvalue] = ""
 SSTATE_EXTRAPATHWILDCARD[vardepvalue] = ""
 
-# For multilib rpm the allarch packagegroup files can overwrite (in theory they're identical)
-SSTATE_ALLOW_OVERLAP_FILES = "${DEPLOY_DIR}/licenses/"
 # Avoid docbook/sgml catalog warnings for now
 SSTATE_ALLOW_OVERLAP_FILES += "${STAGING_ETCDIR_NATIVE}/sgml ${STAGING_DATADIR_NATIVE}/sgml"
 # sdk-provides-dummy-nativesdk and nativesdk-buildtools-perl-dummy overlap for different SDKMACHINE
diff --git a/meta/classes-recipe/license_image.bbclass b/meta/classes-recipe/license_image.bbclass
index fc859c7c659..19b3dc55ba2 100644
--- a/meta/classes-recipe/license_image.bbclass
+++ b/meta/classes-recipe/license_image.bbclass
@@ -18,7 +18,7 @@ python() {
 
 python write_package_manifest() {
     # Get list of installed packages
-    license_image_dir = d.expand('${LICENSE_DIRECTORY}/${IMAGE_NAME}')
+    license_image_dir = d.expand('${LICENSE_DIRECTORY}/${SSTATE_PKGARCH}/${IMAGE_NAME}')
     bb.utils.mkdirhier(license_image_dir)
     from oe.rootfs import image_list_installed_packages
     from oe.utils import format_pkg_list
@@ -49,7 +49,7 @@ python license_create_manifest() {
             pkg_dic[pkg_name]["LICENSE"] = pkg_dic[pkg_name][pkg_lic_name]
 
     rootfs_license_manifest = os.path.join(d.getVar('LICENSE_DIRECTORY'),
-                        d.getVar('IMAGE_NAME'), 'license.manifest')
+                        d.getVar('SSTATE_PKGARCH'), d.getVar('IMAGE_NAME'), 'license.manifest')
     write_license_files(d, rootfs_license_manifest, pkg_dic, rootfs=True)
 }
 
@@ -59,6 +59,8 @@ def write_license_files(d, license_manifest, pkg_dic, rootfs=True):
 
     bad_licenses = (d.getVar("INCOMPATIBLE_LICENSE") or "").split()
     bad_licenses = expand_wildcard_licenses(d, bad_licenses)
+    pkgarchs = d.getVar("SSTATE_ARCHS").split()
+    pkgarchs.reverse()
 
     exceptions = (d.getVar("INCOMPATIBLE_LICENSE_EXCEPTIONS") or "").split()
     with open(license_manifest, "w") as license_file:
@@ -98,9 +100,13 @@ def write_license_files(d, license_manifest, pkg_dic, rootfs=True):
                 license_file.write("FILES: %s\n\n" % pkg_dic[pkg]["FILES"])
 
             for lic in pkg_dic[pkg]["LICENSES"]:
-                lic_file = os.path.join(d.getVar('LICENSE_DIRECTORY'),
-                                        pkg_dic[pkg]["PN"], "generic_%s" % 
-                                        re.sub(r'\+', '', lic))
+                for pkgarch in pkgarchs:
+                    lic_file = os.path.join(d.getVar('LICENSE_DIRECTORY'),
+                                            pkgarch,
+                                            pkg_dic[pkg]["PN"], "generic_%s" %
+                                            re.sub(r'\+', '', lic))
+                    if os.path.exists(lic_file):
+                        break
                 # add explicity avoid of CLOSED license because isn't generic
                 if lic == "CLOSED":
                    continue
@@ -130,8 +136,13 @@ def write_license_files(d, license_manifest, pkg_dic, rootfs=True):
             for pkg in sorted(pkg_dic):
                 pkg_rootfs_license_dir = os.path.join(rootfs_license_dir, pkg)
                 bb.utils.mkdirhier(pkg_rootfs_license_dir)
-                pkg_license_dir = os.path.join(d.getVar('LICENSE_DIRECTORY'),
-                                            pkg_dic[pkg]["PN"]) 
+                for pkgarch in pkgarchs:
+                    pkg_license_dir = os.path.join(d.getVar('LICENSE_DIRECTORY'),
+                                                   pkgarch, pkg_dic[pkg]["PN"])
+                    if os.path.exists(pkg_license_dir):
+                        break
+                if not os.path.exists(pkg_license_dir ):
+                    bb.fatal("Couldn't find license information for dependency %s" % pkg)
 
                 pkg_manifest_licenses = [canonical_license(d, lic) \
                         for lic in pkg_dic[pkg]["LICENSES"]]
@@ -183,7 +194,7 @@ def write_license_files(d, license_manifest, pkg_dic, rootfs=True):
                     os.lchown(p, 0, 0)
                     os.chmod(p, stat.S_IRWXU | stat.S_IRGRP | stat.S_IXGRP | stat.S_IROTH | stat.S_IXOTH)
 
-
+write_license_files[vardepsexclude] = "SSTATE_ARCHS"
 
 def license_deployed_manifest(d):
     """
@@ -195,6 +206,8 @@ def license_deployed_manifest(d):
     dep_dic = {}
     man_dic = {}
     lic_dir = d.getVar("LICENSE_DIRECTORY")
+    pkgarchs = d.getVar("SSTATE_ARCHS").split()
+    pkgarchs.reverse()
 
     dep_dic = get_deployed_dependencies(d)
     for dep in dep_dic.keys():
@@ -204,12 +217,19 @@ def license_deployed_manifest(d):
         man_dic[dep]["PN"] = dep
         man_dic[dep]["FILES"] = \
             " ".join(get_deployed_files(dep_dic[dep]))
-        with open(os.path.join(lic_dir, dep, "recipeinfo"), "r") as f:
+
+        for pkgarch in pkgarchs:
+            licfile = os.path.join(lic_dir, pkgarch, dep, "recipeinfo")
+            if os.path.exists(licfile):
+                break
+        if not os.path.exists(licfile):
+            bb.fatal("Couldn't find license information for dependency %s" % dep)
+        with open(licfile, "r") as f:
             for line in f.readlines():
                 key,val = line.split(": ", 1)
                 man_dic[dep][key] = val[:-1]
 
-    lic_manifest_dir = os.path.join(d.getVar('LICENSE_DIRECTORY'),
+    lic_manifest_dir = os.path.join(d.getVar('LICENSE_DIRECTORY'), d.getVar('SSTATE_PKGARCH'),
                                     d.getVar('IMAGE_NAME'))
     bb.utils.mkdirhier(lic_manifest_dir)
     image_license_manifest = os.path.join(lic_manifest_dir, 'image_license.manifest')
@@ -217,7 +237,7 @@ def license_deployed_manifest(d):
 
     link_name = d.getVar('IMAGE_LINK_NAME')
     if link_name:
-        lic_manifest_symlink_dir = os.path.join(d.getVar('LICENSE_DIRECTORY'),
+        lic_manifest_symlink_dir = os.path.join(d.getVar('LICENSE_DIRECTORY'), d.getVar('SSTATE_PKGARCH'),
                                     link_name)
         # remove old symlink
         if os.path.islink(lic_manifest_symlink_dir):
@@ -227,6 +247,8 @@ def license_deployed_manifest(d):
         if lic_manifest_dir != lic_manifest_symlink_dir:
             os.symlink(lic_manifest_dir, lic_manifest_symlink_dir)
 
+license_deployed_manifest[vardepsexclude] = "SSTATE_ARCHS"
+
 def get_deployed_dependencies(d):
     """
     Get all the deployed dependencies of an image
@@ -255,7 +277,7 @@ def get_deployed_dependencies(d):
                 break
 
     return deploy
-get_deployed_dependencies[vardepsexclude] = "BB_TASKDEPDATA"
+get_deployed_dependencies[vardepsexclude] = "BB_TASKDEPDATA SSTATE_ARCHS"
 
 def get_deployed_files(man_file):
     """
-- 
2.39.2

[PATCH 2/6 v2] create-spdx/sbom: Ensure files don't overlap between machines

[Richard Purdie]

Currently the by-id and by-namespace SPDX files are created without reference
to PACKAGE_ARCH. This means that for two machines using a common package architecture
(e.g. genericx86-64 and qqemux86-64), there would be overlapping files. This means
that the build of one can remove files from the other leading to build failures. An
example would be:

MACHINE=qemux86-64 bitbake core-image-minimal
MACHINE=genericx86-64 bitbake core-image-minimal
MACHINE=qemux86-64 bitbake linux-yocto -c clean
MACHINE=genericx86-64 bitbake core-image-minimal -C rootfs

To fix this, add PACKAGE_ARCH to the path used for the files and use a search
path based upon PACKAGE_ARCHS to access them.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/create-spdx-2.2.bbclass | 31 ++++++++++++++++++----------
 meta/lib/oe/sbom.py                  | 20 ++++++++++++------
 2 files changed, 34 insertions(+), 17 deletions(-)

v2 - Use reversed order of SSTATE_ARCHS

diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass
index 9b28d124c78..39110823c0c 100644
--- a/meta/classes/create-spdx-2.2.bbclass
+++ b/meta/classes/create-spdx-2.2.bbclass
@@ -349,6 +349,8 @@ def collect_dep_recipes(d, doc, spdx_recipe):
 
     deploy_dir_spdx = Path(d.getVar("DEPLOY_DIR_SPDX"))
     spdx_deps_file = Path(d.getVar("SPDXDEPS"))
+    package_archs = d.getVar("SSTATE_ARCHS").split()
+    package_archs.reverse()
 
     dep_recipes = []
 
@@ -356,7 +358,7 @@ def collect_dep_recipes(d, doc, spdx_recipe):
         deps = json.load(f)
 
     for dep_pn, dep_hashfn in deps:
-        dep_recipe_path = oe.sbom.doc_path_by_hashfn(deploy_dir_spdx, "recipe-" + dep_pn, dep_hashfn)
+        dep_recipe_path = oe.sbom.doc_path_by_hashfn(deploy_dir_spdx, package_archs, "recipe-" + dep_pn, dep_hashfn)
 
         spdx_dep_doc, spdx_dep_sha1 = oe.sbom.read_doc(dep_recipe_path)
 
@@ -385,6 +387,7 @@ def collect_dep_recipes(d, doc, spdx_recipe):
 
     return dep_recipes
 
+collect_dep_recipes[vardepsexclude] = "SSTATE_ARCHS"
 
 def collect_dep_sources(d, dep_recipes):
     import oe.sbom
@@ -533,6 +536,7 @@ python do_create_spdx() {
     include_sources = d.getVar("SPDX_INCLUDE_SOURCES") == "1"
     archive_sources = d.getVar("SPDX_ARCHIVE_SOURCES") == "1"
     archive_packaged = d.getVar("SPDX_ARCHIVE_PACKAGED") == "1"
+    pkg_arch = d.getVar("SSTATE_PKGARCH")
 
     creation_time = datetime.now(tz=timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
 
@@ -620,7 +624,7 @@ python do_create_spdx() {
 
     dep_recipes = collect_dep_recipes(d, doc, recipe)
 
-    doc_sha1 = oe.sbom.write_doc(d, doc, d.getVar("SSTATE_PKGARCH"), "recipes", indent=get_json_indent(d))
+    doc_sha1 = oe.sbom.write_doc(d, doc, pkg_arch, "recipes", indent=get_json_indent(d))
     dep_recipes.append(oe.sbom.DepRecipe(doc, doc_sha1, recipe))
 
     recipe_ref = oe.spdx.SPDXExternalDocumentRef()
@@ -685,7 +689,7 @@ python do_create_spdx() {
 
             add_package_sources_from_debug(d, package_doc, spdx_package, package, package_files, sources)
 
-            oe.sbom.write_doc(d, package_doc, d.getVar("SSTATE_PKGARCH"), "packages", indent=get_json_indent(d))
+            oe.sbom.write_doc(d, package_doc, pkg_arch, "packages", indent=get_json_indent(d))
 }
 do_create_spdx[vardepsexclude] += "BB_NUMBER_THREADS"
 # NOTE: depending on do_unpack is a hack that is necessary to get it's dependencies for archive the source
@@ -756,6 +760,9 @@ python do_create_runtime_spdx() {
     creation_time = datetime.now(tz=timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
 
     providers = collect_package_providers(d)
+    pkg_arch = d.getVar("SSTATE_PKGARCH")
+    package_archs = d.getVar("SSTATE_ARCHS").split()
+    package_archs.reverse()
 
     if not is_native:
         bb.build.exec_func("read_subpackage_metadata", d)
@@ -772,7 +779,7 @@ python do_create_runtime_spdx() {
             if not oe.packagedata.packaged(package, localdata):
                 continue
 
-            pkg_spdx_path = oe.sbom.doc_path(deploy_dir_spdx, pkg_name, d.getVar("SSTATE_PKGARCH"), "packages")
+            pkg_spdx_path = oe.sbom.doc_path(deploy_dir_spdx, pkg_name, pkg_arch, "packages")
 
             package_doc, package_doc_sha1 = oe.sbom.read_doc(pkg_spdx_path)
 
@@ -827,7 +834,7 @@ python do_create_runtime_spdx() {
                 if dep in dep_package_cache:
                     (dep_spdx_package, dep_package_ref) = dep_package_cache[dep]
                 else:
-                    dep_path = oe.sbom.doc_path_by_hashfn(deploy_dir_spdx, dep_pkg, dep_hashfn)
+                    dep_path = oe.sbom.doc_path_by_hashfn(deploy_dir_spdx, package_archs, dep_pkg, dep_hashfn)
 
                     spdx_dep_doc, spdx_dep_sha1 = oe.sbom.read_doc(dep_path)
 
@@ -855,10 +862,10 @@ python do_create_runtime_spdx() {
                 )
                 seen_deps.add(dep)
 
-            oe.sbom.write_doc(d, runtime_doc, d.getVar("SSTATE_PKGARCH"), "runtime", spdx_deploy, indent=get_json_indent(d))
+            oe.sbom.write_doc(d, runtime_doc, pkg_arch, "runtime", spdx_deploy, indent=get_json_indent(d))
 }
 
-do_create_runtime_spdx[vardepsexclude] += "OVERRIDES"
+do_create_runtime_spdx[vardepsexclude] += "OVERRIDES SSTATE_ARCHS"
 
 addtask do_create_runtime_spdx after do_create_spdx before do_build do_rm_work
 SSTATETASKS += "do_create_runtime_spdx"
@@ -993,6 +1000,8 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx
     import bb.compress.zstd
 
     providers = collect_package_providers(d)
+    package_archs = d.getVar("SSTATE_ARCHS").split()
+    package_archs.reverse()
 
     creation_time = datetime.now(tz=timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
     deploy_dir_spdx = Path(d.getVar("DEPLOY_DIR_SPDX"))
@@ -1022,7 +1031,7 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx
 
         pkg_name, pkg_hashfn = providers[name]
 
-        pkg_spdx_path = oe.sbom.doc_path_by_hashfn(deploy_dir_spdx, pkg_name, pkg_hashfn)
+        pkg_spdx_path = oe.sbom.doc_path_by_hashfn(deploy_dir_spdx, package_archs, pkg_name, pkg_hashfn)
         pkg_doc, pkg_doc_sha1 = oe.sbom.read_doc(pkg_spdx_path)
 
         for p in pkg_doc.packages:
@@ -1039,7 +1048,7 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx
         else:
             bb.fatal("Unable to find package with name '%s' in SPDX file %s" % (name, pkg_spdx_path))
 
-        runtime_spdx_path = oe.sbom.doc_path_by_hashfn(deploy_dir_spdx, "runtime-" + name, pkg_hashfn)
+        runtime_spdx_path = oe.sbom.doc_path_by_hashfn(deploy_dir_spdx, package_archs, "runtime-" + name, pkg_hashfn)
         runtime_doc, runtime_doc_sha1 = oe.sbom.read_doc(runtime_spdx_path)
 
         runtime_ref = oe.spdx.SPDXExternalDocumentRef()
@@ -1111,7 +1120,7 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx
                     })
 
                 for ref in doc.externalDocumentRefs:
-                    ref_path = oe.sbom.doc_path_by_namespace(deploy_dir_spdx, ref.spdxDocument)
+                    ref_path = oe.sbom.doc_path_by_namespace(deploy_dir_spdx, package_archs, ref.spdxDocument)
                     collect_spdx_document(ref_path)
 
             collect_spdx_document(image_spdx_path)
@@ -1134,4 +1143,4 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx
 
             tar.addfile(info, fileobj=index_str)
 
-combine_spdx[vardepsexclude] += "BB_NUMBER_THREADS"
+combine_spdx[vardepsexclude] += "BB_NUMBER_THREADS SSTATE_ARCHS"
diff --git a/meta/lib/oe/sbom.py b/meta/lib/oe/sbom.py
index 1130fa668bd..cddbf3cc51c 100644
--- a/meta/lib/oe/sbom.py
+++ b/meta/lib/oe/sbom.py
@@ -38,12 +38,20 @@ def get_sdk_spdxid(sdk):
     return "SPDXRef-SDK-%s" % sdk
 
 
-def doc_path_by_namespace(spdx_deploy, doc_namespace):
-    return spdx_deploy / "by-namespace" / doc_namespace.replace("/", "_")
+def doc_path_by_namespace(spdx_deploy, archs, doc_namespace):
+    for pkgarch in archs:
+        filename = spdx_deploy / "by-namespace" / pkgarch / doc_namespace.replace("/", "_")
+        if os.path.exists(filename):
+            break
+    return filename
 
 
-def doc_path_by_hashfn(spdx_deploy, doc_name, hashfn):
-    return spdx_deploy / "by-hash" / hashfn.split()[1] / (doc_name + ".spdx.json")
+def doc_path_by_hashfn(spdx_deploy, archs, doc_name, hashfn):
+    for pkgarch in archs:
+        filename = spdx_deploy / "by-hash" / pkgarch / hashfn.split()[1] / (doc_name + ".spdx.json")
+        if os.path.exists(filename):
+            break
+    return filename
 
 
 def doc_path(spdx_deploy, doc_name, arch, subdir):
@@ -61,11 +69,11 @@ def write_doc(d, spdx_doc, arch, subdir, spdx_deploy=None, indent=None):
     with dest.open("wb") as f:
         doc_sha1 = spdx_doc.to_json(f, sort_keys=True, indent=indent)
 
-    l = doc_path_by_namespace(spdx_deploy, spdx_doc.documentNamespace)
+    l = doc_path_by_namespace(spdx_deploy, [arch], spdx_doc.documentNamespace)
     l.parent.mkdir(exist_ok=True, parents=True)
     l.symlink_to(os.path.relpath(dest, l.parent))
 
-    l = doc_path_by_hashfn(spdx_deploy, spdx_doc.name, d.getVar("BB_HASHFILENAME"))
+    l = doc_path_by_hashfn(spdx_deploy, [arch], spdx_doc.name, d.getVar("BB_HASHFILENAME"))
     l.parent.mkdir(exist_ok=True, parents=True)
     l.symlink_to(os.path.relpath(dest, l.parent))
 
-- 
2.39.2

[PATCH 3/6] sstate: Stop allowing overlapping symlinks from sstate

[Richard Purdie]

When originally implemented, overlapping symlinks in DEPLOY_DIR were common. That
is no longer the case and these overlapping links are causing bugs in other areas
(e.g. bug 14123).

Therefore start showing errors for overlapping symlinks in shared areas. Whilst here,
fix a broken file reference in the grep command to match current file layouts and
update the message shown to users to match current times. Most of the message content
is obsolete now due to other advances and changes in the way the staging code
now works.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes-global/sstate.bbclass | 30 ++++++------------------------
 1 file changed, 6 insertions(+), 24 deletions(-)

diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass
index 706c2ae9388..afcda2980b3 100644
--- a/meta/classes-global/sstate.bbclass
+++ b/meta/classes-global/sstate.bbclass
@@ -266,7 +266,7 @@ def sstate_install(ss, d):
     overlap_allowed = (d.getVar("SSTATE_ALLOW_OVERLAP_FILES") or "").split()
     match = []
     for f in sharedfiles:
-        if os.path.exists(f) and not os.path.islink(f):
+        if os.path.exists(f):
             f = os.path.normpath(f)
             realmatch = True
             for w in overlap_allowed:
@@ -276,36 +276,18 @@ def sstate_install(ss, d):
                     break
             if realmatch:
                 match.append(f)
-                sstate_search_cmd = "grep -rlF '%s' %s --exclude=master.list | sed -e 's:^.*/::'" % (f, d.expand("${SSTATE_MANIFESTS}"))
+                sstate_search_cmd = "grep -rlF '%s' %s --exclude=index-* | sed -e 's:^.*/::'" % (f, d.expand("${SSTATE_MANIFESTS}"))
                 search_output = subprocess.Popen(sstate_search_cmd, shell=True, stdout=subprocess.PIPE).communicate()[0]
                 if search_output:
                     match.append("  (matched in %s)" % search_output.decode('utf-8').rstrip())
                 else:
                     match.append("  (not matched to any task)")
     if match:
-        bb.error("The recipe %s is trying to install files into a shared " \
-          "area when those files already exist. Those files and their manifest " \
-          "location are:\n  %s\nPlease verify which recipe should provide the " \
-          "above files.\n\nThe build has stopped, as continuing in this scenario WILL " \
-          "break things - if not now, possibly in the future (we've seen builds fail " \
-          "several months later). If the system knew how to recover from this " \
-          "automatically it would, however there are several different scenarios " \
-          "which can result in this and we don't know which one this is. It may be " \
-          "you have switched providers of something like virtual/kernel (e.g. from " \
-          "linux-yocto to linux-yocto-dev), in that case you need to execute the " \
-          "clean task for both recipes and it will resolve this error. It may be " \
-          "you changed DISTRO_FEATURES from systemd to udev or vice versa. Cleaning " \
-          "those recipes should again resolve this error, however switching " \
-          "DISTRO_FEATURES on an existing build directory is not supported - you " \
-          "should really clean out tmp and rebuild (reusing sstate should be safe). " \
-          "It could be the overlapping files detected are harmless in which case " \
-          "adding them to SSTATE_ALLOW_OVERLAP_FILES may be the correct solution. It could " \
-          "also be your build is including two different conflicting versions of " \
-          "things (e.g. bluez 4 and bluez 5 and the correct solution for that would " \
-          "be to resolve the conflict. If in doubt, please ask on the mailing list, " \
-          "sharing the error and filelist above." % \
+        bb.fatal("Recipe %s is trying to install files into a shared " \
+          "area when those files already exist. The files and the manifests listing " \
+          "them are:\n  %s\n"
+          "Please adjust the recipes so only one recipe provides a given file. " % \
           (d.getVar('PN'), "\n  ".join(match)))
-        bb.fatal("If the above message is too much, the simpler version is you're advised to wipe out tmp and rebuild (reusing sstate is fine). That will likely fix things in most (but not all) cases.")
 
     if ss['fixmedir'] and os.path.exists(ss['fixmedir'] + "/fixmepath.cmd"):
         sharedfiles.append(ss['fixmedir'] + "/fixmepath.cmd")
-- 
2.39.2

[PATCH 4/6] sstate: Fix nativesdk entry in SSTATE_ARCHS

[Richard Purdie]

PACKAGE_ARCH gets refined by each target so this value isn't valid in all contexts.
Tweak to use underlying variables to build it so that it remains valid in wider
contexts and works with new usage in sdpx and license classes.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes-global/sstate.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass
index afcda2980b3..67070440a6c 100644
--- a/meta/classes-global/sstate.bbclass
+++ b/meta/classes-global/sstate.bbclass
@@ -88,7 +88,7 @@ SSTATE_ARCHS = " \
     ${BUILD_ARCH}_${ORIGNATIVELSBSTRING} \
     ${BUILD_ARCH}_${SDK_ARCH}_${SDK_OS} \
     ${SDK_ARCH}_${SDK_OS} \
-    ${SDK_ARCH}_${PACKAGE_ARCH} \
+    ${SDK_ARCH}_${SDK_ARCH}-${SDKPKGSUFFIX} \
     allarch \
     ${PACKAGE_ARCH} \
     ${PACKAGE_EXTRA_ARCHS} \
-- 
2.39.2

[PATCH 5/6 v2] multilib: fix SSTATE_ARCHS for multilib usage

[Richard Purdie]

When building multilibs, we need to inject the multilib sstate pkgarch
into SSTATE_ARCHS so the list forms a complete search path. Add a tweak
to do this.

PACKAGE_ARCH defaults to TUNE_PKGARCH so this is equivalent and just
guards against recipes changing the value which may have other unwanted
side effects.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes-global/sstate.bbclass | 3 ++-
 meta/classes-recipe/image.bbclass  | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

v2 - rework patch as we need multilib arches in 'normal' builds

diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass
index 67070440a6c..2676f18e0a2 100644
--- a/meta/classes-global/sstate.bbclass
+++ b/meta/classes-global/sstate.bbclass
@@ -83,6 +83,7 @@ SSTATE_HASHEQUIV_FILEMAP ?= " \
 
 BB_HASHFILENAME = "False ${SSTATE_PKGSPEC} ${SSTATE_SWSPEC}"
 
+SSTATE_ARCHS_TUNEPKG ??= "${TUNE_PKGARCH}"
 SSTATE_ARCHS = " \
     ${BUILD_ARCH} \
     ${BUILD_ARCH}_${ORIGNATIVELSBSTRING} \
@@ -90,7 +91,7 @@ SSTATE_ARCHS = " \
     ${SDK_ARCH}_${SDK_OS} \
     ${SDK_ARCH}_${SDK_ARCH}-${SDKPKGSUFFIX} \
     allarch \
-    ${PACKAGE_ARCH} \
+    ${SSTATE_ARCHS_TUNEPKG} \
     ${PACKAGE_EXTRA_ARCHS} \
     ${MACHINE_ARCH}"
 SSTATE_ARCHS[vardepsexclude] = "ORIGNATIVELSBSTRING"
diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index 4f00162e789..7231fad940d 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -96,6 +96,7 @@ USE_DEPMOD ?= "1"
 PID = "${@os.getpid()}"
 
 PACKAGE_ARCH = "${MACHINE_ARCH}"
+SSTATE_ARCHS_TUNEPKG = "${@all_multilib_tune_values(d, 'TUNE_PKGARCH')}"
 
 LDCONFIGDEPEND ?= "ldconfig-native:do_populate_sysroot"
 LDCONFIGDEPEND:libc-musl = ""
-- 
2.39.2

[PATCH 6/6] oeqa/selftest/bbtests: Improve and update test_non_gplv3

[Richard Purdie]

This test no longer worked with fixes to avoid overlapping license files
in deploy. Fix the test but also improve the logging messages so we don't
get "False is not True" messages upon failure.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/lib/oeqa/selftest/cases/bbtests.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/bbtests.py b/meta/lib/oeqa/selftest/cases/bbtests.py
index 31aa5680f01..d242352ea2d 100644
--- a/meta/lib/oeqa/selftest/cases/bbtests.py
+++ b/meta/lib/oeqa/selftest/cases/bbtests.py
@@ -236,8 +236,11 @@ INHERIT:remove = \"report-error\"
         result = bitbake('selftest-ed', ignore_status=True)
         self.assertEqual(result.status, 0, "Bitbake failed, exit code %s, output %s" % (result.status, result.output))
         lic_dir = get_bb_var('LICENSE_DIRECTORY')
-        self.assertFalse(os.path.isfile(os.path.join(lic_dir, 'selftest-ed/generic_GPL-3.0-or-later')))
-        self.assertTrue(os.path.isfile(os.path.join(lic_dir, 'selftest-ed/generic_GPL-2.0-or-later')))
+        arch = get_bb_var('SSTATE_PKGARCH')
+        filename = os.path.join(lic_dir, arch, 'selftest-ed', 'generic_GPL-3.0-or-later')
+        self.assertFalse(os.path.isfile(filename), msg="License file %s exists and shouldn't" % filename)
+        filename = os.path.join(lic_dir, arch, 'selftest-ed', 'generic_GPL-2.0-or-later')
+        self.assertTrue(os.path.isfile(filename), msg="License file %s doesn't exist" % filename)
 
     def test_setscene_only(self):
         """ Bitbake option to restore from sstate only within a build (i.e. execute no real tasks, only setscene)"""
-- 
2.39.2

Re: [OE-core] [PATCH 3/6] sstate: Stop allowing overlapping symlinks from sstate

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 7300 bytes --]

Just FYI I think this change is now causing few more recipes to be mutually
exclusive, when they build the same library (even when it's packaged in
differently named package), in world builds I'm seeing e.g. libslirp and
libslirp-virt (from meta-virtualization) causing packagedata failure for
one of them (depending which one was built second):

DEBUG: Staging files from
TOPDIR/BUILD/work/raspberrypi4_64-oe-linux/libslirp-virt/4.6.1+git/pkgdata-pdata-input
to TOPDIR/BUILD/pkgdata/raspberrypi4-64
ERROR: Recipe libslirp-virt is trying to install files into a shared area
when those files already exist. The files and the manifests listing them
are:
  TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp-dev
    (matched in manifest-raspberrypi4_64-libslirp.packagedata)
  TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp0
    (matched in manifest-raspberrypi4_64-libslirp.packagedata)
  TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp-dbg
    (matched in manifest-raspberrypi4_64-libslirp.packagedata)
  TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp-src
    (matched in manifest-raspberrypi4_64-libslirp.packagedata)
Please adjust the recipes so only one recipe provides a given file.
DEBUG: Python function sstate_task_postfunc finished

Bruce is 4.6.1 version in meta-virtualization still needed or can you
update to libslirp 4.7.0 from oe-core?
From the git log
https://git.yoctoproject.org/meta-virtualization/log/recipes-networking/slirp
it looks like it was originally imported from meta-retro and later renamed
from libslirp to libslirt-virt until the oe-core version is validated in
runtime.

And I'm seeing the same with some internal recipes (e.g. we have
faultmanager recipe which provides libfm - completely different from libfm
from oe-core, just library name coincidence).

Cheers,

On Wed, Sep 20, 2023 at 12:58 PM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> When originally implemented, overlapping symlinks in DEPLOY_DIR were
> common. That
> is no longer the case and these overlapping links are causing bugs in
> other areas
> (e.g. bug 14123).
>
> Therefore start showing errors for overlapping symlinks in shared areas.
> Whilst here,
> fix a broken file reference in the grep command to match current file
> layouts and
> update the message shown to users to match current times. Most of the
> message content
> is obsolete now due to other advances and changes in the way the staging
> code
> now works.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/classes-global/sstate.bbclass | 30 ++++++------------------------
>  1 file changed, 6 insertions(+), 24 deletions(-)
>
> diff --git a/meta/classes-global/sstate.bbclass
> b/meta/classes-global/sstate.bbclass
> index 706c2ae9388..afcda2980b3 100644
> --- a/meta/classes-global/sstate.bbclass
> +++ b/meta/classes-global/sstate.bbclass
> @@ -266,7 +266,7 @@ def sstate_install(ss, d):
>      overlap_allowed = (d.getVar("SSTATE_ALLOW_OVERLAP_FILES") or
> "").split()
>      match = []
>      for f in sharedfiles:
> -        if os.path.exists(f) and not os.path.islink(f):
> +        if os.path.exists(f):
>              f = os.path.normpath(f)
>              realmatch = True
>              for w in overlap_allowed:
> @@ -276,36 +276,18 @@ def sstate_install(ss, d):
>                      break
>              if realmatch:
>                  match.append(f)
> -                sstate_search_cmd = "grep -rlF '%s' %s
> --exclude=master.list | sed -e 's:^.*/::'" % (f,
> d.expand("${SSTATE_MANIFESTS}"))
> +                sstate_search_cmd = "grep -rlF '%s' %s --exclude=index-*
> | sed -e 's:^.*/::'" % (f, d.expand("${SSTATE_MANIFESTS}"))
>                  search_output = subprocess.Popen(sstate_search_cmd,
> shell=True, stdout=subprocess.PIPE).communicate()[0]
>                  if search_output:
>                      match.append("  (matched in %s)" %
> search_output.decode('utf-8').rstrip())
>                  else:
>                      match.append("  (not matched to any task)")
>      if match:
> -        bb.error("The recipe %s is trying to install files into a shared
> " \
> -          "area when those files already exist. Those files and their
> manifest " \
> -          "location are:\n  %s\nPlease verify which recipe should provide
> the " \
> -          "above files.\n\nThe build has stopped, as continuing in this
> scenario WILL " \
> -          "break things - if not now, possibly in the future (we've seen
> builds fail " \
> -          "several months later). If the system knew how to recover from
> this " \
> -          "automatically it would, however there are several different
> scenarios " \
> -          "which can result in this and we don't know which one this is.
> It may be " \
> -          "you have switched providers of something like virtual/kernel
> (e.g. from " \
> -          "linux-yocto to linux-yocto-dev), in that case you need to
> execute the " \
> -          "clean task for both recipes and it will resolve this error. It
> may be " \
> -          "you changed DISTRO_FEATURES from systemd to udev or vice
> versa. Cleaning " \
> -          "those recipes should again resolve this error, however
> switching " \
> -          "DISTRO_FEATURES on an existing build directory is not
> supported - you " \
> -          "should really clean out tmp and rebuild (reusing sstate should
> be safe). " \
> -          "It could be the overlapping files detected are harmless in
> which case " \
> -          "adding them to SSTATE_ALLOW_OVERLAP_FILES may be the correct
> solution. It could " \
> -          "also be your build is including two different conflicting
> versions of " \
> -          "things (e.g. bluez 4 and bluez 5 and the correct solution for
> that would " \
> -          "be to resolve the conflict. If in doubt, please ask on the
> mailing list, " \
> -          "sharing the error and filelist above." % \
> +        bb.fatal("Recipe %s is trying to install files into a shared " \
> +          "area when those files already exist. The files and the
> manifests listing " \
> +          "them are:\n  %s\n"
> +          "Please adjust the recipes so only one recipe provides a given
> file. " % \
>            (d.getVar('PN'), "\n  ".join(match)))
> -        bb.fatal("If the above message is too much, the simpler version
> is you're advised to wipe out tmp and rebuild (reusing sstate is fine).
> That will likely fix things in most (but not all) cases.")
>
>      if ss['fixmedir'] and os.path.exists(ss['fixmedir'] +
> "/fixmepath.cmd"):
>          sharedfiles.append(ss['fixmedir'] + "/fixmepath.cmd")
> --
> 2.39.2
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#187919):
> https://lists.openembedded.org/g/openembedded-core/message/187919
> Mute This Topic: https://lists.openembedded.org/mt/101475773/3617156
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> martin.jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 9148 bytes --]

Re: [OE-core] [PATCH 3/6] sstate: Stop allowing overlapping symlinks from sstate

[Richard Purdie]

On Sun, 2023-09-24 at 11:14 +0200, Martin Jansa wrote:
> Just FYI I think this change is now causing few more recipes to be mutually exclusive, when they build the same library (even when it's packaged in differently named package), in world builds I'm seeing e.g. libslirp and libslirp-virt (from meta-virtualization) causing packagedata failure for one of them (depending which one was built second):
> 
> DEBUG: Staging files from TOPDIR/BUILD/work/raspberrypi4_64-oe-linux/libslirp-virt/4.6.1+git/pkgdata-pdata-input to TOPDIR/BUILD/pkgdata/raspberrypi4-64
> ERROR: Recipe libslirp-virt is trying to install files into a shared area when those files already exist. The files and the manifests listing them are:
>   TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp-dev
>     (matched in manifest-raspberrypi4_64-libslirp.packagedata)
>   TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp0
>     (matched in manifest-raspberrypi4_64-libslirp.packagedata)
>   TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp-dbg
>     (matched in manifest-raspberrypi4_64-libslirp.packagedata)
>   TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp-src
>     (matched in manifest-raspberrypi4_64-libslirp.packagedata)
> Please adjust the recipes so only one recipe provides a given file. 
> DEBUG: Python function sstate_task_postfunc finished
> 
> Bruce is 4.6.1 version in meta-virtualization still needed or can you update to libslirp 4.7.0 from oe-core?
> From the git log https://git.yoctoproject.org/meta-virtualization/log/recipes-networking/slirp it looks like it was originally imported from meta-retro and later renamed from libslirp to libslirt-virt until the oe-core version is validated in runtime.
> 
> And I'm seeing the same with some internal recipes (e.g. we have faultmanager recipe which provides libfm - completely different from libfm from oe-core, just library name coincidence).

This might be safe to exclude due to the way pkgdata works, it is
handled per workdir now. I'd need to check a few things but offhand I
think it will be ok to allow specifically.

Cheers,

Richard

Re: [OE-core] [PATCH 3/6] sstate: Stop allowing overlapping symlinks from sstate

[Richard Purdie]

On Sun, 2023-09-24 at 11:14 +0200, Martin Jansa wrote:
> Just FYI I think this change is now causing few more recipes to be mutually exclusive, when they build the same library (even when it's packaged in differently named package), in world builds I'm seeing e.g. libslirp and libslirp-virt (from meta-virtualization) causing packagedata failure for one of them (depending which one was built second):
> 
> DEBUG: Staging files from TOPDIR/BUILD/work/raspberrypi4_64-oe-linux/libslirp-virt/4.6.1+git/pkgdata-pdata-input to TOPDIR/BUILD/pkgdata/raspberrypi4-64
> ERROR: Recipe libslirp-virt is trying to install files into a shared area when those files already exist. The files and the manifests listing them are:
>   TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp-dev
>     (matched in manifest-raspberrypi4_64-libslirp.packagedata)
>   TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp0
>     (matched in manifest-raspberrypi4_64-libslirp.packagedata)
>   TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp-dbg
>     (matched in manifest-raspberrypi4_64-libslirp.packagedata)
>   TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp-src
>     (matched in manifest-raspberrypi4_64-libslirp.packagedata)
> Please adjust the recipes so only one recipe provides a given file. 
> DEBUG: Python function sstate_task_postfunc finished
> 
> Bruce is 4.6.1 version in meta-virtualization still needed or can you update to libslirp 4.7.0 from oe-core?
> From the git log https://git.yoctoproject.org/meta-virtualization/log/recipes-networking/slirp it looks like it was originally imported from meta-retro and later renamed from libslirp to libslirt-virt until the oe-core version is validated in runtime.
> 
> And I'm seeing the same with some internal recipes (e.g. we have faultmanager recipe which provides libfm - completely different from libfm from oe-core, just library name coincidence).

I did look into this and it *is* a real issue/bug. The output will be
non-deterministic depending upon which is built first. The issue would
"corrupt" anything which is reading data from pkgdata related to the
recipes in question.

The error is therefore correct and we need to do something about this.

Cheers,

Richard

Re: [OE-core] [PATCH 3/6] sstate: Stop allowing overlapping symlinks from sstate

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3348 bytes --]

In the cases I've seen in our internal recipes I can just PNBLACKLIST the
one we never use (libfm) to make sure it's never built with faultmanager
(as it was built only as part of world build anyway, nothing in our images
really depends on it). So it wasn't something which would happen in regular
builds, but I agree that ihe error is real and good that it will force us
to make this a bit more deterministic.

For some other internal cases I can also add SkipRecipe to make them
mutually exclusive (they are MACHINE_ARCH implementations of the same
thing, just not restricted with COMPATIBLE_MACHINE or such to make sure
that only the preferred provider is built for given MACHINE).

And for libslirp and libslirp-virt it looks like it was meant to be only
temporary and this error will force it to be resolved a bit sooner.

Regards,

On Mon, Sep 25, 2023 at 4:59 PM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Sun, 2023-09-24 at 11:14 +0200, Martin Jansa wrote:
> > Just FYI I think this change is now causing few more recipes to be
> mutually exclusive, when they build the same library (even when it's
> packaged in differently named package), in world builds I'm seeing e.g.
> libslirp and libslirp-virt (from meta-virtualization) causing packagedata
> failure for one of them (depending which one was built second):
> >
> > DEBUG: Staging files from
> TOPDIR/BUILD/work/raspberrypi4_64-oe-linux/libslirp-virt/4.6.1+git/pkgdata-pdata-input
> to TOPDIR/BUILD/pkgdata/raspberrypi4-64
> > ERROR: Recipe libslirp-virt is trying to install files into a shared
> area when those files already exist. The files and the manifests listing
> them are:
> >   TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp-dev
> >     (matched in manifest-raspberrypi4_64-libslirp.packagedata)
> >   TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp0
> >     (matched in manifest-raspberrypi4_64-libslirp.packagedata)
> >   TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp-dbg
> >     (matched in manifest-raspberrypi4_64-libslirp.packagedata)
> >   TOPDIR/BUILD/pkgdata/raspberrypi4-64/runtime-reverse/libslirp-src
> >     (matched in manifest-raspberrypi4_64-libslirp.packagedata)
> > Please adjust the recipes so only one recipe provides a given file.
> > DEBUG: Python function sstate_task_postfunc finished
> >
> > Bruce is 4.6.1 version in meta-virtualization still needed or can you
> update to libslirp 4.7.0 from oe-core?
> > From the git log
> https://git.yoctoproject.org/meta-virtualization/log/recipes-networking/slirp
> it looks like it was originally imported from meta-retro and later renamed
> from libslirp to libslirt-virt until the oe-core version is validated in
> runtime.
> >
> > And I'm seeing the same with some internal recipes (e.g. we have
> faultmanager recipe which provides libfm - completely different from libfm
> from oe-core, just library name coincidence).
>
> I did look into this and it *is* a real issue/bug. The output will be
> non-deterministic depending upon which is built first. The issue would
> "corrupt" anything which is reading data from pkgdata related to the
> recipes in question.
>
> The error is therefore correct and we need to do something about this.
>
> Cheers,
>
> Richard
>
>

[-- Attachment #2: Type: text/html, Size: 3948 bytes --]

Re: [OE-core] [PATCH 1/6 v2] license/license_image: Fix license file layout to avoid overlapping files

[Russ Dill]

> Currently DEPLOY_DIR/licenses is added to SSTATE_ALLOW_OVERLAP_FILES. This
> leads to bugs since when one MACHINE_ARCH recipes is cleaned, it removes the
> files for another which then results in later build failures as license files
> disappear.
> 
> The solution is to include SSTAGE_PKGARCH in the path names to the license files.
> That does mean a search has to be used to find the correct license files for a
> given PN but that can be done via SSTATE_ARCHS.
> 
> The implication for other tools is the layout has changed so tools will need to
> adapt to the new paths. The benefit is no more strange build failures such as from
> patterns like:
> 
> MACHINE=qemux86-64 bitbake core-image-minimal
> MACHINE=genericx86-64 bitbake core-image-minimal
> MACHINE=qemux86-64 bitbake linux-yocto -c clean
> MACHINE=genericx86-64 bitbake core-image-minimal -C rootfs
> 
> [YOCTO #14123]
> 
> For anyone finding this commit, I'd question how much people should be relying on
> this code for tooling and suggest the SPDX manifests should be the preferred data
> format going forward anyway.
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/classes-global/license.bbclass       |  2 +-
>  meta/classes-global/sstate.bbclass        |  2 -
>  meta/classes-recipe/license_image.bbclass | 46 +++++++++++++++++------
>  3 files changed, 35 insertions(+), 15 deletions(-)
> 
> v2 - use reversed order of SSTATE_ARCHS
> 
> diff --git a/meta/classes-global/license.bbclass b/meta/classes-global/license.bbclass
> index 23625f0104f..b2e0d3fabaf 100644
> --- a/meta/classes-global/license.bbclass
> +++ b/meta/classes-global/license.bbclass
> @@ -29,7 +29,7 @@ python do_populate_lic() {
>      lic_files_paths = find_license_files(d)
> 
>      # The base directory we wrangle licenses to
> -    destdir = os.path.join(d.getVar('LICSSTATEDIR'), d.getVar('PN'))
> +    destdir = os.path.join(d.getVar('LICSSTATEDIR'), d.getVar('SSTATE_PKGARCH'), d.getVar('PN'))
>      copy_license_files(lic_files_paths, destdir)
>      info = get_recipe_info(d)
>      with open(os.path.join(destdir, "recipeinfo"), "w") as f:

It looks like an update to do_populate_lic[dirs] was missed here.

Re: [PATCH 1/6 v2] license/license_image: Fix license file layout to avoid overlapping files

[Livius]

[scarthgap][master] license archiving is failed to tmp/deploy/licenses
https://lists.openembedded.org/g/openembedded-core/message/202179

Please check my report. What goes wrong?