[PATCH 0/2] SDKs w/ multilibs and spdx30 'isfile'

[PATCH 0/2] SDKs w/ multilibs and spdx30 'isfile'

[Mark Hatle]

From: Mark Hatle <mark.hatle@amd.com>

This resolves problems when building an SDK that includes multilibs.
It also adds the isfile change that was previou put in for spdx 2.2.

Introduce a new multilib SSTATE_ARCHs, as an optimization this just
defaults to SSTATE_ARCHs in most cases.  If building an SDK we do
expand it with the multilibs.

Additionally fixes the issue where were a debug component may refer
to something that is not a file.  This change was already put into
the spdx 2.2 class.  See:

commit a798d00d54a1424f9972de43ff05a0ca8950d7de
Author: Mark Hatle <mark.hatle@amd.com>
Date:   Mon Jul 15 14:56:06 2024 -0500

    create-sdpx-2.2.bbclass: Switch from exists to isfile checking debugsrc
    
    While debugsrc is almost always a file (or link), there are apparently
    cases where a directory could be returned from the dwarfsrcfiles
    processing.  When this happens, the hashing fails and an error results
    when building the SPDX documents.
    
    (From OE-Core rev: 02e262c291c0b2066132b4cb2ca5fda8145284a9)
    
    Signed-off-by: Mark Hatle <mark.hatle@amd.com>
    Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>


Mark Hatle (2):
  spdx30_tasks.py: switch from exists to isfile checking debugsrc
  create-spdx-*: Support multilibs via SPDX_MULTILIB_SSTATE_ARCHS

 meta/classes-recipe/populate_sdk_base.bbclass |  4 ++++
 meta/classes/create-spdx-2.2.bbclass          | 12 ++++++------
 meta/classes/create-spdx-3.0.bbclass          |  4 ++--
 meta/classes/spdx-common.bbclass              |  2 ++
 meta/lib/oe/sbom30.py                         |  2 +-
 meta/lib/oe/spdx30_tasks.py                   |  3 ++-
 6 files changed, 17 insertions(+), 10 deletions(-)

-- 
2.34.1

[PATCH 1/2] spdx30_tasks.py: switch from exists to isfile checking debugsrc

[Mark Hatle]

From: Mark Hatle <mark.hatle@amd.com>

Same change as previously made to the create-spdx-2.2.bbclass,
while debugsrc is almost always a file (or link), there are apparently
cases where a directory could be returned from the dwarfsrcfiles
processing.  When this happens, the hashing fails and an error results
when building the SPDX documents.

Signed-off-by: Mark Hatle <mark.hatle@amd.com>
---
 meta/lib/oe/spdx30_tasks.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py
index 7baa6be70e..9d5bbadc0f 100644
--- a/meta/lib/oe/spdx30_tasks.py
+++ b/meta/lib/oe/spdx30_tasks.py
@@ -238,7 +238,8 @@ def get_package_sources_from_debug(
                     if file_sha256 is None:
                         continue
                 else:
-                    if not debugsrc_path.exists():
+                    # We can only hash files below, skip directories, links, etc.
+                    if not debugsrc_path.isfile():
                         source_hash_cache[debugsrc_path] = None
                         continue
 
-- 
2.34.1

[PATCH 2/2] create-spdx-*: Support multilibs via SPDX_MULTILIB_SSTATE_ARCHS

[Mark Hatle]

From: Mark Hatle <mark.hatle@amd.com>

When a create-spdx-* classes is processing documents, it needs to
find the document in a path that is related to the SSTATE_ARCH
when a packge is generated.  The SSTATE_ARCH can be affected by
multilib configurations, resulting is something like armv8a-mlib.

When the image (or SDK) is being generated and the components are
collected, the system has no knowledge of the multilib arch and
will fail to find it, such as:

  ERROR: meta-toolchain-1.0-r0 do_populate_sdk: No SPDX file found
   for package libilp32-libgcc-dbg,
   False sstate:libilp32-libgcc:armv8a-ilp32-mllibilp32-elf:14.1.0:r0:armv8a-ilp32:12:
   sstate:libilp32-libgcc::14.1.0:r0::12:

Adding in the new SPDX_MULTILIB_SSTATE_ARCHS will provide a full
set of SSTATE_ARCHS including ones that contain the multilib
extension which will allow create-spdx-* to correctly find the
document it is looking for.  This would also be valuable to any
other function doing a similar search through SSTATE_ARCH that may
have been extended with multilib configurations.

Signed-off-by: Mark Hatle <mark.hatle@amd.com>
---
 meta/classes-recipe/populate_sdk_base.bbclass |  4 ++++
 meta/classes/create-spdx-2.2.bbclass          | 12 ++++++------
 meta/classes/create-spdx-3.0.bbclass          |  4 ++--
 meta/classes/spdx-common.bbclass              |  2 ++
 meta/lib/oe/sbom30.py                         |  2 +-
 5 files changed, 15 insertions(+), 9 deletions(-)

diff --git a/meta/classes-recipe/populate_sdk_base.bbclass b/meta/classes-recipe/populate_sdk_base.bbclass
index 8d79d88ebe..61a8b5e300 100644
--- a/meta/classes-recipe/populate_sdk_base.bbclass
+++ b/meta/classes-recipe/populate_sdk_base.bbclass
@@ -6,6 +6,10 @@
 
 PACKAGES = ""
 
+# This exists as an optimization for SPDX processing to only run in image and
+# SDK processing context.  This class happens to be common to these usages.
+SPDX_MULTILIB_SSTATE_ARCHS = "${@all_multilib_tune_values(d, 'SSTATE_ARCHS')}"
+
 inherit image-postinst-intercepts image-artifact-names
 
 # Wildcards specifying complementary packages to install for every package that has been explicitly
diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass
index 12b78cb3f6..509d3b58b6 100644
--- a/meta/classes/create-spdx-2.2.bbclass
+++ b/meta/classes/create-spdx-2.2.bbclass
@@ -267,7 +267,7 @@ def collect_dep_recipes(d, doc, spdx_recipe):
     import oe.spdx_common
 
     deploy_dir_spdx = Path(d.getVar("DEPLOY_DIR_SPDX"))
-    package_archs = d.getVar("SSTATE_ARCHS").split()
+    package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split()
     package_archs.reverse()
 
     dep_recipes = []
@@ -312,7 +312,7 @@ def collect_dep_recipes(d, doc, spdx_recipe):
 
     return dep_recipes
 
-collect_dep_recipes[vardepsexclude] = "SSTATE_ARCHS"
+collect_dep_recipes[vardepsexclude] = "SPDX_MULTILIB_SSTATE_ARCHS"
 
 def collect_dep_sources(d, dep_recipes):
     import oe.sbom
@@ -610,7 +610,7 @@ python do_create_runtime_spdx() {
 
     providers = oe.spdx_common.collect_package_providers(d)
     pkg_arch = d.getVar("SSTATE_PKGARCH")
-    package_archs = d.getVar("SSTATE_ARCHS").split()
+    package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split()
     package_archs.reverse()
 
     if not is_native:
@@ -716,7 +716,7 @@ python do_create_runtime_spdx() {
             oe.sbom.write_doc(d, runtime_doc, pkg_arch, "runtime", spdx_deploy, indent=get_json_indent(d))
 }
 
-do_create_runtime_spdx[vardepsexclude] += "OVERRIDES SSTATE_ARCHS"
+do_create_runtime_spdx[vardepsexclude] += "OVERRIDES SPDX_MULTILIB_SSTATE_ARCHS"
 
 addtask do_create_runtime_spdx after do_create_spdx before do_build do_rm_work
 SSTATETASKS += "do_create_runtime_spdx"
@@ -798,7 +798,7 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx
     import bb.compress.zstd
 
     providers = oe.spdx_common.collect_package_providers(d)
-    package_archs = d.getVar("SSTATE_ARCHS").split()
+    package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split()
     package_archs.reverse()
 
     creation_time = datetime.now(tz=timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
@@ -950,4 +950,4 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx
 
             tar.addfile(info, fileobj=index_str)
 
-combine_spdx[vardepsexclude] += "BB_NUMBER_THREADS SSTATE_ARCHS"
+combine_spdx[vardepsexclude] += "BB_NUMBER_THREADS SPDX_MULTILIB_SSTATE_ARCHS"
diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass
index 41840d9d1a..2eb44ba808 100644
--- a/meta/classes/create-spdx-3.0.bbclass
+++ b/meta/classes/create-spdx-3.0.bbclass
@@ -118,7 +118,7 @@ IMAGE_CLASSES:append = " create-spdx-image-3.0"
 
 oe.spdx30_tasks.set_timestamp_now[vardepsexclude] = "SPDX_INCLUDE_TIMESTAMPS"
 oe.spdx30_tasks.get_package_sources_from_debug[vardepsexclude] += "STAGING_KERNEL_DIR"
-oe.spdx30_tasks.collect_dep_objsets[vardepsexclude] = "SSTATE_ARCHS"
+oe.spdx30_tasks.collect_dep_objsets[vardepsexclude] = "SPDX_MULTILIB_SSTATE_ARCHS"
 
 
 
@@ -164,7 +164,7 @@ python do_create_package_spdx() {
     import oe.spdx30_tasks
     oe.spdx30_tasks.create_package_spdx(d)
 }
-do_create_package_spdx[vardepsexclude] += "OVERRIDES SSTATE_ARCHS"
+do_create_package_spdx[vardepsexclude] += "OVERRIDES SPDX_MULTILIB_SSTATE_ARCHS"
 
 addtask do_create_package_spdx after do_create_spdx before do_build do_rm_work
 SSTATETASKS += "do_create_package_spdx"
diff --git a/meta/classes/spdx-common.bbclass b/meta/classes/spdx-common.bbclass
index d3110a9bdb..e1528b6d0b 100644
--- a/meta/classes/spdx-common.bbclass
+++ b/meta/classes/spdx-common.bbclass
@@ -37,6 +37,8 @@ SPDX_LICENSES ??= "${COREBASE}/meta/files/spdx-licenses.json"
 
 SPDX_CUSTOM_ANNOTATION_VARS ??= ""
 
+SPDX_MULTILIB_SSTATE_ARCHS ??= "${SSTATE_ARCHS}"
+
 python() {
     import oe.spdx_common
     oe.spdx_common.load_spdx_license_data(d)
diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py
index 2532d19dad..27ed74f810 100644
--- a/meta/lib/oe/sbom30.py
+++ b/meta/lib/oe/sbom30.py
@@ -921,7 +921,7 @@ def load_jsonld_by_arch(d, arch, subdir, name, *, required=False):
 
 
 def find_jsonld(d, subdir, name, *, required=False):
-    package_archs = d.getVar("SSTATE_ARCHS").split()
+    package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split()
     package_archs.reverse()
 
     for arch in package_archs:
-- 
2.34.1