[PATCH 1/3] meta/lib/oe/sbom30.py: correct alias

[PATCH 1/3] meta/lib/oe/sbom30.py: correct alias

[Hongxu Jia]

After commit [spdx 3.0: Rework how SPDX aliases are linked] applied,
it added extra "/" to namespace, which causing the replacement of
UNIHASH missing a "/"

  http://spdxdocs.org/openembedded-alias/by-doc-hash/0b308e4b9ad979f642d8787c61f76c31bdcad04837eeaaf8bc383f33f99bbeb8/flex-nativeUNIHASH/build/recipe

After applying this commit to remove "/" from namespace.

  http://spdxdocs.org/openembedded-alias/by-doc-hash/0b308e4b9ad979f642d8787c61f76c31bdcad04837eeaaf8bc383f33f99bbeb8/flex-native/UNIHASH/build/recipe

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 meta/lib/oe/sbom30.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py
index 29cb9e45ad..08fea2aca3 100644
--- a/meta/lib/oe/sbom30.py
+++ b/meta/lib/oe/sbom30.py
@@ -267,7 +267,7 @@ class ObjectSet(oe.spdx30.SHACLObjectSet):
 
     def new_alias_id(self, obj, replace):
         unihash = self.d.getVar("BB_UNIHASH")
-        namespace = self.get_namespace() + "/"
+        namespace = self.get_namespace()
         if unihash not in obj._id:
             bb.warn(f"Unihash {unihash} not found in {obj._id}")
             return None
-- 
2.25.1

[PATCH 2/3] meta/lib/oe/spdx30_tasks.py: fix hash link conflict while multiconfig enabled

[Hongxu Jia]

While enabling multiconfig, build recipe (such as linux-libc-headers) in
different arch directories but generating duplicated hash link, and these
directories are listed in ${SPDX_MULTILIB_SSTATE_ARCHS}

In the following case, build recipe linux-libc-headers in multiconfig to
generate duplicated hash link in core2-32 and core2-64
$ echo 'MACHINE = qemux86-64' >> conf/local.conf
$ mkdir conf/multiconfig
$ echo 'MACHINE = "qemux86"' >> conf/multiconfig/qemux86.conf
$ echo 'BBMULTICONFIG = qemux86' >> conf/local.conf
$ bitbake linux-libc-headers mc:qemux86:linux-libc-headers
$ ls -al tmp/deploy/spdx/3.0.1/*/by-spdxid-hash/*/*.spdx.json |grep recipe-linux-libc-headers.spdx.json
tmp/deploy/spdx/3.0.1/core2-32/by-spdxid-hash/c5/c57f1b9dbf27d2c83c77ca49f971b31d6d6c5830434104c88a12c6e2eb687094.spdx.json -> ../../recipes/recipe-linux-libc-headers.spdx.json
tmp/deploy/spdx/3.0.1/core2-64/by-spdxid-hash/c5/c57f1b9dbf27d2c83c77ca49f971b31d6d6c5830434104c88a12c6e2eb687094.spdx.json -> ../../recipes/recipe-linux-libc-headers.spdx.json

According to [1], and after [spdx 3.0: Rework how SPDX aliases are linked]
applied, SPDX 3.0 have a loop on ${SPDX_MULTILIB_SSTATE_ARCHS} to search for
the hash link to find jsonld file, return on first match

If hash link duplicated, the first match may not be right. Add ${SSTATE_PKGARCH}
to link prefix which could distinguish arch directories in spdxId prefix.

After applying this commit, the hash link of linux-libc-headers differs

ls -al tmp/deploy/spdx/3.0.1/*/by-spdxid-hash/*/*.spdx.json |grep recipe-linux-libc-headers.spdx.json
tmp/deploy/spdx/3.0.1/core2-32/by-spdxid-hash/4f/4f80c220686f345bce561f6d745165ee803e6390ba5e047f54cb28f4e23c167f.spdx.json -> ../../recipes/recipe-linux-libc-headers.spdx.json
tmp/deploy/spdx/3.0.1/core2-64/by-spdxid-hash/22/22db6f7b9cf02bbd3e64ffe2b9bceb17bf6f2f1366db1e72d8d82c337954b112.spdx.json -> ../../recipes/recipe-linux-libc-headers.spdx.json

[1] https://github.com/openembedded/openembedded-core/commit/f1499c36c1054fc90f7b7268cc95285f2eca72f7#diff-253805ec3eaba58c6c3967711921d5a2748974ba533c6d864d86a8e5ad4df4c4L923

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 meta/lib/oe/sbom30.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py
index 08fea2aca3..65603e69b7 100644
--- a/meta/lib/oe/sbom30.py
+++ b/meta/lib/oe/sbom30.py
@@ -786,7 +786,7 @@ class ObjectSet(oe.spdx30.SHACLObjectSet):
             OEIdAliasExtension(
                 alias=objset.new_alias_id(
                     document,
-                    OE_DOC_ALIAS_PREFIX + d.getVar("PN") + "/" + name + "/",
+                    OE_DOC_ALIAS_PREFIX + d.getVar("SSTATE_PKGARCH") + "/" + d.getVar("PN") + "/" + name + "/",
                 ),
             )
         )
-- 
2.25.1

[PATCH 3/3] gcc-cross.inc: add var-SSTATE_ARCHS_TUNEPKG to vardeps of do_create_spdx

[Hongxu Jia]

The recipe gcc-cross inherited cross.bbclass which had class-native
recipe behavior, but depends on class-target recipe linux-libc-headers.
It caused the tweaking of ${TUNE_PKGARCH} for 32bit arm did not trigger
rebuild of do_create_spdx in gcc-cross-arm

Due to commit [meta/lib/oe/spdx30_tasks.py: fix hash link conflict while
multiconfig enabled] applied, it added ${SSTATE_PKGARCH} to link prefix.
While tweaking ${TUNE_PKGARCH} for 32bit arm, use do_create_spdx from
sstate cache will caused ${SSTATE_PKGARCH} did not match

In the following case, the link hash has no change while machine was changed
$ echo 'MACHINE = "qemuarmv5"' >> conf/local.conf
$ bitbake gcc-cross-arm
$ grep linux-libc-headers tmp/deploy/spdx/3.0.1/x86_64/recipes/recipe-gcc-cross-arm.spdx.json
        "http://spdxdocs.org/openembedded-alias/by-doc-hash/09523e44e7b6fe1aed03347e30191c00d93500ce6e1f453e8dfdfae98d325f3e/linux-libc-headers/UNIHASH/build/recipe",
$ echo 'MACHINE = "qemuarm"' >> conf/local.conf
$ bitbake gcc-cross-arm
$ grep linux-libc-headers tmp/deploy/spdx/3.0.1/x86_64/recipes/recipe-gcc-cross-arm.spdx.json
        "http://spdxdocs.org/openembedded-alias/by-doc-hash/09523e44e7b6fe1aed03347e30191c00d93500ce6e1f453e8dfdfae98d325f3e/linux-libc-headers/UNIHASH/build/recipe",

Explicitly add var-SSTATE_ARCHS_TUNEPKG to vardeps of do_create_spdx to
trigger rebuild in this situation. Add the code along with adding
linux-libc-headers to gcc-cross's depends

After applying this commit, the link hash has changed along with machine changed

$ echo 'MACHINE = "qemuarmv5"' >> conf/local.conf
$ bitbake gcc-cross-arm
$ grep linux-libc-headers tmp/deploy/spdx/3.0.1/x86_64/recipes/recipe-gcc-cross-arm.spdx.json
        "http://spdxdocs.org/openembedded-alias/by-doc-hash/09523e44e7b6fe1aed03347e30191c00d93500ce6e1f453e8dfdfae98d325f3e/linux-libc-headers/UNIHASH/build/recipe",
$ echo 'MACHINE = "qemuarm"' >> conf/local.conf
$ bitbake gcc-cross-arm
$ grep linux-libc-headers tmp/deploy/spdx/3.0.1/x86_64/recipes/recipe-gcc-cross-arm.spdx.json
        "http://spdxdocs.org/openembedded-alias/by-doc-hash/087e29d3627a06f5377effa627b99907607f5af0fe8f6d116b4490650c404a55/linux-libc-headers/UNIHASH/build/recipe",

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 meta/recipes-devtools/gcc/gcc-cross.inc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-devtools/gcc/gcc-cross.inc b/meta/recipes-devtools/gcc/gcc-cross.inc
index f85ccd5c23..d6936d063a 100644
--- a/meta/recipes-devtools/gcc/gcc-cross.inc
+++ b/meta/recipes-devtools/gcc/gcc-cross.inc
@@ -7,6 +7,7 @@ PROVIDES = "virtual/${TARGET_PREFIX}gcc virtual/${TARGET_PREFIX}g++"
 python () {
     if d.getVar("TARGET_OS").startswith("linux"):
         d.setVar("EXTRADEPENDS", "linux-libc-headers")
+        d.appendVarFlag("do_create_spdx", "vardeps", " SSTATE_ARCHS_TUNEPKG")
 }
 
 PN = "gcc-cross-${TARGET_ARCH}"
-- 
2.25.1

Re: [PATCH 3/3] gcc-cross.inc: add var-SSTATE_ARCHS_TUNEPKG to vardeps of do_create_spdx

[Joshua Watt]

On Wed, Dec 11, 2024 at 5:07 AM Hongxu Jia <hongxu.jia@windriver.com> wrote:
>
> The recipe gcc-cross inherited cross.bbclass which had class-native
> recipe behavior, but depends on class-target recipe linux-libc-headers.
> It caused the tweaking of ${TUNE_PKGARCH} for 32bit arm did not trigger
> rebuild of do_create_spdx in gcc-cross-arm
>
> Due to commit [meta/lib/oe/spdx30_tasks.py: fix hash link conflict while
> multiconfig enabled] applied, it added ${SSTATE_PKGARCH} to link prefix.
> While tweaking ${TUNE_PKGARCH} for 32bit arm, use do_create_spdx from
> sstate cache will caused ${SSTATE_PKGARCH} did not match
>
> In the following case, the link hash has no change while machine was changed
> $ echo 'MACHINE = "qemuarmv5"' >> conf/local.conf
> $ bitbake gcc-cross-arm
> $ grep linux-libc-headers tmp/deploy/spdx/3.0.1/x86_64/recipes/recipe-gcc-cross-arm.spdx.json
>         "http://spdxdocs.org/openembedded-alias/by-doc-hash/09523e44e7b6fe1aed03347e30191c00d93500ce6e1f453e8dfdfae98d325f3e/linux-libc-headers/UNIHASH/build/recipe",
> $ echo 'MACHINE = "qemuarm"' >> conf/local.conf
> $ bitbake gcc-cross-arm
> $ grep linux-libc-headers tmp/deploy/spdx/3.0.1/x86_64/recipes/recipe-gcc-cross-arm.spdx.json
>         "http://spdxdocs.org/openembedded-alias/by-doc-hash/09523e44e7b6fe1aed03347e30191c00d93500ce6e1f453e8dfdfae98d325f3e/linux-libc-headers/UNIHASH/build/recipe",
>
> Explicitly add var-SSTATE_ARCHS_TUNEPKG to vardeps of do_create_spdx to
> trigger rebuild in this situation. Add the code along with adding
> linux-libc-headers to gcc-cross's depends
>
> After applying this commit, the link hash has changed along with machine changed
>
> $ echo 'MACHINE = "qemuarmv5"' >> conf/local.conf
> $ bitbake gcc-cross-arm
> $ grep linux-libc-headers tmp/deploy/spdx/3.0.1/x86_64/recipes/recipe-gcc-cross-arm.spdx.json
>         "http://spdxdocs.org/openembedded-alias/by-doc-hash/09523e44e7b6fe1aed03347e30191c00d93500ce6e1f453e8dfdfae98d325f3e/linux-libc-headers/UNIHASH/build/recipe",
> $ echo 'MACHINE = "qemuarm"' >> conf/local.conf
> $ bitbake gcc-cross-arm
> $ grep linux-libc-headers tmp/deploy/spdx/3.0.1/x86_64/recipes/recipe-gcc-cross-arm.spdx.json
>         "http://spdxdocs.org/openembedded-alias/by-doc-hash/087e29d3627a06f5377effa627b99907607f5af0fe8f6d116b4490650c404a55/linux-libc-headers/UNIHASH/build/recipe",
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
>  meta/recipes-devtools/gcc/gcc-cross.inc | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/meta/recipes-devtools/gcc/gcc-cross.inc b/meta/recipes-devtools/gcc/gcc-cross.inc
> index f85ccd5c23..d6936d063a 100644
> --- a/meta/recipes-devtools/gcc/gcc-cross.inc
> +++ b/meta/recipes-devtools/gcc/gcc-cross.inc
> @@ -7,6 +7,7 @@ PROVIDES = "virtual/${TARGET_PREFIX}gcc virtual/${TARGET_PREFIX}g++"
>  python () {
>      if d.getVar("TARGET_OS").startswith("linux"):
>          d.setVar("EXTRADEPENDS", "linux-libc-headers")
> +        d.appendVarFlag("do_create_spdx", "vardeps", " SSTATE_ARCHS_TUNEPKG")

Does this need to be conditional? Maybe it should just always be a
vardep of do_create_spdx?

>  }
>
>  PN = "gcc-cross-${TARGET_ARCH}"
> --
> 2.25.1
>

Re: [PATCH 3/3] gcc-cross.inc: add var-SSTATE_ARCHS_TUNEPKG to vardeps of do_create_spdx

[hongxu]

[-- Attachment #1: Type: text/plain, Size: 231 bytes --]

I think the condition is required, only if set linux-libc-headers to EXTRADEPENDS, it makes sense to add vardep of do_create_spdx.
That's why adding the code along with adding linux-libc-headers to gcc-cross's depends.
//Hongxu

[-- Attachment #2: Type: text/html, Size: 264 bytes --]

Re: [OE-core] [PATCH 1/3] meta/lib/oe/sbom30.py: correct alias

[Mathieu Dubois-Briand]

On Wed Dec 11, 2024 at 1:07 PM CET, hongxu via lists.openembedded.org wrote:
> After commit [spdx 3.0: Rework how SPDX aliases are linked] applied,
> it added extra "/" to namespace, which causing the replacement of
> UNIHASH missing a "/"
>
>   http://spdxdocs.org/openembedded-alias/by-doc-hash/0b308e4b9ad979f642d8787c61f76c31bdcad04837eeaaf8bc383f33f99bbeb8/flex-nativeUNIHASH/build/recipe
>
> After applying this commit to remove "/" from namespace.
>
>   http://spdxdocs.org/openembedded-alias/by-doc-hash/0b308e4b9ad979f642d8787c61f76c31bdcad04837eeaaf8bc383f33f99bbeb8/flex-native/UNIHASH/build/recipe
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---

Hi Hongxu,

We have some warnings on the autobuilder while picking these patches,
such as:

WARNING: core-image-sato-1.0-r0 do_create_image_sbom_spdx: The following SPDX IDs were unable to be resolved:
  http://spdxdocs.org/openembedded-alias/by-doc-hash/7f7bda815cc04b5b265ac8602753d177a96016826f71217624323d1afbb6a65e/gdk-pixbuf/UNIHASH/build/recipe
  http://spdxdocs.org/openembedded-alias/by-doc-hash/c0aa4c9cb1de03839f2a5e077627543f00e6269d6a16bfccf156b395df054f2d/gtk+3/UNIHASH/build/recipe

https://valkyrie.yoctoproject.org/#/builders/13/builds/557/steps/13/logs/stdio
https://valkyrie.yoctoproject.org/#/builders/89/builds/579/steps/17/logs/stdio
https://valkyrie.yoctoproject.org/#/builders/95/builds/610/steps/13/logs/stdio

Can you have a look at this issue please ?

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com