[PATCH wrynose 0/4] spdx: fix the compiled-sources filter

[PATCH wrynose 0/4] spdx: fix the compiled-sources filter

[kazuyoshi_akiyama]

SPDX_INCLUDE_COMPILED_SOURCES is meant to filter all source files down to
only the ones that were compiled before adding them to the SBOM,
but on wrynose it doesn't work.
add_package_files() is what picks which files go into the SBOM.
It walks ${SPDXWORK} itself to get file paths,
then keeps only the ones that match the list of compiled sources
returned by get_compiled_sources().
get_compiled_sources() just returns what save_debugsources_info() stored,
and that data uses paths rooted at ${TARGET_DBGSRC_DIR}
(= /usr/src/debug/${PN}/${PV}), not the ${SPDXWORK}-rooted paths
add_package_files() compares against.
So the list get_compiled_sources() returns never matches
what add_package_files() is comparing,
and almost every file ends up marked as not compiled.
This happens for every recipe except the kernel,
which has its own special-cased path.

This isn't just a matter of the SBOM missing the right files.
sbom-cve-check uses the same match result to decide whether
a vulnerable file was actually built, for its vulnerability triage.
If the match count is always zero, that triage doesn't work either.
On yocto-6.0.2 we confirmed the misjudgment affects cpio and perl.

master already has this fixed,
and this backport brings that fix to wrynose.
b567c2f0d9 is the commit that fixes it.
Everything else in the series is a prerequisite for that patch.
Backporting b567c2f0d9 along with its prerequisites
brings wrynose's behavior in line with master.
With this, compiled sources get counted correctly,
and sbom-cve-check's triage works correctly too.

On wrynose (genericx86-64, core-image-full-cmdline),
the match rate goes from effectively zero to 93.9%.

One thing changes.
What the SBOM records as a source file name changes too.
aa44a0f0eb re-unpacks under UNPACKDIR instead of WORKDIR,
so a recorded source file name changes
(e.g. sources/acl-2.3.2/... becomes acl-2.3.2/...).
This is a change to SBOM metadata only.
The actual build output is unaffected.
Anyone diffing SBOMs across LTS point releases will see it.

Signed-off-by: Kazuyoshi Akiyama <kazuyoshi_akiyama@jp.honda>
---
Benjamin Robin (4):
      spdx_common: simplify get_patched_src() implementation
      spdx_common: modify UNPACKDIR instead of WORKDIR in get_patched_src()
      spdx_common: In get_patched_src() ensure kernel dir name to be ${BP}
      package: no longer modify paths in save_debugsources_info()

 meta/lib/oe/package.py      |  54 +++++++++----------
 meta/lib/oe/spdx30_tasks.py |   1 -
 meta/lib/oe/spdx_common.py  | 125 +++++++++++++++++++++++++++-----------------
 3 files changed, 101 insertions(+), 79 deletions(-)
---
base-commit: 9da814ca3685ada3fce46b8987f04ed3d57247b7
change-id: 20260917-backport-wrynose-8fa831054074

Best regards,
--  
Kazuyoshi Akiyama <kazuyoshi_akiyama@jp.honda>

[PATCH wrynose 1/4] spdx_common: simplify get_patched_src() implementation

[kazuyoshi_akiyama]

From: Benjamin Robin <benjamin.robin@bootlin.com>

The behavior remains unchanged:
- Instead of saving and restoring the values of various variables, use a
  copy of the data store via `d.createCopy()`. This prevents side effects
  for other tasks.
- This allows removing the `try ... finally` block.
- The function `is_work_shared_spdx()` no longer needs to be called twice.
- In the work-shared case, there is no need to modify `WORKDIR` since no
  task is executed from that branch.

Signed-off-by: Benjamin Robin <benjamin.robin@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

(cherry picked from commit e0bf8e7c5be3d87376a6c2aab5aca07fd0813763)
Signed-off-by: Kazuyoshi Akiyama <kazuyoshi_akiyama@jp.honda>
---
 meta/lib/oe/spdx_common.py | 81 ++++++++++++++++++++++------------------------
 1 file changed, 39 insertions(+), 42 deletions(-)

diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py
index 6b1a409c40..9d1e22629b 100644
--- a/meta/lib/oe/spdx_common.py
+++ b/meta/lib/oe/spdx_common.py
@@ -172,50 +172,47 @@ def get_patched_src(d):
     Save patched source of the recipe in SPDX_WORKDIR.
     """
     spdx_workdir = d.getVar("SPDXWORK")
-    spdx_sysroot_native = d.getVar("STAGING_DIR_NATIVE")
 
-    workdir = d.getVar("WORKDIR")
+    # The kernel class functions require it to be on work-shared, so we don't change WORKDIR
+    if not is_work_shared_spdx(d):
+        spdx_sysroot_native = d.getVar("STAGING_DIR_NATIVE")
+        localdata = d.createCopy()
 
-    try:
-        # The kernel class functions require it to be on work-shared, so we dont change WORKDIR
-        if not is_work_shared_spdx(d):
-            # Change the WORKDIR to make do_unpack do_patch run in another dir.
-            d.setVar("WORKDIR", spdx_workdir)
-            # Restore the original path to recipe's native sysroot (it's relative to WORKDIR).
-            d.setVar("STAGING_DIR_NATIVE", spdx_sysroot_native)
-
-            # The changed 'WORKDIR' also caused 'B' changed, create dir 'B' for the
-            # possibly requiring of the following tasks (such as some recipes's
-            # do_patch required 'B' existed).
-            bb.utils.mkdirhier(d.getVar("B"))
-
-            bb.build.exec_func("do_unpack", d)
-
-            if d.getVar("SRC_URI") != "":
-                if bb.data.inherits_class("dos2unix", d):
-                    bb.build.exec_func("do_convert_crlf_to_lf", d)
-                bb.build.exec_func("do_patch", d)
-
-        # Copy source from work-share to spdx_workdir
-        if is_work_shared_spdx(d):
-            share_src = d.getVar("S")
-            d.setVar("WORKDIR", spdx_workdir)
-            d.setVar("STAGING_DIR_NATIVE", spdx_sysroot_native)
-            # Copy source to ${SPDXWORK}, same basename dir of ${S};
-            src_dir = spdx_workdir + "/" + os.path.basename(share_src)
-            # For kernel souce, rename suffix dir 'kernel-source'
-            # to ${BP} (${BPN}-${PV})
-            if bb.data.inherits_class("kernel", d):
-                src_dir = spdx_workdir + "/" + d.getVar("BP")
-
-            bb.note(f"copyhardlinktree {share_src} to {src_dir}")
-            oe.path.copyhardlinktree(share_src, src_dir)
-
-        # Some userland has no source.
-        if not os.path.exists(spdx_workdir):
-            bb.utils.mkdirhier(spdx_workdir)
-    finally:
-        d.setVar("WORKDIR", workdir)
+        # Change the WORKDIR to make do_unpack do_patch run in another dir.
+        localdata.setVar("WORKDIR", spdx_workdir)
+        # Restore the original path to recipe's native sysroot (it's relative to WORKDIR).
+        localdata.setVar("STAGING_DIR_NATIVE", spdx_sysroot_native)
+
+        # The changed 'WORKDIR' also caused 'B' changed, create dir 'B' for the
+        # possibly requiring of the following tasks (such as some recipe's
+        # do_patch required 'B' existed).
+        bb.utils.mkdirhier(localdata.getVar("B"))
+
+        bb.build.exec_func("do_unpack", localdata)
+
+        if localdata.getVar("SRC_URI") != "":
+            if bb.data.inherits_class("dos2unix", localdata):
+                bb.build.exec_func("do_convert_crlf_to_lf", localdata)
+            bb.build.exec_func("do_patch", localdata)
+
+    # Copy source from work-shared to spdx_workdir
+    else:
+        share_src = d.getVar("S")
+
+        if bb.data.inherits_class("kernel", d):
+            # For kernel source, rename suffix dir 'kernel-source' to ${BP} (${BPN}-${PV})
+            dir_name = d.getVar("BP")
+        else:
+            # Copy source to ${SPDXWORK}, same basename dir of ${S}
+            dir_name = os.path.basename(share_src)
+
+        src_dir = f"{spdx_workdir}/{dir_name}"
+        bb.note(f"copyhardlinktree {share_src} to {src_dir}")
+        oe.path.copyhardlinktree(share_src, src_dir)
+
+    # Some userland has no source.
+    if not os.path.exists(spdx_workdir):
+        bb.utils.mkdirhier(spdx_workdir)
 
 
 def has_task(d, task):

-- 
2.43.0

[PATCH wrynose 2/4] spdx_common: modify UNPACKDIR instead of WORKDIR in get_patched_src()

[kazuyoshi_akiyama]

From: Benjamin Robin <benjamin.robin@bootlin.com>

This change will affect the SPDX output: the source file path will no
longer be prefixed by `sources/`.

The rest of the changes are safe because:
- `S` should reference `UNPACKDIR` and not directly reference `WORKDIR`.
- After analyzing all public layers, no `do_unpack()` or `do_patch()` was
  found to reference the `${B}` variable. The comment mentioning the
  `${B}` variable traces back to [1] (though no further explanation was
  provided).

Additionally, this ensures proper capture of sources if the recipe
modifies `UNPACKDIR` outside of `WORKDIR` (though such cases should be
rare).

With these changes, SPDXWORK is no longer a working directory which is
"temporary" replacing WORKDIR, it is now a "temporary" directory containing
unpacked source files.
The run.do_patch.XXXX and associated files are now stored in ${WORKDIR}/temp.

[1] https://github.com/doubleopen-io/meta-doubleopen/blob/
e84f0c32663da9c743e4f1802042b733df0e36d7/classes/doubleopen.bbclass#L285

Signed-off-by: Benjamin Robin <benjamin.robin@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

(cherry picked from commit aa44a0f0eb5e260959ebd77f439a5d25b3c1aa2a)
Signed-off-by: Kazuyoshi Akiyama <kazuyoshi_akiyama@jp.honda>
---
 meta/lib/oe/spdx30_tasks.py |  1 -
 meta/lib/oe/spdx_common.py  | 18 +++++-------------
 2 files changed, 5 insertions(+), 14 deletions(-)

diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py
index 1821dd7de4..862e57c3d1 100644
--- a/meta/lib/oe/spdx30_tasks.py
+++ b/meta/lib/oe/spdx30_tasks.py
@@ -873,7 +873,6 @@ def create_spdx(d):
             lambda filepath: [oe.spdx30.software_SoftwarePurpose.source],
             license_data,
             ignore_dirs=[".git"],
-            ignore_top_level_dirs=["temp"],
             archive=None,
         )
         build_inputs |= files
diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py
index 9d1e22629b..013f23ae7c 100644
--- a/meta/lib/oe/spdx_common.py
+++ b/meta/lib/oe/spdx_common.py
@@ -169,28 +169,20 @@ def collect_package_providers(d, direct_deps):
 
 def get_patched_src(d):
     """
-    Save patched source of the recipe in SPDX_WORKDIR.
+    Save patched source of the recipe in SPDXWORK.
     """
     spdx_workdir = d.getVar("SPDXWORK")
 
-    # The kernel class functions require it to be on work-shared, so we don't change WORKDIR
+    # Do not unpack the sources again for the recipe using work-shared
     if not is_work_shared_spdx(d):
-        spdx_sysroot_native = d.getVar("STAGING_DIR_NATIVE")
         localdata = d.createCopy()
 
-        # Change the WORKDIR to make do_unpack do_patch run in another dir.
-        localdata.setVar("WORKDIR", spdx_workdir)
-        # Restore the original path to recipe's native sysroot (it's relative to WORKDIR).
-        localdata.setVar("STAGING_DIR_NATIVE", spdx_sysroot_native)
-
-        # The changed 'WORKDIR' also caused 'B' changed, create dir 'B' for the
-        # possibly requiring of the following tasks (such as some recipe's
-        # do_patch required 'B' existed).
-        bb.utils.mkdirhier(localdata.getVar("B"))
+        # Change the UNPACKDIR to make do_unpack do_patch run in another dir.
+        localdata.setVar("UNPACKDIR", spdx_workdir)
 
         bb.build.exec_func("do_unpack", localdata)
 
-        if localdata.getVar("SRC_URI") != "":
+        if localdata.getVar("SRC_URI"):
             if bb.data.inherits_class("dos2unix", localdata):
                 bb.build.exec_func("do_convert_crlf_to_lf", localdata)
             bb.build.exec_func("do_patch", localdata)

-- 
2.43.0

[PATCH wrynose 3/4] spdx_common: In get_patched_src() ensure kernel dir name to be ${BP}

[kazuyoshi_akiyama]

From: Benjamin Robin <benjamin.robin@bootlin.com>

If a kernel recipe is using a tar file instead of git repository, it is
necessary to override `S` to point to the unpacked tar archive.
In this case `${S}` is not going to be in work-shared. So we must handle
this case in get_patched_src() to ensure that the kernel is extracted in
the following path: `${SPDXWORK}/${BP}`.

For a full detailed analysis see [1], but in summary this change is
necessary because:
- In `save_debugsources_info()` the sources are extracted from the debug
  symbol. For the kernel Linux the source file path is modified to look
  like `${BP}/init/main.c` (`${KERNEL_SRC_PATH}` is replaced by `${BP}`).
- In `get_patched_src()` the sources of the recipe are extracted (again)
  in a sub-directory of `${SPDXWORK}`. If `${S}` is in work-shared and if
  the recipe inherits the kernel class, then the sources are extracted in
  `${SPDXWORK}/${BP}`.
- In `add_package_files()`, with topdir equal to `${SPDXWORK}`, all the
  files (recursively) found in topdir are listed. For each source file,
  if the file path (relative to topdir) is in the list of source files
  retrieved by save_debugsources_info, then the file is added to the SPDX
  SBoM.
- If we are using a tar archive, `${S}` is set for example to
  `${UNPACKDIR}/linux-${PV}`, so in `get_patched_src()` the sources are
  extracted in `${SPDXWORK}/sources/linux-${PV}` since
  `UNPACKDIR = ${WORKDIR}/sources`. `${BP}/init/main.c` is not in
  `${SPDXWORK}`, but `sources/linux-${PV}/init/main.c` is.

[1] https://github.com/bootlin/yocto-kiss/pull/26#discussion_r3626224833

Signed-off-by: Benjamin Robin <benjamin.robin@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

(cherry picked from commit 9bb9763c4c857edfc64aa2ad189a78a4bfb0c897)
Signed-off-by: Kazuyoshi Akiyama <kazuyoshi_akiyama@jp.honda>
---
 meta/lib/oe/spdx_common.py | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py
index 013f23ae7c..e40cf36bba 100644
--- a/meta/lib/oe/spdx_common.py
+++ b/meta/lib/oe/spdx_common.py
@@ -187,6 +187,14 @@ def get_patched_src(d):
                 bb.build.exec_func("do_convert_crlf_to_lf", localdata)
             bb.build.exec_func("do_patch", localdata)
 
+        if bb.data.inherits_class("kernel", localdata):
+            # For kernel source, rename suffix dir to ${BP} (${BPN}-${PV})
+            dir_name = localdata.getVar("BP")
+            kernel_dst_path = f"{spdx_workdir}/{dir_name}"
+            kernel_src_path = localdata.getVar("S")
+            if not os.path.exists(kernel_dst_path):
+                shutil.move(kernel_src_path, kernel_dst_path)
+
     # Copy source from work-shared to spdx_workdir
     else:
         share_src = d.getVar("S")

-- 
2.43.0

[PATCH wrynose 4/4] package: no longer modify paths in save_debugsources_info()

[kazuyoshi_akiyama]

From: Benjamin Robin <benjamin.robin@bootlin.com>

The source files saved in `${PN}-debugsources.json.zstd` are now saved as
is: "/usr/src/kernel/" is no longer replaced by "${BP}/". However, the
file list is still filtered to exclude "<internal>" or "<built-in>", ...

With this modification, the content of `${PN}-debugsources.json.zstd`
is no longer specific or tied to SPDX generation.

The logic that modified the source file paths to match what SPDX
generation expects is now in `oe.spdx_common.get_compiled_sources()`.

Furthermore, previously, for all non-kernel recipes the source paths were
never "resolved", since the KERNEL_SRC_PATH variable is always defined.
So the returned source file paths always started with
`/usr/src/debug/${PN}/${PV}` (the value of TARGET_DBGSRC_DIR).

Update the source file paths to match what the SPDX generation expects:
- In `get_patched_src()`, the sources of the recipe are extracted
  (again). The unpack task runs in a modified local context with
  UNPACKDIR set to the value of `${SPDXWORK}`, so the sources end up
  extracted in a sub-directory of `${SPDXWORK}`.
- In `add_package_files()`, all files below topdir, set to `${SPDXWORK}`,
  are (recursively) listed. For each source file, if its path (relative
  to topdir) is in the list returned by `get_compiled_sources()`, the
  file is added to the SPDX SBoM.

Handle this by replacing ${TARGET_DBGSRC_DIR} with the relative path of
${S} relative to ${UNPACKDIR}. If ${S} is not relative to ${UNPACKDIR},
do nothing.

Signed-off-by: Benjamin Robin <benjamin.robin@bootlin.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

(cherry picked from commit b567c2f0d91fbf36886d0c3a6f6ea90c07771044)
Signed-off-by: Kazuyoshi Akiyama <kazuyoshi_akiyama@jp.honda>
---
 meta/lib/oe/package.py     | 54 +++++++++++++++++++++-------------------------
 meta/lib/oe/spdx_common.py | 42 ++++++++++++++++++++++++++++++------
 2 files changed, 61 insertions(+), 35 deletions(-)

diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index c375acc124..bef3c92a25 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -1060,37 +1060,33 @@ def copydebugsources(debugsrcdir, sources, d):
                 os.rmdir(p)
 
 @bb.parse.vardepsexclude("BB_NUMBER_THREADS")
-def save_debugsources_info(debugsrcdir, sources_raw, d):
+def save_debugsources_info(sources_raw, d):
     import json
     import bb.compress.zstd
-    if debugsrcdir and sources_raw:
-        debugsources_file = d.expand("${PKGDESTWORK}/debugsources/${PN}-debugsources.json.zstd")
-        debugsources_dir = os.path.dirname(debugsources_file)
-        if not os.path.isdir(debugsources_dir):
-            bb.utils.mkdirhier(debugsources_dir)
-        bb.utils.remove(debugsources_file)
 
-        workdir = d.getVar("WORKDIR")
-        pn = d.getVar('PN')
-
-        # Kernel sources are in a different directory and are special case
-        # we format the sources as expected by spdx by replacing /usr/src/kernel/
-        # into BP/
-        kernel_src = d.getVar('KERNEL_SRC_PATH')
-        bp = d.getVar('BP')
-        sources_dict = {}
-        for file, src_files in sources_raw:
-            file_clean = file.replace(f"{workdir}/package/","")
-            sources_clean = [
-                src.replace(f"{debugsrcdir}/{pn}/", "")
-                if not kernel_src else src.replace(f"{kernel_src}/", f"{bp}/")
-                for src in src_files
-                if not any(keyword in src for keyword in ("<internal>", "<built-in>")) and not src.endswith("/")
-            ]
-            sources_dict[file_clean] = sorted(sources_clean)
-        num_threads = int(d.getVar("BB_NUMBER_THREADS"))
-        with bb.compress.zstd.open(debugsources_file, "wt", encoding="utf-8", num_threads=num_threads) as f:
-            json.dump(sources_dict, f, sort_keys=True)
+    if not sources_raw:
+        return
+
+    debugsources_file = d.expand("${PKGDESTWORK}/debugsources/${PN}-debugsources.json.zstd")
+    debugsources_dir = os.path.dirname(debugsources_file)
+    if not os.path.isdir(debugsources_dir):
+        bb.utils.mkdirhier(debugsources_dir)
+    bb.utils.remove(debugsources_file)
+
+    workdir = d.getVar("WORKDIR")
+
+    def _filter_src_file(src):
+        if src.endswith("/"):
+            return False
+        return not any(keyword in src for keyword in ("<internal>", "<built-in>"))
+
+    sources_dict = {}
+    for file, src_files in sources_raw:
+        file_clean = file.replace(f"{workdir}/package/", "")
+        sources_dict[file_clean] = sorted(filter(_filter_src_file, src_files))
+    num_threads = int(d.getVar("BB_NUMBER_THREADS"))
+    with bb.compress.zstd.open(debugsources_file, "wt", encoding="utf-8", num_threads=num_threads) as f:
+        json.dump(sources_dict, f, sort_keys=True)
 
 @bb.parse.vardepsexclude("BB_NUMBER_THREADS")
 def read_debugsources_info(d):
@@ -1337,7 +1333,7 @@ def process_split_and_strip_files(d):
         copydebugsources(dv["srcdir"], sources, d)
 
         # Save source info to be accessible to other tasks
-        save_debugsources_info(dv["srcdir"], results, d)
+        save_debugsources_info(results, d)
     #
     # End of debug splitting
     #
diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py
index e40cf36bba..f885f1832a 100644
--- a/meta/lib/oe/spdx_common.py
+++ b/meta/lib/oe/spdx_common.py
@@ -277,14 +277,44 @@ def get_compiled_sources(d):
         bb.debug(1, "Do not have debugsources.list. Skipping")
         return [], []
 
-    # Sources are not split now in SPDX, so we aggregate them
-    sources = set(itertools.chain.from_iterable(source_info.values()))
-    # Check extensions of files
+    unpackdir = d.getVar("UNPACKDIR")
+    srcdir = d.getVar("S")
+    bp = d.getVar("BP")
+    kernel_src = d.getVar("KERNEL_SRC_PATH")
+    dbgsrc_dir = d.getVar("TARGET_DBGSRC_DIR")
+
+    # Compute the relative path of source directory from ${UNPACKDIR}.
+    # The goal is to replace ${TARGET_DBGSRC_DIR} by this relative path.
+    srcdir_rel = None
+    if srcdir and unpackdir:
+        srcdir_rel = os.path.relpath(srcdir, unpackdir)
+        if srcdir_rel.startswith(".."):
+            srcdir_rel = None
+
+    sources = set()
     types = set()
-    for src in sources:
+
+    # Sources are not split now in SPDX, so we aggregate them
+    for src in set(itertools.chain.from_iterable(source_info.values())):
+        # In the common case, the sources are located in ${S}. To format them as
+        # expected by SPDX, we replace /usr/src/debug/${PN}/${PV} with the path
+        # of ${S} relative to ${UNPACKDIR}.
+        if dbgsrc_dir and srcdir_rel:
+            src = src.replace(f"{dbgsrc_dir}/", f"{srcdir_rel}/")
+
+        # Kernel sources are in a different directory and are special case
+        # we format the sources as expected by spdx by replacing /usr/src/kernel/
+        # into ${BP}/
+        if kernel_src and bp:
+            src = src.replace(f"{kernel_src}/", f"{bp}/")
+
+        sources.add(src)
+
+        # Check extensions of files
         basename = os.path.basename(src)
         ext = basename.partition(".")[2]
-        if ext not in types and ext:
+        if ext:
             types.add(ext)
-    bb.debug(1, f"Num of sources: {len(sources)} and types: {len(types)} {str(types)}")
+
+    bb.debug(1, f"Num of sources: {len(sources)} and types: {len(types)} {types!s}")
     return sources, types

-- 
2.43.0

Re: [OE-core] [PATCH wrynose 0/4] spdx: fix the compiled-sources filter

[Yoann Congal]

On Thu Sep 17, 2026 at 6:24 AM CEST, KAZUYOSHI AKIYAMA via lists.openembedded.org wrote:
> SPDX_INCLUDE_COMPILED_SOURCES is meant to filter all source files down to
> only the ones that were compiled before adding them to the SBOM,
> but on wrynose it doesn't work.
> add_package_files() is what picks which files go into the SBOM.
> It walks ${SPDXWORK} itself to get file paths,
> then keeps only the ones that match the list of compiled sources
> returned by get_compiled_sources().
> get_compiled_sources() just returns what save_debugsources_info() stored,
> and that data uses paths rooted at ${TARGET_DBGSRC_DIR}
> (= /usr/src/debug/${PN}/${PV}), not the ${SPDXWORK}-rooted paths
> add_package_files() compares against.
> So the list get_compiled_sources() returns never matches
> what add_package_files() is comparing,
> and almost every file ends up marked as not compiled.
> This happens for every recipe except the kernel,
> which has its own special-cased path.
>
> This isn't just a matter of the SBOM missing the right files.
> sbom-cve-check uses the same match result to decide whether
> a vulnerable file was actually built, for its vulnerability triage.
> If the match count is always zero, that triage doesn't work either.
> On yocto-6.0.2 we confirmed the misjudgment affects cpio and perl.
>
> master already has this fixed,
> and this backport brings that fix to wrynose.
> b567c2f0d9 is the commit that fixes it.
> Everything else in the series is a prerequisite for that patch.
> Backporting b567c2f0d9 along with its prerequisites
> brings wrynose's behavior in line with master.
> With this, compiled sources get counted correctly,
> and sbom-cve-check's triage works correctly too.
>
> On wrynose (genericx86-64, core-image-full-cmdline),
> the match rate goes from effectively zero to 93.9%.
>
> One thing changes.
> What the SBOM records as a source file name changes too.
> aa44a0f0eb re-unpacks under UNPACKDIR instead of WORKDIR,
> so a recorded source file name changes
> (e.g. sources/acl-2.3.2/... becomes acl-2.3.2/...).
> This is a change to SBOM metadata only.
> The actual build output is unaffected.
> Anyone diffing SBOMs across LTS point releases will see it.

Hello,

Thanks for the series, this sounds like a worthy fix.
But, this SBOM change is a bit more intrusive than I'm confortable
with... This will create a lot of noise for users that precisely track
their SBOMs.

Can we do the fix without this change?

Thanks!
-- 
Yoann Congal
Smile ECS

Re: [OE-core] [PATCH wrynose 0/4] spdx: fix the compiled-sources filter

[Benjamin Robin]

On Saturday, September 26, 2026 at 11:19 AM, Yoann Congal wrote:
> On Thu Sep 17, 2026 at 6:24 AM CEST, KAZUYOSHI AKIYAMA via lists.openembedded.org wrote:
> > SPDX_INCLUDE_COMPILED_SOURCES is meant to filter all source files down to
> > only the ones that were compiled before adding them to the SBOM,
> > but on wrynose it doesn't work.
> > add_package_files() is what picks which files go into the SBOM.
> > It walks ${SPDXWORK} itself to get file paths,
> > then keeps only the ones that match the list of compiled sources
> > returned by get_compiled_sources().
> > get_compiled_sources() just returns what save_debugsources_info() stored,
> > and that data uses paths rooted at ${TARGET_DBGSRC_DIR}
> > (= /usr/src/debug/${PN}/${PV}), not the ${SPDXWORK}-rooted paths
> > add_package_files() compares against.
> > So the list get_compiled_sources() returns never matches
> > what add_package_files() is comparing,
> > and almost every file ends up marked as not compiled.
> > This happens for every recipe except the kernel,
> > which has its own special-cased path.
> >
> > This isn't just a matter of the SBOM missing the right files.
> > sbom-cve-check uses the same match result to decide whether
> > a vulnerable file was actually built, for its vulnerability triage.
> > If the match count is always zero, that triage doesn't work either.
> > On yocto-6.0.2 we confirmed the misjudgment affects cpio and perl.
> >
> > master already has this fixed,
> > and this backport brings that fix to wrynose.
> > b567c2f0d9 is the commit that fixes it.
> > Everything else in the series is a prerequisite for that patch.
> > Backporting b567c2f0d9 along with its prerequisites
> > brings wrynose's behavior in line with master.
> > With this, compiled sources get counted correctly,
> > and sbom-cve-check's triage works correctly too.
> >
> > On wrynose (genericx86-64, core-image-full-cmdline),
> > the match rate goes from effectively zero to 93.9%.
> >
> > One thing changes.
> > What the SBOM records as a source file name changes too.
> > aa44a0f0eb re-unpacks under UNPACKDIR instead of WORKDIR,
> > so a recorded source file name changes
> > (e.g. sources/acl-2.3.2/... becomes acl-2.3.2/...).
> > This is a change to SBOM metadata only.
> > The actual build output is unaffected.
> > Anyone diffing SBOMs across LTS point releases will see it.
> 
> Hello,
> 
> Thanks for the series, this sounds like a worthy fix.
> But, this SBOM change is a bit more intrusive than I'm confortable
> with... This will create a lot of noise for users that precisely track
> their SBOMs.

Hello Yoann,

> Can we do the fix without this change?

Technically, we could do it. But it wouldn't be exactly the same series,
which I don't really like, and the code will diverge.
For information I could develop a solution after ELCE capable of
generating a similar SBOM if Kazuyoshi doesn't feel comfortable doing it.


-- 
Benjamin Robin, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

RE: [OE-core] [PATCH wrynose 0/4] spdx: fix the compiled-sources filter

[KAZUYOSHI AKIYAMA (秋山 和慶)]

Thank you for your comments, Yoann and Benjamin.

I agree with Benjamin.
This series does introduce a visible change that the source path in SPDX will be changed from `sources/XXX/YYY.c` to `XXX/YYY.c`.
However, the design change of the patch 0002 is reasonable, and I believe it is better to avoid code divergence whenever possible.
The impact of this change is very small.
I could write a patch to preserve the `sources/` prefix, but even then, wrynose users would still need to adapt to this change in the next LTS release.
Furthermore, if I write such a patch, the path change would occur silently in the next LTS release, without sufficient context to document it properly in the release notes.

-----Original Message-----
From: Benjamin Robin <benjamin.robin@bootlin.com> 
Sent: Monday, September 28, 2026 5:57 PM
To: KAZUYOSHI AKIYAMA (秋山 和慶) <kazuyoshi_akiyama@jp.honda>; openembedded-core@lists.openembedded.org; Yoann Congal <yoann.congal@smile.fr>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>; Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Subject: Re: [OE-core] [PATCH wrynose 0/4] spdx: fix the compiled-sources filter

On Saturday, September 26, 2026 at 11:19 AM, Yoann Congal wrote:
> On Thu Sep 17, 2026 at 6:24 AM CEST, KAZUYOSHI AKIYAMA via lists.openembedded.org wrote:
> > SPDX_INCLUDE_COMPILED_SOURCES is meant to filter all source files 
> > down to only the ones that were compiled before adding them to the 
> > SBOM, but on wrynose it doesn't work.
> > add_package_files() is what picks which files go into the SBOM.
> > It walks ${SPDXWORK} itself to get file paths, then keeps only the 
> > ones that match the list of compiled sources returned by 
> > get_compiled_sources().
> > get_compiled_sources() just returns what save_debugsources_info() 
> > stored, and that data uses paths rooted at ${TARGET_DBGSRC_DIR} (= 
> > /usr/src/debug/${PN}/${PV}), not the ${SPDXWORK}-rooted paths
> > add_package_files() compares against.
> > So the list get_compiled_sources() returns never matches what 
> > add_package_files() is comparing, and almost every file ends up 
> > marked as not compiled.
> > This happens for every recipe except the kernel, which has its own 
> > special-cased path.
> >
> > This isn't just a matter of the SBOM missing the right files.
> > sbom-cve-check uses the same match result to decide whether a 
> > vulnerable file was actually built, for its vulnerability triage.
> > If the match count is always zero, that triage doesn't work either.
> > On yocto-6.0.2 we confirmed the misjudgment affects cpio and perl.
> >
> > master already has this fixed,
> > and this backport brings that fix to wrynose.
> > b567c2f0d9 is the commit that fixes it.
> > Everything else in the series is a prerequisite for that patch.
> > Backporting b567c2f0d9 along with its prerequisites brings wrynose's 
> > behavior in line with master.
> > With this, compiled sources get counted correctly, and 
> > sbom-cve-check's triage works correctly too.
> >
> > On wrynose (genericx86-64, core-image-full-cmdline), the match rate 
> > goes from effectively zero to 93.9%.
> >
> > One thing changes.
> > What the SBOM records as a source file name changes too.
> > aa44a0f0eb re-unpacks under UNPACKDIR instead of WORKDIR, so a 
> > recorded source file name changes (e.g. sources/acl-2.3.2/... 
> > becomes acl-2.3.2/...).
> > This is a change to SBOM metadata only.
> > The actual build output is unaffected.
> > Anyone diffing SBOMs across LTS point releases will see it.
> 
> Hello,
> 
> Thanks for the series, this sounds like a worthy fix.
> But, this SBOM change is a bit more intrusive than I'm confortable 
> with... This will create a lot of noise for users that precisely track 
> their SBOMs.

Hello Yoann,

> Can we do the fix without this change?

Technically, we could do it. But it wouldn't be exactly the same series, which I don't really like, and the code will diverge.
For information I could develop a solution after ELCE capable of generating a similar SBOM if Kazuyoshi doesn't feel comfortable doing it.


--
Benjamin Robin, Bootlin
Embedded Linux and Kernel engineering
https://urldefense.com/v3/__https://bootlin.com__;!!KEiQlu1B30Bt!3IKXBRtAGpk1LcoaKtP_q1pM91WPOIIVa-2qIAJhzvnrSCmUbNDoWnu6RU-TaPNhiDq0ol8mRpUivxOLMiHHZ6clSr7ziv8FvOs$ 

Re: [OE-core] [PATCH wrynose 0/4] spdx: fix the compiled-sources filter

[Yoann Congal]

Hello,

@Joshua (and other SPDX/SBOM interested parties):
As our "SPDX/SBOM guy", I would love to have your input on this thread.
FYI, We (Paul or I) plan to raise this at the next tech call.

>> -----Original Message-----
>> From: Benjamin Robin <benjamin.robin@bootlin.com> 
>> Sent: Monday, September 28, 2026 5:57 PM
>> To: KAZUYOSHI AKIYAMA (秋山 和慶) <kazuyoshi_akiyama@jp.honda>; openembedded-core@lists.openembedded.org; Yoann Congal <yoann.congal@smile.fr>
>> Cc: Richard Purdie <richard.purdie@linuxfoundation.org>; Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>> Subject: Re: [OE-core] [PATCH wrynose 0/4] spdx: fix the compiled-sources filter
>>
>> On Saturday, September 26, 2026 at 11:19 AM, Yoann Congal wrote:
>>> On Thu Sep 17, 2026 at 6:24 AM CEST, KAZUYOSHI AKIYAMA via lists.openembedded.org wrote:
>>> > SPDX_INCLUDE_COMPILED_SOURCES is meant to filter all source files 
>>> > down to only the ones that were compiled before adding them to the 
>>> > SBOM, but on wrynose it doesn't work.
>>> > add_package_files() is what picks which files go into the SBOM.
>>> > It walks ${SPDXWORK} itself to get file paths, then keeps only the 
>>> > ones that match the list of compiled sources returned by 
>>> > get_compiled_sources().
>>> > get_compiled_sources() just returns what save_debugsources_info() 
>>> > stored, and that data uses paths rooted at ${TARGET_DBGSRC_DIR} (= 
>>> > /usr/src/debug/${PN}/${PV}), not the ${SPDXWORK}-rooted paths
>>> > add_package_files() compares against.
>>> > So the list get_compiled_sources() returns never matches what 
>>> > add_package_files() is comparing, and almost every file ends up 
>>> > marked as not compiled.
>>> > This happens for every recipe except the kernel, which has its own 
>>> > special-cased path.
>>> >
>>> > This isn't just a matter of the SBOM missing the right files.
>>> > sbom-cve-check uses the same match result to decide whether a 
>>> > vulnerable file was actually built, for its vulnerability triage.
>>> > If the match count is always zero, that triage doesn't work either.
>>> > On yocto-6.0.2 we confirmed the misjudgment affects cpio and perl.
>>> >
>>> > master already has this fixed,
>>> > and this backport brings that fix to wrynose.
>>> > b567c2f0d9 is the commit that fixes it.
>>> > Everything else in the series is a prerequisite for that patch.
>>> > Backporting b567c2f0d9 along with its prerequisites brings wrynose's 
>>> > behavior in line with master.
>>> > With this, compiled sources get counted correctly, and 
>>> > sbom-cve-check's triage works correctly too.
>>> >
>>> > On wrynose (genericx86-64, core-image-full-cmdline), the match rate 
>>> > goes from effectively zero to 93.9%.
>>> >
>>> > One thing changes.
>>> > What the SBOM records as a source file name changes too.
>>> > aa44a0f0eb re-unpacks under UNPACKDIR instead of WORKDIR, so a 
>>> > recorded source file name changes (e.g. sources/acl-2.3.2/... 
>>> > becomes acl-2.3.2/...).
>>> > This is a change to SBOM metadata only.
>>> > The actual build output is unaffected.
>>> > Anyone diffing SBOMs across LTS point releases will see it.
>>> 
>>> Hello,
>>> 
>>> Thanks for the series, this sounds like a worthy fix.
>>> But, this SBOM change is a bit more intrusive than I'm confortable 
>>> with... This will create a lot of noise for users that precisely track 
>>> their SBOMs.
>>
>> Hello Yoann,
>>
>>> Can we do the fix without this change?
>>
>> Technically, we could do it. But it wouldn't be exactly the same series, which I don't really like, and the code will diverge.
>> For information I could develop a solution after ELCE capable of generating a similar SBOM if Kazuyoshi doesn't feel comfortable doing it.
>> --
>> Benjamin Robin, Bootlin
>> Embedded Linux and Kernel engineering

On Mon Sep 28, 2026 at 11:39 AM CEST, KAZUYOSHI AKIYAMA (秋山 和慶) wrote:
> Thank you for your comments, Yoann and Benjamin.
>
> I agree with Benjamin.
> This series does introduce a visible change that the source path in
> SPDX will be changed from `sources/XXX/YYY.c` to `XXX/YYY.c`.
> However, the design change of the patch 0002 is reasonable, and I
> believe it is better to avoid code divergence whenever possible.
> The impact of this change is very small.
Do you have anything to support this affirmation?
I can imagine users having quite strict processes around SBOMs that
could be disrupted by a change like this (e.g. diff'ing SBOMs across
versions).

> I could write a patch to preserve the `sources/` prefix, but even
> then, wrynose users would still need to adapt to this change in the
> next LTS release.
> Furthermore, if I write such a patch, the path change would occur
> silently in the next LTS release, without sufficient context to
> document it properly in the release notes.
Breaking changes like this when changing release is expected and not a
problem at all.

Regards,
-- 
Yoann Congal
Smile ECS

Re: [OE-core] [PATCH wrynose 0/4] spdx: fix the compiled-sources filter

[Yoann Congal]

On Thu Sep 17, 2026 at 6:24 AM CEST, KAZUYOSHI AKIYAMA via lists.openembedded.org wrote:
> SPDX_INCLUDE_COMPILED_SOURCES is meant to filter all source files down to
> only the ones that were compiled before adding them to the SBOM,
> but on wrynose it doesn't work.
> add_package_files() is what picks which files go into the SBOM.
> It walks ${SPDXWORK} itself to get file paths,
> then keeps only the ones that match the list of compiled sources
> returned by get_compiled_sources().
> get_compiled_sources() just returns what save_debugsources_info() stored,
> and that data uses paths rooted at ${TARGET_DBGSRC_DIR}
> (= /usr/src/debug/${PN}/${PV}), not the ${SPDXWORK}-rooted paths
> add_package_files() compares against.
> So the list get_compiled_sources() returns never matches
> what add_package_files() is comparing,
> and almost every file ends up marked as not compiled.
> This happens for every recipe except the kernel,
> which has its own special-cased path.
>
> This isn't just a matter of the SBOM missing the right files.
> sbom-cve-check uses the same match result to decide whether
> a vulnerable file was actually built, for its vulnerability triage.
> If the match count is always zero, that triage doesn't work either.
> On yocto-6.0.2 we confirmed the misjudgment affects cpio and perl.
>
> master already has this fixed,
> and this backport brings that fix to wrynose.
> b567c2f0d9 is the commit that fixes it.
> Everything else in the series is a prerequisite for that patch.
> Backporting b567c2f0d9 along with its prerequisites
> brings wrynose's behavior in line with master.
> With this, compiled sources get counted correctly,
> and sbom-cve-check's triage works correctly too.
>
> On wrynose (genericx86-64, core-image-full-cmdline),
> the match rate goes from effectively zero to 93.9%.
>
> One thing changes.
> What the SBOM records as a source file name changes too.
> aa44a0f0eb re-unpacks under UNPACKDIR instead of WORKDIR,
> so a recorded source file name changes
> (e.g. sources/acl-2.3.2/... becomes acl-2.3.2/...).
> This is a change to SBOM metadata only.
> The actual build output is unaffected.
> Anyone diffing SBOMs across LTS point releases will see it.
>
> Signed-off-by: Kazuyoshi Akiyama <kazuyoshi_akiyama@jp.honda>
> ---
> Benjamin Robin (4):
>       spdx_common: simplify get_patched_src() implementation
>       spdx_common: modify UNPACKDIR instead of WORKDIR in get_patched_src()
>       spdx_common: In get_patched_src() ensure kernel dir name to be ${BP}
>       package: no longer modify paths in save_debugsources_info()
>
>  meta/lib/oe/package.py      |  54 +++++++++----------
>  meta/lib/oe/spdx30_tasks.py |   1 -
>  meta/lib/oe/spdx_common.py  | 125 +++++++++++++++++++++++++++-----------------
>  3 files changed, 101 insertions(+), 79 deletions(-)
> ---
> base-commit: 9da814ca3685ada3fce46b8987f04ed3d57247b7
> change-id: 20260917-backport-wrynose-8fa831054074
>
> Best regards,
> --  
> Kazuyoshi Akiyama <kazuyoshi_akiyama@jp.honda>

Independently of the "stable-worthyness" in the sibling thread. I tested
this and got a failure that I think is related to this series:
oe-selftest-fedora rocky8-vk-1 wrynose completed at 2026-10-05 18:01:07+00:00
https://autobuilder.yoctoproject.org/valkyrie/?#/builders/48/builds/4820

oe-selftest-debian debian12-vk-9 wrynose completed at 2026-10-08 11:44:05+00:00
https://autobuilder.yoctoproject.org/valkyrie/?#/builders/35/builds/5026
Re-run as:
  oe-selftest-debian debian12-vk-9 wrynose completed at 2026-10-08 11:44:05+00:00
  https://autobuilder.yoctoproject.org/valkyrie/?#/builders/35/builds/5026

oe-selftest-fedora opensuse160-vk-2 wrynose completed at 2026-10-08 11:45:38+00:00
https://autobuilder.yoctoproject.org/valkyrie/?#/builders/48/builds/4840
Re-run as:
  oe-selftest-fedora opensuse160-vk-2 wrynose completed at 2026-10-08 11:45:38+00:00
  https://autobuilder.yoctoproject.org/valkyrie/?#/builders/48/builds/4840

I started a build without this series:
oe-selftest-debian debian12-vk-6 wrynose still running
https://autobuilder.yoctoproject.org/valkyrie/?#/builders/35/builds/5027

Regards,
-- 
Yoann Congal
Smile ECS

Re: [OE-core] [PATCH wrynose 0/4] spdx: fix the compiled-sources filter

[Yoann Congal]

On Thu Oct 8, 2026 at 3:50 PM CEST, Yoann Congal wrote:
> On Thu Sep 17, 2026 at 6:24 AM CEST, KAZUYOSHI AKIYAMA via lists.openembedded.org wrote:
>> SPDX_INCLUDE_COMPILED_SOURCES is meant to filter all source files down to
>> only the ones that were compiled before adding them to the SBOM,
>> but on wrynose it doesn't work.
>> add_package_files() is what picks which files go into the SBOM.
>> It walks ${SPDXWORK} itself to get file paths,
>> then keeps only the ones that match the list of compiled sources
>> returned by get_compiled_sources().
>> get_compiled_sources() just returns what save_debugsources_info() stored,
>> and that data uses paths rooted at ${TARGET_DBGSRC_DIR}
>> (= /usr/src/debug/${PN}/${PV}), not the ${SPDXWORK}-rooted paths
>> add_package_files() compares against.
>> So the list get_compiled_sources() returns never matches
>> what add_package_files() is comparing,
>> and almost every file ends up marked as not compiled.
>> This happens for every recipe except the kernel,
>> which has its own special-cased path.
>>
>> This isn't just a matter of the SBOM missing the right files.
>> sbom-cve-check uses the same match result to decide whether
>> a vulnerable file was actually built, for its vulnerability triage.
>> If the match count is always zero, that triage doesn't work either.
>> On yocto-6.0.2 we confirmed the misjudgment affects cpio and perl.
>>
>> master already has this fixed,
>> and this backport brings that fix to wrynose.
>> b567c2f0d9 is the commit that fixes it.
>> Everything else in the series is a prerequisite for that patch.
>> Backporting b567c2f0d9 along with its prerequisites
>> brings wrynose's behavior in line with master.
>> With this, compiled sources get counted correctly,
>> and sbom-cve-check's triage works correctly too.
>>
>> On wrynose (genericx86-64, core-image-full-cmdline),
>> the match rate goes from effectively zero to 93.9%.
>>
>> One thing changes.
>> What the SBOM records as a source file name changes too.
>> aa44a0f0eb re-unpacks under UNPACKDIR instead of WORKDIR,
>> so a recorded source file name changes
>> (e.g. sources/acl-2.3.2/... becomes acl-2.3.2/...).
>> This is a change to SBOM metadata only.
>> The actual build output is unaffected.
>> Anyone diffing SBOMs across LTS point releases will see it.
>>
>> Signed-off-by: Kazuyoshi Akiyama <kazuyoshi_akiyama@jp.honda>
>> ---
>> Benjamin Robin (4):
>>       spdx_common: simplify get_patched_src() implementation
>>       spdx_common: modify UNPACKDIR instead of WORKDIR in get_patched_src()
>>       spdx_common: In get_patched_src() ensure kernel dir name to be ${BP}
>>       package: no longer modify paths in save_debugsources_info()
>>
>>  meta/lib/oe/package.py      |  54 +++++++++----------
>>  meta/lib/oe/spdx30_tasks.py |   1 -
>>  meta/lib/oe/spdx_common.py  | 125 +++++++++++++++++++++++++++-----------------
>>  3 files changed, 101 insertions(+), 79 deletions(-)
>> ---
>> base-commit: 9da814ca3685ada3fce46b8987f04ed3d57247b7
>> change-id: 20260917-backport-wrynose-8fa831054074
>>
>> Best regards,
>> --  
>> Kazuyoshi Akiyama <kazuyoshi_akiyama@jp.honda>
>
> Independently of the "stable-worthyness" in the sibling thread. I tested
> this and got a failure that I think is related to this series:
> oe-selftest-fedora rocky8-vk-1 wrynose completed at 2026-10-05 18:01:07+00:00
> https://autobuilder.yoctoproject.org/valkyrie/?#/builders/48/builds/4820
>
> oe-selftest-debian debian12-vk-9 wrynose completed at 2026-10-08 11:44:05+00:00
> https://autobuilder.yoctoproject.org/valkyrie/?#/builders/35/builds/5026
> Re-run as:
>   oe-selftest-debian debian12-vk-9 wrynose completed at 2026-10-08 11:44:05+00:00
>   https://autobuilder.yoctoproject.org/valkyrie/?#/builders/35/builds/5026
>
> oe-selftest-fedora opensuse160-vk-2 wrynose completed at 2026-10-08 11:45:38+00:00
> https://autobuilder.yoctoproject.org/valkyrie/?#/builders/48/builds/4840
> Re-run as:
>   oe-selftest-fedora opensuse160-vk-2 wrynose completed at 2026-10-08 11:45:38+00:00
>   https://autobuilder.yoctoproject.org/valkyrie/?#/builders/48/builds/4840
>
> I started a build without this series:
> oe-selftest-debian debian12-vk-6 wrynose still running
> https://autobuilder.yoctoproject.org/valkyrie/?#/builders/35/builds/5027

That build (5027) also failed like those before but did not have this
series. I conclude that this series does not cause these failures.
Sorry for the noise.
-- 
Yoann Congal
Smile ECS