* [PATCH 0/2] package: improve save_debugsources_info()
@ 2026-08-10 7:11 Benjamin Robin
2026-08-10 7:11 ` [PATCH 1/2] package: Fix source info when PACKAGE_DEBUG_STATIC_SPLIT is set Benjamin Robin
2026-08-10 7:11 ` [PATCH 2/2] package: fix source path in save_debugsources_info() Benjamin Robin
0 siblings, 2 replies; 8+ messages in thread
From: Benjamin Robin @ 2026-08-10 7:11 UTC (permalink / raw)
To: openembedded-core
Cc: jpewhacker, antonin.godard, mathieu.dubois-briand,
thomas.petazzoni, daniel.turull, Benjamin Robin
The overall goal of this series is to fix SPDX SBoM generation in such a way
that it correctly includes compiled sources for "normal" recipes (a recipe
which does not inherit the kernel class).
This was tested with the busybox recipe. The following command list the
effective list of compiled sources.
python3 -m json.tool \
tmp/deploy/spdx/3.0.1/cortexa15t2hf-neon/builds/build-busybox.spdx.json | \
grep -F '.c"'
This is the same list as the extracted sources from the debug symbol:
zstdcat tmp/pkgdata/qemuarm/debugsources/busybox-debugsources.json.zstd | \
python3 -m json.tool
Signed-off-by: Benjamin Robin <benjamin.robin@bootlin.com>
---
Benjamin Robin (2):
package: Fix source info when PACKAGE_DEBUG_STATIC_SPLIT is set
package: fix source path in save_debugsources_info()
meta/lib/oe/package.py | 79 ++++++++++++++++++++++++++++++++------------------
1 file changed, 51 insertions(+), 28 deletions(-)
---
base-commit: 20f678d825d1b8a1e8bfa88dedd51eb628c96d51
change-id: 20260810-fix-save-debugsources-info-b8d0ac4ea206
Best regards,
--
Benjamin Robin <benjamin.robin@bootlin.com>
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 1/2] package: Fix source info when PACKAGE_DEBUG_STATIC_SPLIT is set 2026-08-10 7:11 [PATCH 0/2] package: improve save_debugsources_info() Benjamin Robin @ 2026-08-10 7:11 ` Benjamin Robin 2026-08-10 7:11 ` [PATCH 2/2] package: fix source path in save_debugsources_info() Benjamin Robin 1 sibling, 0 replies; 8+ messages in thread From: Benjamin Robin @ 2026-08-10 7:11 UTC (permalink / raw) To: openembedded-core Cc: jpewhacker, antonin.godard, mathieu.dubois-briand, thomas.petazzoni, daniel.turull, Benjamin Robin Do not override the list of ELF files, with associated source info, when PACKAGE_DEBUG_STATIC_SPLIT is set. Extend the list instead. When PACKAGE_DEBUG_STATIC_SPLIT is not set, use a generator expression instead of calling append for each element: Small cleanup. Signed-off-by: Benjamin Robin <benjamin.robin@bootlin.com> --- meta/lib/oe/package.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index d047e41a78ec..4a244ec9801e 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -1292,10 +1292,9 @@ def process_split_and_strip_files(d): if dv["srcdir"] and not hostos.startswith("mingw"): if (d.getVar('PACKAGE_DEBUG_STATIC_SPLIT') == '1'): - results = oe.utils.multiprocess_launch(splitstaticdebuginfo, staticlibs, d, extraargs=(dvar, dv, d)) + results.extend(oe.utils.multiprocess_launch(splitstaticdebuginfo, staticlibs, d, extraargs=(dvar, dv, d))) else: - for file in staticlibs: - results.append( (file,source_info(file, d)) ) + results.extend((file, source_info(file, d)) for file in staticlibs) d.setVar("PKGDEBUGSOURCES", {strip_pkgd_prefix(f): sorted(s) for f, s in results}) -- 2.55.0 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 2/2] package: fix source path in save_debugsources_info() 2026-08-10 7:11 [PATCH 0/2] package: improve save_debugsources_info() Benjamin Robin 2026-08-10 7:11 ` [PATCH 1/2] package: Fix source info when PACKAGE_DEBUG_STATIC_SPLIT is set Benjamin Robin @ 2026-08-10 7:11 ` Benjamin Robin 2026-08-16 10:46 ` Paul Barker 1 sibling, 1 reply; 8+ messages in thread From: Benjamin Robin @ 2026-08-10 7:11 UTC (permalink / raw) To: openembedded-core Cc: jpewhacker, antonin.godard, mathieu.dubois-briand, thomas.petazzoni, daniel.turull, Benjamin Robin Previously, except for a kernel recipe, the source file paths were never "resolved" since the KERNEL_SRC_PATH variable is always defined. So in the ${PN}-debugsources.json.zstd file the source file paths always started with /usr/src/debug/${PN}/${PV} (which is the value of TARGET_DBGSRC_DIR). Currently the debugsources.json file is only used by the spdx generation. - In `get_patched_src()` the sources of the recipe are extracted (again). The unpack task is executed from a modified local context with UNPACKDIR set to the value of `${SPDXWORK}`. So in summary the sources are extracted in a sub-directory of `${SPDXWORK}`. - In `add_package_files()`, with topdir equal to `${SPDXWORK}`, all the files (recursively) found in topdir are listed. For each source file, if the file path (relative to topdir) is in the list of source files retrieved by save_debugsources_info, then the file is added to the SPDX SBoM. So try to handle that by replacing ${TARGET_DBGSRC_DIR} by the relative path of ${S} relative to ${UNPACKDIR}. If ${S} is not relative to ${UNPACKDIR}, do nothing. Signed-off-by: Benjamin Robin <benjamin.robin@bootlin.com> --- meta/lib/oe/package.py | 74 +++++++++++++++++++++++++++++++++----------------- 1 file changed, 49 insertions(+), 25 deletions(-) diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index 4a244ec9801e..229b9caa5d82 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -1088,37 +1088,61 @@ def copydebugsources(debugsrcdir, sources, d): os.rmdir(p) @bb.parse.vardepsexclude("BB_NUMBER_THREADS") -def save_debugsources_info(debugsrcdir, sources_raw, d): +def save_debugsources_info(sources_raw, d): import json import bb.compress.zstd - if debugsrcdir and sources_raw: - debugsources_file = d.expand("${PKGDESTWORK}/debugsources/${PN}-debugsources.json.zstd") - debugsources_dir = os.path.dirname(debugsources_file) - if not os.path.isdir(debugsources_dir): - bb.utils.mkdirhier(debugsources_dir) - bb.utils.remove(debugsources_file) - workdir = d.getVar("WORKDIR") - pn = d.getVar('PN') + if not sources_raw: + return + + debugsources_file = d.expand("${PKGDESTWORK}/debugsources/${PN}-debugsources.json.zstd") + debugsources_dir = os.path.dirname(debugsources_file) + if not os.path.isdir(debugsources_dir): + bb.utils.mkdirhier(debugsources_dir) + bb.utils.remove(debugsources_file) + + workdir = d.getVar("WORKDIR") + unpackdir = d.getVar("UNPACKDIR") + srcdir = d.getVar("S") + bp = d.getVar("BP") + kernel_src = d.getVar("KERNEL_SRC_PATH") + dbgsrc_dir = d.getVar("TARGET_DBGSRC_DIR") + + # Compute the relative path of source directory from ${UNPACKDIR}. + # The goal is to replace ${TARGET_DBGSRC_DIR} by this relative path. + srcdir_rel = None + if srcdir and unpackdir: + srcdir_rel = os.path.relpath(srcdir, unpackdir) + if srcdir_rel.startswith(".."): + srcdir_rel = None + + def _resolve_source_path(p): + # In the common case, the sources are located in ${S}. To format them as + # expected by SPDX, we replace /usr/src/debug/${PN}/${PV} with the path + # of ${S} relative to ${UNPACKDIR}. + if dbgsrc_dir and srcdir_rel: + p = p.replace(f"{dbgsrc_dir}/", f"{srcdir_rel}/") # Kernel sources are in a different directory and are special case # we format the sources as expected by spdx by replacing /usr/src/kernel/ # into BP/ - kernel_src = d.getVar('KERNEL_SRC_PATH') - bp = d.getVar('BP') - sources_dict = {} - for file, src_files in sources_raw: - file_clean = file.replace(f"{workdir}/package/","") - sources_clean = [ - src.replace(f"{debugsrcdir}/{pn}/", "") - if not kernel_src else src.replace(f"{kernel_src}/", f"{bp}/") - for src in src_files - if not any(keyword in src for keyword in ("<internal>", "<built-in>")) and not src.endswith("/") - ] - sources_dict[file_clean] = sorted(sources_clean) - num_threads = int(d.getVar("BB_NUMBER_THREADS")) - with bb.compress.zstd.open(debugsources_file, "wt", encoding="utf-8", num_threads=num_threads) as f: - json.dump(sources_dict, f, sort_keys=True) + if kernel_src and bp: + p = p.replace(f"{kernel_src}/", f"{bp}/") + + return p + + sources_dict = {} + for file, src_files in sources_raw: + file_clean = file.replace(f"{workdir}/package/", "") + sources_clean = [ + _resolve_source_path(src) + for src in src_files + if not any(keyword in src for keyword in ("<internal>", "<built-in>")) and not src.endswith("/") + ] + sources_dict[file_clean] = sorted(sources_clean) + num_threads = int(d.getVar("BB_NUMBER_THREADS")) + with bb.compress.zstd.open(debugsources_file, "wt", encoding="utf-8", num_threads=num_threads) as f: + json.dump(sources_dict, f, sort_keys=True) @bb.parse.vardepsexclude("BB_NUMBER_THREADS") def read_debugsources_info(d): @@ -1364,7 +1388,7 @@ def process_split_and_strip_files(d): copydebugsources(dv["srcdir"], sources, d) # Save source info to be accessible to other tasks - save_debugsources_info(dv["srcdir"], results, d) + save_debugsources_info(results, d) # # End of debug splitting # -- 2.55.0 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 2/2] package: fix source path in save_debugsources_info() 2026-08-10 7:11 ` [PATCH 2/2] package: fix source path in save_debugsources_info() Benjamin Robin @ 2026-08-16 10:46 ` Paul Barker 2026-08-16 11:12 ` Benjamin Robin 0 siblings, 1 reply; 8+ messages in thread From: Paul Barker @ 2026-08-16 10:46 UTC (permalink / raw) To: Benjamin Robin, openembedded-core Cc: jpewhacker, antonin.godard, mathieu.dubois-briand, thomas.petazzoni, daniel.turull On Mon, 2026-08-10 at 09:11 +0200, Benjamin Robin wrote: > Previously, except for a kernel recipe, the source file paths were never > "resolved" since the KERNEL_SRC_PATH variable is always defined. So in > the ${PN}-debugsources.json.zstd file the source file paths always started > with /usr/src/debug/${PN}/${PV} (which is the value of TARGET_DBGSRC_DIR). > > Currently the debugsources.json file is only used by the spdx generation. > - In `get_patched_src()` the sources of the recipe are extracted (again). > The unpack task is executed from a modified local context with UNPACKDIR > set to the value of `${SPDXWORK}`. So in summary the sources are > extracted in a sub-directory of `${SPDXWORK}`. > - In `add_package_files()`, with topdir equal to `${SPDXWORK}`, all the > files (recursively) found in topdir are listed. For each source file, > if the file path (relative to topdir) is in the list of source files > retrieved by save_debugsources_info, then the file is added to the SPDX > SBoM. > > So try to handle that by replacing ${TARGET_DBGSRC_DIR} by the relative > path of ${S} relative to ${UNPACKDIR}. If ${S} is not relative to > ${UNPACKDIR}, do nothing. > > Signed-off-by: Benjamin Robin <benjamin.robin@bootlin.com> The paths in ${PN}-debugsources.json currently match where the files will be installed on the target. If we change these to be relative paths within ${UNPACKDIR} then we would break other ways that the debugsources json files may be used. What is currently broken? Can this be fixed at the point where the debugsources json file is parsed instead of where it is generated? Sorry if I'm missing some context here. Best regards, -- Paul Barker ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 2/2] package: fix source path in save_debugsources_info() 2026-08-16 10:46 ` Paul Barker @ 2026-08-16 11:12 ` Benjamin Robin 2026-08-16 11:25 ` Benjamin Robin 0 siblings, 1 reply; 8+ messages in thread From: Benjamin Robin @ 2026-08-16 11:12 UTC (permalink / raw) To: openembedded-core, Paul Barker Cc: jpewhacker, antonin.godard, mathieu.dubois-briand, thomas.petazzoni, daniel.turull On Sunday, August 16, 2026 at 12:46 PM, Paul Barker wrote: > On Mon, 2026-08-10 at 09:11 +0200, Benjamin Robin wrote: > > Previously, except for a kernel recipe, the source file paths were never > > "resolved" since the KERNEL_SRC_PATH variable is always defined. So in > > the ${PN}-debugsources.json.zstd file the source file paths always started > > with /usr/src/debug/${PN}/${PV} (which is the value of TARGET_DBGSRC_DIR). > > > > Currently the debugsources.json file is only used by the spdx generation. > > - In `get_patched_src()` the sources of the recipe are extracted (again). > > The unpack task is executed from a modified local context with UNPACKDIR > > set to the value of `${SPDXWORK}`. So in summary the sources are > > extracted in a sub-directory of `${SPDXWORK}`. > > - In `add_package_files()`, with topdir equal to `${SPDXWORK}`, all the > > files (recursively) found in topdir are listed. For each source file, > > if the file path (relative to topdir) is in the list of source files > > retrieved by save_debugsources_info, then the file is added to the SPDX > > SBoM. > > > > So try to handle that by replacing ${TARGET_DBGSRC_DIR} by the relative > > path of ${S} relative to ${UNPACKDIR}. If ${S} is not relative to > > ${UNPACKDIR}, do nothing. > > > > Signed-off-by: Benjamin Robin <benjamin.robin@bootlin.com> > > The paths in ${PN}-debugsources.json currently match where the files > will be installed on the target. If we change these to be relative > paths within ${UNPACKDIR} then we would break other ways that the > debugsources json files may be used. Hello Paul, This was never the purpose of ${PN}-debugsources.json if I am not mistaken. If you look at the code (before my patches) the path should have been modified to be somewhat relative to WORKDIR. But the code had a bug, and the paths were never modified. Also, for the kernel, the kernel sources paths were already modified to be the same as the path in SPDX, so starting with ${BP}. This part was mostly working for the "main" use case. In a previous RFC series that was merged, I fix that to be working for any kernel recipe. > What is currently broken? SPDX_INCLUDE_COMPILED_SOURCES for a normal recipe (not the kernel) is not working. > Can this be fixed at the point where the > debugsources json file is parsed instead of where it is generated? Sorry > if I'm missing some context here. For the full context see: https://lore.kernel.org/all/20260727-fix-get-patched-src-v1-0-f5054ca10e14@bootlin.com/ https://github.com/bootlin/yocto-kiss/pull/26#discussion_r3626224833 -- Benjamin Robin, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 2/2] package: fix source path in save_debugsources_info() 2026-08-16 11:12 ` Benjamin Robin @ 2026-08-16 11:25 ` Benjamin Robin 2026-08-16 11:37 ` Paul Barker 0 siblings, 1 reply; 8+ messages in thread From: Benjamin Robin @ 2026-08-16 11:25 UTC (permalink / raw) To: openembedded-core, Paul Barker Cc: jpewhacker, antonin.godard, mathieu.dubois-briand, thomas.petazzoni, daniel.turull On Sunday, August 16, 2026 at 1:12 PM, Benjamin Robin wrote: > On Sunday, August 16, 2026 at 12:46 PM, Paul Barker wrote: > > On Mon, 2026-08-10 at 09:11 +0200, Benjamin Robin wrote: > > > Previously, except for a kernel recipe, the source file paths were never > > > "resolved" since the KERNEL_SRC_PATH variable is always defined. So in > > > the ${PN}-debugsources.json.zstd file the source file paths always started > > > with /usr/src/debug/${PN}/${PV} (which is the value of TARGET_DBGSRC_DIR). > > > > > > Currently the debugsources.json file is only used by the spdx generation. > > > - In `get_patched_src()` the sources of the recipe are extracted (again). > > > The unpack task is executed from a modified local context with UNPACKDIR > > > set to the value of `${SPDXWORK}`. So in summary the sources are > > > extracted in a sub-directory of `${SPDXWORK}`. > > > - In `add_package_files()`, with topdir equal to `${SPDXWORK}`, all the > > > files (recursively) found in topdir are listed. For each source file, > > > if the file path (relative to topdir) is in the list of source files > > > retrieved by save_debugsources_info, then the file is added to the SPDX > > > SBoM. > > > > > > So try to handle that by replacing ${TARGET_DBGSRC_DIR} by the relative > > > path of ${S} relative to ${UNPACKDIR}. If ${S} is not relative to > > > ${UNPACKDIR}, do nothing. > > > > > > Signed-off-by: Benjamin Robin <benjamin.robin@bootlin.com> > > > > The paths in ${PN}-debugsources.json currently match where the files > > will be installed on the target. If we change these to be relative > > paths within ${UNPACKDIR} then we would break other ways that the > > debugsources json files may be used. > > Hello Paul, > > This was never the purpose of ${PN}-debugsources.json if I am not mistaken. > If you look at the code (before my patches) the path should have been > modified to be somewhat relative to WORKDIR. But the code had a bug, and > the paths were never modified. > > Also, for the kernel, the kernel sources paths were already modified to be > the same as the path in SPDX, so starting with ${BP}. This part was > mostly working for the "main" use case. In a previous RFC series that was > merged, I fix that to be working for any kernel recipe. > > > What is currently broken? > > SPDX_INCLUDE_COMPILED_SOURCES for a normal recipe (not the kernel) is not > working. > > > Can this be fixed at the point where the > > debugsources json file is parsed instead of where it is generated? Yes, I guess we could modify how oe.spdx_common.get_compiled_sources() is implemented. In that case I would recommend to no longer modify the paths of kernel sources in save_debugsources_info(). We would keep the paths as is, we would only filter for "<internal>", "<built-in>", ... In any cases, this is kind of a breaking change since the ${PN}-debugsources.json is not going to contain the same paths as before. But what you are "proposing" (modifying get_compiled_sources()) is a bit cleaner from my point of view. This is a bit more work, that is why I did not do that. Joshua do you have an option on that? > > Sorry if I'm missing some context here. > > For the full context see: > https://lore.kernel.org/all/20260727-fix-get-patched-src-v1-0-f5054ca10e14@bootlin.com/ > https://github.com/bootlin/yocto-kiss/pull/26#discussion_r3626224833 -- Benjamin Robin, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 2/2] package: fix source path in save_debugsources_info() 2026-08-16 11:25 ` Benjamin Robin @ 2026-08-16 11:37 ` Paul Barker 2026-08-16 12:19 ` Benjamin Robin 0 siblings, 1 reply; 8+ messages in thread From: Paul Barker @ 2026-08-16 11:37 UTC (permalink / raw) To: Benjamin Robin, openembedded-core Cc: jpewhacker, antonin.godard, mathieu.dubois-briand, thomas.petazzoni, daniel.turull On Sun, 2026-08-16 at 13:25 +0200, Benjamin Robin wrote: > On Sunday, August 16, 2026 at 1:12 PM, Benjamin Robin wrote: > > On Sunday, August 16, 2026 at 12:46 PM, Paul Barker wrote: > > > On Mon, 2026-08-10 at 09:11 +0200, Benjamin Robin wrote: > > > > Previously, except for a kernel recipe, the source file paths were never > > > > "resolved" since the KERNEL_SRC_PATH variable is always defined. So in > > > > the ${PN}-debugsources.json.zstd file the source file paths always started > > > > with /usr/src/debug/${PN}/${PV} (which is the value of TARGET_DBGSRC_DIR). > > > > > > > > Currently the debugsources.json file is only used by the spdx generation. > > > > - In `get_patched_src()` the sources of the recipe are extracted (again). > > > > The unpack task is executed from a modified local context with UNPACKDIR > > > > set to the value of `${SPDXWORK}`. So in summary the sources are > > > > extracted in a sub-directory of `${SPDXWORK}`. > > > > - In `add_package_files()`, with topdir equal to `${SPDXWORK}`, all the > > > > files (recursively) found in topdir are listed. For each source file, > > > > if the file path (relative to topdir) is in the list of source files > > > > retrieved by save_debugsources_info, then the file is added to the SPDX > > > > SBoM. > > > > > > > > So try to handle that by replacing ${TARGET_DBGSRC_DIR} by the relative > > > > path of ${S} relative to ${UNPACKDIR}. If ${S} is not relative to > > > > ${UNPACKDIR}, do nothing. > > > > > > > > Signed-off-by: Benjamin Robin <benjamin.robin@bootlin.com> > > > > > > The paths in ${PN}-debugsources.json currently match where the files > > > will be installed on the target. If we change these to be relative > > > paths within ${UNPACKDIR} then we would break other ways that the > > > debugsources json files may be used. > > > > Hello Paul, > > > > This was never the purpose of ${PN}-debugsources.json if I am not mistaken. > > If you look at the code (before my patches) the path should have been > > modified to be somewhat relative to WORKDIR. But the code had a bug, and > > the paths were never modified. > > > > Also, for the kernel, the kernel sources paths were already modified to be > > the same as the path in SPDX, so starting with ${BP}. This part was > > mostly working for the "main" use case. In a previous RFC series that was > > merged, I fix that to be working for any kernel recipe. > > > > > What is currently broken? > > > > SPDX_INCLUDE_COMPILED_SOURCES for a normal recipe (not the kernel) is not > > working. > > > > > Can this be fixed at the point where the > > > debugsources json file is parsed instead of where it is generated? > > Yes, I guess we could modify how oe.spdx_common.get_compiled_sources() is > implemented. In that case I would recommend to no longer modify the paths > of kernel sources in save_debugsources_info(). We would keep the paths as is, > we would only filter for "<internal>", "<built-in>", ... > > In any cases, this is kind of a breaking change since the > ${PN}-debugsources.json is not going to contain the same paths as before. > > But what you are "proposing" (modifying get_compiled_sources()) is a bit > cleaner from my point of view. This is a bit more work, that is why I did > not do that. > Joshua do you have an option on that? > > > > Sorry if I'm missing some context here. > > > > For the full context see: > > https://lore.kernel.org/all/20260727-fix-get-patched-src-v1-0-f5054ca10e14@bootlin.com/ > > https://github.com/bootlin/yocto-kiss/pull/26#discussion_r3626224833 Hi Benjamin, Yes, I was missing some context! I was replying based off discussions we had on the patch review call on Thursday. If the paths in debugsources.json files are currently inconsistent between kernel and non-kernel recipes then we should fix that. So perhaps your patch is correct after all. I'll let Joshua give an opinion. Best regards, -- Paul Barker ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 2/2] package: fix source path in save_debugsources_info() 2026-08-16 11:37 ` Paul Barker @ 2026-08-16 12:19 ` Benjamin Robin 0 siblings, 0 replies; 8+ messages in thread From: Benjamin Robin @ 2026-08-16 12:19 UTC (permalink / raw) To: openembedded-core, Paul Barker Cc: jpewhacker, antonin.godard, mathieu.dubois-briand, thomas.petazzoni, daniel.turull On Sunday, August 16, 2026 at 1:37 PM, Paul Barker wrote: > On Sun, 2026-08-16 at 13:25 +0200, Benjamin Robin wrote: > > On Sunday, August 16, 2026 at 1:12 PM, Benjamin Robin wrote: > > > On Sunday, August 16, 2026 at 12:46 PM, Paul Barker wrote: > > > > On Mon, 2026-08-10 at 09:11 +0200, Benjamin Robin wrote: > > > > > Previously, except for a kernel recipe, the source file paths were never > > > > > "resolved" since the KERNEL_SRC_PATH variable is always defined. So in > > > > > the ${PN}-debugsources.json.zstd file the source file paths always started > > > > > with /usr/src/debug/${PN}/${PV} (which is the value of TARGET_DBGSRC_DIR). > > > > > > > > > > Currently the debugsources.json file is only used by the spdx generation. > > > > > - In `get_patched_src()` the sources of the recipe are extracted (again). > > > > > The unpack task is executed from a modified local context with UNPACKDIR > > > > > set to the value of `${SPDXWORK}`. So in summary the sources are > > > > > extracted in a sub-directory of `${SPDXWORK}`. > > > > > - In `add_package_files()`, with topdir equal to `${SPDXWORK}`, all the > > > > > files (recursively) found in topdir are listed. For each source file, > > > > > if the file path (relative to topdir) is in the list of source files > > > > > retrieved by save_debugsources_info, then the file is added to the SPDX > > > > > SBoM. > > > > > > > > > > So try to handle that by replacing ${TARGET_DBGSRC_DIR} by the relative > > > > > path of ${S} relative to ${UNPACKDIR}. If ${S} is not relative to > > > > > ${UNPACKDIR}, do nothing. > > > > > > > > > > Signed-off-by: Benjamin Robin <benjamin.robin@bootlin.com> > > > > > > > > The paths in ${PN}-debugsources.json currently match where the files > > > > will be installed on the target. If we change these to be relative > > > > paths within ${UNPACKDIR} then we would break other ways that the > > > > debugsources json files may be used. > > > > > > Hello Paul, > > > > > > This was never the purpose of ${PN}-debugsources.json if I am not mistaken. > > > If you look at the code (before my patches) the path should have been > > > modified to be somewhat relative to WORKDIR. But the code had a bug, and > > > the paths were never modified. > > > > > > Also, for the kernel, the kernel sources paths were already modified to be > > > the same as the path in SPDX, so starting with ${BP}. This part was > > > mostly working for the "main" use case. In a previous RFC series that was > > > merged, I fix that to be working for any kernel recipe. > > > > > > > What is currently broken? > > > > > > SPDX_INCLUDE_COMPILED_SOURCES for a normal recipe (not the kernel) is not > > > working. > > > > > > > Can this be fixed at the point where the > > > > debugsources json file is parsed instead of where it is generated? > > > > Yes, I guess we could modify how oe.spdx_common.get_compiled_sources() is > > implemented. In that case I would recommend to no longer modify the paths > > of kernel sources in save_debugsources_info(). We would keep the paths as is, > > we would only filter for "<internal>", "<built-in>", ... > > > > In any cases, this is kind of a breaking change since the > > ${PN}-debugsources.json is not going to contain the same paths as before. > > > > But what you are "proposing" (modifying get_compiled_sources()) is a bit > > cleaner from my point of view. This is a bit more work, that is why I did > > not do that. > > Joshua do you have an option on that? > > > > > > Sorry if I'm missing some context here. > > > > > > For the full context see: > > > https://lore.kernel.org/all/20260727-fix-get-patched-src-v1-0-f5054ca10e14@bootlin.com/ > > > https://github.com/bootlin/yocto-kiss/pull/26#discussion_r3626224833 > > Hi Benjamin, > > Yes, I was missing some context! I was replying based off discussions we > had on the patch review call on Thursday. > > If the paths in debugsources.json files are currently inconsistent > between kernel and non-kernel recipes then we should fix that. So > perhaps your patch is correct after all. I'll let Joshua give an > opinion. Be aware that with all my patches, the content of debugsources.json will still contain paths starting with /usr/src/debug/ since there are paths from others recipes (header files from glibc for example, ...). I am not sure I will have time this week to take a look at the second solution (modifying get_compiled_sources which is a bit cleaner from my point of view). I'll let Joshua give his opinion first :) -- Benjamin Robin, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-08-16 12:19 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-08-10 7:11 [PATCH 0/2] package: improve save_debugsources_info() Benjamin Robin 2026-08-10 7:11 ` [PATCH 1/2] package: Fix source info when PACKAGE_DEBUG_STATIC_SPLIT is set Benjamin Robin 2026-08-10 7:11 ` [PATCH 2/2] package: fix source path in save_debugsources_info() Benjamin Robin 2026-08-16 10:46 ` Paul Barker 2026-08-16 11:12 ` Benjamin Robin 2026-08-16 11:25 ` Benjamin Robin 2026-08-16 11:37 ` Paul Barker 2026-08-16 12:19 ` Benjamin Robin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox