* [PATCH v2 0/6] Check compiled files to filter kernel CVEs
@ 2025-04-28 13:41 daniel.turull
2025-04-28 13:42 ` [PATCH v2 1/6] linux-vulns: fetch kernel.org CNA info daniel.turull
` (5 more replies)
0 siblings, 6 replies; 14+ messages in thread
From: daniel.turull @ 2025-04-28 13:41 UTC (permalink / raw)
To: openembedded-core
Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun,
Daniel Turull
From: Daniel Turull <daniel.turull@ericsson.com>
Since kernel.org became a CNA, more information is available in the published CVEs, including details about which files are affected by a given CVE.
This series adds functionality to fetch the database from kernel.org CNA information:
git.kernel.org/pub/scm/linux/security/vulns.git
I have rewritten the original patch and integrated it with cve_check.py, cve_check, vex and create-spdx.
To filter out CVEs that are not applicable, we extract the files used during the kernel compilation and
compare it with the metadata in the CVE.
The CVE_CHECK_KERNEL is enabled by default, but not the CVE_CHECK_KERNEL_CONFIG, since it
needs to build the kernel and it is dependent on kernel configuration.
To include the files into SPDX, SPDX_INCLUDE_SOURCES needs to be enabled.
This could use as a base to run the vulnerability check independently and run an external tool to filter the CVEs.
In addition, we also integrate the data with the output of cve_check, vex, and create-spdx.
As a side effect of using better data from directly the kernel CNA, the number of vulnerabilities reported
increases.
Numbers from 2025-04-28
Standing CVEs:
- Before: 18 CVEs, mostly old that needs to be checked and clean up in a new commit. There is only one from 2025
- Checking with kernel vulns database: 82 CVEs
- Checking with only compiled files: 46 CVEs
v1: initial proposal
v2:
- rewrite kernel_vulns to fetch similarly as cve-update-db-native
- add functionality into cve_check.py, for the classes that uses oe.get_patched_cves function
- add linux-vulns into the cve-check results
- add only compiled files in the spdx, so the check can be done outside the build
- include compiled files into spdx when CVE_CHECK_KERNEL_CONFIG and SPDX_INCLUDE_SOURCES is enabled
Daniel Turull
Daniel Turull (6):
linux-vulns: fetch kernel.org CNA info
cve-check: fix debug message
kernel: add support to extract compiled files
cve-check: move message outsite check_cves and sort
cve-check, vex, spdx: use metadata from linux-vulns to enhance CVE
reporting
spdx: add option to include only compiled kernel files
meta/classes-recipe/kernel.bbclass | 11 ++
meta/classes/create-spdx-2.2.bbclass | 8 +
meta/classes/cve-check.bbclass | 35 +++-
meta/classes/spdx-common.bbclass | 7 +
meta/classes/vex.bbclass | 10 ++
meta/conf/distro/include/maintainers.inc | 1 +
meta/lib/oe/cve_check.py | 210 +++++++++++++++++++++-
meta/lib/oe/spdx30_tasks.py | 8 +
meta/lib/oe/spdx_common.py | 34 ++++
meta/recipes-core/meta/linux-vulns_git.bb | 76 ++++++++
10 files changed, 391 insertions(+), 9 deletions(-)
create mode 100644 meta/recipes-core/meta/linux-vulns_git.bb
^ permalink raw reply [flat|nested] 14+ messages in thread* [PATCH v2 1/6] linux-vulns: fetch kernel.org CNA info 2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull @ 2025-04-28 13:42 ` daniel.turull 2025-04-28 13:42 ` [PATCH v2 2/6] cve-check: fix debug message daniel.turull ` (4 subsequent siblings) 5 siblings, 0 replies; 14+ messages in thread From: daniel.turull @ 2025-04-28 13:42 UTC (permalink / raw) To: openembedded-core Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun, Daniel Turull, Peter Marko From: Daniel Turull <daniel.turull@ericsson.com> Add CVE data source for kernel.org. It includes more information than the one provided by NVD. Use similar mechanism and same variables as cve-check to define when to update. To use without internet access, change variable VULNS_URL to a local copy or mirror. CC: Peter Marko <peter.marko@siemens.com> Signed-off-by: Daniel Turull <daniel.turull@ericsson.com> --- meta/conf/distro/include/maintainers.inc | 1 + meta/recipes-core/meta/linux-vulns_git.bb | 76 +++++++++++++++++++++++ 2 files changed, 77 insertions(+) create mode 100644 meta/recipes-core/meta/linux-vulns_git.bb diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc index 8065287c17..ec427fe6a4 100644 --- a/meta/conf/distro/include/maintainers.inc +++ b/meta/conf/distro/include/maintainers.inc @@ -468,6 +468,7 @@ RECIPE_MAINTAINER:pn-lighttpd = "Unassigned <unassigned@yoctoproject.org>" RECIPE_MAINTAINER:pn-linux-dummy = "Unassigned <unassigned@yoctoproject.org>" RECIPE_MAINTAINER:pn-linux-firmware = "Otavio Salvador <otavio.salvador@ossystems.com.br>" RECIPE_MAINTAINER:pn-linux-libc-headers = "Bruce Ashfield <bruce.ashfield@gmail.com>" +RECIPE_MAINTAINER:pn-linux-vulns = "Unassigned <unassigned@yoctoproject.org>" RECIPE_MAINTAINER:pn-linux-yocto = "Bruce Ashfield <bruce.ashfield@gmail.com>" RECIPE_MAINTAINER:pn-linux-yocto-dev = "Bruce Ashfield <bruce.ashfield@gmail.com>" RECIPE_MAINTAINER:pn-linux-yocto-rt = "Bruce Ashfield <bruce.ashfield@gmail.com>" diff --git a/meta/recipes-core/meta/linux-vulns_git.bb b/meta/recipes-core/meta/linux-vulns_git.bb new file mode 100644 index 0000000000..fc48558eb8 --- /dev/null +++ b/meta/recipes-core/meta/linux-vulns_git.bb @@ -0,0 +1,76 @@ +SUMMARY = "CVE information from kernel.org" +DESCRIPTION = "Repo for tracking and maintaining the CVE identifiers reserved \ +and assigned to the Linux kernel project." +HOMEPAGE = "https://git.kernel.org/pub/scm/linux/security/vulns.git/about/" +LICENSE = "GPL-2.0-only & cve-tou" +SECTION = "base" + +INHIBIT_DEFAULT_DEPS = "1" + +inherit native +inherit nopackages + +VULNS_URL ?= "https://git.kernel.org/pub/scm/linux/security/vulns" +CVE_CHECK_KERNEL_DB_DIR ??= "${DL_DIR}/CVE_CHECK/vulns" + +# Use same intervals as cve-update-db-native. By default: once a day (24*60*60). +# Use 0 to force the update +# Use a negative value to skip the update + +CVE_DB_UPDATE_INTERVAL ??= "86400" + +python do_fetch(){ + import os + import bb.utils + + bb.utils.export_proxies(d) + db_file = d.getVar("CVE_CHECK_KERNEL_DB_DIR") + repo_url = d.getVar("VULNS_URL") + + try: + import time + update_interval = int(d.getVar("CVE_DB_UPDATE_INTERVAL")) + + if update_interval < 0: + bb.note("Kernel CVE database update skipped") + return + if time.time() - os.path.getmtime(db_file) < update_interval: + bb.debug(2,"Kernel CVE database, recently updated, skipping") + return + + except OSError: + pass + + bb.utils.mkdirhier(os.path.dirname(db_file)) + # Configure cmd + if not os.path.exists(db_file): + cmd = f"git clone {repo_url} {db_file}" + else: + cmd = f"git -C {db_file} pull" + try: + bb.fetch2.runfetchcmd(cmd, d) + except bb.fetch2.FetchError as e: + bb.warn(f"Kernel vulns repo url not accessible. {repo_url}") + bb.warn("Set VULNS_URL in local.conf to point to a local copy or mirror") +} + +do_clean() { + rm -rf ${CVE_CHECK_KERNEL_DB_DIR} +} + +deltask do_patch +deltask do_unpack +deltask do_configure +deltask do_compile +deltask do_install +deltask do_populate_sysroot +deltask do_runtime_spdx +deltask do_create_spdx +deltask do_populate_lic +deltask do_cve_check + +do_fetch[nostamp] = "1" +do_fetch[file-checksums] = "" +do_fetch[vardeps] = "" + +EXCLUDE_FROM_WORLD = "1" ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 2/6] cve-check: fix debug message 2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull 2025-04-28 13:42 ` [PATCH v2 1/6] linux-vulns: fetch kernel.org CNA info daniel.turull @ 2025-04-28 13:42 ` daniel.turull 2025-04-28 13:42 ` [PATCH v2 3/6] kernel: add support to extract compiled files daniel.turull ` (3 subsequent siblings) 5 siblings, 0 replies; 14+ messages in thread From: daniel.turull @ 2025-04-28 13:42 UTC (permalink / raw) To: openembedded-core Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun, Daniel Turull From: Daniel Turull <daniel.turull@ericsson.com> Debug level was not added as a parameter, causing a warning. Signed-off-by: Daniel Turull <daniel.turull@ericsson.com> --- meta/classes/cve-check.bbclass | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass index 1aef00d297..86ddfaae5f 100644 --- a/meta/classes/cve-check.bbclass +++ b/meta/classes/cve-check.bbclass @@ -288,16 +288,16 @@ def cve_update(d, cve_data, cve, entry): if entry['status'] == "version-in-range" and cve_data[cve]['status'] == "version-not-in-range": # New result from the scan, vulnerable cve_data[cve] = entry - bb.debug("CVE entry %s update from Patched to Unpatched from the scan result" % cve) + bb.debug(1, "CVE entry %s update from Patched to Unpatched from the scan result" % cve) return if entry['abbrev-status'] == "Patched" and cve_data[cve]['abbrev-status'] == "Unpatched": if entry['status'] == "version-not-in-range" and cve_data[cve]['status'] == "version-in-range": # Range does not match the scan, but we already have a vulnerable match, ignore - bb.debug("CVE entry %s update from Patched to Unpatched from the scan result - not applying" % cve) + bb.debug(1, "CVE entry %s update from Patched to Unpatched from the scan result - not applying" % cve) return # If we have an "Ignored", it has a priority if cve_data[cve]['abbrev-status'] == "Ignored": - bb.debug("CVE %s not updating because Ignored" % cve) + bb.debug(1, "CVE %s not updating because Ignored" % cve) return bb.warn("Unhandled CVE entry update for %s from %s to %s" % (cve, cve_data[cve], entry)) ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 3/6] kernel: add support to extract compiled files 2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull 2025-04-28 13:42 ` [PATCH v2 1/6] linux-vulns: fetch kernel.org CNA info daniel.turull 2025-04-28 13:42 ` [PATCH v2 2/6] cve-check: fix debug message daniel.turull @ 2025-04-28 13:42 ` daniel.turull 2025-04-28 14:24 ` [OE-core] " Bruce Ashfield 2025-04-28 13:42 ` [PATCH v2 4/6] cve-check: move message outsite check_cves and sort daniel.turull ` (2 subsequent siblings) 5 siblings, 1 reply; 14+ messages in thread From: daniel.turull @ 2025-04-28 13:42 UTC (permalink / raw) To: openembedded-core Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun, Daniel Turull, Peter Marko From: Daniel Turull <daniel.turull@ericsson.com> Use gen_compile_commands.py to extract files used during compilation for the used kernel configuration. CC: Peter Marko <peter.marko@siemens.com> Signed-off-by: Daniel Turull <daniel.turull@ericsson.com> --- meta/classes-recipe/kernel.bbclass | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass index 36ce659762..56060f2c91 100644 --- a/meta/classes-recipe/kernel.bbclass +++ b/meta/classes-recipe/kernel.bbclass @@ -867,3 +867,14 @@ EXPORT_FUNCTIONS do_deploy # Add using Device Tree support inherit kernel-devicetree + +KERNEL_FILES_DIR ?= "${LOG_DIR}/cve/kernel_files" +KERNEL_SRC_FILES ?= "${KERNEL_FILES_DIR}/compile_commands.json" + +do_save_compiled_files() { + bbdebug 1 "Saving compiled files in ${KERNEL_SRC_FILES}" + mkdir -p ${KERNEL_FILES_DIR} + ${S}/scripts/clang-tools/gen_compile_commands.py -o ${KERNEL_SRC_FILES} -d ${B} +} + +addtask do_save_compiled_files after do_compile do_compile_kernelmodules ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [OE-core] [PATCH v2 3/6] kernel: add support to extract compiled files 2025-04-28 13:42 ` [PATCH v2 3/6] kernel: add support to extract compiled files daniel.turull @ 2025-04-28 14:24 ` Bruce Ashfield 2025-04-28 14:28 ` Daniel Turull 0 siblings, 1 reply; 14+ messages in thread From: Bruce Ashfield @ 2025-04-28 14:24 UTC (permalink / raw) To: daniel.turull Cc: openembedded-core, rybczynska, steve, Peter.Marko, ross.burton, skandigraun [-- Attachment #1: Type: text/plain, Size: 2067 bytes --] On Mon, Apr 28, 2025 at 9:42 AM Daniel Turull via lists.openembedded.org <daniel.turull=ericsson.com@lists.openembedded.org> wrote: > From: Daniel Turull <daniel.turull@ericsson.com> > > Use gen_compile_commands.py to extract files used during compilation > for the used kernel configuration. > > CC: Peter Marko <peter.marko@siemens.com> > Signed-off-by: Daniel Turull <daniel.turull@ericsson.com> > --- > meta/classes-recipe/kernel.bbclass | 11 +++++++++++ > 1 file changed, 11 insertions(+) > > diff --git a/meta/classes-recipe/kernel.bbclass > b/meta/classes-recipe/kernel.bbclass > index 36ce659762..56060f2c91 100644 > --- a/meta/classes-recipe/kernel.bbclass > +++ b/meta/classes-recipe/kernel.bbclass > @@ -867,3 +867,14 @@ EXPORT_FUNCTIONS do_deploy > > # Add using Device Tree support > inherit kernel-devicetree > + > +KERNEL_FILES_DIR ?= "${LOG_DIR}/cve/kernel_files" > +KERNEL_SRC_FILES ?= "${KERNEL_FILES_DIR}/compile_commands.json" > + > +do_save_compiled_files() { > + bbdebug 1 "Saving compiled files in ${KERNEL_SRC_FILES}" > + mkdir -p ${KERNEL_FILES_DIR} > + ${S}/scripts/clang-tools/gen_compile_commands.py -o > ${KERNEL_SRC_FILES} -d ${B} > +} > + > +addtask do_save_compiled_files after do_compile do_compile_kernelmodules > This is something that I wouldn't want running in my development environment. I can't tell fully. Is this opt in ? Is there a way to opt-out ? Bruce > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#215604): > https://lists.openembedded.org/g/openembedded-core/message/215604 > Mute This Topic: https://lists.openembedded.org/mt/112499916/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > > -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II [-- Attachment #2: Type: text/html, Size: 3892 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [OE-core] [PATCH v2 3/6] kernel: add support to extract compiled files 2025-04-28 14:24 ` [OE-core] " Bruce Ashfield @ 2025-04-28 14:28 ` Daniel Turull 0 siblings, 0 replies; 14+ messages in thread From: Daniel Turull @ 2025-04-28 14:28 UTC (permalink / raw) To: Bruce Ashfield Cc: openembedded-core@lists.openembedded.org, rybczynska@gmail.com, steve@sakoman.com, Peter.Marko@siemens.com, ross.burton@arm.com, skandigraun@gmail.com [-- Attachment #1: Type: text/plain, Size: 2810 bytes --] Hi, I have put it as default, but you have a good point. I’ll add the task conditionally if CVE_CHECK_KERNEL_CONFIG is set to 1 in the next patch version series. Best regards, Daniel From: Bruce Ashfield <bruce.ashfield@gmail.com> Sent: Monday, 28 April 2025 16:25 To: Daniel Turull <daniel.turull@ericsson.com> Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; steve@sakoman.com; Peter.Marko@siemens.com; ross.burton@arm.com; skandigraun@gmail.com Subject: Re: [OE-core] [PATCH v2 3/6] kernel: add support to extract compiled files On Mon, Apr 28, 2025 at 9:42 AM Daniel Turull via lists.openembedded.org<http://lists.openembedded.org/> <daniel.turull=ericsson.com@lists.openembedded.org<mailto:ericsson.com@lists.openembedded.org>> wrote: From: Daniel Turull <daniel.turull@ericsson.com<mailto:daniel.turull@ericsson.com>> Use gen_compile_commands.py to extract files used during compilation for the used kernel configuration. CC: Peter Marko <peter.marko@siemens.com<mailto:peter.marko@siemens.com>> Signed-off-by: Daniel Turull <daniel.turull@ericsson.com<mailto:daniel.turull@ericsson.com>> --- meta/classes-recipe/kernel.bbclass | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass index 36ce659762..56060f2c91 100644 --- a/meta/classes-recipe/kernel.bbclass +++ b/meta/classes-recipe/kernel.bbclass @@ -867,3 +867,14 @@ EXPORT_FUNCTIONS do_deploy # Add using Device Tree support inherit kernel-devicetree + +KERNEL_FILES_DIR ?= "${LOG_DIR}/cve/kernel_files" +KERNEL_SRC_FILES ?= "${KERNEL_FILES_DIR}/compile_commands.json" + +do_save_compiled_files() { + bbdebug 1 "Saving compiled files in ${KERNEL_SRC_FILES}" + mkdir -p ${KERNEL_FILES_DIR} + ${S}/scripts/clang-tools/gen_compile_commands.py -o ${KERNEL_SRC_FILES} -d ${B} +} + +addtask do_save_compiled_files after do_compile do_compile_kernelmodules This is something that I wouldn't want running in my development environment. I can't tell fully. Is this opt in ? Is there a way to opt-out ? Bruce -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#215604): https://lists.openembedded.org/g/openembedded-core/message/215604 Mute This Topic: https://lists.openembedded.org/mt/112499916/1050810 Group Owner: openembedded-core+owner@lists.openembedded.org<mailto:openembedded-core%2Bowner@lists.openembedded.org> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com<mailto:bruce.ashfield@gmail.com>] -=-=-=-=-=-=-=-=-=-=-=- -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II [-- Attachment #2: Type: text/html, Size: 7570 bytes --] ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 4/6] cve-check: move message outsite check_cves and sort 2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull ` (2 preceding siblings ...) 2025-04-28 13:42 ` [PATCH v2 3/6] kernel: add support to extract compiled files daniel.turull @ 2025-04-28 13:42 ` daniel.turull 2025-04-28 13:42 ` [PATCH v2 5/6] cve-check, vex, spdx: use metadata from linux-vulns to enhance CVE reporting daniel.turull 2025-04-28 13:42 ` [PATCH v2 6/6] spdx: add option to include only compiled kernel files daniel.turull 5 siblings, 0 replies; 14+ messages in thread From: daniel.turull @ 2025-04-28 13:42 UTC (permalink / raw) To: openembedded-core Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun, Daniel Turull, Peter Marko From: Daniel Turull <daniel.turull@ericsson.com> When adding corrections from multiple sources of CVEs, the message showing standing CVES should be at the end. Also sort them. CC: Peter Marko <peter.marko@siemens.com> Signed-off-by: Daniel Turull <daniel.turull@ericsson.com> --- meta/classes/cve-check.bbclass | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass index 86ddfaae5f..12159a98d2 100644 --- a/meta/classes/cve-check.bbclass +++ b/meta/classes/cve-check.bbclass @@ -173,6 +173,12 @@ python do_cve_check () { if len(cve_data) or (d.getVar("CVE_CHECK_COVERAGE") == "1" and status): get_cve_info(d, cve_data) cve_write_data(d, cve_data, status) + + if d.getVar("CVE_CHECK_SHOW_WARNINGS") == "1": + unpatched_cves = [cve for cve in cve_data if cve_data[cve]["abbrev-status"] == "Unpatched"] + if unpatched_cves: + bb.warn("Found unpatched CVE (%s)" % " ".join(sorted(unpatched_cves))) + else: bb.note("No CVE database found, skipping CVE check") @@ -422,10 +428,6 @@ def check_cves(d, cve_data): if not cves_in_recipe: bb.note("No CVE records for products in recipe %s" % (pn)) - if d.getVar("CVE_CHECK_SHOW_WARNINGS") == "1": - unpatched_cves = [cve for cve in cve_data if cve_data[cve]["abbrev-status"] == "Unpatched"] - if unpatched_cves: - bb.warn("Found unpatched CVE (%s)" % " ".join(unpatched_cves)) return (cve_data, cves_status) ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 5/6] cve-check, vex, spdx: use metadata from linux-vulns to enhance CVE reporting 2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull ` (3 preceding siblings ...) 2025-04-28 13:42 ` [PATCH v2 4/6] cve-check: move message outsite check_cves and sort daniel.turull @ 2025-04-28 13:42 ` daniel.turull 2025-04-28 13:42 ` [PATCH v2 6/6] spdx: add option to include only compiled kernel files daniel.turull 5 siblings, 0 replies; 14+ messages in thread From: daniel.turull @ 2025-04-28 13:42 UTC (permalink / raw) To: openembedded-core Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun, Daniel Turull, Peter Marko From: Daniel Turull <daniel.turull@ericsson.com> Introducing two new options to check kernel CVEs using linux-vulns. - CVE_CHECK_KERNEL = "1" To add linux-vulns metadata on cve-report for cve_check, vex and spdx. This will check for kernel CVEs, and use the metadata to resolve them. Enabled by default - CVE_CHECK_KERNEL_CONFIG = "1" CVE Check using kernel compiled files, disabled by default since it requires a compiled kernel and it will increase cve-check times. Metadata in the CVE information includes the affected files, and using the compiled files by the kernel we can ignore some of the cves. The above variables are defined in cve_check.bbclass, vex.bbclass and spdx.bbclass in case not all classes are used at the same time. The ones in cve_check has priority. Example of output with CVE_CHECK_KERNEL_CONFIG when using cve-check: { "id": "CVE-2024-26710", "status": "Ignored", "link": "https://nvd.nist.gov/vuln/detail/CVE-2024-26710", "summary": "In the Linux kernel, the following vulnerability [...]", "scorev2": "0.0", "scorev3": "5.5", "scorev4": "0.0", "modified": "2025-03-17T15:36:11.620", "vector": "LOCAL", "vectorString": "CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H", "detail": "not-applicable-config", "description": "Source code not compiled by config. ['arch/powerpc/include/asm/thread_info.h']" }, And same with vex: { "id": "CVE-2024-26710", "status": "Ignored", "link": "https://nvd.nist.gov/vuln/detail/CVE-2024-26710", "detail": "not-applicable-config", "description": "Source code not compiled by config. ['arch/powerpc/include/asm/thread_info.h']" }, And new information for the Unpatched showing where the fix (if any) is available: Tested with 6.12.22 kernel { "id": "CVE-2025-39728", "status": "Unpatched", "link": "https://nvd.nist.gov/vuln/detail/CVE-2025-39728", "summary": "In the Linux kernel, the following vulnerability has been [...], "scorev2": "0.0", "scorev3": "0.0", "scorev4": "0.0", "modified": "2025-04-21T14:23:45.950", "vector": "UNKNOWN", "vectorString": "UNKNOWN", "detail": "version-in-range", "description": "Needs backporting (fixed from 6.12.23)" }, Tested with cve-check, create-spdx2.2, create-spdx3.0, vex CC: Peter Marko <peter.marko@siemens.com> Signed-off-by: Daniel Turull <daniel.turull@ericsson.com> --- meta/classes/cve-check.bbclass | 19 ++- meta/classes/spdx-common.bbclass | 7 ++ meta/classes/vex.bbclass | 10 ++ meta/lib/oe/cve_check.py | 210 ++++++++++++++++++++++++++++++- 4 files changed, 244 insertions(+), 2 deletions(-) diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass index 12159a98d2..02266f171f 100644 --- a/meta/classes/cve-check.bbclass +++ b/meta/classes/cve-check.bbclass @@ -34,6 +34,13 @@ CVE_VERSION ??= "${PV}" # Possible database sources: NVD1, NVD2, FKIE NVD_DB_VERSION ?= "FKIE" +# CVE Check using kernel CNA and compiled files +CVE_CHECK_KERNEL ?= "1" +# CVE Check using kernel compiled files +CVE_CHECK_KERNEL_CONFIG ?= "0" +# Location of the linux-vulns data +CVE_CHECK_KERNEL_DB_DIR ?= "${DL_DIR}/CVE_CHECK/vulns" + # Use different file names for each database source, as they synchronize at different moments, so may be slightly different CVE_CHECK_DB_FILENAME ?= "${@'nvdcve_2-2.db' if d.getVar('NVD_DB_VERSION') == 'NVD2' else 'nvdcve_1-3.db' if d.getVar('NVD_DB_VERSION') == 'NVD1' else 'nvdfkie_1-1.db'}" CVE_CHECK_DB_FETCHER ?= "${@'cve-update-nvd2-native' if d.getVar('NVD_DB_VERSION') == 'NVD2' else 'cve-update-db-native'}" @@ -111,6 +118,9 @@ python () { if nvd_database_type not in ("NVD1", "NVD2", "FKIE"): bb.erroronce("Malformed NVD_DB_VERSION, must be one of: NVD1, NVD2, FKIE. Defaulting to NVD2") d.setVar("NVD_DB_VERSION", "NVD2") + + from oe.cve_check import extend_cve_kernel_config + extend_cve_kernel_config(d, "do_cve_check") } def generate_json_report(d, out_path, link_path): @@ -161,7 +171,7 @@ python do_cve_check () { """ Check recipe for patched and unpatched CVEs """ - from oe.cve_check import get_patched_cves + from oe.cve_check import get_patched_cves, get_kernel_cves with bb.utils.fileslocked([d.getVar("CVE_CHECK_DB_FILE_LOCK")], shared=True): if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")): @@ -170,6 +180,13 @@ python do_cve_check () { except FileNotFoundError: bb.fatal("Failure in searching patches") cve_data, status = check_cves(d, patched_cves) + + cve_check_kernel = d.getVar("CVE_CHECK_KERNEL") + if "linux_kernel" in d.getVar("CVE_PRODUCT") and cve_check_kernel == "1": + kernel_unpatched_cves, _kernel_patched_cves = get_kernel_cves(d) + # Patched CVEs are already returned in the get_patched_cves(). Adding the unpatched + cve_data.update(kernel_unpatched_cves) + if len(cve_data) or (d.getVar("CVE_CHECK_COVERAGE") == "1" and status): get_cve_info(d, cve_data) cve_write_data(d, cve_data, status) diff --git a/meta/classes/spdx-common.bbclass b/meta/classes/spdx-common.bbclass index 713a7fc651..d2b9c6993a 100644 --- a/meta/classes/spdx-common.bbclass +++ b/meta/classes/spdx-common.bbclass @@ -37,9 +37,16 @@ SPDX_CUSTOM_ANNOTATION_VARS ??= "" SPDX_MULTILIB_SSTATE_ARCHS ??= "${SSTATE_ARCHS}" +CVE_CHECK_KERNEL ??= "0" +CVE_CHECK_KERNEL_CONFIG ??= "0" +CVE_CHECK_KERNEL_DB_DIR ?= "${DL_DIR}/CVE_CHECK/vulns" + python () { from oe.cve_check import extend_cve_status extend_cve_status(d) + + from oe.cve_check import extend_cve_kernel_config + extend_cve_kernel_config(d, "do_create_spdx") } def create_spdx_source_deps(d): diff --git a/meta/classes/vex.bbclass b/meta/classes/vex.bbclass index 905d67b47d..56ebdb0580 100644 --- a/meta/classes/vex.bbclass +++ b/meta/classes/vex.bbclass @@ -26,6 +26,13 @@ CVE_PRODUCT ??= "${BPN}" CVE_VERSION ??= "${PV}" +# CVE Check using kernel CNA and compiled files +CVE_CHECK_KERNEL ?= "0" +# CVE Check using kernel compiled files +CVE_CHECK_KERNEL_CONFIG ?= "0" + +CVE_CHECK_KERNEL_DB_DIR ?= "${DL_DIR}/CVE_CHECK/vulns" + CVE_CHECK_SUMMARY_DIR ?= "${LOG_DIR}/cve" CVE_CHECK_SUMMARY_FILE_NAME_JSON = "cve-summary.json" @@ -78,6 +85,9 @@ python () { from oe.cve_check import extend_cve_status extend_cve_status(d) + + from oe.cve_check import extend_cve_kernel_config + extend_cve_kernel_config(d, "do_generate_vex") } def generate_json_report(d, out_path, link_path): diff --git a/meta/lib/oe/cve_check.py b/meta/lib/oe/cve_check.py index ae194f27cf..9f34da0e91 100644 --- a/meta/lib/oe/cve_check.py +++ b/meta/lib/oe/cve_check.py @@ -47,6 +47,7 @@ class Version(): self._version.pre_l, self._version.pre_v ) + self.version = version def __eq__(self, other): if not isinstance(other, Version): @@ -58,6 +59,9 @@ class Version(): return NotImplemented return self._key > other._key + def __str__(self) -> str: + return self.version + def _cmpkey(release, patch_l, pre_l, pre_v): # remove leading 0 _release = tuple( @@ -202,8 +206,14 @@ def get_patched_cves(d): "affected-product": decoded_status["product"], } - return patched_cves + # If we are parsing the kernel, check compiled files + cve_check_kernel = d.getVar("CVE_CHECK_KERNEL") + if "linux_kernel" in d.getVar("CVE_PRODUCT") and cve_check_kernel == "1": + bb.debug(1, "Checking kernel CVEs") + _kernel_unpatched_cves, kernel_patched_cves = get_kernel_cves(d) + patched_cves.update(kernel_patched_cves) + return patched_cves def get_cpe_ids(cve_product, version): """ @@ -376,3 +386,201 @@ def extend_cve_status(d): d.setVarFlag("CVE_STATUS", cve, d.getVarFlag(cve_status_group, "status")) else: bb.warn("CVE_STATUS_GROUPS contains undefined variable %s" % cve_status_group) + +def extend_cve_kernel_config(d, task): + pn = d.getVar('PN') + # For kernel CVEs, add required dependencies + if "linux_kernel" in d.getVar("CVE_PRODUCT"): + if d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1": + bb.debug(1, "Checking kernel CVEs using kernel config") + depends = f" {pn}:do_save_compiled_files " + d.appendVarFlag(task, "depends", depends) + d.setVar('CVE_CHECK_KERNEL','1') + if d.getVar("CVE_CHECK_KERNEL") == "1": + bb.debug(1, "Checking kernel CVEs using linux-vulns") + d.appendVarFlag(task, "depends", " linux-vulns:do_fetch ") + +def get_kernel_cves(d): + """ + Get CVEs for the kernel + """ + import glob + import json + patched_cves = {} + unpatched_cves = {} + datadir = f"{d.getVar('CVE_CHECK_KERNEL_DB_DIR')}/cve/published/" + version_str = d.getVar("LINUX_VERSION") + check_config = d.getVar("CVE_CHECK_KERNEL_CONFIG") + version = Version(version_str) + base_version = Version(".".join(version_str.split(".")[0:2])) + + # Check all CVES from kernel vulns + pattern = os.path.join(datadir, '**', f"CVE-*.json") + cve_files = glob.glob(pattern, recursive=True) + not_applicable_config = 0 + fixed_as_later_backport = 0 + for cve_file in cve_files: + cve_info = {} + with open(cve_file, "r") as f: + cve_info = json.load(f) + + if len(cve_info) == 0: + bb.error(f"Not valid data in {cve_file}. Aborting") + break + cve_id = cve_info["cveMetadata"]["cveID"] + + first_affected, fixed, backport_ver = get_kernel_fixed_versions(cve_info, base_version) + if not fixed: + if check_config == "1": + is_affected, affected_files = check_kernel_compiled_files(d, cve_info) + else: + is_affected = True + if not is_affected and len(affected_files) > 0: + bb.debug(1, f"{cve_id} - not applicable configuration since affected files not compiled: {affected_files}") + patched_cves[cve_id] = { + "abbrev-status": "Ignored", + "status": "not-applicable-config", + "justification": f"Source code not compiled by config. {affected_files}" + } + else: + bb.debug(1, f"{cve_id} not fixed usptream") + description = cve_info["containers"]["cna"]["descriptions"][0]["value"] + unpatched_cves[cve_id] = { + "abbrev-status": "Unpatched", + "status": "version-in-range", + "summary": description, + "justification": f"No fix available upstream" + } + elif first_affected and version < first_affected: + bb.debug(1, f'{cve_id} - "fixed-version: only affects {first_affected} onwards"') + patched_cves[cve_id] = { + "abbrev-status": "Patched", + "status": "fixed-version", + "justification": f"only affects {first_affected} onwards" + } + elif fixed <= version: + bb.debug(1, f'{cve_id} - "fixed-version: Fixed from version {fixed}"') + patched_cves[cve_id] = { + "abbrev-status": "Patched", + "status": "fixed-version", + "justification": f"fixed-version: Fixed from version {fixed}" + } + else: + if backport_ver: + if backport_ver <= version: + bb.debug(1, f'{cve_id} - "cpe-stable-backport: Backported in {backport_ver}"') + patched_cves[cve_id] = { + "abbrev-status": "Patched", + "status": "cpe-stable-backport", + "justification": f"Backported in {backport_ver}" + } + else: + bb.debug(1, f"{cve_id}: needs backporting (fixed from {backport_ver})") + description = cve_info["containers"]["cna"]["descriptions"][0]["value"] + unpatched_cves[cve_id] = { + "abbrev-status": "Unpatched", + "status": "version-in-range", + "summary": description, + "justification": f"Needs backporting (fixed from {backport_ver})" + } + fixed_as_later_backport += 1 + else: + # Check if file affected + if check_config == "1": + is_affected, affected_files = check_kernel_compiled_files(d, cve_info) + else: + is_affected = True + if not is_affected and len(affected_files) > 0: + bb.debug(1, f"{cve_id} - not applicable configuration since affected files not compiled: {affected_files}") + patched_cves[cve_id] = { + "abbrev-status": "Ignored", + "status": "not-applicable-config", + "justification": f"Source code not compiled by config. {affected_files}" + } + not_applicable_config +=1 + else: + bb.debug(1, f"{cve_id}: needs backporting (fixed from {fixed})") + description = cve_info["containers"]["cna"]["descriptions"][0]["value"] + unpatched_cves[cve_id] = { + "abbrev-status": "Unpatched", + "status": "version-in-range", + "summary": description, + "justification": f"Needs backporting (fixed from {fixed})" + } + if len(cve_files) > 0: + bb.debug(1, f"Total CVEs ignored due to not applicable config {not_applicable_config}") + bb.debug(1, f"Total vulnerable CVEs: {len(unpatched_cves)}") + bb.debug(1, f"Total CVEs already backported in {base_version}: {fixed_as_later_backport}") + return unpatched_cves, patched_cves + +def check_kernel_compiled_files(d, cve_info): + """ + Return if a CVE affected us depending on compiled files + """ + import json + files_affected = [] + kfiles = [] + is_affected = False + with open(d.getVar('KERNEL_SRC_FILES'), 'r') as file: + for item in json.load(file): + kfiles.append(item['file'].replace(f"{d.getVar('S')}/","")) + + for item in cve_info['containers']['cna']['affected']: + if item["defaultStatus"] == "affected": + if "programFiles" in item: + files = item['programFiles'] + files_affected.extend(files) + + if len(files_affected) > 0: + for f in files_affected: + if f in kfiles: + bb.debug(1, f"File match: {f}") + is_affected = True + return is_affected, files_affected + +def get_kernel_fixed_versions(cve_info, base_version): + ''' + Get fixed versions for a given CVE + ''' + first_affected = None + fixed = None + fixed_backport = None + next_version = Version(str(base_version) + ".5000") + for affected in cve_info["containers"]["cna"]["affected"]: + # In case the CVE info is not complete, it might not have default status and therefore + # we don't know the status of this CVE. + if not "defaultStatus" in affected: + return first_affected, fixed, fixed_backport + if affected["defaultStatus"] == "affected": + for version in affected["versions"]: + v = Version(version["version"]) + if v == 0: + # Skiping non-affected + continue + if version["status"] == "affected" and not first_affected: + first_affected = v + elif (version["status"] == "unaffected" and + version['versionType'] == "original_commit_for_fix"): + fixed = v + elif base_version < v and v < next_version: + fixed_backport = v + elif affected["defaultStatus"] == "unaffected": + # Only specific versions are affected. We care only about our base version + if "versions" not in affected: + continue + for version in affected["versions"]: + if "versionType" not in version: + continue + if version["versionType"] == "git": + continue + v = Version(version["version"]) + # in case it is not in our base version + less_than = Version(version["lessThan"]) + + if not first_affected: + first_affected = v + fixed = less_than + if base_version < v and v < next_version: + fixed_backport = less_than + + return first_affected, fixed, fixed_backport ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v2 6/6] spdx: add option to include only compiled kernel files 2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull ` (4 preceding siblings ...) 2025-04-28 13:42 ` [PATCH v2 5/6] cve-check, vex, spdx: use metadata from linux-vulns to enhance CVE reporting daniel.turull @ 2025-04-28 13:42 ` daniel.turull 2025-04-28 13:57 ` Joshua Watt 5 siblings, 1 reply; 14+ messages in thread From: daniel.turull @ 2025-04-28 13:42 UTC (permalink / raw) To: openembedded-core Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun, Daniel Turull, Joshua Watt, Peter Marko From: Daniel Turull <daniel.turull@ericsson.com> When CVE_CHECK_KERNEL_CONFIG is enabled, only include the source code (.c, .h) files that are used during compilation. This enables an external tool to use the SPDX information to disregard vulnerabilities that are not compiled. CC: Joshua Watt <JPEWhacker@gmail.com> CC: Peter Marko <peter.marko@siemens.com> Signed-off-by: Daniel Turull <daniel.turull@ericsson.com> --- meta/classes/create-spdx-2.2.bbclass | 8 +++++++ meta/lib/oe/spdx30_tasks.py | 8 +++++++ meta/lib/oe/spdx_common.py | 34 ++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+) diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass index 7e8f8b9ff5..6bf0c70bd4 100644 --- a/meta/classes/create-spdx-2.2.bbclass +++ b/meta/classes/create-spdx-2.2.bbclass @@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv spdx_files = [] file_counter = 1 + + check_kernel_compiled = bb.data.inherits_class("kernel", d) and d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" + if check_kernel_compiled: + kernel_sources = oe.spdx_common.get_kernel_compiled_files(d) for subdir, dirs, files in os.walk(topdir): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir): @@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv filename = str(filepath.relative_to(topdir)) if not filepath.is_symlink() and filepath.is_file(): + # When creating spdx for the kernel, we only include compiled files. + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): + break spdx_file = oe.spdx.SPDXFile() spdx_file.SPDXID = get_spdxid(file_counter) for t in get_types(filepath): diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index ba965821f8..14f26773c5 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -156,6 +156,10 @@ def add_package_files( bb.note(f"Skip {topdir}") return spdx_files + check_kernel_compiled = bb.data.inherits_class("kernel", d) and d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" + if check_kernel_compiled: + kernel_sources = oe.spdx_common.get_kernel_compiled_files(d) + for subdir, dirs, files in os.walk(topdir, onerror=walk_error): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir): @@ -167,6 +171,10 @@ def add_package_files( filepath = Path(subdir) / file if filepath.is_symlink() or not filepath.is_file(): continue + # When creating spdx for the kernel, we only include compiled files + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): + break filename = str(filepath.relative_to(topdir)) file_purposes = get_purposes(filepath) diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py index 4caefc7673..c87e3875c7 100644 --- a/meta/lib/oe/spdx_common.py +++ b/meta/lib/oe/spdx_common.py @@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name): uri = uri + "@" + fd.revision return uri + +def is_kernel_compiled(filename, kernel_sources, d): + """ + Check if the file, is a kernel compiled file + """ + import os + + _, extension = os.path.splitext(filename) + # Special case, that we need to ignore, since this is not a source file + if filename.rfind(".mod.c") > 0: + return True + # We filter .c files and header files + if extension not in [".c", ".h"]: + return True + # Check that the c file is in the list + if filename in kernel_sources: + return True + return False + +def get_kernel_compiled_files(d): + """ + Get results from the save_compiled files and include also header files + """ + import json + import os + kfiles = [] + with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f: + for item in json.load(f): + kfile = os.path.basename(item['file']) + # Return also the correspondig header file + hfile = ".h".join(kfile.rsplit(".c", 1)) + kfiles.append(kfile) + kfiles.append(hfile) + return kfiles ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH v2 6/6] spdx: add option to include only compiled kernel files 2025-04-28 13:42 ` [PATCH v2 6/6] spdx: add option to include only compiled kernel files daniel.turull @ 2025-04-28 13:57 ` Joshua Watt 2025-04-28 13:59 ` Marko, Peter 0 siblings, 1 reply; 14+ messages in thread From: Joshua Watt @ 2025-04-28 13:57 UTC (permalink / raw) To: daniel.turull Cc: openembedded-core, rybczynska, steve, Peter.Marko, ross.burton, skandigraun I see the appeal here; however I've been trying to avoid too much kernel specific stuff in SPDX. Is there a way to make this more generic so it could be used by any recipe? On Mon, Apr 28, 2025 at 7:42 AM <daniel.turull@ericsson.com> wrote: > > From: Daniel Turull <daniel.turull@ericsson.com> > > When CVE_CHECK_KERNEL_CONFIG is enabled, only include the > source code (.c, .h) files that are used during compilation. > > This enables an external tool to use the SPDX information to disregard > vulnerabilities that are not compiled. > > CC: Joshua Watt <JPEWhacker@gmail.com> > CC: Peter Marko <peter.marko@siemens.com> > Signed-off-by: Daniel Turull <daniel.turull@ericsson.com> > --- > meta/classes/create-spdx-2.2.bbclass | 8 +++++++ > meta/lib/oe/spdx30_tasks.py | 8 +++++++ > meta/lib/oe/spdx_common.py | 34 ++++++++++++++++++++++++++++ > 3 files changed, 50 insertions(+) > > diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass > index 7e8f8b9ff5..6bf0c70bd4 100644 > --- a/meta/classes/create-spdx-2.2.bbclass > +++ b/meta/classes/create-spdx-2.2.bbclass > @@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv > spdx_files = [] > > file_counter = 1 > + > + check_kernel_compiled = bb.data.inherits_class("kernel", d) and d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" > + if check_kernel_compiled: > + kernel_sources = oe.spdx_common.get_kernel_compiled_files(d) > for subdir, dirs, files in os.walk(topdir): > dirs[:] = [d for d in dirs if d not in ignore_dirs] > if subdir == str(topdir): > @@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv > filename = str(filepath.relative_to(topdir)) > > if not filepath.is_symlink() and filepath.is_file(): > + # When creating spdx for the kernel, we only include compiled files. > + if check_kernel_compiled: > + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): > + break > spdx_file = oe.spdx.SPDXFile() > spdx_file.SPDXID = get_spdxid(file_counter) > for t in get_types(filepath): > diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py > index ba965821f8..14f26773c5 100644 > --- a/meta/lib/oe/spdx30_tasks.py > +++ b/meta/lib/oe/spdx30_tasks.py > @@ -156,6 +156,10 @@ def add_package_files( > bb.note(f"Skip {topdir}") > return spdx_files > > + check_kernel_compiled = bb.data.inherits_class("kernel", d) and d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" > + if check_kernel_compiled: > + kernel_sources = oe.spdx_common.get_kernel_compiled_files(d) > + > for subdir, dirs, files in os.walk(topdir, onerror=walk_error): > dirs[:] = [d for d in dirs if d not in ignore_dirs] > if subdir == str(topdir): > @@ -167,6 +171,10 @@ def add_package_files( > filepath = Path(subdir) / file > if filepath.is_symlink() or not filepath.is_file(): > continue > + # When creating spdx for the kernel, we only include compiled files > + if check_kernel_compiled: > + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): > + break > > filename = str(filepath.relative_to(topdir)) > file_purposes = get_purposes(filepath) > diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py > index 4caefc7673..c87e3875c7 100644 > --- a/meta/lib/oe/spdx_common.py > +++ b/meta/lib/oe/spdx_common.py > @@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name): > uri = uri + "@" + fd.revision > > return uri > + > +def is_kernel_compiled(filename, kernel_sources, d): > + """ > + Check if the file, is a kernel compiled file > + """ > + import os > + > + _, extension = os.path.splitext(filename) > + # Special case, that we need to ignore, since this is not a source file > + if filename.rfind(".mod.c") > 0: > + return True > + # We filter .c files and header files > + if extension not in [".c", ".h"]: > + return True > + # Check that the c file is in the list > + if filename in kernel_sources: > + return True > + return False > + > +def get_kernel_compiled_files(d): > + """ > + Get results from the save_compiled files and include also header files > + """ > + import json > + import os > + kfiles = [] > + with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f: > + for item in json.load(f): > + kfile = os.path.basename(item['file']) > + # Return also the correspondig header file > + hfile = ".h".join(kfile.rsplit(".c", 1)) This feels a little bit wrong to me. I think a better option would be to include all headers, since we don't have a good way of knowing which are actually used. > + kfiles.append(kfile) > + kfiles.append(hfile) > + return kfiles ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files 2025-04-28 13:57 ` Joshua Watt @ 2025-04-28 13:59 ` Marko, Peter 2025-04-28 14:12 ` Daniel Turull 2025-04-28 14:25 ` Daniel Turull 0 siblings, 2 replies; 14+ messages in thread From: Marko, Peter @ 2025-04-28 13:59 UTC (permalink / raw) To: Joshua Watt, daniel.turull@ericsson.com Cc: openembedded-core@lists.openembedded.org, rybczynska@gmail.com, steve@sakoman.com, ross.burton@arm.com, skandigraun@gmail.com I think that only kernel provides file-based CVE information right now. Peter > -----Original Message----- > From: Joshua Watt <jpewhacker@gmail.com> > Sent: Monday, April 28, 2025 15:58 > To: daniel.turull@ericsson.com > Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; > steve@sakoman.com; Marko, Peter (FT D EU SK BFS1) > <Peter.Marko@siemens.com>; ross.burton@arm.com; skandigraun@gmail.com > Subject: Re: [PATCH v2 6/6] spdx: add option to include only compiled kernel files > > I see the appeal here; however I've been trying to avoid too much > kernel specific stuff in SPDX. Is there a way to make this more > generic so it could be used by any recipe? > > On Mon, Apr 28, 2025 at 7:42 AM <daniel.turull@ericsson.com> wrote: > > > > From: Daniel Turull <daniel.turull@ericsson.com> > > > > When CVE_CHECK_KERNEL_CONFIG is enabled, only include the > > source code (.c, .h) files that are used during compilation. > > > > This enables an external tool to use the SPDX information to disregard > > vulnerabilities that are not compiled. > > > > CC: Joshua Watt <JPEWhacker@gmail.com> > > CC: Peter Marko <peter.marko@siemens.com> > > Signed-off-by: Daniel Turull <daniel.turull@ericsson.com> > > --- > > meta/classes/create-spdx-2.2.bbclass | 8 +++++++ > > meta/lib/oe/spdx30_tasks.py | 8 +++++++ > > meta/lib/oe/spdx_common.py | 34 ++++++++++++++++++++++++++++ > > 3 files changed, 50 insertions(+) > > > > diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx- > 2.2.bbclass > > index 7e8f8b9ff5..6bf0c70bd4 100644 > > --- a/meta/classes/create-spdx-2.2.bbclass > > +++ b/meta/classes/create-spdx-2.2.bbclass > > @@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, > get_spdxid, get_types, *, archiv > > spdx_files = [] > > > > file_counter = 1 > > + > > + check_kernel_compiled = bb.data.inherits_class("kernel", d) and > d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" > > + if check_kernel_compiled: > > + kernel_sources = oe.spdx_common.get_kernel_compiled_files(d) > > for subdir, dirs, files in os.walk(topdir): > > dirs[:] = [d for d in dirs if d not in ignore_dirs] > > if subdir == str(topdir): > > @@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, > get_spdxid, get_types, *, archiv > > filename = str(filepath.relative_to(topdir)) > > > > if not filepath.is_symlink() and filepath.is_file(): > > + # When creating spdx for the kernel, we only include compiled files. > > + if check_kernel_compiled: > > + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, > d): > > + break > > spdx_file = oe.spdx.SPDXFile() > > spdx_file.SPDXID = get_spdxid(file_counter) > > for t in get_types(filepath): > > diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py > > index ba965821f8..14f26773c5 100644 > > --- a/meta/lib/oe/spdx30_tasks.py > > +++ b/meta/lib/oe/spdx30_tasks.py > > @@ -156,6 +156,10 @@ def add_package_files( > > bb.note(f"Skip {topdir}") > > return spdx_files > > > > + check_kernel_compiled = bb.data.inherits_class("kernel", d) and > d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" > > + if check_kernel_compiled: > > + kernel_sources = oe.spdx_common.get_kernel_compiled_files(d) > > + > > for subdir, dirs, files in os.walk(topdir, onerror=walk_error): > > dirs[:] = [d for d in dirs if d not in ignore_dirs] > > if subdir == str(topdir): > > @@ -167,6 +171,10 @@ def add_package_files( > > filepath = Path(subdir) / file > > if filepath.is_symlink() or not filepath.is_file(): > > continue > > + # When creating spdx for the kernel, we only include compiled files > > + if check_kernel_compiled: > > + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): > > + break > > > > filename = str(filepath.relative_to(topdir)) > > file_purposes = get_purposes(filepath) > > diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py > > index 4caefc7673..c87e3875c7 100644 > > --- a/meta/lib/oe/spdx_common.py > > +++ b/meta/lib/oe/spdx_common.py > > @@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name): > > uri = uri + "@" + fd.revision > > > > return uri > > + > > +def is_kernel_compiled(filename, kernel_sources, d): > > + """ > > + Check if the file, is a kernel compiled file > > + """ > > + import os > > + > > + _, extension = os.path.splitext(filename) > > + # Special case, that we need to ignore, since this is not a source file > > + if filename.rfind(".mod.c") > 0: > > + return True > > + # We filter .c files and header files > > + if extension not in [".c", ".h"]: > > + return True > > + # Check that the c file is in the list > > + if filename in kernel_sources: > > + return True > > + return False > > + > > +def get_kernel_compiled_files(d): > > + """ > > + Get results from the save_compiled files and include also header files > > + """ > > + import json > > + import os > > + kfiles = [] > > + with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f: > > + for item in json.load(f): > > + kfile = os.path.basename(item['file']) > > + # Return also the correspondig header file > > + hfile = ".h".join(kfile.rsplit(".c", 1)) > > This feels a little bit wrong to me. I think a better option would be > to include all headers, since we don't have a good way of knowing > which are actually used. > > > + kfiles.append(kfile) > > + kfiles.append(hfile) > > + return kfiles ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files 2025-04-28 13:59 ` Marko, Peter @ 2025-04-28 14:12 ` Daniel Turull 2025-04-28 14:44 ` Joshua Watt 2025-04-28 14:25 ` Daniel Turull 1 sibling, 1 reply; 14+ messages in thread From: Daniel Turull @ 2025-04-28 14:12 UTC (permalink / raw) To: Marko, Peter, Joshua Watt Cc: openembedded-core@lists.openembedded.org, rybczynska@gmail.com, steve@sakoman.com, ross.burton@arm.com, skandigraun@gmail.com Hi, Yes, from what I saw and Peter says only the kernel has file information that can be machine readable, and we can also extract the files that we compile. I could try to normalize how the files are input into the spdx generation and then have a generic variable that any recipe can set to enable the inclusion of a subset of files. In this case, if/when more recipes provides this, spdx doesn't need to change. Best regards Daniel -----Original Message----- From: Marko, Peter <Peter.Marko@siemens.com> Sent: Monday, 28 April 2025 16:00 To: Joshua Watt <jpewhacker@gmail.com>; Daniel Turull <daniel.turull@ericsson.com> Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; steve@sakoman.com; ross.burton@arm.com; skandigraun@gmail.com Subject: RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files I think that only kernel provides file-based CVE information right now. Peter > -----Original Message----- > From: Joshua Watt <jpewhacker@gmail.com> > Sent: Monday, April 28, 2025 15:58 > To: daniel.turull@ericsson.com > Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; > steve@sakoman.com; Marko, Peter (FT D EU SK BFS1) > <Peter.Marko@siemens.com>; ross.burton@arm.com; skandigraun@gmail.com > Subject: Re: [PATCH v2 6/6] spdx: add option to include only compiled > kernel files > > I see the appeal here; however I've been trying to avoid too much > kernel specific stuff in SPDX. Is there a way to make this more > generic so it could be used by any recipe? > > On Mon, Apr 28, 2025 at 7:42 AM <daniel.turull@ericsson.com> wrote: > > > > From: Daniel Turull <daniel.turull@ericsson.com> > > > > When CVE_CHECK_KERNEL_CONFIG is enabled, only include the source > > code (.c, .h) files that are used during compilation. > > > > This enables an external tool to use the SPDX information to > > disregard vulnerabilities that are not compiled. > > > > CC: Joshua Watt <JPEWhacker@gmail.com> > > CC: Peter Marko <peter.marko@siemens.com> > > Signed-off-by: Daniel Turull <daniel.turull@ericsson.com> > > --- > > meta/classes/create-spdx-2.2.bbclass | 8 +++++++ > > meta/lib/oe/spdx30_tasks.py | 8 +++++++ > > meta/lib/oe/spdx_common.py | 34 ++++++++++++++++++++++++++++ > > 3 files changed, 50 insertions(+) > > > > diff --git a/meta/classes/create-spdx-2.2.bbclass > > b/meta/classes/create-spdx- > 2.2.bbclass > > index 7e8f8b9ff5..6bf0c70bd4 100644 > > --- a/meta/classes/create-spdx-2.2.bbclass > > +++ b/meta/classes/create-spdx-2.2.bbclass > > @@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, > get_spdxid, get_types, *, archiv > > spdx_files = [] > > > > file_counter = 1 > > + > > + check_kernel_compiled = bb.data.inherits_class("kernel", d) and > d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" > > + if check_kernel_compiled: > > + kernel_sources = > > + oe.spdx_common.get_kernel_compiled_files(d) > > for subdir, dirs, files in os.walk(topdir): > > dirs[:] = [d for d in dirs if d not in ignore_dirs] > > if subdir == str(topdir): > > @@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, > get_spdxid, get_types, *, archiv > > filename = str(filepath.relative_to(topdir)) > > > > if not filepath.is_symlink() and filepath.is_file(): > > + # When creating spdx for the kernel, we only include compiled files. > > + if check_kernel_compiled: > > + if not oe.spdx_common.is_kernel_compiled(file, > > + kernel_sources, > d): > > + break > > spdx_file = oe.spdx.SPDXFile() > > spdx_file.SPDXID = get_spdxid(file_counter) > > for t in get_types(filepath): > > diff --git a/meta/lib/oe/spdx30_tasks.py > > b/meta/lib/oe/spdx30_tasks.py index ba965821f8..14f26773c5 100644 > > --- a/meta/lib/oe/spdx30_tasks.py > > +++ b/meta/lib/oe/spdx30_tasks.py > > @@ -156,6 +156,10 @@ def add_package_files( > > bb.note(f"Skip {topdir}") > > return spdx_files > > > > + check_kernel_compiled = bb.data.inherits_class("kernel", d) and > d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" > > + if check_kernel_compiled: > > + kernel_sources = > > + oe.spdx_common.get_kernel_compiled_files(d) > > + > > for subdir, dirs, files in os.walk(topdir, onerror=walk_error): > > dirs[:] = [d for d in dirs if d not in ignore_dirs] > > if subdir == str(topdir): > > @@ -167,6 +171,10 @@ def add_package_files( > > filepath = Path(subdir) / file > > if filepath.is_symlink() or not filepath.is_file(): > > continue > > + # When creating spdx for the kernel, we only include compiled files > > + if check_kernel_compiled: > > + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): > > + break > > > > filename = str(filepath.relative_to(topdir)) > > file_purposes = get_purposes(filepath) diff --git > > a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py index > > 4caefc7673..c87e3875c7 100644 > > --- a/meta/lib/oe/spdx_common.py > > +++ b/meta/lib/oe/spdx_common.py > > @@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name): > > uri = uri + "@" + fd.revision > > > > return uri > > + > > +def is_kernel_compiled(filename, kernel_sources, d): > > + """ > > + Check if the file, is a kernel compiled file > > + """ > > + import os > > + > > + _, extension = os.path.splitext(filename) > > + # Special case, that we need to ignore, since this is not a source file > > + if filename.rfind(".mod.c") > 0: > > + return True > > + # We filter .c files and header files > > + if extension not in [".c", ".h"]: > > + return True > > + # Check that the c file is in the list > > + if filename in kernel_sources: > > + return True > > + return False > > + > > +def get_kernel_compiled_files(d): > > + """ > > + Get results from the save_compiled files and include also header files > > + """ > > + import json > > + import os > > + kfiles = [] > > + with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f: > > + for item in json.load(f): > > + kfile = os.path.basename(item['file']) > > + # Return also the correspondig header file > > + hfile = ".h".join(kfile.rsplit(".c", 1)) > > This feels a little bit wrong to me. I think a better option would be > to include all headers, since we don't have a good way of knowing > which are actually used. > > > + kfiles.append(kfile) > > + kfiles.append(hfile) > > + return kfiles ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v2 6/6] spdx: add option to include only compiled kernel files 2025-04-28 14:12 ` Daniel Turull @ 2025-04-28 14:44 ` Joshua Watt 0 siblings, 0 replies; 14+ messages in thread From: Joshua Watt @ 2025-04-28 14:44 UTC (permalink / raw) To: Daniel Turull Cc: Marko, Peter, openembedded-core@lists.openembedded.org, rybczynska@gmail.com, steve@sakoman.com, ross.burton@arm.com, skandigraun@gmail.com On Mon, Apr 28, 2025 at 8:12 AM Daniel Turull <daniel.turull@ericsson.com> wrote: > > Hi, > > Yes, from what I saw and Peter says only the kernel has file information that can be machine readable, > and we can also extract the files that we compile. > > I could try to normalize how the files are input into the spdx generation and then have a generic variable that any recipe can set > to enable the inclusion of a subset of files. In this case, if/when more recipes provides this, > spdx doesn't need to change. I was thinking that maybe the way to do this is to have this somehow call a function when enabled, and then have the recipe (or even better the build system bbclass) implement that function; that way each recipe or build system can implement the correct method for collecting the data. I'm not quite sure if that's the best way to do it though > > Best regards > Daniel > > -----Original Message----- > From: Marko, Peter <Peter.Marko@siemens.com> > Sent: Monday, 28 April 2025 16:00 > To: Joshua Watt <jpewhacker@gmail.com>; Daniel Turull <daniel.turull@ericsson.com> > Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; steve@sakoman.com; ross.burton@arm.com; skandigraun@gmail.com > Subject: RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files > > I think that only kernel provides file-based CVE information right now. > > Peter > > > -----Original Message----- > > From: Joshua Watt <jpewhacker@gmail.com> > > Sent: Monday, April 28, 2025 15:58 > > To: daniel.turull@ericsson.com > > Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; > > steve@sakoman.com; Marko, Peter (FT D EU SK BFS1) > > <Peter.Marko@siemens.com>; ross.burton@arm.com; skandigraun@gmail.com > > Subject: Re: [PATCH v2 6/6] spdx: add option to include only compiled > > kernel files > > > > I see the appeal here; however I've been trying to avoid too much > > kernel specific stuff in SPDX. Is there a way to make this more > > generic so it could be used by any recipe? > > > > On Mon, Apr 28, 2025 at 7:42 AM <daniel.turull@ericsson.com> wrote: > > > > > > From: Daniel Turull <daniel.turull@ericsson.com> > > > > > > When CVE_CHECK_KERNEL_CONFIG is enabled, only include the source > > > code (.c, .h) files that are used during compilation. > > > > > > This enables an external tool to use the SPDX information to > > > disregard vulnerabilities that are not compiled. > > > > > > CC: Joshua Watt <JPEWhacker@gmail.com> > > > CC: Peter Marko <peter.marko@siemens.com> > > > Signed-off-by: Daniel Turull <daniel.turull@ericsson.com> > > > --- > > > meta/classes/create-spdx-2.2.bbclass | 8 +++++++ > > > meta/lib/oe/spdx30_tasks.py | 8 +++++++ > > > meta/lib/oe/spdx_common.py | 34 ++++++++++++++++++++++++++++ > > > 3 files changed, 50 insertions(+) > > > > > > diff --git a/meta/classes/create-spdx-2.2.bbclass > > > b/meta/classes/create-spdx- > > 2.2.bbclass > > > index 7e8f8b9ff5..6bf0c70bd4 100644 > > > --- a/meta/classes/create-spdx-2.2.bbclass > > > +++ b/meta/classes/create-spdx-2.2.bbclass > > > @@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, > > get_spdxid, get_types, *, archiv > > > spdx_files = [] > > > > > > file_counter = 1 > > > + > > > + check_kernel_compiled = bb.data.inherits_class("kernel", d) and > > d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" > > > + if check_kernel_compiled: > > > + kernel_sources = > > > + oe.spdx_common.get_kernel_compiled_files(d) > > > for subdir, dirs, files in os.walk(topdir): > > > dirs[:] = [d for d in dirs if d not in ignore_dirs] > > > if subdir == str(topdir): > > > @@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, > > get_spdxid, get_types, *, archiv > > > filename = str(filepath.relative_to(topdir)) > > > > > > if not filepath.is_symlink() and filepath.is_file(): > > > + # When creating spdx for the kernel, we only include compiled files. > > > + if check_kernel_compiled: > > > + if not oe.spdx_common.is_kernel_compiled(file, > > > + kernel_sources, > > d): > > > + break > > > spdx_file = oe.spdx.SPDXFile() > > > spdx_file.SPDXID = get_spdxid(file_counter) > > > for t in get_types(filepath): > > > diff --git a/meta/lib/oe/spdx30_tasks.py > > > b/meta/lib/oe/spdx30_tasks.py index ba965821f8..14f26773c5 100644 > > > --- a/meta/lib/oe/spdx30_tasks.py > > > +++ b/meta/lib/oe/spdx30_tasks.py > > > @@ -156,6 +156,10 @@ def add_package_files( > > > bb.note(f"Skip {topdir}") > > > return spdx_files > > > > > > + check_kernel_compiled = bb.data.inherits_class("kernel", d) and > > d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" > > > + if check_kernel_compiled: > > > + kernel_sources = > > > + oe.spdx_common.get_kernel_compiled_files(d) > > > + > > > for subdir, dirs, files in os.walk(topdir, onerror=walk_error): > > > dirs[:] = [d for d in dirs if d not in ignore_dirs] > > > if subdir == str(topdir): > > > @@ -167,6 +171,10 @@ def add_package_files( > > > filepath = Path(subdir) / file > > > if filepath.is_symlink() or not filepath.is_file(): > > > continue > > > + # When creating spdx for the kernel, we only include compiled files > > > + if check_kernel_compiled: > > > + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): > > > + break > > > > > > filename = str(filepath.relative_to(topdir)) > > > file_purposes = get_purposes(filepath) diff --git > > > a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py index > > > 4caefc7673..c87e3875c7 100644 > > > --- a/meta/lib/oe/spdx_common.py > > > +++ b/meta/lib/oe/spdx_common.py > > > @@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name): > > > uri = uri + "@" + fd.revision > > > > > > return uri > > > + > > > +def is_kernel_compiled(filename, kernel_sources, d): > > > + """ > > > + Check if the file, is a kernel compiled file > > > + """ > > > + import os > > > + > > > + _, extension = os.path.splitext(filename) > > > + # Special case, that we need to ignore, since this is not a source file > > > + if filename.rfind(".mod.c") > 0: > > > + return True > > > + # We filter .c files and header files > > > + if extension not in [".c", ".h"]: > > > + return True > > > + # Check that the c file is in the list > > > + if filename in kernel_sources: > > > + return True > > > + return False > > > + > > > +def get_kernel_compiled_files(d): > > > + """ > > > + Get results from the save_compiled files and include also header files > > > + """ > > > + import json > > > + import os > > > + kfiles = [] > > > + with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f: > > > + for item in json.load(f): > > > + kfile = os.path.basename(item['file']) > > > + # Return also the correspondig header file > > > + hfile = ".h".join(kfile.rsplit(".c", 1)) > > > > This feels a little bit wrong to me. I think a better option would be > > to include all headers, since we don't have a good way of knowing > > which are actually used. > > > > > + kfiles.append(kfile) > > > + kfiles.append(hfile) > > > + return kfiles ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files 2025-04-28 13:59 ` Marko, Peter 2025-04-28 14:12 ` Daniel Turull @ 2025-04-28 14:25 ` Daniel Turull 1 sibling, 0 replies; 14+ messages in thread From: Daniel Turull @ 2025-04-28 14:25 UTC (permalink / raw) To: Marko, Peter, Joshua Watt Cc: openembedded-core@lists.openembedded.org, rybczynska@gmail.com, steve@sakoman.com, ross.burton@arm.com, skandigraun@gmail.com Hi, I missed to answer your comment. Daniel -----Original Message----- From: Marko, Peter <Peter.Marko@siemens.com> Sent: Monday, 28 April 2025 16:00 To: Joshua Watt <jpewhacker@gmail.com>; Daniel Turull <daniel.turull@ericsson.com> Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; steve@sakoman.com; ross.burton@arm.com; skandigraun@gmail.com Subject: RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files I think that only kernel provides file-based CVE information right now. Peter > -----Original Message----- > From: Joshua Watt <jpewhacker@gmail.com> > Sent: Monday, April 28, 2025 15:58 > To: daniel.turull@ericsson.com > Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; > steve@sakoman.com; Marko, Peter (FT D EU SK BFS1) > <Peter.Marko@siemens.com>; ross.burton@arm.com; skandigraun@gmail.com > Subject: Re: [PATCH v2 6/6] spdx: add option to include only compiled > kernel files > > I see the appeal here; however I've been trying to avoid too much > kernel specific stuff in SPDX. Is there a way to make this more > generic so it could be used by any recipe? > > On Mon, Apr 28, 2025 at 7:42 AM <daniel.turull@ericsson.com> wrote: > > > > From: Daniel Turull <daniel.turull@ericsson.com> > > > > When CVE_CHECK_KERNEL_CONFIG is enabled, only include the source > > code (.c, .h) files that are used during compilation. > > > > This enables an external tool to use the SPDX information to > > disregard vulnerabilities that are not compiled. > > > > CC: Joshua Watt <JPEWhacker@gmail.com> > > CC: Peter Marko <peter.marko@siemens.com> > > Signed-off-by: Daniel Turull <daniel.turull@ericsson.com> > > --- > > meta/classes/create-spdx-2.2.bbclass | 8 +++++++ > > meta/lib/oe/spdx30_tasks.py | 8 +++++++ > > meta/lib/oe/spdx_common.py | 34 ++++++++++++++++++++++++++++ > > 3 files changed, 50 insertions(+) > > > > diff --git a/meta/classes/create-spdx-2.2.bbclass > > b/meta/classes/create-spdx- > 2.2.bbclass > > index 7e8f8b9ff5..6bf0c70bd4 100644 > > --- a/meta/classes/create-spdx-2.2.bbclass > > +++ b/meta/classes/create-spdx-2.2.bbclass > > @@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, > get_spdxid, get_types, *, archiv > > spdx_files = [] > > > > file_counter = 1 > > + > > + check_kernel_compiled = bb.data.inherits_class("kernel", d) and > d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" > > + if check_kernel_compiled: > > + kernel_sources = > > + oe.spdx_common.get_kernel_compiled_files(d) > > for subdir, dirs, files in os.walk(topdir): > > dirs[:] = [d for d in dirs if d not in ignore_dirs] > > if subdir == str(topdir): > > @@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, > get_spdxid, get_types, *, archiv > > filename = str(filepath.relative_to(topdir)) > > > > if not filepath.is_symlink() and filepath.is_file(): > > + # When creating spdx for the kernel, we only include compiled files. > > + if check_kernel_compiled: > > + if not oe.spdx_common.is_kernel_compiled(file, > > + kernel_sources, > d): > > + break > > spdx_file = oe.spdx.SPDXFile() > > spdx_file.SPDXID = get_spdxid(file_counter) > > for t in get_types(filepath): > > diff --git a/meta/lib/oe/spdx30_tasks.py > > b/meta/lib/oe/spdx30_tasks.py index ba965821f8..14f26773c5 100644 > > --- a/meta/lib/oe/spdx30_tasks.py > > +++ b/meta/lib/oe/spdx30_tasks.py > > @@ -156,6 +156,10 @@ def add_package_files( > > bb.note(f"Skip {topdir}") > > return spdx_files > > > > + check_kernel_compiled = bb.data.inherits_class("kernel", d) and > d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" > > + if check_kernel_compiled: > > + kernel_sources = > > + oe.spdx_common.get_kernel_compiled_files(d) > > + > > for subdir, dirs, files in os.walk(topdir, onerror=walk_error): > > dirs[:] = [d for d in dirs if d not in ignore_dirs] > > if subdir == str(topdir): > > @@ -167,6 +171,10 @@ def add_package_files( > > filepath = Path(subdir) / file > > if filepath.is_symlink() or not filepath.is_file(): > > continue > > + # When creating spdx for the kernel, we only include compiled files > > + if check_kernel_compiled: > > + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): > > + break > > > > filename = str(filepath.relative_to(topdir)) > > file_purposes = get_purposes(filepath) diff --git > > a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py index > > 4caefc7673..c87e3875c7 100644 > > --- a/meta/lib/oe/spdx_common.py > > +++ b/meta/lib/oe/spdx_common.py > > @@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name): > > uri = uri + "@" + fd.revision > > > > return uri > > + > > +def is_kernel_compiled(filename, kernel_sources, d): > > + """ > > + Check if the file, is a kernel compiled file > > + """ > > + import os > > + > > + _, extension = os.path.splitext(filename) > > + # Special case, that we need to ignore, since this is not a source file > > + if filename.rfind(".mod.c") > 0: > > + return True > > + # We filter .c files and header files > > + if extension not in [".c", ".h"]: > > + return True > > + # Check that the c file is in the list > > + if filename in kernel_sources: > > + return True > > + return False > > + > > +def get_kernel_compiled_files(d): > > + """ > > + Get results from the save_compiled files and include also header files > > + """ > > + import json > > + import os > > + kfiles = [] > > + with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f: > > + for item in json.load(f): > > + kfile = os.path.basename(item['file']) > > + # Return also the correspondig header file > > + hfile = ".h".join(kfile.rsplit(".c", 1)) > > This feels a little bit wrong to me. I think a better option would be > to include all headers, since we don't have a good way of knowing > which are actually used. Yes, I had my doubt regarding this. It probably doesn't have an impact in the rest of the functionality for the CVEs I'll keep only the "c" files and try to make it more generic. > > > + kfiles.append(kfile) > > + kfiles.append(hfile) > > + return kfiles ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2025-04-28 14:44 UTC | newest] Thread overview: 14+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull 2025-04-28 13:42 ` [PATCH v2 1/6] linux-vulns: fetch kernel.org CNA info daniel.turull 2025-04-28 13:42 ` [PATCH v2 2/6] cve-check: fix debug message daniel.turull 2025-04-28 13:42 ` [PATCH v2 3/6] kernel: add support to extract compiled files daniel.turull 2025-04-28 14:24 ` [OE-core] " Bruce Ashfield 2025-04-28 14:28 ` Daniel Turull 2025-04-28 13:42 ` [PATCH v2 4/6] cve-check: move message outsite check_cves and sort daniel.turull 2025-04-28 13:42 ` [PATCH v2 5/6] cve-check, vex, spdx: use metadata from linux-vulns to enhance CVE reporting daniel.turull 2025-04-28 13:42 ` [PATCH v2 6/6] spdx: add option to include only compiled kernel files daniel.turull 2025-04-28 13:57 ` Joshua Watt 2025-04-28 13:59 ` Marko, Peter 2025-04-28 14:12 ` Daniel Turull 2025-04-28 14:44 ` Joshua Watt 2025-04-28 14:25 ` Daniel Turull
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox