Openembedded Core Discussions
 help / color / mirror / Atom feed
* [PATCH v2 0/6] Check compiled files to filter kernel CVEs
@ 2025-04-28 13:41 daniel.turull
  2025-04-28 13:42 ` [PATCH v2 1/6] linux-vulns: fetch kernel.org CNA info daniel.turull
                   ` (5 more replies)
  0 siblings, 6 replies; 14+ messages in thread
From: daniel.turull @ 2025-04-28 13:41 UTC (permalink / raw)
  To: openembedded-core
  Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun,
	Daniel Turull

From: Daniel Turull <daniel.turull@ericsson.com>

Since kernel.org became a CNA, more information is available in the published CVEs, including details about which files are affected by a given CVE.

This series adds functionality to fetch the database from kernel.org CNA information:
git.kernel.org/pub/scm/linux/security/vulns.git

I have rewritten the original patch and integrated it with cve_check.py, cve_check, vex and create-spdx.

To filter out CVEs that are not applicable, we extract the files used during the kernel compilation and
compare it with the metadata in the CVE.

The CVE_CHECK_KERNEL is enabled by default, but not the CVE_CHECK_KERNEL_CONFIG, since it
needs to build the kernel and it is dependent on kernel configuration.

To include the files into SPDX, SPDX_INCLUDE_SOURCES needs to be enabled.

This could use as a base to run the vulnerability check independently and run an external tool to filter the CVEs.

In addition, we also integrate the data with the output of cve_check, vex, and create-spdx.

As a side effect of using better data from directly the kernel CNA, the number of vulnerabilities reported
increases.

Numbers from 2025-04-28

Standing CVEs:
- Before: 18 CVEs, mostly old that needs to be checked and clean up in a new commit. There is only one from 2025
- Checking with kernel vulns database: 82 CVEs
- Checking with only compiled files: 46 CVEs

v1: initial proposal
v2:
  - rewrite kernel_vulns to fetch similarly as cve-update-db-native
  - add functionality into cve_check.py, for the classes that uses oe.get_patched_cves function
  - add linux-vulns into the cve-check results
  - add only compiled files in the spdx, so the check can be done outside the build
  - include compiled files into spdx when CVE_CHECK_KERNEL_CONFIG and SPDX_INCLUDE_SOURCES is enabled

Daniel Turull

Daniel Turull (6):
  linux-vulns: fetch kernel.org CNA info
  cve-check: fix debug message
  kernel: add support to extract compiled files
  cve-check: move message outsite check_cves and sort
  cve-check, vex, spdx: use metadata from linux-vulns to enhance CVE
    reporting
  spdx: add option to include only compiled kernel files

 meta/classes-recipe/kernel.bbclass        |  11 ++
 meta/classes/create-spdx-2.2.bbclass      |   8 +
 meta/classes/cve-check.bbclass            |  35 +++-
 meta/classes/spdx-common.bbclass          |   7 +
 meta/classes/vex.bbclass                  |  10 ++
 meta/conf/distro/include/maintainers.inc  |   1 +
 meta/lib/oe/cve_check.py                  | 210 +++++++++++++++++++++-
 meta/lib/oe/spdx30_tasks.py               |   8 +
 meta/lib/oe/spdx_common.py                |  34 ++++
 meta/recipes-core/meta/linux-vulns_git.bb |  76 ++++++++
 10 files changed, 391 insertions(+), 9 deletions(-)
 create mode 100644 meta/recipes-core/meta/linux-vulns_git.bb



^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH v2 1/6] linux-vulns: fetch kernel.org CNA info
  2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull
@ 2025-04-28 13:42 ` daniel.turull
  2025-04-28 13:42 ` [PATCH v2 2/6] cve-check: fix debug message daniel.turull
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 14+ messages in thread
From: daniel.turull @ 2025-04-28 13:42 UTC (permalink / raw)
  To: openembedded-core
  Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun,
	Daniel Turull, Peter Marko

From: Daniel Turull <daniel.turull@ericsson.com>

Add CVE data source for kernel.org.

It includes more information than the one provided by NVD.
Use similar mechanism and same variables as cve-check to define
when to update.

To use without internet access, change variable VULNS_URL to a local
copy or mirror.

CC: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
---
 meta/conf/distro/include/maintainers.inc  |  1 +
 meta/recipes-core/meta/linux-vulns_git.bb | 76 +++++++++++++++++++++++
 2 files changed, 77 insertions(+)
 create mode 100644 meta/recipes-core/meta/linux-vulns_git.bb

diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 8065287c17..ec427fe6a4 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -468,6 +468,7 @@ RECIPE_MAINTAINER:pn-lighttpd = "Unassigned <unassigned@yoctoproject.org>"
 RECIPE_MAINTAINER:pn-linux-dummy = "Unassigned <unassigned@yoctoproject.org>"
 RECIPE_MAINTAINER:pn-linux-firmware = "Otavio Salvador <otavio.salvador@ossystems.com.br>"
 RECIPE_MAINTAINER:pn-linux-libc-headers = "Bruce Ashfield <bruce.ashfield@gmail.com>"
+RECIPE_MAINTAINER:pn-linux-vulns = "Unassigned <unassigned@yoctoproject.org>"
 RECIPE_MAINTAINER:pn-linux-yocto = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-linux-yocto-dev = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-linux-yocto-rt = "Bruce Ashfield <bruce.ashfield@gmail.com>"
diff --git a/meta/recipes-core/meta/linux-vulns_git.bb b/meta/recipes-core/meta/linux-vulns_git.bb
new file mode 100644
index 0000000000..fc48558eb8
--- /dev/null
+++ b/meta/recipes-core/meta/linux-vulns_git.bb
@@ -0,0 +1,76 @@
+SUMMARY = "CVE information from kernel.org"
+DESCRIPTION = "Repo for tracking and maintaining the CVE identifiers reserved \
+and assigned to the Linux kernel project."
+HOMEPAGE = "https://git.kernel.org/pub/scm/linux/security/vulns.git/about/"
+LICENSE = "GPL-2.0-only & cve-tou"
+SECTION = "base"
+
+INHIBIT_DEFAULT_DEPS = "1"
+
+inherit native
+inherit nopackages
+
+VULNS_URL ?= "https://git.kernel.org/pub/scm/linux/security/vulns"
+CVE_CHECK_KERNEL_DB_DIR ??= "${DL_DIR}/CVE_CHECK/vulns"
+
+# Use same intervals as cve-update-db-native. By default: once a day (24*60*60).
+# Use 0 to force the update
+# Use a negative value to skip the update
+
+CVE_DB_UPDATE_INTERVAL ??= "86400"
+
+python do_fetch(){
+    import os
+    import bb.utils
+
+    bb.utils.export_proxies(d)
+    db_file = d.getVar("CVE_CHECK_KERNEL_DB_DIR")
+    repo_url = d.getVar("VULNS_URL")
+
+    try:
+        import time
+        update_interval = int(d.getVar("CVE_DB_UPDATE_INTERVAL"))
+
+        if update_interval < 0:
+            bb.note("Kernel CVE database update skipped")
+            return
+        if time.time() - os.path.getmtime(db_file) < update_interval:
+            bb.debug(2,"Kernel CVE database, recently updated, skipping")
+            return
+
+    except OSError:
+        pass
+
+    bb.utils.mkdirhier(os.path.dirname(db_file))
+    # Configure cmd
+    if not os.path.exists(db_file):
+        cmd = f"git clone {repo_url} {db_file}"
+    else:
+        cmd = f"git -C {db_file} pull"
+    try:
+        bb.fetch2.runfetchcmd(cmd, d)
+    except bb.fetch2.FetchError as e:
+        bb.warn(f"Kernel vulns repo url not accessible. {repo_url}")
+        bb.warn("Set VULNS_URL in local.conf to point to a local copy or mirror")
+}
+
+do_clean() {
+    rm -rf ${CVE_CHECK_KERNEL_DB_DIR}
+}
+
+deltask do_patch
+deltask do_unpack
+deltask do_configure
+deltask do_compile
+deltask do_install
+deltask do_populate_sysroot
+deltask do_runtime_spdx
+deltask do_create_spdx
+deltask do_populate_lic
+deltask do_cve_check
+
+do_fetch[nostamp] = "1"
+do_fetch[file-checksums] = ""
+do_fetch[vardeps] = ""
+
+EXCLUDE_FROM_WORLD = "1"


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH v2 2/6] cve-check: fix debug message
  2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull
  2025-04-28 13:42 ` [PATCH v2 1/6] linux-vulns: fetch kernel.org CNA info daniel.turull
@ 2025-04-28 13:42 ` daniel.turull
  2025-04-28 13:42 ` [PATCH v2 3/6] kernel: add support to extract compiled files daniel.turull
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 14+ messages in thread
From: daniel.turull @ 2025-04-28 13:42 UTC (permalink / raw)
  To: openembedded-core
  Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun,
	Daniel Turull

From: Daniel Turull <daniel.turull@ericsson.com>

Debug level was not added as a parameter, causing a warning.

Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
---
 meta/classes/cve-check.bbclass | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 1aef00d297..86ddfaae5f 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -288,16 +288,16 @@ def cve_update(d, cve_data, cve, entry):
         if entry['status'] == "version-in-range" and cve_data[cve]['status'] == "version-not-in-range":
             # New result from the scan, vulnerable
             cve_data[cve] = entry
-            bb.debug("CVE entry %s update from Patched to Unpatched from the scan result" % cve)
+            bb.debug(1, "CVE entry %s update from Patched to Unpatched from the scan result" % cve)
             return
     if entry['abbrev-status'] == "Patched" and cve_data[cve]['abbrev-status'] == "Unpatched":
         if entry['status'] == "version-not-in-range" and cve_data[cve]['status'] == "version-in-range":
             # Range does not match the scan, but we already have a vulnerable match, ignore
-            bb.debug("CVE entry %s update from Patched to Unpatched from the scan result - not applying" % cve)
+            bb.debug(1, "CVE entry %s update from Patched to Unpatched from the scan result - not applying" % cve)
             return
     # If we have an "Ignored", it has a priority
     if cve_data[cve]['abbrev-status'] == "Ignored":
-        bb.debug("CVE %s not updating because Ignored" % cve)
+        bb.debug(1, "CVE %s not updating because Ignored" % cve)
         return
     bb.warn("Unhandled CVE entry update for %s from %s to %s" % (cve, cve_data[cve], entry))
 


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH v2 3/6] kernel: add support to extract compiled files
  2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull
  2025-04-28 13:42 ` [PATCH v2 1/6] linux-vulns: fetch kernel.org CNA info daniel.turull
  2025-04-28 13:42 ` [PATCH v2 2/6] cve-check: fix debug message daniel.turull
@ 2025-04-28 13:42 ` daniel.turull
  2025-04-28 14:24   ` [OE-core] " Bruce Ashfield
  2025-04-28 13:42 ` [PATCH v2 4/6] cve-check: move message outsite check_cves and sort daniel.turull
                   ` (2 subsequent siblings)
  5 siblings, 1 reply; 14+ messages in thread
From: daniel.turull @ 2025-04-28 13:42 UTC (permalink / raw)
  To: openembedded-core
  Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun,
	Daniel Turull, Peter Marko

From: Daniel Turull <daniel.turull@ericsson.com>

Use gen_compile_commands.py to extract files used during compilation
for the used kernel configuration.

CC: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
---
 meta/classes-recipe/kernel.bbclass | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass
index 36ce659762..56060f2c91 100644
--- a/meta/classes-recipe/kernel.bbclass
+++ b/meta/classes-recipe/kernel.bbclass
@@ -867,3 +867,14 @@ EXPORT_FUNCTIONS do_deploy
 
 # Add using Device Tree support
 inherit kernel-devicetree
+
+KERNEL_FILES_DIR ?= "${LOG_DIR}/cve/kernel_files"
+KERNEL_SRC_FILES ?= "${KERNEL_FILES_DIR}/compile_commands.json"
+
+do_save_compiled_files() {
+    bbdebug 1 "Saving compiled files in ${KERNEL_SRC_FILES}"
+    mkdir -p ${KERNEL_FILES_DIR}
+    ${S}/scripts/clang-tools/gen_compile_commands.py -o ${KERNEL_SRC_FILES} -d ${B}
+}
+
+addtask do_save_compiled_files after do_compile do_compile_kernelmodules


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH v2 4/6] cve-check: move message outsite check_cves and sort
  2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull
                   ` (2 preceding siblings ...)
  2025-04-28 13:42 ` [PATCH v2 3/6] kernel: add support to extract compiled files daniel.turull
@ 2025-04-28 13:42 ` daniel.turull
  2025-04-28 13:42 ` [PATCH v2 5/6] cve-check, vex, spdx: use metadata from linux-vulns to enhance CVE reporting daniel.turull
  2025-04-28 13:42 ` [PATCH v2 6/6] spdx: add option to include only compiled kernel files daniel.turull
  5 siblings, 0 replies; 14+ messages in thread
From: daniel.turull @ 2025-04-28 13:42 UTC (permalink / raw)
  To: openembedded-core
  Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun,
	Daniel Turull, Peter Marko

From: Daniel Turull <daniel.turull@ericsson.com>

When adding corrections from multiple sources of CVEs, the
message showing standing CVES should be at the end. Also sort them.

CC: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
---
 meta/classes/cve-check.bbclass | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 86ddfaae5f..12159a98d2 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -173,6 +173,12 @@ python do_cve_check () {
             if len(cve_data) or (d.getVar("CVE_CHECK_COVERAGE") == "1" and status):
                 get_cve_info(d, cve_data)
                 cve_write_data(d, cve_data, status)
+
+            if d.getVar("CVE_CHECK_SHOW_WARNINGS") == "1":
+                unpatched_cves = [cve for cve in cve_data if cve_data[cve]["abbrev-status"] == "Unpatched"]
+                if unpatched_cves:
+                    bb.warn("Found unpatched CVE (%s)" % " ".join(sorted(unpatched_cves)))
+
         else:
             bb.note("No CVE database found, skipping CVE check")
 
@@ -422,10 +428,6 @@ def check_cves(d, cve_data):
     if not cves_in_recipe:
         bb.note("No CVE records for products in recipe %s" % (pn))
 
-    if d.getVar("CVE_CHECK_SHOW_WARNINGS") == "1":
-        unpatched_cves = [cve for cve in cve_data if cve_data[cve]["abbrev-status"] == "Unpatched"]
-        if unpatched_cves:
-            bb.warn("Found unpatched CVE (%s)" % " ".join(unpatched_cves))
 
     return (cve_data, cves_status)
 


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH v2 5/6] cve-check, vex, spdx: use metadata from linux-vulns to enhance CVE reporting
  2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull
                   ` (3 preceding siblings ...)
  2025-04-28 13:42 ` [PATCH v2 4/6] cve-check: move message outsite check_cves and sort daniel.turull
@ 2025-04-28 13:42 ` daniel.turull
  2025-04-28 13:42 ` [PATCH v2 6/6] spdx: add option to include only compiled kernel files daniel.turull
  5 siblings, 0 replies; 14+ messages in thread
From: daniel.turull @ 2025-04-28 13:42 UTC (permalink / raw)
  To: openembedded-core
  Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun,
	Daniel Turull, Peter Marko

From: Daniel Turull <daniel.turull@ericsson.com>

Introducing two new options to check kernel CVEs using linux-vulns.

- CVE_CHECK_KERNEL = "1"
To add linux-vulns metadata on cve-report for cve_check, vex and spdx.
This will check for kernel CVEs, and use the metadata to resolve them.
Enabled by default

- CVE_CHECK_KERNEL_CONFIG = "1"
CVE Check using kernel compiled files, disabled by default since it requires
a compiled kernel and it will increase cve-check times.
Metadata in the CVE information includes the affected files, and using the
compiled files by the kernel we can ignore some of the cves.

The above variables are defined in cve_check.bbclass, vex.bbclass and
spdx.bbclass in case not all classes are used at the same time. The ones in
cve_check has priority.

Example of output with CVE_CHECK_KERNEL_CONFIG when using cve-check:

{
  "id": "CVE-2024-26710",
  "status": "Ignored",
  "link": "https://nvd.nist.gov/vuln/detail/CVE-2024-26710",
  "summary": "In the Linux kernel, the following vulnerability [...]",
  "scorev2": "0.0",
  "scorev3": "5.5",
  "scorev4": "0.0",
  "modified": "2025-03-17T15:36:11.620",
  "vector": "LOCAL",
  "vectorString": "CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H",
  "detail": "not-applicable-config",
  "description": "Source code not compiled by config. ['arch/powerpc/include/asm/thread_info.h']"
},

And same with vex:
{
  "id": "CVE-2024-26710",
  "status": "Ignored",
  "link": "https://nvd.nist.gov/vuln/detail/CVE-2024-26710",
  "detail": "not-applicable-config",
  "description": "Source code not compiled by config. ['arch/powerpc/include/asm/thread_info.h']"
},

And new information for the Unpatched showing where the fix (if any) is available:
Tested with 6.12.22 kernel
{
  "id": "CVE-2025-39728",
  "status": "Unpatched",
  "link": "https://nvd.nist.gov/vuln/detail/CVE-2025-39728",
  "summary": "In the Linux kernel, the following vulnerability has been [...],
  "scorev2": "0.0",
  "scorev3": "0.0",
  "scorev4": "0.0",
  "modified": "2025-04-21T14:23:45.950",
  "vector": "UNKNOWN",
  "vectorString": "UNKNOWN",
  "detail": "version-in-range",
  "description": "Needs backporting (fixed from 6.12.23)"
},

Tested with cve-check, create-spdx2.2, create-spdx3.0, vex

CC: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
---
 meta/classes/cve-check.bbclass   |  19 ++-
 meta/classes/spdx-common.bbclass |   7 ++
 meta/classes/vex.bbclass         |  10 ++
 meta/lib/oe/cve_check.py         | 210 ++++++++++++++++++++++++++++++-
 4 files changed, 244 insertions(+), 2 deletions(-)

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 12159a98d2..02266f171f 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -34,6 +34,13 @@ CVE_VERSION ??= "${PV}"
 # Possible database sources: NVD1, NVD2, FKIE
 NVD_DB_VERSION ?= "FKIE"
 
+# CVE Check using kernel CNA and compiled files
+CVE_CHECK_KERNEL ?= "1"
+# CVE Check using kernel compiled files
+CVE_CHECK_KERNEL_CONFIG ?= "0"
+# Location of the linux-vulns data
+CVE_CHECK_KERNEL_DB_DIR ?= "${DL_DIR}/CVE_CHECK/vulns"
+
 # Use different file names for each database source, as they synchronize at different moments, so may be slightly different
 CVE_CHECK_DB_FILENAME ?= "${@'nvdcve_2-2.db' if d.getVar('NVD_DB_VERSION') == 'NVD2' else 'nvdcve_1-3.db' if d.getVar('NVD_DB_VERSION') == 'NVD1' else 'nvdfkie_1-1.db'}"
 CVE_CHECK_DB_FETCHER ?= "${@'cve-update-nvd2-native' if d.getVar('NVD_DB_VERSION') == 'NVD2' else 'cve-update-db-native'}"
@@ -111,6 +118,9 @@ python () {
     if nvd_database_type not in ("NVD1", "NVD2", "FKIE"):
         bb.erroronce("Malformed NVD_DB_VERSION, must be one of: NVD1, NVD2, FKIE. Defaulting to NVD2")
         d.setVar("NVD_DB_VERSION", "NVD2")
+
+    from oe.cve_check import extend_cve_kernel_config
+    extend_cve_kernel_config(d, "do_cve_check")
 }
 
 def generate_json_report(d, out_path, link_path):
@@ -161,7 +171,7 @@ python do_cve_check () {
     """
     Check recipe for patched and unpatched CVEs
     """
-    from oe.cve_check import get_patched_cves
+    from oe.cve_check import get_patched_cves, get_kernel_cves
 
     with bb.utils.fileslocked([d.getVar("CVE_CHECK_DB_FILE_LOCK")], shared=True):
         if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")):
@@ -170,6 +180,13 @@ python do_cve_check () {
             except FileNotFoundError:
                 bb.fatal("Failure in searching patches")
             cve_data, status = check_cves(d, patched_cves)
+
+            cve_check_kernel = d.getVar("CVE_CHECK_KERNEL")
+            if "linux_kernel" in d.getVar("CVE_PRODUCT") and cve_check_kernel == "1":
+                kernel_unpatched_cves, _kernel_patched_cves = get_kernel_cves(d)
+                # Patched CVEs are already returned in the get_patched_cves(). Adding the unpatched
+                cve_data.update(kernel_unpatched_cves)
+
             if len(cve_data) or (d.getVar("CVE_CHECK_COVERAGE") == "1" and status):
                 get_cve_info(d, cve_data)
                 cve_write_data(d, cve_data, status)
diff --git a/meta/classes/spdx-common.bbclass b/meta/classes/spdx-common.bbclass
index 713a7fc651..d2b9c6993a 100644
--- a/meta/classes/spdx-common.bbclass
+++ b/meta/classes/spdx-common.bbclass
@@ -37,9 +37,16 @@ SPDX_CUSTOM_ANNOTATION_VARS ??= ""
 
 SPDX_MULTILIB_SSTATE_ARCHS ??= "${SSTATE_ARCHS}"
 
+CVE_CHECK_KERNEL ??= "0"
+CVE_CHECK_KERNEL_CONFIG ??= "0"
+CVE_CHECK_KERNEL_DB_DIR ?= "${DL_DIR}/CVE_CHECK/vulns"
+
 python () {
     from oe.cve_check import extend_cve_status
     extend_cve_status(d)
+
+    from oe.cve_check import extend_cve_kernel_config
+    extend_cve_kernel_config(d, "do_create_spdx")
 }
 
 def create_spdx_source_deps(d):
diff --git a/meta/classes/vex.bbclass b/meta/classes/vex.bbclass
index 905d67b47d..56ebdb0580 100644
--- a/meta/classes/vex.bbclass
+++ b/meta/classes/vex.bbclass
@@ -26,6 +26,13 @@
 CVE_PRODUCT ??= "${BPN}"
 CVE_VERSION ??= "${PV}"
 
+# CVE Check using kernel CNA and compiled files
+CVE_CHECK_KERNEL ?= "0"
+# CVE Check using kernel compiled files
+CVE_CHECK_KERNEL_CONFIG ?= "0"
+
+CVE_CHECK_KERNEL_DB_DIR ?= "${DL_DIR}/CVE_CHECK/vulns"
+
 CVE_CHECK_SUMMARY_DIR ?= "${LOG_DIR}/cve"
 
 CVE_CHECK_SUMMARY_FILE_NAME_JSON = "cve-summary.json"
@@ -78,6 +85,9 @@ python () {
 
     from oe.cve_check import extend_cve_status
     extend_cve_status(d)
+
+    from oe.cve_check import extend_cve_kernel_config
+    extend_cve_kernel_config(d, "do_generate_vex")
 }
 
 def generate_json_report(d, out_path, link_path):
diff --git a/meta/lib/oe/cve_check.py b/meta/lib/oe/cve_check.py
index ae194f27cf..9f34da0e91 100644
--- a/meta/lib/oe/cve_check.py
+++ b/meta/lib/oe/cve_check.py
@@ -47,6 +47,7 @@ class Version():
             self._version.pre_l,
             self._version.pre_v
         )
+        self.version = version
 
     def __eq__(self, other):
         if not isinstance(other, Version):
@@ -58,6 +59,9 @@ class Version():
             return NotImplemented
         return self._key > other._key
 
+    def __str__(self) -> str:
+        return self.version
+
 def _cmpkey(release, patch_l, pre_l, pre_v):
     # remove leading 0
     _release = tuple(
@@ -202,8 +206,14 @@ def get_patched_cves(d):
                 "affected-product": decoded_status["product"],
             }
 
-    return patched_cves
+    # If we are parsing the kernel, check compiled files
+    cve_check_kernel = d.getVar("CVE_CHECK_KERNEL")
+    if "linux_kernel" in d.getVar("CVE_PRODUCT") and cve_check_kernel == "1":
+         bb.debug(1, "Checking kernel CVEs")
+         _kernel_unpatched_cves, kernel_patched_cves = get_kernel_cves(d)
+         patched_cves.update(kernel_patched_cves)
 
+    return patched_cves
 
 def get_cpe_ids(cve_product, version):
     """
@@ -376,3 +386,201 @@ def extend_cve_status(d):
                 d.setVarFlag("CVE_STATUS", cve, d.getVarFlag(cve_status_group, "status"))
         else:
             bb.warn("CVE_STATUS_GROUPS contains undefined variable %s" % cve_status_group)
+
+def extend_cve_kernel_config(d, task):
+    pn = d.getVar('PN')
+    # For kernel CVEs, add required dependencies
+    if "linux_kernel" in d.getVar("CVE_PRODUCT"):
+        if d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1":
+            bb.debug(1, "Checking kernel CVEs using kernel config")
+            depends = f" {pn}:do_save_compiled_files "
+            d.appendVarFlag(task, "depends", depends)
+            d.setVar('CVE_CHECK_KERNEL','1')
+        if d.getVar("CVE_CHECK_KERNEL") == "1":
+            bb.debug(1, "Checking kernel CVEs using linux-vulns")
+            d.appendVarFlag(task, "depends", " linux-vulns:do_fetch ")
+
+def get_kernel_cves(d):
+    """
+    Get CVEs for the kernel
+    """
+    import glob
+    import json
+    patched_cves = {}
+    unpatched_cves = {}
+    datadir = f"{d.getVar('CVE_CHECK_KERNEL_DB_DIR')}/cve/published/"
+    version_str = d.getVar("LINUX_VERSION")
+    check_config = d.getVar("CVE_CHECK_KERNEL_CONFIG")
+    version = Version(version_str)
+    base_version = Version(".".join(version_str.split(".")[0:2]))
+
+    # Check all CVES from kernel vulns
+    pattern = os.path.join(datadir, '**', f"CVE-*.json")
+    cve_files = glob.glob(pattern, recursive=True)
+    not_applicable_config = 0
+    fixed_as_later_backport = 0
+    for cve_file in cve_files:
+        cve_info = {}
+        with open(cve_file, "r") as f:
+            cve_info = json.load(f)
+
+        if len(cve_info) == 0:
+            bb.error(f"Not valid data in {cve_file}. Aborting")
+            break
+        cve_id = cve_info["cveMetadata"]["cveID"]
+
+        first_affected, fixed, backport_ver = get_kernel_fixed_versions(cve_info, base_version)
+        if not fixed:
+            if check_config == "1":
+                is_affected, affected_files = check_kernel_compiled_files(d, cve_info)
+            else:
+                is_affected = True
+            if not is_affected and len(affected_files) > 0:
+                bb.debug(1, f"{cve_id} - not applicable configuration since affected files not compiled: {affected_files}")
+                patched_cves[cve_id] = {
+                    "abbrev-status": "Ignored",
+                    "status": "not-applicable-config",
+                    "justification": f"Source code not compiled by config. {affected_files}"
+                }
+            else:
+                bb.debug(1, f"{cve_id} not fixed usptream")
+                description = cve_info["containers"]["cna"]["descriptions"][0]["value"]
+                unpatched_cves[cve_id] = {
+                    "abbrev-status": "Unpatched",
+                    "status": "version-in-range",
+                    "summary": description,
+                    "justification": f"No fix available upstream"
+                }
+        elif first_affected and version < first_affected:
+            bb.debug(1, f'{cve_id} - "fixed-version: only affects {first_affected} onwards"')
+            patched_cves[cve_id] = {
+                "abbrev-status": "Patched",
+                "status": "fixed-version",
+                "justification": f"only affects {first_affected} onwards"
+            }
+        elif fixed <= version:
+            bb.debug(1, f'{cve_id} - "fixed-version: Fixed from version {fixed}"')
+            patched_cves[cve_id] = {
+                "abbrev-status": "Patched",
+                "status": "fixed-version",
+                "justification": f"fixed-version: Fixed from version {fixed}"
+            }
+        else:
+            if backport_ver:
+                if backport_ver <= version:
+                    bb.debug(1, f'{cve_id} - "cpe-stable-backport: Backported in {backport_ver}"')
+                    patched_cves[cve_id] = {
+                        "abbrev-status": "Patched",
+                        "status": "cpe-stable-backport",
+                        "justification": f"Backported in {backport_ver}"
+                    }
+                else:
+                    bb.debug(1, f"{cve_id}: needs backporting (fixed from {backport_ver})")
+                    description = cve_info["containers"]["cna"]["descriptions"][0]["value"]
+                    unpatched_cves[cve_id] = {
+                        "abbrev-status": "Unpatched",
+                        "status": "version-in-range",
+                        "summary": description,
+                        "justification": f"Needs backporting (fixed from {backport_ver})"
+                    }
+                    fixed_as_later_backport += 1
+            else:
+                # Check if file affected
+                if check_config == "1":
+                    is_affected, affected_files = check_kernel_compiled_files(d, cve_info)
+                else:
+                    is_affected = True
+                if not is_affected and len(affected_files) > 0:
+                    bb.debug(1, f"{cve_id} - not applicable configuration since affected files not compiled: {affected_files}")
+                    patched_cves[cve_id] = {
+                         "abbrev-status": "Ignored",
+                         "status": "not-applicable-config",
+                         "justification": f"Source code not compiled by config. {affected_files}"
+                    }
+                    not_applicable_config +=1
+                else:
+                    bb.debug(1, f"{cve_id}: needs backporting (fixed from {fixed})")
+                    description = cve_info["containers"]["cna"]["descriptions"][0]["value"]
+                    unpatched_cves[cve_id] = {
+                        "abbrev-status": "Unpatched",
+                        "status": "version-in-range",
+                        "summary": description,
+                        "justification": f"Needs backporting (fixed from {fixed})"
+                    }
+    if len(cve_files) > 0:
+        bb.debug(1, f"Total CVEs ignored due to not applicable config {not_applicable_config}")
+        bb.debug(1, f"Total vulnerable CVEs: {len(unpatched_cves)}")
+        bb.debug(1, f"Total CVEs already backported in {base_version}: {fixed_as_later_backport}")
+    return unpatched_cves, patched_cves
+
+def check_kernel_compiled_files(d, cve_info):
+    """
+    Return if a CVE affected us depending on compiled files
+    """
+    import json
+    files_affected = []
+    kfiles = []
+    is_affected = False
+    with open(d.getVar('KERNEL_SRC_FILES'), 'r') as file:
+        for item in json.load(file):
+            kfiles.append(item['file'].replace(f"{d.getVar('S')}/",""))
+
+    for item in cve_info['containers']['cna']['affected']:
+        if item["defaultStatus"] == "affected":
+            if "programFiles" in item:
+                files = item['programFiles']
+                files_affected.extend(files)
+
+    if len(files_affected) > 0:
+        for f in files_affected:
+            if f in kfiles:
+                bb.debug(1, f"File match: {f}")
+                is_affected = True
+    return is_affected, files_affected
+
+def get_kernel_fixed_versions(cve_info, base_version):
+    '''
+    Get fixed versions for a given CVE
+    '''
+    first_affected = None
+    fixed = None
+    fixed_backport = None
+    next_version = Version(str(base_version) + ".5000")
+    for affected in cve_info["containers"]["cna"]["affected"]:
+        # In case the CVE info is not complete, it might not have default status and therefore
+        # we don't know the status of this CVE.
+        if not "defaultStatus" in affected:
+            return first_affected, fixed, fixed_backport
+        if affected["defaultStatus"] == "affected":
+            for version in affected["versions"]:
+                v = Version(version["version"])
+                if v == 0:
+                    # Skiping non-affected
+                    continue
+                if version["status"] == "affected" and not first_affected:
+                    first_affected = v
+                elif (version["status"] == "unaffected" and
+                    version['versionType'] == "original_commit_for_fix"):
+                    fixed = v
+                elif base_version < v and v < next_version:
+                    fixed_backport = v
+        elif affected["defaultStatus"] == "unaffected":
+            # Only specific versions are affected. We care only about our base version
+            if "versions" not in affected:
+                continue
+            for version in affected["versions"]:
+                if "versionType" not in version:
+                    continue
+                if version["versionType"] == "git":
+                    continue
+                v = Version(version["version"])
+                # in case it is not in our base version
+                less_than = Version(version["lessThan"])
+
+                if not first_affected:
+                    first_affected = v
+                fixed = less_than
+                if base_version < v and v < next_version:
+                    fixed_backport = less_than
+
+    return first_affected, fixed, fixed_backport


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH v2 6/6] spdx: add option to include only compiled kernel files
  2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull
                   ` (4 preceding siblings ...)
  2025-04-28 13:42 ` [PATCH v2 5/6] cve-check, vex, spdx: use metadata from linux-vulns to enhance CVE reporting daniel.turull
@ 2025-04-28 13:42 ` daniel.turull
  2025-04-28 13:57   ` Joshua Watt
  5 siblings, 1 reply; 14+ messages in thread
From: daniel.turull @ 2025-04-28 13:42 UTC (permalink / raw)
  To: openembedded-core
  Cc: rybczynska, steve, Peter.Marko, ross.burton, skandigraun,
	Daniel Turull, Joshua Watt, Peter Marko

From: Daniel Turull <daniel.turull@ericsson.com>

When CVE_CHECK_KERNEL_CONFIG is enabled, only include the
source code (.c, .h) files that are used during compilation.

This enables an external tool to use the SPDX information to disregard
vulnerabilities that are not compiled.

CC: Joshua Watt <JPEWhacker@gmail.com>
CC: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
---
 meta/classes/create-spdx-2.2.bbclass |  8 +++++++
 meta/lib/oe/spdx30_tasks.py          |  8 +++++++
 meta/lib/oe/spdx_common.py           | 34 ++++++++++++++++++++++++++++
 3 files changed, 50 insertions(+)

diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass
index 7e8f8b9ff5..6bf0c70bd4 100644
--- a/meta/classes/create-spdx-2.2.bbclass
+++ b/meta/classes/create-spdx-2.2.bbclass
@@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
     spdx_files = []
 
     file_counter = 1
+
+    check_kernel_compiled = bb.data.inherits_class("kernel", d) and d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1"
+    if check_kernel_compiled:
+        kernel_sources = oe.spdx_common.get_kernel_compiled_files(d)
     for subdir, dirs, files in os.walk(topdir):
         dirs[:] = [d for d in dirs if d not in ignore_dirs]
         if subdir == str(topdir):
@@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
             filename = str(filepath.relative_to(topdir))
 
             if not filepath.is_symlink() and filepath.is_file():
+                # When creating spdx for the kernel, we only include compiled files.
+                if check_kernel_compiled:
+                     if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d):
+                          break
                 spdx_file = oe.spdx.SPDXFile()
                 spdx_file.SPDXID = get_spdxid(file_counter)
                 for t in get_types(filepath):
diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py
index ba965821f8..14f26773c5 100644
--- a/meta/lib/oe/spdx30_tasks.py
+++ b/meta/lib/oe/spdx30_tasks.py
@@ -156,6 +156,10 @@ def add_package_files(
         bb.note(f"Skip {topdir}")
         return spdx_files
 
+    check_kernel_compiled = bb.data.inherits_class("kernel", d) and d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1"
+    if check_kernel_compiled:
+        kernel_sources = oe.spdx_common.get_kernel_compiled_files(d)
+
     for subdir, dirs, files in os.walk(topdir, onerror=walk_error):
         dirs[:] = [d for d in dirs if d not in ignore_dirs]
         if subdir == str(topdir):
@@ -167,6 +171,10 @@ def add_package_files(
             filepath = Path(subdir) / file
             if filepath.is_symlink() or not filepath.is_file():
                 continue
+            # When creating spdx for the kernel, we only include compiled files
+            if check_kernel_compiled:
+                 if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d):
+                      break
 
             filename = str(filepath.relative_to(topdir))
             file_purposes = get_purposes(filepath)
diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py
index 4caefc7673..c87e3875c7 100644
--- a/meta/lib/oe/spdx_common.py
+++ b/meta/lib/oe/spdx_common.py
@@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name):
         uri = uri + "@" + fd.revision
 
     return uri
+
+def is_kernel_compiled(filename, kernel_sources, d):
+    """
+    Check if the file, is a kernel compiled file
+    """
+    import os
+
+    _, extension = os.path.splitext(filename)
+    # Special case, that we need to ignore, since this is not a source file
+    if filename.rfind(".mod.c") > 0:
+        return True
+    # We filter .c files and header files
+    if extension not in [".c", ".h"]:
+        return True
+    # Check that the c file is in the list
+    if filename in kernel_sources:
+        return True
+    return False
+
+def get_kernel_compiled_files(d):
+    """
+    Get results from the save_compiled files and include also header files
+    """
+    import json
+    import os
+    kfiles = []
+    with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f:
+        for item in json.load(f):
+            kfile = os.path.basename(item['file'])
+            # Return also the correspondig header file
+            hfile = ".h".join(kfile.rsplit(".c", 1))
+            kfiles.append(kfile)
+            kfiles.append(hfile)
+    return kfiles


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
  2025-04-28 13:42 ` [PATCH v2 6/6] spdx: add option to include only compiled kernel files daniel.turull
@ 2025-04-28 13:57   ` Joshua Watt
  2025-04-28 13:59     ` Marko, Peter
  0 siblings, 1 reply; 14+ messages in thread
From: Joshua Watt @ 2025-04-28 13:57 UTC (permalink / raw)
  To: daniel.turull
  Cc: openembedded-core, rybczynska, steve, Peter.Marko, ross.burton,
	skandigraun

I see the appeal here; however I've been trying to avoid too much
kernel specific stuff in SPDX. Is there a way to make this more
generic so it could be used by any recipe?

On Mon, Apr 28, 2025 at 7:42 AM <daniel.turull@ericsson.com> wrote:
>
> From: Daniel Turull <daniel.turull@ericsson.com>
>
> When CVE_CHECK_KERNEL_CONFIG is enabled, only include the
> source code (.c, .h) files that are used during compilation.
>
> This enables an external tool to use the SPDX information to disregard
> vulnerabilities that are not compiled.
>
> CC: Joshua Watt <JPEWhacker@gmail.com>
> CC: Peter Marko <peter.marko@siemens.com>
> Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
> ---
>  meta/classes/create-spdx-2.2.bbclass |  8 +++++++
>  meta/lib/oe/spdx30_tasks.py          |  8 +++++++
>  meta/lib/oe/spdx_common.py           | 34 ++++++++++++++++++++++++++++
>  3 files changed, 50 insertions(+)
>
> diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass
> index 7e8f8b9ff5..6bf0c70bd4 100644
> --- a/meta/classes/create-spdx-2.2.bbclass
> +++ b/meta/classes/create-spdx-2.2.bbclass
> @@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
>      spdx_files = []
>
>      file_counter = 1
> +
> +    check_kernel_compiled = bb.data.inherits_class("kernel", d) and d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1"
> +    if check_kernel_compiled:
> +        kernel_sources = oe.spdx_common.get_kernel_compiled_files(d)
>      for subdir, dirs, files in os.walk(topdir):
>          dirs[:] = [d for d in dirs if d not in ignore_dirs]
>          if subdir == str(topdir):
> @@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
>              filename = str(filepath.relative_to(topdir))
>
>              if not filepath.is_symlink() and filepath.is_file():
> +                # When creating spdx for the kernel, we only include compiled files.
> +                if check_kernel_compiled:
> +                     if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d):
> +                          break
>                  spdx_file = oe.spdx.SPDXFile()
>                  spdx_file.SPDXID = get_spdxid(file_counter)
>                  for t in get_types(filepath):
> diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py
> index ba965821f8..14f26773c5 100644
> --- a/meta/lib/oe/spdx30_tasks.py
> +++ b/meta/lib/oe/spdx30_tasks.py
> @@ -156,6 +156,10 @@ def add_package_files(
>          bb.note(f"Skip {topdir}")
>          return spdx_files
>
> +    check_kernel_compiled = bb.data.inherits_class("kernel", d) and d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1"
> +    if check_kernel_compiled:
> +        kernel_sources = oe.spdx_common.get_kernel_compiled_files(d)
> +
>      for subdir, dirs, files in os.walk(topdir, onerror=walk_error):
>          dirs[:] = [d for d in dirs if d not in ignore_dirs]
>          if subdir == str(topdir):
> @@ -167,6 +171,10 @@ def add_package_files(
>              filepath = Path(subdir) / file
>              if filepath.is_symlink() or not filepath.is_file():
>                  continue
> +            # When creating spdx for the kernel, we only include compiled files
> +            if check_kernel_compiled:
> +                 if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d):
> +                      break
>
>              filename = str(filepath.relative_to(topdir))
>              file_purposes = get_purposes(filepath)
> diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py
> index 4caefc7673..c87e3875c7 100644
> --- a/meta/lib/oe/spdx_common.py
> +++ b/meta/lib/oe/spdx_common.py
> @@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name):
>          uri = uri + "@" + fd.revision
>
>      return uri
> +
> +def is_kernel_compiled(filename, kernel_sources, d):
> +    """
> +    Check if the file, is a kernel compiled file
> +    """
> +    import os
> +
> +    _, extension = os.path.splitext(filename)
> +    # Special case, that we need to ignore, since this is not a source file
> +    if filename.rfind(".mod.c") > 0:
> +        return True
> +    # We filter .c files and header files
> +    if extension not in [".c", ".h"]:
> +        return True
> +    # Check that the c file is in the list
> +    if filename in kernel_sources:
> +        return True
> +    return False
> +
> +def get_kernel_compiled_files(d):
> +    """
> +    Get results from the save_compiled files and include also header files
> +    """
> +    import json
> +    import os
> +    kfiles = []
> +    with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f:
> +        for item in json.load(f):
> +            kfile = os.path.basename(item['file'])
> +            # Return also the correspondig header file
> +            hfile = ".h".join(kfile.rsplit(".c", 1))

This feels a little bit wrong to me. I think a better option would be
to include all headers, since we don't have a good way of knowing
which are actually used.

> +            kfiles.append(kfile)
> +            kfiles.append(hfile)
> +    return kfiles


^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
  2025-04-28 13:57   ` Joshua Watt
@ 2025-04-28 13:59     ` Marko, Peter
  2025-04-28 14:12       ` Daniel Turull
  2025-04-28 14:25       ` Daniel Turull
  0 siblings, 2 replies; 14+ messages in thread
From: Marko, Peter @ 2025-04-28 13:59 UTC (permalink / raw)
  To: Joshua Watt, daniel.turull@ericsson.com
  Cc: openembedded-core@lists.openembedded.org, rybczynska@gmail.com,
	steve@sakoman.com, ross.burton@arm.com, skandigraun@gmail.com

I think that only kernel provides file-based CVE information right now.

Peter

> -----Original Message-----
> From: Joshua Watt <jpewhacker@gmail.com>
> Sent: Monday, April 28, 2025 15:58
> To: daniel.turull@ericsson.com
> Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com;
> steve@sakoman.com; Marko, Peter (FT D EU SK BFS1)
> <Peter.Marko@siemens.com>; ross.burton@arm.com; skandigraun@gmail.com
> Subject: Re: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
> 
> I see the appeal here; however I've been trying to avoid too much
> kernel specific stuff in SPDX. Is there a way to make this more
> generic so it could be used by any recipe?
> 
> On Mon, Apr 28, 2025 at 7:42 AM <daniel.turull@ericsson.com> wrote:
> >
> > From: Daniel Turull <daniel.turull@ericsson.com>
> >
> > When CVE_CHECK_KERNEL_CONFIG is enabled, only include the
> > source code (.c, .h) files that are used during compilation.
> >
> > This enables an external tool to use the SPDX information to disregard
> > vulnerabilities that are not compiled.
> >
> > CC: Joshua Watt <JPEWhacker@gmail.com>
> > CC: Peter Marko <peter.marko@siemens.com>
> > Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
> > ---
> >  meta/classes/create-spdx-2.2.bbclass |  8 +++++++
> >  meta/lib/oe/spdx30_tasks.py          |  8 +++++++
> >  meta/lib/oe/spdx_common.py           | 34 ++++++++++++++++++++++++++++
> >  3 files changed, 50 insertions(+)
> >
> > diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-
> 2.2.bbclass
> > index 7e8f8b9ff5..6bf0c70bd4 100644
> > --- a/meta/classes/create-spdx-2.2.bbclass
> > +++ b/meta/classes/create-spdx-2.2.bbclass
> > @@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
> get_spdxid, get_types, *, archiv
> >      spdx_files = []
> >
> >      file_counter = 1
> > +
> > +    check_kernel_compiled = bb.data.inherits_class("kernel", d) and
> d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1"
> > +    if check_kernel_compiled:
> > +        kernel_sources = oe.spdx_common.get_kernel_compiled_files(d)
> >      for subdir, dirs, files in os.walk(topdir):
> >          dirs[:] = [d for d in dirs if d not in ignore_dirs]
> >          if subdir == str(topdir):
> > @@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
> get_spdxid, get_types, *, archiv
> >              filename = str(filepath.relative_to(topdir))
> >
> >              if not filepath.is_symlink() and filepath.is_file():
> > +                # When creating spdx for the kernel, we only include compiled files.
> > +                if check_kernel_compiled:
> > +                     if not oe.spdx_common.is_kernel_compiled(file, kernel_sources,
> d):
> > +                          break
> >                  spdx_file = oe.spdx.SPDXFile()
> >                  spdx_file.SPDXID = get_spdxid(file_counter)
> >                  for t in get_types(filepath):
> > diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py
> > index ba965821f8..14f26773c5 100644
> > --- a/meta/lib/oe/spdx30_tasks.py
> > +++ b/meta/lib/oe/spdx30_tasks.py
> > @@ -156,6 +156,10 @@ def add_package_files(
> >          bb.note(f"Skip {topdir}")
> >          return spdx_files
> >
> > +    check_kernel_compiled = bb.data.inherits_class("kernel", d) and
> d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1"
> > +    if check_kernel_compiled:
> > +        kernel_sources = oe.spdx_common.get_kernel_compiled_files(d)
> > +
> >      for subdir, dirs, files in os.walk(topdir, onerror=walk_error):
> >          dirs[:] = [d for d in dirs if d not in ignore_dirs]
> >          if subdir == str(topdir):
> > @@ -167,6 +171,10 @@ def add_package_files(
> >              filepath = Path(subdir) / file
> >              if filepath.is_symlink() or not filepath.is_file():
> >                  continue
> > +            # When creating spdx for the kernel, we only include compiled files
> > +            if check_kernel_compiled:
> > +                 if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d):
> > +                      break
> >
> >              filename = str(filepath.relative_to(topdir))
> >              file_purposes = get_purposes(filepath)
> > diff --git a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py
> > index 4caefc7673..c87e3875c7 100644
> > --- a/meta/lib/oe/spdx_common.py
> > +++ b/meta/lib/oe/spdx_common.py
> > @@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name):
> >          uri = uri + "@" + fd.revision
> >
> >      return uri
> > +
> > +def is_kernel_compiled(filename, kernel_sources, d):
> > +    """
> > +    Check if the file, is a kernel compiled file
> > +    """
> > +    import os
> > +
> > +    _, extension = os.path.splitext(filename)
> > +    # Special case, that we need to ignore, since this is not a source file
> > +    if filename.rfind(".mod.c") > 0:
> > +        return True
> > +    # We filter .c files and header files
> > +    if extension not in [".c", ".h"]:
> > +        return True
> > +    # Check that the c file is in the list
> > +    if filename in kernel_sources:
> > +        return True
> > +    return False
> > +
> > +def get_kernel_compiled_files(d):
> > +    """
> > +    Get results from the save_compiled files and include also header files
> > +    """
> > +    import json
> > +    import os
> > +    kfiles = []
> > +    with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f:
> > +        for item in json.load(f):
> > +            kfile = os.path.basename(item['file'])
> > +            # Return also the correspondig header file
> > +            hfile = ".h".join(kfile.rsplit(".c", 1))
> 
> This feels a little bit wrong to me. I think a better option would be
> to include all headers, since we don't have a good way of knowing
> which are actually used.
> 
> > +            kfiles.append(kfile)
> > +            kfiles.append(hfile)
> > +    return kfiles

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
  2025-04-28 13:59     ` Marko, Peter
@ 2025-04-28 14:12       ` Daniel Turull
  2025-04-28 14:44         ` Joshua Watt
  2025-04-28 14:25       ` Daniel Turull
  1 sibling, 1 reply; 14+ messages in thread
From: Daniel Turull @ 2025-04-28 14:12 UTC (permalink / raw)
  To: Marko, Peter, Joshua Watt
  Cc: openembedded-core@lists.openembedded.org, rybczynska@gmail.com,
	steve@sakoman.com, ross.burton@arm.com, skandigraun@gmail.com

Hi,

Yes, from what I saw and Peter says only the kernel has file information that can be machine readable,
and we can also extract the files that we compile.

I could try to normalize how the files are input into the spdx generation and then have a generic variable that any recipe can set
to enable the inclusion of a subset of files. In this case, if/when more recipes provides this,
spdx doesn't need to change.

Best regards
Daniel

-----Original Message-----
From: Marko, Peter <Peter.Marko@siemens.com> 
Sent: Monday, 28 April 2025 16:00
To: Joshua Watt <jpewhacker@gmail.com>; Daniel Turull <daniel.turull@ericsson.com>
Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; steve@sakoman.com; ross.burton@arm.com; skandigraun@gmail.com
Subject: RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files

I think that only kernel provides file-based CVE information right now.

Peter

> -----Original Message-----
> From: Joshua Watt <jpewhacker@gmail.com>
> Sent: Monday, April 28, 2025 15:58
> To: daniel.turull@ericsson.com
> Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; 
> steve@sakoman.com; Marko, Peter (FT D EU SK BFS1) 
> <Peter.Marko@siemens.com>; ross.burton@arm.com; skandigraun@gmail.com
> Subject: Re: [PATCH v2 6/6] spdx: add option to include only compiled 
> kernel files
> 
> I see the appeal here; however I've been trying to avoid too much 
> kernel specific stuff in SPDX. Is there a way to make this more 
> generic so it could be used by any recipe?
> 
> On Mon, Apr 28, 2025 at 7:42 AM <daniel.turull@ericsson.com> wrote:
> >
> > From: Daniel Turull <daniel.turull@ericsson.com>
> >
> > When CVE_CHECK_KERNEL_CONFIG is enabled, only include the source 
> > code (.c, .h) files that are used during compilation.
> >
> > This enables an external tool to use the SPDX information to 
> > disregard vulnerabilities that are not compiled.
> >
> > CC: Joshua Watt <JPEWhacker@gmail.com>
> > CC: Peter Marko <peter.marko@siemens.com>
> > Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
> > ---
> >  meta/classes/create-spdx-2.2.bbclass |  8 +++++++
> >  meta/lib/oe/spdx30_tasks.py          |  8 +++++++
> >  meta/lib/oe/spdx_common.py           | 34 ++++++++++++++++++++++++++++
> >  3 files changed, 50 insertions(+)
> >
> > diff --git a/meta/classes/create-spdx-2.2.bbclass 
> > b/meta/classes/create-spdx-
> 2.2.bbclass
> > index 7e8f8b9ff5..6bf0c70bd4 100644
> > --- a/meta/classes/create-spdx-2.2.bbclass
> > +++ b/meta/classes/create-spdx-2.2.bbclass
> > @@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
> get_spdxid, get_types, *, archiv
> >      spdx_files = []
> >
> >      file_counter = 1
> > +
> > +    check_kernel_compiled = bb.data.inherits_class("kernel", d) and
> d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1"
> > +    if check_kernel_compiled:
> > +        kernel_sources = 
> > + oe.spdx_common.get_kernel_compiled_files(d)
> >      for subdir, dirs, files in os.walk(topdir):
> >          dirs[:] = [d for d in dirs if d not in ignore_dirs]
> >          if subdir == str(topdir):
> > @@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
> get_spdxid, get_types, *, archiv
> >              filename = str(filepath.relative_to(topdir))
> >
> >              if not filepath.is_symlink() and filepath.is_file():
> > +                # When creating spdx for the kernel, we only include compiled files.
> > +                if check_kernel_compiled:
> > +                     if not oe.spdx_common.is_kernel_compiled(file, 
> > + kernel_sources,
> d):
> > +                          break
> >                  spdx_file = oe.spdx.SPDXFile()
> >                  spdx_file.SPDXID = get_spdxid(file_counter)
> >                  for t in get_types(filepath):
> > diff --git a/meta/lib/oe/spdx30_tasks.py 
> > b/meta/lib/oe/spdx30_tasks.py index ba965821f8..14f26773c5 100644
> > --- a/meta/lib/oe/spdx30_tasks.py
> > +++ b/meta/lib/oe/spdx30_tasks.py
> > @@ -156,6 +156,10 @@ def add_package_files(
> >          bb.note(f"Skip {topdir}")
> >          return spdx_files
> >
> > +    check_kernel_compiled = bb.data.inherits_class("kernel", d) and
> d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1"
> > +    if check_kernel_compiled:
> > +        kernel_sources = 
> > + oe.spdx_common.get_kernel_compiled_files(d)
> > +
> >      for subdir, dirs, files in os.walk(topdir, onerror=walk_error):
> >          dirs[:] = [d for d in dirs if d not in ignore_dirs]
> >          if subdir == str(topdir):
> > @@ -167,6 +171,10 @@ def add_package_files(
> >              filepath = Path(subdir) / file
> >              if filepath.is_symlink() or not filepath.is_file():
> >                  continue
> > +            # When creating spdx for the kernel, we only include compiled files
> > +            if check_kernel_compiled:
> > +                 if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d):
> > +                      break
> >
> >              filename = str(filepath.relative_to(topdir))
> >              file_purposes = get_purposes(filepath) diff --git 
> > a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py index 
> > 4caefc7673..c87e3875c7 100644
> > --- a/meta/lib/oe/spdx_common.py
> > +++ b/meta/lib/oe/spdx_common.py
> > @@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name):
> >          uri = uri + "@" + fd.revision
> >
> >      return uri
> > +
> > +def is_kernel_compiled(filename, kernel_sources, d):
> > +    """
> > +    Check if the file, is a kernel compiled file
> > +    """
> > +    import os
> > +
> > +    _, extension = os.path.splitext(filename)
> > +    # Special case, that we need to ignore, since this is not a source file
> > +    if filename.rfind(".mod.c") > 0:
> > +        return True
> > +    # We filter .c files and header files
> > +    if extension not in [".c", ".h"]:
> > +        return True
> > +    # Check that the c file is in the list
> > +    if filename in kernel_sources:
> > +        return True
> > +    return False
> > +
> > +def get_kernel_compiled_files(d):
> > +    """
> > +    Get results from the save_compiled files and include also header files
> > +    """
> > +    import json
> > +    import os
> > +    kfiles = []
> > +    with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f:
> > +        for item in json.load(f):
> > +            kfile = os.path.basename(item['file'])
> > +            # Return also the correspondig header file
> > +            hfile = ".h".join(kfile.rsplit(".c", 1))
> 
> This feels a little bit wrong to me. I think a better option would be 
> to include all headers, since we don't have a good way of knowing 
> which are actually used.
> 
> > +            kfiles.append(kfile)
> > +            kfiles.append(hfile)
> > +    return kfiles

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [OE-core] [PATCH v2 3/6] kernel: add support to extract compiled files
  2025-04-28 13:42 ` [PATCH v2 3/6] kernel: add support to extract compiled files daniel.turull
@ 2025-04-28 14:24   ` Bruce Ashfield
  2025-04-28 14:28     ` Daniel Turull
  0 siblings, 1 reply; 14+ messages in thread
From: Bruce Ashfield @ 2025-04-28 14:24 UTC (permalink / raw)
  To: daniel.turull
  Cc: openembedded-core, rybczynska, steve, Peter.Marko, ross.burton,
	skandigraun

[-- Attachment #1: Type: text/plain, Size: 2067 bytes --]

On Mon, Apr 28, 2025 at 9:42 AM Daniel Turull via lists.openembedded.org
<daniel.turull=ericsson.com@lists.openembedded.org> wrote:

> From: Daniel Turull <daniel.turull@ericsson.com>
>
> Use gen_compile_commands.py to extract files used during compilation
> for the used kernel configuration.
>
> CC: Peter Marko <peter.marko@siemens.com>
> Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
> ---
>  meta/classes-recipe/kernel.bbclass | 11 +++++++++++
>  1 file changed, 11 insertions(+)
>
> diff --git a/meta/classes-recipe/kernel.bbclass
> b/meta/classes-recipe/kernel.bbclass
> index 36ce659762..56060f2c91 100644
> --- a/meta/classes-recipe/kernel.bbclass
> +++ b/meta/classes-recipe/kernel.bbclass
> @@ -867,3 +867,14 @@ EXPORT_FUNCTIONS do_deploy
>
>  # Add using Device Tree support
>  inherit kernel-devicetree
> +
> +KERNEL_FILES_DIR ?= "${LOG_DIR}/cve/kernel_files"
> +KERNEL_SRC_FILES ?= "${KERNEL_FILES_DIR}/compile_commands.json"
> +
> +do_save_compiled_files() {
> +    bbdebug 1 "Saving compiled files in ${KERNEL_SRC_FILES}"
> +    mkdir -p ${KERNEL_FILES_DIR}
> +    ${S}/scripts/clang-tools/gen_compile_commands.py -o
> ${KERNEL_SRC_FILES} -d ${B}
> +}
> +
> +addtask do_save_compiled_files after do_compile do_compile_kernelmodules
>

This is something that I wouldn't want running in my development
environment.

I can't tell fully. Is this opt in ? Is there a way to opt-out ?

Bruce



>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#215604):
> https://lists.openembedded.org/g/openembedded-core/message/215604
> Mute This Topic: https://lists.openembedded.org/mt/112499916/1050810
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 3892 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
  2025-04-28 13:59     ` Marko, Peter
  2025-04-28 14:12       ` Daniel Turull
@ 2025-04-28 14:25       ` Daniel Turull
  1 sibling, 0 replies; 14+ messages in thread
From: Daniel Turull @ 2025-04-28 14:25 UTC (permalink / raw)
  To: Marko, Peter, Joshua Watt
  Cc: openembedded-core@lists.openembedded.org, rybczynska@gmail.com,
	steve@sakoman.com, ross.burton@arm.com, skandigraun@gmail.com

Hi,
I missed to answer your comment.

Daniel

-----Original Message-----
From: Marko, Peter <Peter.Marko@siemens.com> 
Sent: Monday, 28 April 2025 16:00
To: Joshua Watt <jpewhacker@gmail.com>; Daniel Turull <daniel.turull@ericsson.com>
Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; steve@sakoman.com; ross.burton@arm.com; skandigraun@gmail.com
Subject: RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files

I think that only kernel provides file-based CVE information right now.

Peter

> -----Original Message-----
> From: Joshua Watt <jpewhacker@gmail.com>
> Sent: Monday, April 28, 2025 15:58
> To: daniel.turull@ericsson.com
> Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; 
> steve@sakoman.com; Marko, Peter (FT D EU SK BFS1) 
> <Peter.Marko@siemens.com>; ross.burton@arm.com; skandigraun@gmail.com
> Subject: Re: [PATCH v2 6/6] spdx: add option to include only compiled 
> kernel files
> 
> I see the appeal here; however I've been trying to avoid too much 
> kernel specific stuff in SPDX. Is there a way to make this more 
> generic so it could be used by any recipe?
> 
> On Mon, Apr 28, 2025 at 7:42 AM <daniel.turull@ericsson.com> wrote:
> >
> > From: Daniel Turull <daniel.turull@ericsson.com>
> >
> > When CVE_CHECK_KERNEL_CONFIG is enabled, only include the source 
> > code (.c, .h) files that are used during compilation.
> >
> > This enables an external tool to use the SPDX information to 
> > disregard vulnerabilities that are not compiled.
> >
> > CC: Joshua Watt <JPEWhacker@gmail.com>
> > CC: Peter Marko <peter.marko@siemens.com>
> > Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
> > ---
> >  meta/classes/create-spdx-2.2.bbclass |  8 +++++++
> >  meta/lib/oe/spdx30_tasks.py          |  8 +++++++
> >  meta/lib/oe/spdx_common.py           | 34 ++++++++++++++++++++++++++++
> >  3 files changed, 50 insertions(+)
> >
> > diff --git a/meta/classes/create-spdx-2.2.bbclass 
> > b/meta/classes/create-spdx-
> 2.2.bbclass
> > index 7e8f8b9ff5..6bf0c70bd4 100644
> > --- a/meta/classes/create-spdx-2.2.bbclass
> > +++ b/meta/classes/create-spdx-2.2.bbclass
> > @@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
> get_spdxid, get_types, *, archiv
> >      spdx_files = []
> >
> >      file_counter = 1
> > +
> > +    check_kernel_compiled = bb.data.inherits_class("kernel", d) and
> d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1"
> > +    if check_kernel_compiled:
> > +        kernel_sources = 
> > + oe.spdx_common.get_kernel_compiled_files(d)
> >      for subdir, dirs, files in os.walk(topdir):
> >          dirs[:] = [d for d in dirs if d not in ignore_dirs]
> >          if subdir == str(topdir):
> > @@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
> get_spdxid, get_types, *, archiv
> >              filename = str(filepath.relative_to(topdir))
> >
> >              if not filepath.is_symlink() and filepath.is_file():
> > +                # When creating spdx for the kernel, we only include compiled files.
> > +                if check_kernel_compiled:
> > +                     if not oe.spdx_common.is_kernel_compiled(file, 
> > + kernel_sources,
> d):
> > +                          break
> >                  spdx_file = oe.spdx.SPDXFile()
> >                  spdx_file.SPDXID = get_spdxid(file_counter)
> >                  for t in get_types(filepath):
> > diff --git a/meta/lib/oe/spdx30_tasks.py 
> > b/meta/lib/oe/spdx30_tasks.py index ba965821f8..14f26773c5 100644
> > --- a/meta/lib/oe/spdx30_tasks.py
> > +++ b/meta/lib/oe/spdx30_tasks.py
> > @@ -156,6 +156,10 @@ def add_package_files(
> >          bb.note(f"Skip {topdir}")
> >          return spdx_files
> >
> > +    check_kernel_compiled = bb.data.inherits_class("kernel", d) and
> d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1"
> > +    if check_kernel_compiled:
> > +        kernel_sources = 
> > + oe.spdx_common.get_kernel_compiled_files(d)
> > +
> >      for subdir, dirs, files in os.walk(topdir, onerror=walk_error):
> >          dirs[:] = [d for d in dirs if d not in ignore_dirs]
> >          if subdir == str(topdir):
> > @@ -167,6 +171,10 @@ def add_package_files(
> >              filepath = Path(subdir) / file
> >              if filepath.is_symlink() or not filepath.is_file():
> >                  continue
> > +            # When creating spdx for the kernel, we only include compiled files
> > +            if check_kernel_compiled:
> > +                 if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d):
> > +                      break
> >
> >              filename = str(filepath.relative_to(topdir))
> >              file_purposes = get_purposes(filepath) diff --git 
> > a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py index 
> > 4caefc7673..c87e3875c7 100644
> > --- a/meta/lib/oe/spdx_common.py
> > +++ b/meta/lib/oe/spdx_common.py
> > @@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name):
> >          uri = uri + "@" + fd.revision
> >
> >      return uri
> > +
> > +def is_kernel_compiled(filename, kernel_sources, d):
> > +    """
> > +    Check if the file, is a kernel compiled file
> > +    """
> > +    import os
> > +
> > +    _, extension = os.path.splitext(filename)
> > +    # Special case, that we need to ignore, since this is not a source file
> > +    if filename.rfind(".mod.c") > 0:
> > +        return True
> > +    # We filter .c files and header files
> > +    if extension not in [".c", ".h"]:
> > +        return True
> > +    # Check that the c file is in the list
> > +    if filename in kernel_sources:
> > +        return True
> > +    return False
> > +
> > +def get_kernel_compiled_files(d):
> > +    """
> > +    Get results from the save_compiled files and include also header files
> > +    """
> > +    import json
> > +    import os
> > +    kfiles = []
> > +    with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f:
> > +        for item in json.load(f):
> > +            kfile = os.path.basename(item['file'])
> > +            # Return also the correspondig header file
> > +            hfile = ".h".join(kfile.rsplit(".c", 1))
> 
> This feels a little bit wrong to me. I think a better option would be 
> to include all headers, since we don't have a good way of knowing 
> which are actually used.

Yes, I had my doubt regarding this. It probably doesn't have an impact in the rest of the functionality for the CVEs
I'll keep only the "c" files and try to make it more generic.

> 
> > +            kfiles.append(kfile)
> > +            kfiles.append(hfile)
> > +    return kfiles

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE: [OE-core] [PATCH v2 3/6] kernel: add support to extract compiled files
  2025-04-28 14:24   ` [OE-core] " Bruce Ashfield
@ 2025-04-28 14:28     ` Daniel Turull
  0 siblings, 0 replies; 14+ messages in thread
From: Daniel Turull @ 2025-04-28 14:28 UTC (permalink / raw)
  To: Bruce Ashfield
  Cc: openembedded-core@lists.openembedded.org, rybczynska@gmail.com,
	steve@sakoman.com, Peter.Marko@siemens.com, ross.burton@arm.com,
	skandigraun@gmail.com

[-- Attachment #1: Type: text/plain, Size: 2810 bytes --]

Hi,

I have put it as default, but you have a good point. I’ll add the task conditionally if CVE_CHECK_KERNEL_CONFIG is set to 1 in the next patch version series.

Best regards,
Daniel
From: Bruce Ashfield <bruce.ashfield@gmail.com>
Sent: Monday, 28 April 2025 16:25
To: Daniel Turull <daniel.turull@ericsson.com>
Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; steve@sakoman.com; Peter.Marko@siemens.com; ross.burton@arm.com; skandigraun@gmail.com
Subject: Re: [OE-core] [PATCH v2 3/6] kernel: add support to extract compiled files



On Mon, Apr 28, 2025 at 9:42 AM Daniel Turull via lists.openembedded.org<http://lists.openembedded.org/> <daniel.turull=ericsson.com@lists.openembedded.org<mailto:ericsson.com@lists.openembedded.org>> wrote:
From: Daniel Turull <daniel.turull@ericsson.com<mailto:daniel.turull@ericsson.com>>

Use gen_compile_commands.py to extract files used during compilation
for the used kernel configuration.

CC: Peter Marko <peter.marko@siemens.com<mailto:peter.marko@siemens.com>>
Signed-off-by: Daniel Turull <daniel.turull@ericsson.com<mailto:daniel.turull@ericsson.com>>
---
 meta/classes-recipe/kernel.bbclass | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/meta/classes-recipe/kernel.bbclass b/meta/classes-recipe/kernel.bbclass
index 36ce659762..56060f2c91 100644
--- a/meta/classes-recipe/kernel.bbclass
+++ b/meta/classes-recipe/kernel.bbclass
@@ -867,3 +867,14 @@ EXPORT_FUNCTIONS do_deploy

 # Add using Device Tree support
 inherit kernel-devicetree
+
+KERNEL_FILES_DIR ?= "${LOG_DIR}/cve/kernel_files"
+KERNEL_SRC_FILES ?= "${KERNEL_FILES_DIR}/compile_commands.json"
+
+do_save_compiled_files() {
+    bbdebug 1 "Saving compiled files in ${KERNEL_SRC_FILES}"
+    mkdir -p ${KERNEL_FILES_DIR}
+    ${S}/scripts/clang-tools/gen_compile_commands.py -o ${KERNEL_SRC_FILES} -d ${B}
+}
+
+addtask do_save_compiled_files after do_compile do_compile_kernelmodules

This is something that I wouldn't want running in my development environment.

I can't tell fully. Is this opt in ? Is there a way to opt-out ?

Bruce




-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#215604): https://lists.openembedded.org/g/openembedded-core/message/215604
Mute This Topic: https://lists.openembedded.org/mt/112499916/1050810
Group Owner: openembedded-core+owner@lists.openembedded.org<mailto:openembedded-core%2Bowner@lists.openembedded.org>
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com<mailto:bruce.ashfield@gmail.com>]
-=-=-=-=-=-=-=-=-=-=-=-


--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 7570 bytes --]

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
  2025-04-28 14:12       ` Daniel Turull
@ 2025-04-28 14:44         ` Joshua Watt
  0 siblings, 0 replies; 14+ messages in thread
From: Joshua Watt @ 2025-04-28 14:44 UTC (permalink / raw)
  To: Daniel Turull
  Cc: Marko, Peter, openembedded-core@lists.openembedded.org,
	rybczynska@gmail.com, steve@sakoman.com, ross.burton@arm.com,
	skandigraun@gmail.com

On Mon, Apr 28, 2025 at 8:12 AM Daniel Turull
<daniel.turull@ericsson.com> wrote:
>
> Hi,
>
> Yes, from what I saw and Peter says only the kernel has file information that can be machine readable,
> and we can also extract the files that we compile.
>
> I could try to normalize how the files are input into the spdx generation and then have a generic variable that any recipe can set
> to enable the inclusion of a subset of files. In this case, if/when more recipes provides this,
> spdx doesn't need to change.

I was thinking that maybe the way to do this is to have this somehow
call a function when enabled, and then have the recipe (or even better
the build system bbclass) implement that function; that way each
recipe or build system can implement the correct method for collecting
the data. I'm not quite sure if that's the best way to do it though

>
> Best regards
> Daniel
>
> -----Original Message-----
> From: Marko, Peter <Peter.Marko@siemens.com>
> Sent: Monday, 28 April 2025 16:00
> To: Joshua Watt <jpewhacker@gmail.com>; Daniel Turull <daniel.turull@ericsson.com>
> Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com; steve@sakoman.com; ross.burton@arm.com; skandigraun@gmail.com
> Subject: RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
>
> I think that only kernel provides file-based CVE information right now.
>
> Peter
>
> > -----Original Message-----
> > From: Joshua Watt <jpewhacker@gmail.com>
> > Sent: Monday, April 28, 2025 15:58
> > To: daniel.turull@ericsson.com
> > Cc: openembedded-core@lists.openembedded.org; rybczynska@gmail.com;
> > steve@sakoman.com; Marko, Peter (FT D EU SK BFS1)
> > <Peter.Marko@siemens.com>; ross.burton@arm.com; skandigraun@gmail.com
> > Subject: Re: [PATCH v2 6/6] spdx: add option to include only compiled
> > kernel files
> >
> > I see the appeal here; however I've been trying to avoid too much
> > kernel specific stuff in SPDX. Is there a way to make this more
> > generic so it could be used by any recipe?
> >
> > On Mon, Apr 28, 2025 at 7:42 AM <daniel.turull@ericsson.com> wrote:
> > >
> > > From: Daniel Turull <daniel.turull@ericsson.com>
> > >
> > > When CVE_CHECK_KERNEL_CONFIG is enabled, only include the source
> > > code (.c, .h) files that are used during compilation.
> > >
> > > This enables an external tool to use the SPDX information to
> > > disregard vulnerabilities that are not compiled.
> > >
> > > CC: Joshua Watt <JPEWhacker@gmail.com>
> > > CC: Peter Marko <peter.marko@siemens.com>
> > > Signed-off-by: Daniel Turull <daniel.turull@ericsson.com>
> > > ---
> > >  meta/classes/create-spdx-2.2.bbclass |  8 +++++++
> > >  meta/lib/oe/spdx30_tasks.py          |  8 +++++++
> > >  meta/lib/oe/spdx_common.py           | 34 ++++++++++++++++++++++++++++
> > >  3 files changed, 50 insertions(+)
> > >
> > > diff --git a/meta/classes/create-spdx-2.2.bbclass
> > > b/meta/classes/create-spdx-
> > 2.2.bbclass
> > > index 7e8f8b9ff5..6bf0c70bd4 100644
> > > --- a/meta/classes/create-spdx-2.2.bbclass
> > > +++ b/meta/classes/create-spdx-2.2.bbclass
> > > @@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
> > get_spdxid, get_types, *, archiv
> > >      spdx_files = []
> > >
> > >      file_counter = 1
> > > +
> > > +    check_kernel_compiled = bb.data.inherits_class("kernel", d) and
> > d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1"
> > > +    if check_kernel_compiled:
> > > +        kernel_sources =
> > > + oe.spdx_common.get_kernel_compiled_files(d)
> > >      for subdir, dirs, files in os.walk(topdir):
> > >          dirs[:] = [d for d in dirs if d not in ignore_dirs]
> > >          if subdir == str(topdir):
> > > @@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
> > get_spdxid, get_types, *, archiv
> > >              filename = str(filepath.relative_to(topdir))
> > >
> > >              if not filepath.is_symlink() and filepath.is_file():
> > > +                # When creating spdx for the kernel, we only include compiled files.
> > > +                if check_kernel_compiled:
> > > +                     if not oe.spdx_common.is_kernel_compiled(file,
> > > + kernel_sources,
> > d):
> > > +                          break
> > >                  spdx_file = oe.spdx.SPDXFile()
> > >                  spdx_file.SPDXID = get_spdxid(file_counter)
> > >                  for t in get_types(filepath):
> > > diff --git a/meta/lib/oe/spdx30_tasks.py
> > > b/meta/lib/oe/spdx30_tasks.py index ba965821f8..14f26773c5 100644
> > > --- a/meta/lib/oe/spdx30_tasks.py
> > > +++ b/meta/lib/oe/spdx30_tasks.py
> > > @@ -156,6 +156,10 @@ def add_package_files(
> > >          bb.note(f"Skip {topdir}")
> > >          return spdx_files
> > >
> > > +    check_kernel_compiled = bb.data.inherits_class("kernel", d) and
> > d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1"
> > > +    if check_kernel_compiled:
> > > +        kernel_sources =
> > > + oe.spdx_common.get_kernel_compiled_files(d)
> > > +
> > >      for subdir, dirs, files in os.walk(topdir, onerror=walk_error):
> > >          dirs[:] = [d for d in dirs if d not in ignore_dirs]
> > >          if subdir == str(topdir):
> > > @@ -167,6 +171,10 @@ def add_package_files(
> > >              filepath = Path(subdir) / file
> > >              if filepath.is_symlink() or not filepath.is_file():
> > >                  continue
> > > +            # When creating spdx for the kernel, we only include compiled files
> > > +            if check_kernel_compiled:
> > > +                 if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d):
> > > +                      break
> > >
> > >              filename = str(filepath.relative_to(topdir))
> > >              file_purposes = get_purposes(filepath) diff --git
> > > a/meta/lib/oe/spdx_common.py b/meta/lib/oe/spdx_common.py index
> > > 4caefc7673..c87e3875c7 100644
> > > --- a/meta/lib/oe/spdx_common.py
> > > +++ b/meta/lib/oe/spdx_common.py
> > > @@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name):
> > >          uri = uri + "@" + fd.revision
> > >
> > >      return uri
> > > +
> > > +def is_kernel_compiled(filename, kernel_sources, d):
> > > +    """
> > > +    Check if the file, is a kernel compiled file
> > > +    """
> > > +    import os
> > > +
> > > +    _, extension = os.path.splitext(filename)
> > > +    # Special case, that we need to ignore, since this is not a source file
> > > +    if filename.rfind(".mod.c") > 0:
> > > +        return True
> > > +    # We filter .c files and header files
> > > +    if extension not in [".c", ".h"]:
> > > +        return True
> > > +    # Check that the c file is in the list
> > > +    if filename in kernel_sources:
> > > +        return True
> > > +    return False
> > > +
> > > +def get_kernel_compiled_files(d):
> > > +    """
> > > +    Get results from the save_compiled files and include also header files
> > > +    """
> > > +    import json
> > > +    import os
> > > +    kfiles = []
> > > +    with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f:
> > > +        for item in json.load(f):
> > > +            kfile = os.path.basename(item['file'])
> > > +            # Return also the correspondig header file
> > > +            hfile = ".h".join(kfile.rsplit(".c", 1))
> >
> > This feels a little bit wrong to me. I think a better option would be
> > to include all headers, since we don't have a good way of knowing
> > which are actually used.
> >
> > > +            kfiles.append(kfile)
> > > +            kfiles.append(hfile)
> > > +    return kfiles


^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2025-04-28 14:44 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-04-28 13:41 [PATCH v2 0/6] Check compiled files to filter kernel CVEs daniel.turull
2025-04-28 13:42 ` [PATCH v2 1/6] linux-vulns: fetch kernel.org CNA info daniel.turull
2025-04-28 13:42 ` [PATCH v2 2/6] cve-check: fix debug message daniel.turull
2025-04-28 13:42 ` [PATCH v2 3/6] kernel: add support to extract compiled files daniel.turull
2025-04-28 14:24   ` [OE-core] " Bruce Ashfield
2025-04-28 14:28     ` Daniel Turull
2025-04-28 13:42 ` [PATCH v2 4/6] cve-check: move message outsite check_cves and sort daniel.turull
2025-04-28 13:42 ` [PATCH v2 5/6] cve-check, vex, spdx: use metadata from linux-vulns to enhance CVE reporting daniel.turull
2025-04-28 13:42 ` [PATCH v2 6/6] spdx: add option to include only compiled kernel files daniel.turull
2025-04-28 13:57   ` Joshua Watt
2025-04-28 13:59     ` Marko, Peter
2025-04-28 14:12       ` Daniel Turull
2025-04-28 14:44         ` Joshua Watt
2025-04-28 14:25       ` Daniel Turull

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox