[PATCH v5 00/15] add SPDX SBOM generation script

[Luis <luis.augenstein@tngtech.com>]

This patch series introduces a Python-based script for generating SBOM
documents in the SPDX 3.0.1 format for kernel builds.

A Software Bill of Materials (SBOM) describes the individual components
of a software product. For the kernel, the goal is to describe the
distributable build outputs (typically the kernel image and modules),
the source files involved in producing these outputs, and the build
process that connects the source and output files.

To achieve this, the sbom script generates three SPDX documents:

- sbom-output.spdx.json
  Describes the final build outputs together with high-level
  build metadata.

- sbom-source.spdx.json
  Describes all source files involved in the build, including
  licensing information and additional file metadata.

- sbom-build.spdx.json
  Describes the entire build process, linking source files
  from the source SBOM to output files in the output SBOM.

The sbom script is optional. It can be invoked via the `make sbom` target.
This target depends on `all` and triggers a standard kernel build. Once all
output artifacts have been generated, starting from the kernel image and
modules as root nodes, the script reconstructs the dependency graph up
to the original source files. Build dependencies are primarily derived from
the `.cmd` files generated by Kbuild, which record the full command used
to build each output file.

Currently, the script only supports x86 and arm64 architectures.

This series was developed with assistance from AI tools, namely Cursor
with Claude Sonnet 4.5 and OpenCode with GLM-4.7. The AI was used for
documentation, exploring the repository, and iterating on design
questions and implementation details such as regex patterns.

Assisted-by: Cursor:claude-sonnet-4-5
Assisted-by: OpenCode:GLM-4-7
Co-developed-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Maximilian Huber <maximilian.huber@tngtech.com>
Signed-off-by: Luis Augenstein <luis.augenstein@tngtech.com>
---
Changes in v5:
- simplify make integration. No dedicated Makefile. Only top level.
- support dynamic command parser templates based on environment variable overwrites: 
  CC, LD, AR, NM, OBJCOPY, STRIP.
- add --write-output-on-error cli argument when invoked from the Makefile which
  allows to generate spdx documents despite errors.
  This was for example a problem in https://birdcloud.org/bc/Linux_Kernel_Missing_SPDX_ID_lines_from_build_SBOMs
  The script still exits with a non-zero exit code.
- split savedcmd_parser.py file into multiple files for better readability.
- add command parser for gen-hyprel command.
- remove the created cli argument for CreationInfo object which does not need to be a configurable. 
  Instead, the latest modified timestamp of the provided root artifacts is used.
- remove the originatedBy property from Package elements which was set to the agent who created the SBOM.
  This was a wrong usage of the property.
- remove some spdx properties that were not used in the current version, namely SoftwareArtifact.additionalPurpose,
  Package.downloadLocation, Artifact.builtTime, and Agent.externalIdentifier
---
Luis Augenstein (15):
  scripts/sbom: add documentation
  scripts/sbom: integrate script in make process
  scripts/sbom: setup sbom logging
  scripts/sbom: add command parsers
  scripts/sbom: add cmd graph generation
  scripts/sbom: add additional dependency sources for cmd graph
  scripts/sbom: add SPDX classes
  scripts/sbom: add JSON-LD serialization
  scripts/sbom: add shared SPDX elements
  scripts/sbom: collect file metadata
  scripts/sbom: add SPDX output graph
  scripts/sbom: add SPDX source graph
  scripts/sbom: add SPDX build graph
  scripts/sbom: add unit tests for command parsers
  scripts/sbom: add unit tests for SPDX-License-Identifier parsing

 .gitignore                                    |   1 +
 Documentation/tools/index.rst                 |   1 +
 Documentation/tools/sbom/sbom.rst             | 206 ++++++++
 MAINTAINERS                                   |   6 +
 Makefile                                      |  28 +-
 scripts/sbom/sbom.py                          | 129 +++++
 scripts/sbom/sbom/__init__.py                 |   0
 scripts/sbom/sbom/cmd_graph/__init__.py       |   7 +
 scripts/sbom/sbom/cmd_graph/cmd_file.py       | 149 ++++++
 scripts/sbom/sbom/cmd_graph/cmd_graph.py      |  46 ++
 scripts/sbom/sbom/cmd_graph/cmd_graph_node.py | 142 ++++++
 scripts/sbom/sbom/cmd_graph/deps_parser.py    |  52 ++
 .../sbom/cmd_graph/hardcoded_dependencies.py  |  83 +++
 scripts/sbom/sbom/cmd_graph/incbin_parser.py  |  42 ++
 .../cmd_graph/savedcmd_parser/__init__.py     |   6 +
 .../command_parser_registry.py                | 474 ++++++++++++++++++
 .../savedcmd_parser/command_splitter.py       | 124 +++++
 .../savedcmd_parser/savedcmd_parser.py        |  68 +++
 .../cmd_graph/savedcmd_parser/tokenizer.py    |  94 ++++
 scripts/sbom/sbom/config.py                   | 318 ++++++++++++
 scripts/sbom/sbom/environment.py              | 192 +++++++
 scripts/sbom/sbom/path_utils.py               |  11 +
 scripts/sbom/sbom/sbom_logging.py             |  88 ++++
 scripts/sbom/sbom/spdx/__init__.py            |   7 +
 scripts/sbom/sbom/spdx/build.py               |  17 +
 scripts/sbom/sbom/spdx/core.py                | 170 +++++++
 scripts/sbom/sbom/spdx/serialization.py       |  56 +++
 scripts/sbom/sbom/spdx/simplelicensing.py     |  20 +
 scripts/sbom/sbom/spdx/software.py            |  69 +++
 scripts/sbom/sbom/spdx/spdxId.py              |  36 ++
 scripts/sbom/sbom/spdx_graph/__init__.py      |   7 +
 .../sbom/sbom/spdx_graph/build_spdx_graphs.py |  82 +++
 scripts/sbom/sbom/spdx_graph/kernel_file.py   | 310 ++++++++++++
 .../sbom/spdx_graph/shared_spdx_elements.py   |  32 ++
 .../sbom/sbom/spdx_graph/spdx_build_graph.py  | 317 ++++++++++++
 .../sbom/sbom/spdx_graph/spdx_graph_model.py  |  36 ++
 .../sbom/sbom/spdx_graph/spdx_output_graph.py | 187 +++++++
 .../sbom/sbom/spdx_graph/spdx_source_graph.py | 126 +++++
 scripts/sbom/tests/__init__.py                |   0
 scripts/sbom/tests/cmd_graph/__init__.py      |   0
 .../tests/cmd_graph/test_savedcmd_parser.py   | 412 +++++++++++++++
 scripts/sbom/tests/spdx_graph/__init__.py     |   0
 .../sbom/tests/spdx_graph/test_kernel_file.py |  32 ++
 43 files changed, 4181 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/tools/sbom/sbom.rst
 create mode 100644 scripts/sbom/sbom.py
 create mode 100644 scripts/sbom/sbom/__init__.py
 create mode 100644 scripts/sbom/sbom/cmd_graph/__init__.py
 create mode 100644 scripts/sbom/sbom/cmd_graph/cmd_file.py
 create mode 100644 scripts/sbom/sbom/cmd_graph/cmd_graph.py
 create mode 100644 scripts/sbom/sbom/cmd_graph/cmd_graph_node.py
 create mode 100644 scripts/sbom/sbom/cmd_graph/deps_parser.py
 create mode 100644 scripts/sbom/sbom/cmd_graph/hardcoded_dependencies.py
 create mode 100644 scripts/sbom/sbom/cmd_graph/incbin_parser.py
 create mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/__init__.py
 create mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/command_parser_registry.py
 create mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/command_splitter.py
 create mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/savedcmd_parser.py
 create mode 100644 scripts/sbom/sbom/cmd_graph/savedcmd_parser/tokenizer.py
 create mode 100644 scripts/sbom/sbom/config.py
 create mode 100644 scripts/sbom/sbom/environment.py
 create mode 100644 scripts/sbom/sbom/path_utils.py
 create mode 100644 scripts/sbom/sbom/sbom_logging.py
 create mode 100644 scripts/sbom/sbom/spdx/__init__.py
 create mode 100644 scripts/sbom/sbom/spdx/build.py
 create mode 100644 scripts/sbom/sbom/spdx/core.py
 create mode 100644 scripts/sbom/sbom/spdx/serialization.py
 create mode 100644 scripts/sbom/sbom/spdx/simplelicensing.py
 create mode 100644 scripts/sbom/sbom/spdx/software.py
 create mode 100644 scripts/sbom/sbom/spdx/spdxId.py
 create mode 100644 scripts/sbom/sbom/spdx_graph/__init__.py
 create mode 100644 scripts/sbom/sbom/spdx_graph/build_spdx_graphs.py
 create mode 100644 scripts/sbom/sbom/spdx_graph/kernel_file.py
 create mode 100644 scripts/sbom/sbom/spdx_graph/shared_spdx_elements.py
 create mode 100644 scripts/sbom/sbom/spdx_graph/spdx_build_graph.py
 create mode 100644 scripts/sbom/sbom/spdx_graph/spdx_graph_model.py
 create mode 100644 scripts/sbom/sbom/spdx_graph/spdx_output_graph.py
 create mode 100644 scripts/sbom/sbom/spdx_graph/spdx_source_graph.py
 create mode 100644 scripts/sbom/tests/__init__.py
 create mode 100644 scripts/sbom/tests/cmd_graph/__init__.py
 create mode 100644 scripts/sbom/tests/cmd_graph/test_savedcmd_parser.py
 create mode 100644 scripts/sbom/tests/spdx_graph/__init__.py
 create mode 100644 scripts/sbom/tests/spdx_graph/test_kernel_file.py

-- 
2.43.0