* [docs] ref-manual: Document SPDX 3.0.1 variables
@ 2025-11-14 14:52 Stefano Tondo
2025-11-14 15:12 ` [docs][PATCH v2] " Stefano Tondo
[not found] ` <1877E8B8F84BDDDE.1114802@lists.yoctoproject.org>
0 siblings, 2 replies; 4+ messages in thread
From: Stefano Tondo @ 2025-11-14 14:52 UTC (permalink / raw)
To: docs; +Cc: Stefano Tondo, peter.marko, adrian.freihofer
From: Stefano Tondo <stefano.tondo.ext@siemens.com>
Add comprehensive documentation for SPDX-related variables in the
Yocto reference manual. This includes documenting previously
undocumented variables and updating existing documentation with
SPDX 3.0.1 specific information.
Variables documented:
- SPDX_INCLUDE_SOURCES: Control source file inclusion in SBOM
Updated with concrete SPDX 3.0.1 size data (~260 MB without sources,
~2.5-2.6 GB with sources)
- SPDX_INCLUDE_COMPILED_SOURCES: Control compiled source inclusion
- SPDX_UUID_NAMESPACE: Namespace for UUID generation in SPDX documents
- SPDX_NAMESPACE_PREFIX: URI prefix for SPDX documents
- SPDX_PRETTY: JSON output formatting control
- SPDX_LICENSES: Path to SPDX license identifier mapping file
- SPDX_CUSTOM_ANNOTATION_VARS: Custom annotation variables
- SPDX_MULTILIB_SSTATE_ARCHS: Architecture list for dependency collection
These variables are defined in meta/classes/spdx-common.bbclass and
are now properly documented in the reference manual with detailed
descriptions and practical usage information.
Signed-off-by: Stefano Tondo <stefano.tondo.ext@siemens.com>
---
documentation/ref-manual/variables.rst | 186 ++++---------------------
1 file changed, 30 insertions(+), 156 deletions(-)
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index a80ef364e..6a4472531 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -1526,11 +1526,6 @@ system and gives an overview of their function and contents.
:term:`CC`
The minimal command and arguments used to run the C compiler.
- :term:`CCACHE_DISABLE`
- When inheriting the :ref:`ref-classes-ccache` class, the
- :term:`CCACHE_DISABLE` variable can be set to "1" in a recipe to disable
- `Ccache` support. This is useful when the recipe is known to not support it.
-
:term:`CCLD`
The minimal command and arguments used to run the linker when the C
compiler is being used as the linker.
@@ -3961,70 +3956,6 @@ system and gives an overview of their function and contents.
material for Wic is located in the
":doc:`/ref-manual/kickstart`" chapter.
- :term:`IMAGE_EXTRA_PARTITION_FILES`
- A space-separated list of files installed into the extra partition(s)
- when preparing an image using the Wic tool with the
- ``extra_partition`` source plugin. By default,
- the files are
- installed under the same name as the source files. To change the
- installed name, separate it from the original name with a semi-colon
- (;). Source files need to be located in
- :term:`DEPLOY_DIR_IMAGE`. Here is an
- example::
-
- IMAGE_EXTRA_PARTITION_FILES = "foobar file.conf;config"
-
- In the above example, the file ``foobar`` is installed with its original name
- ``foobar``, while the file ``file.conf`` is installed and renamed to ``config``.
-
- Alternatively, source files can be picked up using a glob pattern.
- However, hidden files are ignored, and the pattern is non-recursive
- (subdirectories are ignored).
- The destination file will have the same name as the base
- name of the source file path. To install files into a renamed directory
- within the target location, pass its name after a semi-colon (;).
- Here are two examples::
-
- IMAGE_EXTRA_PARTITION_FILES = "foo/*"
- IMAGE_EXTRA_PARTITION_FILES = "foo/*;bar/"
-
- The first line in this example
- installs all files from ``foo`` directory
- into the root of the target partition. The second line in this example installs
- the same files into a ``bar`` directory within the target partition.
- The ``bar/`` directory is automatically created if it does not exist.
-
- You can also specify the target by label, UUID or partition name if multiple
- extra partitions coexist. Let's take the following example. This would be
- the WKS file for the image currently being built::
-
- part --source extra_partition --fstype=ext4 --label foo
- part --source extra_partition --fstype=ext4 --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d
- part --source extra_partition --fstype=ext4 --part-name config
-
- And the following configuration::
-
- IMAGE_EXTRA_PARTITION_FILES_label-foo = "foo/*"
- IMAGE_EXTRA_PARTITION_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d = "foo/*;bar/"
- IMAGE_EXTRA_PARTITION_FILES_part-name-config = "config"
-
- Then:
-
- - The partition labeled "foo" would get all files from the ``foo``
- directory.
-
- - The partition whose UUID is "e7d0824e-cda3-4bed-9f54-9ef5312d105d"
- would get all files from the ``foo`` directory, installed into a
- ``bar`` directory.
-
- - The partition named "config" would get the file ``config``.
-
- You can find information on how to use the Wic tool in the
- ":ref:`dev-manual/wic:creating partitioned images using wic`"
- section of the Yocto Project Development Tasks Manual. Reference
- material for Wic is located in the
- ":doc:`/ref-manual/kickstart`" chapter.
-
:term:`IMAGE_FEATURES`
The primary list of features to include in an image. Typically, you
configure this variable in an image recipe. Although you can use this
@@ -5523,7 +5454,7 @@ system and gives an overview of their function and contents.
information on how this variable is used.
:term:`LAYERDEPENDS`
- Lists the layers, separated by spaces, on which this layer depends.
+ Lists the layers, separated by spaces, on which this recipe depends.
Optionally, you can specify a specific layer version for a dependency
by adding it to the end of the layer name. Here is an example::
@@ -7940,16 +7871,6 @@ system and gives an overview of their function and contents.
:term:`REPODIR`
See :term:`bitbake:REPODIR` in the BitBake manual.
- :term:`REQUIRED_COMBINED_FEATURES`
- When inheriting the :ref:`ref-classes-features_check` class, this variable
- identifies combined features (the intersection of :term:`MACHINE_FEATURES`
- and :term:`DISTRO_FEATURES`) that must exist in the current configuration
- in order for the :term:`OpenEmbedded Build System` to build the recipe. In
- other words, if the :term:`REQUIRED_COMBINED_FEATURES` variable lists a
- feature that does not appear in :term:`COMBINED_FEATURES` within the
- current configuration, then the recipe will be skipped, and if the build
- system attempts to build the recipe then an error will be triggered.
-
:term:`REQUIRED_DISTRO_FEATURES`
When inheriting the :ref:`ref-classes-features_check`
class, this variable identifies distribution features that must exist
@@ -7960,41 +7881,6 @@ system and gives an overview of their function and contents.
the recipe will be skipped, and if the build system attempts to build
the recipe then an error will be triggered.
- :term:`REQUIRED_IMAGE_FEATURES`
- When inheriting the :ref:`ref-classes-features_check` class, this variable
- identifies image features that must exist in the current
- configuration in order for the :term:`OpenEmbedded Build System` to build
- the recipe. In other words, if the :term:`REQUIRED_IMAGE_FEATURES` variable
- lists a feature that does not appear in :term:`IMAGE_FEATURES` within the
- current configuration, then the recipe will be skipped, and if the build
- system attempts to build the recipe then an error will be triggered.
-
- Compared to other ``REQUIRED_*_FEATURES`` variables, the
- :term:`REQUIRED_IMAGE_FEATURES` varible only targets image recipes, as the
- :term:`IMAGE_FEATURES` variable is handled by the :ref:`ref-classes-core-image`
- class). However, the :term:`REQUIRED_IMAGE_FEATURES` varible can also be
- set from a :term:`Configuration File`, such as a distro
- configuration file, if the list of required image features should apply to
- all images using this :term:`DISTRO`.
-
- :term:`REQUIRED_MACHINE_FEATURES`
- When inheriting the :ref:`ref-classes-features_check` class, this variable
- identifies :term:`MACHINE_FEATURES` that must exist in the current
- configuration in order for the :term:`OpenEmbedded Build System` to build
- the recipe. In other words, if the :term:`REQUIRED_MACHINE_FEATURES` variable
- lists a feature that does not appear in :term:`MACHINE_FEATURES` within the
- current configuration, then the recipe will be skipped, and if the build
- system attempts to build the recipe then an error will be triggered.
-
- :term:`REQUIRED_TUNE_FEATURES`
- When inheriting the :ref:`ref-classes-features_check` class, this variable
- identifies tune features that must exist in the current configuration in
- order for the :term:`OpenEmbedded Build System` to build the recipe. In
- other words, if the :term:`REQUIRED_TUNE_FEATURES` variable lists a
- feature that does not appear in :term:`TUNE_FEATURES` within the current
- configuration, then the recipe will be skipped, and if the build system
- attempts to build the recipe then an error will be triggered.
-
:term:`REQUIRED_VERSION`
If there are multiple versions of a recipe available, this variable
determines which version should be given preference.
@@ -8962,10 +8848,21 @@ system and gives an overview of their function and contents.
SPDX_INCLUDE_COMPILED_SOURCES = "1"
- According to our tests, building ``core-image-minimal`` for the
- ``qemux86-64`` machine, enabling this option compared with the
- :term:`SPDX_INCLUDE_SOURCES` reduces the size of the ``tmp/deploy/spdx``
- directory from 2GB to 1.6GB.
+ According to our tests on release 4.1 "langdale" (SPDX 2.2 format), building
+ ``core-image-minimal`` for the ``qemux86-64`` machine, enabling this
+ option compared with the :term:`SPDX_INCLUDE_SOURCES` reduced the size
+ of the ``tmp/deploy/spdx`` directory from 2GB to 1.6GB.
+
+ With SPDX 3.0.1 JSON-LD format, the uncompressed image SBOM file
+ (``core-image-minimal-qemux86-64.rootfs.spdx.json``) is approximately
+ **260 MB without sources** (``SPDX_INCLUDE_SOURCES = "0"``), and increases to
+ **~2.5-2.6 GB when sources are included** (either ``SPDX_INCLUDE_SOURCES = "1"``
+ or ``SPDX_INCLUDE_COMPILED_SOURCES = "1"``). This represents approximately
+ a **10x size increase** when including source files.
+
+ Note: SPDX 3.0.1 JSON-LD files are not compressed by default, unlike the
+ tar.zst format used in SPDX 2.2. Compression (e.g. zstd) can significantly
+ reduce the file size.
:term:`SPDX_INCLUDE_SOURCES`
This option allows to add a description of the source files used to build
@@ -8979,8 +8876,8 @@ system and gives an overview of their function and contents.
SPDX_INCLUDE_SOURCES = "1"
- According to our tests on release 4.1 "langdale", building
- ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
+ According to our tests on release 4.1 "langdale" (SPDX 2.2 format),
+ building ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
this option multiplied the total size of the ``tmp/deploy/spdx``
directory by a factor of 3 (+291 MiB for this image),
and the size of the ``IMAGE-MACHINE.spdx.tar.zst`` in
@@ -8988,6 +8885,17 @@ system and gives an overview of their function and contents.
image), compared to just using the :ref:`ref-classes-create-spdx` class
with no option.
+ With SPDX 3.0.1 JSON-LD format, the uncompressed image SBOM file
+ (``core-image-minimal-qemux86-64.rootfs.spdx.json``) is approximately
+ **260 MB without sources** (default: ``SPDX_INCLUDE_SOURCES = "0"``),
+ and increases to **~2.5-2.6 GB when sources are included**
+ (``SPDX_INCLUDE_SOURCES = "1"``). This represents approximately a
+ **10x size increase** when including source files.
+
+ Note: SPDX 3.0.1 JSON-LD files are not compressed by default, unlike the
+ tar.zst format used in SPDX 2.2. Compression (e.g. zstd) can significantly
+ reduce the file size.
+
:term:`SPDX_NAMESPACE_PREFIX`
This option could be used in order to change the prefix of ``spdxDocument``
and the prefix of ``documentNamespace``. It is set by default to
@@ -10906,12 +10814,6 @@ system and gives an overview of their function and contents.
Please see the "Selection of Processor Architecture and Board Type"
section in the U-Boot README for valid values for this variable.
- :term:`UBOOT_MAKE_OPTS`
- The :term:`UBOOT_MAKE_OPTS` variable can be used to pass extra options to
- ``make`` when U-Boot is configured and compiled.
-
- See the :ref:`ref-classes-uboot-config` class for more information.
-
:term:`UBOOT_MAKE_TARGET`
Specifies the target called in the ``Makefile``. The default target
is "all".
@@ -11034,22 +10936,6 @@ system and gives an overview of their function and contents.
`Unified Kernel Image (UKI) <https://uapi-group.org/specifications/specs/unified_kernel_image/>`__.
Defaults to ``ukify build``.
- :term:`UNINATIVE_CHECKSUM`
- When inheriting the :ref:`ref-classes-uninative` class, the
- :term:`UNINATIVE_CHECKSUM` variable flags contain the checksums of the
- uninative tarball as specified by the :term:`UNINATIVE_URL` variable.
- There should be one checksum per tarballs published at
- :term:`UNINATIVE_URL`, which match architectures. For example::
-
- UNINATIVE_CHECKSUM[aarch64] ?= "812045d826b7fda88944055e8526b95a5a9440bfef608d5b53fd52faab49bf85"
- UNINATIVE_CHECKSUM[i686] ?= "5cc28efd0c15a75de4bcb147c6cce65f1c1c9d442173a220f08427f40a3ffa09"
- UNINATIVE_CHECKSUM[x86_64] ?= "4c03d1ed2b7b4e823aca4a1a23d8f2e322f1770fc10e859adcede5777aff4f3a"
-
- :term:`UNINATIVE_URL`
- When inheriting the :ref:`ref-classes-uninative` class, the
- :term:`UNINATIVE_URL` variable contains the URL where the uninative
- tarballs are published.
-
:term:`UNKNOWN_CONFIGURE_OPT_IGNORE`
Specifies a list of options that, if reported by the configure script
as being invalid, should not generate a warning during the
@@ -11069,7 +10955,7 @@ system and gives an overview of their function and contents.
:term:`UNPACKDIR`
This variable, used by the :ref:`ref-classes-base` class,
- specifies where fetched sources should be unpacked by the
+ specifies where fetches sources should be unpacked by the
:ref:`ref-tasks-unpack` task.
:term:`UPDATERCPN`
@@ -11145,18 +11031,6 @@ system and gives an overview of their function and contents.
the Yocto Project Development Tasks Manual for information on how to
use this variable.
- :term:`USE_NLS`
- Determine if language translations should be built for recipes that can
- build them. This variable can be equal to:
-
- - ``yes``: translations are enabled.
- - ``no``: translation are disabled.
-
- Recipes can use the value of this variable to enable language
- translations in their build. Classes such as :ref:`ref-classes-gettext`
- use the value of this variable to enable :wikipedia:`Gettext <Gettext>`
- support.
-
:term:`USE_VT`
When using
:ref:`SysVinit <dev-manual/new-recipe:enabling system services>`,
--
2.51.1
^ permalink raw reply related [flat|nested] 4+ messages in thread* [docs][PATCH v2] ref-manual: Document SPDX 3.0.1 variables
2025-11-14 14:52 [docs] ref-manual: Document SPDX 3.0.1 variables Stefano Tondo
@ 2025-11-14 15:12 ` Stefano Tondo
2025-11-24 15:48 ` Antonin Godard
[not found] ` <1877E8B8F84BDDDE.1114802@lists.yoctoproject.org>
1 sibling, 1 reply; 4+ messages in thread
From: Stefano Tondo @ 2025-11-14 15:12 UTC (permalink / raw)
To: docs; +Cc: Stefano Tondo, peter.marko, adrian.freihofer
From: Stefano Tondo <stefano.tondo.ext@siemens.com>
Add comprehensive documentation for SPDX-related variables in the
Yocto reference manual. This includes documenting previously
undocumented variables and updating existing documentation with
SPDX 3.0.1 specific information.
New variables documented:
- SPDX_LICENSES: Path to SPDX license identifier mapping file
- SPDX_MULTILIB_SSTATE_ARCHS: Architecture list for dependency collection
- SPDX_UUID_NAMESPACE: Namespace for UUID generation in SPDX documents
Updated existing variables with SPDX 3.0.1 data:
- SPDX_INCLUDE_SOURCES: Added concrete SPDX 3.0.1 size data
(~260 MB without sources, ~2.5-2.6 GB with sources)
- SPDX_INCLUDE_COMPILED_SOURCES: Updated with SPDX 3.0.1 format information
The size information is based on building core-image-minimal for
qemux86-64 with SPDX 3.0.1 JSON-LD format. The uncompressed SBOM
file is approximately 260 MB without sources and increases to
~2.5-2.6 GB when sources are included (approximately 10x increase).
Note: SPDX 3.0.1 uses uncompressed JSON-LD format by default,
unlike the tar.zst compression used in SPDX 2.2.
These variables are defined in meta/classes/spdx-common.bbclass.
Signed-off-by: Stefano Tondo <stefano.tondo.ext@siemens.com>
---
Changes in v2:
- Fixed unintended removal of non-SPDX variables (CCACHE_DISABLE,
IMAGE_EXTRA_PARTITION_FILES, REQUIRED_*_FEATURES)
- Fixed unintended change to LAYERDEPENDS description
- Only SPDX-related variables are now modified
documentation/ref-manual/variables.rst | 72 +++++++++++++++++++++++---
1 file changed, 66 insertions(+), 6 deletions(-)
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index a80ef364e..c098227f2 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -8962,10 +8962,21 @@ system and gives an overview of their function and contents.
SPDX_INCLUDE_COMPILED_SOURCES = "1"
- According to our tests, building ``core-image-minimal`` for the
- ``qemux86-64`` machine, enabling this option compared with the
- :term:`SPDX_INCLUDE_SOURCES` reduces the size of the ``tmp/deploy/spdx``
- directory from 2GB to 1.6GB.
+ According to our tests on release 4.1 "langdale" (SPDX 2.2 format), building
+ ``core-image-minimal`` for the ``qemux86-64`` machine, enabling this
+ option compared with the :term:`SPDX_INCLUDE_SOURCES` reduced the size
+ of the ``tmp/deploy/spdx`` directory from 2GB to 1.6GB.
+
+ With SPDX 3.0.1 JSON-LD format, the uncompressed image SBOM file
+ (``core-image-minimal-qemux86-64.rootfs.spdx.json``) is approximately
+ **260 MB without sources** (``SPDX_INCLUDE_SOURCES = "0"``), and increases to
+ **~2.5-2.6 GB when sources are included** (either ``SPDX_INCLUDE_SOURCES = "1"``
+ or ``SPDX_INCLUDE_COMPILED_SOURCES = "1"``). This represents approximately
+ a **10x size increase** when including source files.
+
+ Note: SPDX 3.0.1 JSON-LD files are not compressed by default, unlike the
+ tar.zst format used in SPDX 2.2. Compression (e.g. zstd) can significantly
+ reduce the file size.
:term:`SPDX_INCLUDE_SOURCES`
This option allows to add a description of the source files used to build
@@ -8979,8 +8990,8 @@ system and gives an overview of their function and contents.
SPDX_INCLUDE_SOURCES = "1"
- According to our tests on release 4.1 "langdale", building
- ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
+ According to our tests on release 4.1 "langdale" (SPDX 2.2 format),
+ building ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
this option multiplied the total size of the ``tmp/deploy/spdx``
directory by a factor of 3 (+291 MiB for this image),
and the size of the ``IMAGE-MACHINE.spdx.tar.zst`` in
@@ -8988,6 +8999,42 @@ system and gives an overview of their function and contents.
image), compared to just using the :ref:`ref-classes-create-spdx` class
with no option.
+ With SPDX 3.0.1 JSON-LD format, the uncompressed image SBOM file
+ (``core-image-minimal-qemux86-64.rootfs.spdx.json``) is approximately
+ **260 MB without sources** (default: ``SPDX_INCLUDE_SOURCES = "0"``),
+ and increases to **~2.5-2.6 GB when sources are included**
+ (``SPDX_INCLUDE_SOURCES = "1"``). This represents approximately a
+ **10x size increase** when including source files.
+
+ Note: SPDX 3.0.1 JSON-LD files are not compressed by default, unlike the
+ tar.zst format used in SPDX 2.2. Compression (e.g. zstd) can significantly
+ reduce the file size.
+
+ :term:`SPDX_LICENSES`
+ Path to the JSON file containing SPDX license identifier mappings. This
+ file maps common license names to official SPDX license identifiers used
+ during SBOM generation.
+
+ The default value is::
+
+ SPDX_LICENSES = "${COREBASE}/meta/files/spdx-licenses.json"
+
+ You can override this variable to use a custom license mapping file if
+ your organization uses different license naming conventions.
+
+ :term:`SPDX_MULTILIB_SSTATE_ARCHS`
+ The list of sstate architectures to consider when collecting SPDX
+ dependencies. This includes multilib architectures when multilib is
+ enabled.
+
+ The default value is::
+
+ SPDX_MULTILIB_SSTATE_ARCHS = "${SSTATE_ARCHS}"
+
+ This variable is used internally by the SPDX generation classes to
+ ensure all relevant dependencies are included in the SBOM, regardless
+ of whether multilib is enabled or not.
+
:term:`SPDX_NAMESPACE_PREFIX`
This option could be used in order to change the prefix of ``spdxDocument``
and the prefix of ``documentNamespace``. It is set by default to
@@ -9015,6 +9062,19 @@ system and gives an overview of their function and contents.
this option is recommended if you want to inspect the SPDX
output files with a text editor.
+ :term:`SPDX_UUID_NAMESPACE`
+ The namespace used for generating UUIDs in SPDX documents. This should
+ be a domain name or unique identifier for your organization to ensure
+ globally unique SPDX IDs across different builds and organizations.
+
+ The default value is::
+
+ SPDX_UUID_NAMESPACE = "sbom.openembedded.org"
+
+ You can override this variable to use your organization's domain::
+
+ SPDX_UUID_NAMESPACE = "sbom.example.com"
+
:term:`SPDXLICENSEMAP`
Maps commonly used license names to their SPDX counterparts found in
``meta/files/common-licenses/``. For the default :term:`SPDXLICENSEMAP`
--
2.51.1
^ permalink raw reply related [flat|nested] 4+ messages in thread* Re: [docs][PATCH v2] ref-manual: Document SPDX 3.0.1 variables
2025-11-14 15:12 ` [docs][PATCH v2] " Stefano Tondo
@ 2025-11-24 15:48 ` Antonin Godard
0 siblings, 0 replies; 4+ messages in thread
From: Antonin Godard @ 2025-11-24 15:48 UTC (permalink / raw)
To: stondo, docs; +Cc: Stefano Tondo, peter.marko, adrian.freihofer
Hi,
On Fri Nov 14, 2025 at 4:12 PM CET, Stefano Tondo via lists.yoctoproject.org wrote:
> From: Stefano Tondo <stefano.tondo.ext@siemens.com>
>
> Add comprehensive documentation for SPDX-related variables in the
> Yocto reference manual. This includes documenting previously
> undocumented variables and updating existing documentation with
> SPDX 3.0.1 specific information.
>
> New variables documented:
> - SPDX_LICENSES: Path to SPDX license identifier mapping file
> - SPDX_MULTILIB_SSTATE_ARCHS: Architecture list for dependency collection
> - SPDX_UUID_NAMESPACE: Namespace for UUID generation in SPDX documents
>
> Updated existing variables with SPDX 3.0.1 data:
> - SPDX_INCLUDE_SOURCES: Added concrete SPDX 3.0.1 size data
> (~260 MB without sources, ~2.5-2.6 GB with sources)
> - SPDX_INCLUDE_COMPILED_SOURCES: Updated with SPDX 3.0.1 format information
>
> The size information is based on building core-image-minimal for
> qemux86-64 with SPDX 3.0.1 JSON-LD format. The uncompressed SBOM
> file is approximately 260 MB without sources and increases to
> ~2.5-2.6 GB when sources are included (approximately 10x increase).
>
> Note: SPDX 3.0.1 uses uncompressed JSON-LD format by default,
> unlike the tar.zst compression used in SPDX 2.2.
>
> These variables are defined in meta/classes/spdx-common.bbclass.
>
> Signed-off-by: Stefano Tondo <stefano.tondo.ext@siemens.com>
> ---
>
> Changes in v2:
> - Fixed unintended removal of non-SPDX variables (CCACHE_DISABLE,
> IMAGE_EXTRA_PARTITION_FILES, REQUIRED_*_FEATURES)
> - Fixed unintended change to LAYERDEPENDS description
> - Only SPDX-related variables are now modified
>
> documentation/ref-manual/variables.rst | 72 +++++++++++++++++++++++---
> 1 file changed, 66 insertions(+), 6 deletions(-)
>
> diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
> index a80ef364e..c098227f2 100644
> --- a/documentation/ref-manual/variables.rst
> +++ b/documentation/ref-manual/variables.rst
> @@ -8962,10 +8962,21 @@ system and gives an overview of their function and contents.
>
> SPDX_INCLUDE_COMPILED_SOURCES = "1"
>
> - According to our tests, building ``core-image-minimal`` for the
> - ``qemux86-64`` machine, enabling this option compared with the
> - :term:`SPDX_INCLUDE_SOURCES` reduces the size of the ``tmp/deploy/spdx``
> - directory from 2GB to 1.6GB.
> + According to our tests on release 4.1 "langdale" (SPDX 2.2 format), building
Did you really test this on langdale, or did you add langdale here because it
was mentioned in the other variable definitions?
> + ``core-image-minimal`` for the ``qemux86-64`` machine, enabling this
> + option compared with the :term:`SPDX_INCLUDE_SOURCES` reduced the size
> + of the ``tmp/deploy/spdx`` directory from 2GB to 1.6GB.
> +
> + With SPDX 3.0.1 JSON-LD format, the uncompressed image SBOM file
I'm not sure it's worth mentioning JSON-LD here, as it can add confusion. I
would just go with JSON.
> + (``core-image-minimal-qemux86-64.rootfs.spdx.json``) is approximately
> + **260 MB without sources** (``SPDX_INCLUDE_SOURCES = "0"``), and increases to
> + **~2.5-2.6 GB when sources are included** (either ``SPDX_INCLUDE_SOURCES = "1"``
> + or ``SPDX_INCLUDE_COMPILED_SOURCES = "1"``). This represents approximately
So SPDX_INCLUDE_COMPILED_SOURCES has the same impact on size compared to
SPDX_INCLUDE_SOURCES? I thought the point of that variable was to optimize
what's included in the file.
> + a **10x size increase** when including source files.
> +
> + Note: SPDX 3.0.1 JSON-LD files are not compressed by default, unlike the
> + tar.zst format used in SPDX 2.2. Compression (e.g. zstd) can significantly
> + reduce the file size.
We could use a ..note :: block here and in the other notes:
"""
.. note::
SPDX 3.0.1 JSON files are not compressed by default, unlike the tar.zst
format used in SPDX 2.2. Compression (e.g. zstd) can significantly reduce the
file size.
"""
Also, how can one enable compression? If you know, it would be worth adding it
the note. Either the way to do it or a link to the variable that controls this.
>
> :term:`SPDX_INCLUDE_SOURCES`
> This option allows to add a description of the source files used to build
> @@ -8979,8 +8990,8 @@ system and gives an overview of their function and contents.
>
> SPDX_INCLUDE_SOURCES = "1"
>
> - According to our tests on release 4.1 "langdale", building
> - ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
> + According to our tests on release 4.1 "langdale" (SPDX 2.2 format),
> + building ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
> this option multiplied the total size of the ``tmp/deploy/spdx``
> directory by a factor of 3 (+291 MiB for this image),
> and the size of the ``IMAGE-MACHINE.spdx.tar.zst`` in
> @@ -8988,6 +8999,42 @@ system and gives an overview of their function and contents.
> image), compared to just using the :ref:`ref-classes-create-spdx` class
> with no option.
>
> + With SPDX 3.0.1 JSON-LD format, the uncompressed image SBOM file
> + (``core-image-minimal-qemux86-64.rootfs.spdx.json``) is approximately
> + **260 MB without sources** (default: ``SPDX_INCLUDE_SOURCES = "0"``),
> + and increases to **~2.5-2.6 GB when sources are included**
> + (``SPDX_INCLUDE_SOURCES = "1"``). This represents approximately a
> + **10x size increase** when including source files.
> +
> + Note: SPDX 3.0.1 JSON-LD files are not compressed by default, unlike the
> + tar.zst format used in SPDX 2.2. Compression (e.g. zstd) can significantly
> + reduce the file size.
> +
> + :term:`SPDX_LICENSES`
> + Path to the JSON file containing SPDX license identifier mappings. This
> + file maps common license names to official SPDX license identifiers used
> + during SBOM generation.
> +
> + The default value is::
> +
> + SPDX_LICENSES = "${COREBASE}/meta/files/spdx-licenses.json"
I would avoid hardcoding a path in the docs, to decrease the maintenance
effort. Perhaps:
"""
The default value points to a copy of the license mappings defined by SPDX
(https://github.com/spdx/license-list-data) stored in :term:`OpenEmbedded-Core
(OE-Core)`.
"""
Same remark for SPDX_MULTILIB_SSTATE_ARCHS and SPDX_UUID_NAMESPACE.
> +
> + You can override this variable to use a custom license mapping file if
> + your organization uses different license naming conventions.
> +
> + :term:`SPDX_MULTILIB_SSTATE_ARCHS`
> + The list of sstate architectures to consider when collecting SPDX
> + dependencies. This includes multilib architectures when multilib is
> + enabled.
> +
> + The default value is::
> +
> + SPDX_MULTILIB_SSTATE_ARCHS = "${SSTATE_ARCHS}"
> +
> + This variable is used internally by the SPDX generation classes to
> + ensure all relevant dependencies are included in the SBOM, regardless
> + of whether multilib is enabled or not.
> +
> :term:`SPDX_NAMESPACE_PREFIX`
> This option could be used in order to change the prefix of ``spdxDocument``
> and the prefix of ``documentNamespace``. It is set by default to
> @@ -9015,6 +9062,19 @@ system and gives an overview of their function and contents.
> this option is recommended if you want to inspect the SPDX
> output files with a text editor.
>
> + :term:`SPDX_UUID_NAMESPACE`
> + The namespace used for generating UUIDs in SPDX documents. This should
> + be a domain name or unique identifier for your organization to ensure
> + globally unique SPDX IDs across different builds and organizations.
> +
> + The default value is::
> +
> + SPDX_UUID_NAMESPACE = "sbom.openembedded.org"
> +
> + You can override this variable to use your organization's domain::
> +
> + SPDX_UUID_NAMESPACE = "sbom.example.com"
> +
> :term:`SPDXLICENSEMAP`
> Maps commonly used license names to their SPDX counterparts found in
> ``meta/files/common-licenses/``. For the default :term:`SPDXLICENSEMAP`
Thanks!
Antonin
--
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 4+ messages in thread
[parent not found: <1877E8B8F84BDDDE.1114802@lists.yoctoproject.org>]
* Re: [docs][PATCH v2] ref-manual: Document SPDX 3.0.1 variables
[not found] ` <1877E8B8F84BDDDE.1114802@lists.yoctoproject.org>
@ 2025-11-24 21:42 ` Stefano Tondo
0 siblings, 0 replies; 4+ messages in thread
From: Stefano Tondo @ 2025-11-24 21:42 UTC (permalink / raw)
To: stondo; +Cc: docs, Stefano Tondo, peter.marko, adrian.freihofer
[-- Attachment #1: Type: text/plain, Size: 8593 bytes --]
Hi Antonin,
Thank you very much for the review.
I've addressed all your points in v3:
1. Langdale testing claim - Moved to SPDX 2.2 context only, removed
untested claims for SPDX 3.0.1
2. JSON-LD terminology - Changed to "JSON" throughout for clarity
3. SPDX_INCLUDE_COMPILED_SOURCES size data - Removed untested claims, added
explanatory note
4. RST formatting - Converted all paragraphs to proper `.. note::` blocks
5. Hardcoded values - Replaced with generic examples and added link to
upstream SPDX license-list-data
I also researched compression and added verified instructions with zstd.
Testing showed 94-97% compression ratios (e.g., 262 MB SBOM → 8.9 MB),
which should be very helpful for users dealing with large SPDX 3.0.1 files.
I'll send v3 shortly.
Best regards,
Stefano
On Fri, Nov 14, 2025 at 4:21 PM Stefano Tondo via lists.yoctoproject.org
<stondo=gmail.com@lists.yoctoproject.org> wrote:
> From: Stefano Tondo <stefano.tondo.ext@siemens.com>
>
> Add comprehensive documentation for SPDX-related variables in the
> Yocto reference manual. This includes documenting previously
> undocumented variables and updating existing documentation with
> SPDX 3.0.1 specific information.
>
> New variables documented:
> - SPDX_LICENSES: Path to SPDX license identifier mapping file
> - SPDX_MULTILIB_SSTATE_ARCHS: Architecture list for dependency collection
> - SPDX_UUID_NAMESPACE: Namespace for UUID generation in SPDX documents
>
> Updated existing variables with SPDX 3.0.1 data:
> - SPDX_INCLUDE_SOURCES: Added concrete SPDX 3.0.1 size data
> (~260 MB without sources, ~2.5-2.6 GB with sources)
> - SPDX_INCLUDE_COMPILED_SOURCES: Updated with SPDX 3.0.1 format information
>
> The size information is based on building core-image-minimal for
> qemux86-64 with SPDX 3.0.1 JSON-LD format. The uncompressed SBOM
> file is approximately 260 MB without sources and increases to
> ~2.5-2.6 GB when sources are included (approximately 10x increase).
>
> Note: SPDX 3.0.1 uses uncompressed JSON-LD format by default,
> unlike the tar.zst compression used in SPDX 2.2.
>
> These variables are defined in meta/classes/spdx-common.bbclass.
>
> Signed-off-by: Stefano Tondo <stefano.tondo.ext@siemens.com>
> ---
>
> Changes in v2:
> - Fixed unintended removal of non-SPDX variables (CCACHE_DISABLE,
> IMAGE_EXTRA_PARTITION_FILES, REQUIRED_*_FEATURES)
> - Fixed unintended change to LAYERDEPENDS description
> - Only SPDX-related variables are now modified
>
> documentation/ref-manual/variables.rst | 72 +++++++++++++++++++++++---
> 1 file changed, 66 insertions(+), 6 deletions(-)
>
> diff --git a/documentation/ref-manual/variables.rst
> b/documentation/ref-manual/variables.rst
> index a80ef364e..c098227f2 100644
> --- a/documentation/ref-manual/variables.rst
> +++ b/documentation/ref-manual/variables.rst
> @@ -8962,10 +8962,21 @@ system and gives an overview of their function and
> contents.
>
> SPDX_INCLUDE_COMPILED_SOURCES = "1"
>
> - According to our tests, building ``core-image-minimal`` for the
> - ``qemux86-64`` machine, enabling this option compared with the
> - :term:`SPDX_INCLUDE_SOURCES` reduces the size of the
> ``tmp/deploy/spdx``
> - directory from 2GB to 1.6GB.
> + According to our tests on release 4.1 "langdale" (SPDX 2.2 format),
> building
> + ``core-image-minimal`` for the ``qemux86-64`` machine, enabling this
> + option compared with the :term:`SPDX_INCLUDE_SOURCES` reduced the
> size
> + of the ``tmp/deploy/spdx`` directory from 2GB to 1.6GB.
> +
> + With SPDX 3.0.1 JSON-LD format, the uncompressed image SBOM file
> + (``core-image-minimal-qemux86-64.rootfs.spdx.json``) is
> approximately
> + **260 MB without sources** (``SPDX_INCLUDE_SOURCES = "0"``), and
> increases to
> + **~2.5-2.6 GB when sources are included** (either
> ``SPDX_INCLUDE_SOURCES = "1"``
> + or ``SPDX_INCLUDE_COMPILED_SOURCES = "1"``). This represents
> approximately
> + a **10x size increase** when including source files.
> +
> + Note: SPDX 3.0.1 JSON-LD files are not compressed by default,
> unlike the
> + tar.zst format used in SPDX 2.2. Compression (e.g. zstd) can
> significantly
> + reduce the file size.
>
> :term:`SPDX_INCLUDE_SOURCES`
> This option allows to add a description of the source files used to
> build
> @@ -8979,8 +8990,8 @@ system and gives an overview of their function and
> contents.
>
> SPDX_INCLUDE_SOURCES = "1"
>
> - According to our tests on release 4.1 "langdale", building
> - ``core-image-minimal`` for the ``qemux86-64`` machine, enabling
> + According to our tests on release 4.1 "langdale" (SPDX 2.2 format),
> + building ``core-image-minimal`` for the ``qemux86-64`` machine,
> enabling
> this option multiplied the total size of the ``tmp/deploy/spdx``
> directory by a factor of 3 (+291 MiB for this image),
> and the size of the ``IMAGE-MACHINE.spdx.tar.zst`` in
> @@ -8988,6 +8999,42 @@ system and gives an overview of their function and
> contents.
> image), compared to just using the :ref:`ref-classes-create-spdx`
> class
> with no option.
>
> + With SPDX 3.0.1 JSON-LD format, the uncompressed image SBOM file
> + (``core-image-minimal-qemux86-64.rootfs.spdx.json``) is
> approximately
> + **260 MB without sources** (default: ``SPDX_INCLUDE_SOURCES =
> "0"``),
> + and increases to **~2.5-2.6 GB when sources are included**
> + (``SPDX_INCLUDE_SOURCES = "1"``). This represents approximately a
> + **10x size increase** when including source files.
> +
> + Note: SPDX 3.0.1 JSON-LD files are not compressed by default,
> unlike the
> + tar.zst format used in SPDX 2.2. Compression (e.g. zstd) can
> significantly
> + reduce the file size.
> +
> + :term:`SPDX_LICENSES`
> + Path to the JSON file containing SPDX license identifier mappings.
> This
> + file maps common license names to official SPDX license identifiers
> used
> + during SBOM generation.
> +
> + The default value is::
> +
> + SPDX_LICENSES = "${COREBASE}/meta/files/spdx-licenses.json"
> +
> + You can override this variable to use a custom license mapping file
> if
> + your organization uses different license naming conventions.
> +
> + :term:`SPDX_MULTILIB_SSTATE_ARCHS`
> + The list of sstate architectures to consider when collecting SPDX
> + dependencies. This includes multilib architectures when multilib is
> + enabled.
> +
> + The default value is::
> +
> + SPDX_MULTILIB_SSTATE_ARCHS = "${SSTATE_ARCHS}"
> +
> + This variable is used internally by the SPDX generation classes to
> + ensure all relevant dependencies are included in the SBOM,
> regardless
> + of whether multilib is enabled or not.
> +
> :term:`SPDX_NAMESPACE_PREFIX`
> This option could be used in order to change the prefix of
> ``spdxDocument``
> and the prefix of ``documentNamespace``. It is set by default to
> @@ -9015,6 +9062,19 @@ system and gives an overview of their function and
> contents.
> this option is recommended if you want to inspect the SPDX
> output files with a text editor.
>
> + :term:`SPDX_UUID_NAMESPACE`
> + The namespace used for generating UUIDs in SPDX documents. This
> should
> + be a domain name or unique identifier for your organization to
> ensure
> + globally unique SPDX IDs across different builds and organizations.
> +
> + The default value is::
> +
> + SPDX_UUID_NAMESPACE = "sbom.openembedded.org"
> +
> + You can override this variable to use your organization's domain::
> +
> + SPDX_UUID_NAMESPACE = "sbom.example.com"
> +
> :term:`SPDXLICENSEMAP`
> Maps commonly used license names to their SPDX counterparts found in
> ``meta/files/common-licenses/``. For the default
> :term:`SPDXLICENSEMAP`
> --
> 2.51.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#8044):
> https://lists.yoctoproject.org/g/docs/message/8044
> Mute This Topic: https://lists.yoctoproject.org/mt/116292422/10120103
> Group Owner: docs+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [stondo@gmail.com
> ]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
[-- Attachment #2: Type: text/html, Size: 10342 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2025-11-24 21:43 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-11-14 14:52 [docs] ref-manual: Document SPDX 3.0.1 variables Stefano Tondo
2025-11-14 15:12 ` [docs][PATCH v2] " Stefano Tondo
2025-11-24 15:48 ` Antonin Godard
[not found] ` <1877E8B8F84BDDDE.1114802@lists.yoctoproject.org>
2025-11-24 21:42 ` Stefano Tondo
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox