[OE-core][PATCH 0/2] spdx3: support SBOM compression with Zstd

[OE-core][PATCH 0/2] spdx3: support SBOM compression with Zstd

[Jérémie Dautheribes (Schneider Electric )]

Hi,

This patch series adds support for compressing all types of SBOMs (image,
recipe, SDK) using zstd, similar to what we had previously with SPDX 2.2.

To do so, we introduce a new SPDX_SBOM_EXT variable containing the SBOM
extension name. Based on this extension, we decide whether SBOMs should be
compressed or not.

This is optional and by default SBOMs are not compressed to keep the
current behavior and not to break compatibility.

This work was tested on the qemuarm64 machine on the following SBOMs:
  - core-image-minimal SBOM (image SBOM)
  - busybox SBOM (recipe SBOM)
  - core-image-minimal SDK SBOM (SDK SBOM)

At first, instead of SPDX_SBOM_EXT, I used a boolean SPDX_COMPRESSED_SBOM
variable to decide whether or not a SBOM should be compressed, but it led
to a lot of code additions to SBOM consumers (for instance sbom-cve-check)
to check whether the SBOM filename extension was ".spdx.json" or
".spdx.json.zst".

Signed-off-by: Jérémie Dautheribes (Schneider Electric) <jeremie.dautheribes@bootlin.com>
---
Jérémie Dautheribes (Schneider Electric) (2):
      spdx3: introduce SPDX_SBOM_EXT variable
      spdx3: support SBOM compression based on SPDX_SBOM_EXT

 meta/classes-recipe/sbom-cve-check.bbclass |  2 +-
 meta/classes/create-spdx-3.0.bbclass       |  4 ++++
 meta/classes/sbom-cve-check-recipe.bbclass |  2 +-
 meta/lib/oe/sbom30.py                      | 11 +++++++++--
 meta/lib/oe/spdx30_tasks.py                | 12 +++++++-----
 5 files changed, 22 insertions(+), 9 deletions(-)
---
base-commit: 4f7d1a0885d7d6f2a533f7388ed5f5a35d6f99bc
change-id: 20260512-sbom-zstd-support-7bd9b13881e2

Best regards,
--  
Jérémie Dautheribes (Schneider Electric) <jeremie.dautheribes@bootlin.com>

[OE-core][PATCH 1/2] spdx3: introduce SPDX_SBOM_EXT variable

[Jérémie Dautheribes (Schneider Electric )]

In preparation for upcoming work, introduce a new SPDX_SBOM_EXT variable
explicitly telling the file extension name for SBOMs.

Keep the default value ".spdx.json" to maintain compatibility with the
current behavior.

Co-authored-by: Benjamin Robin (Schneider Electric) <benjamin.robin@bootlin.com>
Signed-off-by: Jérémie Dautheribes (Schneider Electric) <jeremie.dautheribes@bootlin.com>
---
 meta/classes-recipe/sbom-cve-check.bbclass |  2 +-
 meta/classes/create-spdx-3.0.bbclass       |  3 +++
 meta/classes/sbom-cve-check-recipe.bbclass |  2 +-
 meta/lib/oe/spdx30_tasks.py                | 12 +++++++-----
 4 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/meta/classes-recipe/sbom-cve-check.bbclass b/meta/classes-recipe/sbom-cve-check.bbclass
index fe145a2212..ddecb82e52 100644
--- a/meta/classes-recipe/sbom-cve-check.bbclass
+++ b/meta/classes-recipe/sbom-cve-check.bbclass
@@ -14,7 +14,7 @@ python do_sbom_cve_check() {
     """
     Task: Run sbom-cve-check analysis on SBOM.
     """
-    sbom_path = d.expand("${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.spdx.json")
+    sbom_path = d.expand("${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}${SPDX_SBOM_EXT}")
     image_name = d.getVar("IMAGE_NAME")
     link_name = d.getVar("IMAGE_LINK_NAME")
     run_sbom_cve_check(d, sbom_path, image_name, link_name)
diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass
index 56fd01fd53..785edb9865 100644
--- a/meta/classes/create-spdx-3.0.bbclass
+++ b/meta/classes/create-spdx-3.0.bbclass
@@ -74,6 +74,9 @@ SPDX_IMPORTS[doc] = "SPDX_IMPORTS is the base variable that describes how to \
             algorithms, as described by the HashAlgorithm vocabulary in the\
             SPDX 3 spec. Optional but recommended"
 
+SPDX_SBOM_EXT ??= ".spdx.json"
+SPDX_SBOM_EXT[doc] = "SBOM file extension name."
+
 # Agents
 #   Bitbake variables can be used to describe an SPDX Agent that may be used
 #   during the build. An Agent is specified using a set of variables which all
diff --git a/meta/classes/sbom-cve-check-recipe.bbclass b/meta/classes/sbom-cve-check-recipe.bbclass
index c80b8ac83f..eaad73ddaf 100644
--- a/meta/classes/sbom-cve-check-recipe.bbclass
+++ b/meta/classes/sbom-cve-check-recipe.bbclass
@@ -16,7 +16,7 @@ python do_sbom_cve_check_recipe() {
     """
     Task: Run sbom-cve-check analysis on a recipe SBOM.
     """
-    sbom_path = d.expand("${DEPLOY_DIR_IMAGE}/${SPDX_RECIPE_SBOM_NAME}.spdx.json")
+    sbom_path = d.expand("${DEPLOY_DIR_IMAGE}/${SPDX_RECIPE_SBOM_NAME}${SPDX_SBOM_EXT}")
     recipe = d.getVar("SPDX_RECIPE_SBOM_NAME")
     run_sbom_cve_check(d, sbom_path, recipe)
 }
diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py
index 1821dd7de4..63d93c7901 100644
--- a/meta/lib/oe/spdx30_tasks.py
+++ b/meta/lib/oe/spdx30_tasks.py
@@ -1526,8 +1526,9 @@ def create_image_sbom_spdx(d):
     image_link_name = d.getVar("IMAGE_LINK_NAME")
     imgdeploydir = Path(d.getVar("SPDXIMAGEDEPLOYDIR"))
     machine = d.getVar("MACHINE")
+    sbom_ext = d.getVar("SPDX_SBOM_EXT")
 
-    spdx_path = imgdeploydir / (image_name + ".spdx.json")
+    spdx_path = imgdeploydir / f"{image_name}{sbom_ext}"
 
     root_elements = []
 
@@ -1567,7 +1568,7 @@ def create_image_sbom_spdx(d):
             if link != target_path:
                 link.symlink_to(os.path.relpath(target_path, link.parent))
 
-    make_image_link(spdx_path, ".spdx.json")
+    make_image_link(spdx_path, sbom_ext)
 
 
 def sdk_create_spdx(d, sdk_type, spdx_work_dir, toolchain_outputname):
@@ -1603,6 +1604,7 @@ def sdk_create_spdx(d, sdk_type, spdx_work_dir, toolchain_outputname):
 
 
 def create_sdk_sbom(d, sdk_deploydir, spdx_work_dir, toolchain_outputname):
+    sbom_ext = d.getVar("SPDX_SBOM_EXT")
     # Load the document written earlier
     rootfs_objset = oe.sbom30.load_jsonld(
         d, spdx_work_dir / "sdk-rootfs.spdx.json", required=True
@@ -1681,15 +1683,15 @@ def create_sdk_sbom(d, sdk_deploydir, spdx_work_dir, toolchain_outputname):
                 elem.suppliedBy = supplier_id
 
     oe.sbom30.write_jsonld_doc(
-        d, objset, sdk_deploydir / (toolchain_outputname + ".spdx.json")
+        d, objset, sdk_deploydir / f"{toolchain_outputname}{sbom_ext}"
     )
 
 
 def create_recipe_sbom(d, deploydir):
     sbom_name = d.getVar("SPDX_RECIPE_SBOM_NAME")
-
+    sbom_ext = d.getVar("SPDX_SBOM_EXT")
     recipe, recipe_objset = load_recipe_spdx(d)
 
     objset, sbom = oe.sbom30.create_sbom(d, sbom_name, [recipe], [recipe_objset])
 
-    oe.sbom30.write_jsonld_doc(d, objset, deploydir / (sbom_name + ".spdx.json"))
+    oe.sbom30.write_jsonld_doc(d, objset, deploydir / f"{sbom_name}{sbom_ext}")

-- 
2.54.0

[OE-core][PATCH 2/2] spdx3: support SBOM compression based on SPDX_SBOM_EXT

[Jérémie Dautheribes (Schneider Electric )]

Add support for optional zstd compression for all types of SBOMs,
including:
  - image SBOM
  - recipe SBOM
  - SDK SBOM

Zstd compression is applied if SPDX_SBOM_EXT ends with ".zst".

Co-authored-by: Benjamin Robin (Schneider Electric) <benjamin.robin@bootlin.com>
Signed-off-by: Jérémie Dautheribes (Schneider Electric) <jeremie.dautheribes@bootlin.com>
---
 meta/classes/create-spdx-3.0.bbclass |  3 ++-
 meta/lib/oe/sbom30.py                | 11 +++++++++--
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass
index 785edb9865..6cf8fa4688 100644
--- a/meta/classes/create-spdx-3.0.bbclass
+++ b/meta/classes/create-spdx-3.0.bbclass
@@ -75,7 +75,8 @@ SPDX_IMPORTS[doc] = "SPDX_IMPORTS is the base variable that describes how to \
             SPDX 3 spec. Optional but recommended"
 
 SPDX_SBOM_EXT ??= ".spdx.json"
-SPDX_SBOM_EXT[doc] = "SBOM file extension name."
+SPDX_SBOM_EXT[doc] = "SBOM file extension name.\
+    If it ends with '.zst', SBOMs are automatically compressed using Zstd."
 
 # Agents
 #   Bitbake variables can be used to describe an SPDX Agent that may be used
diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py
index 0f1f9281ad..2184c1a07f 100644
--- a/meta/lib/oe/sbom30.py
+++ b/meta/lib/oe/sbom30.py
@@ -1036,8 +1036,15 @@ def write_jsonld_doc(d, objset, dest):
         serializer = oe.spdx30.JSONLDInlineSerializer()
 
     objset.objects.add(objset.doc)
-    with dest.open("wb") as f:
-        serializer.write(objset, f, force_at_graph=True)
+
+    if dest.name.endswith(".zst"):
+        num_threads = int(d.getVar("BB_NUMBER_THREADS"))
+        with bb.compress.zstd.open(dest, "w", num_threads=num_threads) as f:
+            serializer.write(objset, f, force_at_graph=True)
+    else:
+        with dest.open("wb") as f:
+            serializer.write(objset, f, force_at_graph=True)
+
     objset.objects.remove(objset.doc)
 
 

-- 
2.54.0

Re: [OE-core][PATCH 2/2] spdx3: support SBOM compression based on SPDX_SBOM_EXT

[Richard Purdie]

On Tue, 2026-05-12 at 19:01 +0200, Jérémie Dautheribes via lists.openembedded.org wrote:
> 
>      objset.objects.add(objset.doc)
> -    with dest.open("wb") as f:
> -        serializer.write(objset, f, force_at_graph=True)
> +
> +    if dest.name.endswith(".zst"):
> +        num_threads = int(d.getVar("BB_NUMBER_THREADS"))
> +        with bb.compress.zstd.open(dest, "w", num_threads=num_threads) as f:

This should be derived from PARALLEL_MAKE, not BB_NUMBER_THREADS.

The latter is how many tasks bitbake runs in parallel, the former is
how many threads the task should have.

There is a function somewhere which extracts the number of jobs from
PARALLEL_MAKE...

Cheers,

Richard

Re: [OE-core][PATCH 1/2] spdx3: introduce SPDX_SBOM_EXT variable

[Joshua Watt]

On Tue, May 12, 2026 at 11:02 AM Jérémie Dautheribes via
lists.openembedded.org
<jeremie.dautheribes=bootlin.com@lists.openembedded.org> wrote:
>
> In preparation for upcoming work, introduce a new SPDX_SBOM_EXT variable
> explicitly telling the file extension name for SBOMs.
>
> Keep the default value ".spdx.json" to maintain compatibility with the
> current behavior.
>
> Co-authored-by: Benjamin Robin (Schneider Electric) <benjamin.robin@bootlin.com>
> Signed-off-by: Jérémie Dautheribes (Schneider Electric) <jeremie.dautheribes@bootlin.com>
> ---
>  meta/classes-recipe/sbom-cve-check.bbclass |  2 +-
>  meta/classes/create-spdx-3.0.bbclass       |  3 +++
>  meta/classes/sbom-cve-check-recipe.bbclass |  2 +-
>  meta/lib/oe/spdx30_tasks.py                | 12 +++++++-----
>  4 files changed, 12 insertions(+), 7 deletions(-)
>
> diff --git a/meta/classes-recipe/sbom-cve-check.bbclass b/meta/classes-recipe/sbom-cve-check.bbclass
> index fe145a2212..ddecb82e52 100644
> --- a/meta/classes-recipe/sbom-cve-check.bbclass
> +++ b/meta/classes-recipe/sbom-cve-check.bbclass
> @@ -14,7 +14,7 @@ python do_sbom_cve_check() {
>      """
>      Task: Run sbom-cve-check analysis on SBOM.
>      """
> -    sbom_path = d.expand("${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.spdx.json")
> +    sbom_path = d.expand("${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}${SPDX_SBOM_EXT}")
>      image_name = d.getVar("IMAGE_NAME")
>      link_name = d.getVar("IMAGE_LINK_NAME")
>      run_sbom_cve_check(d, sbom_path, image_name, link_name)
> diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass
> index 56fd01fd53..785edb9865 100644
> --- a/meta/classes/create-spdx-3.0.bbclass
> +++ b/meta/classes/create-spdx-3.0.bbclass
> @@ -74,6 +74,9 @@ SPDX_IMPORTS[doc] = "SPDX_IMPORTS is the base variable that describes how to \
>              algorithms, as described by the HashAlgorithm vocabulary in the\
>              SPDX 3 spec. Optional but recommended"
>
> +SPDX_SBOM_EXT ??= ".spdx.json"

We should perhaps consider making this SPDX_SBOM_EXT_SUFFIX instead;
.spdx.json is the ISO standard extension for SPDX documents and is
non-optional.

> +SPDX_SBOM_EXT[doc] = "SBOM file extension name."
> +
>  # Agents
>  #   Bitbake variables can be used to describe an SPDX Agent that may be used
>  #   during the build. An Agent is specified using a set of variables which all
> diff --git a/meta/classes/sbom-cve-check-recipe.bbclass b/meta/classes/sbom-cve-check-recipe.bbclass
> index c80b8ac83f..eaad73ddaf 100644
> --- a/meta/classes/sbom-cve-check-recipe.bbclass
> +++ b/meta/classes/sbom-cve-check-recipe.bbclass
> @@ -16,7 +16,7 @@ python do_sbom_cve_check_recipe() {
>      """
>      Task: Run sbom-cve-check analysis on a recipe SBOM.
>      """
> -    sbom_path = d.expand("${DEPLOY_DIR_IMAGE}/${SPDX_RECIPE_SBOM_NAME}.spdx.json")
> +    sbom_path = d.expand("${DEPLOY_DIR_IMAGE}/${SPDX_RECIPE_SBOM_NAME}${SPDX_SBOM_EXT}")
>      recipe = d.getVar("SPDX_RECIPE_SBOM_NAME")
>      run_sbom_cve_check(d, sbom_path, recipe)
>  }
> diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py
> index 1821dd7de4..63d93c7901 100644
> --- a/meta/lib/oe/spdx30_tasks.py
> +++ b/meta/lib/oe/spdx30_tasks.py
> @@ -1526,8 +1526,9 @@ def create_image_sbom_spdx(d):
>      image_link_name = d.getVar("IMAGE_LINK_NAME")
>      imgdeploydir = Path(d.getVar("SPDXIMAGEDEPLOYDIR"))
>      machine = d.getVar("MACHINE")
> +    sbom_ext = d.getVar("SPDX_SBOM_EXT")
>
> -    spdx_path = imgdeploydir / (image_name + ".spdx.json")
> +    spdx_path = imgdeploydir / f"{image_name}{sbom_ext}"
>
>      root_elements = []
>
> @@ -1567,7 +1568,7 @@ def create_image_sbom_spdx(d):
>              if link != target_path:
>                  link.symlink_to(os.path.relpath(target_path, link.parent))
>
> -    make_image_link(spdx_path, ".spdx.json")
> +    make_image_link(spdx_path, sbom_ext)
>
>
>  def sdk_create_spdx(d, sdk_type, spdx_work_dir, toolchain_outputname):
> @@ -1603,6 +1604,7 @@ def sdk_create_spdx(d, sdk_type, spdx_work_dir, toolchain_outputname):
>
>
>  def create_sdk_sbom(d, sdk_deploydir, spdx_work_dir, toolchain_outputname):
> +    sbom_ext = d.getVar("SPDX_SBOM_EXT")
>      # Load the document written earlier
>      rootfs_objset = oe.sbom30.load_jsonld(
>          d, spdx_work_dir / "sdk-rootfs.spdx.json", required=True
> @@ -1681,15 +1683,15 @@ def create_sdk_sbom(d, sdk_deploydir, spdx_work_dir, toolchain_outputname):
>                  elem.suppliedBy = supplier_id
>
>      oe.sbom30.write_jsonld_doc(
> -        d, objset, sdk_deploydir / (toolchain_outputname + ".spdx.json")
> +        d, objset, sdk_deploydir / f"{toolchain_outputname}{sbom_ext}"
>      )
>
>
>  def create_recipe_sbom(d, deploydir):
>      sbom_name = d.getVar("SPDX_RECIPE_SBOM_NAME")
> -
> +    sbom_ext = d.getVar("SPDX_SBOM_EXT")
>      recipe, recipe_objset = load_recipe_spdx(d)
>
>      objset, sbom = oe.sbom30.create_sbom(d, sbom_name, [recipe], [recipe_objset])
>
> -    oe.sbom30.write_jsonld_doc(d, objset, deploydir / (sbom_name + ".spdx.json"))
> +    oe.sbom30.write_jsonld_doc(d, objset, deploydir / f"{sbom_name}{sbom_ext}")
>
> --
> 2.54.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#236896): https://lists.openembedded.org/g/openembedded-core/message/236896
> Mute This Topic: https://lists.openembedded.org/mt/119282963/3616693
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [JPEWhacker@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core][PATCH 2/2] spdx3: support SBOM compression based on SPDX_SBOM_EXT

[Joshua Watt]

On Tue, May 12, 2026 at 11:02 AM Jérémie Dautheribes via
lists.openembedded.org
<jeremie.dautheribes=bootlin.com@lists.openembedded.org> wrote:
>
> Add support for optional zstd compression for all types of SBOMs,
> including:
>   - image SBOM
>   - recipe SBOM
>   - SDK SBOM
>
> Zstd compression is applied if SPDX_SBOM_EXT ends with ".zst".
>
> Co-authored-by: Benjamin Robin (Schneider Electric) <benjamin.robin@bootlin.com>
> Signed-off-by: Jérémie Dautheribes (Schneider Electric) <jeremie.dautheribes@bootlin.com>
> ---
>  meta/classes/create-spdx-3.0.bbclass |  3 ++-
>  meta/lib/oe/sbom30.py                | 11 +++++++++--
>  2 files changed, 11 insertions(+), 3 deletions(-)
>
> diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass
> index 785edb9865..6cf8fa4688 100644
> --- a/meta/classes/create-spdx-3.0.bbclass
> +++ b/meta/classes/create-spdx-3.0.bbclass
> @@ -75,7 +75,8 @@ SPDX_IMPORTS[doc] = "SPDX_IMPORTS is the base variable that describes how to \
>              SPDX 3 spec. Optional but recommended"
>
>  SPDX_SBOM_EXT ??= ".spdx.json"
> -SPDX_SBOM_EXT[doc] = "SBOM file extension name."
> +SPDX_SBOM_EXT[doc] = "SBOM file extension name.\
> +    If it ends with '.zst', SBOMs are automatically compressed using Zstd."
>
>  # Agents
>  #   Bitbake variables can be used to describe an SPDX Agent that may be used
> diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py
> index 0f1f9281ad..2184c1a07f 100644
> --- a/meta/lib/oe/sbom30.py
> +++ b/meta/lib/oe/sbom30.py
> @@ -1036,8 +1036,15 @@ def write_jsonld_doc(d, objset, dest):
>          serializer = oe.spdx30.JSONLDInlineSerializer()
>
>      objset.objects.add(objset.doc)
> -    with dest.open("wb") as f:
> -        serializer.write(objset, f, force_at_graph=True)
> +
> +    if dest.name.endswith(".zst"):

I'm not sure I like this detection mechanism; I think we usually do
something more explicit for compression rather than relying on the
suffix in other places?

> +        num_threads = int(d.getVar("BB_NUMBER_THREADS"))

The API is oe.utils.parallel_make_argument()

> +        with bb.compress.zstd.open(dest, "w", num_threads=num_threads) as f:
> +            serializer.write(objset, f, force_at_graph=True)
> +    else:
> +        with dest.open("wb") as f:
> +            serializer.write(objset, f, force_at_graph=True)
> +
>      objset.objects.remove(objset.doc)
>
>
>
> --
> 2.54.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#236897): https://lists.openembedded.org/g/openembedded-core/message/236897
> Mute This Topic: https://lists.openembedded.org/mt/119282964/3616693
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [JPEWhacker@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core][PATCH 2/2] spdx3: support SBOM compression based on SPDX_SBOM_EXT

[Joshua Watt]

On Tue, May 12, 2026 at 4:27 PM Joshua Watt <jpewhacker@gmail.com> wrote:
>
> On Tue, May 12, 2026 at 11:02 AM Jérémie Dautheribes via
> lists.openembedded.org
> <jeremie.dautheribes=bootlin.com@lists.openembedded.org> wrote:
> >
> > Add support for optional zstd compression for all types of SBOMs,
> > including:
> >   - image SBOM
> >   - recipe SBOM
> >   - SDK SBOM

We should perhaps also implement decompression when reading in
documents, so that the intermediate documents are compressed as well;
if we are allowing the final documents to be compressed, I don't see a
compelling reason why we wouldn't just compress all of them.

> >
> > Zstd compression is applied if SPDX_SBOM_EXT ends with ".zst".
> >
> > Co-authored-by: Benjamin Robin (Schneider Electric) <benjamin.robin@bootlin.com>
> > Signed-off-by: Jérémie Dautheribes (Schneider Electric) <jeremie.dautheribes@bootlin.com>
> > ---
> >  meta/classes/create-spdx-3.0.bbclass |  3 ++-
> >  meta/lib/oe/sbom30.py                | 11 +++++++++--
> >  2 files changed, 11 insertions(+), 3 deletions(-)
> >
> > diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass
> > index 785edb9865..6cf8fa4688 100644
> > --- a/meta/classes/create-spdx-3.0.bbclass
> > +++ b/meta/classes/create-spdx-3.0.bbclass
> > @@ -75,7 +75,8 @@ SPDX_IMPORTS[doc] = "SPDX_IMPORTS is the base variable that describes how to \
> >              SPDX 3 spec. Optional but recommended"
> >
> >  SPDX_SBOM_EXT ??= ".spdx.json"
> > -SPDX_SBOM_EXT[doc] = "SBOM file extension name."
> > +SPDX_SBOM_EXT[doc] = "SBOM file extension name.\
> > +    If it ends with '.zst', SBOMs are automatically compressed using Zstd."
> >
> >  # Agents
> >  #   Bitbake variables can be used to describe an SPDX Agent that may be used
> > diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py
> > index 0f1f9281ad..2184c1a07f 100644
> > --- a/meta/lib/oe/sbom30.py
> > +++ b/meta/lib/oe/sbom30.py
> > @@ -1036,8 +1036,15 @@ def write_jsonld_doc(d, objset, dest):
> >          serializer = oe.spdx30.JSONLDInlineSerializer()
> >
> >      objset.objects.add(objset.doc)
> > -    with dest.open("wb") as f:
> > -        serializer.write(objset, f, force_at_graph=True)
> > +
> > +    if dest.name.endswith(".zst"):
>
> I'm not sure I like this detection mechanism; I think we usually do
> something more explicit for compression rather than relying on the
> suffix in other places?
>
> > +        num_threads = int(d.getVar("BB_NUMBER_THREADS"))
>
> The API is oe.utils.parallel_make_argument()
>
> > +        with bb.compress.zstd.open(dest, "w", num_threads=num_threads) as f:
> > +            serializer.write(objset, f, force_at_graph=True)
> > +    else:
> > +        with dest.open("wb") as f:
> > +            serializer.write(objset, f, force_at_graph=True)
> > +
> >      objset.objects.remove(objset.doc)
> >
> >
> >
> > --
> > 2.54.0
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#236897): https://lists.openembedded.org/g/openembedded-core/message/236897
> > Mute This Topic: https://lists.openembedded.org/mt/119282964/3616693
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [JPEWhacker@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >

Re: [OE-core][PATCH 2/2] spdx3: support SBOM compression based on SPDX_SBOM_EXT

[Benjamin Robin]

Hello Joshua,

On Wednesday, May 13, 2026 at 12:29 AM, Joshua Watt wrote:
> On Tue, May 12, 2026 at 4:27 PM Joshua Watt <jpewhacker@gmail.com> wrote:
> >
> > On Tue, May 12, 2026 at 11:02 AM Jérémie Dautheribes via
> > lists.openembedded.org
> > <jeremie.dautheribes=bootlin.com@lists.openembedded.org> wrote:
> > >
> > > Add support for optional zstd compression for all types of SBOMs,
> > > including:
> > >   - image SBOM
> > >   - recipe SBOM
> > >   - SDK SBOM
> 
> We should perhaps also implement decompression when reading in
> documents, so that the intermediate documents are compressed as well;
> if we are allowing the final documents to be compressed, I don't see a
> compelling reason why we wouldn't just compress all of them.

I am not sure this is a good idea performance-wise, mainly because
Yocto is currently relying on an external program to compress and
decompress. We need to wait for Python 3.14 to be the minimum required
Python version to be able to use the native implementation of zstd.
Indeed intermediate documents are pretty "small".

Also with SPDX2, intermediate documents were not compressed.

The goal is not to reduce the size of the build directory, but only
the size of deployed artifacts.


-- 
Benjamin Robin, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core][PATCH 2/2] spdx3: support SBOM compression based on SPDX_SBOM_EXT

[Benjamin Robin]

On Wednesday, May 13, 2026 at 12:27 AM, Joshua Watt wrote:
> On Tue, May 12, 2026 at 11:02 AM Jérémie Dautheribes via
> lists.openembedded.org
> <jeremie.dautheribes=bootlin.com@lists.openembedded.org> wrote:
> >
> > Add support for optional zstd compression for all types of SBOMs,
> > including:
> >   - image SBOM
> >   - recipe SBOM
> >   - SDK SBOM
> >
> > Zstd compression is applied if SPDX_SBOM_EXT ends with ".zst".
> >
> > Co-authored-by: Benjamin Robin (Schneider Electric) <benjamin.robin@bootlin.com>
> > Signed-off-by: Jérémie Dautheribes (Schneider Electric) <jeremie.dautheribes@bootlin.com>
> > ---
> >  meta/classes/create-spdx-3.0.bbclass |  3 ++-
> >  meta/lib/oe/sbom30.py                | 11 +++++++++--
> >  2 files changed, 11 insertions(+), 3 deletions(-)
> >
> > diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass
> > index 785edb9865..6cf8fa4688 100644
> > --- a/meta/classes/create-spdx-3.0.bbclass
> > +++ b/meta/classes/create-spdx-3.0.bbclass
> > @@ -75,7 +75,8 @@ SPDX_IMPORTS[doc] = "SPDX_IMPORTS is the base variable that describes how to \
> >              SPDX 3 spec. Optional but recommended"
> >
> >  SPDX_SBOM_EXT ??= ".spdx.json"
> > -SPDX_SBOM_EXT[doc] = "SBOM file extension name."
> > +SPDX_SBOM_EXT[doc] = "SBOM file extension name.\
> > +    If it ends with '.zst', SBOMs are automatically compressed using Zstd."
> >
> >  # Agents
> >  #   Bitbake variables can be used to describe an SPDX Agent that may be used
> > diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py
> > index 0f1f9281ad..2184c1a07f 100644
> > --- a/meta/lib/oe/sbom30.py
> > +++ b/meta/lib/oe/sbom30.py
> > @@ -1036,8 +1036,15 @@ def write_jsonld_doc(d, objset, dest):
> >          serializer = oe.spdx30.JSONLDInlineSerializer()
> >
> >      objset.objects.add(objset.doc)
> > -    with dest.open("wb") as f:
> > -        serializer.write(objset, f, force_at_graph=True)
> > +
> > +    if dest.name.endswith(".zst"):
> 
> I'm not sure I like this detection mechanism; I think we usually do
> something more explicit for compression rather than relying on the
> suffix in other places?

Do you have an example somewhere in the code base?
I am not opposed to use a variable like `SPDX_COMPRESSED_SBOM`
and to have the following code "duplicated" (or create a function for it):

sbom_file_extension = ".spdx.json.zst" if compressed_sbom else ".spdx.json"

The goal was to simplify the code, and to allow user flexibility.
The user could choose any other extension (even if it violate the
ISO standard extension for SPDX documents).

> 
> > +        num_threads = int(d.getVar("BB_NUMBER_THREADS"))
> 
> The API is oe.utils.parallel_make_argument()

Thanks, but for information all code instance calling
`bb.compress.zstd.open` use the BB_NUMBER_THREADS variable :)

So maybe this should be fixed by another patch?

> 
> > +        with bb.compress.zstd.open(dest, "w", num_threads=num_threads) as f:
> > +            serializer.write(objset, f, force_at_graph=True)
> > +    else:
> > +        with dest.open("wb") as f:
> > +            serializer.write(objset, f, force_at_graph=True)
> > +
> >      objset.objects.remove(objset.doc)
> >
> >
> >
> > --
> > 2.54.0
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#236897): https://lists.openembedded.org/g/openembedded-core/message/236897
> > Mute This Topic: https://lists.openembedded.org/mt/119282964/3616693
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [JPEWhacker@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
> 


-- 
Benjamin Robin, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core][PATCH 2/2] spdx3: support SBOM compression based on SPDX_SBOM_EXT

[Jérémie Dautheribes]

Hello Joshua, Benjamin,

On 13/05/2026 09:07, Benjamin Robin wrote:
> Hello Joshua,
> 
> On Wednesday, May 13, 2026 at 12:29 AM, Joshua Watt wrote:
>> On Tue, May 12, 2026 at 4:27 PM Joshua Watt <jpewhacker@gmail.com> wrote:
>>>
>>> On Tue, May 12, 2026 at 11:02 AM Jérémie Dautheribes via
>>> lists.openembedded.org
>>> <jeremie.dautheribes=bootlin.com@lists.openembedded.org> wrote:
>>>>
>>>> Add support for optional zstd compression for all types of SBOMs,
>>>> including:
>>>>    - image SBOM
>>>>    - recipe SBOM
>>>>    - SDK SBOM
>>
>> We should perhaps also implement decompression when reading in
>> documents, so that the intermediate documents are compressed as well;
>> if we are allowing the final documents to be compressed, I don't see a
>> compelling reason why we wouldn't just compress all of them.
> 
> I am not sure this is a good idea performance-wise, mainly because
> Yocto is currently relying on an external program to compress and
> decompress. We need to wait for Python 3.14 to be the minimum required
> Python version to be able to use the native implementation of zstd.
> Indeed intermediate documents are pretty "small".
> 
> Also with SPDX2, intermediate documents were not compressed.
> 
> The goal is not to reduce the size of the build directory, but only
> the size of deployed artifacts.

In addition to what Benjamin already explained, our typical use-case is
storing the deployed SBOMs to an external location (typically a cloud
provider) and we encountered some cases where the uncompressed image 
SBOM size is ~180 MB.

We could compress them outside of Yocto of course, but we thought it 
would be great to have this feature directly in Yocto, especially since 
it was already supported in the SPDX 2.2 implementation.

Best regards,
-- 
Jérémie Dautheribes, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core][PATCH 2/2] spdx3: support SBOM compression based on SPDX_SBOM_EXT

[Jérémie Dautheribes]

Hello Joshua,

On 13/05/2026 00:27, Joshua Watt wrote:
> On Tue, May 12, 2026 at 11:02 AM Jérémie Dautheribes via
> lists.openembedded.org
> <jeremie.dautheribes=bootlin.com@lists.openembedded.org> wrote:
>>
>> Add support for optional zstd compression for all types of SBOMs,
>> including:
>>    - image SBOM
>>    - recipe SBOM
>>    - SDK SBOM
>>
>> Zstd compression is applied if SPDX_SBOM_EXT ends with ".zst".
>>
>> Co-authored-by: Benjamin Robin (Schneider Electric) <benjamin.robin@bootlin.com>
>> Signed-off-by: Jérémie Dautheribes (Schneider Electric) <jeremie.dautheribes@bootlin.com>
>> ---
>>   meta/classes/create-spdx-3.0.bbclass |  3 ++-
>>   meta/lib/oe/sbom30.py                | 11 +++++++++--
>>   2 files changed, 11 insertions(+), 3 deletions(-)
>>
>> diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass
>> index 785edb9865..6cf8fa4688 100644
>> --- a/meta/classes/create-spdx-3.0.bbclass
>> +++ b/meta/classes/create-spdx-3.0.bbclass
>> @@ -75,7 +75,8 @@ SPDX_IMPORTS[doc] = "SPDX_IMPORTS is the base variable that describes how to \
>>               SPDX 3 spec. Optional but recommended"
>>
>>   SPDX_SBOM_EXT ??= ".spdx.json"
>> -SPDX_SBOM_EXT[doc] = "SBOM file extension name."
>> +SPDX_SBOM_EXT[doc] = "SBOM file extension name.\
>> +    If it ends with '.zst', SBOMs are automatically compressed using Zstd."
>>
>>   # Agents
>>   #   Bitbake variables can be used to describe an SPDX Agent that may be used
>> diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py
>> index 0f1f9281ad..2184c1a07f 100644
>> --- a/meta/lib/oe/sbom30.py
>> +++ b/meta/lib/oe/sbom30.py
>> @@ -1036,8 +1036,15 @@ def write_jsonld_doc(d, objset, dest):
>>           serializer = oe.spdx30.JSONLDInlineSerializer()
>>
>>       objset.objects.add(objset.doc)
>> -    with dest.open("wb") as f:
>> -        serializer.write(objset, f, force_at_graph=True)
>> +
>> +    if dest.name.endswith(".zst"):
> 
> I'm not sure I like this detection mechanism; I think we usually do
> something more explicit for compression rather than relying on the
> suffix in other places?

Maybe we should then introduce a SPDX_COMPRESSED_SBOM boolean variable,
which would be used by SPDX_SBOM_EXT_SUFFIX to determine whether ".zst"
is appended to the SBOM file name or not. Then, we could check in the
`write_jsonld_doc` function whether compression is enabled based on this
SPDX_COMPRESSED_SBOM variable.

What do you think? Do you have any other suggestions?

Best regards,
-- 
Jérémie Dautheribes, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

RE: [OE-core][PATCH 2/2] spdx3: support SBOM compression based on SPDX_SBOM_EXT

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Jérémie Dautheribes via lists.openembedded.org
> Sent: den 13 maj 2026 09:47
> To: Joshua Watt <jpewhacker@gmail.com>
> Cc: openembedded-core@lists.openembedded.org; miquel.raynal@bootlin.com; thomas.petazzoni@bootlin.com; benjamin.robin@bootlin.com
> Subject: Re: [OE-core][PATCH 2/2] spdx3: support SBOM compression based on SPDX_SBOM_EXT
> 
> Hello Joshua,
> 
> On 13/05/2026 00:27, Joshua Watt wrote:
> > On Tue, May 12, 2026 at 11:02 AM Jérémie Dautheribes via lists.openembedded.org <jeremie.dautheribes=bootlin.com@lists.openembedded.org> wrote:
> >>
> >> Add support for optional zstd compression for all types of SBOMs,
> >> including:
> >>    - image SBOM
> >>    - recipe SBOM
> >>    - SDK SBOM
> >>
> >> Zstd compression is applied if SPDX_SBOM_EXT ends with ".zst".
> >>
> >> Co-authored-by: Benjamin Robin (Schneider Electric) <benjamin.robin@bootlin.com>
> >> Signed-off-by: Jérémie Dautheribes (Schneider Electric) <jeremie.dautheribes@bootlin.com>
> >> ---
> >>   meta/classes/create-spdx-3.0.bbclass |  3 ++-
> >>   meta/lib/oe/sbom30.py                | 11 +++++++++--
> >>   2 files changed, 11 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass
> >> index 785edb9865..6cf8fa4688 100644
> >> --- a/meta/classes/create-spdx-3.0.bbclass
> >> +++ b/meta/classes/create-spdx-3.0.bbclass
> >> @@ -75,7 +75,8 @@ SPDX_IMPORTS[doc] = "SPDX_IMPORTS is the base variable that describes how to \
> >>               SPDX 3 spec. Optional but recommended"
> >>
> >>   SPDX_SBOM_EXT ??= ".spdx.json"
> >> -SPDX_SBOM_EXT[doc] = "SBOM file extension name."
> >> +SPDX_SBOM_EXT[doc] = "SBOM file extension name.\
> >> +    If it ends with '.zst', SBOMs are automatically compressed using Zstd."
> >>
> >>   # Agents
> >>   #   Bitbake variables can be used to describe an SPDX Agent that may be used
> >> diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py
> >> index 0f1f9281ad..2184c1a07f 100644
> >> --- a/meta/lib/oe/sbom30.py
> >> +++ b/meta/lib/oe/sbom30.py
> >> @@ -1036,8 +1036,15 @@ def write_jsonld_doc(d, objset, dest):
> >>           serializer = oe.spdx30.JSONLDInlineSerializer()
> >>
> >>       objset.objects.add(objset.doc)
> >> -    with dest.open("wb") as f:
> >> -        serializer.write(objset, f, force_at_graph=True)
> >> +
> >> +    if dest.name.endswith(".zst"):
> >
> > I'm not sure I like this detection mechanism; I think we usually do
> > something more explicit for compression rather than relying on the
> > suffix in other places?
> 
> Maybe we should then introduce a SPDX_COMPRESSED_SBOM boolean variable,
> which would be used by SPDX_SBOM_EXT_SUFFIX to determine whether ".zst"
> is appended to the SBOM file name or not. Then, we could check in the
> `write_jsonld_doc` function whether compression is enabled based on this
> SPDX_COMPRESSED_SBOM variable.
> 
> What do you think? Do you have any other suggestions?

If you use something like:

SPDX_COMPRESSION = "zstd"

then you make it more future proof if someone wants to add support for 
some other compression format.

> 
> Best regards,
> --
> Jérémie Dautheribes, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

//Peter

Re: [OE-core][PATCH 2/2] spdx3: support SBOM compression based on SPDX_SBOM_EXT

[Jérémie Dautheribes]

On 13/05/2026 09:47, Jérémie Dautheribes via lists.openembedded.org wrote:
> Hello Joshua,
> 
> On 13/05/2026 00:27, Joshua Watt wrote:
>> On Tue, May 12, 2026 at 11:02 AM Jérémie Dautheribes via
>> lists.openembedded.org
>> <jeremie.dautheribes=bootlin.com@lists.openembedded.org> wrote:
>>>
>>> Add support for optional zstd compression for all types of SBOMs,
>>> including:
>>>    - image SBOM
>>>    - recipe SBOM
>>>    - SDK SBOM
>>>
>>> Zstd compression is applied if SPDX_SBOM_EXT ends with ".zst".
>>>
>>> Co-authored-by: Benjamin Robin (Schneider Electric) 
>>> <benjamin.robin@bootlin.com>
>>> Signed-off-by: Jérémie Dautheribes (Schneider Electric) 
>>> <jeremie.dautheribes@bootlin.com>
>>> ---
>>>   meta/classes/create-spdx-3.0.bbclass |  3 ++-
>>>   meta/lib/oe/sbom30.py                | 11 +++++++++--
>>>   2 files changed, 11 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/ 
>>> create-spdx-3.0.bbclass
>>> index 785edb9865..6cf8fa4688 100644
>>> --- a/meta/classes/create-spdx-3.0.bbclass
>>> +++ b/meta/classes/create-spdx-3.0.bbclass
>>> @@ -75,7 +75,8 @@ SPDX_IMPORTS[doc] = "SPDX_IMPORTS is the base 
>>> variable that describes how to \
>>>               SPDX 3 spec. Optional but recommended"
>>>
>>>   SPDX_SBOM_EXT ??= ".spdx.json"
>>> -SPDX_SBOM_EXT[doc] = "SBOM file extension name."
>>> +SPDX_SBOM_EXT[doc] = "SBOM file extension name.\
>>> +    If it ends with '.zst', SBOMs are automatically compressed using 
>>> Zstd."
>>>
>>>   # Agents
>>>   #   Bitbake variables can be used to describe an SPDX Agent that 
>>> may be used
>>> diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py
>>> index 0f1f9281ad..2184c1a07f 100644
>>> --- a/meta/lib/oe/sbom30.py
>>> +++ b/meta/lib/oe/sbom30.py
>>> @@ -1036,8 +1036,15 @@ def write_jsonld_doc(d, objset, dest):
>>>           serializer = oe.spdx30.JSONLDInlineSerializer()
>>>
>>>       objset.objects.add(objset.doc)
>>> -    with dest.open("wb") as f:
>>> -        serializer.write(objset, f, force_at_graph=True)
>>> +
>>> +    if dest.name.endswith(".zst"):
>>
>> I'm not sure I like this detection mechanism; I think we usually do
>> something more explicit for compression rather than relying on the
>> suffix in other places?
> 
> Maybe we should then introduce a SPDX_COMPRESSED_SBOM boolean variable,
> which would be used by SPDX_SBOM_EXT_SUFFIX to determine whether ".zst"
> is appended to the SBOM file name or not. Then, we could check in the
> `write_jsonld_doc` function whether compression is enabled based on this
> SPDX_COMPRESSED_SBOM variable.
> 

After further thought, that solution would not work well since
`write_jsonld_doc` is not only used for SBOM generation.

-- 
Jérémie Dautheribes, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com