[Buildroot] [RFC PATCH 0/5] Support SBOM in CycloneDX format

[Buildroot] [RFC PATCH 0/5] Support SBOM in CycloneDX format

[Thomas Perale via buildroot]

This RFC patch series propose to add support for the CycloneDX
SBOM format.

There is a growing need to generate SBOM from buildroot
configurations. Right now, there are different solutions available
for buildroot users `show-info`, `legal-info` and `pkg-stats`.
They all generate similar information (`show-info` showing more) but
in a format that is specific to buildroot.

CycloneDX is a format already supported by tools such as
https://dependencytrack.org/ that helps track software,
vulnerabilities, etc ...

A DependencyTrack instance has been deployed at https://dependencytrack-lqgs7zhwmq-ew.a.run.app/
so you can play with CycloneDX and see the possibility. Be mindful
that I used a Google Cloud instance with cold start enabled so it can
take several minutes to initialize if no one has used the service.
Login with username `buildroot` and password `buildroot`.
All the project you create will be publicly accessible and removable,
just so you know.

To generate a CycloneDX SBOM compatible with DependencyTrack I found out
that one line minified JSON does not work and I had to format it. I
used the tool `json_reformat` to do it but tools are also available
online. I use the following command:

  make cyclonedx | json_reformat > sbom.json

This is a first sketch and I hope to gather comments on functionality
the community want me to include. I already have a todo list of feature
I plan to work on:

- [ ] Find a solution to handle versioning. The "version" property
      should be incremented every SBOM generation.
- [ ] Add more buildroot 'property' to components
  - [ ] Infrastructure (cmake, cargo etc).
  - [ ] Match the column of the `pkg-stat` command and add the missing
        one that don't have a place in the current CycloneDX spec to a
        corresponding property
- [ ] Make it configurable like pkg-stat
- [ ] Use PACKAGES_ALL if no .config
- [ ] Fix licenses list if there are commas inside parenthesis. Example
      of wrong license list:
              {
                  "license": {
                      "name": "LGPL-2.1+ (libblkid"
                  }
              },
              {
                  "license": {
                      "name": "libfdisk"
                  }
              },
              {
                  "license": {
                      "name": "libmount)"
                  }
              },
- [ ] Introduce a variable `_UPSTREAM_NAME` that defaults to `_RAWNAME`.
      For most python packages for instance, buildroot `python-` prefix
      shouldn't be in the upstream name. This `_UPSTREAM_NAME` can also be
      used e.g. for the default of CPE, PURL, and maybe in other places
      too.
- [ ] For patches downloaded using `_PATCH` variable (rarely used).
      Modify the pedigree to specify the URL instead of writting the
      content.
- [ ] Include the list of upstream URL with `externalReferences` the type
      will be `distribution`
      https://cyclonedx.org/docs/1.5/json/#components_items_externalReferences
- [ ] Add rule to the gitlab ci that check the SPDX licenses are up to
      date.
- [ ] Upstream changes to Dependency Track
  - [ ] Support 1-line minified JSON
  - [ ] Support showing patches and property

Thank you !

Thomas Perale (5):
  package/pkg-generic.mk: add PURL package variable
  package/pkg-utils.mk: urlencode/urldecode macros
  support/misc/cyclonedx.mk: support CycloneDX format
  support/misc/cyclonedx.mk: support spdx license check
  Makefile: add command to generate SBOM in CycloneDX format

 Makefile                       |  13 +
 package/pkg-generic.mk         |  12 +
 package/pkg-utils.mk           |  12 +
 support/misc/cyclonedx-spdx.mk | 617 +++++++++++++++++++++++++++++++++
 support/misc/cyclonedx.mk      | 230 ++++++++++++
 5 files changed, 884 insertions(+)
 create mode 100644 support/misc/cyclonedx-spdx.mk
 create mode 100644 support/misc/cyclonedx.mk

--
2.44.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [RFC PATCH 1/5] package/pkg-generic.mk: add PURL package variable

[Thomas Perale via buildroot]

PURL stands for 'package URL', it's a specification that standardize
how packages are identified and located.

PURL is used to reference the same package across different package
manager, tracking tools, API and databases.

A purl is a URL composed of seven components:

  scheme:type/namespace/name@version?qualifiers#subpath

  - scheme: always 'pkg' (required)
  - type: package manager used to install the package, download origin,
          type of package (required)
  - namespace: name prefix, type specific additional information (optional)
  - name: package name (required)
  - version: package version (optional)
  - qualifiers: extra information (optional)
  - subpath: extra subpath relative to package root (optional)

A PURL for the purl-spec repository looks like this:

  pkg:github/package-url/purl-spec@346589846130317464b677bc4eab30bf5040183a

It contains information like the provenance (github), organization
(package-url), name (purl-spec) and version (34658984...).

This patch introduces an auto-generated PURL for non internal packages
(packages with a `<pkg>_SOURCE` variable), with the possibility for each
package to define their own PURL by defining the following variable:

  <pkg>_PURL

If the variable is not defined it will be generated as follows:

  $(2)_PURL = pkg:generic/$$($(2)_RAWNAME)@$$($(2)_VERSION)

The type 'generic' is used by default but can be extended in the future
to support github, gitlab, etc ...

For more information, see https://github.com/package-url/purl-spec

Signed-off-by: Thomas Perale <thomas.perale@mind.be>
---
 package/pkg-generic.mk | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 577a148c1e..0966b714ff 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -754,6 +754,18 @@ ifeq ($$($(2)_CPE_ID_VALID),YES)
  $(2)_CPE_ID = $$($(2)_CPE_ID_PREFIX):$$($(2)_CPE_ID_VENDOR):$$($(2)_CPE_ID_PRODUCT):$$($(2)_CPE_ID_VERSION):$$($(2)_CPE_ID_UPDATE):*:*:*:*:*:*
 endif # ifeq ($$($(2)_CPE_ID_VALID),YES)

+# If no package url (purl) is set, a generic purl is created for non internal
+# packages.
+# see https://github.com/package-url/purl-spec
+ifndef $(2)_PURL
+ ifdef $(3)_PURL
+  $(2)_PURL = $$($(3)_PURL)
+ endif
+ ifdef $(2)_SOURCE
+   $(2)_PURL = pkg:generic/$$($(2)_RAWNAME)@$$($(2)_VERSION)
+ endif
+endif
+
 # When a target package is a toolchain dependency set this variable to
 # 'NO' so the 'toolchain' dependency is not added to prevent a circular
 # dependency.
--
2.44.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [RFC PATCH 2/5] package/pkg-utils.mk: urlencode/urldecode macros

[Thomas Perale via buildroot]

This patch introduces two new macros:

  - urlencode
  - urldecode

URL encoding consists of converting ASCII characters into
a percent symbol followed by a two digit hexadecimal code.
And the other way around for URL decoding.

The macros encode/decode a string passed as an argument by
escaping the following characters:

  - '%' is replaced by %25
  - 'space' is replaced by %20

The characters covered by this patch is non exhaustive.

Because the Make language treats spaces as different entry of a list,
the aim of those macros is to provide a well known encoding method
to escape text containing spaces into a string that won't be
treated as a list by the Make language.

Signed-off-by: Thomas Perale <thomas.perale@mind.be>
---
 package/pkg-utils.mk | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk
index 723bbe4e24..b8cfb85fca 100644
--- a/package/pkg-utils.mk
+++ b/package/pkg-utils.mk
@@ -329,3 +329,15 @@ define NXP_EXTRACT_HELPER
 	find $(@D)/$(basename $(notdir $(1))) -mindepth 1 -maxdepth 1 -exec mv {} $(@D) \;
 	rmdir $(@D)/$(basename $(notdir $(1)))
 endef
+
+# urlencode -- returns an url encoded string.
+#   - encode % into %25
+#   - encode spaces into %20
+#
+# $(1): text
+urlencode = $(subst $(space),%20,$(subst %,%25,$(1)))
+
+# urldecode -- decode an url encoded string.
+#
+# $(1): text
+urldecode = $(subst %25,%,$(subst %20,$(space),$(1)))
--
2.44.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [RFC PATCH 2/5] package/pkg-utils.mk: urlencode/urldecode macros

[Yann E. MORIN]

Thomas, All,

On 2024-04-04 14:43 +0200, Thomas Perale via buildroot spake thusly:
> This patch introduces two new macros:
> 
>   - urlencode
>   - urldecode
> 
> URL encoding consists of converting ASCII characters into
> a percent symbol followed by a two digit hexadecimal code.
> And the other way around for URL decoding.
> 
> The macros encode/decode a string passed as an argument by
> escaping the following characters:
> 
>   - '%' is replaced by %25
>   - 'space' is replaced by %20
> 
> The characters covered by this patch is non exhaustive.

Not sure what you meant with this sentence... Can you elaborate?
Or did you mean that we can add more chars in the future?

Regards,
Yann E. MORIN.

> Because the Make language treats spaces as different entry of a list,
> the aim of those macros is to provide a well known encoding method
> to escape text containing spaces into a string that won't be
> treated as a list by the Make language.
> 
> Signed-off-by: Thomas Perale <thomas.perale@mind.be>
> ---
>  package/pkg-utils.mk | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk
> index 723bbe4e24..b8cfb85fca 100644
> --- a/package/pkg-utils.mk
> +++ b/package/pkg-utils.mk
> @@ -329,3 +329,15 @@ define NXP_EXTRACT_HELPER
>  	find $(@D)/$(basename $(notdir $(1))) -mindepth 1 -maxdepth 1 -exec mv {} $(@D) \;
>  	rmdir $(@D)/$(basename $(notdir $(1)))
>  endef
> +
> +# urlencode -- returns an url encoded string.
> +#   - encode % into %25
> +#   - encode spaces into %20
> +#
> +# $(1): text
> +urlencode = $(subst $(space),%20,$(subst %,%25,$(1)))
> +
> +# urldecode -- decode an url encoded string.
> +#
> +# $(1): text
> +urldecode = $(subst %25,%,$(subst %20,$(space),$(1)))
> --
> 2.44.0
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [RFC PATCH 2/5] package/pkg-utils.mk: urlencode/urldecode macros

[Arnout Vandecappelle via buildroot]

On 07/04/2024 19:44, Yann E. MORIN wrote:
> Thomas, All,
> 
> On 2024-04-04 14:43 +0200, Thomas Perale via buildroot spake thusly:
>> This patch introduces two new macros:
>>
>>    - urlencode
>>    - urldecode
>>
>> URL encoding consists of converting ASCII characters into
>> a percent symbol followed by a two digit hexadecimal code.
>> And the other way around for URL decoding.
>>
>> The macros encode/decode a string passed as an argument by
>> escaping the following characters:
>>
>>    - '%' is replaced by %25
>>    - 'space' is replaced by %20
>>
>> The characters covered by this patch is non exhaustive.
> 
> Not sure what you meant with this sentence... Can you elaborate?

  It means that not all characters that normally are urlencoded (like /, :, #) 
are handled by this patch.

> Or did you mean that we can add more chars in the future?

  So yes, that.

  Regards,
  Arnout

> 
> Regards,
> Yann E. MORIN.
> 
>> Because the Make language treats spaces as different entry of a list,
>> the aim of those macros is to provide a well known encoding method
>> to escape text containing spaces into a string that won't be
>> treated as a list by the Make language.
>>
>> Signed-off-by: Thomas Perale <thomas.perale@mind.be>
>> ---
>>   package/pkg-utils.mk | 12 ++++++++++++
>>   1 file changed, 12 insertions(+)
>>
>> diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk
>> index 723bbe4e24..b8cfb85fca 100644
>> --- a/package/pkg-utils.mk
>> +++ b/package/pkg-utils.mk
>> @@ -329,3 +329,15 @@ define NXP_EXTRACT_HELPER
>>   	find $(@D)/$(basename $(notdir $(1))) -mindepth 1 -maxdepth 1 -exec mv {} $(@D) \;
>>   	rmdir $(@D)/$(basename $(notdir $(1)))
>>   endef
>> +
>> +# urlencode -- returns an url encoded string.
>> +#   - encode % into %25
>> +#   - encode spaces into %20
>> +#
>> +# $(1): text
>> +urlencode = $(subst $(space),%20,$(subst %,%25,$(1)))
>> +
>> +# urldecode -- decode an url encoded string.
>> +#
>> +# $(1): text
>> +urldecode = $(subst %25,%,$(subst %20,$(space),$(1)))
>> --
>> 2.44.0
>>
>> _______________________________________________
>> buildroot mailing list
>> buildroot@buildroot.org
>> https://lists.buildroot.org/mailman/listinfo/buildroot
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [RFC PATCH 3/5] support/misc/cyclonedx.mk: support CycloneDX format

[Thomas Perale via buildroot]

CycloneDX is a software bill of materials (SBOM) specification.

There is a growing need to generate SBOM from buildroot
configurations. Right now there are different solutions available
for buildroot users `show-info`, `legal-info` and `pkg-stats`.
They all generate similar information (`show-info` showing more) but in
a format that is specific to buildroot.

This is the reason this patch introduces a new SBOM output type for
buildroot: CycloneDX. CycloneDX is a format already supported by tools
such as https://dependencytrack.org/ that helps track softwares,
vulnerabilities, etc ...

To match the functionality of `show-info`, buildroot internal packages
will also be present in the SBOM with a reduced set of property.
Internal packages are defined as packages without `<pkg>_SOURCE`
defined.
In a future patch more properties can be added to cover the
functionality of `show-info`, `legal-info` and `pkg-stats`.

The CycloneDX SBOM output as a stripped JSON formatted line as there are
already macros available to work with JSON in buildroot.

For more information, see https://cyclonedx.org/
and https://cyclonedx.org/docs/1.5/json/

Signed-off-by: Thomas Perale <thomas.perale@mind.be>
---
 support/misc/cyclonedx.mk | 197 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 197 insertions(+)
 create mode 100644 support/misc/cyclonedx.mk

diff --git a/support/misc/cyclonedx.mk b/support/misc/cyclonedx.mk
new file mode 100644
index 0000000000..3906a3b60a
--- /dev/null
+++ b/support/misc/cyclonedx.mk
@@ -0,0 +1,197 @@
+################################################################################
+#
+# This file contains various utility functions used to create a SBOM
+# in the JSON CycloneDX format.
+#
+# https://cyclonedx.org/docs/1.5/json/
+#
+################################################################################
+
+# Note: to avoid conflict with <pkg>_VERSION `_SPEC` is added
+CYCLONEDX_VERSION_SPEC = 1.5
+
+#
+# Licenses list helper functions
+# Since licenses in buildroot are comma separated list and the 'make' language
+# uses spaces to create list we need to replace the spaces of licenses name
+# by a character not used in any licenses name.
+#
+# _cyclonedx-licenses-as-list: create a list from the url-encoded comma
+#                              separeted license list.
+#
+# $(1): an url-encoded comma separeted list
+#
+# Turns "Public%20Domain,%20GPL-2.0" into "Public%20Domain GPL-2.0"
+_cyclonedx-licenses-as-list = $(subst $(comma)%20,$(space),$(1))
+
+# _cyclonedx-license -- create an entry of a cyclonedx component license list
+#
+# For more information on license object see
+# https://cyclonedx.org/docs/1.5/json/#components_items_licenses_oneOf_i0_items_license
+#
+# $(1): a single url-encoded license name
+define _cyclonedx-license
+	{
+		"license": {
+			"name": $(call mk-json-str,$(1))
+		}
+	},
+endef
+
+# _cyclonedx-licenses -- create a licenses list formatted for a CycloneDX
+#                        component
+#
+# $(1): a comma separated license list
+#
+# KNOWN ISSUE: Licenses name that include a parenthesis with comma inside,
+#   will result be misinterpreted as multiple licenses name:
+#   - host-util-linux: LGPL-2.1+ (libblkid, libfdisk, libmount)
+define _cyclonedx-licenses
+	$(foreach license,$(call _cyclonedx-licenses-as-list,$(call urlencode,$(1))),
+		$(call urldecode,$(call _cyclonedx-license,$(license)))
+	)
+endef
+
+# Note about patch list: this patch list might not be complete.
+# There is no variable yet that stores the patch list without applying them.
+_cyclonedx-patches-list = $(foreach patchdir,\
+	$(addsuffix /$($(1)_PKGDIR),$(CURDIR)) $(addsuffix /$($(1)_RAWNAME),$(call qstrip,$(BR2_GLOBAL_PATCH_DIR))),\
+	$(wildcard $(addsuffix /*.patch,$(patchdir)))\
+)
+
+# _cyclonedx-patch -- single entry of a patch list.
+#                     It's required to pass the type of the patch it can be
+#                     either: unofficial, monkey, backport or cherry-pick.
+#                     Since there is no information available about each
+#                     patches, we mark them as "unofficial".
+#
+# $(1): single patch path
+define _cyclonedx-patch
+	{
+		"type": "unofficial",
+		"diff": {
+			"text": {
+				"content": $(call mk-json-str,$(file < $(1)))
+			}
+		}
+	},
+endef
+
+# _cyclonedx-patches -- patch list are stored under the pedigree entry used to
+#                       document how a component is modified.
+#
+# $(1): patch path list
+define _cyclonedx-patches
+	$(intcmp $(words $(1)),0,,,
+		"pedigree": {\
+			"patches": [\
+				$(foreach patch,$(1),\
+					$(call _cyclonedx-patch,$(patch))\
+				)\
+			]\
+		}$(comma)\
+	)
+endef
+
+# _cyclonedx-component -- representation of a package for the CycloneDX format
+#  - bom-ref: is a unique identifier used to refer to this component in the
+#      'dependencies' section.
+#  - type: is a required property since we don't have enough information about
+#      the package from its definition CycloneDX spec recommend setting it
+#      to 'library'.
+#  - properties: is used to add additional information that doesn't fit the
+#      current CycloneDX specification.
+#    - BR_TYPE: {host|target}
+#
+# $(1): upper-case package name
+#
+# KNWON ISSUE: packages with a custom tarball (linux,uboot,...) will have the
+#   'version' property set to 'custom'.
+define _cyclonedx-component
+	{
+		"bom-ref": $(call mk-json-str,$($(1)_NAME)),
+		"name": $(call mk-json-str,$(if $($(1)_RAWNAME),$($(1)_RAWNAME),$($(1)_NAME))),
+		"type": "library",
+		$(if $($(1)_SOURCE),
+			"version": $(call mk-json-str,$($(1)_DL_VERSION))$(comma)
+			"licenses": [
+				$(call _cyclonedx-licenses,$($(1)_LICENSE)) \
+			]$(comma)
+		)
+		$(if $($(1)_PURL), \
+			"purl": $(call mk-json-str,$($(1)_PURL))$(comma) \
+		)
+		$(if $($(1)_CPE_ID_VALID), \
+			"cpe": $(call mk-json-str,$($(1)_CPE_ID))$(comma) \
+		)
+		$(call _cyclonedx-patches,$(call _cyclonedx-patches-list,$(1)))
+		"properties": [{
+			"name": "BR_TYPE",
+			"value": $(call mk-json-str,$($(1)_TYPE))
+		}],
+	},
+endef
+
+# _cyclonedx-dependency -- create dependency relationships between components.
+#  - ref: reference to a component bom-ref.
+#  - dependsOn: array of component bom-ref identifier to create the dependencies.
+#
+# $(1): upper-case package name
+define _cyclonedx-dependency
+	$(if $($(1)_FINAL_RECURSIVE_DEPENDENCIES),
+		{
+			"ref": $(call mk-json-str,$($(1)_NAME))$(comma)
+			"dependsOn": [
+				$(call make-comma-list,$(foreach p,\
+					$($(1)_FINAL_RECURSIVE_DEPENDENCIES), \
+					$(call mk-json-str,$(p))\
+				))
+			]
+		}$(comma)
+	)
+endef
+
+# cyclonedx-json -- return a CycloneDX SBOM formatted as a JSON dictionnary.
+#  - bomFormat: required field is always "CycloneDX"
+#  - specVersion: required field with CycloneDX spec version
+#  - version: is used by software that accept CycloneDX SBOM to differentiate
+#    the different SBOM. The bigger the number the newer the SBOM is.
+#    Here it's set to '1' and it should be incremented when the resulting
+#    SBOM is edited later.
+#    The CycloneDX spec mentions that an 'uuid' property can also be used to
+#    differentiate SBOM but is not included because there is no native command
+#    to generate an uuid in buildroot.
+#
+# $(1): packages list
+define cyclonedx-json
+	$(call clean-json,{
+		"bomFormat": "CycloneDX"$(comma)
+		"$$schema": "http://cyclonedx.org/schema/bom-$(CYCLONEDX_VERSION_SPEC).schema.json"$(comma)
+		"specVersion": $(call mk-json-str,$(CYCLONEDX_VERSION_SPEC))$(comma)
+		"version": 1$(comma)
+		"components": [ \
+			$(foreach p,$(1), \
+				$(call _cyclonedx-component,$(call UPPERCASE,$(p))) \
+			) \
+		]$(comma)
+		"dependencies": [
+			{
+				"ref": "buildroot"$(comma)\
+				"dependsOn": [$(call make-comma-list,\
+					$(foreach p,$(1),$(call mk-json-str,$(p)))\
+				)]\
+			}$(comma)
+			$(foreach p,$(1),\
+				$(call _cyclonedx-dependency,$(call UPPERCASE,$(p)),$(2)) \
+			) \
+		]$(comma)
+		"metadata": {
+			"component": {
+				"bom-ref": "buildroot"$(comma)
+				"name": "buildroot"$(comma)
+				"type": "firmware"$(comma)
+				"version": $(call mk-json-str,$(BR2_VERSION_FULL))
+			}
+		}
+	})
+endef
--
2.44.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [RFC PATCH 4/5] support/misc/cyclonedx.mk: support spdx license check

[Thomas Perale via buildroot]

To improve the tracking of the licenses of the components, the file
`support/misc/cyclonedx-spdx.mk` that contains a definition of every
approved SPDX licenses ID is included in this patch.

This file has been generated by the `support/misc/cyclonedx-spdx.mk`
rule included `support/misc/cyclonedx.mk`. It will remain there to
re-generate the file if it's updated.

Knowing if a license name is a valid SPDX ID or not, allows tools
such as Dependency Track to directly show the license content of
components with a known SPDX ID.

Signed-off-by: Thomas Perale <thomas.perale@mind.be>
---
 support/misc/cyclonedx-spdx.mk | 617 +++++++++++++++++++++++++++++++++
 support/misc/cyclonedx.mk      |  35 +-
 2 files changed, 651 insertions(+), 1 deletion(-)
 create mode 100644 support/misc/cyclonedx-spdx.mk

diff --git a/support/misc/cyclonedx-spdx.mk b/support/misc/cyclonedx-spdx.mk
new file mode 100644
index 0000000000..81c387fd23
--- /dev/null
+++ b/support/misc/cyclonedx-spdx.mk
@@ -0,0 +1,617 @@
+# List of approved SPDX license
+# See https://raw.githubusercontent.com/CycloneDX/specification/1.5/schema/spdx.schema.json
+define spdx
+0BSD
+AAL
+Abstyles
+AdaCore-doc
+Adobe-2006
+Adobe-Glyph
+ADSL
+AFL-1.1
+AFL-1.2
+AFL-2.0
+AFL-2.1
+AFL-3.0
+Afmparse
+AGPL-1.0
+AGPL-1.0-only
+AGPL-1.0-or-later
+AGPL-3.0
+AGPL-3.0-only
+AGPL-3.0-or-later
+Aladdin
+AMDPLPA
+AML
+AMPAS
+ANTLR-PD
+ANTLR-PD-fallback
+Apache-1.0
+Apache-1.1
+Apache-2.0
+APAFML
+APL-1.0
+App-s2p
+APSL-1.0
+APSL-1.1
+APSL-1.2
+APSL-2.0
+Arphic-1999
+Artistic-1.0
+Artistic-1.0-cl8
+Artistic-1.0-Perl
+Artistic-2.0
+ASWF-Digital-Assets-1.0
+ASWF-Digital-Assets-1.1
+Baekmuk
+Bahyph
+Barr
+Beerware
+Bitstream-Charter
+Bitstream-Vera
+BitTorrent-1.0
+BitTorrent-1.1
+blessing
+BlueOak-1.0.0
+Boehm-GC
+Borceux
+Brian-Gladman-3-Clause
+BSD-1-Clause
+BSD-2-Clause
+BSD-2-Clause-FreeBSD
+BSD-2-Clause-NetBSD
+BSD-2-Clause-Patent
+BSD-2-Clause-Views
+BSD-3-Clause
+BSD-3-Clause-Attribution
+BSD-3-Clause-Clear
+BSD-3-Clause-LBNL
+BSD-3-Clause-Modification
+BSD-3-Clause-No-Military-License
+BSD-3-Clause-No-Nuclear-License
+BSD-3-Clause-No-Nuclear-License-2014
+BSD-3-Clause-No-Nuclear-Warranty
+BSD-3-Clause-Open-MPI
+BSD-4-Clause
+BSD-4-Clause-Shortened
+BSD-4-Clause-UC
+BSD-4.3RENO
+BSD-4.3TAHOE
+BSD-Advertising-Acknowledgement
+BSD-Attribution-HPND-disclaimer
+BSD-Protection
+BSD-Source-Code
+BSL-1.0
+BUSL-1.1
+bzip2-1.0.5
+bzip2-1.0.6
+C-UDA-1.0
+CAL-1.0
+CAL-1.0-Combined-Work-Exception
+Caldera
+CATOSL-1.1
+CC-BY-1.0
+CC-BY-2.0
+CC-BY-2.5
+CC-BY-2.5-AU
+CC-BY-3.0
+CC-BY-3.0-AT
+CC-BY-3.0-DE
+CC-BY-3.0-IGO
+CC-BY-3.0-NL
+CC-BY-3.0-US
+CC-BY-4.0
+CC-BY-NC-1.0
+CC-BY-NC-2.0
+CC-BY-NC-2.5
+CC-BY-NC-3.0
+CC-BY-NC-3.0-DE
+CC-BY-NC-4.0
+CC-BY-NC-ND-1.0
+CC-BY-NC-ND-2.0
+CC-BY-NC-ND-2.5
+CC-BY-NC-ND-3.0
+CC-BY-NC-ND-3.0-DE
+CC-BY-NC-ND-3.0-IGO
+CC-BY-NC-ND-4.0
+CC-BY-NC-SA-1.0
+CC-BY-NC-SA-2.0
+CC-BY-NC-SA-2.0-DE
+CC-BY-NC-SA-2.0-FR
+CC-BY-NC-SA-2.0-UK
+CC-BY-NC-SA-2.5
+CC-BY-NC-SA-3.0
+CC-BY-NC-SA-3.0-DE
+CC-BY-NC-SA-3.0-IGO
+CC-BY-NC-SA-4.0
+CC-BY-ND-1.0
+CC-BY-ND-2.0
+CC-BY-ND-2.5
+CC-BY-ND-3.0
+CC-BY-ND-3.0-DE
+CC-BY-ND-4.0
+CC-BY-SA-1.0
+CC-BY-SA-2.0
+CC-BY-SA-2.0-UK
+CC-BY-SA-2.1-JP
+CC-BY-SA-2.5
+CC-BY-SA-3.0
+CC-BY-SA-3.0-AT
+CC-BY-SA-3.0-DE
+CC-BY-SA-3.0-IGO
+CC-BY-SA-4.0
+CC-PDDC
+CC0-1.0
+CDDL-1.0
+CDDL-1.1
+CDL-1.0
+CDLA-Permissive-1.0
+CDLA-Permissive-2.0
+CDLA-Sharing-1.0
+CECILL-1.0
+CECILL-1.1
+CECILL-2.0
+CECILL-2.1
+CECILL-B
+CECILL-C
+CERN-OHL-1.1
+CERN-OHL-1.2
+CERN-OHL-P-2.0
+CERN-OHL-S-2.0
+CERN-OHL-W-2.0
+CFITSIO
+checkmk
+ClArtistic
+Clips
+CMU-Mach
+CNRI-Jython
+CNRI-Python
+CNRI-Python-GPL-Compatible
+COIL-1.0
+Community-Spec-1.0
+Condor-1.1
+copyleft-next-0.3.0
+copyleft-next-0.3.1
+Cornell-Lossless-JPEG
+CPAL-1.0
+CPL-1.0
+CPOL-1.02
+Crossword
+CrystalStacker
+CUA-OPL-1.0
+Cube
+curl
+D-FSL-1.0
+diffmark
+DL-DE-BY-2.0
+DOC
+Dotseqn
+DRL-1.0
+DSDP
+dtoa
+dvipdfm
+ECL-1.0
+ECL-2.0
+eCos-2.0
+EFL-1.0
+EFL-2.0
+eGenix
+Elastic-2.0
+Entessa
+EPICS
+EPL-1.0
+EPL-2.0
+ErlPL-1.1
+etalab-2.0
+EUDatagrid
+EUPL-1.0
+EUPL-1.1
+EUPL-1.2
+Eurosym
+Fair
+FDK-AAC
+Frameworx-1.0
+FreeBSD-DOC
+FreeImage
+FSFAP
+FSFUL
+FSFULLR
+FSFULLRWD
+FTL
+GD
+GFDL-1.1
+GFDL-1.1-invariants-only
+GFDL-1.1-invariants-or-later
+GFDL-1.1-no-invariants-only
+GFDL-1.1-no-invariants-or-later
+GFDL-1.1-only
+GFDL-1.1-or-later
+GFDL-1.2
+GFDL-1.2-invariants-only
+GFDL-1.2-invariants-or-later
+GFDL-1.2-no-invariants-only
+GFDL-1.2-no-invariants-or-later
+GFDL-1.2-only
+GFDL-1.2-or-later
+GFDL-1.3
+GFDL-1.3-invariants-only
+GFDL-1.3-invariants-or-later
+GFDL-1.3-no-invariants-only
+GFDL-1.3-no-invariants-or-later
+GFDL-1.3-only
+GFDL-1.3-or-later
+Giftware
+GL2PS
+Glide
+Glulxe
+GLWTPL
+gnuplot
+GPL-1.0
+GPL-1.0+
+GPL-1.0-only
+GPL-1.0-or-later
+GPL-2.0
+GPL-2.0+
+GPL-2.0-only
+GPL-2.0-or-later
+GPL-2.0-with-autoconf-exception
+GPL-2.0-with-bison-exception
+GPL-2.0-with-classpath-exception
+GPL-2.0-with-font-exception
+GPL-2.0-with-GCC-exception
+GPL-3.0
+GPL-3.0+
+GPL-3.0-only
+GPL-3.0-or-later
+GPL-3.0-with-autoconf-exception
+GPL-3.0-with-GCC-exception
+Graphics-Gems
+gSOAP-1.3b
+HaskellReport
+Hippocratic-2.1
+HP-1986
+HPND
+HPND-export-US
+HPND-Markus-Kuhn
+HPND-sell-variant
+HPND-sell-variant-MIT-disclaimer
+HTMLTIDY
+IBM-pibs
+ICU
+IEC-Code-Components-EULA
+IJG
+IJG-short
+ImageMagick
+iMatix
+Imlib2
+Info-ZIP
+Inner-Net-2.0
+Intel
+Intel-ACPI
+Interbase-1.0
+IPA
+IPL-1.0
+ISC
+Jam
+JasPer-2.0
+JPL-image
+JPNIC
+JSON
+Kazlib
+Knuth-CTAN
+LAL-1.2
+LAL-1.3
+Latex2e
+Latex2e-translated-notice
+Leptonica
+LGPL-2.0
+LGPL-2.0+
+LGPL-2.0-only
+LGPL-2.0-or-later
+LGPL-2.1
+LGPL-2.1+
+LGPL-2.1-only
+LGPL-2.1-or-later
+LGPL-3.0
+LGPL-3.0+
+LGPL-3.0-only
+LGPL-3.0-or-later
+LGPLLR
+Libpng
+libpng-2.0
+libselinux-1.0
+libtiff
+libutil-David-Nugent
+LiLiQ-P-1.1
+LiLiQ-R-1.1
+LiLiQ-Rplus-1.1
+Linux-man-pages-1-para
+Linux-man-pages-copyleft
+Linux-man-pages-copyleft-2-para
+Linux-man-pages-copyleft-var
+Linux-OpenIB
+LOOP
+LPL-1.0
+LPL-1.02
+LPPL-1.0
+LPPL-1.1
+LPPL-1.2
+LPPL-1.3a
+LPPL-1.3c
+LZMA-SDK-9.11-to-9.20
+LZMA-SDK-9.22
+MakeIndex
+Martin-Birgmeier
+metamail
+Minpack
+MirOS
+MIT
+MIT-0
+MIT-advertising
+MIT-CMU
+MIT-enna
+MIT-feh
+MIT-Festival
+MIT-Modern-Variant
+MIT-open-group
+MIT-Wu
+MITNFA
+Motosoto
+mpi-permissive
+mpich2
+MPL-1.0
+MPL-1.1
+MPL-2.0
+MPL-2.0-no-copyleft-exception
+mplus
+MS-LPL
+MS-PL
+MS-RL
+MTLL
+MulanPSL-1.0
+MulanPSL-2.0
+Multics
+Mup
+NAIST-2003
+NASA-1.3
+Naumen
+NBPL-1.0
+NCGL-UK-2.0
+NCSA
+Net-SNMP
+NetCDF
+Newsletr
+NGPL
+NICTA-1.0
+NIST-PD
+NIST-PD-fallback
+NIST-Software
+NLOD-1.0
+NLOD-2.0
+NLPL
+Nokia
+NOSL
+Noweb
+NPL-1.0
+NPL-1.1
+NPOSL-3.0
+NRL
+NTP
+NTP-0
+Nunit
+O-UDA-1.0
+OCCT-PL
+OCLC-2.0
+ODbL-1.0
+ODC-By-1.0
+OFFIS
+OFL-1.0
+OFL-1.0-no-RFN
+OFL-1.0-RFN
+OFL-1.1
+OFL-1.1-no-RFN
+OFL-1.1-RFN
+OGC-1.0
+OGDL-Taiwan-1.0
+OGL-Canada-2.0
+OGL-UK-1.0
+OGL-UK-2.0
+OGL-UK-3.0
+OGTSL
+OLDAP-1.1
+OLDAP-1.2
+OLDAP-1.3
+OLDAP-1.4
+OLDAP-2.0
+OLDAP-2.0.1
+OLDAP-2.1
+OLDAP-2.2
+OLDAP-2.2.1
+OLDAP-2.2.2
+OLDAP-2.3
+OLDAP-2.4
+OLDAP-2.5
+OLDAP-2.6
+OLDAP-2.7
+OLDAP-2.8
+OLFL-1.3
+OML
+OpenPBS-2.3
+OpenSSL
+OPL-1.0
+OPL-UK-3.0
+OPUBL-1.0
+OSET-PL-2.1
+OSL-1.0
+OSL-1.1
+OSL-2.0
+OSL-2.1
+OSL-3.0
+Parity-6.0.0
+Parity-7.0.0
+PDDL-1.0
+PHP-3.0
+PHP-3.01
+Plexus
+PolyForm-Noncommercial-1.0.0
+PolyForm-Small-Business-1.0.0
+PostgreSQL
+PSF-2.0
+psfrag
+psutils
+Python-2.0
+Python-2.0.1
+Qhull
+QPL-1.0
+QPL-1.0-INRIA-2004
+Rdisc
+RHeCos-1.1
+RPL-1.1
+RPL-1.5
+RPSL-1.0
+RSA-MD
+RSCPL
+Ruby
+SAX-PD
+Saxpath
+SCEA
+SchemeReport
+Sendmail
+Sendmail-8.23
+SGI-B-1.0
+SGI-B-1.1
+SGI-B-2.0
+SGP4
+SHL-0.5
+SHL-0.51
+SimPL-2.0
+SISSL
+SISSL-1.2
+Sleepycat
+SMLNJ
+SMPPL
+SNIA
+snprintf
+Spencer-86
+Spencer-94
+Spencer-99
+SPL-1.0
+SSH-OpenSSH
+SSH-short
+SSPL-1.0
+StandardML-NJ
+SugarCRM-1.1.3
+SunPro
+SWL
+Symlinks
+TAPR-OHL-1.0
+TCL
+TCP-wrappers
+TermReadKey
+TMate
+TORQUE-1.1
+TOSL
+TPDL
+TPL-1.0
+TTWL
+TU-Berlin-1.0
+TU-Berlin-2.0
+UCAR
+UCL-1.0
+Unicode-DFS-2015
+Unicode-DFS-2016
+Unicode-TOU
+UnixCrypt
+Unlicense
+UPL-1.0
+Vim
+VOSTROM
+VSL-1.0
+W3C
+W3C-19980720
+W3C-20150513
+w3m
+Watcom-1.0
+Widget-Workshop
+Wsuipa
+WTFPL
+wxWindows
+X11
+X11-distribute-modifications-variant
+Xdebug-1.03
+Xerox
+Xfig
+XFree86-1.1
+xinetd
+xlock
+Xnet
+xpp
+XSkat
+YPL-1.0
+YPL-1.1
+Zed
+Zend-2.0
+Zimbra-1.3
+Zimbra-1.4
+Zlib
+zlib-acknowledgement
+ZPL-1.1
+ZPL-2.0
+ZPL-2.1
+389-exception
+Asterisk-exception
+Autoconf-exception-2.0
+Autoconf-exception-3.0
+Autoconf-exception-generic
+Autoconf-exception-macro
+Bison-exception-2.2
+Bootloader-exception
+Classpath-exception-2.0
+CLISP-exception-2.0
+cryptsetup-OpenSSL-exception
+DigiRule-FOSS-exception
+eCos-exception-2.0
+Fawkes-Runtime-exception
+FLTK-exception
+Font-exception-2.0
+freertos-exception-2.0
+GCC-exception-2.0
+GCC-exception-3.1
+GNAT-exception
+gnu-javamail-exception
+GPL-3.0-interface-exception
+GPL-3.0-linking-exception
+GPL-3.0-linking-source-exception
+GPL-CC-1.0
+GStreamer-exception-2005
+GStreamer-exception-2008
+i2p-gpl-java-exception
+KiCad-libraries-exception
+LGPL-3.0-linking-exception
+libpri-OpenH323-exception
+Libtool-exception
+Linux-syscall-note
+LLGPL
+LLVM-exception
+LZMA-exception
+mif-exception
+Nokia-Qt-exception-1.1
+OCaml-LGPL-linking-exception
+OCCT-exception-1.0
+OpenJDK-assembly-exception-1.0
+openvpn-openssl-exception
+PS-or-PDF-font-exception-20170817
+QPL-1.0-INRIA-2004-exception
+Qt-GPL-exception-1.0
+Qt-LGPL-exception-1.1
+Qwt-exception-1.0
+SHL-2.0
+SHL-2.1
+SWI-exception
+Swift-exception
+u-boot-exception-2.0
+Universal-FOSS-exception-1.0
+vsftpd-openssl-exception
+WxWindows-exception-3.1
+x11vnc-openssl-exception
+endef
diff --git a/support/misc/cyclonedx.mk b/support/misc/cyclonedx.mk
index 3906a3b60a..1d7199c92c 100644
--- a/support/misc/cyclonedx.mk
+++ b/support/misc/cyclonedx.mk
@@ -7,6 +7,8 @@
 #
 ################################################################################

+include support/misc/cyclonedx-spdx.mk
+
 # Note: to avoid conflict with <pkg>_VERSION `_SPEC` is added
 CYCLONEDX_VERSION_SPEC = 1.5

@@ -24,6 +26,22 @@ CYCLONEDX_VERSION_SPEC = 1.5
 # Turns "Public%20Domain,%20GPL-2.0" into "Public%20Domain GPL-2.0"
 _cyclonedx-licenses-as-list = $(subst $(comma)%20,$(space),$(1))

+# _cyclonedx-license-attribute -- according to CycloneDX spec correct SPDX
+#                                 licenses must use the 'id' key while
+#                                 other use the 'name' key.
+#                                 If a SPDX license is followed by parenthesis
+#                                 to describe its scope it will be threated as
+#                                 a non SPDX license.
+#
+# $(1): a license name with space encoded. Since all official SPDX license names
+#       are a single word (no spaces), it's not an issue to keep them url-encoded.
+define _cyclonedx-license-attribute
+	$(if $(filter $(spdx),$(1)), \
+		"id", \
+		"name" \
+	)
+endef
+
 # _cyclonedx-license -- create an entry of a cyclonedx component license list
 #
 # For more information on license object see
@@ -33,7 +51,8 @@ _cyclonedx-licenses-as-list = $(subst $(comma)%20,$(space),$(1))
 define _cyclonedx-license
 	{
 		"license": {
-			"name": $(call mk-json-str,$(1))
+			$(call _cyclonedx-license-attribute,$(1)):
+				$(call mk-json-str,$(1))
 		}
 	},
 endef
@@ -195,3 +214,17 @@ define cyclonedx-json
 		}
 	})
 endef
+
+# Use this rule to update the `cyclonedx-spdx.mk` file. The rule will
+# override the cyclonedx-spdx.mk file with a variable called 'spdx' that
+# contains the list of the SPDX license supported by CycloneDX spec.
+.PHONY: support/misc/cyclonedx-spdx.mk
+support/misc/cyclonedx-spdx.mk:
+	$(WGET) -O - https://raw.githubusercontent.com/CycloneDX/specification/$(CYCLONEDX_VERSION_SPEC)/schema/spdx.schema.json | \
+		$(JQ) jq -r '.enum[]' | { \
+			echo '# List of approved SPDX license'; \
+			echo '# See https://raw.githubusercontent.com/CycloneDX/specification/1.5/schema/spdx.schema.json'; \
+			echo 'define spdx'; \
+			cat; \
+			echo 'endef'; \
+		} > $@
--
2.44.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [RFC PATCH 5/5] Makefile: add command to generate SBOM in CycloneDX format

[Thomas Perale via buildroot]

This patch adds a new command to generate a JSON SBOM in the CycloneDX
format based on the packages selected in the dot config.

Usage:

  make cyclonedx

Signed-off-by: Thomas Perale <thomas.perale@mind.be>
---
 Makefile | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/Makefile b/Makefile
index 91973cca60..43344673c0 100644
--- a/Makefile
+++ b/Makefile
@@ -155,6 +155,8 @@ MAKEOVERRIDES :=

 # Include some helper macros and variables
 include support/misc/utils.mk
+# Include CycloneDX SBOM support
+include support/misc/cyclonedx.mk

 # Set variables related to in-tree or out-of-tree build.
 # Here, both $(O) and $(CURDIR) are absolute canonical paths.
@@ -924,6 +926,16 @@ show-info:
 		) \
 	)

+.PHONY: cyclonedx
+cyclonedx:
+	@:
+	$(info $(call cyclonedx-json, \
+			$(sort $(foreach p,$(PACKAGES) $(TARGETS_ROOTFS), \
+				$(p) $($(call UPPERCASE,$(p))_FINAL_RECURSIVE_DEPENDENCIES)) \
+			) \
+		) \
+	)
+
 .PHONY: pkg-stats
 pkg-stats:
 	@cd "$(CONFIG_DIR)" ; \
@@ -1185,6 +1197,7 @@ help:
 	@echo '  source                 - download all sources needed for offline-build'
 	@echo '  external-deps          - list external packages used'
 	@echo '  legal-info             - generate info about license compliance'
+	@echo '  cyclonedx              - generate info about packages, as a CycloneDX formatted json blurb'
 	@echo '  show-info              - generate info about packages, as a JSON blurb'
 	@echo '  pkg-stats              - generate info about packages as JSON and HTML'
 	@echo '  printvars              - dump internal variables selected with VARS=...'
--
2.44.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [RFC PATCH 0/5] Support SBOM in CycloneDX format

[Michael Nosthoff via buildroot]

Hi Thomas,

Great that you're taking up this topic. I'm highly interested because I'm currently generating my sbom "by hand" using a small python script which
converts the output of show-info. But yours is already better as it adds the dependency info and also fills the purl field (to the best of its knowledge).

Did you see the work of Robert (i added him in CC)? He submitted a script to the list some time ago which can run on the manifest.csv [0]. 
Maybe there are some synergies.

First feedback:
I created my SBOM using your patchset and put it into our dependency track instance. Looks promising.

One thing that I noticed: 
when running `make cyclonedx` I first get the output of wget which retrieves the cyclonedx schema. 
So I have to cut it from the output to have proper json. Maybe you want to silence this in some way.

I'll keep playing around and will give you further feedback if I'm noticing things.

Regards,
Michael

[0] https://github.com/CycloneDX/cyclonedx-buildroot

On Thursday, April 04, 2024 14:43 CEST, Thomas Perale via buildroot <buildroot@buildroot.org> wrote:

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [RFC PATCH 0/5] Support SBOM in CycloneDX format

[Thomas Perale via buildroot]

Hi Michael,

Thank you for taking the time checking my changes.

> Did you see the work of Robert (i added him in CC)? He submitted a script to the list some time ago which can run on the manifest.csv [0].
> Maybe there are some synergies.

I saw the work of Robert and used it as a reference to write the first 
iterations of this patch series but started working on a change for 
different reasons:

- Provide a solution that doesn't rely on python and external library to 
generate the SBOM. And a Makefile implementation just make sense to 
access all the package variable.
- Generating something based on `show-info` and `legal-info` just add 
more commands to run.

I would highly appreciate any contribution/reviews from Robert.

> One thing that I noticed:
> when running `make cyclonedx` I first get the output of wget which retrieves the cyclonedx schema.
> So I have to cut it from the output to have proper json. Maybe you want to silence this in some way.

Did you mean the `cyclondx-spdx` file from the 
`support/misc/cyclonedx-spdx.mk` rule ? It should not be ran by default 
and don't see why it's run ..

Regards,
PERALE Thomas

On 4/5/24 11:21 AM, Michael Nosthoff wrote:
> Hi Thomas,
>
> Great that you're taking up this topic. I'm highly interested because I'm currently generating my sbom "by hand" using a small python script which
> converts the output of show-info. But yours is already better as it adds the dependency info and also fills the purl field (to the best of its knowledge).
>
> Did you see the work of Robert (i added him in CC)? He submitted a script to the list some time ago which can run on the manifest.csv [0].
> Maybe there are some synergies.
>
> First feedback:
> I created my SBOM using your patchset and put it into our dependency track instance. Looks promising.
>
> One thing that I noticed:
> when running `make cyclonedx` I first get the output of wget which retrieves the cyclonedx schema.
> So I have to cut it from the output to have proper json. Maybe you want to silence this in some way.
>
> I'll keep playing around and will give you further feedback if I'm noticing things.
>
> Regards,
> Michael
>
> [0] https://github.com/CycloneDX/cyclonedx-buildroot
>
> On Thursday, April 04, 2024 14:43 CEST, Thomas Perale via buildroot <buildroot@buildroot.org> wrote:
>
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [RFC PATCH 0/5] Support SBOM in CycloneDX format

[Thomas Petazzoni via buildroot]

Hello Thomas,

On Thu,  4 Apr 2024 14:43:24 +0200
Thomas Perale <thomas.perale@essensium.com> wrote:

> This RFC patch series propose to add support for the CycloneDX
> SBOM format.
> 
> There is a growing need to generate SBOM from buildroot
> configurations. Right now, there are different solutions available
> for buildroot users `show-info`, `legal-info` and `pkg-stats`.
> They all generate similar information (`show-info` showing more) but
> in a format that is specific to buildroot.

Thanks a lot for your work on this, this is really useful.

My initial reaction is whether it is really the right solution to have
"make" spit out the CycloneDX format, or whether we should use "make
show-info", and then process the JSON output using some Python script
to generate the CycloneDX SBoM. Some of the mangling needed to generate
the CycloneDX stuff is a bit tricky to write in make, and having a
utility Python script doing that work based on a JSON input would I
believe be simpler and easier to extend. Have you explored this idea?

Of course, if "make show-info" doesn't provide enough information in
the generated JSON blurb, we can always extend that with more
information.

> This is a first sketch and I hope to gather comments on functionality
> the community want me to include. I already have a todo list of feature
> I plan to work on:
> 
> - [ ] Find a solution to handle versioning. The "version" property
>       should be incremented every SBOM generation.

Perhaps easier to handle this with the Python script approach proposed
above.

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [RFC PATCH 0/5] Support SBOM in CycloneDX format

[Yann E. MORIN]

Thomas², All,

On 2024-04-07 23:15 +0200, Thomas Petazzoni via buildroot spake thusly:
> On Thu,  4 Apr 2024 14:43:24 +0200
> Thomas Perale <thomas.perale@essensium.com> wrote:
> 
> > This RFC patch series propose to add support for the CycloneDX
> > SBOM format.
> > 
> > There is a growing need to generate SBOM from buildroot
> > configurations. Right now, there are different solutions available
> > for buildroot users `show-info`, `legal-info` and `pkg-stats`.
> > They all generate similar information (`show-info` showing more) but
> > in a format that is specific to buildroot.
> 
> Thanks a lot for your work on this, this is really useful.
> 
> My initial reaction is whether it is really the right solution to have
> "make" spit out the CycloneDX format, or whether we should use "make
> show-info", and then process the JSON output using some Python script
> to generate the CycloneDX SBoM. Some of the mangling needed to generate
> the CycloneDX stuff is a bit tricky to write in make, and having a
> utility Python script doing that work based on a JSON input would I
> believe be simpler and easier to extend. Have you explored this idea?
> 
> Of course, if "make show-info" doesn't provide enough information in
> the generated JSON blurb, we can always extend that with more
> information.

This was also my reaction: when I introduced show-info, the goal was to
expose details about the current configuration, so that it could be
digested by various scripts to generate the required SBOM formats.

I would prefer that we do get a support/script/cyclonedx script (python
or whatever) that gets show-info output and generates CycloneDX SBOM.

> > This is a first sketch and I hope to gather comments on functionality
> > the community want me to include. I already have a todo list of feature
> > I plan to work on:
> > 
> > - [ ] Find a solution to handle versioning. The "version" property
> >       should be incremented every SBOM generation.
> 
> Perhaps easier to handle this with the Python script approach proposed
> above.

Also I don't see how that versioning works.

If it gets incremented everytime the SBOM is generated, then (with the
current state of this series):

    git clone blablabla/buildroot
    make -C buildroot foo_defconfig
    make -C buildroot cyclonedx
    make -C buildroot cyclonedx

would yield two different versions, although nothing changed in-between.

Also:

    git clone blablabla/buildroot
    make -C buildroot foo_defconfig
    make -C buildroot
    make -C buildroot cyclonedx
    rm -rf buildroot
    git clone blablabla/buildroot
    make -C buildroot foo_defconfig
    make -C buildroot
    make -C buildroot cyclonedx

would have to yield two different versions, but where would the "latest"
version be stored? Certainly not in the user's home directory.

And arguably, if two different people build the cyclonedx SBOM for the
same project, on two different machines (with an otherwise identical
environment), at different time, then how do they know the version used
by the other? This also applies to "two CI pipelines" instead of "two
users".

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [RFC PATCH 0/5] Support SBOM in CycloneDX format

[Arnout Vandecappelle via buildroot]

On 08/04/2024 21:15, Yann E. MORIN wrote:
> Thomas², All,
> 
> On 2024-04-07 23:15 +0200, Thomas Petazzoni via buildroot spake thusly:
>> On Thu,  4 Apr 2024 14:43:24 +0200
>> Thomas Perale <thomas.perale@essensium.com> wrote:
>>
>>> This RFC patch series propose to add support for the CycloneDX
>>> SBOM format.
>>>
>>> There is a growing need to generate SBOM from buildroot
>>> configurations. Right now, there are different solutions available
>>> for buildroot users `show-info`, `legal-info` and `pkg-stats`.
>>> They all generate similar information (`show-info` showing more) but
>>> in a format that is specific to buildroot.
>>
>> Thanks a lot for your work on this, this is really useful.
>>
>> My initial reaction is whether it is really the right solution to have
>> "make" spit out the CycloneDX format, or whether we should use "make
>> show-info", and then process the JSON output using some Python script
>> to generate the CycloneDX SBoM. Some of the mangling needed to generate
>> the CycloneDX stuff is a bit tricky to write in make, and having a
>> utility Python script doing that work based on a JSON input would I
>> believe be simpler and easier to extend. Have you explored this idea?
>>
>> Of course, if "make show-info" doesn't provide enough information in
>> the generated JSON blurb, we can always extend that with more
>> information.
> 
> This was also my reaction: when I introduced show-info, the goal was to
> expose details about the current configuration, so that it could be
> digested by various scripts to generate the required SBOM formats.
> 
> I would prefer that we do get a support/script/cyclonedx script (python
> or whatever) that gets show-info output and generates CycloneDX SBOM.

  This was (of course) on my instigation.

  My premise is that we should (in the longer term) _always_ generate the 
CycloneDX SBoM as part of the world target, so without needing a separate "make 
cyclonedx" step. I think we need to kind of force people to generate SBoMs, so 
there is no excuse not to have it.

  This has a number of implications: it has to be quite lightweight (because it 
is called for every build), so that basically limits it so make and shell (I 
didn't like the idea of "always build host-python"). It should also be possible 
to generate it without internet connection on condition that a 'make source' was 
called before (that's why the license list in support/misc/cyclonedx-spdx.mk is 
stored in the git repo and not downloaded while building). On the other hand, 
the cyclonedx generation can depend on the presence of downloads and build 
artifacts, if that is needed.

  On that background, with the addition of cyclonedx, we now have 5 things that 
do somewhat similar things: cyclonedx, legal-info, show-info, pkg-stats, and 
printvars. So a second goal is to try to unify all that in a single mechanism. 
And because cyclonedx is a standard, it makes sense to me to try to use that as 
the primary mechanism, and make the others derivations from it.

- For show-info: as long as the same information is available in the cyclonedx 
file, we can simply use the cyclonedx file and deprecate show-info (possibly 
with a conversion script, which is allowed to be written in python, but might 
even be possible to write with jq, to convert to the legacy format).
- For legal-info, it's quite obvious that cyclonedx should basically replace 
that - though legal-info currently still does a number of additional steps (e.g. 
collect the actual source) that cyclonedx doesn't cover. I also think the 
manifest.csv file makes a lot of sense. I'm not sure how to solve that, but I 
can imagine that there could still be an explicit legal-info target that 
consumes the cyclonedx file and does some additional steps.
- For pkg-stats, it turns out that DependencyTrack already covers a large number 
of features that we have in there (CVEs, obviously, but also upstream version). 
Some very buildroot-specific things are not in there (e.g. the package's infra) 
but it could be possible to add those to DependencyTrack.
- For printvars I have no solution, I guess it will have to stay.


  So, why is it not a Python script based on show-info? Because I want the 
CycloneDX file to be _always_ generated, which means no Python. And processing 
JSON in shell is horror, then it's better to just generate it twice.



>>> This is a first sketch and I hope to gather comments on functionality
>>> the community want me to include. I already have a todo list of feature
>>> I plan to work on:
>>>
>>> - [ ] Find a solution to handle versioning. The "version" property
>>>        should be incremented every SBOM generation.
>>
>> Perhaps easier to handle this with the Python script approach proposed
>> above.
> 
> Also I don't see how that versioning works.

  In my understanding, the version should be updated every time you edit a 
CycloneDX file. So we always generate version 1. It is updated by tools that 
_process_ CycloneDX, e.g. to annotate with additional information.


> If it gets incremented everytime the SBOM is generated, then (with the
> current state of this series):
> 
>      git clone blablabla/buildroot
>      make -C buildroot foo_defconfig
>      make -C buildroot cyclonedx
>      make -C buildroot cyclonedx
> 
> would yield two different versions, although nothing changed in-between.

  So this would generate the same version 1 both times.


> Also:
> 
>      git clone blablabla/buildroot
>      make -C buildroot foo_defconfig
>      make -C buildroot
>      make -C buildroot cyclonedx
>      rm -rf buildroot
>      git clone blablabla/buildroot
>      make -C buildroot foo_defconfig
>      make -C buildroot
>      make -C buildroot cyclonedx
> 
> would have to yield two different versions, but where would the "latest"
> version be stored? Certainly not in the user's home directory.

  IMHO the cyclonedx file should be bit-for-bit reproducible, so this must 
_definitely_ generate the same version.


  Regards,
  Arnout

> And arguably, if two different people build the cyclonedx SBOM for the
> same project, on two different machines (with an otherwise identical
> environment), at different time, then how do they know the version used
> by the other? This also applies to "two CI pipelines" instead of "two
> users".
> 
> Regards,
> Yann E. MORIN.
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [RFC PATCH 0/5] Support SBOM in CycloneDX format

[Yann E. MORIN]

Arnout, All,

On 2024-04-09 14:17 +0200, Arnout Vandecappelle spake thusly:
> On 08/04/2024 21:15, Yann E. MORIN wrote:
> > On 2024-04-07 23:15 +0200, Thomas Petazzoni via buildroot spake thusly:
> > > On Thu,  4 Apr 2024 14:43:24 +0200
> > > Thomas Perale <thomas.perale@essensium.com> wrote:
> > > > This RFC patch series propose to add support for the CycloneDX
> > > > SBOM format.
[--SNIP--]

> > On 2024-04-07 23:15 +0200, Thomas Petazzoni via buildroot spake thusly:
> > > My initial reaction is whether it is really the right solution to have
> > > "make" spit out the CycloneDX format, or whether we should use "make
> > > show-info", and then process the JSON output using some Python script
> > > to generate the CycloneDX SBoM.
[--SNIP--]

> On 08/04/2024 21:15, Yann E. MORIN wrote:
> > This was also my reaction: when I introduced show-info,
[--SNIP--]

On 2024-04-09 14:17 +0200, Arnout Vandecappelle spake thusly:
>  My premise is that we should (in the longer term) _always_ generate the
> CycloneDX SBoM as part of the world target, so without needing a separate
> "make cyclonedx" step. I think we need to kind of force people to generate
> SBoMs, so there is no excuse not to have it.

An SBOM is totally worthless when generated on the developers machines,
as the build environment there is uncontrollable; it is useless during
development as well [0]. An SBOM only ever makes sense when generated in
a delivery pipeline, like in a CI/CD pipeline.

[0] except in the rare cases of those developpers explicitly looking at
or working on the SBOM: evaluating the SBOM, fixing bugs in the SBOM
generation tooling, adding SBOM-related metadata...

>  This has a number of implications: it has to be quite lightweight (because
> it is called for every build), so that basically limits it so make and shell
> (I didn't like the idea of "always build host-python"). It should also be
> possible to generate it without internet connection on condition that a
> 'make source' was called before (that's why the license list in
> support/misc/cyclonedx-spdx.mk is stored in the git repo and not downloaded
> while building). On the other hand, the cyclonedx generation can depend on
> the presence of downloads and build artifacts, if that is needed.
> 
>  On that background, with the addition of cyclonedx, we now have 5 things
> that do somewhat similar things: cyclonedx, legal-info, show-info,
> pkg-stats, and printvars.

Forget about printvars: it is unusable and legacy. We have show-vars
instead, which does provide a reliable extract of the internal
variables (except for those that need a $(@) or $(@D) or such, because
there is no actual target in that case, and thus it is not possible to
properly extract the internal state of variables).

We should now drop printvars.

> So a second goal is to try to unify all that in a
> single mechanism. And because cyclonedx is a standard, it makes sense to me
> to try to use that as the primary mechanism, and make the others derivations
> from it.

The nice things about standards, is that there are so many to pick from.
;-)

SPDX is also an SBOM standard, which predates cyclonedx by 6 years. It
is more mature (older!), so why don't we settle on that instead?

> - For show-info: as long as the same information is available in the
> cyclonedx file, we can simply use the cyclonedx file and deprecate show-info
> (possibly with a conversion script, which is allowed to be written in
> python, but might even be possible to write with jq, to convert to the
> legacy format).

I strongly disagree. show-info has other usages than generating an SBOM,
and contains a lot of info that does not belong in a SBOM. For example,
as you noticed, the package infra is mising in cyclonedx, and there is
no reason to add it there, while it has its place in show-info.

Another example, is that (AFAICS) there is no dedicated way to describe
reverse depdndencies in cylonedx, except by injected an arbitrary
property, which means that it does not really fit... show-info provides
that.

More generally: there is information that belongs to show-info, as it is
usefull for generating various reports, that have absolutely no meaning
in an SBOM, and so there is no reason to inject those in cyclonedx just
because something else may need it. That's what show-info is for.

And as Thomas said, if show-info is missing data, we can easily add it.

> - For legal-info, it's quite obvious that cyclonedx should basically replace
> that - though legal-info currently still does a number of additional steps
> (e.g. collect the actual source) that cyclonedx doesn't cover. I also think
> the manifest.csv file makes a lot of sense. I'm not sure how to solve that,
> but I can imagine that there could still be an explicit legal-info target
> that consumes the cyclonedx file and does some additional steps.

Or the other way around: cyclonedx consumes the output of legal-info.

> - For pkg-stats, it turns out that DependencyTrack already covers a large
> number of features that we have in there (CVEs, obviously, but also upstream
> version). Some very buildroot-specific things are not in there (e.g. the
> package's infra) but it could be possible to add those to DependencyTrack.
> - For printvars I have no solution, I guess it will have to stay.

See above: we should drop it in favour of show-vars. But yes, this wil
have to stay, indeed.

>  So, why is it not a Python script based on show-info? Because I want the
> CycloneDX file to be _always_ generated, which means no Python. And
> processing JSON in shell is horror, then it's better to just generate it
> twice.

As I said above: it does not make sense to always generate it, because
it is only relevant when it is generated in a controlled environment,
like a delivery pipeline for a delivery tag (or such), and not during
development on developers machines.

[--SNIP--]
> > > > - [ ] Find a solution to handle versioning. The "version" property
> > > >        should be incremented every SBOM generation.
> > Also I don't see how that versioning works.
>  In my understanding, the version should be updated every time you edit a
> CycloneDX file. So we always generate version 1. It is updated by tools that
> _process_ CycloneDX, e.g. to annotate with additional information.

Ah, so there are tools that have to run on the cyclonedx data after it
has been generated, because they need to inject data we have no way to
know in Buildroot?

This means that those tools can very well consume show-info to generate
the final cyclonedx, to begin with...

>  IMHO the cyclonedx file should be bit-for-bit reproducible, so this must
> _definitely_ generate the same version.

I do agree that it should be reproducible, but then I don't see how this
articulates around the "version property should be incremented every
SBOM generation."

Note that, if the version really has to be incremented every time the
cyclonedx data is generated, we could just use the UNIX timestamp, with
milli- or microsecond precision, which is in practice going to be
monotonically incrementing. It's not going to be a sequential +1 every
time, though...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [RFC PATCH 0/5] Support SBOM in CycloneDX format

[Arnout Vandecappelle via buildroot]

On 10/04/2024 19:21, Yann E. MORIN wrote:
> Arnout, All,
> 
> On 2024-04-09 14:17 +0200, Arnout Vandecappelle spake thusly:
>> On 08/04/2024 21:15, Yann E. MORIN wrote:
>>> On 2024-04-07 23:15 +0200, Thomas Petazzoni via buildroot spake thusly:
>>>> On Thu,  4 Apr 2024 14:43:24 +0200
>>>> Thomas Perale <thomas.perale@essensium.com> wrote:
>>>>> This RFC patch series propose to add support for the CycloneDX
>>>>> SBOM format.
> [--SNIP--]
> 
>>> On 2024-04-07 23:15 +0200, Thomas Petazzoni via buildroot spake thusly:
>>>> My initial reaction is whether it is really the right solution to have
>>>> "make" spit out the CycloneDX format, or whether we should use "make
>>>> show-info", and then process the JSON output using some Python script
>>>> to generate the CycloneDX SBoM.
> [--SNIP--]
> 
>> On 08/04/2024 21:15, Yann E. MORIN wrote:
>>> This was also my reaction: when I introduced show-info,
> [--SNIP--]
> 
> On 2024-04-09 14:17 +0200, Arnout Vandecappelle spake thusly:
>>   My premise is that we should (in the longer term) _always_ generate the
>> CycloneDX SBoM as part of the world target, so without needing a separate
>> "make cyclonedx" step. I think we need to kind of force people to generate
>> SBoMs, so there is no excuse not to have it.
> 
> An SBOM is totally worthless when generated on the developers machines,
> as the build environment there is uncontrollable; it is useless during
> development as well [0]. An SBOM only ever makes sense when generated in
> a delivery pipeline, like in a CI/CD pipeline.

  Huh? It's equally worthless as the images created by Buildroot then. Those are 
also uncontrollable. Of course, you have no guarantee that the SBoM corresponds 
to the image that gets shipped to customers, but that's not really relevant. You 
can still use the SBoM to see which packages get installed, which patches are 
applied, which CVEs apply, which licenses are used, etc.

  And anyway, I don't think it's very relevant if this SBoM generation happens 
on a developer machine or in CI. In either case, I don't think we want to rely 
on a host-installed Python version.


> [0] except in the rare cases of those developpers explicitly looking at
> or working on the SBOM: evaluating the SBOM, fixing bugs in the SBOM
> generation tooling, adding SBOM-related metadata...

  So you're saying that for developers who do not use the SBoM, it's not 
relevant to generate them locally. That's true of course. Not very relevant IMHO 
though.



>>   This has a number of implications: it has to be quite lightweight (because
>> it is called for every build), so that basically limits it so make and shell
>> (I didn't like the idea of "always build host-python"). It should also be
>> possible to generate it without internet connection on condition that a
>> 'make source' was called before (that's why the license list in
>> support/misc/cyclonedx-spdx.mk is stored in the git repo and not downloaded
>> while building). On the other hand, the cyclonedx generation can depend on
>> the presence of downloads and build artifacts, if that is needed.
>>
>>   On that background, with the addition of cyclonedx, we now have 5 things
>> that do somewhat similar things: cyclonedx, legal-info, show-info,
>> pkg-stats, and printvars.
> 
> Forget about printvars: it is unusable and legacy. We have show-vars
> instead, which does provide a reliable extract of the internal
> variables (except for those that need a $(@) or $(@D) or such, because
> there is no actual target in that case, and thus it is not possible to
> properly extract the internal state of variables).
> 
> We should now drop printvars.

  OMG, I completely forgot about show-vars! But indeed, we still need to keep 
show-vars. And we should drop printvars.


>> So a second goal is to try to unify all that in a
>> single mechanism. And because cyclonedx is a standard, it makes sense to me
>> to try to use that as the primary mechanism, and make the others derivations
>> from it.
> 
> The nice things about standards, is that there are so many to pick from.
> ;-)
> 
> SPDX is also an SBOM standard, which predates cyclonedx by 6 years. It
> is more mature (older!), so why don't we settle on that instead?

  Because SPDX is horribly messy and complicated.


>> - For show-info: as long as the same information is available in the
>> cyclonedx file, we can simply use the cyclonedx file and deprecate show-info
>> (possibly with a conversion script, which is allowed to be written in
>> python, but might even be possible to write with jq, to convert to the
>> legacy format).
> 
> I strongly disagree. show-info has other usages than generating an SBOM,
> and contains a lot of info that does not belong in a SBOM. For example,
> as you noticed, the package infra is mising in cyclonedx, and there is
> no reason to add it there, while it has its place in show-info.

  Excellent example :-). The SBoM JSON (or show-info BTW) is not really usable 
for a human as is. So you'll want a tool to visualize it. You can make your own 
tool (pkg-stats), or you can use something that exist already (DependencyTrack). 
In the latter case, you want to make sure that that tool displays all relevant 
information - so the SBoM needs to contain that information. Fortunately, 
CycloneDX allows to add additional properties. Now we just need to extend 
DependencyTrack to display that information. Thomas Perale is looking at that.


> Another example, is that (AFAICS) there is no dedicated way to describe
> reverse depdndencies in cylonedx, except by injected an arbitrary
> property, which means that it does not really fit... show-info provides
> that.

  True.

> More generally: there is information that belongs to show-info, as it is
> usefull for generating various reports, that have absolutely no meaning
> in an SBOM,

  What kind of information would that be? show-info and an SBoM are pretty much 
the same thing: a list of packages and their properties.

> and so there is no reason to inject those in cyclonedx just
> because something else may need it. That's what show-info is for
  Anyway, for me replacing show-info is not the most important thing, it's more 
a nice-to-have. What I don't want is to generate CycloneDX by post-processing 
the show-info output, because it turns out that that is still fairly complicated 
and it's just easier to directly generate CycloneDX from make.


> And as Thomas said, if show-info is missing data, we can easily add it.
> 
>> - For legal-info, it's quite obvious that cyclonedx should basically replace
>> that - though legal-info currently still does a number of additional steps
>> (e.g. collect the actual source) that cyclonedx doesn't cover. I also think
>> the manifest.csv file makes a lot of sense. I'm not sure how to solve that,
>> but I can imagine that there could still be an explicit legal-info target
>> that consumes the cyclonedx file and does some additional steps.
> 
> Or the other way around: cyclonedx consumes the output of legal-info.

  Again, it's going to be much easier to generate everything from make rather 
than post-processing the output of legal-info and show-info.


>> - For pkg-stats, it turns out that DependencyTrack already covers a large
>> number of features that we have in there (CVEs, obviously, but also upstream
>> version). Some very buildroot-specific things are not in there (e.g. the
>> package's infra) but it could be possible to add those to DependencyTrack.
>> - For printvars I have no solution, I guess it will have to stay.
> 
> See above: we should drop it in favour of show-vars. But yes, this wil
> have to stay, indeed.
> 
>>   So, why is it not a Python script based on show-info? Because I want the
>> CycloneDX file to be _always_ generated, which means no Python. And
>> processing JSON in shell is horror, then it's better to just generate it
>> twice.
> 
> As I said above: it does not make sense to always generate it, because
> it is only relevant when it is generated in a controlled environment,
> like a delivery pipeline for a delivery tag (or such), and not during
> development on developers machines.
> 
> [--SNIP--]
>>>>> - [ ] Find a solution to handle versioning. The "version" property
>>>>>         should be incremented every SBOM generation.
>>> Also I don't see how that versioning works.
>>   In my understanding, the version should be updated every time you edit a
>> CycloneDX file. So we always generate version 1. It is updated by tools that
>> _process_ CycloneDX, e.g. to annotate with additional information.
> 
> Ah, so there are tools that have to run on the cyclonedx data after it
> has been generated, because they need to inject data we have no way to
> know in Buildroot?

  I don't know if those tools actually exist. I'm just saying that the version 
field is supposed to be updated by such tools.


> This means that those tools can very well consume show-info to generate
> the final cyclonedx, to begin with...

  I am _very_ sure that no such tool exists :-)


>>   IMHO the cyclonedx file should be bit-for-bit reproducible, so this must
>> _definitely_ generate the same version.
> 
> I do agree that it should be reproducible, but then I don't see how this
> articulates around the "version property should be incremented every
> SBOM generation."

  The statement "version property should be incremented every SBOM generation." 
from Thomas Perale is simply wrong (at least, AFAIU). It has to be incremented 
every time the SBoM is updated.

  There is also a UUID (serialNumber) field defined [1]. That one is supposed to 
be re-generated every time the SBoM is generated (but it can use any kind of 
UUID generation mechanism, e.g. random or time based). I don't like that very 
much though, because that makes the SBoM not reproducible.

  Regards,
  Arnout

[1] https://cyclonedx.org/docs/1.5/json/#serialNumber

> Note that, if the version really has to be incremented every time the
> cyclonedx data is generated, we could just use the UNIX timestamp, with
> milli- or microsecond precision, which is in practice going to be
> monotonically incrementing. It's not going to be a sequential +1 every
> time, though...
> 
> Regards,
> Yann E. MORIN.
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [RFC PATCH 0/5] Support SBOM in CycloneDX format

[Thomas Petazzoni via buildroot]

Hello,

On Wed, 10 Apr 2024 21:26:09 +0200
Arnout Vandecappelle <arnout@mind.be> wrote:

> > An SBOM is totally worthless when generated on the developers machines,
> > as the build environment there is uncontrollable; it is useless during
> > development as well [0]. An SBOM only ever makes sense when generated in
> > a delivery pipeline, like in a CI/CD pipeline.  
> 
>   Huh? It's equally worthless as the images created by Buildroot then. Those are 
> also uncontrollable. Of course, you have no guarantee that the SBoM corresponds 
> to the image that gets shipped to customers, but that's not really relevant. You 
> can still use the SBoM to see which packages get installed, which patches are 
> applied, which CVEs apply, which licenses are used, etc.

Yes, I agree the argument from Yann here is a bit dubious, but do we
care?

>   And anyway, I don't think it's very relevant if this SBoM generation happens 
> on a developer machine or in CI. In either case, I don't think we want to rely 
> on a host-installed Python version.

Why?

> > We should now drop printvars.  
> 
>   OMG, I completely forgot about show-vars! But indeed, we still need to keep 
> show-vars. And we should drop printvars.

We don't care about backward compatibility?

> > Another example, is that (AFAICS) there is no dedicated way to describe
> > reverse depdndencies in cylonedx, except by injected an arbitrary
> > property, which means that it does not really fit... show-info provides
> > that.  
> 
>   True.

So, how do you propose that CycloneDX replaces show-info, if CycloneDX
cannot represent all information that is emitted today in show-info?

> > More generally: there is information that belongs to show-info, as it is
> > usefull for generating various reports, that have absolutely no meaning
> > in an SBOM,  
> 
>   What kind of information would that be? show-info and an SBoM are pretty much 
> the same thing: a list of packages and their properties.

Well, see above?

> > and so there is no reason to inject those in cyclonedx just
> > because something else may need it. That's what show-info is for  
>   Anyway, for me replacing show-info is not the most important thing, it's more 
> a nice-to-have. What I don't want is to generate CycloneDX by post-processing 
> the show-info output, because it turns out that that is still fairly complicated 
> and it's just easier to directly generate CycloneDX from make.

Why is it "fairly complicated"? It sounds to me that post-processing
show-info into a simple and nice Python script to generate CycloneDX is
not at all "fairly complicated" but rather "fairly simpler" than doing
it in make.

Even from a pure architecture point of view, it makes a lot of sense to
have a Buildroot-specific show-info emitted a JSON blurb with as much
information as we want (we don't need to comply with any standard), and
*then* have post-processing tool consume this Buildroot-specific JSON
blurb and turn it into CycloneDX, SPDX, whatever people want, by using
the relevant information provided in the Buildroot-specific JSON blurb.

So I'm sorry, but your argumentation is not convincing at all :-/

> > And as Thomas said, if show-info is missing data, we can easily add it.
> >   
> >> - For legal-info, it's quite obvious that cyclonedx should basically replace
> >> that - though legal-info currently still does a number of additional steps
> >> (e.g. collect the actual source) that cyclonedx doesn't cover. I also think
> >> the manifest.csv file makes a lot of sense. I'm not sure how to solve that,
> >> but I can imagine that there could still be an explicit legal-info target
> >> that consumes the cyclonedx file and does some additional steps.  
> > 
> > Or the other way around: cyclonedx consumes the output of legal-info.  
> 
>   Again, it's going to be much easier to generate everything from make rather 
> than post-processing the output of legal-info and show-info.

Do you have some real argument to back this affirmation? Python code
is typically much more readable than make code, and it's easier to do
text manipulation in Python than in make.

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [RFC PATCH 0/5] Support SBOM in CycloneDX format

[Yann E. MORIN]

Thomas, All,

On 2024-04-10 22:10 +0200, Thomas Petazzoni spake thusly:
> On Wed, 10 Apr 2024 21:26:09 +0200
> Arnout Vandecappelle <arnout@mind.be> wrote:
> > > An SBOM is totally worthless when generated on the developers machines,
> > > as the build environment there is uncontrollable; it is useless during
> > > development as well [0]. An SBOM only ever makes sense when generated in
> > > a delivery pipeline, like in a CI/CD pipeline.  
> >   Huh? It's equally worthless as the images created by Buildroot then. Those are 
> > also uncontrollable. Of course, you have no guarantee that the SBoM corresponds 
> > to the image that gets shipped to customers, but that's not really relevant. You 
> > can still use the SBoM to see which packages get installed, which patches are 
> > applied, which CVEs apply, which licenses are used, etc.
> Yes, I agree the argument from Yann here is a bit dubious, but do we
> care?

What I meant here, is that the environment on a developper's machine is
unknown, so there is no way one can produce an SBOm there, that is
meaningful. If the developper has an override-srcdir or a local.mk with
arbitrary stuff, then this is not reproducible, and this should not be
used to tag a release.

Where an SBOM is usefull, is when it is generated for a release, so that
it can traval along it. A release should be done in a constrained,
reproducible environment, like a CD pipeline.

> >   And anyway, I don't think it's very relevant if this SBoM generation happens 
> > on a developer machine or in CI. In either case, I don't think we want to rely 
> > on a host-installed Python version.
> Why?

Indeed, why? As far as I see, such a script would only need modules from
the stdlib (json, maybe subprocess, re, and similar). We'd just need to
state that Buildroot requires a python3.8+ on the host, which to be
honest would hugely help in simpifying our other scripts...

We indeed currently do not require such a python to be present, as
that's not part of our requirements.

> > > We should now drop printvars.  
> >   OMG, I completely forgot about show-vars! But indeed, we still need to keep 
> > show-vars. And we should drop printvars.
> We don't care about backward compatibility?

Yes, but printvars is riddled with issues, and show-vars as been around
for a while already...

> > > and so there is no reason to inject those in cyclonedx just
> > > because something else may need it. That's what show-info is for  
> >   Anyway, for me replacing show-info is not the most important thing, it's more 
> > a nice-to-have. What I don't want is to generate CycloneDX by post-processing 
> > the show-info output, because it turns out that that is still fairly complicated 
> > and it's just easier to directly generate CycloneDX from make.
> Why is it "fairly complicated"? It sounds to me that post-processing
> show-info into a simple and nice Python script to generate CycloneDX is
> not at all "fairly complicated" but rather "fairly simpler" than doing
> it in make.

Here 's even a little patch that would allow to write everything in
python, and generate the thign as part of the standard build process
(which I would not suggest, but for the sake of the argument):

    commit ccf4722ae0a9755aa73c9c1eaa2a6935b77d7a7c
    Author:     Yann E. MORIN <yann.morin.1998@free.fr>
    AuthorDate: Wed Apr 10 22:25:11 2024 +0200
    Commit:     Yann E. MORIN <yann.morin.1998@free.fr>
    CommitDate: Wed Apr 10 22:42:51 2024 +0200

        Makefile: cyclonedx from show-info ezpz

        Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>

    diff --git a/Makefile b/Makefile
    index d1caec63b5..344f72244a 100644
    --- a/Makefile
    +++ b/Makefile
    @@ -497,7 +497,7 @@ export BASE_DIR
     #
     ################################################################################

    -all: world
    +all: world sbom-all

     # Include legacy before the other things, because package .mk files
     # may rely on it.
    @@ -918,20 +918,37 @@ check-dependencies:
         @cd "$(CONFIG_DIR)"; \
         $(TOPDIR)/support/scripts/graph-depends -C

    +SHOW_INFO = \
    +    $(call clean-json, \
    +           { $(foreach p, \
    +                   $(sort $(foreach i,$(PACKAGES) $(TARGETS_ROOTFS), \
    +                                   $(i) \
    +                                   $($(call UPPERCASE,$(i))_FINAL_RECURSIVE_DEPENDENCIES) \
    +                           ) \
    +                   ), \
    +                   $(call json-info,$(call UPPERCASE,$(p)))$(comma) \
    +           ) } \
    +    ) \
    +
     .PHONY: show-info
     show-info:
         @:
    -    $(info $(call clean-json, \
    -                   { $(foreach p, \
    -                           $(sort $(foreach i,$(PACKAGES) $(TARGETS_ROOTFS), \
    -                                           $(i) \
    -                                           $($(call UPPERCASE,$(i))_FINAL_RECURSIVE_DEPENDENCIES) \
    -                                   ) \
    -                           ), \
    -                           $(call json-info,$(call UPPERCASE,$(p)))$(comma) \
    -                   ) } \
    -           ) \
    -    )
    +    # Use $(foreach...) like for show-vars, just in case
    +    $(foreach i,$(SHOW_INFO),$(info $(i)))
    +
    +sbom-all: sbom-buildroot sbom-cyclonedx
    +
    +.PHONY: $(O)/info.json
    +$(O)/info.json:
    +    $(file > $(@),$(SHOW_INFO))
    +
    +.PHONY: sbom-buildroot
    +sbom-buildroot: $(O)/sbom.json
    +    @:
    +
    +.PHONY: sbom-cyclonedx
    +sbom-cyclonedx: $(O)/sbom.json world legal-info
    +    $(TOPDIR)/support/scripts/cyclonedx --show-info=$(<) --legal-info=$(LEGAL_INFO_DIR)

     .PHONY: pkg-stats
     pkg-stats:

Also: badly tab-mangled, but that's just for the example, not meant to
be applied. Totally untested, although I already had the SHOW_INFO
variable implemented a while back already, and IIRC that was working.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot