[Buildroot] [PATCH 0/3] Add tainting support

[Buildroot] [PATCH 0/3] Add tainting support

[Adam Duskett]

Originally taken from Angelo Compagnucci's patch series:
https://patchwork.ozlabs.org/project/buildroot/list/?series=64340

When the original patch series was submitted, Buildroot was quite a bit
smaller! With almost 1,000 packages added since then and more and more external
package managers being available for various languages, it is not possible to
package every single dependency needed for some packages.

Indeed, looking at NPM, for example, some packages rely on tens, if not
hundreds, of others. While this could be better and is indeed quite bad, it is,
unfortunately, the reality we must deal with.

With this patch series, we add initial tainting support to Buildroot with the
added option to turn the support off by way of the BR2_DISABLE_TAINT_CHECKING
option. This option gives us the best of both worlds: A sane default and an
option for advanced users to turn off the check if they understand the risks
and can guarantee their build is reproducible.

This patch series has two significant benefits:
  - Taint checking paves the way for additional package managers to be
    incorporated into Buildroot while maintaining reproducible integrity for
    packages provided by Buildroot.

  - It tells the user their build is tainted and what packages they
    have selected are causing the taint.

  - It makes support easier. If a user has a build that is tainted and the
    the problem they are experiencing is with a tainted package; it is
    more straightforward to tell the user they are on their own.

Adam Duskett (3):
  Makefile: add tainting support
  docs/manual: add information about tainting
  package/nodejs: taint the build when using external modules

 Config.in                                |  9 +++++++++
 Makefile                                 | 17 +++++++++++++++++
 docs/manual/adding-packages-generic.adoc |  9 +++++++++
 docs/manual/legal-notice.adoc            | 24 ++++++++++++++++++++++++
 package/nodejs/Config.in                 |  3 +++
 package/nodejs/nodejs-src/nodejs-src.mk  |  1 +
 package/pkg-generic.mk                   | 19 +++++++++++++++++++
 7 files changed, 82 insertions(+)

-- 
2.41.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 1/3] Makefile: add tainting support

[Adam Duskett]

Packages that may harm the build reproducibility or licensing of a build
should declare the ${PKG_NAME}_TAINTS variable. If a package taints the build,
add its name to a list of tainting packages. The build ends with a warning
message if the tainting packages list is not empty. Moreover, legal info will
show a warning in the presence of a tainting package.

However, as pointed out by Yann, a user may be using a local repository for
their packages (I.E., NPM). Below are three examples:

BR2_PACKAGE_NODEJS_MODULES_ADDITIONAL="http://myserver/node-mods/VERSION/foo"
  - The user manages the repository and guarantees that it is reproducible.

BR2_PACKAGE_NODEJS_MODULES_ADDITIONAL="$(BR2_EXTERANL_MY_TREE_PATH)/mods/foo"
  - Reproducible by way of being in a git-versioned br2-external tree.

BR2_PACKAGE_NODEJS_MODULES_ADDITIONAL="foo@1.2.3"
  - Reproducible because the version is specified.

As there is no way of reading a user's mind, we introduce a new option:
BR2_DISABLE_TAINT_CHECKING, which does what the option name suggests: The
option allows a user to explicitly turn off taint checking if they want to do
so.

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
---
 Config.in              |  9 +++++++++
 Makefile               | 17 +++++++++++++++++
 package/pkg-generic.mk | 19 +++++++++++++++++++
 3 files changed, 45 insertions(+)

diff --git a/Config.in b/Config.in
index 556b6c2575..3fc2d67bd9 100644
--- a/Config.in
+++ b/Config.in
@@ -685,6 +685,15 @@ config BR2_GLOBAL_PATCH_DIR
 
 menu "Advanced"
 
+config BR2_DISABLE_TAINT_CHECKING
+	bool "Disable taint checking"
+	help
+	  By default, selecting and using package managers such as NPM sets
+	  the tainted build flag and display a warning message; however, if
+	  you are confident that what you are building is from a trusted
+	  source, such as a local package repository, then this option
+	  disables taint checking.
+
 config BR2_FORCE_HOST_BUILD
 	bool "Force the building of host dependencies"
 	help
diff --git a/Makefile b/Makefile
index 3e85d5ef09..053924c7c8 100644
--- a/Makefile
+++ b/Makefile
@@ -807,6 +807,18 @@ endif # merged /usr
 
 	touch $(TARGET_DIR)/usr
 
+.PHONY: check-tainted
+check-tainted:
+ifneq ($(BR2_DISABLE_TAINT_CHECKING),y)
+ifneq ($(BR2_TAINTED_BY),)
+	@echo "Your buildroot configuration is tainted by: $(BR2_TAINTED_BY)"
+else
+	@echo "Your buildroot configuration is not tainted"
+endif
+else
+	@echo "Taint checking disabled"
+endif
+
 # Note: this will run in the filesystem context, so will use a copy
 # of target/, not the real one, so the files are still available on
 # re-builds (foo-rebuild, etc...)
@@ -822,6 +834,10 @@ target-post-image: $(TARGETS_ROOTFS) target-finalize staging-finalize
 	@$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_IMAGE_SCRIPT)), \
 		$(call MESSAGE,"Executing post-image script $(s)"); \
 		$(EXTRA_ENV) $(s) $(BINARIES_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep))
+ifneq ($(BR2_DISABLE_TAINT_CHECKING),y)
+	$(if $(BR2_TAINTED_BY), @echo "WARNING: Your buildroot configuration is tainted by: $(BR2_TAINTED_BY)!")
+endif
+
 
 .PHONY: source
 source: $(foreach p,$(PACKAGES),$(p)-all-source)
@@ -1188,6 +1204,7 @@ help:
 	@echo '  source                 - download all sources needed for offline-build'
 	@echo '  external-deps          - list external packages used'
 	@echo '  legal-info             - generate info about license compliance'
+	@echo '  check-tainted          - check if any selected package harms build reproducibility or licensing'
 	@echo '  show-info              - generate info about packages, as a JSON blurb'
 	@echo '  pkg-stats              - generate info about packages as JSON and HTML'
 	@echo '  printvars              - dump internal variables selected with VARS=...'
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 289fa552b4..7ffb544300 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -1102,6 +1102,17 @@ else
 $(2)_KCONFIG_VAR = BR2_PACKAGE_$(2)
 endif
 
+ifneq ($(BR2_DISABLE_TAINT_CHECKING),y)
+ifdef $(2)_TAINTS
+ifeq ($$($$($(2)_KCONFIG_VAR)),y)
+BR2_TAINTED_BY+=$$($(2)_RAWNAME)
+endif
+ifeq ($$($(2)_TYPE),host)
+$$(error "Host package $(1) has $(2)_TAINTS set: not supported)
+endif
+endif
+endif
+
 # legal-info: declare dependencies and set values used later for the manifest
 ifneq ($$($(2)_LICENSE_FILES),)
 $(2)_MANIFEST_LICENSE_FILES = $$($(2)_LICENSE_FILES)
@@ -1141,6 +1152,14 @@ else
 	$(Q)$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$($(2)_RAWNAME),$$($(2)_BASENAME_RAW),$$($(2)_HASH_FILE),$$(F),$$($(2)_DIR)/$$(F),$$(call UPPERCASE,$(4)))$$(sep))
 endif # license files
 
+ifneq ($(BR2_DISABLE_TAINT_CHECKING),y)
+ifeq ($$(call qstrip,$$($(2)_TAINTS)),YES)
+ifeq ($$($$($(2)_KCONFIG_VAR)),y)
+	$(Q)$$(call legal-warning-pkg,$$($(2)_RAWNAME),unknown license for additional modules or dependencies)
+endif
+endif
+endif
+
 ifeq ($$($(2)_REDISTRIBUTE),YES)
 ifeq ($$($(2)_SITE_METHOD),local)
 # Packages without a tarball: don't save and warn
-- 
2.41.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 2/3] docs/manual: add information about tainting

[Adam Duskett]

Add documentation about the usage of LIBFOO_TAINTS and what the make target
"check-tainted" does. Also, add documentation about turning off taint
checking and a few scenarios of why a user would want to do so.

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
---
 docs/manual/adding-packages-generic.adoc |  9 +++++++++
 docs/manual/legal-notice.adoc            | 24 ++++++++++++++++++++++++
 2 files changed, 33 insertions(+)

diff --git a/docs/manual/adding-packages-generic.adoc b/docs/manual/adding-packages-generic.adoc
index 76b037f436..12083b07d5 100644
--- a/docs/manual/adding-packages-generic.adoc
+++ b/docs/manual/adding-packages-generic.adoc
@@ -460,6 +460,15 @@ not and can not work as people would expect it should:
   to let you know, and +not saved+ will appear in the +license files+ field
   of the manifest file for this package.
 
+* +LIBFOO_TAINTS+ should be set to YES if a package taints a Buildroot
+  configuration. A Buildroot configuration is tainted when a package uses
+  external dependencies for which Buildroot cannot recover licensing
+  information, such as using a package manager (e.g., NPM) during the build.
+  If a configuration is tainted, the licensing information produced by
+  +make legal-info+ may not be accurate. If you wish to turn off taint
+  checking, it is possible to do so by enabling the BR2_DISABLE_TAINT_CHECKING
+  option.
+
 * +LIBFOO_ACTUAL_SOURCE_TARBALL+ only applies to packages whose
   +LIBFOO_SITE+ / +LIBFOO_SOURCE+ pair points to an archive that does
   not actually contain source code, but binary code. This a very
diff --git a/docs/manual/legal-notice.adoc b/docs/manual/legal-notice.adoc
index 179aa6b179..55a2120f8e 100644
--- a/docs/manual/legal-notice.adoc
+++ b/docs/manual/legal-notice.adoc
@@ -72,6 +72,30 @@ some of the external toolchains and the Buildroot source code itself.
 When you run +make legal-info+, Buildroot produces warnings in the +README+
 file to inform you of relevant material that could not be saved.
 
+Furthermore, if a package uses custom external dependencies from the Buildroot
+tree, the configuration may be tainted. An example could be a package manager
+for a software stack that downloads the required dependencies while building a
+package, such as NPM. In such cases, Buildroot cannot check the licensing of
+the downloaded software and, thus, give accurate licensing information.
+However, it is possible to turn off taint checking by enabling the
+BR2_DISABLE_TAINT_CHECKING option. This option is helpful in cases where you
+can guarantee the reproducibility of the build. Here are three examples:
+
+BR2_PACKAGE_NODEJS_MODULES_ADDITIONAL="http://myserver/node-mods/VERSION/foo"
+  - The user manages the repository and guarantees that it is reproducible.
+
+BR2_PACKAGE_NODEJS_MODULES_ADDITIONAL="$(BR2_EXTERANL_MY_TREE_PATH)/mods/foo"
+  - Reproducible by way of being in a git-versioned br2-external tree.
+
+BR2_PACKAGE_NODEJS_MODULES_ADDITIONAL="foo@1.2.3"
+  - Reproducible because the version is specified.
+
+To check if your configuration is tainted, run:
+
+--------------------
+make check-tainted
+--------------------
+
 Finally, keep in mind that the output of +make legal-info+ is based on
 declarative statements in each of the packages recipes. The Buildroot
 developers try to do their best to keep those declarative statements as
-- 
2.41.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 3/3] package/nodejs: taint the build when using external modules

[Adam Duskett]

By default, there is no guarantee of reproducibility when a user enables
additional NPM packages. As Yann pointed out the last time this patch series
was submitted, there are several legitimate scenarios where enabling
additional NPM packages is reproducible. However, the
BR2_DISABLE_TAINT_CHECKING option addresses these concerns.

Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
---
 package/nodejs/Config.in                | 3 +++
 package/nodejs/nodejs-src/nodejs-src.mk | 1 +
 2 files changed, 4 insertions(+)

diff --git a/package/nodejs/Config.in b/package/nodejs/Config.in
index c684f8eb86..80c58b2e94 100644
--- a/package/nodejs/Config.in
+++ b/package/nodejs/Config.in
@@ -74,6 +74,9 @@ config BR2_PACKAGE_NODEJS_MODULES_ADDITIONAL
 	  version), the uglify-js module at 1.3.4, a module from a
 	  filesystem path, and a module from a git repository.
 
+	  Warning:
+	  By default, using this option taints the build!
+
 config BR2_PACKAGE_NODEJS_MODULES_ADDITIONAL_DEPS
 	string "Additional module dependencies"
 	help
diff --git a/package/nodejs/nodejs-src/nodejs-src.mk b/package/nodejs/nodejs-src/nodejs-src.mk
index 3452c93728..d04e0416e9 100644
--- a/package/nodejs/nodejs-src/nodejs-src.mk
+++ b/package/nodejs/nodejs-src/nodejs-src.mk
@@ -242,6 +242,7 @@ NODEJS_SRC_MODULES_LIST= $(call qstrip,\
 #
 ifneq ($(NODEJS_SRC_MODULES_LIST),)
 NODEJS_SRC_DEPENDENCIES += host-nodejs
+NODEJS_TAINTS = YES
 define NODEJS_SRC_INSTALL_MODULES
 	# If you're having trouble with module installation, adding -d to the
 	# npm install call below and setting npm_config_rollback=false can both
-- 
2.41.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 0/3] Add tainting support

[Yann E. MORIN]

Adam, All,

On 2023-11-03 12:27 -0600, Adam Duskett spake thusly:
> Originally taken from Angelo Compagnucci's patch series:
> https://patchwork.ozlabs.org/project/buildroot/list/?series=64340
> 
> When the original patch series was submitted, Buildroot was quite a bit
> smaller! With almost 1,000 packages added since then and more and more external
> package managers being available for various languages, it is not possible to
> package every single dependency needed for some packages.
> 
> Indeed, looking at NPM, for example, some packages rely on tens, if not
> hundreds, of others. While this could be better and is indeed quite bad, it is,
> unfortunately, the reality we must deal with.
> 
> With this patch series, we add initial tainting support to Buildroot with the
> added option to turn the support off by way of the BR2_DISABLE_TAINT_CHECKING
> option. This option gives us the best of both worlds: A sane default and an
> option for advanced users to turn off the check if they understand the risks
> and can guarantee their build is reproducible.

It does not require any such thing as tainted, and is just as simple as
this:

    diff --git a/package/nodejs/nodejs-src/nodejs-src.mk b/package/nodejs/nodejs-src/nodejs-src.mk
    index 3452c93728..2d716d8547 100644
    --- a/package/nodejs/nodejs-src/nodejs-src.mk
    +++ b/package/nodejs/nodejs-src/nodejs-src.mk
    @@ -241,6 +241,7 @@ NODEJS_SRC_MODULES_LIST= $(call qstrip,\
     # We can only call NPM if there's something to install.
     #
     ifneq ($(NODEJS_SRC_MODULES_LIST),)
    +NODEJS_SRC_LICENSE += , vendored dependencies licenses probably not listed
     NODEJS_SRC_DEPENDENCIES += host-nodejs
     define NODEJS_SRC_INSTALL_MODULES
        # If you're having trouble with module installation, adding -d
        # to the

As for reproducibility: if the package is not reproducible, either it is
fixed so that it is reproducible, or if that is not possible, then the
package should be hidden away behind depends on !BR2_REPRODUCIBLE

That's as simple as that, I would say.

Regards,
Yann E. MORIN.

> 
> This patch series has two significant benefits:
>   - Taint checking paves the way for additional package managers to be
>     incorporated into Buildroot while maintaining reproducible integrity for
>     packages provided by Buildroot.
> 
>   - It tells the user their build is tainted and what packages they
>     have selected are causing the taint.
> 
>   - It makes support easier. If a user has a build that is tainted and the
>     the problem they are experiencing is with a tainted package; it is
>     more straightforward to tell the user they are on their own.
> 
> Adam Duskett (3):
>   Makefile: add tainting support
>   docs/manual: add information about tainting
>   package/nodejs: taint the build when using external modules
> 
>  Config.in                                |  9 +++++++++
>  Makefile                                 | 17 +++++++++++++++++
>  docs/manual/adding-packages-generic.adoc |  9 +++++++++
>  docs/manual/legal-notice.adoc            | 24 ++++++++++++++++++++++++
>  package/nodejs/Config.in                 |  3 +++
>  package/nodejs/nodejs-src/nodejs-src.mk  |  1 +
>  package/pkg-generic.mk                   | 19 +++++++++++++++++++
>  7 files changed, 82 insertions(+)
> 
> -- 
> 2.41.0
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 0/3] Add tainting support

[Adam Duskett]

[-- Attachment #1.1: Type: text/plain, Size: 5108 bytes --]

Yes, but to your original points
https://patchwork.ozlabs.org/project/buildroot/list/?series=64340, this is
not
an acceptable solution to put  "NODEJS_SRC_LICENSE += , vendored
dependencies licenses probably not listed"

in the makefile, because there are several valid instances where you may be
able to guarantee the integrity of

the licenses and provide hashsums for each of them. Hence, the ability to
disable the taint check.


Adam Duskett

Senior Embedded Systems Developer

M. +1208-515-8102

adam.duskett@amarulasolutions.com

__________________________________


Amarula Solutions BV

Joop Geesinkweg 125, 1114 AB, Amsterdam, NL

T. +31 (0)85 111 9170
info@amarulasolutions.com

www.amarulasolutions.com



On Fri, Nov 3, 2023 at 3:31 PM Yann E. MORIN <yann.morin.1998@free.fr>
wrote:

> Adam, All,
>
> On 2023-11-03 12:27 -0600, Adam Duskett spake thusly:
> > Originally taken from Angelo Compagnucci's patch series:
> > https://patchwork.ozlabs.org/project/buildroot/list/?series=64340
> >
> > When the original patch series was submitted, Buildroot was quite a bit
> > smaller! With almost 1,000 packages added since then and more and more
> external
> > package managers being available for various languages, it is not
> possible to
> > package every single dependency needed for some packages.
> >
> > Indeed, looking at NPM, for example, some packages rely on tens, if not
> > hundreds, of others. While this could be better and is indeed quite bad,
> it is,
> > unfortunately, the reality we must deal with.
> >
> > With this patch series, we add initial tainting support to Buildroot
> with the
> > added option to turn the support off by way of the
> BR2_DISABLE_TAINT_CHECKING
> > option. This option gives us the best of both worlds: A sane default and
> an
> > option for advanced users to turn off the check if they understand the
> risks
> > and can guarantee their build is reproducible.
>
> It does not require any such thing as tainted, and is just as simple as
> this:
>
>     diff --git a/package/nodejs/nodejs-src/nodejs-src.mk
> b/package/nodejs/nodejs-src/nodejs-src.mk
>     index 3452c93728..2d716d8547 100644
>     --- a/package/nodejs/nodejs-src/nodejs-src.mk
>     +++ b/package/nodejs/nodejs-src/nodejs-src.mk
>     @@ -241,6 +241,7 @@ NODEJS_SRC_MODULES_LIST= $(call qstrip,\
>      # We can only call NPM if there's something to install.
>      #
>      ifneq ($(NODEJS_SRC_MODULES_LIST),)
>     +NODEJS_SRC_LICENSE += , vendored dependencies licenses probably not
> listed
>      NODEJS_SRC_DEPENDENCIES += host-nodejs
>      define NODEJS_SRC_INSTALL_MODULES
>         # If you're having trouble with module installation, adding -d
>         # to the
>
> As for reproducibility: if the package is not reproducible, either it is
> fixed so that it is reproducible, or if that is not possible, then the
> package should be hidden away behind depends on !BR2_REPRODUCIBLE
>
> That's as simple as that, I would say.
>
> Regards,
> Yann E. MORIN.
>
> >
> > This patch series has two significant benefits:
> >   - Taint checking paves the way for additional package managers to be
> >     incorporated into Buildroot while maintaining reproducible integrity
> for
> >     packages provided by Buildroot.
> >
> >   - It tells the user their build is tainted and what packages they
> >     have selected are causing the taint.
> >
> >   - It makes support easier. If a user has a build that is tainted and
> the
> >     the problem they are experiencing is with a tainted package; it is
> >     more straightforward to tell the user they are on their own.
> >
> > Adam Duskett (3):
> >   Makefile: add tainting support
> >   docs/manual: add information about tainting
> >   package/nodejs: taint the build when using external modules
> >
> >  Config.in                                |  9 +++++++++
> >  Makefile                                 | 17 +++++++++++++++++
> >  docs/manual/adding-packages-generic.adoc |  9 +++++++++
> >  docs/manual/legal-notice.adoc            | 24 ++++++++++++++++++++++++
> >  package/nodejs/Config.in                 |  3 +++
> >  package/nodejs/nodejs-src/nodejs-src.mk  |  1 +
> >  package/pkg-generic.mk                   | 19 +++++++++++++++++++
> >  7 files changed, 82 insertions(+)
> >
> > --
> > 2.41.0
> >
> > _______________________________________________
> > buildroot mailing list
> > buildroot@buildroot.org
> > https://lists.buildroot.org/mailman/listinfo/buildroot
>
> --
>
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics'
> conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___
>      |
> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is
> no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v
>  conspiracy.  |
>
> '------------------------------^-------^------------------^--------------------'
>

[-- Attachment #1.2: Type: text/html, Size: 10370 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 0/3] Add tainting support

[Peter Korsgaard]

>>>>> "Adam" == Adam Duskett <adam.duskett@amarulasolutions.com> writes:

 > Yes, but to your original points
 > https://patchwork.ozlabs.org/project/buildroot/list/?series=64340, this is
 > not
 > an acceptable solution to put  "NODEJS_SRC_LICENSE += , vendored
 > dependencies licenses probably not listed"

 > in the makefile, because there are several valid instances where you may be
 > able to guarantee the integrity of

 > the licenses and provide hashsums for each of them. Hence, the ability to
 > disable the taint check.

In that case you would probably make individual packages for those to
have the download/license/hash stuff integrated?

As is this seems very vague and not something that will be tested by the
autobuilders, so I prefer Yann's simple npm license addition.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot