Re: [Buildroot] [PATCH 1/3] package/pkg-golang: record vendored Go modules in show-info

[Thomas Perale via buildroot <buildroot@buildroot.org>]

Hi Mattias,

In reply of:
> The SBOM only listed Go packages as a whole, the modules vendored into
> them were invisible to CVE scanners. Read them from the build info the
> Go toolchain embeds in each binary, so the list matches what ships.
> 
> Signed-off-by: Mattias Walström <lazzer@gmail.com>

I haven't totally check the series yet but during the ELCE2026 Buildroot dev
meeting we have discussed this issue. Go is not the only case this would be
required: Rust, the kernel, potentially NPM packages and other standalone
packages that support generating an SBOM (such as wolfssl, brotli, ...) would
also benefit from a proper SBOM generation support.

The idea was to add a "<pkg>_SBOM" step to the generic package infrastructure
so we can both have something shared per infra such as Go and individual package
implementing their own procedure.
There are still some missing links before we move forward this way:

- How do we support the variety of SBOM format (CycloneDX, SPDX, compilation
  database)
- Do we include in the CycloneDX SBOM or just reference it ?

Some question for you for the Go case:

- Would it make sense to use the utility CycloneDX maintain for Go instead [1] ?
  If we go with the "<pkg>_SBOM" step I described it's something we could add
  to the dependencies. And Rust packages could also use the CycloneDX utility
  for Rust.
- Can you compare the output of this utility to the one you propose so we can
  better assess the pros and cons ?

[1] https://github.com/CycloneDX/cyclonedx-gomod

Best Regards,
PERALE Thomas

> ---
>  package/pkg-golang.mk           |  8 ++++
>  package/pkg-utils.mk            |  7 ++++
>  support/scripts/go-modules-info | 67 +++++++++++++++++++++++++++++++++
>  3 files changed, 82 insertions(+)
>  create mode 100755 support/scripts/go-modules-info
> 
> diff --git a/package/pkg-golang.mk b/package/pkg-golang.mk
> index cf12dd56b0..043efc94bd 100644
> --- a/package/pkg-golang.mk
> +++ b/package/pkg-golang.mk
> @@ -206,6 +206,14 @@ define $(2)_INSTALL_CMDS
>  endef
>  endif
>  
> +# Record the Go modules linked into the built binaries, for show-info
> +define $(2)_GO_MODULES_INFO
> +	$$(Q)$$(TOPDIR)/support/scripts/go-modules-info --go $$(GO_BIN) \
> +		-o $$(@D)/$$(PKG_VENDORED_DEPS_FILE) \
> +		$$$$(find $$(@D)/bin -maxdepth 1 -type f 2>/dev/null)
> +endef
> +$(2)_POST_BUILD_HOOKS += $(2)_GO_MODULES_INFO
> +
>  # Call the generic package infrastructure to generate the necessary make
>  # targets
>  $(call inner-generic-package,$(1),$(2),$(3),$(4))
> diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk
> index 17b0aa1760..21b434c6bd 100644
> --- a/package/pkg-utils.mk
> +++ b/package/pkg-utils.mk
> @@ -111,6 +111,10 @@ define json-info
>  	}
>  endef
>  
> +# File in a package build directory listing the dependencies vendored into
> +# it, as a JSON list of {name, version, purl}, added to show-info when present
> +PKG_VENDORED_DEPS_FILE = .br_vendored_dependencies.json
> +
>  # _json-info-pkg, _json-info-pkg-details, _json-info-fs: private helpers
>  # for json-info, above
>  define _json-info-pkg
> @@ -210,6 +214,9 @@ define _json-info-pkg-details
>  			$(call mk-json-str,$(patsubst $(CONFIG_DIR)/%,%,$(patch)))$(comma) \
>  		)
>  	],
> +	$(if $(wildcard $($(1)_DIR)/$(PKG_VENDORED_DEPS_FILE)), \
> +		"vendored_dependencies": $(shell cat $($(1)_DIR)/$(PKG_VENDORED_DEPS_FILE))$(comma) \
> +	)
>  	$(if $($(1)_PROVIDES), \
>  		"provides": [ \
>  			$(foreach provide, \
> diff --git a/support/scripts/go-modules-info b/support/scripts/go-modules-info
> new file mode 100755
> index 0000000000..476a4de333
> --- /dev/null
> +++ b/support/scripts/go-modules-info
> @@ -0,0 +1,67 @@
> +#!/usr/bin/env python3
> +"""List the Go modules linked into a set of Go binaries.
> +
> +Reads the build info that the Go toolchain embeds in every binary
> +(go version -m -json) and writes the third-party modules, deduplicated
> +across binaries, as a compact JSON list. Modules replaced by a local
> +directory are part of the package source and are left out.
> +
> +Each entry has the name, version and purl of the module that was built.
> +When a replace directive swapped in another module, the original is
> +recorded under "replaces".
> +"""
> +
> +import argparse
> +import json
> +import subprocess
> +import sys
> +import urllib.parse
> +
> +
> +def purl(path, version):
> +    segments = [urllib.parse.quote(s, safe="") for s in path.split("/")]
> +    return f"pkg:golang/{'/'.join(segments)}@{urllib.parse.quote(version, safe='')}"
> +
> +
> +def is_local(path):
> +    return path.startswith(("./", "../", "/"))
> +
> +
> +def module(path, version):
> +    return {"name": path, "version": version, "purl": purl(path, version)}
> +
> +
> +def binary_deps(go, binary):
> +    out = subprocess.run([go, "version", "-m", "-json", binary],
> +                         check=True, capture_output=True, text=True).stdout
> +    return json.loads(out).get("Deps") or []
> +
> +
> +def main():
> +    parser = argparse.ArgumentParser(description=__doc__,
> +                                     formatter_class=argparse.RawDescriptionHelpFormatter)
> +    parser.add_argument("--go", required=True, help="Path to the go tool")
> +    parser.add_argument("-o", "--output", required=True, help="Output JSON file")
> +    parser.add_argument("binaries", nargs="*", help="Go binaries to inspect")
> +    args = parser.parse_args()
> +
> +    modules = {}
> +    for binary in args.binaries:
> +        for dep in binary_deps(args.go, binary):
> +            repl = dep.get("Replace")
> +            if repl:
> +                if is_local(repl["Path"]):
> +                    continue
> +                entry = module(repl["Path"], repl["Version"])
> +                entry["replaces"] = module(dep["Path"], dep["Version"])
> +            else:
> +                entry = module(dep["Path"], dep["Version"])
> +            modules[entry["purl"]] = entry
> +
> +    with open(args.output, "w") as f:
> +        json.dump([modules[k] for k in sorted(modules)], f, separators=(",", ":"))
> +        f.write("\n")
> +
> +
> +if __name__ == "__main__":
> +    sys.exit(main())
> -- 
> 2.43.0
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot