[2.18][PATCH 0/2] fetch/{npm,npmsw}: fix security issue and re-enable fetchers

[2.18][PATCH 0/2] fetch/{npm,npmsw}: fix security issue and re-enable fetchers

[Antonin Godard]

This is a proposal to backport patches re-enabling the npm and npmsw
fetcher on 2.18 (Wrynose). The two patches from this series are simple
cherry-picks, and had no conflicts applying.

Applying these two patches locally is also fine to me, but as this seems
to be working as-is I figured this might be helpful to others - up to the
maintainer(s) to decide.

Tests are passing locally:

$ bitbake-selftest -v bb.tests.fetch.NPMTest
test_npm (bb.tests.fetch.NPMTest.test_npm) ... ok
test_npm_bad_checksum (bb.tests.fetch.NPMTest.test_npm_bad_checksum) ... ok
test_npm_bad_recipe_checksum_rejected (bb.tests.fetch.NPMTest.test_npm_bad_recipe_checksum_rejected)
A wrong sha512sum param in SRC_URI causes the fetch to fail. ... ok
test_npm_destsuffix_downloadfilename (bb.tests.fetch.NPMTest.test_npm_destsuffix_downloadfilename) ... ok
test_npm_mirrors (bb.tests.fetch.NPMTest.test_npm_mirrors) ... ok
test_npm_no_checksum_rejected (bb.tests.fetch.NPMTest.test_npm_no_checksum_rejected)
A missing checksum in SRC_URI is rejected regardless of BB_STRICT_CHECKSUM. ... ok
test_npm_no_network_no_tarball (bb.tests.fetch.NPMTest.test_npm_no_network_no_tarball) ... ok
test_npm_no_network_with_tarball (bb.tests.fetch.NPMTest.test_npm_no_network_with_tarball) ... ok
test_npm_package_invalid (bb.tests.fetch.NPMTest.test_npm_package_invalid) ... ok
test_npm_package_none (bb.tests.fetch.NPMTest.test_npm_package_none) ... ok
test_npm_premirrors (bb.tests.fetch.NPMTest.test_npm_premirrors) ... ok
test_npm_premirrors_with_specified_filename (bb.tests.fetch.NPMTest.test_npm_premirrors_with_specified_filename) ... ok
test_npm_recipe_checksum (bb.tests.fetch.NPMTest.test_npm_recipe_checksum)
A sha512sum param in SRC_URI is forwarded to the proxy and verified. ... ok
test_npm_registry_alternate (bb.tests.fetch.NPMTest.test_npm_registry_alternate) ... ok
test_npm_registry_invalid (bb.tests.fetch.NPMTest.test_npm_registry_invalid) ... ok
test_npm_registry_none (bb.tests.fetch.NPMTest.test_npm_registry_none) ... ok
test_npm_version_invalid (bb.tests.fetch.NPMTest.test_npm_version_invalid) ... ok
test_npm_version_latest_rejected (bb.tests.fetch.NPMTest.test_npm_version_latest_rejected) ... ok
test_npm_version_none (bb.tests.fetch.NPMTest.test_npm_version_none) ... ok
test_npmsw (bb.tests.fetch.NPMTest.test_npmsw) ... ok
test_npmsw_bad_checksum (bb.tests.fetch.NPMTest.test_npmsw_bad_checksum) ... ok
test_npmsw_bundled (bb.tests.fetch.NPMTest.test_npmsw_bundled) ... ok
test_npmsw_destsuffix (bb.tests.fetch.NPMTest.test_npmsw_destsuffix) ... ok
test_npmsw_dev (bb.tests.fetch.NPMTest.test_npmsw_dev) ... ok
test_npmsw_git (bb.tests.fetch.NPMTest.test_npmsw_git) ... ok
test_npmsw_mirrors (bb.tests.fetch.NPMTest.test_npmsw_mirrors) ... ok
test_npmsw_no_network_no_tarball (bb.tests.fetch.NPMTest.test_npmsw_no_network_no_tarball) ... ok
test_npmsw_no_network_with_tarball (bb.tests.fetch.NPMTest.test_npmsw_no_network_with_tarball) ... ok
test_npmsw_npm_reusability (bb.tests.fetch.NPMTest.test_npmsw_npm_reusability) ... ok
test_npmsw_premirrors (bb.tests.fetch.NPMTest.test_npmsw_premirrors) ... ok

----------------------------------------------------------------------
Ran 30 tests in 101.185s

OK

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
Thomas Perrot (2):
      fetch/{npm,npmsw}: re-enable fetchers now that checksums come from SRC_URI
      tests/fetch: restore and extend npm/npmsw test coverage

 lib/bb/fetch2/npm.py   | 111 +++++++++++++++++++++++++-----------------
 lib/bb/fetch2/npmsw.py |  12 ++---
 lib/bb/tests/fetch.py  | 129 +++++++++++++++++++++++++++++++++++++------------
 3 files changed, 169 insertions(+), 83 deletions(-)
---
base-commit: 0ad6c1c34a5e07a5f8dd66ab248c1e7b37b69fa9
change-id: 20260825-enable-npmsw-fetcher-wrynose-2574d377ec2b

[2.18][PATCH 1/2] fetch/{npm,npmsw}: re-enable fetchers now that checksums come from SRC_URI

[Antonin Godard]

From: Thomas Perrot <thomas.perrot@bootlin.com>

The npm and npmsw fetchers were disabled in 355cd226 because the npm
fetcher accepted checksums from the remote registry rather than from
the recipe. A compromised registry controls both the tarball and its
advertised hash, making checksum verification meaningless.

npmsw was also disabled at that point, but its security model is
already correct: checksums come from the locally-committed
npm-shrinkwrap.json file, not from the network. Re-enable it with a
fix for the missing FetchError import that would cause a NameError on
malformed shrinkwrap files. Also fix two correctness bugs:
- change 'if not packages' to 'if packages is None' in
  foreach_dependencies so a valid zero-dependency shrinkwrap
  (packages={}) no longer raises FetchError.
- add 'elif resolved is None' guard before the startswith chain in
  _resolve_dependency so missing 'resolved' fields raise
  ParameterError instead of AttributeError.

For npm, fix the root cause by separating URL resolution from checksum
handling:
- _resolve_proxy_url now stores only the bare tarball URL in the
  .resolved file; the registry-supplied dist.integrity / dist.shasum
  values are ignored entirely.
- _setup_proxy builds the proxy URL from that bare tarball URL and
  injects the checksum from the recipe's SRC_URI parameters
  (sha512sum=, sha256sum=, etc.).  uri.params is cleared before
  rebuilding so that a .resolved file written by the npmsw fetcher
  (which stores the full URI with checksum params) cannot smuggle a
  registry-sourced checksum into the npm proxy URL.
- A checksum is now mandatory: urldata_init raises
  MissingParameterError if none of sha512sum/sha256sum/sha384sum/
  sha1sum is present in SRC_URI. Unlike wget, npm has no independent
  anchor for the fetched content (no pinned commit, no fixed path) --
  the registry is exactly the party this fetcher was disabled for not
  trusting -- so, unlike the rest of BitBake's fetchers, this
  requirement is unconditional and does not depend on
  BB_STRICT_CHECKSUM.
- version=latest is now a hard ParameterError instead of a warning;
  it is inherently non-reproducible. The dead
  'if ud.version == "latest": return True' branch in need_update() is
  also removed, since version=latest is rejected at urldata_init time.
- Narrow the broad 'except Exception' in _npm_view to only catch
  json.JSONDecodeError; FetchError and ParameterError now propagate
  directly to the caller instead of being re-wrapped as a generic
  FetchError, fixing typed exception handling for version mismatches.
  Fall back to str(error) when 'summary' is absent in the registry
  error dict so the message is never silently None.

Note: existing .resolved files written by the old fetcher embed a
registry-sourced checksum in the URL and must be removed before
rebuilding.

[YOCTO #16105]

Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9e45e00a2009f28829417c9517f5ab623f66dd5d)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 lib/bb/fetch2/npm.py   | 111 +++++++++++++++++++++++++++++--------------------
 lib/bb/fetch2/npmsw.py |  12 +++---
 2 files changed, 72 insertions(+), 51 deletions(-)

diff --git a/lib/bb/fetch2/npm.py b/lib/bb/fetch2/npm.py
index ed9ed167946..0b4dc69fd96 100644
--- a/lib/bb/fetch2/npm.py
+++ b/lib/bb/fetch2/npm.py
@@ -16,6 +16,11 @@ Supported SRC_URI options are:
 - version
     The npm package version. This is a mandatory parameter.
 
+- sha512sum / sha256sum / sha384sum / sha1sum
+    The expected checksum of the downloaded tarball. Exactly one is
+    mandatory: the registry cannot be trusted to supply its own tamper
+    detection, so the checksum must come from the recipe.
+
 - downloadfilename
     Specifies the filename used when storing the downloaded file.
 
@@ -33,6 +38,7 @@ import bb
 from bb.fetch2 import Fetch
 from bb.fetch2 import FetchError
 from bb.fetch2 import FetchMethod
+from bb.fetch2 import MalformedUrl
 from bb.fetch2 import MissingParameterError
 from bb.fetch2 import ParameterError
 from bb.fetch2 import URI
@@ -40,6 +46,9 @@ from bb.fetch2 import check_network_access
 from bb.fetch2 import runfetchcmd
 from bb.utils import is_semver
 
+# Preference order matches strength; the first one present in SRC_URI wins.
+CHECKSUM_PARMS = ("sha512sum", "sha256sum", "sha384sum", "sha1sum")
+
 def npm_package(package):
     """Convert the npm package name to remove unsupported character"""
     # For scoped package names ('@user/package') the '/' is replaced by a '-'.
@@ -150,11 +159,7 @@ class Npm(FetchMethod):
 
     def supports(self, ud, d):
         """Check if a given url can be fetched with npm"""
-        #return ud.type in ["npm"]
-        if ud.type in ["npm"]:
-            from bb.parse import SkipRecipe
-            raise SkipRecipe("The npm fetcher has been disabled due to security issues and there is no maintainer to address them")
-        return False
+        return ud.type in ["npm"]
 
     def urldata_init(self, ud, d):
         """Init npm specific variables within url data"""
@@ -176,11 +181,32 @@ class Npm(FetchMethod):
         if not ud.version:
             raise MissingParameterError("Parameter 'version' required", ud.url)
 
-        if not is_semver(ud.version) and not ud.version == "latest":
+        if ud.version == "latest":
+            raise ParameterError(
+                "Version 'latest' is not reproducible; specify an exact semver version",
+                ud.url)
+
+        if not is_semver(ud.version):
             raise ParameterError("Invalid 'version' parameter", ud.url)
 
         # Extract the 'registry' part of the url
         ud.registry = re.sub(r"^npm://", "https://", ud.url.split(";")[0])
+        if not ud.url.split(";")[0][len("npm://"):]:
+            raise MalformedUrl(ud.url)
+
+        # A checksum is mandatory: the registry cannot be trusted to supply
+        # its own tamper detection, so the recipe must commit to one.
+        for cp in CHECKSUM_PARMS:
+            if cp in ud.parm:
+                ud.checksum_name = cp
+                ud.checksum_expected = ud.parm[cp]
+                break
+        else:
+            raise MissingParameterError(
+                "Missing checksum for npm package '%s@%s': a compromised "
+                "registry could otherwise serve a tampered tarball undetected. "
+                "Add one of %s to SRC_URI." % (ud.package, ud.version, ", ".join(CHECKSUM_PARMS)),
+                ud.url)
 
         # Using the 'downloadfilename' parameter as local filename
         # or the npm package name.
@@ -202,6 +228,13 @@ class Npm(FetchMethod):
         ud.resolvefile = self.localpath(ud, d) + ".resolved"
 
     def _resolve_proxy_url(self, ud, d):
+        """Resolve the tarball URL from the registry and cache it without any checksum.
+
+        Checksums must never be sourced from the registry: a compromised registry
+        controls both the tarball and its advertised hash, so any checksum obtained
+        there provides no tamper detection. Checksums are applied in _setup_proxy
+        from the recipe-provided SRC_URI parameters instead.
+        """
         def _npm_view():
             args = []
             args.append(("json", "true"))
@@ -217,50 +250,27 @@ class Npm(FetchMethod):
 
             try:
                 view = json.loads(view_string)
-
-                error = view.get("error")
-                if error is not None:
-                    raise FetchError(error.get("summary"), ud.url)
-
-                if ud.version == "latest":
-                    bb.warn("The npm package %s is using the latest " \
-                            "version available. This could lead to " \
-                            "non-reproducible builds." % pkgver)
-                elif ud.version != view.get("version"):
-                    raise ParameterError("Invalid 'version' parameter", ud.url)
-
-                return view
-
-            except Exception as e:
+            except json.JSONDecodeError as e:
                 raise FetchError("Invalid view from npm: %s" % str(e), ud.url)
 
-        def _get_url(view):
-            tarball_url = view.get("dist", {}).get("tarball")
+            error = view.get("error")
+            if error is not None:
+                raise FetchError(error.get("summary") or str(error), ud.url)
 
-            if tarball_url is None:
-                raise FetchError("Invalid 'dist.tarball' in view", ud.url)
+            if ud.version != view.get("version"):
+                raise ParameterError("Invalid 'version' parameter", ud.url)
 
-            uri = URI(tarball_url)
-            uri.params["downloadfilename"] = ud.localfile
-
-            integrity = view.get("dist", {}).get("integrity")
-            shasum = view.get("dist", {}).get("shasum")
+            return view
 
-            if integrity is not None:
-                checksum_name, checksum_expected = npm_integrity(integrity)
-                uri.params[checksum_name] = checksum_expected
-            elif shasum is not None:
-                uri.params["sha1sum"] = shasum
-            else:
-                raise FetchError("Invalid 'dist.integrity' in view", ud.url)
+        view = _npm_view()
+        tarball_url = view.get("dist", {}).get("tarball")
 
-            return str(uri)
-
-        url = _get_url(_npm_view())
+        if tarball_url is None:
+            raise FetchError("Invalid 'dist.tarball' in view", ud.url)
 
         bb.utils.mkdirhier(os.path.dirname(ud.resolvefile))
         with open(ud.resolvefile, "w") as f:
-            f.write(url)
+            f.write(tarball_url)
 
     def _setup_proxy(self, ud, d):
         if ud.proxy is None:
@@ -268,13 +278,26 @@ class Npm(FetchMethod):
                 self._resolve_proxy_url(ud, d)
 
             with open(ud.resolvefile, "r") as f:
-                url = f.read()
+                tarball_url = f.read().strip()
+
+            uri = URI(tarball_url)
+            # Discard any params that may have been embedded in the stored URL
+            # (e.g. from an npmsw-written .resolved file) and rebuild from
+            # scratch so that registry-sourced checksums can never flow through.
+            uri.params = {}
+            uri.params["downloadfilename"] = ud.localfile
+
+            # Inject the recipe-provided checksum into the proxy URL.
+            # Checksums from the remote registry are never used; only the
+            # value validated in urldata_init, sourced from the recipe, is
+            # trusted.
+            uri.params[ud.checksum_name] = ud.checksum_expected
 
             # Avoid conflicts between the environment data and:
             # - the proxy url checksum
             data = bb.data.createCopy(d)
             data.delVarFlags("SRC_URI")
-            ud.proxy = Fetch([url], data)
+            ud.proxy = Fetch([str(uri)], data)
 
     def _get_proxy_method(self, ud, d):
         self._setup_proxy(ud, d)
@@ -298,8 +321,6 @@ class Npm(FetchMethod):
         """Force a fetch, even if localpath exists ?"""
         if not os.path.exists(ud.resolvefile):
             return True
-        if ud.version == "latest":
-            return True
         proxy_m, proxy_ud, proxy_d = self._get_proxy_method(ud, d)
         return proxy_m.need_update(proxy_ud, proxy_d)
 
diff --git a/lib/bb/fetch2/npmsw.py b/lib/bb/fetch2/npmsw.py
index 85f4482ad7d..6066a1d3c55 100644
--- a/lib/bb/fetch2/npmsw.py
+++ b/lib/bb/fetch2/npmsw.py
@@ -21,6 +21,7 @@ import json
 import os
 import re
 import bb
+from bb.fetch2 import FetchError
 from bb.fetch2 import Fetch
 from bb.fetch2 import FetchMethod
 from bb.fetch2 import ParameterError
@@ -44,7 +45,7 @@ def foreach_dependencies(shrinkwrap, callback=None, dev=False):
             location = the location of the package (string)
     """
     packages = shrinkwrap.get("packages")
-    if not packages:
+    if packages is None:
         raise FetchError("Invalid shrinkwrap file format")
 
     for location, data in packages.items():
@@ -63,11 +64,7 @@ class NpmShrinkWrap(FetchMethod):
 
     def supports(self, ud, d):
         """Check if a given url can be fetched with npmsw"""
-        #return ud.type in ["npmsw"]
-        if ud.type in ["npmsw"]:
-            from bb.parse import SkipRecipe
-            raise SkipRecipe("The npmsw fetcher has been disabled due to security issues and there is no maintainer to address them")
-        return False
+        return ud.type in ["npmsw"]
 
     def urldata_init(self, ud, d):
         """Init npmsw specific variables within url data"""
@@ -126,6 +123,9 @@ class NpmShrinkWrap(FetchMethod):
                 extrapaths.append(resolvefile)
 
             # Handle http tarball sources
+            elif resolved is None:
+                raise ParameterError("Missing 'resolved' field for dependency '%s'" % name, ud.url)
+
             elif resolved.startswith("http") and integrity:
                 localfile = npm_localfile(os.path.basename(resolved))
 

-- 
2.55.0

[2.18][PATCH 2/2] tests/fetch: restore and extend npm/npmsw test coverage

[Antonin Godard]

From: Thomas Perrot <thomas.perrot@bootlin.com>

Re-enable all NPMTest cases that were disabled by returning an
unconditional unittest.skip():
- Fix skipIfNoNpm() dead code: the shutil.which check was unreachable
  after the early return.
- Remove the return-skip guard from all test_npmsw_* tests; the npmsw
  fetcher is now re-enabled.
- Restore test_npm_no_network_no_tarball with a proper @skipIfNoNpm()
  decorator.

Adapt tests for the new npm fetcher behaviour:
- Replace test_npm_version_latest (which asserted success) with
  test_npm_version_latest_rejected, which asserts ParameterError since
  version=latest is no longer accepted.
- Add a checksum to every direct npm:// fetch in this file (test_npm,
  test_npm_premirrors, test_npm_premirrors_with_specified_filename,
  test_npm_mirrors, test_npm_destsuffix_downloadfilename,
  test_npm_no_network_no_tarball, test_npm_no_network_with_tarball,
  test_npm_registry_alternate, test_npm_registry_invalid,
  test_npm_package_invalid, and the array-flatten fetches used by the
  npmsw download-cache-reuse tests), since urldata_init now rejects
  SRC_URI without one.

Add new tests:
- test_npm_recipe_checksum: verifies that a sha512sum/sha256sum param
  in SRC_URI is forwarded to the proxy fetcher and the download
  succeeds when it matches.
- test_npm_bad_recipe_checksum_rejected: verifies that a wrong checksum
  in the recipe causes the fetch to fail.
- test_npm_no_checksum_rejected: verifies that a missing checksum is
  rejected regardless of BB_STRICT_CHECKSUM (unset, '0', '1', or
  'ignore'), confirming the requirement is unconditional rather than
  the usual opt-in strict-checksum behaviour.

[YOCTO #16105]

Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f7608f3195af9abb405af9be95286e2267221331)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 lib/bb/tests/fetch.py | 129 +++++++++++++++++++++++++++++++++++++-------------
 1 file changed, 97 insertions(+), 32 deletions(-)

diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py
index c071401e6ca..c3f36fc6b96 100644
--- a/lib/bb/tests/fetch.py
+++ b/lib/bb/tests/fetch.py
@@ -18,6 +18,7 @@ import collections
 import os
 import signal
 import subprocess
+import json
 import tarfile
 import threading
 from bb.fetch2 import URI
@@ -2951,7 +2952,6 @@ class CrateTest(FetcherTest):
 
 class NPMTest(FetcherTest):
     def skipIfNoNpm():
-        return unittest.skip('npm disabled due to security issues')
         if not shutil.which('npm'):
             return unittest.skip('npm not installed')
         return lambda f: f
@@ -2959,7 +2959,9 @@ class NPMTest(FetcherTest):
     @skipIfNoNpm()
     @skipIfNoNetwork()
     def test_npm(self):
-        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0']
+        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0'
+                ';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7'
+                '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223']
         fetcher = bb.fetch.Fetch(urls, self.d)
         ud = fetcher.ud[fetcher.urls[0]]
         fetcher.download()
@@ -2973,7 +2975,9 @@ class NPMTest(FetcherTest):
     @skipIfNoNpm()
     @skipIfNoNetwork()
     def test_npm_bad_checksum(self):
-        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0']
+        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0'
+                ';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7'
+                '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223']
         # Fetch once to get a tarball
         fetcher = bb.fetch.Fetch(urls, self.d)
         ud = fetcher.ud[fetcher.urls[0]]
@@ -2992,7 +2996,9 @@ class NPMTest(FetcherTest):
     @skipIfNoNpm()
     @skipIfNoNetwork()
     def test_npm_premirrors(self):
-        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0']
+        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0'
+                ';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7'
+                '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223']
         # Fetch once to get a tarball
         fetcher = bb.fetch.Fetch(urls, self.d)
         ud = fetcher.ud[fetcher.urls[0]]
@@ -3022,7 +3028,9 @@ class NPMTest(FetcherTest):
     @skipIfNoNpm()
     @skipIfNoNetwork()
     def test_npm_premirrors_with_specified_filename(self):
-        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0']
+        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0'
+                ';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7'
+                '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223']
         # Fetch once to get a tarball
         fetcher = bb.fetch.Fetch(urls, self.d)
         ud = fetcher.ud[fetcher.urls[0]]
@@ -3044,7 +3052,9 @@ class NPMTest(FetcherTest):
     @skipIfNoNetwork()
     def test_npm_mirrors(self):
         # Fetch once to get a tarball
-        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0']
+        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0'
+                ';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7'
+                '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223']
         fetcher = bb.fetch.Fetch(urls, self.d)
         ud = fetcher.ud[fetcher.urls[0]]
         fetcher.download()
@@ -3069,7 +3079,10 @@ class NPMTest(FetcherTest):
     @skipIfNoNpm()
     @skipIfNoNetwork()
     def test_npm_destsuffix_downloadfilename(self):
-        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0;destsuffix=foo/bar;downloadfilename=foo-bar.tgz']
+        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0'
+                ';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7'
+                '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223'
+                ';destsuffix=foo/bar;downloadfilename=foo-bar.tgz']
         fetcher = bb.fetch.Fetch(urls, self.d)
         fetcher.download()
         self.assertTrue(os.path.exists(os.path.join(self.dldir, 'npm2', 'foo-bar.tgz')))
@@ -3077,9 +3090,11 @@ class NPMTest(FetcherTest):
         unpackdir = os.path.join(self.unpackdir, 'foo', 'bar')
         self.assertTrue(os.path.exists(os.path.join(unpackdir, 'package.json')))
 
+    @skipIfNoNpm()
     def test_npm_no_network_no_tarball(self):
-        return unittest.skip('npm disabled due to security issues')
-        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0']
+        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0'
+                ';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7'
+                '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223']
         self.d.setVar('BB_NO_NETWORK', '1')
         fetcher = bb.fetch.Fetch(urls, self.d)
         with self.assertRaises(bb.fetch2.NetworkAccess):
@@ -3088,7 +3103,9 @@ class NPMTest(FetcherTest):
     @skipIfNoNpm()
     @skipIfNoNetwork()
     def test_npm_no_network_with_tarball(self):
-        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0']
+        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0'
+                ';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7'
+                '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223']
         # Fetch once to get a tarball
         fetcher = bb.fetch.Fetch(urls, self.d)
         fetcher.download()
@@ -3103,7 +3120,9 @@ class NPMTest(FetcherTest):
     @skipIfNoNpm()
     @skipIfNoNetwork()
     def test_npm_registry_alternate(self):
-        urls = ['npm://skimdb.npmjs.com;package=@savoirfairelinux/node-server-example;version=1.0.0']
+        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0'
+                ';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7'
+                '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223']
         fetcher = bb.fetch.Fetch(urls, self.d)
         fetcher.download()
         fetcher.unpack(self.unpackdir)
@@ -3111,19 +3130,17 @@ class NPMTest(FetcherTest):
         self.assertTrue(os.path.exists(os.path.join(unpackdir, 'package.json')))
 
     @skipIfNoNpm()
-    @skipIfNoNetwork()
-    def test_npm_version_latest(self):
+    def test_npm_version_latest_rejected(self):
         url = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=latest']
-        fetcher = bb.fetch.Fetch(url, self.d)
-        fetcher.download()
-        fetcher.unpack(self.unpackdir)
-        unpackdir = os.path.join(self.unpackdir, 'npm')
-        self.assertTrue(os.path.exists(os.path.join(unpackdir, 'package.json')))
+        with self.assertRaises(bb.fetch2.ParameterError):
+            bb.fetch.Fetch(url, self.d)
 
     @skipIfNoNpm()
     @skipIfNoNetwork()
     def test_npm_registry_invalid(self):
-        urls = ['npm://registry.invalid.org;package=@savoirfairelinux/node-server-example;version=1.0.0']
+        urls = ['npm://registry.invalid.org;package=@savoirfairelinux/node-server-example;version=1.0.0'
+                ';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7'
+                '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223']
         fetcher = bb.fetch.Fetch(urls, self.d)
         with self.assertRaises(bb.fetch2.FetchError):
             fetcher.download()
@@ -3131,7 +3148,9 @@ class NPMTest(FetcherTest):
     @skipIfNoNpm()
     @skipIfNoNetwork()
     def test_npm_package_invalid(self):
-        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/invalid;version=1.0.0']
+        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/invalid;version=1.0.0'
+                ';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7'
+                '83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223']
         fetcher = bb.fetch.Fetch(urls, self.d)
         with self.assertRaises(bb.fetch2.FetchError):
             fetcher.download()
@@ -3143,6 +3162,60 @@ class NPMTest(FetcherTest):
         with self.assertRaises(bb.fetch2.ParameterError):
             fetcher = bb.fetch.Fetch(urls, self.d)
 
+    @skipIfNoNpm()
+    @skipIfNoNetwork()
+    def test_npm_recipe_checksum(self):
+        """A sha512sum param in SRC_URI is forwarded to the proxy and verified."""
+        import subprocess
+        from bb.fetch2.npm import npm_integrity
+        result = subprocess.run(
+            ['npm', 'view', '--json', '@savoirfairelinux/node-server-example@1.0.0'],
+            capture_output=True, text=True)
+        if result.returncode != 0:
+            self.skipTest('npm view failed: %s' % result.stderr.strip())
+        try:
+            view = json.loads(result.stdout)
+        except json.JSONDecodeError:
+            self.skipTest('npm view returned invalid JSON')
+        integrity = view.get('dist', {}).get('integrity')
+        if not integrity:
+            self.skipTest('npm view response missing dist.integrity')
+        checksum_name, hexsum = npm_integrity(integrity)
+        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example'
+                ';version=1.0.0;%s=%s' % (checksum_name, hexsum)]
+        fetcher = bb.fetch.Fetch(urls, self.d)
+        ud = fetcher.ud[fetcher.urls[0]]
+        fetcher.download()
+        self.assertTrue(os.path.exists(ud.localpath))
+
+    @skipIfNoNpm()
+    @skipIfNoNetwork()
+    def test_npm_bad_recipe_checksum_rejected(self):
+        """A wrong sha512sum param in SRC_URI causes the fetch to fail."""
+        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example'
+                ';version=1.0.0;sha512sum=deadbeef00']
+        fetcher = bb.fetch.Fetch(urls, self.d)
+        with self.assertRaises(bb.fetch2.FetchError):
+            fetcher.download()
+
+    @skipIfNoNpm()
+    def test_npm_no_checksum_rejected(self):
+        """A missing checksum in SRC_URI is rejected regardless of BB_STRICT_CHECKSUM.
+
+        Unlike wget and other fetchers, npm must not fall back to an
+        unverified download: the registry cannot be trusted to supply its
+        own tamper detection, so the checksum requirement is not gated by
+        the usual opt-in strict-checksum setting.
+        """
+        urls = ['npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0']
+        for strict in (None, '0', '1', 'ignore'):
+            if strict is None:
+                self.d.delVar('BB_STRICT_CHECKSUM')
+            else:
+                self.d.setVar('BB_STRICT_CHECKSUM', strict)
+            with self.assertRaises(bb.fetch2.MissingParameterError):
+                bb.fetch.Fetch(urls, self.d)
+
     @skipIfNoNpm()
     @skipIfNoNetwork()
     def test_npm_registry_none(self):
@@ -3175,7 +3248,6 @@ class NPMTest(FetcherTest):
 
     @skipIfNoNetwork()
     def test_npmsw(self):
-        return unittest.skip('npm disabled due to security issues')
         swfile = self.create_shrinkwrap_file({
             'packages': {
                 'node_modules/array-flatten': {
@@ -3212,7 +3284,6 @@ class NPMTest(FetcherTest):
 
     @skipIfNoNetwork()
     def test_npmsw_git(self):
-        return unittest.skip('npm disabled due to security issues')
         swfile = self.create_shrinkwrap_file({
             'packages': {
                 'node_modules/cookie': {
@@ -3226,7 +3297,6 @@ class NPMTest(FetcherTest):
 
     @skipIfNoNetwork()
     def test_npmsw_dev(self):
-        return unittest.skip('npm disabled due to security issues')
         swfile = self.create_shrinkwrap_file({
             'packages': {
                 'node_modules/array-flatten': {
@@ -3255,7 +3325,6 @@ class NPMTest(FetcherTest):
 
     @skipIfNoNetwork()
     def test_npmsw_destsuffix(self):
-        return unittest.skip('npm disabled due to security issues')
         swfile = self.create_shrinkwrap_file({
             'packages': {
                 'node_modules/array-flatten': {
@@ -3271,7 +3340,6 @@ class NPMTest(FetcherTest):
         self.assertTrue(os.path.exists(os.path.join(self.unpackdir, 'foo', 'bar', 'node_modules', 'array-flatten', 'package.json')))
 
     def test_npmsw_no_network_no_tarball(self):
-        return unittest.skip('npm disabled due to security issues')
         swfile = self.create_shrinkwrap_file({
             'packages': {
                 'node_modules/array-flatten': {
@@ -3290,7 +3358,7 @@ class NPMTest(FetcherTest):
     @skipIfNoNetwork()
     def test_npmsw_no_network_with_tarball(self):
         # Fetch once to get a tarball
-        fetcher = bb.fetch.Fetch(['npm://registry.npmjs.org;package=array-flatten;version=1.1.1'], self.d)
+        fetcher = bb.fetch.Fetch(['npm://registry.npmjs.org;package=array-flatten;version=1.1.1;sha1sum=9a5f699051b1e7073328f2a008968b64ea2955d2'], self.d)
         fetcher.download()
         # Disable network access
         self.d.setVar('BB_NO_NETWORK', '1')
@@ -3311,7 +3379,6 @@ class NPMTest(FetcherTest):
 
     @skipIfNoNetwork()
     def test_npmsw_npm_reusability(self):
-        return unittest.skip('npm disabled due to security issues')
         # Fetch once with npmsw
         swfile = self.create_shrinkwrap_file({
             'packages': {
@@ -3327,14 +3394,13 @@ class NPMTest(FetcherTest):
         # Disable network access
         self.d.setVar('BB_NO_NETWORK', '1')
         # Fetch again with npm
-        fetcher = bb.fetch.Fetch(['npm://registry.npmjs.org;package=array-flatten;version=1.1.1'], self.d)
+        fetcher = bb.fetch.Fetch(['npm://registry.npmjs.org;package=array-flatten;version=1.1.1;sha1sum=9a5f699051b1e7073328f2a008968b64ea2955d2'], self.d)
         fetcher.download()
         fetcher.unpack(self.unpackdir)
         self.assertTrue(os.path.exists(os.path.join(self.unpackdir, 'npm', 'package.json')))
 
     @skipIfNoNetwork()
     def test_npmsw_bad_checksum(self):
-        return unittest.skip('npm disabled due to security issues')
         # Try to fetch with bad checksum
         swfile = self.create_shrinkwrap_file({
             'packages': {
@@ -3376,7 +3442,7 @@ class NPMTest(FetcherTest):
     @skipIfNoNetwork()
     def test_npmsw_premirrors(self):
         # Fetch once to get a tarball
-        fetcher = bb.fetch.Fetch(['npm://registry.npmjs.org;package=array-flatten;version=1.1.1'], self.d)
+        fetcher = bb.fetch.Fetch(['npm://registry.npmjs.org;package=array-flatten;version=1.1.1;sha1sum=9a5f699051b1e7073328f2a008968b64ea2955d2'], self.d)
         ud = fetcher.ud[fetcher.urls[0]]
         fetcher.download()
         self.assertTrue(os.path.exists(ud.localpath))
@@ -3405,7 +3471,7 @@ class NPMTest(FetcherTest):
     @skipIfNoNetwork()
     def test_npmsw_mirrors(self):
         # Fetch once to get a tarball
-        fetcher = bb.fetch.Fetch(['npm://registry.npmjs.org;package=array-flatten;version=1.1.1'], self.d)
+        fetcher = bb.fetch.Fetch(['npm://registry.npmjs.org;package=array-flatten;version=1.1.1;sha1sum=9a5f699051b1e7073328f2a008968b64ea2955d2'], self.d)
         ud = fetcher.ud[fetcher.urls[0]]
         fetcher.download()
         self.assertTrue(os.path.exists(ud.localpath))
@@ -3431,7 +3497,6 @@ class NPMTest(FetcherTest):
 
     @skipIfNoNetwork()
     def test_npmsw_bundled(self):
-        return unittest.skip('npm disabled due to security issues')
         swfile = self.create_shrinkwrap_file({
             'packages': {
                 'node_modules/array-flatten': {

-- 
2.55.0

Re: [bitbake-devel] [2.18][PATCH 0/2] fetch/{npm,npmsw}: fix security issue and re-enable fetchers

[Yoann Congal]

Hello,

On Tue Aug 25, 2026 at 3:02 PM CEST, Antonin Godard via lists.openembedded.org wrote:
> This is a proposal to backport patches re-enabling the npm and npmsw
> fetcher on 2.18 (Wrynose). The two patches from this series are simple
> cherry-picks, and had no conflicts applying.
>
> Applying these two patches locally is also fine to me, but as this seems
> to be working as-is I figured this might be helpful to others - up to the
> maintainer(s) to decide.

I understand this is tempting but that'd be a feature addition from
wrynose point of view and so, unacceptable per stable policy.

Without exception to the rules, we'll have to live with wrynose not
having a NPM fetcher.

Regards,
-- 
Yoann Congal
Smile ECS

Re: [bitbake-devel] [2.18][PATCH 0/2] fetch/{npm,npmsw}: fix security issue and re-enable fetchers

[Antonin Godard]

Hi Yoann,

On Tue Aug 25, 2026 at 3:10 PM CEST, Yoann Congal via lists.openembedded.org wrote:
> Hello,
>
> On Tue Aug 25, 2026 at 3:02 PM CEST, Antonin Godard via lists.openembedded.org wrote:
>> This is a proposal to backport patches re-enabling the npm and npmsw
>> fetcher on 2.18 (Wrynose). The two patches from this series are simple
>> cherry-picks, and had no conflicts applying.
>>
>> Applying these two patches locally is also fine to me, but as this seems
>> to be working as-is I figured this might be helpful to others - up to the
>> maintainer(s) to decide.
>
> I understand this is tempting but that'd be a feature addition from
> wrynose point of view and so, unacceptable per stable policy.
>
> Without exception to the rules, we'll have to live with wrynose not
> having a NPM fetcher.

Understood, and fine by me. You'll have a link to share to others wondering
about it now. :)

Thanks for the swift answer!
Antonin