From: Stefano Tondo <stondo@gmail.com>
To: openembedded-core@lists.openembedded.org
Cc: Alexander Kanavin <alex.kanavin@gmail.com>,
Joshua Watt <JPEWhacker@gmail.com>,
Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>,
Peter Marko <peter.marko@siemens.com>,
stefano.tondo.ext@siemens.com
Subject: [PATCH 10/10] recipetool: create_npm: List the npm dependencies in SRC_URI
Date: Fri, 20 Nov 2026 14:53:41 +0100 [thread overview]
Message-ID: <20261120135342.2628228-11-stondo@gmail.com> (raw)
In-Reply-To: <20261120135342.2628228-1-stondo@gmail.com>
From: Stefano Tondo <stefano.tondo.ext@siemens.com>
recipetool adds an npmsw:// entry without destsuffix to the recipe.
Since bitbake commit 84f102954, the npmsw fetcher unpacks into the root
of UNPACKDIR and not into ${S}. So npm-shrinkwrap.json is not in ${S},
and npm.bbclass silently packs no dependencies. The package then has no
node_modules.
Create the recipe with the explicit list of dependencies instead:
- Add npm-shrinkwrap.json to SRC_URI, so that it unpacks into ${S}.
- Inherit npm-update-dependencies and require ${BPN}-npm-deps.inc.
- Run the update task, which writes the dependencies to the .inc file.
create_recipe() gives the source subdirectory to the handlers in
extravalues, so that the npm handler knows where ${S} is.
Fetch the dependencies from the same list for the license scan.
The selftest now checks the .inc file.
Signed-off-by: Stefano Tondo <stefano.tondo.ext@siemens.com>
---
meta/lib/oeqa/selftest/cases/recipetool.py | 16 +++++++--
scripts/lib/recipetool/create.py | 4 +++
scripts/lib/recipetool/create_npm.py | 42 +++++++++++++++-------
3 files changed, 48 insertions(+), 14 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/recipetool.py b/meta/lib/oeqa/selftest/cases/recipetool.py
index ba4f674b4c..181d95804f 100644
--- a/meta/lib/oeqa/selftest/cases/recipetool.py
+++ b/meta/lib/oeqa/selftest/cases/recipetool.py
@@ -5,11 +5,13 @@
#
import errno
+import json
import os
import shutil
import tempfile
import urllib.parse
+import oe.npm
import oe.spdx_license
from oeqa.utils.commands import runCmd, bitbake, get_bb_var
@@ -452,16 +454,26 @@ class RecipetoolCreateTests(RecipetoolBase):
urls.append('npm://registry.npmjs.org/;package=@savoirfairelinux/node-server-example;version=${PV}'
';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7'
'83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223')
- urls.append('npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json')
+ urls.append('file://npm-shrinkwrap.json;subdir=npm')
checkvars['SRC_URI'] = set(urls)
checkvars['S'] = '${UNPACKDIR}/npm'
checkvars['LICENSE:${PN}'] = 'MIT'
checkvars['LICENSE:${PN}-base64'] = 'Unknown'
checkvars['LICENSE:${PN}-accepts'] = 'MIT'
checkvars['LICENSE:${PN}-inherits'] = 'ISC'
- inherits = ['npm']
+ inherits = ['npm', 'npm-update-dependencies']
self._test_recipe_contents(recipefile, checkvars, inherits)
+ # The update task writes one SRC_URI entry for each dependency
+ with open(shrinkwrap) as f:
+ expected = oe.npm.shrinkwrap_src_uri(json.load(f), 'npm')
+ deps_require_file = os.path.join(temprecipe, 'savoirfairelinux-node-server-example-npm-deps.inc')
+ self.assertFileExists(deps_require_file)
+ checkvars = {'SRC_URI': set(expected)}
+ self._test_recipe_contents(deps_require_file, checkvars, [])
+ self.assertTrue(any(url.startswith('npm://registry.npmjs.org/;package=accepts;')
+ and url.endswith(';destsuffix=npm/node_modules/accepts') for url in expected))
+
def test_recipetool_create_github(self):
# Basic test to see if github URL mangling works. Deliberately use an
# older release of Meson at present so we don't need a toml parser.
diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index b2cc92c90b..46a321a866 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -784,8 +784,12 @@ def create_recipe(args):
classes.append('bin_package')
handled.append('buildsystem')
+ # Handlers can read the source subdirectory (S relative to UNPACKDIR),
+ # when S is set to it above
+ extravalues['srcsubdir'] = srcsubdir if not args.binary else ''
for handler in handlers:
handler.process(srctree_use, classes, lines_before, lines_after, handled, extravalues)
+ extravalues.pop('srcsubdir', None)
# native and nativesdk classes are special and must be inherited last
# If present, put them at the end of the classes list
diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py
index 50dd45c7d2..1b03aef612 100644
--- a/scripts/lib/recipetool/create_npm.py
+++ b/scripts/lib/recipetool/create_npm.py
@@ -12,6 +12,7 @@ import re
import sys
import tempfile
import bb
+import oe.npm
from bb.fetch.npm import NpmEnvironment
from bb.fetch.npm import npm_package
from bb.fetch.npmsw import foreach_dependencies
@@ -257,33 +258,39 @@ class NpmRecipeHandler(RecipeHandler):
extravalues.setdefault("extrafiles", {})
extravalues["extrafiles"]["npm-shrinkwrap.json"] = shrinkwrap_copy
- url_local = "npmsw://%s" % shrinkwrap_file
- url_recipe= "npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json"
+ has_dependencies = "dependencies" in shrinkwrap.get("packages", {}).get("", {})
- if dev:
- url_local += ";dev=1"
- url_recipe += ";dev=1"
+ # The shrinkwrap file must be in ${S}, where npm.bbclass reads it
+ srcsubdir = extravalues.get("srcsubdir") or "${BP}"
+ url_recipe = "file://npm-shrinkwrap.json;subdir=%s" % srcsubdir
- # Add the npmsw url in the SRC_URI of the generated recipe
+ # Add the shrinkwrap file to the SRC_URI of the generated recipe
def _handle_srcuri(varname, origvalue, op, newlines):
- """Update the version value and add the 'npmsw://' url"""
+ """Update the version value and add the shrinkwrap file"""
value = origvalue.replace("version=" + data["version"], "version=${PV}")
value = value.replace("version=latest", "version=${PV}")
values = [line.strip() for line in value.strip('\n').splitlines()]
- if "dependencies" in shrinkwrap.get("packages", {}).get("", {}):
+ if has_dependencies:
values.append(url_recipe)
return values, None, 4, False
(_, newlines) = bb.utils.edit_metadata(lines_before, ["SRC_URI"], _handle_srcuri)
lines_before[:] = [line.rstrip('\n') for line in newlines]
+ try:
+ dependency_urls = oe.npm.shrinkwrap_src_uri(shrinkwrap, "", dev)
+ except ValueError as e:
+ bb.error("%s: %s" % (shrinkwrap_file, e))
+ sys.exit(14)
+
# In order to generate correct licence checksums in the recipe the
- # dependencies have to be fetched again using the npmsw url
+ # dependencies have to be fetched again
bb.note("Fetching npm dependencies ...")
bb.utils.remove(os.path.join(srctree, "node_modules"), recurse=True)
- fetcher = bb.fetch.Fetch([url_local], d)
- fetcher.download()
- fetcher.unpack(srctree)
+ if dependency_urls:
+ fetcher = bb.fetch.Fetch(dependency_urls, d)
+ fetcher.download()
+ fetcher.unpack(srctree)
bb.note("Handling licences ...")
(licfiles, packages, fallback_licenses) = self._handle_licenses(srctree, shrinkwrap_file, dev)
@@ -297,6 +304,17 @@ class NpmRecipeHandler(RecipeHandler):
classes.append("npm")
handled.append("buildsystem")
+ if has_dependencies:
+ # npm-update-dependencies writes the dependencies to
+ # ${BPN}-npm-deps.inc. Write a stub file, which the task replaces.
+ classes.append("npm-update-dependencies")
+ extravalues["run_tasks"] = "update_npm_deps"
+ stub = os.path.join(tempfile.mkdtemp(prefix="recipetool-npm-"), "npm-deps.inc")
+ with open(stub, "w") as f:
+ f.write("# FROM RECIPETOOL\n")
+ extravalues["extrafiles"]["../{pn}-npm-deps.inc"] = stub
+ lines_before.append("require ${BPN}-npm-deps.inc")
+
# Check if package has peer dependencies and inform the user
self._handle_peer_dependency(shrinkwrap_file)
--
2.56.1
prev parent reply other threads:[~2026-11-20 13:54 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-20 13:53 [PATCH 00/10] npm: list the dependencies in SRC_URI Stefano Tondo
2026-11-20 13:53 ` [PATCH 01/10] recipetool: create_npm: Give the fallback licenses as a string Stefano Tondo
2026-11-20 13:53 ` [PATCH 02/10] recipetool: create: Get the checksum of npm packages from the registry Stefano Tondo
2026-11-20 13:53 ` [PATCH 03/10] recipetool: create_npm: Give the package fallback licenses as strings Stefano Tondo
2026-11-20 13:53 ` [PATCH 04/10] recipetool: create_npm: Remove the "@" of a scope from generated names Stefano Tondo
2026-11-20 14:04 ` Patchtest results for " patchtest
2026-11-20 13:53 ` [PATCH 05/10] oeqa/selftest: recipetool: Expect the npm source in UNPACKDIR Stefano Tondo
2026-11-20 13:53 ` [PATCH 06/10] lib/oe/npm: Add SRC_URI entries for the dependencies of a shrinkwrap file Stefano Tondo
2026-11-20 13:53 ` [PATCH 07/10] npm-update-dependencies: Add a class to list the npm dependencies Stefano Tondo
2026-11-20 13:53 ` [PATCH 08/10] npm: Check that SRC_URI lists all dependencies of the shrinkwrap file Stefano Tondo
2026-11-20 13:53 ` [PATCH 09/10] recipetool: create: Clean the recipe after its tasks Stefano Tondo
2026-11-20 13:53 ` Stefano Tondo [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261120135342.2628228-11-stondo@gmail.com \
--to=stondo@gmail.com \
--cc=JPEWhacker@gmail.com \
--cc=alex.kanavin@gmail.com \
--cc=openembedded-core@lists.openembedded.org \
--cc=peter.marko@siemens.com \
--cc=stefan.herbrechtsmeier@weidmueller.com \
--cc=stefano.tondo.ext@siemens.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox