From: Stefano Tondo <stondo@gmail.com>
To: openembedded-core@lists.openembedded.org
Cc: Alexander Kanavin <alex.kanavin@gmail.com>,
Joshua Watt <JPEWhacker@gmail.com>,
Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>,
Peter Marko <peter.marko@siemens.com>,
stefano.tondo.ext@siemens.com
Subject: [PATCH 10/10] recipetool: create_npm: List the npm dependencies in SRC_URI
Date: Fri, 20 Nov 2026 14:53:41 +0100 [thread overview]
Message-ID: <20261120135342.2628228-11-stondo@gmail.com> (raw)
In-Reply-To: <20261120135342.2628228-1-stondo@gmail.com>
From: Stefano Tondo <stefano.tondo.ext@siemens.com>
recipetool adds an npmsw:// entry without destsuffix to the recipe.
Since bitbake commit 84f102954, the npmsw fetcher unpacks into the root
of UNPACKDIR and not into ${S}. So npm-shrinkwrap.json is not in ${S},
and npm.bbclass silently packs no dependencies. The package then has no
node_modules.
Create the recipe with the explicit list of dependencies instead:
- Add npm-shrinkwrap.json to SRC_URI, so that it unpacks into ${S}.
- Inherit npm-update-dependencies and require ${BPN}-npm-deps.inc.
- Run the update task, which writes the dependencies to the .inc file.
create_recipe() gives the source subdirectory to the handlers in
extravalues, so that the npm handler knows where ${S} is.
Fetch the dependencies from the same list for the license scan.
The selftest now checks the .inc file.
Signed-off-by: Stefano Tondo <stefano.tondo.ext@siemens.com>
---
meta/lib/oeqa/selftest/cases/recipetool.py | 16 +++++++--
scripts/lib/recipetool/create.py | 4 +++
scripts/lib/recipetool/create_npm.py | 42 +++++++++++++++-------
3 files changed, 48 insertions(+), 14 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/recipetool.py b/meta/lib/oeqa/selftest/cases/recipetool.py
index ba4f674b4c..181d95804f 100644
--- a/meta/lib/oeqa/selftest/cases/recipetool.py
+++ b/meta/lib/oeqa/selftest/cases/recipetool.py
@@ -5,11 +5,13 @@
#
import errno
+import json
import os
import shutil
import tempfile
import urllib.parse
+import oe.npm
import oe.spdx_license
from oeqa.utils.commands import runCmd, bitbake, get_bb_var
@@ -452,16 +454,26 @@ class RecipetoolCreateTests(RecipetoolBase):
urls.append('npm://registry.npmjs.org/;package=@savoirfairelinux/node-server-example;version=${PV}'
';sha512sum=f2dd7d88cb9a129fbb97eb87a8b5103bab24f783420fd7587f8000a355a12bf7'
'83f1263230afc588123958b73e36bb241f63eaf08119aac5aa2a870bc4de9223')
- urls.append('npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json')
+ urls.append('file://npm-shrinkwrap.json;subdir=npm')
checkvars['SRC_URI'] = set(urls)
checkvars['S'] = '${UNPACKDIR}/npm'
checkvars['LICENSE:${PN}'] = 'MIT'
checkvars['LICENSE:${PN}-base64'] = 'Unknown'
checkvars['LICENSE:${PN}-accepts'] = 'MIT'
checkvars['LICENSE:${PN}-inherits'] = 'ISC'
- inherits = ['npm']
+ inherits = ['npm', 'npm-update-dependencies']
self._test_recipe_contents(recipefile, checkvars, inherits)
+ # The update task writes one SRC_URI entry for each dependency
+ with open(shrinkwrap) as f:
+ expected = oe.npm.shrinkwrap_src_uri(json.load(f), 'npm')
+ deps_require_file = os.path.join(temprecipe, 'savoirfairelinux-node-server-example-npm-deps.inc')
+ self.assertFileExists(deps_require_file)
+ checkvars = {'SRC_URI': set(expected)}
+ self._test_recipe_contents(deps_require_file, checkvars, [])
+ self.assertTrue(any(url.startswith('npm://registry.npmjs.org/;package=accepts;')
+ and url.endswith(';destsuffix=npm/node_modules/accepts') for url in expected))
+
def test_recipetool_create_github(self):
# Basic test to see if github URL mangling works. Deliberately use an
# older release of Meson at present so we don't need a toml parser.
diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index b2cc92c90b..46a321a866 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -784,8 +784,12 @@ def create_recipe(args):
classes.append('bin_package')
handled.append('buildsystem')
+ # Handlers can read the source subdirectory (S relative to UNPACKDIR),
+ # when S is set to it above
+ extravalues['srcsubdir'] = srcsubdir if not args.binary else ''
for handler in handlers:
handler.process(srctree_use, classes, lines_before, lines_after, handled, extravalues)
+ extravalues.pop('srcsubdir', None)
# native and nativesdk classes are special and must be inherited last
# If present, put them at the end of the classes list
diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py
index 50dd45c7d2..1b03aef612 100644
--- a/scripts/lib/recipetool/create_npm.py
+++ b/scripts/lib/recipetool/create_npm.py
@@ -12,6 +12,7 @@ import re
import sys
import tempfile
import bb
+import oe.npm
from bb.fetch.npm import NpmEnvironment
from bb.fetch.npm import npm_package
from bb.fetch.npmsw import foreach_dependencies
@@ -257,33 +258,39 @@ class NpmRecipeHandler(RecipeHandler):
extravalues.setdefault("extrafiles", {})
extravalues["extrafiles"]["npm-shrinkwrap.json"] = shrinkwrap_copy
- url_local = "npmsw://%s" % shrinkwrap_file
- url_recipe= "npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json"
+ has_dependencies = "dependencies" in shrinkwrap.get("packages", {}).get("", {})
- if dev:
- url_local += ";dev=1"
- url_recipe += ";dev=1"
+ # The shrinkwrap file must be in ${S}, where npm.bbclass reads it
+ srcsubdir = extravalues.get("srcsubdir") or "${BP}"
+ url_recipe = "file://npm-shrinkwrap.json;subdir=%s" % srcsubdir
- # Add the npmsw url in the SRC_URI of the generated recipe
+ # Add the shrinkwrap file to the SRC_URI of the generated recipe
def _handle_srcuri(varname, origvalue, op, newlines):
- """Update the version value and add the 'npmsw://' url"""
+ """Update the version value and add the shrinkwrap file"""
value = origvalue.replace("version=" + data["version"], "version=${PV}")
value = value.replace("version=latest", "version=${PV}")
values = [line.strip() for line in value.strip('\n').splitlines()]
- if "dependencies" in shrinkwrap.get("packages", {}).get("", {}):
+ if has_dependencies:
values.append(url_recipe)
return values, None, 4, False
(_, newlines) = bb.utils.edit_metadata(lines_before, ["SRC_URI"], _handle_srcuri)
lines_before[:] = [line.rstrip('\n') for line in newlines]
+ try:
+ dependency_urls = oe.npm.shrinkwrap_src_uri(shrinkwrap, "", dev)
+ except ValueError as e:
+ bb.error("%s: %s" % (shrinkwrap_file, e))
+ sys.exit(14)
+
# In order to generate correct licence checksums in the recipe the
- # dependencies have to be fetched again using the npmsw url
+ # dependencies have to be fetched again
bb.note("Fetching npm dependencies ...")
bb.utils.remove(os.path.join(srctree, "node_modules"), recurse=True)
- fetcher = bb.fetch.Fetch([url_local], d)
- fetcher.download()
- fetcher.unpack(srctree)
+ if dependency_urls:
+ fetcher = bb.fetch.Fetch(dependency_urls, d)
+ fetcher.download()
+ fetcher.unpack(srctree)
bb.note("Handling licences ...")
(licfiles, packages, fallback_licenses) = self._handle_licenses(srctree, shrinkwrap_file, dev)
@@ -297,6 +304,17 @@ class NpmRecipeHandler(RecipeHandler):
classes.append("npm")
handled.append("buildsystem")
+ if has_dependencies:
+ # npm-update-dependencies writes the dependencies to
+ # ${BPN}-npm-deps.inc. Write a stub file, which the task replaces.
+ classes.append("npm-update-dependencies")
+ extravalues["run_tasks"] = "update_npm_deps"
+ stub = os.path.join(tempfile.mkdtemp(prefix="recipetool-npm-"), "npm-deps.inc")
+ with open(stub, "w") as f:
+ f.write("# FROM RECIPETOOL\n")
+ extravalues["extrafiles"]["../{pn}-npm-deps.inc"] = stub
+ lines_before.append("require ${BPN}-npm-deps.inc")
+
# Check if package has peer dependencies and inform the user
self._handle_peer_dependency(shrinkwrap_file)
--
2.56.1
prev parent reply other threads:[~2026-11-20 13:54 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-20 13:53 [PATCH 00/10] npm: list the dependencies in SRC_URI Stefano Tondo
2026-11-20 13:53 ` [PATCH 01/10] recipetool: create_npm: Give the fallback licenses as a string Stefano Tondo
2026-11-20 13:53 ` [PATCH 02/10] recipetool: create: Get the checksum of npm packages from the registry Stefano Tondo
2026-11-20 13:53 ` [PATCH 03/10] recipetool: create_npm: Give the package fallback licenses as strings Stefano Tondo
2026-11-20 13:53 ` [PATCH 04/10] recipetool: create_npm: Remove the "@" of a scope from generated names Stefano Tondo
2026-11-20 14:04 ` Patchtest results for " patchtest
2026-11-20 13:53 ` [PATCH 05/10] oeqa/selftest: recipetool: Expect the npm source in UNPACKDIR Stefano Tondo
2026-11-20 13:53 ` [PATCH 06/10] lib/oe/npm: Add SRC_URI entries for the dependencies of a shrinkwrap file Stefano Tondo
2026-11-20 13:53 ` [PATCH 07/10] npm-update-dependencies: Add a class to list the npm dependencies Stefano Tondo
2026-11-20 13:53 ` [PATCH 08/10] npm: Check that SRC_URI lists all dependencies of the shrinkwrap file Stefano Tondo
2026-11-20 13:53 ` [PATCH 09/10] recipetool: create: Clean the recipe after its tasks Stefano Tondo
2026-11-20 13:53 ` Stefano Tondo [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261120135342.2628228-11-stondo@gmail.com \
--to=stondo@gmail.com \
--cc=JPEWhacker@gmail.com \
--cc=alex.kanavin@gmail.com \
--cc=openembedded-core@lists.openembedded.org \
--cc=peter.marko@siemens.com \
--cc=stefan.herbrechtsmeier@weidmueller.com \
--cc=stefano.tondo.ext@siemens.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.