From: Stefano Tondo <stondo@gmail.com>
To: openembedded-core@lists.openembedded.org
Cc: Joshua Watt <JPEWhacker@gmail.com>,
Alexander Kanavin <alex.kanavin@gmail.com>,
Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>,
Paul Barker <paul@pbarker.dev>,
Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>,
Peter Marko <peter.marko@siemens.com>,
stefano.tondo.ext@siemens.com
Subject: [PATCH v3 2/4] spdx30: Record the lifecycle scope of crate dependencies
Date: Thu, 19 Nov 2026 21:49:07 +0100 [thread overview]
Message-ID: <20261119204909.2931233-3-stondo@gmail.com> (raw)
In-Reply-To: <20261119204909.2931233-1-stondo@gmail.com>
From: Stefano Tondo <stefano.tondo.ext@siemens.com>
A Cargo.lock file lists the crates for all targets, and some crates
only run at build time (build scripts, proc-macros and the crates
they use). The fetcher downloads all of them, but only some end up in
the binary.
Add a do_cargo_metadata task that writes the dependency graph for the
target with cargo metadata --filter-platform. The create_spdx task
reads this graph. It links the build to each crate download with a
scoped dependsOn relationship: runtime, build or test. A crate that is
not used on the target gets an annotation instead.
The walk expands each workspace member only once. Members can depend
on each other in a cycle, and a member can be a dev dependency of
itself, as uv-preview is in python3-uv-build.
Add unit tests for the scopes, with member cycles.
Signed-off-by: Stefano Tondo <stefano.tondo.ext@siemens.com>
---
meta/classes-recipe/cargo.bbclass | 17 ++++
meta/lib/oe/spdx30_tasks.py | 90 +++++++++++++++++++
.../oeqa/selftest/cases/oelib/cargo_scopes.py | 73 +++++++++++++++
3 files changed, 180 insertions(+)
create mode 100644 meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py
diff --git a/meta/classes-recipe/cargo.bbclass b/meta/classes-recipe/cargo.bbclass
index 586071988a..b3e82f9d5b 100644
--- a/meta/classes-recipe/cargo.bbclass
+++ b/meta/classes-recipe/cargo.bbclass
@@ -26,6 +26,23 @@ cargo_do_compile () {
"${CARGO}" build ${CARGO_BUILD_FLAGS} ${PACKAGECONFIG_CONFARGS} "$@"
}
+# The crate dependency graph for the target, with the crates that are not
+# used on it removed. create-spdx uses it to give each crate download a
+# lifecycle scope.
+CARGO_METADATA_FILE = "${WORKDIR}/cargo-metadata.json"
+
+do_cargo_metadata () {
+ rm -f "${CARGO_METADATA_FILE}"
+ # Some recipes inherit cargo without being a cargo project
+ if [ ! -f "${CARGO_MANIFEST_PATH}" ]; then
+ return
+ fi
+ "${CARGO}" metadata --format-version 1 --offline --locked \
+ --manifest-path "${CARGO_MANIFEST_PATH}" \
+ --filter-platform "${RUST_HOST_SYS}" > "${CARGO_METADATA_FILE}"
+}
+addtask cargo_metadata after do_configure before do_compile do_create_spdx
+
cargo_do_install () {
local have_installed=false
for tgt in "${B}/target/${CARGO_TARGET_SUBDIR}/"*; do
diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py
index eded315819..01e4eefa2d 100644
--- a/meta/lib/oe/spdx30_tasks.py
+++ b/meta/lib/oe/spdx30_tasks.py
@@ -476,6 +476,94 @@ def set_package_identity(d, dl, fd):
dl.software_packageUrl = identity.purl
+def get_cargo_scopes(d):
+ """Return {(crate, version): scope} from CARGO_METADATA_FILE.
+
+ The file is the output of cargo metadata --filter-platform for the
+ target, so crates used only on other targets are absent. Walks the
+ dependency graph from the workspace members: normal dependencies are
+ runtime, build dependencies and proc-macro crates (and everything they
+ pull in) are build, dev dependencies of the members are test.
+ """
+ path = d.getVar("CARGO_METADATA_FILE")
+ if not path or not os.path.exists(path):
+ return None
+
+ with open(path) as f:
+ metadata = json.load(f)
+
+ packages = {p["id"]: p for p in metadata["packages"]}
+ nodes = {n["id"]: n for n in metadata["resolve"]["nodes"]}
+ members = set(metadata["workspace_members"])
+ rank = {"test": 1, "build": 2, "runtime": 3}
+
+ def is_proc_macro(pkg_id):
+ return any("proc-macro" in t["kind"] for t in packages[pkg_id]["targets"])
+
+ scopes = {}
+ expanded_members = set()
+ stack = [(m, "runtime") for m in members]
+ while stack:
+ pkg_id, scope = stack.pop()
+ if pkg_id in members:
+ # A member is a root with the runtime scope. Expand it only
+ # once: members can depend on each other in a cycle, and a
+ # member can even be a dev dependency of itself.
+ if pkg_id in expanded_members:
+ continue
+ expanded_members.add(pkg_id)
+ scope = "runtime"
+ else:
+ if scope == "runtime" and is_proc_macro(pkg_id):
+ scope = "build"
+ if rank.get(scopes.get(pkg_id), 0) >= rank[scope]:
+ continue
+ scopes[pkg_id] = scope
+ for dep in nodes[pkg_id]["deps"]:
+ for kind in (k["kind"] for k in dep["dep_kinds"]):
+ if kind == "dev":
+ if pkg_id in members:
+ stack.append((dep["pkg"], "test"))
+ elif kind == "build":
+ stack.append((dep["pkg"], "test" if scope == "test" else "build"))
+ else:
+ stack.append((dep["pkg"], scope))
+
+ return {
+ (packages[i]["name"], packages[i]["version"]): s for i, s in scopes.items()
+ }
+
+
+def add_dependency_scopes(d, objset, build, source_files):
+ """Give crate downloads a lifecycle scope, see get_cargo_scopes()."""
+ scopes = get_cargo_scopes(d)
+ if scopes is None:
+ return
+
+ by_scope = {}
+ for dl in source_files:
+ if not (getattr(dl, "software_packageUrl", None) or "").startswith("pkg:cargo/"):
+ continue
+ scope = scopes.get((dl.name, dl.software_packageVersion))
+ if scope:
+ by_scope.setdefault(scope, []).append(dl)
+ else:
+ objset.new_annotation(
+ dl,
+ "%s %s is fetched but not built: not used on target %s"
+ % (dl.name, dl.software_packageVersion, d.getVar("RUST_HOST_SYS")),
+ oe.spdx30.AnnotationType.other,
+ )
+
+ for scope, dls in sorted(by_scope.items()):
+ objset.new_scoped_relationship(
+ [build],
+ oe.spdx30.RelationshipType.dependsOn,
+ getattr(oe.spdx30.LifecycleScopeType, scope),
+ sorted(oe.sbom30.get_element_link_id(dl) for dl in dls),
+ )
+
+
def add_download_files(d, objset):
inputs = set()
release_date = get_release_date(d)
@@ -978,6 +1066,8 @@ def create_spdx(d):
source_files = add_download_files(d, build_objset)
build_inputs |= source_files
+ if bb.data.inherits_class("cargo", d):
+ add_dependency_scopes(d, build_objset, build, source_files)
recipe_spdx_license = add_license_expression(
d, build_objset, d.getVar("LICENSE"), license_data, [recipe_objset]
diff --git a/meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py b/meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py
new file mode 100644
index 0000000000..6273346668
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py
@@ -0,0 +1,73 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+import json
+import os
+import tempfile
+from unittest.case import TestCase
+
+import bb.data
+import oe.spdx30_tasks
+
+
+def package(name, proc_macro=False):
+ kind = ["proc-macro"] if proc_macro else ["lib"]
+ return {"id": name, "name": name, "version": "1.0.0", "targets": [{"kind": kind}]}
+
+
+def dep(name, *kinds):
+ return {"pkg": name, "dep_kinds": [{"kind": kind} for kind in kinds or (None,)]}
+
+
+class TestCargoScopes(TestCase):
+ def get_scopes(self, members, packages, deps):
+ metadata = {
+ "packages": packages,
+ "workspace_members": members,
+ "resolve": {"nodes": [{"id": p["id"], "deps": deps.get(p["id"], [])} for p in packages]},
+ }
+ with tempfile.TemporaryDirectory() as tmpdir:
+ path = os.path.join(tmpdir, "cargo-metadata.json")
+ with open(path, "w") as f:
+ json.dump(metadata, f)
+ d = bb.data.init()
+ d.setVar("CARGO_METADATA_FILE", path)
+ scopes = oe.spdx30_tasks.get_cargo_scopes(d)
+ return {name: scope for (name, version), scope in scopes.items()}
+
+ def test_scopes(self):
+ packages = [package(n) for n in ("app", "runtime-dep", "build-dep", "build-dep-dep",
+ "dev-dep", "shared")]
+ packages.append(package("derive", proc_macro=True))
+ deps = {
+ "app": [dep("runtime-dep"), dep("build-dep", "build"), dep("dev-dep", "dev"),
+ dep("derive"), dep("shared", "dev")],
+ "build-dep": [dep("build-dep-dep")],
+ "runtime-dep": [dep("shared")],
+ }
+ self.assertEqual(self.get_scopes(["app"], packages, deps), {
+ "runtime-dep": "runtime",
+ "shared": "runtime",
+ "build-dep": "build",
+ "build-dep-dep": "build",
+ "derive": "build",
+ "dev-dep": "test",
+ })
+
+ def test_member_cycles(self):
+ # Workspace members that depend on each other in a cycle, and a member
+ # that is a dev dependency of itself, as in uv. Each member is a root
+ # and is expanded once.
+ packages = [package(n) for n in ("member-a", "member-b", "member-c", "crate-a", "crate-c")]
+ deps = {
+ "member-a": [dep("member-b"), dep("member-a", "dev"), dep("crate-a")],
+ "member-b": [dep("member-c"), dep("member-a", "dev")],
+ "member-c": [dep("member-a"), dep("member-c", None, "dev"), dep("crate-c", "dev")],
+ }
+ self.assertEqual(self.get_scopes(["member-a", "member-b", "member-c"], packages, deps), {
+ "crate-a": "runtime",
+ "crate-c": "test",
+ })
--
2.56.1
next prev parent reply other threads:[~2026-11-19 20:49 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 20:49 [PATCH v3 0/4] spdx30: identity and purls of dependency downloads Stefano Tondo
2026-11-19 20:49 ` [PATCH v3 1/4] spdx30: Take name, version and PURL of downloads from the fetcher Stefano Tondo
2026-11-19 20:49 ` Stefano Tondo [this message]
2026-11-19 20:49 ` [PATCH v3 3/4] spdx30: Add the PURLs of the source downloads to the output packages Stefano Tondo
2026-11-19 20:49 ` [PATCH v3 4/4] oeqa/selftest: spdx: Test crate identity, scope and output PURLs Stefano Tondo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119204909.2931233-3-stondo@gmail.com \
--to=stondo@gmail.com \
--cc=JPEWhacker@gmail.com \
--cc=alex.kanavin@gmail.com \
--cc=mathieu.dubois-briand@bootlin.com \
--cc=openembedded-core@lists.openembedded.org \
--cc=paul@pbarker.dev \
--cc=peter.marko@siemens.com \
--cc=stefan.herbrechtsmeier@weidmueller.com \
--cc=stefano.tondo.ext@siemens.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.