Openembedded Core Discussions
 help / color / mirror / Atom feed
From: Stefano Tondo <stondo@gmail.com>
To: openembedded-core@lists.openembedded.org
Cc: Joshua Watt <JPEWhacker@gmail.com>,
	Alexander Kanavin <alex.kanavin@gmail.com>,
	Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>,
	Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>,
	stefano.tondo.ext@siemens.com
Subject: [PATCH v2 2/4] spdx30: Record the lifecycle scope of crate dependencies
Date: Thu, 19 Nov 2026 00:44:23 +0100	[thread overview]
Message-ID: <20261118234425.3681880-3-stondo@gmail.com> (raw)
In-Reply-To: <20261118234425.3681880-1-stondo@gmail.com>

From: Stefano Tondo <stefano.tondo.ext@siemens.com>

A Cargo.lock file lists the crates for all targets, and some crates
only run at build time (build scripts, proc-macros and the crates
they use). The fetcher downloads all of them, but only some end up in
the binary.

Add a do_cargo_metadata task that writes the dependency graph for the
target with cargo metadata --filter-platform. The create_spdx task
reads this graph. It links the build to each crate download with a
scoped dependsOn relationship: runtime, build or test. A crate that is
not used on the target gets an annotation instead.

The walk expands each workspace member only once. Members can depend
on each other in a cycle, and a member can be a dev dependency of
itself, as uv-preview is in python3-uv-build.

Add unit tests for the scopes, with member cycles.

Signed-off-by: Stefano Tondo <stefano.tondo.ext@siemens.com>
---
 meta/classes-recipe/cargo.bbclass             | 17 ++++
 meta/lib/oe/spdx30_tasks.py                   | 90 +++++++++++++++++++
 .../oeqa/selftest/cases/oelib/cargo_scopes.py | 73 +++++++++++++++
 3 files changed, 180 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py

diff --git a/meta/classes-recipe/cargo.bbclass b/meta/classes-recipe/cargo.bbclass
index 586071988a..b3e82f9d5b 100644
--- a/meta/classes-recipe/cargo.bbclass
+++ b/meta/classes-recipe/cargo.bbclass
@@ -26,6 +26,23 @@ cargo_do_compile () {
 	"${CARGO}" build ${CARGO_BUILD_FLAGS} ${PACKAGECONFIG_CONFARGS} "$@"
 }
 
+# The crate dependency graph for the target, with the crates that are not
+# used on it removed. create-spdx uses it to give each crate download a
+# lifecycle scope.
+CARGO_METADATA_FILE = "${WORKDIR}/cargo-metadata.json"
+
+do_cargo_metadata () {
+	rm -f "${CARGO_METADATA_FILE}"
+	# Some recipes inherit cargo without being a cargo project
+	if [ ! -f "${CARGO_MANIFEST_PATH}" ]; then
+		return
+	fi
+	"${CARGO}" metadata --format-version 1 --offline --locked \
+		--manifest-path "${CARGO_MANIFEST_PATH}" \
+		--filter-platform "${RUST_HOST_SYS}" > "${CARGO_METADATA_FILE}"
+}
+addtask cargo_metadata after do_configure before do_compile do_create_spdx
+
 cargo_do_install () {
 	local have_installed=false
 	for tgt in "${B}/target/${CARGO_TARGET_SUBDIR}/"*; do
diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py
index 58fe45ec7e..7aae73ed0d 100644
--- a/meta/lib/oe/spdx30_tasks.py
+++ b/meta/lib/oe/spdx30_tasks.py
@@ -476,6 +476,94 @@ def set_package_identity(d, dl, fd):
         dl.software_packageUrl = identity["purl"]
 
 
+def get_cargo_scopes(d):
+    """Return {(crate, version): scope} from CARGO_METADATA_FILE.
+
+    The file is the output of cargo metadata --filter-platform for the
+    target, so crates used only on other targets are absent. Walks the
+    dependency graph from the workspace members: normal dependencies are
+    runtime, build dependencies and proc-macro crates (and everything they
+    pull in) are build, dev dependencies of the members are test.
+    """
+    path = d.getVar("CARGO_METADATA_FILE")
+    if not path or not os.path.exists(path):
+        return None
+
+    with open(path) as f:
+        metadata = json.load(f)
+
+    packages = {p["id"]: p for p in metadata["packages"]}
+    nodes = {n["id"]: n for n in metadata["resolve"]["nodes"]}
+    members = set(metadata["workspace_members"])
+    rank = {"test": 1, "build": 2, "runtime": 3}
+
+    def is_proc_macro(pkg_id):
+        return any("proc-macro" in t["kind"] for t in packages[pkg_id]["targets"])
+
+    scopes = {}
+    expanded_members = set()
+    stack = [(m, "runtime") for m in members]
+    while stack:
+        pkg_id, scope = stack.pop()
+        if pkg_id in members:
+            # A member is a root with the runtime scope. Expand it only
+            # once: members can depend on each other in a cycle, and a
+            # member can even be a dev dependency of itself.
+            if pkg_id in expanded_members:
+                continue
+            expanded_members.add(pkg_id)
+            scope = "runtime"
+        else:
+            if scope == "runtime" and is_proc_macro(pkg_id):
+                scope = "build"
+            if rank.get(scopes.get(pkg_id), 0) >= rank[scope]:
+                continue
+            scopes[pkg_id] = scope
+        for dep in nodes[pkg_id]["deps"]:
+            for kind in (k["kind"] for k in dep["dep_kinds"]):
+                if kind == "dev":
+                    if pkg_id in members:
+                        stack.append((dep["pkg"], "test"))
+                elif kind == "build":
+                    stack.append((dep["pkg"], "test" if scope == "test" else "build"))
+                else:
+                    stack.append((dep["pkg"], scope))
+
+    return {
+        (packages[i]["name"], packages[i]["version"]): s for i, s in scopes.items()
+    }
+
+
+def add_dependency_scopes(d, objset, build, source_files):
+    """Give crate downloads a lifecycle scope, see get_cargo_scopes()."""
+    scopes = get_cargo_scopes(d)
+    if scopes is None:
+        return
+
+    by_scope = {}
+    for dl in source_files:
+        if not (getattr(dl, "software_packageUrl", None) or "").startswith("pkg:cargo/"):
+            continue
+        scope = scopes.get((dl.name, dl.software_packageVersion))
+        if scope:
+            by_scope.setdefault(scope, []).append(dl)
+        else:
+            objset.new_annotation(
+                dl,
+                "%s %s is fetched but not built: not used on target %s"
+                % (dl.name, dl.software_packageVersion, d.getVar("RUST_HOST_SYS")),
+                oe.spdx30.AnnotationType.other,
+            )
+
+    for scope, dls in sorted(by_scope.items()):
+        objset.new_scoped_relationship(
+            [build],
+            oe.spdx30.RelationshipType.dependsOn,
+            getattr(oe.spdx30.LifecycleScopeType, scope),
+            sorted(oe.sbom30.get_element_link_id(dl) for dl in dls),
+        )
+
+
 def add_download_files(d, objset):
     inputs = set()
     release_date = get_release_date(d)
@@ -978,6 +1066,8 @@ def create_spdx(d):
 
     source_files = add_download_files(d, build_objset)
     build_inputs |= source_files
+    if bb.data.inherits_class("cargo", d):
+        add_dependency_scopes(d, build_objset, build, source_files)
 
     recipe_spdx_license = add_license_expression(
         d, build_objset, d.getVar("LICENSE"), license_data, [recipe_objset]
diff --git a/meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py b/meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py
new file mode 100644
index 0000000000..6273346668
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/oelib/cargo_scopes.py
@@ -0,0 +1,73 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+import json
+import os
+import tempfile
+from unittest.case import TestCase
+
+import bb.data
+import oe.spdx30_tasks
+
+
+def package(name, proc_macro=False):
+    kind = ["proc-macro"] if proc_macro else ["lib"]
+    return {"id": name, "name": name, "version": "1.0.0", "targets": [{"kind": kind}]}
+
+
+def dep(name, *kinds):
+    return {"pkg": name, "dep_kinds": [{"kind": kind} for kind in kinds or (None,)]}
+
+
+class TestCargoScopes(TestCase):
+    def get_scopes(self, members, packages, deps):
+        metadata = {
+            "packages": packages,
+            "workspace_members": members,
+            "resolve": {"nodes": [{"id": p["id"], "deps": deps.get(p["id"], [])} for p in packages]},
+        }
+        with tempfile.TemporaryDirectory() as tmpdir:
+            path = os.path.join(tmpdir, "cargo-metadata.json")
+            with open(path, "w") as f:
+                json.dump(metadata, f)
+            d = bb.data.init()
+            d.setVar("CARGO_METADATA_FILE", path)
+            scopes = oe.spdx30_tasks.get_cargo_scopes(d)
+        return {name: scope for (name, version), scope in scopes.items()}
+
+    def test_scopes(self):
+        packages = [package(n) for n in ("app", "runtime-dep", "build-dep", "build-dep-dep",
+                                         "dev-dep", "shared")]
+        packages.append(package("derive", proc_macro=True))
+        deps = {
+            "app": [dep("runtime-dep"), dep("build-dep", "build"), dep("dev-dep", "dev"),
+                    dep("derive"), dep("shared", "dev")],
+            "build-dep": [dep("build-dep-dep")],
+            "runtime-dep": [dep("shared")],
+        }
+        self.assertEqual(self.get_scopes(["app"], packages, deps), {
+            "runtime-dep": "runtime",
+            "shared": "runtime",
+            "build-dep": "build",
+            "build-dep-dep": "build",
+            "derive": "build",
+            "dev-dep": "test",
+        })
+
+    def test_member_cycles(self):
+        # Workspace members that depend on each other in a cycle, and a member
+        # that is a dev dependency of itself, as in uv. Each member is a root
+        # and is expanded once.
+        packages = [package(n) for n in ("member-a", "member-b", "member-c", "crate-a", "crate-c")]
+        deps = {
+            "member-a": [dep("member-b"), dep("member-a", "dev"), dep("crate-a")],
+            "member-b": [dep("member-c"), dep("member-a", "dev")],
+            "member-c": [dep("member-a"), dep("member-c", None, "dev"), dep("crate-c", "dev")],
+        }
+        self.assertEqual(self.get_scopes(["member-a", "member-b", "member-c"], packages, deps), {
+            "crate-a": "runtime",
+            "crate-c": "test",
+        })
-- 
2.56.1


  parent reply	other threads:[~2026-11-18 23:44 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 23:44 [PATCH v2 0/4] spdx30: identity and purls of dependency downloads Stefano Tondo
2026-11-18 23:44 ` [PATCH v2 1/4] spdx30: Take name, version and PURL of downloads from the fetcher Stefano Tondo
2026-11-18 23:44 ` Stefano Tondo [this message]
2026-11-18 23:44 ` [PATCH v2 3/4] spdx30: Add the PURLs of the source downloads to the output packages Stefano Tondo
2026-11-18 23:44 ` [PATCH v2 4/4] oeqa/selftest: spdx: Test crate identity, scope and output PURLs Stefano Tondo
2026-11-19 10:03   ` Paul Barker
2026-11-19 10:30     ` Stefano Tondo
2026-11-19 12:34 ` [PATCH v2 0/4] spdx30: identity and purls of dependency downloads Mathieu Dubois-Briand
2026-11-19 13:35   ` Stefano Tondo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118234425.3681880-3-stondo@gmail.com \
    --to=stondo@gmail.com \
    --cc=JPEWhacker@gmail.com \
    --cc=alex.kanavin@gmail.com \
    --cc=mathieu.dubois-briand@bootlin.com \
    --cc=openembedded-core@lists.openembedded.org \
    --cc=stefan.herbrechtsmeier@weidmueller.com \
    --cc=stefano.tondo.ext@siemens.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox