[RFC] spdx30: name, version and purl for dependency downloads

[RFC] spdx30: name, version and purl for dependency downloads

[Stefano Tondo]

Hi all,

I would like your opinion on a direction for the SPDX output of
recipes with language dependencies. I have working patches for it,
described below, but I would rather agree on the approach before
sending them.

Yocto already knows each language dependency at fetch time for the
recipes that list them in SRC_URI (crate://, gomod://, gomodgit://, and
npmsw:// through its shrinkwrap file). The SPDX output loses most of
that: the downloads are recorded, but without a name, version or
package URL that a scanner can use, and dependencies fetched
implicitly (git submodules, shrinkwrap dependencies) are not recorded
at all.

Measured on current master (oe-core 3a3eef7afb, bitbake 0f33b68cb,
meta-openembedded 33f74f5dc5), source downloads in the build SPDX of
each recipe, today and with the patches:

                  today                   with the patches
  python3-bcrypt  42 crates, none with    all 42 with name,
                  version or purl         version and purl
  crucible        36 Go modules, none     all 36 with name,
                  with version or purl    version and purl
  p11-kit         gitsm, submodule        submodule recorded with
                  pkcs11-json missing     its own revision
  webserver-oob   2 downloads, the 70     72 downloads, 71 with
                  shrinkwrap packages     a purl
                  missing

webserver-oob is from TI's meta-tisdk, built on qemux86-64 without its
am335x and meta-ti runtime dependencies and without one git.ti.com
source that failed to clone, since neither oe-core nor
meta-openembedded has an npmsw recipe. crucible also fetches 67
go.mod-only entries (;mod=1), which are not compiled in and stay
without an identity.

Today a crate shows up as name "cipher-0.4.4.crate" with download
location https://static.crates.io/crates/cipher/0.4.4/download, and a
Go module as gomod://github.com/felixge/httpsnoop, with the version
only in the file name "github.com.felixge.httpsnoop@v1.0.4.zip".
Consumers end up parsing URLs and file names per ecosystem to recover
what the fetcher already knew.

I am aware of two earlier threads and would like to build on them, not
reopen them:

- Joshua's "lib: spdx30_tasks: Use Fetch.expanded_urldata()" (March
  2025), which stalled on an intermittent edk2-firmware gitsm failure
  on the autobuilder. I think I found the cause: the patch passes each
  expanded URL to oe.patch.patch_path(), which calls
  Fetch.localpath(url). For a URL that is not in SRC_URI, that creates
  a new FetchData with the recipe datastore, where the
  SRCREV_<submodule> that the gitsm fetcher sets in its own datastore
  copy does not exist. A recipe with only named SRCREVs, like
  edk2-firmware, then fails with "Please set a valid SRCREV". It
  reproduces every time with meta-selftest's git-submodule-test given
  a named SRCREV. My guess for the intermittency is that gitsm only
  lists the submodules when the clone is in DL_DIR.
- Stefan's "Concept for tightly coupled package manager" RFC (December
  2024). My reading of that thread is that explicit SRC_URI lists
  generated in layers (go-mod-update-modules,
  cargo-update-recipe-crates) are the preferred model and that
  fetchers should stay thin. This proposal assumes that model and does
  not add any dependency resolution to bitbake.

The patches have two parts.

1. Record implicit downloads

   add_download_files() also records what implicit_urldata() returns
   for each SRC_URI entry, but keeps the patch check on the SRC_URI
   entries, so implicit downloads are never set up again with the
   recipe datastore. The npmsw fetcher gets an implicit_urldata(),
   since it already builds the list of URLs internally (ud.proxy). The
   gitsm fetcher warns when the repository is not in DL_DIR, instead
   of silently returning no submodules.

2. Let the fetcher state the identity of what it downloads

   FetchData gets a package_identity attribute, None by default and
   set in urldata_init() by the fetchers that know it:

     crate     ("cargo", "cipher", "0.4.4", {})
     gomod     ("golang", "github.com/felixge/httpsnoop", "v1.0.4", {})
     gomodgit  same as gomod
     npm       ("npm", "@scope/name", "1.2.3", {})
     npmsw     the same for each registry dependency

   The last element holds purl qualifiers, so a crate or npm package
   from another registry gets a repository_url. go.mod-only entries
   get None. create-spdx-3.0 sets the name, software_packageVersion and
   software_packageUrl from it and builds the purl in oe-core, so
   bitbake does not need to know the purl specification.

   I first had a FetchMethod method in mind, but npmsw fetches its
   dependencies through plain https URLs, so it has to attach the
   identity to each of them, and an attribute is enough for that. No
   URL syntax changes, unlike the dn=/dv= parameters in Stefan's
   series.

Both parts are small, about 35 lines in bitbake and 40 in oe-core plus
tests. I tested them on master with new bitbake fetcher unit tests and
new oe-selftest cases (submodules, including the named SRCREV case,
and crate identity), which fail without the patches, and with the
existing fetcher tests and the SPDX selftests closest to the change.
Two bugs I found in the existing Git handling while doing this (gitsm
skipped, changes not reaching the task signature) are fixed in a
separate series, which the oe-core part builds on:
https://lore.kernel.org/openembedded-core/20261104112257.866914-1-stondo@gmail.com/

Questions:

a) Is the fetcher the right place for the identity, or would you
   prefer oe-core to derive it from each fetcher's FetchData? The
   latter avoids touching bitbake, but for npmsw it means reading the
   internals of its proxy fetcher.

b) Should bitbake keep returning the parts and leave the purl to
   oe-core, as above, or return a purl directly?

c) For ecosystems without a fetcher (.NET/NuGet, Maven), would a
   fetcher plus an update class following go-mod-update-modules be
   welcome, or is that better kept in a layer?

d) Dependency scope. Identity alone does not give a correct SBOM: a
   lock file lists what may be needed on any platform. python3-bcrypt
   fetches wasi, wasip2 and wit-bindgen (WebAssembly only) and r-efi
   (UEFI), and with the patches they show up with a clean purl as if
   they were part of the product. As a follow-up I tried running
   cargo metadata --filter-platform ${RUST_HOST_SYS} in cargo.bbclass:
   of the 42 crates, 25 are runtime, 12 only run at build time (build
   scripts, proc-macros) and 5 are not used on the target. I recorded
   them as LifecycleScopedRelationship dependsOn with scope runtime or
   build, and an annotation on the unused ones. Go (go list -deps) and
   npm (dev, optional, os and cpu in the shrinkwrap) could work the
   same way. Does that split sound right: declared scope from the
   fetcher where the lock file has it, observed scope from the
   language classes? And is there a preferred SPDX representation for
   "fetched but not built"?

Thanks,
Stefano

Re: [RFC] spdx30: name, version and purl for dependency downloads

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 618 bytes --]

On Thu, 5 Nov 2026 at 13:21, Stefano Tondo <stondo@gmail.com> wrote:

> I would like your opinion on a direction for the SPDX output of
> recipes with language dependencies. I have working patches for it,
> described below, but I would rather agree on the approach before
> sending them.
>

I'm afraid I am missing context, which is hardly surprising because I did
no spdx work until now. What are the language dependencies? Does something
require or recommend them? Is there a normative document that defines them?
The email talks about how the implementation works, but says nothing about
what the problem is.

Alex

[-- Attachment #2: Type: text/html, Size: 978 bytes --]

Re: [RFC] spdx30: name, version and purl for dependency downloads

[Stefano Tondo]

On Thu, 5 Nov 2026 at 21:26, Alexander Kanavin <alex.kanavin@gmail.com> wrote:
> I'm afraid I am missing context, which is hardly surprising because I
> did no spdx work until now. What are the language dependencies? Does
> something require or recommend them? Is there a normative document
> that defines them? The email talks about how the implementation
> works, but says nothing about what the problem is.

Hi Alex,

You are right: I jumped straight into the how and never explained
the why. Let me start from the problem.

Take crucible from meta-oe. It is a Go program, and besides its own
code it compiles in 36 Go modules fetched with gomod://, one of them
golang.org/x/net v0.39.0. Now imagine the next advisory for
golang.org/x/net comes out, and whoever ships a device with crucible
on it is asked: "Is your product affected?"

The SPDX that Yocto writes for the image should answer that, but it
can't. The module is in there, but only as a downloaded file named
"golang.org.x.net@v0.39.0.zip" with the location
gomod://golang.org/x/net: no version field and no package URL, so
no scanner recognises it. cve-check does not see it either, because
it only knows the CPEs of recipes. The answer ends up being someone
reading file names by hand, for every advisory and every image.

That is what I meant by language dependencies: the packages of a
language's own package manager that a recipe compiles in without
having a recipe of their own. Rust crates (crate://), Go modules
(gomod://) and npm packages (npmsw://) are the cases Yocto can fetch
today. For the first two the problem is the one above: present, but
not identifiable. Git submodules and the dependencies listed in an
npm shrinkwrap file are worse off: they are not recorded at all.

Why this matters now, with the documents that ask for it:

- The EU Cyber Resilience Act (Regulation (EU) 2024/2847), Annex I,
  Part II, point 1, requires manufacturers to identify and document
  the components of their products, including by drawing up an SBOM
  in a commonly used, machine-readable format covering at the very
  least the top-level dependencies. It applies from December 2027 to
  products sold in the EU, so many Yocto users will have to deliver
  one.

- BSI TR-03183-2, the German technical guideline that spells out
  what such an SBOM contains, asks for the dependencies of everything
  that is delivered (section 5.1), each with a name, a version and an
  identifier such as a purl or CPE (section 5.2).

- The "2026 Minimum Elements for a Software Bill of Materials",
  published in July by CISA together with BSI, ANSSI and other
  agencies, asks for transitive dependencies and an identifier per
  component, and gives the example above as its reason: a recipient
  should be able to conclude that a new vulnerability does not affect
  them when the SBOM does not list the component.

Yocto already knows all of this at fetch time; the RFC is only about
writing it into the SPDX it already produces, without changing how
anything is fetched. I hope that makes the context clearer, and I am
happy to go into more detail on any of it.

Thanks,
Stefano

Re: [RFC] spdx30: name, version and purl for dependency downloads

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

On Thu, 5 Nov 2026 at 22:05, Stefano Tondo <stondo@gmail.com> wrote:

> Yocto already knows all of this at fetch time; the RFC is only about
> writing it into the SPDX it already produces, without changing how
> anything is fetched. I hope that makes the context clearer, and I am
> happy to go into more detail on any of it.
>

Hello Stefano,

thanks for the detailed response. I need to look at code to give specific
guidance, but I want to make a couple general points:

- do not make fetchers any more complex than they already are. The reason
is that few people understand or have an interest in that code, even fewer
are able to meaningfully review contributions to it. And still fewer
actually show up to patch review meetings where patches are looked at.
Rather try to do as much as possible in oe-core through a clear API.

- gitsm and npmsw are historical mistakes, however 'convenient' they may
be. Doing internal magic just resulted in a ton of corner cases,
maintenance headaches and (in case of npm) bitrot and lack of integrity
protection that caused the whole thing to be disabled until that disabling
forced someone to fix the issues. I would want to replace them with flat,
explicit implementations.

Alex

[-- Attachment #2: Type: text/html, Size: 2002 bytes --]

Re: [OE-core] [RFC] spdx30: name, version and purl for dependency downloads

[Richard Purdie]

On Thu, 2026-11-05 at 13:21 +0100, Stefano Tondo via openembedded-core wrote:
> Yocto already knows each language dependency at fetch time for the
> recipes that list them in SRC_URI (crate://, gomod://, gomodgit://, and
> npmsw:// through its shrinkwrap file). The SPDX output loses most of
> that: the downloads are recorded, but without a name, version or
> package URL that a scanner can use, and dependencies fetched
> implicitly (git submodules, shrinkwrap dependencies) are not recorded
> at all.
> 
> Measured on current master (oe-core 3a3eef7afb, bitbake 0f33b68cb,
> meta-openembedded 33f74f5dc5), source downloads in the build SPDX of
> each recipe, today and with the patches:
> 
>                   today                   with the patches
>   python3-bcrypt  42 crates, none with    all 42 with name,
>                   version or purl         version and purl
>   crucible        36 Go modules, none     all 36 with name,
>                   with version or purl    version and purl
>   p11-kit         gitsm, submodule        submodule recorded with
>                   pkcs11-json missing     its own revision
>   webserver-oob   2 downloads, the 70     72 downloads, 71 with
>                   shrinkwrap packages     a purl
>                   missing
> 
> webserver-oob is from TI's meta-tisdk, built on qemux86-64 without its
> am335x and meta-ti runtime dependencies and without one git.ti.com
> source that failed to clone, since neither oe-core nor
> meta-openembedded has an npmsw recipe. crucible also fetches 67
> go.mod-only entries (;mod=1), which are not compiled in and stay
> without an identity.
> 
> Today a crate shows up as name "cipher-0.4.4.crate" with download
> location https://static.crates.io/crates/cipher/0.4.4/download, and a
> Go module as gomod://github.com/felixge/httpsnoop, with the version
> only in the file name "github.com.felixge.httpsnoop@v1.0.4.zip".
> Consumers end up parsing URLs and file names per ecosystem to recover
> what the fetcher already knew.
> 
> I am aware of two earlier threads and would like to build on them, not
> reopen them:
> 
> - Joshua's "lib: spdx30_tasks: Use Fetch.expanded_urldata()" (March
>   2025), which stalled on an intermittent edk2-firmware gitsm failure
>   on the autobuilder. I think I found the cause: the patch passes each
>   expanded URL to oe.patch.patch_path(), which calls
>   Fetch.localpath(url). For a URL that is not in SRC_URI, that creates
>   a new FetchData with the recipe datastore, where the
>   SRCREV_<submodule> that the gitsm fetcher sets in its own datastore
>   copy does not exist. A recipe with only named SRCREVs, like
>   edk2-firmware, then fails with "Please set a valid SRCREV". It
>   reproduces every time with meta-selftest's git-submodule-test given
>   a named SRCREV. My guess for the intermittency is that gitsm only
>   lists the submodules when the clone is in DL_DIR.

We should revisit that and fix it as I think it exposes general bugs.

That said, I am thinking we should change the fetcher so all the
submodule urls are included in SRC_URI in future and expanded_urldata()
goes away. We need to update the other backends to match. I have no
knowledge of npm so that one will be hard for me.

Once we do that, your problem becomes easier too I think. It also means
I can then fix various mirror issues I'd like to change.

> - Stefan's "Concept for tightly coupled package manager" RFC (December
>   2024). My reading of that thread is that explicit SRC_URI lists
>   generated in layers (go-mod-update-modules,
>   cargo-update-recipe-crates) are the preferred model and that
>   fetchers should stay thin. This proposal assumes that model and does
>   not add any dependency resolution to bitbake.

Yes, although just to be clear, I think bitbake needs to verify that
the dependencies are all listed.

> The patches have two parts.
> 
> 1. Record implicit downloads
> 
>    add_download_files() also records what implicit_urldata() returns
>    for each SRC_URI entry, but keeps the patch check on the SRC_URI
>    entries, so implicit downloads are never set up again with the
>    recipe datastore. The npmsw fetcher gets an implicit_urldata(),
>    since it already builds the list of URLs internally (ud.proxy). The
>    gitsm fetcher warns when the repository is not in DL_DIR, instead
>    of silently returning no submodules.
> 
> 2. Let the fetcher state the identity of what it downloads
> 
>    FetchData gets a package_identity attribute, None by default and
>    set in urldata_init() by the fetchers that know it:
> 
>      crate     ("cargo", "cipher", "0.4.4", {})
>      gomod     ("golang", "github.com/felixge/httpsnoop", "v1.0.4", {})
>      gomodgit  same as gomod
>      npm       ("npm", "@scope/name", "1.2.3", {})
>      npmsw     the same for each registry dependency
> 
>    The last element holds purl qualifiers, so a crate or npm package
>    from another registry gets a repository_url. go.mod-only entries
>    get None. create-spdx-3.0 sets the name, software_packageVersion and
>    software_packageUrl from it and builds the purl in oe-core, so
>    bitbake does not need to know the purl specification.
> 
>    I first had a FetchMethod method in mind, but npmsw fetches its
>    dependencies through plain https URLs, so it has to attach the
>    identity to each of them, and an attribute is enough for that. No
>    URL syntax changes, unlike the dn=/dv= parameters in Stefan's
>    series.
> 
> Both parts are small, about 35 lines in bitbake and 40 in oe-core plus
> tests. I tested them on master with new bitbake fetcher unit tests and
> new oe-selftest cases (submodules, including the named SRCREV case,
> and crate identity), which fail without the patches, and with the
> existing fetcher tests and the SPDX selftests closest to the change.
> Two bugs I found in the existing Git handling while doing this (gitsm
> skipped, changes not reaching the task signature) are fixed in a
> separate series, which the oe-core part builds on:
> https://lore.kernel.org/openembedded-core/20261104112257.866914-1-stondo@gmail.com/
> 
> Questions:
> 
> a) Is the fetcher the right place for the identity, or would you
>    prefer oe-core to derive it from each fetcher's FetchData? The
>    latter avoids touching bitbake, but for npmsw it means reading the
>    internals of its proxy fetcher.

I think that the fetcher is the right place to do this as I would be
very much against poking fetcher internals from OE-Core.

We need to declare and document the stage at which this is expected to
work (i.e. before/after fetch, before/after unpack).

> b) Should bitbake keep returning the parts and leave the purl to
>    oe-core, as above, or return a purl directly?

It should probably return structured data including the purl from some
new API.

> c) For ecosystems without a fetcher (.NET/NuGet, Maven), would a
>    fetcher plus an update class following go-mod-update-modules be
>    welcome, or is that better kept in a layer?

If the fetcher can conform to our APIs and roughly follow how
rust/crates are working today, I think those would be fine,
particularly if we have good tests too.

> d) Dependency scope. Identity alone does not give a correct SBOM: a
>    lock file lists what may be needed on any platform. python3-bcrypt
>    fetches wasi, wasip2 and wit-bindgen (WebAssembly only) and r-efi
>    (UEFI), and with the patches they show up with a clean purl as if
>    they were part of the product. As a follow-up I tried running
>    cargo metadata --filter-platform ${RUST_HOST_SYS} in cargo.bbclass:
>    of the 42 crates, 25 are runtime, 12 only run at build time (build
>    scripts, proc-macros) and 5 are not used on the target. I recorded
>    them as LifecycleScopedRelationship dependsOn with scope runtime or
>    build, and an annotation on the unused ones. Go (go list -deps) and
>    npm (dev, optional, os and cpu in the shrinkwrap) could work the
>    same way. Does that split sound right: declared scope from the
>    fetcher where the lock file has it, observed scope from the
>    language classes? And is there a preferred SPDX representation for
>    "fetched but not built"?

It sounds possibly right but I'd defer to Joshua on that one.

Cheers,

Richard

Re: [RFC] spdx30: name, version and purl for dependency downloads

[Stefano Tondo]

Hi Richard, hi Alex,

Thank you both for the feedback. Your replies point in the same
direction, so I answer them together here.

On Fri, 6 Nov 2026 at 12:04, Alexander Kanavin wrote:
> - do not make fetchers any more complex than they already are.
> [...] Rather try to do as much as possible in oe-core through a
> clear API.

On Fri, 6 Nov 2026 at 12:17, Richard Purdie wrote:
> I think that the fetcher is the right place to do this as I would
> be very much against poking fetcher internals from OE-Core. We need
> to declare and document the stage at which this is expected to work
> (i.e. before/after fetch, before/after unpack).
> [...]
> It should probably return structured data including the purl from
> some new API.

I think these two points fit together. The fetchers already parse the
name and the version of each dependency for their own use. The new
API only returns that data as a documented structure, with the purl.
It does not change the fetch or the unpack. oe-core then writes
the SPDX from it and does not read any fetcher internals.

About the stage: crate, gomod and npmsw know the identity when they
create the FetchData. That is before fetch, and it needs no network
access. npmsw only reads its shrinkwrap file, which is a local file
in the layer. gitsm is the exception, because it can list the
submodules only from the clone. When the submodules are in SRC_URI,
that exception goes away. So I propose to document "before fetch".

> I am thinking we should change the fetcher so all the submodule urls
> are included in SRC_URI in future and expanded_urldata() goes away.
> We need to update the other backends to match. I have no knowledge
> of npm so that one will be hard for me.

Alex wrote:
> - gitsm and npmsw are historical mistakes, however 'convenient' they
> may be. [...] I would want to replace them with flat, explicit
> implementations.

I agree, and I would like to help with npm. My idea is to follow
cargo-update-recipe-crates and go-mod-update-modules: an update class
writes one SRC_URI entry with a checksum for each package in the
shrinkwrap file. Then bitbake checks that the list matches the
shrinkwrap file, as you said in your reply. Is
that what you have in mind? If yes, I can prepare patches.

This also makes my RFC smaller. Its first part records implicit
downloads. With flat SRC_URI lists, there are no implicit downloads
left to record.

> We should revisit that and fix it as I think it exposes general
> bugs.

I can investigate this too. I have a reproducer that fails every time:
meta-selftest's git-submodule-test with a named SRCREV. The bug I
found is in Fetch.localpath() for a URL that is not in SRC_URI. It
creates a new FetchData with the recipe datastore, which does not
have the SRCREV that gitsm set for the submodule. Is this the bug you
mean, or do you know of others in this area? I can start from any
cases you have.

> If the fetcher can conform to our APIs and roughly follow how
> rust/crates are working today, I think those would be fine,
> particularly if we have good tests too.

Thanks, that is clear. I will come back to new fetchers only after
we agree on the API.

Joshua, Richard deferred question d) to you. In short: a lock file
lists packages for all platforms, so the fetcher downloads some
dependencies that are never built for the target. I proposed to take
the declared scope from the lock file in the fetcher, and the
observed scope from the language classes (for example cargo metadata
--filter-platform). Does that split work for you? And how do you want
"fetched but not built" to look in SPDX 3.0?

For the next step: crate and gomod already list every dependency in
SRC_URI, so the identity API can work for them now. Do you want me to
send patches for these two, with tests, and add npm after the
flattening? Or do you prefer that I wait for Joshua's answer first?

Thanks,
Stefano

Re: [RFC] spdx30: name, version and purl for dependency downloads

[Joshua Watt]

[-- Attachment #1: Type: text/plain, Size: 4889 bytes --]

On Tue, Nov 10, 2026 at 2:59 PM Stefano Tondo <stondo@gmail.com> wrote:

> Hi Richard, hi Alex,
>
> Thank you both for the feedback. Your replies point in the same
> direction, so I answer them together here.
>
> On Fri, 6 Nov 2026 at 12:04, Alexander Kanavin wrote:
> > - do not make fetchers any more complex than they already are.
> > [...] Rather try to do as much as possible in oe-core through a
> > clear API.
>
> On Fri, 6 Nov 2026 at 12:17, Richard Purdie wrote:
> > I think that the fetcher is the right place to do this as I would
> > be very much against poking fetcher internals from OE-Core. We need
> > to declare and document the stage at which this is expected to work
> > (i.e. before/after fetch, before/after unpack).
> > [...]
> > It should probably return structured data including the purl from
> > some new API.
>
> I think these two points fit together. The fetchers already parse the
> name and the version of each dependency for their own use. The new
> API only returns that data as a documented structure, with the purl.
> It does not change the fetch or the unpack. oe-core then writes
> the SPDX from it and does not read any fetcher internals.
>
> About the stage: crate, gomod and npmsw know the identity when they
> create the FetchData. That is before fetch, and it needs no network
> access. npmsw only reads its shrinkwrap file, which is a local file
> in the layer. gitsm is the exception, because it can list the
> submodules only from the clone. When the submodules are in SRC_URI,
> that exception goes away. So I propose to document "before fetch".
>
> > I am thinking we should change the fetcher so all the submodule urls
> > are included in SRC_URI in future and expanded_urldata() goes away.
> > We need to update the other backends to match. I have no knowledge
> > of npm so that one will be hard for me.
>
> Alex wrote:
> > - gitsm and npmsw are historical mistakes, however 'convenient' they
> > may be. [...] I would want to replace them with flat, explicit
> > implementations.
>
> I agree, and I would like to help with npm. My idea is to follow
> cargo-update-recipe-crates and go-mod-update-modules: an update class
> writes one SRC_URI entry with a checksum for each package in the
> shrinkwrap file. Then bitbake checks that the list matches the
> shrinkwrap file, as you said in your reply. Is
> that what you have in mind? If yes, I can prepare patches.
>
> This also makes my RFC smaller. Its first part records implicit
> downloads. With flat SRC_URI lists, there are no implicit downloads
> left to record.
>
> > We should revisit that and fix it as I think it exposes general
> > bugs.
>
> I can investigate this too. I have a reproducer that fails every time:
> meta-selftest's git-submodule-test with a named SRCREV. The bug I
> found is in Fetch.localpath() for a URL that is not in SRC_URI. It
> creates a new FetchData with the recipe datastore, which does not
> have the SRCREV that gitsm set for the submodule. Is this the bug you
> mean, or do you know of others in this area? I can start from any
> cases you have.
>
> > If the fetcher can conform to our APIs and roughly follow how
> > rust/crates are working today, I think those would be fine,
> > particularly if we have good tests too.
>
> Thanks, that is clear. I will come back to new fetchers only after
> we agree on the API.
>
> Joshua, Richard deferred question d) to you. In short: a lock file
> lists packages for all platforms, so the fetcher downloads some
> dependencies that are never built for the target. I proposed to take
> the declared scope from the lock file in the fetcher, and the
> observed scope from the language classes (for example cargo metadata
> --filter-platform). Does that split work for you? And how do you want
> "fetched but not built" to look in SPDX 3.0?
>

I'm not sure if we can. The current support we have is that the things
listed in SRC_URI are "build inputs" to the build, which is accurate (they
are inputs, even if they aren't used). We don't currently have a mechanism
to link the outputs of the build back to the SRC_URI inputs.

Perhaps, the better option would be to propagate the SRC_URI purls to the
output packages, but apply some sort of (optional) filter that can remove a
SRC_URI if it is not actually present in the output. That way, if we have
positive confirmation that some input is not present in the output, we can
remove its PURL from the output package (but, barring that, we include it).


>
> For the next step: crate and gomod already list every dependency in
> SRC_URI, so the identity API can work for them now. Do you want me to
> send patches for these two, with tests, and add npm after the
> flattening? Or do you prefer that I wait for Joshua's answer first?
>
> Thanks,
> Stefano
>

[-- Attachment #2: Type: text/html, Size: 5698 bytes --]

Re: [RFC] spdx30: name, version and purl for dependency downloads

[Stefano Tondo]

On Tue, 10 Nov 2026 at 23:53, Joshua Watt <JPEWhacker@gmail.com> wrote:
> Perhaps, the better option would be to propagate the SRC_URI purls to
> the output packages, but apply some sort of (optional) filter that can
> remove a SRC_URI if it is not actually present in the output. That way,
> if we have positive confirmation that some input is not present in the
> output, we can remove its PURL from the output package (but, barring
> that, we include it).

Hi Joshua,

Thanks, that model makes sense to me. A purl stays unless there is
proof that the dependency is not in the output. For vulnerability
checks, that is the safer side.

The cargo prototype already works as that kind of filter.
cargo metadata --filter-platform resolves the dependencies for the
target. For python3-bcrypt, it gives the proof for 17 of the 42
crates: 5 are not used on the target, and 12 only run at build time
(build scripts and proc-macros). None of them end up in the binary,
so they lose their purl, and the 25 runtime crates keep it. For Go,
go version -m on each binary lists the modules compiled into it, so
it can give the same proof.

One question about the output side: how do you want the purls on the
output package? I see two options:

1. Add them to the externalIdentifier list of the output package, the
   way set_purls() handles SPDX_PACKAGE_URLS today.
2. Link the output package to the source download elements, which
   carry the purls, with a relationship such as contains.

Option 1 is simpler. But an external identifier says what the package
itself is, so a scanner can take python3-bcrypt for one of its crates.
Option 2 keeps each dependency as its own element. I prefer option 2,
unless you see a reason against it.

Thanks,
Stefano

Re: [RFC] spdx30: name, version and purl for dependency downloads

[Joshua Watt]

[-- Attachment #1: Type: text/plain, Size: 2333 bytes --]

On Wed, Nov 11, 2026 at 4:58 AM Stefano Tondo <stondo@gmail.com> wrote:

> On Tue, 10 Nov 2026 at 23:53, Joshua Watt <JPEWhacker@gmail.com> wrote:
> > Perhaps, the better option would be to propagate the SRC_URI purls to
> > the output packages, but apply some sort of (optional) filter that can
> > remove a SRC_URI if it is not actually present in the output. That way,
> > if we have positive confirmation that some input is not present in the
> > output, we can remove its PURL from the output package (but, barring
> > that, we include it).
>
> Hi Joshua,
>
> Thanks, that model makes sense to me. A purl stays unless there is
> proof that the dependency is not in the output. For vulnerability
> checks, that is the safer side.
>
> The cargo prototype already works as that kind of filter.
> cargo metadata --filter-platform resolves the dependencies for the
> target. For python3-bcrypt, it gives the proof for 17 of the 42
> crates: 5 are not used on the target, and 12 only run at build time
> (build scripts and proc-macros). None of them end up in the binary,
> so they lose their purl, and the 25 runtime crates keep it. For Go,
> go version -m on each binary lists the modules compiled into it, so
> it can give the same proof.
>
> One question about the output side: how do you want the purls on the
> output package? I see two options:
>
> 1. Add them to the externalIdentifier list of the output package, the
>    way set_purls() handles SPDX_PACKAGE_URLS today.
> 2. Link the output package to the source download elements, which
>    carry the purls, with a relationship such as contains.
>
> Option 1 is simpler. But an external identifier says what the package
> itself is, so a scanner can take python3-bcrypt for one of its crates.
>

This one is likely the easiest. Static linking (of any kind) means you
should inherit the PURLs of the libraries you have linked against. The
alternative would be to try and make separate packages for the static
libraries and then link them into the final package with a `contains` or
something, but I don't think we have enough introspection into the build
systems to do that reliably.


> Option 2 keeps each dependency as its own element. I prefer option 2,
> unless you see a reason against it.
>
> Thanks,
> Stefano
>

[-- Attachment #2: Type: text/html, Size: 2983 bytes --]

Re: [RFC] spdx30: name, version and purl for dependency downloads

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 661 bytes --]

On Tue, 10 Nov 2026 at 22:59, Stefano Tondo <stondo@gmail.com> wrote:

> I agree, and I would like to help with npm. My idea is to follow
> cargo-update-recipe-crates and go-mod-update-modules: an update class
> writes one SRC_URI entry with a checksum for each package in the
> shrinkwrap file. Then bitbake checks that the list matches the
> shrinkwrap file, as you said in your reply. Is
> that what you have in mind? If yes, I can prepare patches.
>

More or less yes. Please write tests together with any new code; ideally it
should be possible to use and test npm fetcher without npm executable, same
as cargo fetcher doesn't need cargo executable.

Alex

[-- Attachment #2: Type: text/html, Size: 1027 bytes --]