Re: [OE-core][PATCH v6 00/12] Add SPDX 3.0 support

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Tue, 2024-07-16 at 14:18 +0100, Richard Purdie via
lists.openembedded.org wrote:
> On Tue, 2024-07-16 at 00:00 +0100, Richard Purdie via
> lists.openembedded.org wrote:
> > On Mon, 2024-07-15 at 15:26 -0600, Joshua Watt wrote:
> > > On Mon, Jul 15, 2024 at 3:07 PM Richard Purdie
> > > <richard.purdie@linuxfoundation.org> wrote:
> > > > 
> > > > On Mon, 2024-07-15 at 14:40 -0600, Joshua Watt wrote:
> > > > > On Sat, Jul 13, 2024 at 12:44 AM Richard Purdie
> > > > > <richard.purdie@linuxfoundation.org> wrote:
> > > > > > 
> > > > > > On Fri, 2024-07-12 at 09:58 -0600, Joshua Watt via
> > > > > > lists.openembedded.org wrote:
> > > > > > > This patch series add support for SPDX 3.0 and sets it as
> > > > > > > the
> > > > > > > default.
> > > > > > > Currently it is not possible to have SPDX 2.2 and SPDX
> > > > > > > 3.0
> > > > > > > enabled at
> > > > > > > the same time
> > > > > > > 
> > > > > > > v2: Added tests and addressed feedback
> > > > > > > v3: Fixed several oe-selftest and build failures
> > > > > > > v4: Fixed silly typo mistake in staging.bbclass
> > > > > > > v5: Reworked to make SPDX 3 output reproducible by
> > > > > > > default.
> > > > > > > Variables
> > > > > > >     that introduce non-reproducible output are documented
> > > > > > > as
> > > > > > > such.
> > > > > > > v6: Many changes:
> > > > > > >   * Fixed bug where building baremetal images would break
> > > > > > > SPDX
> > > > > > > 2.2
> > > > > > >   * Most SPDX code is now in python library files instead
> > > > > > > of
> > > > > > > tasks
> > > > > > >   * Removed dependency on pacakge_write_* tasks
> > > > > > >   * Fixed sstate selftest cases to account for SPDX 3.0
> > > > > > > task
> > > > > > > names
> > > > > > 
> > > > > > This had a lot of failures in testing I'm afraid:
> > > > > > 
> > > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/7134
> > > > > 
> > > > > This appears to be caused because the SPDX tasks are not
> > > > > running
> > > > > for
> > > > > some recipes (e.g. automake). It looks like this like:
> > > > > 
> > > > >   do_create_rootfs_spdx[recrdeptask] += "do_create_spdx
> > > > > do_create_package_spdx"
> > > > > 
> > > > > is not actually strong enough to make sure the SPDX tasks for
> > > > > automake
> > > > > run for e.g. core-image-sato-sdk, but I don't know why. I'll
> > > > > keep
> > > > > looking, but if anyone happens to know off the top of their
> > > > > head
> > > > > let
> > > > > me know
> > > > 
> > > > Can you be specific about which tasks you mean when you say
> > > > "make
> > > > sure
> > > > the SPDX tasks for automake run"? Do you mean do_create_spdx,
> > > > do_create_package_spdx or a different one?
> > > 
> > > Specifically, do_create_package_spdx must be run for each package
> > > installed in the rootfs before do_create_rootfs_spdx runs. I
> > > thought
> > > that
> > > 
> > >   do_create_rootfs_spdx[recrdeptask] += "do_create_spdx
> > > do_create_package_spdx"
> > > 
> > > would do this (the do_create_spdx is probably not necessary),
> > > since
> > > AFIACT, this is also how the packages get generated before being
> > > installed in the root file system via manipulation of
> > > do_rootfs[recrdeptask], but I think I'm missing something?
> > 
> > 
> > In packagegroups.bbclass there is this being added:
> > 
> > do_create_package_spdx[deptask] = "do_create_spdx"
> > do_create_package_spdx[rdeptask] = ""
> > 
> > and 
> > 
> > bitbake core-image-sato-sdk -g -c create_rootfs_spdx
> > 
> > lists
> > 
> > "core-image-sato-sdk.do_create_rootfs_spdx" ->
> > "automake.do_create_package_spdx"
> > 
> > in tasks-depends.dot if I disable it.
> > 
> > I'm not 100% sure what is going on and should sleep but wanted to
> > share
> > that before I did.
> 
> I deleted those two lines and ran a build which passed testing.
> 
> I don't think we need them with spdx 3 since it doesn't have the
> "hash
> changing" issue that we had that caused us to add that for spdx2?
> 
> The commit was:
> https://git.yoctoproject.org/poky/commit/meta/classes-recipe/packagegroup.bbclass?id=06b5f249ced23b6bc442758131832b8640164b44

For the purposes of the archives, Joshua and I agreed we don't need
these lines for SPDX 3.0. I squashed in that fix and merged SPDX 3.0
support (as default). Thanks Joshua, Marta and others for work on this.

Cheers,

Richard