[PATCH 1/2] create-spdx: Use function rather than AVAILABLE_LICENSES

[PATCH 1/2] create-spdx: Use function rather than AVAILABLE_LICENSES

[Richard Purdie]

We can directly call the function rather than using the variable indirection.
As this is the last user of the variable, it then allows removal of it in a
followup patch.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/create-spdx.bbclass | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta/classes/create-spdx.bbclass b/meta/classes/create-spdx.bbclass
index daf99ca676c..1a4804a7c50 100644
--- a/meta/classes/create-spdx.bbclass
+++ b/meta/classes/create-spdx.bbclass
@@ -94,7 +94,7 @@ def convert_license_to_spdx(lic, document, d, existing={}):
     from pathlib import Path
     import oe.spdx
 
-    available_licenses = d.getVar("AVAILABLE_LICENSES").split()
+    avail_licenses = available_licenses(d)
     license_data = d.getVar("SPDX_LICENSE_DATA")
     extracted = {}
 
@@ -112,7 +112,7 @@ def convert_license_to_spdx(lic, document, d, existing={}):
         if name == "PD":
             # Special-case this.
             extracted_info.extractedText = "Software released to the public domain"
-        elif name in available_licenses:
+        elif name in avail_licenses:
             # This license can be found in COMMON_LICENSE_DIR or LICENSE_PATH
             for directory in [d.getVar('COMMON_LICENSE_DIR')] + (d.getVar('LICENSE_PATH') or '').split():
                 try:
@@ -122,11 +122,11 @@ def convert_license_to_spdx(lic, document, d, existing={}):
                 except FileNotFoundError:
                     pass
             if extracted_info.extractedText is None:
-                # Error out, as the license was in available_licenses so should
+                # Error out, as the license was in avail_licenses so should
                 # be on disk somewhere.
                 bb.error("Cannot find text for license %s" % name)
         else:
-            # If it's not SPDX, or PD, or in available licenses, then NO_GENERIC_LICENSE must be set
+            # If it's not SPDX, or PD, or in avail_licenses, then NO_GENERIC_LICENSE must be set
             filename = d.getVarFlag('NO_GENERIC_LICENSE', name)
             if filename:
                 filename = d.expand("${S}/" + filename)
-- 
2.32.0

[PATCH 2/2] sstate: Allow optimisation of do_create_spdx task dependencies

[Richard Purdie]

do_create_spdx tasks don't need their dependencies so we can optimistion
this as we do for some other tasks.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/sstate.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 163bdf0b5f7..1c0cae48938 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -1084,7 +1084,7 @@ def setscene_depvalid(task, taskdependees, notneeded, d, log=None):
 
     logit("Considering setscene task: %s" % (str(taskdependees[task])), log)
 
-    directtasks = ["do_populate_lic", "do_deploy_source_date_epoch", "do_shared_workdir", "do_stash_locale", "do_gcc_stash_builddir"]
+    directtasks = ["do_populate_lic", "do_deploy_source_date_epoch", "do_shared_workdir", "do_stash_locale", "do_gcc_stash_builddir", "do_create_spdx"]
 
     def isNativeCross(x):
         return x.endswith("-native") or "-cross-" in x or "-crosssdk" in x or x.endswith("-cross")
-- 
2.32.0

RE: [OE-core] [PATCH 1/2] create-spdx: Use function rather than AVAILABLE_LICENSES

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Richard Purdie
> Sent: den 15 mars 2022 18:33
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH 1/2] create-spdx: Use function rather than AVAILABLE_LICENSES
> 
> We can directly call the function rather than using the variable
> indirection.
> As this is the last user of the variable, it then allows removal of it in a
> followup patch.
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/classes/create-spdx.bbclass | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/meta/classes/create-spdx.bbclass b/meta/classes/create-spdx.bbclass
> index daf99ca676c..1a4804a7c50 100644
> --- a/meta/classes/create-spdx.bbclass
> +++ b/meta/classes/create-spdx.bbclass
> @@ -94,7 +94,7 @@ def convert_license_to_spdx(lic, document, d, existing={}):
>      from pathlib import Path
>      import oe.spdx
> 
> -    available_licenses = d.getVar("AVAILABLE_LICENSES").split()
> +    avail_licenses = available_licenses(d)

Doesn't this mean that the available licenses will be calculated once 
per recipe? Or isn't it a problem since the time will not be spent 
during recipe parsing, but rather during task execution where no one 
will notice if the create-spdx task for all recipes takes a little 
longer?

That said, if I remember correctly from the last time I looked at this 
function, it should be possible to rewrite it to not rely on 
available_licenses() without any real loss in functionality.

>      license_data = d.getVar("SPDX_LICENSE_DATA")
>      extracted = {}
> 
> @@ -112,7 +112,7 @@ def convert_license_to_spdx(lic, document, d, existing={}):
>          if name == "PD":
>              # Special-case this.
>              extracted_info.extractedText = "Software released to the public domain"
> -        elif name in available_licenses:
> +        elif name in avail_licenses:
>              # This license can be found in COMMON_LICENSE_DIR or LICENSE_PATH
>              for directory in [d.getVar('COMMON_LICENSE_DIR')] + (d.getVar('LICENSE_PATH') or '').split():
>                  try:
> @@ -122,11 +122,11 @@ def convert_license_to_spdx(lic, document, d, existing={}):
>                  except FileNotFoundError:
>                      pass
>              if extracted_info.extractedText is None:
> -                # Error out, as the license was in available_licenses so should
> +                # Error out, as the license was in avail_licenses so should
>                  # be on disk somewhere.
>                  bb.error("Cannot find text for license %s" % name)
>          else:
> -            # If it's not SPDX, or PD, or in available licenses, then NO_GENERIC_LICENSE must be set
> +            # If it's not SPDX, or PD, or in avail_licenses, then NO_GENERIC_LICENSE must be set
>              filename = d.getVarFlag('NO_GENERIC_LICENSE', name)
>              if filename:
>                  filename = d.expand("${S}/" + filename)
> --
> 2.32.0

//Peter

Re: [OE-core] [PATCH 1/2] create-spdx: Use function rather than AVAILABLE_LICENSES

[Richard Purdie]

On Tue, 2022-03-15 at 17:48 +0000, Peter Kjellerstedt wrote:
> > -----Original Message-----
> > From: openembedded-core@lists.openembedded.org
> > <openembedded-core@lists.openembedded.org> On Behalf Of Richard Purdie
> > Sent: den 15 mars 2022 18:33
> > To: openembedded-core@lists.openembedded.org
> > Subject: [OE-core] [PATCH 1/2] create-spdx: Use function rather than
> > AVAILABLE_LICENSES
> > 
> > We can directly call the function rather than using the variable
> > indirection.
> > As this is the last user of the variable, it then allows removal of it in a
> > followup patch.
> > 
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > ---
> >  meta/classes/create-spdx.bbclass | 8 ++++----
> >  1 file changed, 4 insertions(+), 4 deletions(-)
> > 
> > diff --git a/meta/classes/create-spdx.bbclass b/meta/classes/create-
> > spdx.bbclass
> > index daf99ca676c..1a4804a7c50 100644
> > --- a/meta/classes/create-spdx.bbclass
> > +++ b/meta/classes/create-spdx.bbclass
> > @@ -94,7 +94,7 @@ def convert_license_to_spdx(lic, document, d,
> > existing={}):
> >      from pathlib import Path
> >      import oe.spdx
> > 
> > -    available_licenses = d.getVar("AVAILABLE_LICENSES").split()
> > +    avail_licenses = available_licenses(d)
> 
> Doesn't this mean that the available licenses will be calculated once 
> per recipe? Or isn't it a problem since the time will not be spent 
> during recipe parsing, but rather during task execution where no one 
> will notice if the create-spdx task for all recipes takes a little 
> longer?

It will be in a different process so it will already run once per recipe during
task execution anyway. So yes, this just allows us to move execution overhead
away from parsing.

> That said, if I remember correctly from the last time I looked at this 
> function, it should be possible to rewrite it to not rely on 
> available_licenses() without any real loss in functionality.

Patches very welcome. Now we're moving to SPDX identifiers only, this need to
this is likely reduced too. I needed to start unwinding the pieces somewhere
though...

Cheers,

Richard