[PATCH] recipetool: create: allow handers to set license

[PATCH] recipetool: create: allow handers to set license

[Mark D Horn]

Recipetool plugins set through register_recipe_handlers were not able
to impact the license type via setting extravalues['LICENSE']. This is
due to caching the license variables in create_recipe before the handlers
have been executed.

This change moves the call to handle_license_vars well after the
registered plugins (and extravalue functions) have been called.

Signed-off-by: Mark D Horn <mark.d.horn@intel.com>
---
 scripts/lib/recipetool/create.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index 359eb9adfc26..6dd1a81369bb 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -549,8 +549,6 @@ def create_recipe(args):
     lines_before.append('')
 
     handled = []
-    licvalues = handle_license_vars(srctree_use, lines_before, handled, extravalues, tinfoil.config_data)
-
     classes = []
 
     # FIXME This is kind of a hack, we probably ought to be using bitbake to do this
@@ -769,6 +767,8 @@ def create_recipe(args):
         outlines.append('')
     outlines.extend(lines_after)
 
+    licvalues = handle_license_vars(srctree_use, lines_before, handled, extravalues, tinfoil.config_data)
+
     if extravalues:
         if 'LICENSE' in extravalues and not licvalues:
             # Don't blow away 'CLOSED' value that comments say we set
-- 
2.7.4

Re: [PATCH] recipetool: create: allow handers to set license

[Richard Purdie]

On Wed, 2017-08-02 at 16:09 -0700, Mark D Horn wrote:
> Recipetool plugins set through register_recipe_handlers were not able
> to impact the license type via setting extravalues['LICENSE']. This
> is
> due to caching the license variables in create_recipe before the
> handlers
> have been executed.
> 
> This change moves the call to handle_license_vars well after the
> registered plugins (and extravalue functions) have been called.
> 
> Signed-off-by: Mark D Horn <mark.d.horn@intel.com>
> ---
>  scripts/lib/recipetool/create.py | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

I'm afraid this leads to oe-selftest regressions and issues in devtool:

$ oe-selftest -r devtool.DevtoolTests.test_devtool_add
[...]
2017-08-10 10:58:01,398 - oe-selftest - INFO - ======================================================================
2017-08-10 10:58:01,399 - oe-selftest - INFO - FAIL [6.404s]: test_devtool_add (devtool.DevtoolTests)
2017-08-10 10:58:01,399 - oe-selftest - INFO - ----------------------------------------------------------------------
2017-08-10 10:58:01,399 - oe-selftest - INFO - Traceback (most recent call last):
  File "/media/build1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/media/build1/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 206, in test_devtool_add
    result = runCmd('devtool add pv %s' % srcdir)
  File "/media/build1/poky/meta/lib/oeqa/utils/commands.py", line 191, in runCmd
    raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'devtool add pv /tmp/devtoolqaznmpqyo4/pv-1.5.3' returned non-zero exit status 1:
NOTE: Starting bitbake server...
NOTE: Creating workspace layer in /media/build1/poky/build/workspace
NOTE: Enabling workspace layer in bblayers.conf
NOTE: Starting bitbake server...
NOTE: Starting bitbake server...
ERROR: Traceback (most recent call last):
  File "/media/build1/poky/bitbake/lib/bb/command.py", line 83, in runCommand
    result = command_method(self, commandline)
  File "/media/build1/poky/bitbake/lib/bb/command.py", line 566, in parseRecipeFile
    envdata = parser.loadDataFull(fn, appendfiles)
  File "/media/build1/poky/bitbake/lib/bb/cache.py", line 339, in loadDataFull
    bb_data = self.load_bbfile(virtualfn, appends, virtonly=True)
  File "/media/build1/poky/bitbake/lib/bb/cache.py", line 352, in load_bbfile
    datastores = parse_recipe(bb_data, bbfile, appends, mc)
  File "/media/build1/poky/bitbake/lib/bb/cache.py", line 315, in parse_recipe
    bb_data = bb.parse.handle(bbfile, bb_data)
  File "/media/build1/poky/bitbake/lib/bb/parse/__init__.py", line 113, in handle
    return h['handle'](fn, data, include)
  File "/media/build1/poky/bitbake/lib/bb/parse/parse_py/BBHandler.py", line 157, in handle
    return ast.multi_finalize(fn, d)
  File "/media/build1/poky/bitbake/lib/bb/parse/ast.py", line 395, in multi_finalize
    finalize(fn, d)
  File "/media/build1/poky/bitbake/lib/bb/parse/ast.py", line 355, in finalize
    bb.utils.better_exec("\n".join(code), {"d": d})
  File "/media/build1/poky/bitbake/lib/bb/utils.py", line 399, in better_exec
    exec(code, get_context(), context)
  File "<code>", line 3, in <module>
  File "/media/build1/poky/meta/classes/base.bbclass", line 468, in __anon_686__media_build1_poky_meta_classes_base_bbclass
  File "/media/build1/poky/bitbake/lib/bb/__init__.py", line 104, in fatal
    raise BBHandledException()
bb.BBHandledException

ERROR: Parsing newly created recipe failed, moving recipe to /media/build1/poky/build/workspace/recipes/pv/pv_1.5.3.bb.parsefailed for reference. If this looks to be caused by the recipe itself, please report this error.
----------------------------------------------------------------------
2017-08-10 10:58:01,399 - oe-selftest - INFO - Ran 1 test in 7.684s
2017-08-10 10:58:01,399 - oe-selftest - INFO - FAILED
2017-08-10 10:58:01,399 - oe-selftest - INFO -  (failures=1)
2017-08-10 10:58:01,404 - oe-selftest - INFO - RESULTS:
2017-08-10 10:58:01,405 - oe-selftest - INFO - RESULTS - devtool.DevtoolTests.test_devtool_add - Testcase 1159: FAILED
2017-08-10 10:58:01,405 - oe-selftest - INFO - SUMMARY:
2017-08-10 10:58:01,405 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 7.690s
2017-08-10 10:58:01,405 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed


There were other failures but this one seem to illustrate the issue.

Cheers,

Richard

Re: [PATCH] recipetool: create: allow handers to set license

[Paul Eggleton]

On Thursday, 10 August 2017 10:00:39 PM NZST Richard Purdie wrote:
> On Wed, 2017-08-02 at 16:09 -0700, Mark D Horn wrote:
> > Recipetool plugins set through register_recipe_handlers were not able
> > to impact the license type via setting extravalues['LICENSE']. This
> > is
> > due to caching the license variables in create_recipe before the
> > handlers
> > have been executed.
> > 
> > This change moves the call to handle_license_vars well after the
> > registered plugins (and extravalue functions) have been called.
> > 
> > Signed-off-by: Mark D Horn <mark.d.horn@intel.com>
> > ---
> >  scripts/lib/recipetool/create.py | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> I'm afraid this leads to oe-selftest regressions and issues in devtool:
>...

FYI Mark and I have discussed this offline and I have a reworked change for 
which the tests pass - I just need to sort out where some of the other devtool 
patches are and I will send it out.

Cheers,
Paul


-- 

Paul Eggleton
Intel Open Source Technology Centre