* [PATCH] meta: license: fix non-SPDX license being removed from INCOMPATIBLE_LICENSE @ 2019-06-05 15:23 Quentin Schulz 2019-06-06 14:52 ` Adrian Bunk 0 siblings, 1 reply; 4+ messages in thread From: Quentin Schulz @ 2019-06-05 15:23 UTC (permalink / raw) To: openembedded-core; +Cc: Quentin Schulz, andriy.danylovskyy A non-SPDX license (which is not an alias to an SPDX license) cannot currently be marked as incompatible in INCOMPATIBLE_LICENSE. In the current state, we take all INCOMPATIBLE_LICENSE and pass them through expand_wildcard_licenses which is only adding SPDX licenses that match the glob regexp of what is in INCOMPATIBLE_LICENSE (be it a direct match to an SPDX license or via an alias). This does not work well with custom licenses. E.g.: foo.bb: LICENSE = "FooLicense" conf/local.conf: INCOMPATIBLE_LICENSE = "FooLicense" `bitbake foo` Gives no warning, no error, builds and packages successfully, because INCOMPATIBLE_LICENSE is basically empty since FooLicense is neither in SPDXLICENSEMAP nor in SRC_DISTRIBUTE_LICENSES. Let's add the original licenses to the list returned by expand_wildcard_licenses to be able to handle the aforementioned case. INCOMPATIBLE_LICENSE = "FooLicense GPLv2 GPLv3+" used to "resolve" to "GPLv2 GPLv3". It now resolves to "FooLicense GPLv2 GPLv3 GPLv3+" which fixes the issue with custom licenses not being in SPDXLICENSEMAP or SRC_DISTRIBUTE_LICENSES and thus being left out of the blacklisted licenses. Signed-off-by: Quentin Schulz <quentin.schulz@streamunlimited.com> --- meta/classes/license.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass index ed91a4b4db..adca881c85 100644 --- a/meta/classes/license.bbclass +++ b/meta/classes/license.bbclass @@ -268,7 +268,7 @@ def expand_wildcard_licenses(d, wildcard_licenses): wildcards from SPDXLICENSEMAP flags and SRC_DISTRIBUTE_LICENSES values. """ import fnmatch - licenses = [] + licenses = wildcard_licenses[:] spdxmapkeys = d.getVarFlags('SPDXLICENSEMAP').keys() for wld_lic in wildcard_licenses: spdxflags = fnmatch.filter(spdxmapkeys, wld_lic) -- 2.17.1 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] meta: license: fix non-SPDX license being removed from INCOMPATIBLE_LICENSE 2019-06-05 15:23 [PATCH] meta: license: fix non-SPDX license being removed from INCOMPATIBLE_LICENSE Quentin Schulz @ 2019-06-06 14:52 ` Adrian Bunk 2019-06-06 15:11 ` Quentin Schulz 0 siblings, 1 reply; 4+ messages in thread From: Adrian Bunk @ 2019-06-06 14:52 UTC (permalink / raw) To: Quentin Schulz; +Cc: andriy.danylovskyy, openembedded-core This breaks the build for me: ERROR: test-image-1.0-r0 do_rootfs: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:license_create_manifest(d) 0003: File: '/home/bunk/yoctomusl/yocto-test/sources/poky/meta/classes/license_image.bbclass', lineno: 35, function: license_create_manifest 0031: pkg_dic[pkg_name]["LICENSE"] = pkg_dic[pkg_name][pkg_lic_name] 0032: 0033: rootfs_license_manifest = os.path.join(d.getVar('LICENSE_DIRECTORY'), 0034: d.getVar('IMAGE_NAME'), 'license.manifest') *** 0035: write_license_files(d, rootfs_license_manifest, pkg_dic, rootfs=True) 0036:} 0037: 0038:def write_license_files(d, license_manifest, pkg_dic, rootfs=True): 0039: import re File: '/home/bunk/yoctomusl/yocto-test/sources/poky/meta/classes/license_image.bbclass', lineno: 44, function: write_license_files 0040: import stat 0041: 0042: bad_licenses = (d.getVar("INCOMPATIBLE_LICENSE") or "").split() 0043: bad_licenses = map(lambda l: canonical_license(d, l), bad_licenses) *** 0044: bad_licenses = expand_wildcard_licenses(d, bad_licenses) 0045: 0046: with open(license_manifest, "w") as license_file: 0047: for pkg in sorted(pkg_dic): 0048: if bad_licenses: File: '/home/bunk/yoctomusl/yocto-test/sources/poky/meta/classes/license.bbclass', lineno: 271, function: expand_wildcard_licenses 0267: Return actual spdx format license names if wildcard used. We expand 0268: wildcards from SPDXLICENSEMAP flags and SRC_DISTRIBUTE_LICENSES values. 0269: """ 0270: import fnmatch *** 0271: licenses = wildcard_licenses[:] 0272: spdxmapkeys = d.getVarFlags('SPDXLICENSEMAP').keys() 0273: for wld_lic in wildcard_licenses: 0274: spdxflags = fnmatch.filter(spdxmapkeys, wld_lic) 0275: licenses += [d.getVarFlag('SPDXLICENSEMAP', flag) for flag in spdxflags] Exception: TypeError: 'map' object is not subscriptable ERROR: test-image-1.0-r0 do_rootfs: 'map' object is not subscriptable ERROR: test-image-1.0-r0 do_rootfs: Function failed: license_create_manifest ERROR: Logfile of failure stored in: /home/bunk/yoctomusl/yocto-test/build_test/tmp/work/test-test-linux-musl/test-image/1.0-r0/temp/log.do_rootfs.29928 ERROR: Task (/home/bunk/yoctomusl/yocto-test/sources/meta-test/recipes-test/images/test-image.bb:do_rootfs) failed with exit code '1' cu Adrian On Wed, Jun 05, 2019 at 05:23:52PM +0200, Quentin Schulz wrote: > A non-SPDX license (which is not an alias to an SPDX license) cannot > currently be marked as incompatible in INCOMPATIBLE_LICENSE. > In the current state, we take all INCOMPATIBLE_LICENSE and pass them > through expand_wildcard_licenses which is only adding SPDX licenses that > match the glob regexp of what is in INCOMPATIBLE_LICENSE (be it a direct > match to an SPDX license or via an alias). > > This does not work well with custom licenses. > > E.g.: > > foo.bb: > LICENSE = "FooLicense" > > conf/local.conf: > INCOMPATIBLE_LICENSE = "FooLicense" > > `bitbake foo` > > Gives no warning, no error, builds and packages successfully, because > INCOMPATIBLE_LICENSE is basically empty since FooLicense is neither in > SPDXLICENSEMAP nor in SRC_DISTRIBUTE_LICENSES. > > Let's add the original licenses to the list returned by > expand_wildcard_licenses to be able to handle the aforementioned case. > > INCOMPATIBLE_LICENSE = "FooLicense GPLv2 GPLv3+" used to "resolve" to > "GPLv2 GPLv3". It now resolves to "FooLicense GPLv2 GPLv3 GPLv3+" which > fixes the issue with custom licenses not being in SPDXLICENSEMAP or > SRC_DISTRIBUTE_LICENSES and thus being left out of the blacklisted > licenses. > > Signed-off-by: Quentin Schulz <quentin.schulz@streamunlimited.com> > --- > meta/classes/license.bbclass | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass > index ed91a4b4db..adca881c85 100644 > --- a/meta/classes/license.bbclass > +++ b/meta/classes/license.bbclass > @@ -268,7 +268,7 @@ def expand_wildcard_licenses(d, wildcard_licenses): > wildcards from SPDXLICENSEMAP flags and SRC_DISTRIBUTE_LICENSES values. > """ > import fnmatch > - licenses = [] > + licenses = wildcard_licenses[:] > spdxmapkeys = d.getVarFlags('SPDXLICENSEMAP').keys() > for wld_lic in wildcard_licenses: > spdxflags = fnmatch.filter(spdxmapkeys, wld_lic) > -- > 2.17.1 > > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] meta: license: fix non-SPDX license being removed from INCOMPATIBLE_LICENSE 2019-06-06 14:52 ` Adrian Bunk @ 2019-06-06 15:11 ` Quentin Schulz 2019-06-06 15:36 ` Adrian Bunk 0 siblings, 1 reply; 4+ messages in thread From: Quentin Schulz @ 2019-06-06 15:11 UTC (permalink / raw) To: Adrian Bunk; +Cc: andriy.danylovskyy, openembedded-core Hi Adrian, On Thu, Jun 06, 2019 at 05:52:36PM +0300, Adrian Bunk wrote: > This breaks the build for me: > > ERROR: test-image-1.0-r0 do_rootfs: Error executing a python function in exec_python_func() autogenerated: > > The stack trace of python calls that resulted in this exception/failure was: > File: 'exec_python_func() autogenerated', lineno: 2, function: <module> > 0001: > *** 0002:license_create_manifest(d) > 0003: > File: '/home/bunk/yoctomusl/yocto-test/sources/poky/meta/classes/license_image.bbclass', lineno: 35, function: license_create_manifest > 0031: pkg_dic[pkg_name]["LICENSE"] = pkg_dic[pkg_name][pkg_lic_name] > 0032: > 0033: rootfs_license_manifest = os.path.join(d.getVar('LICENSE_DIRECTORY'), > 0034: d.getVar('IMAGE_NAME'), 'license.manifest') > *** 0035: write_license_files(d, rootfs_license_manifest, pkg_dic, rootfs=True) > 0036:} > 0037: > 0038:def write_license_files(d, license_manifest, pkg_dic, rootfs=True): > 0039: import re > File: '/home/bunk/yoctomusl/yocto-test/sources/poky/meta/classes/license_image.bbclass', lineno: 44, function: write_license_files > 0040: import stat > 0041: > 0042: bad_licenses = (d.getVar("INCOMPATIBLE_LICENSE") or "").split() > 0043: bad_licenses = map(lambda l: canonical_license(d, l), bad_licenses) > *** 0044: bad_licenses = expand_wildcard_licenses(d, bad_licenses) > 0045: > 0046: with open(license_manifest, "w") as license_file: > 0047: for pkg in sorted(pkg_dic): > 0048: if bad_licenses: > File: '/home/bunk/yoctomusl/yocto-test/sources/poky/meta/classes/license.bbclass', lineno: 271, function: expand_wildcard_licenses > 0267: Return actual spdx format license names if wildcard used. We expand > 0268: wildcards from SPDXLICENSEMAP flags and SRC_DISTRIBUTE_LICENSES values. > 0269: """ > 0270: import fnmatch > *** 0271: licenses = wildcard_licenses[:] > 0272: spdxmapkeys = d.getVarFlags('SPDXLICENSEMAP').keys() > 0273: for wld_lic in wildcard_licenses: > 0274: spdxflags = fnmatch.filter(spdxmapkeys, wld_lic) > 0275: licenses += [d.getVarFlag('SPDXLICENSEMAP', flag) for flag in spdxflags] > Exception: TypeError: 'map' object is not subscriptable > > ERROR: test-image-1.0-r0 do_rootfs: 'map' object is not subscriptable > ERROR: test-image-1.0-r0 do_rootfs: Function failed: license_create_manifest > ERROR: Logfile of failure stored in: /home/bunk/yoctomusl/yocto-test/build_test/tmp/work/test-test-linux-musl/test-image/1.0-r0/temp/log.do_rootfs.29928 > ERROR: Task (/home/bunk/yoctomusl/yocto-test/sources/meta-test/recipes-test/images/test-image.bb:do_rootfs) failed with exit code '1' > Funnily enough, you sent the log when I was already working on fixing the issue :) I only tested the v1 on package builds and not image builds, my bad. Thanks for the report though, much appreciated. I've sent a v2 (and you're Cc'ed to it). Thanks, Quentin -- StreamUnlimited Engineering GmbH High Tech Campus Vienna, Gutheil-Schoder-Gasse 10, 1100 Vienna, Austria quentin.schulz@streamunlimited.com, www.streamunlimited.com ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] meta: license: fix non-SPDX license being removed from INCOMPATIBLE_LICENSE 2019-06-06 15:11 ` Quentin Schulz @ 2019-06-06 15:36 ` Adrian Bunk 0 siblings, 0 replies; 4+ messages in thread From: Adrian Bunk @ 2019-06-06 15:36 UTC (permalink / raw) To: Quentin Schulz; +Cc: andriy.danylovskyy, openembedded-core On Thu, Jun 06, 2019 at 05:11:37PM +0200, Quentin Schulz wrote: > Hi Adrian, Hi Quentin, >... > I've sent a v2 (and you're Cc'ed to it). thanks, I've confirmed this fixes my problem. > Thanks, > Quentin cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-06-06 15:36 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-06-05 15:23 [PATCH] meta: license: fix non-SPDX license being removed from INCOMPATIBLE_LICENSE Quentin Schulz 2019-06-06 14:52 ` Adrian Bunk 2019-06-06 15:11 ` Quentin Schulz 2019-06-06 15:36 ` Adrian Bunk
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox