[PATCH 0/3] fix whitelist processing does not play well with wildcards in INCOMPATIBLE_LICENSES

[PATCH 0/3] fix whitelist processing does not play well with wildcards in INCOMPATIBLE_LICENSES

[Hongxu Jia]

Test Steps:

mpfr is "GPLv3 & LGPLv3", less is "GPLv3+ | BSD-2-Clause" but listed in WHITELIST_GPL-3.0
So while INCOMPATIBLE_LICENSE = "*GPL-3", we could build less, but skip mpfr.

$ bitbake less
...
NOTE: Tasks Summary: Attempted 587 tasks of which 440 didn't need to be rerun and all succeeded.
...

$ bitbake libiconv
...
ERROR: Nothing PROVIDES 'mpfr'
ERROR: mpfr was skipped: incompatible with license GPLv3 & LGPLv3
...

//Hongxu

The following changes since commit ec6377bcf52d105cd23ac6bbbeddd38fee9337e4:

  bitbake: bitbake-user-manual-metadata.xml: Updated do_package_write example (2014-12-09 22:25:36 +0000)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib hongxu/fix-incompatible_license
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=hongxu/fix-incompatible_license

Hongxu Jia (3):
  base/license.bbclass: expand wildcards in INCOMPATIBLE_LICENSE
  default-versions.inc: let INCOMPATIBLE_LICENSE supports wildcard
  libproxy: let INCOMPATIBLE_LICENSE supports wildcard

 meta/classes/base.bbclass                        |  2 +-
 meta/classes/license.bbclass                     | 25 ++++++++++++++++++++++++
 meta/conf/distro/include/default-versions.inc    |  2 +-
 meta/conf/documentation.conf                     |  2 +-
 meta/recipes-support/libproxy/libproxy_0.4.11.bb |  2 +-
 5 files changed, 29 insertions(+), 4 deletions(-)

-- 
1.9.1

[PATCH 1/3] base/license.bbclass: expand wildcards in INCOMPATIBLE_LICENSE

[Hongxu Jia]

The whitelist processing in code in base.bbclass does not play well with
wildcards in INCOMPATIBLE_LICENSES. The code expects bad_licenses to
contain actual license names, not wildcards.

Add incompatible_license_contains to replace bb.utils.contains(
"INCOMPATIBLE_LICENSE", **, **, **, d)

[YOCTO #5592]

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 meta/classes/base.bbclass    |  2 +-
 meta/classes/license.bbclass | 25 +++++++++++++++++++++++++
 meta/conf/documentation.conf |  2 +-
 3 files changed, 27 insertions(+), 2 deletions(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index de81a7d..06cfe26 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -443,7 +443,7 @@ python () {
                 check_license = False
 
         if check_license and bad_licenses:
-            bad_licenses = map(lambda l: canonical_license(d, l), bad_licenses)
+            bad_licenses = expand_wildcard_licenses(d, bad_licenses)
 
             whitelist = []
             for lic in bad_licenses:
diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
index 14d3107..ea4c880 100644
--- a/meta/classes/license.bbclass
+++ b/meta/classes/license.bbclass
@@ -285,6 +285,31 @@ def canonical_license(d, license):
             lic += '+'
     return lic or license
 
+def expand_wildcard_licenses(d, wildcard_licenses):
+    """
+    Return actual spdx format license names if wildcard used. We expand
+    wildcards from SPDXLICENSEMAP flags and SRC_DISTRIBUTE_LICENSES values.
+    """
+    import fnmatch
+    licenses = []
+    spdxmapkeys = d.getVarFlags('SPDXLICENSEMAP').keys()
+    for wld_lic in wildcard_licenses:
+        spdxflags = fnmatch.filter(spdxmapkeys, wld_lic)
+        licenses += [d.getVarFlag('SPDXLICENSEMAP', flag) for flag in spdxflags]
+
+    spdx_lics = (d.getVar('SRC_DISTRIBUTE_LICENSES') or '').split()
+    for wld_lic in wildcard_licenses:
+        licenses += fnmatch.filter(spdx_lics, wld_lic)
+
+    licenses = list(set(licenses))
+    return licenses
+
+def incompatible_license_contains(license, truevalue, falsevalue, d):
+    license = canonical_license(d, license)
+    bad_licenses = (d.getVar('INCOMPATIBLE_LICENSE', True) or "").split()
+    bad_licenses = expand_wildcard_licenses(d, bad_licenses)
+    return truevalue if license in bad_licenses else falsevalue
+
 def incompatible_license(d, dont_want_licenses, package=None):
     """
     This function checks if a recipe has only incompatible licenses. It also
diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf
index 5564316..2ab86e1 100644
--- a/meta/conf/documentation.conf
+++ b/meta/conf/documentation.conf
@@ -223,7 +223,7 @@ IMAGE_ROOTFS_EXTRA_SPACE[doc] = "Defines additional free disk space created in t
 IMAGE_ROOTFS_SIZE[doc] = "Defines the size in Kbytes for the generated image."
 IMAGE_TYPES[doc] = "Specifies the complete list of supported image types by default."
 INC_PR[doc] = "Helps define the recipe revision for recipes that share a common include file."
-INCOMPATIBLE_LICENSE[doc] = "Specifies a space-separated list of license names (as they would appear in LICENSE) that should be excluded from the build."
+INCOMPATIBLE_LICENSE[doc] = "Specifies a space-separated list of license names (as they would appear in LICENSE) that should be excluded from the build. Wildcard is supported, such as '*GPLv3'"
 INHIBIT_DEFAULT_DEPS[doc] = "Prevents the default dependencies, namely the C compiler and standard C library (libc), from being added to DEPENDS."
 INHIBIT_PACKAGE_STRIP[doc] = "If set to "1", causes the build to not strip binaries in resulting packages."
 INHERIT[doc] = "Causes the named class to be inherited at this point during parsing. The variable is only valid in configuration files."
-- 
1.9.1

[PATCH 2/3] default-versions.inc: let INCOMPATIBLE_LICENSE supports wildcard

[Hongxu Jia]

While wildcard in INCOMPATIBLE_LICENSE, such as INCOMPATIBLE_LICENSE =
"*GPL-3", PREFERRED_VERSION_db-native could have the correct value "5.%"

[YOCTO #5592]

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 meta/conf/distro/include/default-versions.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/distro/include/default-versions.inc b/meta/conf/distro/include/default-versions.inc
index 20e258b..6e92aa5 100644
--- a/meta/conf/distro/include/default-versions.inc
+++ b/meta/conf/distro/include/default-versions.inc
@@ -12,4 +12,4 @@ PREFERRED_VERSION_liberation-fonts ?= "1.04"
 
 # Force db-native's version to keep sync with db while
 # 'AGPL-3.0' in ${INCOMPATIBLE_LICENSE} blacklist
-PREFERRED_VERSION_db-native = "${@base_contains('INCOMPATIBLE_LICENSE', 'AGPL-3.0', '5.%', '6.%', d)}"
+PREFERRED_VERSION_db-native = "${@incompatible_license_contains('AGPL-3.0', '5.%', '6.%', d)}"
-- 
1.9.1

[PATCH 3/3] libproxy: let INCOMPATIBLE_LICENSE supports wildcard

[Hongxu Jia]

While wildcard in INCOMPATIBLE_LICENSE, such as INCOMPATIBLE_LICENSE =
"*GPL-3", libproxy could correct work.

[YOCTO #5592]

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
 meta/recipes-support/libproxy/libproxy_0.4.11.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-support/libproxy/libproxy_0.4.11.bb b/meta/recipes-support/libproxy/libproxy_0.4.11.bb
index a53a197..3367c85 100644
--- a/meta/recipes-support/libproxy/libproxy_0.4.11.bb
+++ b/meta/recipes-support/libproxy/libproxy_0.4.11.bb
@@ -29,7 +29,7 @@ do_configure_prepend() {
 }
 
 python() {
-    if bb.utils.contains("INCOMPATIBLE_LICENSE", "GPLv3", "x", "", d) == "x" or bb.utils.contains("DISTRO_FEATURES", "x11", "x", "", d) == "":
+    if incompatible_license_contains("GPLv3", "x", "", d) == "x" or bb.utils.contains("DISTRO_FEATURES", "x11", "x", "", d) == "":
         d.setVar("EXTRA_OECMAKE", d.getVar("EXTRA_OECMAKE").replace("-DWITH_GNOME=yes", "-DWITH_GNOME=no"))
         d.setVar("DEPENDS", " ".join(i for i in d.getVar("DEPENDS").split() if i != "gconf"))
 }
-- 
1.9.1

Re: [PATCH 0/3] fix whitelist processing does not play well with wildcards in INCOMPATIBLE_LICENSES

[Hongxu Jia]

On 12/15/2014 05:23 PM, Hongxu Jia wrote:
> Test Steps:
>
> mpfr is "GPLv3 & LGPLv3", less is "GPLv3+ | BSD-2-Clause" but listed in WHITELIST_GPL-3.0
> So while INCOMPATIBLE_LICENSE = "*GPL-3", we could build less, but skip mpfr.
>
> $ bitbake less
> ...
> NOTE: Tasks Summary: Attempted 587 tasks of which 440 didn't need to be rerun and all succeeded.
> ...
>
> $ bitbake libiconv

s/libiconv/mpfr/

//Hongxu

> ...
> ERROR: Nothing PROVIDES 'mpfr'
> ERROR: mpfr was skipped: incompatible with license GPLv3 & LGPLv3
> ...
>
> //Hongxu
>
> The following changes since commit ec6377bcf52d105cd23ac6bbbeddd38fee9337e4:
>
>    bitbake: bitbake-user-manual-metadata.xml: Updated do_package_write example (2014-12-09 22:25:36 +0000)
>
> are available in the git repository at:
>
>    git://git.pokylinux.org/poky-contrib hongxu/fix-incompatible_license
>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=hongxu/fix-incompatible_license
>
> Hongxu Jia (3):
>    base/license.bbclass: expand wildcards in INCOMPATIBLE_LICENSE
>    default-versions.inc: let INCOMPATIBLE_LICENSE supports wildcard
>    libproxy: let INCOMPATIBLE_LICENSE supports wildcard
>
>   meta/classes/base.bbclass                        |  2 +-
>   meta/classes/license.bbclass                     | 25 ++++++++++++++++++++++++
>   meta/conf/distro/include/default-versions.inc    |  2 +-
>   meta/conf/documentation.conf                     |  2 +-
>   meta/recipes-support/libproxy/libproxy_0.4.11.bb |  2 +-
>   5 files changed, 29 insertions(+), 4 deletions(-)
>