[PATCH 0/4] copyleft/license bits

[PATCH 0/4] copyleft/license bits

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

The following changes since commit fff18970706913e7fd7f4a119d798dddb44b388a:

  base-files: filesystems: fix mount order (2012-01-06 14:41:19 +0000)

are available in the git repository at:
  https://github.com/kergoth/oe-core license-bits

Christopher Larson (4):
      copyleft_compliance: add control of recipe types to include
      copyleft_compliance: add debug message with the reason for exclusion
      oe.license: add is_included convenience function
      oe.license: avoid the need to catch SyntaxError

 meta/classes/copyleft_compliance.bbclass |   56 ++++++++++++++++------------
 meta/lib/oe/license.py                   |   59 ++++++++++++++++++++++++++++--
 meta/lib/test.py                         |    3 ++
 3 files changed, 90 insertions(+), 28 deletions(-)
 create mode 100644 meta/lib/test.py

Christopher Larson (4):
  copyleft_compliance: add control of recipe types to include
  copyleft_compliance: add debug message with the reason for exclusion
  oe.license: add is_included convenience function
  oe.license: avoid the need to catch SyntaxError

 meta/classes/copyleft_compliance.bbclass |   56 ++++++++++++++++------------
 meta/lib/oe/license.py                   |   59 ++++++++++++++++++++++++++++--
 meta/lib/test.py                         |    3 ++
 3 files changed, 90 insertions(+), 28 deletions(-)
 create mode 100644 meta/lib/test.py

-- 
1.7.8.rc4

[PATCH 1/4] copyleft_compliance: add control of recipe types to include

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

In this context, recipe "type" refers to whether it is 'target', 'native',
'cross', etc. COPYLEFT_RECIPE_TYPES is a space separated list of types to
include. It defaults to 'target'.

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
 meta/classes/copyleft_compliance.bbclass |   21 ++++++++++++++++++---
 1 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/meta/classes/copyleft_compliance.bbclass b/meta/classes/copyleft_compliance.bbclass
index 2fe524b..37ed09e 100644
--- a/meta/classes/copyleft_compliance.bbclass
+++ b/meta/classes/copyleft_compliance.bbclass
@@ -16,6 +16,22 @@ COPYLEFT_LICENSE_EXCLUDE ?= 'CLOSED Proprietary'
 COPYLEFT_LICENSE_EXCLUDE[type] = 'list'
 COPYLEFT_LICENSE_INCLUDE[doc] = 'Space separated list of globs which exclude licenses'
 
+COPYLEFT_RECIPE_TYPE ?= '${@copyleft_recipe_type(d)}'
+COPYLEFT_RECIPE_TYPE[doc] = 'The "type" of the current recipe (e.g. target, native, cross)'
+
+COPYLEFT_RECIPE_TYPES ?= 'target'
+COPYLEFT_RECIPE_TYPES[type] = 'list'
+COPYLEFT_RECIPE_TYPES[doc] = 'Space separated list of recipe types to include'
+
+COPYLEFT_AVAILABLE_RECIPE_TYPES = 'target native nativesdk cross crosssdk cross-canadian'
+COPYLEFT_AVAILABLE_RECIPE_TYPES[type] = 'list'
+COPYLEFT_AVAILABLE_RECIPE_TYPES[doc] = 'Space separated list of available recipe types'
+
+def copyleft_recipe_type(d):
+    for recipe_type in oe.data.typed_value('COPYLEFT_AVAILABLE_RECIPE_TYPES', d):
+        if oe.utils.inherits(d, recipe_type):
+            return recipe_type
+    return 'target'
 
 def copyleft_should_include(d):
     """Determine if this recipe's sources should be deployed for compliance"""
@@ -23,9 +39,8 @@ def copyleft_should_include(d):
     import oe.license
     from fnmatch import fnmatchcase as fnmatch
 
-    if oe.utils.inherits(d, 'native', 'nativesdk', 'cross', 'crossdk'):
-        # not a target recipe
-        return
+    if d.getVar('COPYLEFT_RECIPE_TYPE', True) not in oe.data.typed_value('COPYLEFT_RECIPE_TYPES', d):
+        return False
 
     include = oe.data.typed_value('COPYLEFT_LICENSE_INCLUDE', d)
     exclude = oe.data.typed_value('COPYLEFT_LICENSE_EXCLUDE', d)
-- 
1.7.8.rc4

[PATCH 2/4] copyleft_compliance: add debug message with the reason for exclusion

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
 meta/classes/copyleft_compliance.bbclass |   24 +++++++++++++++++-------
 1 files changed, 17 insertions(+), 7 deletions(-)

diff --git a/meta/classes/copyleft_compliance.bbclass b/meta/classes/copyleft_compliance.bbclass
index 37ed09e..fd04638 100644
--- a/meta/classes/copyleft_compliance.bbclass
+++ b/meta/classes/copyleft_compliance.bbclass
@@ -39,8 +39,9 @@ def copyleft_should_include(d):
     import oe.license
     from fnmatch import fnmatchcase as fnmatch
 
-    if d.getVar('COPYLEFT_RECIPE_TYPE', True) not in oe.data.typed_value('COPYLEFT_RECIPE_TYPES', d):
-        return False
+    recipe_type = d.getVar('COPYLEFT_RECIPE_TYPE', True)
+    if recipe_type not in oe.data.typed_value('COPYLEFT_RECIPE_TYPES', d):
+        return False, 'recipe type "%s" is excluded' % recipe_type
 
     include = oe.data.typed_value('COPYLEFT_LICENSE_INCLUDE', d)
     exclude = oe.data.typed_value('COPYLEFT_LICENSE_EXCLUDE', d)
@@ -63,18 +64,27 @@ def copyleft_should_include(d):
         licenses = oe.license.flattened_licenses(d.getVar('LICENSE', True), choose_licenses)
     except oe.license.InvalidLicense as exc:
         bb.fatal('%s: %s' % (d.getVar('PF', True), exc))
-    except SyntaxError:
-        bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF', True)))
-
-    return all(include_license(lic) for lic in licenses)
+    except SyntaxError as exc:
+        bb.warn('%s: error when parsing the LICENSE variable: %s' % (d.getVar('P', True), exc))
+    else:
+        excluded = filter(lambda lic: not include_license(lic), licenses)
+        if excluded:
+            return False, 'recipe has excluded licenses: %s' % ', '.join(excluded)
+        else:
+            return True, None
 
 python do_prepare_copyleft_sources () {
     """Populate a tree of the recipe sources and emit patch series files"""
     import os.path
     import shutil
 
-    if not copyleft_should_include(d):
+    p = d.getVar('P', True)
+    included, reason = copyleft_should_include(d)
+    if not included:
+        bb.debug(1, 'copyleft: %s is excluded: %s' % (p, reason))
         return
+    else:
+        bb.debug(1, 'copyleft: %s is included' % p)
 
     sources_dir = d.getVar('COPYLEFT_SOURCES_DIR', 1)
     src_uri = d.getVar('SRC_URI', 1).split()
-- 
1.7.8.rc4

[PATCH 3/4] oe.license: add is_included convenience function

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

Given a license string and whitelist and blacklist, determine if the
license string matches the whitelist and does not match the blacklist.

When encountering an OR, it prefers the side with the highest weight (more
included licenses). It then checks the inclusion of the flattened list of
licenses from there.

Returns a tuple holding the boolean state and a list of the applicable
licenses which were excluded (or None, if the state is True)

Examples:

    is_included, excluded = oe.license.is_included(licensestr, ['GPL*', 'LGPL*'])
    is_included, excluded = oe.license.is_included(licensestr, blacklist=['Proprietary', 'CLOSED'])

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
 meta/classes/copyleft_compliance.bbclass |   23 +++---------------
 meta/lib/oe/license.py                   |   36 ++++++++++++++++++++++++++++++
 meta/lib/test.py                         |    3 ++
 3 files changed, 43 insertions(+), 19 deletions(-)
 create mode 100644 meta/lib/test.py

diff --git a/meta/classes/copyleft_compliance.bbclass b/meta/classes/copyleft_compliance.bbclass
index fd04638..6f058e0 100644
--- a/meta/classes/copyleft_compliance.bbclass
+++ b/meta/classes/copyleft_compliance.bbclass
@@ -46,32 +46,17 @@ def copyleft_should_include(d):
     include = oe.data.typed_value('COPYLEFT_LICENSE_INCLUDE', d)
     exclude = oe.data.typed_value('COPYLEFT_LICENSE_EXCLUDE', d)
 
-    def include_license(license):
-        if any(fnmatch(license, pattern) for pattern in exclude):
-            return False
-        if any(fnmatch(license, pattern) for pattern in include):
-            return True
-        return False
-
-    def choose_licenses(a, b):
-        """Select the left option in an OR if all its licenses are to be included"""
-        if all(include_license(lic) for lic in a):
-            return a
-        else:
-            return b
-
     try:
-        licenses = oe.license.flattened_licenses(d.getVar('LICENSE', True), choose_licenses)
+        is_included, excluded = oe.license.is_included(d.getVar('LICENSE', True), include, exclude)
     except oe.license.InvalidLicense as exc:
         bb.fatal('%s: %s' % (d.getVar('PF', True), exc))
     except SyntaxError as exc:
         bb.warn('%s: error when parsing the LICENSE variable: %s' % (d.getVar('P', True), exc))
     else:
-        excluded = filter(lambda lic: not include_license(lic), licenses)
-        if excluded:
-            return False, 'recipe has excluded licenses: %s' % ', '.join(excluded)
-        else:
+        if is_included:
             return True, None
+        else:
+            return False, 'recipe has excluded licenses: %s' % ', '.join(excluded)
 
 python do_prepare_copyleft_sources () {
     """Populate a tree of the recipe sources and emit patch series files"""
diff --git a/meta/lib/oe/license.py b/meta/lib/oe/license.py
index 7ab66e7..3543cfe 100644
--- a/meta/lib/oe/license.py
+++ b/meta/lib/oe/license.py
@@ -3,6 +3,7 @@
 
 import ast
 import re
+from fnmatch import fnmatchcase as fnmatch
 
 class InvalidLicense(StandardError):
     def __init__(self, license):
@@ -60,3 +61,38 @@ def flattened_licenses(licensestr, choose_licenses):
     flatten = FlattenVisitor(choose_licenses)
     flatten.visit_string(licensestr)
     return flatten.licenses
+
+def is_included(licensestr, whitelist=None, blacklist=None):
+    """Given a license string and whitelist and blacklist, determine if the
+    license string matches the whitelist and does not match the blacklist.
+
+    Returns a tuple holding the boolean state and a list of the applicable
+    licenses which were excluded (or None, if the state is True)
+    """
+
+    def include_license(license):
+        return (any(fnmatch(license, pattern) for pattern in whitelist) and not
+                any(fnmatch(license, pattern) for pattern in blacklist))
+
+    def choose_licenses(alpha, beta):
+        """Select the option in an OR which is the 'best' (has the most
+        included licenses)."""
+        alpha_weight = len(filter(include_license, alpha))
+        beta_weight = len(filter(include_license, beta))
+        if alpha_weight > beta_weight:
+            return alpha
+        else:
+            return beta
+
+    if not whitelist:
+        whitelist = ['*']
+
+    if not blacklist:
+        blacklist = []
+
+    licenses = flattened_licenses(licensestr, choose_licenses)
+    excluded = filter(lambda lic: not include_license(lic), licenses)
+    if excluded:
+        return False, excluded
+    else:
+        return True, None
diff --git a/meta/lib/test.py b/meta/lib/test.py
new file mode 100644
index 0000000..12f4d83
--- /dev/null
+++ b/meta/lib/test.py
@@ -0,0 +1,3 @@
+import oe.license
+
+print(oe.license.is_included('LGPLv2.1 | GPLv3', ['*'], []))
-- 
1.7.8.rc4

[PATCH 4/4] oe.license: avoid the need to catch SyntaxError

[Christopher Larson]

From: Christopher Larson <chris_larson@mentor.com>

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
 meta/classes/copyleft_compliance.bbclass |    4 +---
 meta/lib/oe/license.py                   |   23 +++++++++++++++++++----
 2 files changed, 20 insertions(+), 7 deletions(-)

diff --git a/meta/classes/copyleft_compliance.bbclass b/meta/classes/copyleft_compliance.bbclass
index 6f058e0..2eb9ded 100644
--- a/meta/classes/copyleft_compliance.bbclass
+++ b/meta/classes/copyleft_compliance.bbclass
@@ -48,10 +48,8 @@ def copyleft_should_include(d):
 
     try:
         is_included, excluded = oe.license.is_included(d.getVar('LICENSE', True), include, exclude)
-    except oe.license.InvalidLicense as exc:
+    except oe.license.LicenseError as exc:
         bb.fatal('%s: %s' % (d.getVar('PF', True), exc))
-    except SyntaxError as exc:
-        bb.warn('%s: error when parsing the LICENSE variable: %s' % (d.getVar('P', True), exc))
     else:
         if is_included:
             return True, None
diff --git a/meta/lib/oe/license.py b/meta/lib/oe/license.py
index 3543cfe..5914506 100644
--- a/meta/lib/oe/license.py
+++ b/meta/lib/oe/license.py
@@ -5,13 +5,25 @@ import ast
 import re
 from fnmatch import fnmatchcase as fnmatch
 
-class InvalidLicense(StandardError):
+class LicenseError(StandardError):
+    pass
+
+class LicenseSyntaxError(LicenseError):
+    def __init__(self, licensestr, exc):
+        self.licensestr = licensestr
+        self.exc = exc
+        LicenseError.__init__(self)
+
+    def __str__(self):
+        return "error in '%s': %s" % (self.licensestr, self.exc)
+
+class InvalidLicense(LicenseError):
     def __init__(self, license):
         self.license = license
-        StandardError.__init__(self)
+        LicenseError.__init__(self)
 
     def __str__(self):
-        return "invalid license '%s'" % self.license
+        return "invalid characters in license '%s'" % self.license
 
 license_operator = re.compile('([&|() ])')
 license_pattern = re.compile('[a-zA-Z0-9.+_\-]+$')
@@ -59,7 +71,10 @@ class FlattenVisitor(LicenseVisitor):
 def flattened_licenses(licensestr, choose_licenses):
     """Given a license string and choose_licenses function, return a flat list of licenses"""
     flatten = FlattenVisitor(choose_licenses)
-    flatten.visit_string(licensestr)
+    try:
+        flatten.visit_string(licensestr)
+    except SyntaxError as exc:
+        raise LicenseSyntaxError(licensestr, exc)
     return flatten.licenses
 
 def is_included(licensestr, whitelist=None, blacklist=None):
-- 
1.7.8.rc4

Re: [PATCH 0/4] copyleft/license bits

[Saul Wold]

On 01/10/2012 08:31 AM, Christopher Larson wrote:
> From: Christopher Larson<chris_larson@mentor.com>
>
> The following changes since commit fff18970706913e7fd7f4a119d798dddb44b388a:
>
>    base-files: filesystems: fix mount order (2012-01-06 14:41:19 +0000)
>
> are available in the git repository at:
>    https://github.com/kergoth/oe-core license-bits
>
> Christopher Larson (4):
>        copyleft_compliance: add control of recipe types to include
>        copyleft_compliance: add debug message with the reason for exclusion
>        oe.license: add is_included convenience function
>        oe.license: avoid the need to catch SyntaxError
>
>   meta/classes/copyleft_compliance.bbclass |   56 ++++++++++++++++------------
>   meta/lib/oe/license.py                   |   59 ++++++++++++++++++++++++++++--
>   meta/lib/test.py                         |    3 ++
>   3 files changed, 90 insertions(+), 28 deletions(-)
>   create mode 100644 meta/lib/test.py
>
> Christopher Larson (4):
>    copyleft_compliance: add control of recipe types to include
>    copyleft_compliance: add debug message with the reason for exclusion
>    oe.license: add is_included convenience function
>    oe.license: avoid the need to catch SyntaxError
>
>   meta/classes/copyleft_compliance.bbclass |   56 ++++++++++++++++------------
>   meta/lib/oe/license.py                   |   59 ++++++++++++++++++++++++++++--
>   meta/lib/test.py                         |    3 ++
>   3 files changed, 90 insertions(+), 28 deletions(-)
>   create mode 100644 meta/lib/test.py
>
Merged into OE-Core

Thanks
	Sau!