[PATCH 0/3] archiver.bbclass: make it can filter the license

[PATCH 0/3] archiver.bbclass: make it can filter the license

[Robert Yang]

After talked with Chris, we use the copyleft_filter.bbclass which is
from copyleft_compliance.bbclass to filter the license for both
archiver.bbclass and copyleft_compliance.bbclass, and we may remove
copyleft_compliance.bbclass in post 1.6, but we will keep it now.

// Robert

The following changes since commit 4a47c943a76e79af82d47d8fa83c6687e0d915a0:

  busybox: disable fsck.minix and mkfs.minix (2014-03-23 09:08:49 +0000)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib rbt/archiver_license
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/archiver_license

Robert Yang (3):
  archiver.bbclass: make it can filter the license
  archiver.bbclass: emit patch series files for original src
  local.conf.sample.extended: update it for the archiver

 meta/classes/archiver.bbclass            | 40 +++++++++++++++++++--
 meta/classes/copyleft_compliance.bbclass | 56 +----------------------------
 meta/classes/copyleft_filter.bbclass     | 62 ++++++++++++++++++++++++++++++++
 meta/conf/local.conf.sample.extended     | 10 +++++-
 4 files changed, 110 insertions(+), 58 deletions(-)
 create mode 100644 meta/classes/copyleft_filter.bbclass

-- 
1.8.3.1

[PATCH 1/3] archiver.bbclass: make it can filter the license

[Robert Yang]

* Filter the license (default: no), the recipe whose license in
  COPYLEFT_LICENSE_INCLUDE will be included, and in
  COPYLEFT_LICENSE_EXCLUDE will be excluded.

* The user can set the recipe type that would be archived (native,
  target, and so on), deafult to all.

The copyleft_filter.bbclass is come from copyleft_compliance.bbclass,
which is used by both copyleft_compliance.bbclass and archiver.bbclass.

[YOCTO #5740]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/archiver.bbclass            | 27 ++++++++++++--
 meta/classes/copyleft_compliance.bbclass | 56 +----------------------------
 meta/classes/copyleft_filter.bbclass     | 62 ++++++++++++++++++++++++++++++++
 3 files changed, 88 insertions(+), 57 deletions(-)
 create mode 100644 meta/classes/copyleft_filter.bbclass

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index 7780c71..9aa9335 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -12,9 +12,23 @@
 # 5) The environment data, similar to 'bitbake -e recipe':
 #    ARCHIVER_MODE[dumpdata] = "1"
 # 6) The recipe (.bb and .inc): ARCHIVER_MODE[recipe] = "1"
+# 7) Whether output the .src.rpm package:
+#    ARCHIVER_MODE[srpm] = "1"
+# 8) Filter the license, the recipe whose license in
+#    COPYLEFT_LICENSE_INCLUDE will be included, and in
+#    COPYLEFT_LICENSE_EXCLUDE will be excluded.
+#    COPYLEFT_LICENSE_INCLUDE = 'GPL* LGPL*'
+#    COPYLEFT_LICENSE_EXCLUDE = 'CLOSED Proprietary'
+# 9) The recipe type that will be archived:
+#    COPYLEFT_RECIPE_TYPES = 'target'
 #
-# All of the above can be packed into a .src.rpm package: (when PACKAGES != "")
-# ARCHIVER_MODE[srpm] = "1"
+
+# Don't filter the license by default
+COPYLEFT_LICENSE_INCLUDE ?= ''
+COPYLEFT_LICENSE_EXCLUDE ?= ''
+# Create archive for all the recipe types
+COPYLEFT_RECIPE_TYPES ?= 'target native nativesdk cross crosssdk cross-canadian'
+inherit copyleft_filter
 
 ARCHIVER_MODE[srpm] ?= "0"
 ARCHIVER_MODE[src] ?= "patched"
@@ -38,6 +52,15 @@ do_ar_original[dirs] = "${ARCHIVER_OUTDIR} ${ARCHIVER_WORKDIR}"
 python () {
     pn = d.getVar('PN', True)
 
+    if d.getVar('COPYLEFT_LICENSE_INCLUDE', True) or \
+            d.getVar('COPYLEFT_LICENSE_EXCLUDE', True):
+        included, reason = copyleft_should_include(d)
+        if not included:
+            bb.debug(1, 'archiver: %s is excluded: %s' % (pn, reason))
+            return
+        else:
+            bb.debug(1, 'archiver: %s is included: %s' % (pn, reason))
+
     ar_src = d.getVarFlag('ARCHIVER_MODE', 'src', True)
     ar_dumpdata = d.getVarFlag('ARCHIVER_MODE', 'dumpdata', True)
     ar_recipe = d.getVarFlag('ARCHIVER_MODE', 'recipe', True)
diff --git a/meta/classes/copyleft_compliance.bbclass b/meta/classes/copyleft_compliance.bbclass
index b47c2c9..907c183 100644
--- a/meta/classes/copyleft_compliance.bbclass
+++ b/meta/classes/copyleft_compliance.bbclass
@@ -2,66 +2,12 @@
 # Defaults to using symlinks, as it's a quick operation, and one can easily
 # follow the links when making use of the files (e.g. tar with the -h arg).
 #
-# By default, includes all GPL and LGPL, and excludes CLOSED and Proprietary.
-#
 # vi:sts=4:sw=4:et
 
-COPYLEFT_LICENSE_INCLUDE ?= 'GPL* LGPL*'
-COPYLEFT_LICENSE_INCLUDE[type] = 'list'
-COPYLEFT_LICENSE_INCLUDE[doc] = 'Space separated list of globs which include licenses'
-
-COPYLEFT_LICENSE_EXCLUDE ?= 'CLOSED Proprietary'
-COPYLEFT_LICENSE_EXCLUDE[type] = 'list'
-COPYLEFT_LICENSE_EXCLUDE[doc] = 'Space separated list of globs which exclude licenses'
-
-COPYLEFT_RECIPE_TYPE ?= '${@copyleft_recipe_type(d)}'
-COPYLEFT_RECIPE_TYPE[doc] = 'The "type" of the current recipe (e.g. target, native, cross)'
-
-COPYLEFT_RECIPE_TYPES ?= 'target'
-COPYLEFT_RECIPE_TYPES[type] = 'list'
-COPYLEFT_RECIPE_TYPES[doc] = 'Space separated list of recipe types to include'
-
-COPYLEFT_AVAILABLE_RECIPE_TYPES = 'target native nativesdk cross crosssdk cross-canadian'
-COPYLEFT_AVAILABLE_RECIPE_TYPES[type] = 'list'
-COPYLEFT_AVAILABLE_RECIPE_TYPES[doc] = 'Space separated list of available recipe types'
+inherit copyleft_filter
 
 COPYLEFT_SOURCES_DIR ?= '${DEPLOY_DIR}/copyleft_sources'
 
-def copyleft_recipe_type(d):
-    for recipe_type in oe.data.typed_value('COPYLEFT_AVAILABLE_RECIPE_TYPES', d):
-        if oe.utils.inherits(d, recipe_type):
-            return recipe_type
-    return 'target'
-
-def copyleft_should_include(d):
-    """
-    Determine if this recipe's sources should be deployed for compliance
-    """
-    import ast
-    import oe.license
-    from fnmatch import fnmatchcase as fnmatch
-
-    recipe_type = d.getVar('COPYLEFT_RECIPE_TYPE', True)
-    if recipe_type not in oe.data.typed_value('COPYLEFT_RECIPE_TYPES', d):
-        return False, 'recipe type "%s" is excluded' % recipe_type
-
-    include = oe.data.typed_value('COPYLEFT_LICENSE_INCLUDE', d)
-    exclude = oe.data.typed_value('COPYLEFT_LICENSE_EXCLUDE', d)
-
-    try:
-        is_included, reason = oe.license.is_included(d.getVar('LICENSE', True), include, exclude)
-    except oe.license.LicenseError as exc:
-        bb.fatal('%s: %s' % (d.getVar('PF', True), exc))
-    else:
-        if is_included:
-            if reason:
-                return True, 'recipe has included licenses: %s' % ', '.join(reason)
-            else:
-                return False, 'recipe does not include a copyleft license'
-        else:
-            return False, 'recipe has excluded licenses: %s' % ', '.join(reason)
-
-
 python do_prepare_copyleft_sources () {
     """Populate a tree of the recipe sources and emit patch series files"""
     import os.path
diff --git a/meta/classes/copyleft_filter.bbclass b/meta/classes/copyleft_filter.bbclass
new file mode 100644
index 0000000..2c1d8f1
--- /dev/null
+++ b/meta/classes/copyleft_filter.bbclass
@@ -0,0 +1,62 @@
+# Filter the license, the copyleft_should_include returns True for the
+# COPYLEFT_LICENSE_INCLUDE recipe, and False for the
+# COPYLEFT_LICENSE_EXCLUDE.
+#
+# By default, includes all GPL and LGPL, and excludes CLOSED and Proprietary.
+#
+# vi:sts=4:sw=4:et
+
+COPYLEFT_LICENSE_INCLUDE ?= 'GPL* LGPL*'
+COPYLEFT_LICENSE_INCLUDE[type] = 'list'
+COPYLEFT_LICENSE_INCLUDE[doc] = 'Space separated list of globs which include licenses'
+
+COPYLEFT_LICENSE_EXCLUDE ?= 'CLOSED Proprietary'
+COPYLEFT_LICENSE_EXCLUDE[type] = 'list'
+COPYLEFT_LICENSE_EXCLUDE[doc] = 'Space separated list of globs which exclude licenses'
+
+COPYLEFT_RECIPE_TYPE ?= '${@copyleft_recipe_type(d)}'
+COPYLEFT_RECIPE_TYPE[doc] = 'The "type" of the current recipe (e.g. target, native, cross)'
+
+COPYLEFT_RECIPE_TYPES ?= 'target'
+COPYLEFT_RECIPE_TYPES[type] = 'list'
+COPYLEFT_RECIPE_TYPES[doc] = 'Space separated list of recipe types to include'
+
+COPYLEFT_AVAILABLE_RECIPE_TYPES = 'target native nativesdk cross crosssdk cross-canadian'
+COPYLEFT_AVAILABLE_RECIPE_TYPES[type] = 'list'
+COPYLEFT_AVAILABLE_RECIPE_TYPES[doc] = 'Space separated list of available recipe types'
+
+def copyleft_recipe_type(d):
+    for recipe_type in oe.data.typed_value('COPYLEFT_AVAILABLE_RECIPE_TYPES', d):
+        if oe.utils.inherits(d, recipe_type):
+            return recipe_type
+    return 'target'
+
+def copyleft_should_include(d):
+    """
+    Determine if this recipe's sources should be deployed for compliance
+    """
+    import ast
+    import oe.license
+    from fnmatch import fnmatchcase as fnmatch
+
+    recipe_type = d.getVar('COPYLEFT_RECIPE_TYPE', True)
+    if recipe_type not in oe.data.typed_value('COPYLEFT_RECIPE_TYPES', d):
+        return False, 'recipe type "%s" is excluded' % recipe_type
+
+    include = oe.data.typed_value('COPYLEFT_LICENSE_INCLUDE', d)
+    exclude = oe.data.typed_value('COPYLEFT_LICENSE_EXCLUDE', d)
+
+    try:
+        is_included, reason = oe.license.is_included(d.getVar('LICENSE', True), include, exclude)
+    except oe.license.LicenseError as exc:
+        bb.fatal('%s: %s' % (d.getVar('PF', True), exc))
+    else:
+        if is_included:
+            if reason:
+                return True, 'recipe has included licenses: %s' % ', '.join(reason)
+            else:
+                return False, 'recipe does not include a copyleft license'
+        else:
+            return False, 'recipe has excluded licenses: %s' % ', '.join(reason)
+
+
-- 
1.8.3.1

[PATCH 2/3] archiver.bbclass: emit patch series files for original src

[Robert Yang]

Emit patch series files for original src so that the user knows how to
apply the patch orderly.

[YOCTO #5113]

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/archiver.bbclass | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index 9aa9335..8d8e7c4 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -150,6 +150,19 @@ python do_ar_original() {
             tar = tarfile.open(tarname, 'w:gz')
             tar.add('.')
             tar.close()
+
+    # Emit patch series files for 'original'
+    bb.note('Writing patch series files...')
+    for patch in src_patches(d):
+        _, _, local, _, _, parm = bb.fetch.decodeurl(patch)
+        patchdir = parm.get('patchdir')
+        if patchdir:
+            series = os.path.join(ar_outdir, 'series.subdir.%s' % patchdir.replace('/', '_'))
+        else:
+            series = os.path.join(ar_outdir, 'series')
+
+        with open(series, 'a') as s:
+            s.write('%s -p%s\n' % (os.path.basename(local), parm['striplevel']))
 }
 
 python do_ar_patched() {
-- 
1.8.3.1

[PATCH 3/3] local.conf.sample.extended: update it for the archiver

[Robert Yang]

Add the filter license comment for archiver.bbclass.

[YOCTO #5740]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/conf/local.conf.sample.extended | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/meta/conf/local.conf.sample.extended b/meta/conf/local.conf.sample.extended
index aa01917..048aa8b 100644
--- a/meta/conf/local.conf.sample.extended
+++ b/meta/conf/local.conf.sample.extended
@@ -214,9 +214,17 @@
 # 6) the recipe (.bb and .inc):
 #ARCHIVER_MODE[recipe] = "1"
 #
-# Whether output the .src.rpm package:
+# 7) Whether output the .src.rpm package:
 #ARCHIVER_MODE[srpm] = "1"
 #
+# 8) Filter the license, the recipe whose license in
+#    COPYLEFT_LICENSE_INCLUDE will be included, and in
+#    COPYLEFT_LICENSE_EXCLUDE will be excluded.
+#    COPYLEFT_LICENSE_INCLUDE = 'GPL* LGPL*'
+#    COPYLEFT_LICENSE_EXCLUDE = 'CLOSED Proprietary'
+# 9) The recipe type that will be archived:
+#    COPYLEFT_RECIPE_TYPES = '<type1> <type2> ...'
+#    The type can be: target native nativesdk cross crosssdk cross-canadian.
 
 # Remove the old image before the new one generated to save disk space
 #RM_OLD_IMAGE = "1"
-- 
1.8.3.1

Re: [PATCH 3/3] local.conf.sample.extended: update it for the archiver

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

On Sun, Mar 23, 2014 at 10:39 PM, Robert Yang <liezhi.yang@windriver.com>wrote:

> +# 9) The recipe type that will be archived:
> +#    COPYLEFT_RECIPE_TYPES = '<type1> <type2> ...'
> +#    The type can be: target native nativesdk cross crosssdk
> cross-canadian.
>

Hmm, I wonder if we should note that the default is to archive all types,
here.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 910 bytes --]

Re: [PATCH 3/3] local.conf.sample.extended: update it for the archiver

[Robert Yang]

On 03/24/2014 11:34 PM, Chris Larson wrote:
> On Sun, Mar 23, 2014 at 10:39 PM, Robert Yang <liezhi.yang@windriver.com>wrote:
>
>> +# 9) The recipe type that will be archived:
>> +#    COPYLEFT_RECIPE_TYPES = '<type1> <type2> ...'
>> +#    The type can be: target native nativesdk cross crosssdk
>> cross-canadian.
>>
>
> Hmm, I wonder if we should note that the default is to archive all types,
> here.
>

Good idea, I've updated it and pushed to:

git://git.openembedded.org/openembedded-core-contrib rbt/archiver_license
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/archiver_license


diff --git a/meta/conf/local.conf.sample.extended 
b/meta/conf/local.conf.sample.extended
index aa01917..a1f2464 100644
--- a/meta/conf/local.conf.sample.extended
+++ b/meta/conf/local.conf.sample.extended
@@ -214,9 +214,20 @@
  # 6) the recipe (.bb and .inc):
  #ARCHIVER_MODE[recipe] = "1"
  #
-# Whether output the .src.rpm package:
+# 7) Whether output the .src.rpm package:
  #ARCHIVER_MODE[srpm] = "1"
  #
+# 8) Filter the license, the recipe whose license in
+#    COPYLEFT_LICENSE_INCLUDE will be included, and in
+#    COPYLEFT_LICENSE_EXCLUDE will be excluded.
+#COPYLEFT_LICENSE_INCLUDE = 'GPL* LGPL*'
+#COPYLEFT_LICENSE_EXCLUDE = 'CLOSED Proprietary'
+#
+# 9) Config the recipe type that will be archived, the type can be
+#    target, native, nativesdk, cross, crosssdk and cross-canadian,
+#    you can set one or more types. Archive all types by default.
+#COPYLEFT_RECIPE_TYPES = 'target'
+#

-R

Re: [PATCH 3/3] local.conf.sample.extended: update it for the archiver

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 991 bytes --]

On Mon, Mar 24, 2014 at 6:43 PM, Robert Yang <liezhi.yang@windriver.com>wrote:

> On 03/24/2014 11:34 PM, Chris Larson wrote:
>
>> On Sun, Mar 23, 2014 at 10:39 PM, Robert Yang <liezhi.yang@windriver.com>
>> wrote:
>>
>>  +# 9) The recipe type that will be archived:
>>> +#    COPYLEFT_RECIPE_TYPES = '<type1> <type2> ...'
>>> +#    The type can be: target native nativesdk cross crosssdk
>>> cross-canadian.
>>>
>>>
>> Hmm, I wonder if we should note that the default is to archive all types,
>> here.
>>
>>
> Good idea, I've updated it and pushed to:
>
> git://git.openembedded.org/openembedded-core-contrib rbt/archiver_license
> http://cgit.openembedded.org/cgit.cgi/openembedded-core-
> contrib/log/?h=rbt/archiver_license


Looks good to me.

Signed-off-by: Christopher Larson <kergoth@gmail.com>
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 1988 bytes --]