[PATCH 0/4] icecc improvements

[PATCH 0/4] icecc improvements

[Martin Jansa]

Fix for icecc inherit causing allarch and native recipes to have different
signatures for different MACHINES.

Make blacklist more strict and start with empty one - old entries seem all
to be resolved already.

The following changes since commit 8163854adf87ac42a8f08ee25685d0ce1efb4724:

  oe-selftest: separated the SStateBase and SStateTests in different modules (2014-01-16 12:18:44 +0000)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib jansa/icecc
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=jansa/icecc

Martin Jansa (4):
  icecc: Don't replace non-empty PARALLEL_MAKE with empty
    ICECC_PARALLEL_MAKE
  icecc: use bb.utils.which also for 'as'
  icecc: Fix allarch and native recipes having different signatures
  icecc: use exact match in blacklists, re-start with empty
    system_package_blacklist

 meta/classes/icecc.bbclass | 46 +++++++++++++++++++++++++++++-----------------
 1 file changed, 29 insertions(+), 17 deletions(-)

-- 
1.8.5.2

[PATCH 1/4] icecc: Don't replace non-empty PARALLEL_MAKE with empty ICECC_PARALLEL_MAKE

[Martin Jansa]

* it's needed for use-case like this:
  # Inherit icecc here, so that all builders have the same sstate signatures
  INHERIT_DISTRO += "icecc"
  # and then disable its function by default (so that people still need to explicity
  # enable it in local.conf if they have configured icecc and want to use it.
  # You need to set _empty_ value in local.conf to enable icecc function:
  # ICECC_DISABLED = ""
  ICECC_DISABLED ??= "1"
* so default ICECC_PARALLEL_MAKE is still empty, but we want build
  to respect our PARALLEL_MAKE, unfortunately we cannot do something
  like ICECC_PARALLEL_MAKE ??= "${PARALLEL_MAKE}", because that would
  cause PARALLEL_MAKE to reference itself.

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 meta/classes/icecc.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index 7147d2f..5af764d 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -147,7 +147,7 @@ def icc_version(bb, d):
         return ""
 
     parallel = d.getVar('ICECC_PARALLEL_MAKE') or ""
-    if not d.getVar('PARALLEL_MAKE') == "":
+    if not d.getVar('PARALLEL_MAKE') == "" and parallel:
         d.setVar("PARALLEL_MAKE", parallel)
 
     if icc_is_native(bb, d):
-- 
1.8.5.2

[PATCH 2/4] icecc: use bb.utils.which also for 'as'

[Martin Jansa]

* it was introduced in
  commit 3a842ec52e7d010767b13bdcb5629ac07b3ee9e7
  Author: Dmitry Eremin-Solenikov <dbaryshkov@gmail.com>
  Date:   Fri Sep 16 10:55:16 2011 +0400
  Subject: icecc.bbclass: replace with updated version

  without any explanation in which case
    ${ICECC_CC} -print-prog-name=as
  is returning as in current working directory, but will keep old
  behavior just in case

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 meta/classes/icecc.bbclass | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index 5af764d..054e25c 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -251,6 +251,8 @@ set_icecc_env() {
 
     ICECC_CC="${@icc_get_and_check_tool(bb, d, "gcc")}"
     ICECC_CXX="${@icc_get_and_check_tool(bb, d, "g++")}"
+    # cannot use icc_get_and_check_tool here because it assumes as without target_sys prefix
+    ICECC_WHICH_AS="${@bb.utils.which(os.getenv('PATH'), 'as')}"
     if [ ! -x "${ICECC_CC}" -o ! -x "${ICECC_CXX}" ]
     then
         bbwarn "Cannot use icecc: could not get ICECC_CC or ICECC_CXX"
@@ -266,9 +268,12 @@ set_icecc_env() {
     fi
 
     ICECC_AS="`${ICECC_CC} -print-prog-name=as`"
+    # for target recipes should return something like:
+    # /OE/tmp-eglibc/sysroots/x86_64-linux/usr/libexec/arm920tt-oe-linux-gnueabi/gcc/arm-oe-linux-gnueabi/4.8.2/as
+    # and just "as" for native, if it returns "as" in current directory (for whatever reason) use "as" from PATH
     if [ "`dirname "${ICECC_AS}"`" = "." ]
     then
-        ICECC_AS="`which as`"
+        ICECC_AS="${ICECC_WHICH_AS}"
     fi
 
     if [ ! -f "${ICECC_VERSION}.done" ]
-- 
1.8.5.2

[PATCH 3/4] icecc: Fix allarch and native recipes having different signatures

[Martin Jansa]

* for different MACHINES
* is there more elegant way to have "overridable" function so that
  signature handler properly uses only the branch without
  STAGING_BINDIR_TOOLCHAIN?

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 meta/classes/icecc.bbclass | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index 054e25c..549bd69 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -142,6 +142,8 @@ def icc_is_native(bb, d):
         bb.data.inherits_class("cross", d) or \
         bb.data.inherits_class("native", d);
 
+# Don't pollute allarch signatures with TARGET_FPU
+icc_version[vardepsexclude] += "TARGET_FPU"
 def icc_version(bb, d):
     if use_icc(bb, d) == "no":
         return ""
@@ -182,6 +184,8 @@ def icc_get_external_tool(bb, d, tool):
     target_prefix = d.expand('${TARGET_PREFIX}')
     return os.path.join(external_toolchain_bindir, '%s%s' % (target_prefix, tool))
 
+# Don't pollute native signatures with target TUNE_PKGARCH through STAGING_BINDIR_TOOLCHAIN
+icc_get_tool[vardepsexclude] += "STAGING_BINDIR_TOOLCHAIN"
 def icc_get_tool(bb, d, tool):
     if icc_is_native(bb, d):
         return bb.utils.which(os.getenv("PATH"), tool)
-- 
1.8.5.2

[PATCH 4/4] icecc: use exact match in blacklists, re-start with empty system_package_blacklist

[Martin Jansa]

* unify debug messages a bit
* old implementation allowed partial match in blacklist, it's safer
  to explicitly list exact matches
* I was able to build all entries from system_package_blacklist with
  icecc enabled, lets assume that they were already resolved by newer
  versions (we've fixed a lot of parallel issues in recipes which were
  detected even without icecc and this list is very old).

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 meta/classes/icecc.bbclass | 33 ++++++++++++++++++---------------
 1 file changed, 18 insertions(+), 15 deletions(-)

diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index 549bd69..5c9e66c 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -95,36 +95,39 @@ def use_icc(bb,d):
     if icc_is_allarch(bb, d):
         return "no"
 
-    package_tmp = d.expand('${PN}')
+    pn = d.getVar('PN', True)
 
-    system_class_blacklist = [ "none" ]
+    system_class_blacklist = []
     user_class_blacklist = (d.getVar('ICECC_USER_CLASS_BL') or "none").split()
     package_class_blacklist = system_class_blacklist + user_class_blacklist
 
     for black in package_class_blacklist:
         if bb.data.inherits_class(black, d):
-            #bb.note(package_tmp, ' class ', black, ' found in blacklist, disable icecc')
+            bb.debug(1, "%s: class %s found in blacklist, disable icecc" % (pn, black))
             return "no"
 
-    #"system" package blacklist contains a list of packages that can not distribute compile tasks
-    #for one reason or the other
-    system_package_blacklist = [ "uclibc", "glibc", "gcc", "bind", "u-boot", "dhcp-forwarder", "enchant", "connman", "orbit2" ]
+    # "system" recipe blacklist contains a list of packages that can not distribute compile tasks
+    # for one reason or the other
+    # this is the old list (which doesn't seem to be valid anymore, because I was able to build
+    # all these with icecc enabled)
+    # system_package_blacklist = [ "uclibc", "glibc", "gcc", "bind", "u-boot", "dhcp-forwarder", "enchant", "connman", "orbit2" ]
+    # when adding new entry, please document why (how it failed) so that we can re-evaluate it later
+    # e.g. when there is new version
+    system_package_blacklist = []
     user_package_blacklist = (d.getVar('ICECC_USER_PACKAGE_BL') or "").split()
     user_package_whitelist = (d.getVar('ICECC_USER_PACKAGE_WL') or "").split()
     package_blacklist = system_package_blacklist + user_package_blacklist
 
-    for black in package_blacklist:
-        if black in package_tmp:
-            #bb.note(package_tmp, ' found in blacklist, disable icecc')
-            return "no"
+    if pn in package_blacklist:
+        bb.debug(1, "%s: found in blacklist, disable icecc" % pn)
+        return "no"
 
-    for white in user_package_whitelist:
-        if white in package_tmp:
-            bb.debug(1, package_tmp, " ", d.expand('${PV})'), " found in whitelist, enable icecc")
-            return "yes"
+    if pn in user_package_whitelist:
+        bb.debug(1, "%s: found in whitelist, enable icecc" % pn)
+        return "yes"
 
     if d.getVar('PARALLEL_MAKE') == "":
-        bb.debug(1, package_tmp, " ", d.expand('${PV}'), " has empty PARALLEL_MAKE, disable icecc")
+        bb.debug(1, "%s: has empty PARALLEL_MAKE, disable icecc" % pn)
         return "no"
 
     return "yes"
-- 
1.8.5.2