[PATCH 0/6 V2] ccache.bbclass: Refactor it to make it more reliable

[PATCH 0/6 V2] ccache.bbclass: Refactor it to make it more reliable

[Robert Yang]

* V2
  - Fix comments from Martin Jansa and Martin Hundebøll.
  - Fix RP's comments:
    1) Add comments for ccache related code in base.bbclass
    2) Remove all ccache-native's dependencies so that all recipes can depend on
       and use it.
  - Update test_ccache_tool in oeqa

* V1
  - Initial version

// Robert

The following changes since commit 8bc0d2f2197430723f8b2d0785169e48c883eedb:

  mdadm: Drop redundant patches and fix build with clang (2018-12-09 11:08:07 +0000)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib rbt/ccache
  http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/ccache

Robert Yang (6):
  ccache-native: Remove dependencies
  ccache.bbclass: Refactor it to make it more reliable
  apr/openssl10: Enable ccache for them
  cmake.bbclass/goarch.bbclass: Set CCACHE_DISABLE
  base.bbclass: Add comments for gcc links to ccache
  oeqa/selftest/cases: Update test_ccache_tool

 meta/classes/base.bbclass                          |  4 ++++
 meta/classes/ccache.bbclass                        | 25 +++++++++++++++++++---
 meta/classes/cmake.bbclass                         |  4 ++--
 meta/classes/goarch.bbclass                        |  4 ++++
 meta/conf/bitbake.conf                             |  6 +-----
 meta/conf/layer.conf                               |  1 -
 meta/lib/oe/utils.py                               |  3 ++-
 meta/lib/oeqa/selftest/cases/buildoptions.py       | 10 ++++-----
 .../openssl/openssl10_1.0.2q.bb                    |  3 ---
 meta/recipes-devtools/ccache/ccache.inc            | 13 ++++++++++-
 meta/recipes-devtools/ccache/ccache_3.4.2.bb       |  4 ++--
 meta/recipes-support/apr/apr_1.6.5.bb              |  5 -----
 12 files changed, 54 insertions(+), 28 deletions(-)

-- 
2.7.4

[PATCH 1/6] ccache-native: Remove dependencies

[Robert Yang]

Remove ccache-native's dependencies so that all native recipes can depend on
ccache-native and use it except ccache-native itself.

It has the following 4 dependencies:
- xz-native: It is intruduced by ccache-3.4.2.tar.xz, use ccache-3.4.2.tar.gz
             to replace of it can fix the problem.

- zlib-native: Use --with-bundled-zlib to fix it.

- autotools-native: Set INHIBIT_AUTOTOOLS_DEPS and add a do_configure() to fix
                    the problem.

- quilt-native: Set PATCHTOOL = "patch" to fix it.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-devtools/ccache/ccache.inc      | 13 ++++++++++++-
 meta/recipes-devtools/ccache/ccache_3.4.2.bb |  4 ++--
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-devtools/ccache/ccache.inc b/meta/recipes-devtools/ccache/ccache.inc
index 6566328..7f80065 100644
--- a/meta/recipes-devtools/ccache/ccache.inc
+++ b/meta/recipes-devtools/ccache/ccache.inc
@@ -9,8 +9,19 @@ LICENSE = "GPLv3+"
 
 DEPENDS = "zlib"
 
-SRC_URI = "https://download.samba.org/pub/${BPN}/${BP}.tar.xz"
+SRC_URI = "https://download.samba.org/pub/${BPN}/${BP}.tar.gz"
 
 inherit autotools
 
+# Remove ccache-native's dependencies, so that it can be used widely by
+# other native recipes.
+DEPENDS_class-native = ""
+EXTRA_OECONF_class-native = "--with-bundled-zlib"
+INHIBIT_AUTOTOOLS_DEPS_class-native = "1"
+PATCHTOOL = "patch"
+
 BBCLASSEXTEND = "native"
+
+do_configure_class-native() {
+    oe_runconf
+}
diff --git a/meta/recipes-devtools/ccache/ccache_3.4.2.bb b/meta/recipes-devtools/ccache/ccache_3.4.2.bb
index 7230934..921dc3b 100644
--- a/meta/recipes-devtools/ccache/ccache_3.4.2.bb
+++ b/meta/recipes-devtools/ccache/ccache_3.4.2.bb
@@ -3,8 +3,8 @@ require ccache.inc
 LICENSE = "GPLv3+"
 LIC_FILES_CHKSUM = "file://LICENSE.adoc;md5=2722abeaf1750dbf175b9491112374e1"
 
-SRC_URI[md5sum] = "9e048f88f3897125864f9a5e1abfb72d"
-SRC_URI[sha256sum] = "18a8b14367d63d3d37fb6c33cba60e1b7fcd7a63d608df97c9771ae0d234fee2"
+SRC_URI[md5sum] = "07d4324efbb3c216bbd1c512f4553480"
+SRC_URI[sha256sum] = "b2264923c63e2b90a17cf56acb1df3f4229c416fb88e476e5ec7e02919d319c3"
 
 SRC_URI += " \
             file://0002-dev.mk.in-fix-file-name-too-long.patch \
-- 
2.7.4

[PATCH 2/6] ccache.bbclass: Refactor it to make it more reliable

[Robert Yang]

The previous ccache.bbclass has the following problems:
- It uses host's ccache for native recipes, but this may not work on some
  hosts, for example, it nerver works on my Ubuntu 14.04.4, there are always
  build failures (m4-native failed at do_configure, and others will also be
  failed if I disable CCACHE for m4-native)

- native/nativesdk/cross/crosssdk recipes use host's ccahe, but target uses
  ccache-native, this may confuse user.

- The target recipes may use both host's ccache and ccache-native, this may
  cause unexpected problems and hard to debug. This is because ccache-native is
  in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS, so ccache-native may not be present when
  rebuild target recipes, and then it would use hosttools/ccache, but the
  previous ccache files were generated by ccache-native.

- Target recipes can't use ccache when no ccahe is installed on the host:
  CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"

After refactored:
All types recipes (native, target and others) will use ccache-native except
ccache-native itself, host's cache won't be used any more. It is more
reliable now, which will work everywhere when ccache-native can be built.

And now we need use "CCACHE_DISABLE = '1'" to disable ccahe for the recipe
rather than "CCACHE = ''" since we set CCACHE in anonymous function, and
d.getVar('CCACHE') works after "CCACHE ??=" which is set in bitbake.conf, so we
can't check whether CCACHE is set or not in anonymous function since it is
always set. Use CCACHE_DISABLE to disable it would be more clear.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/ccache.bbclass | 25 ++++++++++++++++++++++---
 meta/conf/bitbake.conf      |  6 +-----
 meta/conf/layer.conf        |  1 -
 meta/lib/oe/utils.py        |  3 ++-
 4 files changed, 25 insertions(+), 10 deletions(-)

diff --git a/meta/classes/ccache.bbclass b/meta/classes/ccache.bbclass
index 9609020..59e1022 100644
--- a/meta/classes/ccache.bbclass
+++ b/meta/classes/ccache.bbclass
@@ -1,4 +1,14 @@
-CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"
+#
+# Usage:
+# - Enable ccache
+#   Add the following line to a conffile such as conf/local.conf:
+#   INHERIT += "ccache"
+#
+# - Disable ccache for a recipe
+#   Add the following line to the recipe if it can't be built with ccache:
+#   CCACHE_DISABLE = '1'
+#
+
 export CCACHE_DIR ?= "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}"
 
 # We need to stop ccache considering the current directory or the
@@ -7,5 +17,14 @@ export CCACHE_DIR ?= "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}"
 # ${PV} or ${PR} change.
 export CCACHE_NOHASHDIR ?= "1"
 
-DEPENDS_append_class-target = " ccache-native"
-DEPENDS[vardepvalueexclude] = " ccache-native"
+python() {
+    """
+    Enable ccache for the recipe
+    """
+    pn = d.getVar('PN')
+    # quilt-native doesn't need ccache since no c files
+    if not (pn in ('ccache-native', 'quilt-native') or
+            bb.utils.to_boolean(d.getVar('CCACHE_DISABLE'))):
+        d.appendVar('DEPENDS', ' ccache-native')
+        d.setVar('CCACHE', 'ccache ')
+}
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 6480062..c88d66c 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -492,7 +492,7 @@ HOSTTOOLS += " \
 HOSTTOOLS += "${@'ip ping ps scp ssh stty' if (bb.utils.contains_any('IMAGE_CLASSES', 'testimage testsdk', True, False, d) or any(x in (d.getVar("BBINCLUDED") or "") for x in ["testimage.bbclass", "testsdk.bbclass"])) else ''}"
 
 # Link to these if present
-HOSTTOOLS_NONFATAL += "aws ccache gcc-ar gpg ld.bfd ld.gold nc pigz sftp socat ssh sudo"
+HOSTTOOLS_NONFATAL += "aws gcc-ar gpg ld.bfd ld.gold nc pigz sftp socat ssh sudo"
 
 # Temporary add few more detected in bitbake world
 HOSTTOOLS_NONFATAL += "join nl size yes zcat"
@@ -504,10 +504,6 @@ HOSTTOOLS_NONFATAL += "bzr"
 HOSTTOOLS_NONFATAL += "scp"
 
 CCACHE ??= ""
-# ccache < 3.1.10 will create CCACHE_DIR on startup even if disabled, and
-# autogen sets HOME=/dev/null so in certain situations builds can fail.
-# Explicitly export CCACHE_DIR until we can assume ccache >3.1.10 on the host.
-export CCACHE_DIR ??= "${@os.getenv('HOME')}/.ccache"
 
 TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
 
diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
index c67dec4..4141d6c 100644
--- a/meta/conf/layer.conf
+++ b/meta/conf/layer.conf
@@ -49,7 +49,6 @@ SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS += " \
   *->quilt-native \
   *->subversion-native \
   *->git-native \
-  *->ccache-native \
   *->icecc-create-env-native \
   gcc-cross-${TARGET_ARCH}->virtual/${TARGET_PREFIX}libc-for-gcc \
   gcc-cross-${TARGET_ARCH}->linux-libc-headers \
diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
index 8a584d6..a64f16c 100644
--- a/meta/lib/oe/utils.py
+++ b/meta/lib/oe/utils.py
@@ -362,7 +362,8 @@ def host_gcc_version(d, taskcontextonly=False):
     if taskcontextonly and d.getVar('BB_WORKERCONTEXT') != '1':
         return
 
-    compiler = d.getVar("BUILD_CC")
+    # Get rid of ccache since it is not present when parsing.
+    compiler = d.getVar("BUILD_CC").lstrip("ccache-")
     try:
         env = os.environ.copy()
         env["PATH"] = d.getVar("PATH")
-- 
2.7.4

[PATCH 3/6] apr/openssl10: Enable ccache for them

[Robert Yang]

They work well now.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-connectivity/openssl/openssl10_1.0.2q.bb | 3 ---
 meta/recipes-support/apr/apr_1.6.5.bb                 | 5 -----
 2 files changed, 8 deletions(-)

diff --git a/meta/recipes-connectivity/openssl/openssl10_1.0.2q.bb b/meta/recipes-connectivity/openssl/openssl10_1.0.2q.bb
index 6518dac..1986cf2 100644
--- a/meta/recipes-connectivity/openssl/openssl10_1.0.2q.bb
+++ b/meta/recipes-connectivity/openssl/openssl10_1.0.2q.bb
@@ -76,9 +76,6 @@ EXTRA_OEMAKE = "${@bb.utils.contains('PACKAGECONFIG', 'manpages', '', 'OE_DISABL
 
 export OE_LDFLAGS = "${LDFLAGS}"
 
-# openssl fails with ccache: https://bugzilla.yoctoproject.org/show_bug.cgi?id=12810
-CCACHE = ""
-
 TERMIO ?= "-DTERMIO"
 TERMIO_libc-musl = "-DTERMIOS"
 
diff --git a/meta/recipes-support/apr/apr_1.6.5.bb b/meta/recipes-support/apr/apr_1.6.5.bb
index c2f5900..432e4ed 100644
--- a/meta/recipes-support/apr/apr_1.6.5.bb
+++ b/meta/recipes-support/apr/apr_1.6.5.bb
@@ -37,11 +37,6 @@ CACHED_CONFIGUREVARS += "ac_cv_header_netinet_sctp_h=no ac_cv_header_netinet_sct
 CACHED_CONFIGUREVARS += "ac_cv_sizeof_struct_iovec=yes"
 CACHED_CONFIGUREVARS += "ac_cv_file__dev_zero=yes"
 
-# Otherwise libtool fails to compile apr-utils
-# x86_64-linux-libtool: compile: unable to infer tagged configuration
-# x86_64-linux-libtool:   error: specify a tag with '--tag'
-CCACHE = ""
-
 PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)}"
 PACKAGECONFIG[ipv6] = "--enable-ipv6,--disable-ipv6,"
 
-- 
2.7.4

[PATCH 4/6] cmake.bbclass/goarch.bbclass: Set CCACHE_DISABLE

[Robert Yang]

They can't be built with ccache.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/cmake.bbclass  | 4 ++--
 meta/classes/goarch.bbclass | 4 ++++
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index b364d2b..12fc452 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -4,8 +4,8 @@ OECMAKE_SOURCEPATH ??= "${S}"
 DEPENDS_prepend = "cmake-native "
 B = "${WORKDIR}/build"
 
-# We need to unset CCACHE otherwise cmake gets too confused
-CCACHE = ""
+# We need to disable CCACHE otherwise cmake gets too confused
+CCACHE_DISABLE = "1"
 
 # What CMake generator to use.
 # The supported options are "Unix Makefiles" or "Ninja".
diff --git a/meta/classes/goarch.bbclass b/meta/classes/goarch.bbclass
index b2c94fa..1390f31 100644
--- a/meta/classes/goarch.bbclass
+++ b/meta/classes/goarch.bbclass
@@ -42,6 +42,10 @@ TUNE_CCARGS_remove = "-march=mips32r2"
 SECURITY_CFLAGS_mipsarch = "${SECURITY_NOPIE_CFLAGS}"
 SECURITY_NOPIE_CFLAGS ??= ""
 
+# go can't be built with ccache:
+# gcc: fatal error: no input files
+CCACHE_DISABLE = "1"
+
 def go_map_arch(a, d):
     import re
     if re.match('i.86', a):
-- 
2.7.4

[PATCH 5/6] base.bbclass: Add comments for gcc links to ccache

[Robert Yang]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/base.bbclass | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index e715ffa..e6af673 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -122,6 +122,10 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
         desttool = os.path.join(dest, tool)
         if not os.path.exists(desttool):
             srctool = bb.utils.which(path, tool, executable=True)
+            # gcc/g++ may link to ccache on some hosts, e.g.,
+            # /usr/local/bin/ccache/gcc -> /usr/bin/ccache, then which(gcc)
+            # would return /usr/local/bin/ccache/gcc, but what we need is
+            # /usr/bin/gcc, this code can check and fix that.
             if "ccache" in srctool:
                 srctool = bb.utils.which(path, tool, executable=True, direction=1)
             if srctool:
-- 
2.7.4

[PATCH 6/6] oeqa/selftest/cases: Update test_ccache_tool

[Robert Yang]

Now we can compile m4-native rather than target m4, this can save a lot of
build time.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/lib/oeqa/selftest/cases/buildoptions.py | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/buildoptions.py b/meta/lib/oeqa/selftest/cases/buildoptions.py
index f234bac..181e9a9 100644
--- a/meta/lib/oeqa/selftest/cases/buildoptions.py
+++ b/meta/lib/oeqa/selftest/cases/buildoptions.py
@@ -37,13 +37,13 @@ class ImageOptionsTests(OESelftestTestCase):
         p = bb_vars['SYSROOT_DESTDIR'] + bb_vars['bindir'] + "/" + "ccache"
         self.assertTrue(os.path.isfile(p), msg = "No ccache found (%s)" % p)
         self.write_config('INHERIT += "ccache"')
-        self.add_command_to_tearDown('bitbake -c clean m4')
-        bitbake("m4 -c clean")
-        bitbake("m4 -f -c compile")
-        log_compile = os.path.join(get_bb_var("WORKDIR","m4"), "temp/log.do_compile")
+        self.add_command_to_tearDown('bitbake -c clean m4-native')
+        bitbake("m4-native -c clean")
+        bitbake("m4-native -f -c compile")
+        log_compile = os.path.join(get_bb_var("WORKDIR","m4-native"), "temp/log.do_compile")
         with open(log_compile, "r") as f:
             loglines = "".join(f.readlines())
-        self.assertIn("ccache", loglines, msg="No match for ccache in m4 log.do_compile. For further details: %s" % log_compile)
+        self.assertIn("ccache", loglines, msg="No match for ccache in m4-native log.do_compile. For further details: %s" % log_compile)
 
     @OETestID(1435)
     def test_read_only_image(self):
-- 
2.7.4

Re: [PATCH 5/6] base.bbclass: Add comments for gcc links to ccache

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 1829 bytes --]

This should probably remove the dirname srctool from path and check again
with the same direction, rather than reversing it, in case the toolchain is
in multiple locations in PATH, as  you’re flipping the PATH priority order
doing it this way. I realize you’re just updating the description, not the
code, so just a random th ought for the future here..

On Tue, Dec 11, 2018 at 11:44 PM Robert Yang <liezhi.yang@windriver.com>
wrote:

> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>  meta/classes/base.bbclass | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index e715ffa..e6af673 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -122,6 +122,10 @@ def setup_hosttools_dir(dest, toolsvar, d,
> fatal=True):
>          desttool = os.path.join(dest, tool)
>          if not os.path.exists(desttool):
>              srctool = bb.utils.which(path, tool, executable=True)
> +            # gcc/g++ may link to ccache on some hosts, e.g.,
> +            # /usr/local/bin/ccache/gcc -> /usr/bin/ccache, then
> which(gcc)
> +            # would return /usr/local/bin/ccache/gcc, but what we need is
> +            # /usr/bin/gcc, this code can check and fix that.
>              if "ccache" in srctool:
>                  srctool = bb.utils.which(path, tool, executable=True,
> direction=1)
>              if srctool:
> --
> 2.7.4
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>


-- 
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 2646 bytes --]

Re: [PATCH 5/6] base.bbclass: Add comments for gcc links to ccache

[Robert Yang]

Hi Christopher,

On 12/12/18 11:50 PM, Christopher Larson wrote:
> This should probably remove the dirname srctool from path and check again with 
> the same direction, rather than reversing it, in case the toolchain is in 
> multiple locations in PATH, as  you’re flipping the PATH priority order doing it 
> this way. I realize you’re just updating the description, not the code, so just 
> a random th ought for the future here..

The problem is I don't know whether the code is also used for fixing other
cases. So I'm leaning to keep it as the current status since it works.

// Robert

> 
> On Tue, Dec 11, 2018 at 11:44 PM Robert Yang <liezhi.yang@windriver.com 
> <mailto:liezhi.yang@windriver.com>> wrote:
> 
>     Signed-off-by: Robert Yang <liezhi.yang@windriver.com
>     <mailto:liezhi.yang@windriver.com>>
>     ---
>       meta/classes/base.bbclass | 4 ++++
>       1 file changed, 4 insertions(+)
> 
>     diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
>     index e715ffa..e6af673 100644
>     --- a/meta/classes/base.bbclass
>     +++ b/meta/classes/base.bbclass
>     @@ -122,6 +122,10 @@ def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
>               desttool = os.path.join(dest, tool)
>               if not os.path.exists(desttool):
>                   srctool = bb.utils.which(path, tool, executable=True)
>     +            # gcc/g++ may link to ccache on some hosts, e.g.,
>     +            # /usr/local/bin/ccache/gcc -> /usr/bin/ccache, then which(gcc)
>     +            # would return /usr/local/bin/ccache/gcc, but what we need is
>     +            # /usr/bin/gcc, this code can check and fix that.
>                   if "ccache" in srctool:
>                       srctool = bb.utils.which(path, tool, executable=True,
>     direction=1)
>                   if srctool:
>     -- 
>     2.7.4
> 
>     -- 
>     _______________________________________________
>     Openembedded-core mailing list
>     Openembedded-core@lists.openembedded.org
>     <mailto:Openembedded-core@lists.openembedded.org>
>     http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 
> 
> 
> -- 
> Christopher Larson
> kergoth at gmail dot com
> Founder - BitBake, OpenEmbedded, OpenZaurus
> Senior Software Engineer, Mentor Graphics

Re: [PATCH 2/6] ccache.bbclass: Refactor it to make it more reliable

[Robert Yang]

On 12/12/18 2:59 PM, Robert Yang wrote:
> The previous ccache.bbclass has the following problems:
> - It uses host's ccache for native recipes, but this may not work on some
>    hosts, for example, it nerver works on my Ubuntu 14.04.4, there are always
>    build failures (m4-native failed at do_configure, and others will also be
>    failed if I disable CCACHE for m4-native)
> 
> - native/nativesdk/cross/crosssdk recipes use host's ccahe, but target uses
>    ccache-native, this may confuse user.
> 
> - The target recipes may use both host's ccache and ccache-native, this may
>    cause unexpected problems and hard to debug. This is because ccache-native is
>    in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS, so ccache-native may not be present when
>    rebuild target recipes, and then it would use hosttools/ccache, but the
>    previous ccache files were generated by ccache-native.
> 
> - Target recipes can't use ccache when no ccahe is installed on the host:
>    CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"
> 
> After refactored:
> All types recipes (native, target and others) will use ccache-native except
> ccache-native itself, host's cache won't be used any more. It is more
> reliable now, which will work everywhere when ccache-native can be built.
> 
> And now we need use "CCACHE_DISABLE = '1'" to disable ccahe for the recipe
> rather than "CCACHE = ''" since we set CCACHE in anonymous function, and
> d.getVar('CCACHE') works after "CCACHE ??=" which is set in bitbake.conf, so we
> can't check whether CCACHE is set or not in anonymous function since it is
> always set. Use CCACHE_DISABLE to disable it would be more clear.
> 
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>   meta/classes/ccache.bbclass | 25 ++++++++++++++++++++++---
>   meta/conf/bitbake.conf      |  6 +-----
>   meta/conf/layer.conf        |  1 -
>   meta/lib/oe/utils.py        |  3 ++-
>   4 files changed, 25 insertions(+), 10 deletions(-)
> 
> diff --git a/meta/classes/ccache.bbclass b/meta/classes/ccache.bbclass
> index 9609020..59e1022 100644
> --- a/meta/classes/ccache.bbclass
> +++ b/meta/classes/ccache.bbclass
> @@ -1,4 +1,14 @@
> -CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"
> +#
> +# Usage:
> +# - Enable ccache
> +#   Add the following line to a conffile such as conf/local.conf:
> +#   INHERIT += "ccache"
> +#
> +# - Disable ccache for a recipe
> +#   Add the following line to the recipe if it can't be built with ccache:
> +#   CCACHE_DISABLE = '1'
> +#
> +
>   export CCACHE_DIR ?= "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}"
>   
>   # We need to stop ccache considering the current directory or the
> @@ -7,5 +17,14 @@ export CCACHE_DIR ?= "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}"
>   # ${PV} or ${PR} change.
>   export CCACHE_NOHASHDIR ?= "1"
>   
> -DEPENDS_append_class-target = " ccache-native"
> -DEPENDS[vardepvalueexclude] = " ccache-native"
> +python() {
> +    """
> +    Enable ccache for the recipe
> +    """
> +    pn = d.getVar('PN')
> +    # quilt-native doesn't need ccache since no c files
> +    if not (pn in ('ccache-native', 'quilt-native') or
> +            bb.utils.to_boolean(d.getVar('CCACHE_DISABLE'))):
> +        d.appendVar('DEPENDS', ' ccache-native')
> +        d.setVar('CCACHE', 'ccache ')
> +}
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 6480062..c88d66c 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -492,7 +492,7 @@ HOSTTOOLS += " \
>   HOSTTOOLS += "${@'ip ping ps scp ssh stty' if (bb.utils.contains_any('IMAGE_CLASSES', 'testimage testsdk', True, False, d) or any(x in (d.getVar("BBINCLUDED") or "") for x in ["testimage.bbclass", "testsdk.bbclass"])) else ''}"
>   
>   # Link to these if present
> -HOSTTOOLS_NONFATAL += "aws ccache gcc-ar gpg ld.bfd ld.gold nc pigz sftp socat ssh sudo"
> +HOSTTOOLS_NONFATAL += "aws gcc-ar gpg ld.bfd ld.gold nc pigz sftp socat ssh sudo"
>   
>   # Temporary add few more detected in bitbake world
>   HOSTTOOLS_NONFATAL += "join nl size yes zcat"
> @@ -504,10 +504,6 @@ HOSTTOOLS_NONFATAL += "bzr"
>   HOSTTOOLS_NONFATAL += "scp"
>   
>   CCACHE ??= ""
> -# ccache < 3.1.10 will create CCACHE_DIR on startup even if disabled, and
> -# autogen sets HOME=/dev/null so in certain situations builds can fail.
> -# Explicitly export CCACHE_DIR until we can assume ccache >3.1.10 on the host.
> -export CCACHE_DIR ??= "${@os.getenv('HOME')}/.ccache"
>   
>   TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
>   
> diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
> index c67dec4..4141d6c 100644
> --- a/meta/conf/layer.conf
> +++ b/meta/conf/layer.conf
> @@ -49,7 +49,6 @@ SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS += " \
>     *->quilt-native \
>     *->subversion-native \
>     *->git-native \
> -  *->ccache-native \
>     *->icecc-create-env-native \
>     gcc-cross-${TARGET_ARCH}->virtual/${TARGET_PREFIX}libc-for-gcc \
>     gcc-cross-${TARGET_ARCH}->linux-libc-headers \
> diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
> index 8a584d6..a64f16c 100644
> --- a/meta/lib/oe/utils.py
> +++ b/meta/lib/oe/utils.py
> @@ -362,7 +362,8 @@ def host_gcc_version(d, taskcontextonly=False):
>       if taskcontextonly and d.getVar('BB_WORKERCONTEXT') != '1':
>           return
>   
> -    compiler = d.getVar("BUILD_CC")
> +    # Get rid of ccache since it is not present when parsing.
> +    compiler = d.getVar("BUILD_CC").lstrip("ccache-")

Sorry, here should be:

     compiler = d.getVar("BUILD_CC")
     # Get rid of ccache since it is not present when parsing.
     if compiler.startswith('ccache '):
         compiler = compiler[7:]


I updated it in the repo.

// Robert

>       try:
>           env = os.environ.copy()
>           env["PATH"] = d.getVar("PATH")
>