[PATCH V3 00/11] ccache.bbclass: Refactor it to make it more reliable

[PATCH V3 00/11] ccache.bbclass: Refactor it to make it more reliable

[Robert Yang]

* V3
  - Make cache files can be shared between different build dirs
  - Add task do_cleanccache
  - Make cmake work with ccache
  - Upgrade ccache to 3.5, this version has a debug mode, which is useful for
    figuring out why a cache file is not used.

* V2
  - Fix comments from Martin Jansa and Martin Hundebøll.
  - Fix RP's comments:
    1) Add comments for ccache related code in base.bbclass
    2) Remove all ccache-native's dependencies so that all recipes can depend on
       and use it.
  - Update test_ccache_tool in oeqa

* V1
  Initial version

The following changes since commit 20aea61385e1a53ac245353899277ba20104ed2f:

  gcc: Drop 7.3 since 8.2 is working fine for us (2018-12-15 17:10:44 +0000)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib rbt/ccache
  http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/ccache

Robert Yang (11):
  ccache-native: Remove dependencies
  ccache: 3.4.2 -> 3.5
  ccache.bbclass: Refactor it to make it more reliable
  ccache.bbclass: Make it can be shared between different builds
  ccache.bbclass: Add task do_cleanccache
  ccache.bbclass: Set CCACHE_CONFIGPATH to fix race issues
  bitbake.conf: Enable -fdebug-prefix-map for nativesdk
  goarch.bbclass: Set CCACHE_DISABLE
  cmake.bbclass: Make it work with ccache
  apr/openssl10: Enable ccache for them
  oeqa/selftest/cases: Update test_ccache_tool

 meta/classes/ccache.bbclass                        | 63 ++++++++++++++++++++--
 meta/classes/cmake.bbclass                         | 21 +++++---
 meta/classes/goarch.bbclass                        |  4 ++
 meta/conf/bitbake.conf                             |  8 +--
 meta/conf/ccache.conf                              |  2 +
 meta/conf/layer.conf                               |  1 -
 meta/lib/oe/utils.py                               |  3 ++
 meta/lib/oeqa/selftest/cases/buildoptions.py       | 10 ++--
 .../openssl/openssl10_1.0.2q.bb                    |  3 --
 meta/recipes-devtools/ccache/ccache.inc            | 13 ++++-
 meta/recipes-devtools/ccache/ccache_3.4.2.bb       | 11 ----
 meta/recipes-devtools/ccache/ccache_3.5.bb         | 11 ++++
 meta/recipes-graphics/piglit/piglit_git.bb         |  3 ++
 meta/recipes-graphics/vulkan/vulkan_1.1.73.0.bb    |  3 ++
 meta/recipes-sato/webkit/webkitgtk_2.22.4.bb       |  3 ++
 meta/recipes-support/apr/apr_1.6.5.bb              |  5 --
 16 files changed, 122 insertions(+), 42 deletions(-)
 create mode 100644 meta/conf/ccache.conf
 delete mode 100644 meta/recipes-devtools/ccache/ccache_3.4.2.bb
 create mode 100644 meta/recipes-devtools/ccache/ccache_3.5.bb

-- 
2.7.4

[PATCH V3 01/11] ccache-native: Remove dependencies

[Robert Yang]

Remove ccache-native's dependencies so that all native recipes can depend on
ccache-native and use it except ccache-native itself.

It has the following 4 dependencies:
- xz-native: It is intruduced by ccache-3.4.2.tar.xz, use ccache-3.4.2.tar.gz
             to replace of it can fix the problem.

- zlib-native: Use --with-bundled-zlib to fix it.

- autotools-native: Set INHIBIT_AUTOTOOLS_DEPS and add a do_configure() to fix
                    the problem.

- quilt-native: Set PATCHTOOL = "patch" to fix it.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-devtools/ccache/ccache.inc      | 13 ++++++++++++-
 meta/recipes-devtools/ccache/ccache_3.4.2.bb |  4 ++--
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-devtools/ccache/ccache.inc b/meta/recipes-devtools/ccache/ccache.inc
index 6566328..7f80065 100644
--- a/meta/recipes-devtools/ccache/ccache.inc
+++ b/meta/recipes-devtools/ccache/ccache.inc
@@ -9,8 +9,19 @@ LICENSE = "GPLv3+"
 
 DEPENDS = "zlib"
 
-SRC_URI = "https://download.samba.org/pub/${BPN}/${BP}.tar.xz"
+SRC_URI = "https://download.samba.org/pub/${BPN}/${BP}.tar.gz"
 
 inherit autotools
 
+# Remove ccache-native's dependencies, so that it can be used widely by
+# other native recipes.
+DEPENDS_class-native = ""
+EXTRA_OECONF_class-native = "--with-bundled-zlib"
+INHIBIT_AUTOTOOLS_DEPS_class-native = "1"
+PATCHTOOL = "patch"
+
 BBCLASSEXTEND = "native"
+
+do_configure_class-native() {
+    oe_runconf
+}
diff --git a/meta/recipes-devtools/ccache/ccache_3.4.2.bb b/meta/recipes-devtools/ccache/ccache_3.4.2.bb
index 7230934..921dc3b 100644
--- a/meta/recipes-devtools/ccache/ccache_3.4.2.bb
+++ b/meta/recipes-devtools/ccache/ccache_3.4.2.bb
@@ -3,8 +3,8 @@ require ccache.inc
 LICENSE = "GPLv3+"
 LIC_FILES_CHKSUM = "file://LICENSE.adoc;md5=2722abeaf1750dbf175b9491112374e1"
 
-SRC_URI[md5sum] = "9e048f88f3897125864f9a5e1abfb72d"
-SRC_URI[sha256sum] = "18a8b14367d63d3d37fb6c33cba60e1b7fcd7a63d608df97c9771ae0d234fee2"
+SRC_URI[md5sum] = "07d4324efbb3c216bbd1c512f4553480"
+SRC_URI[sha256sum] = "b2264923c63e2b90a17cf56acb1df3f4229c416fb88e476e5ec7e02919d319c3"
 
 SRC_URI += " \
             file://0002-dev.mk.in-fix-file-name-too-long.patch \
-- 
2.7.4

[PATCH V3 02/11] ccache: 3.4.2 -> 3.5

[Robert Yang]

The LIC_FILES_CHKSUM is changed because copyright year and zlib version
are changed, others are still the same.

This version supports CCACHE_DEBUG which will make the debug easier.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-devtools/ccache/ccache_3.4.2.bb | 11 -----------
 meta/recipes-devtools/ccache/ccache_3.5.bb   | 11 +++++++++++
 2 files changed, 11 insertions(+), 11 deletions(-)
 delete mode 100644 meta/recipes-devtools/ccache/ccache_3.4.2.bb
 create mode 100644 meta/recipes-devtools/ccache/ccache_3.5.bb

diff --git a/meta/recipes-devtools/ccache/ccache_3.4.2.bb b/meta/recipes-devtools/ccache/ccache_3.4.2.bb
deleted file mode 100644
index 921dc3b..0000000
--- a/meta/recipes-devtools/ccache/ccache_3.4.2.bb
+++ /dev/null
@@ -1,11 +0,0 @@
-require ccache.inc
-
-LICENSE = "GPLv3+"
-LIC_FILES_CHKSUM = "file://LICENSE.adoc;md5=2722abeaf1750dbf175b9491112374e1"
-
-SRC_URI[md5sum] = "07d4324efbb3c216bbd1c512f4553480"
-SRC_URI[sha256sum] = "b2264923c63e2b90a17cf56acb1df3f4229c416fb88e476e5ec7e02919d319c3"
-
-SRC_URI += " \
-            file://0002-dev.mk.in-fix-file-name-too-long.patch \
-"
diff --git a/meta/recipes-devtools/ccache/ccache_3.5.bb b/meta/recipes-devtools/ccache/ccache_3.5.bb
new file mode 100644
index 0000000..89062c4
--- /dev/null
+++ b/meta/recipes-devtools/ccache/ccache_3.5.bb
@@ -0,0 +1,11 @@
+require ccache.inc
+
+LICENSE = "GPLv3+"
+LIC_FILES_CHKSUM = "file://LICENSE.adoc;md5=39f34d6ff4df51ed718fd7d243fc4e51"
+
+SRC_URI[md5sum] = "6e62bf2fd2c8fdc749efdca74868da41"
+SRC_URI[sha256sum] = "8094f9bc186be4c9db9b480eeac280926bd44039502d1e596f45371b05a85918"
+
+SRC_URI += " \
+            file://0002-dev.mk.in-fix-file-name-too-long.patch \
+"
-- 
2.7.4

[PATCH V3 03/11] ccache.bbclass: Refactor it to make it more reliable

[Robert Yang]

The previous ccache.bbclass has the following problems:
- It uses host's ccache for native recipes, but this may not work on some
  hosts, for example, it nerver works on my Ubuntu 14.04.4, there are always
  build failures (m4-native failed at do_configure, and others will also be
  failed if I disable CCACHE for m4-native)

- native/nativesdk/cross/crosssdk recipes use host's ccahe, but target uses
  ccache-native, this may confuse user.

- The target recipes may use both host's ccache and ccache-native, this may
  cause unexpected problems and hard to debug. This is because ccache-native is
  in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS, so ccache-native may not be present when
  rebuild target recipes, and then it would use hosttools/ccache, but the
  previous ccache files were generated by ccache-native.

- Target recipes can't use ccache when no ccahe is installed on the host:
  CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"

After refactored:
All types recipes (native, target and others) will use ccache-native except
ccache-native itself, host's cache won't be used any more. It is more
reliable now, which will work everywhere when ccache-native can be built.

And now we need use "CCACHE_DISABLE = '1'" to disable ccahe for the recipe
rather than "CCACHE = ''" since we set CCACHE in anonymous function, and
d.getVar('CCACHE') works after "CCACHE ??=" which is set in bitbake.conf, so we
can't check whether CCACHE is set or not in anonymous function since it is
always set. Use CCACHE_DISABLE to disable it would be more clear.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/ccache.bbclass | 25 ++++++++++++++++++++++---
 meta/conf/bitbake.conf      |  6 +-----
 meta/conf/layer.conf        |  1 -
 meta/lib/oe/utils.py        |  3 +++
 4 files changed, 26 insertions(+), 9 deletions(-)

diff --git a/meta/classes/ccache.bbclass b/meta/classes/ccache.bbclass
index 9609020..59e1022 100644
--- a/meta/classes/ccache.bbclass
+++ b/meta/classes/ccache.bbclass
@@ -1,4 +1,14 @@
-CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"
+#
+# Usage:
+# - Enable ccache
+#   Add the following line to a conffile such as conf/local.conf:
+#   INHERIT += "ccache"
+#
+# - Disable ccache for a recipe
+#   Add the following line to the recipe if it can't be built with ccache:
+#   CCACHE_DISABLE = '1'
+#
+
 export CCACHE_DIR ?= "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}"
 
 # We need to stop ccache considering the current directory or the
@@ -7,5 +17,14 @@ export CCACHE_DIR ?= "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}"
 # ${PV} or ${PR} change.
 export CCACHE_NOHASHDIR ?= "1"
 
-DEPENDS_append_class-target = " ccache-native"
-DEPENDS[vardepvalueexclude] = " ccache-native"
+python() {
+    """
+    Enable ccache for the recipe
+    """
+    pn = d.getVar('PN')
+    # quilt-native doesn't need ccache since no c files
+    if not (pn in ('ccache-native', 'quilt-native') or
+            bb.utils.to_boolean(d.getVar('CCACHE_DISABLE'))):
+        d.appendVar('DEPENDS', ' ccache-native')
+        d.setVar('CCACHE', 'ccache ')
+}
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 6480062..c88d66c 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -492,7 +492,7 @@ HOSTTOOLS += " \
 HOSTTOOLS += "${@'ip ping ps scp ssh stty' if (bb.utils.contains_any('IMAGE_CLASSES', 'testimage testsdk', True, False, d) or any(x in (d.getVar("BBINCLUDED") or "") for x in ["testimage.bbclass", "testsdk.bbclass"])) else ''}"
 
 # Link to these if present
-HOSTTOOLS_NONFATAL += "aws ccache gcc-ar gpg ld.bfd ld.gold nc pigz sftp socat ssh sudo"
+HOSTTOOLS_NONFATAL += "aws gcc-ar gpg ld.bfd ld.gold nc pigz sftp socat ssh sudo"
 
 # Temporary add few more detected in bitbake world
 HOSTTOOLS_NONFATAL += "join nl size yes zcat"
@@ -504,10 +504,6 @@ HOSTTOOLS_NONFATAL += "bzr"
 HOSTTOOLS_NONFATAL += "scp"
 
 CCACHE ??= ""
-# ccache < 3.1.10 will create CCACHE_DIR on startup even if disabled, and
-# autogen sets HOME=/dev/null so in certain situations builds can fail.
-# Explicitly export CCACHE_DIR until we can assume ccache >3.1.10 on the host.
-export CCACHE_DIR ??= "${@os.getenv('HOME')}/.ccache"
 
 TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
 
diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
index c67dec4..4141d6c 100644
--- a/meta/conf/layer.conf
+++ b/meta/conf/layer.conf
@@ -49,7 +49,6 @@ SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS += " \
   *->quilt-native \
   *->subversion-native \
   *->git-native \
-  *->ccache-native \
   *->icecc-create-env-native \
   gcc-cross-${TARGET_ARCH}->virtual/${TARGET_PREFIX}libc-for-gcc \
   gcc-cross-${TARGET_ARCH}->linux-libc-headers \
diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
index 29b4115..ee6f0e6 100644
--- a/meta/lib/oe/utils.py
+++ b/meta/lib/oe/utils.py
@@ -363,6 +363,9 @@ def host_gcc_version(d, taskcontextonly=False):
         return
 
     compiler = d.getVar("BUILD_CC")
+    # Get rid of ccache since it is not present when parsing.
+    if compiler.startswith('ccache '):
+        compiler = compiler[7:]
     try:
         env = os.environ.copy()
         env["PATH"] = d.getVar("PATH")
-- 
2.7.4

[PATCH V3 04/11] ccache.bbclass: Make it can be shared between different builds

[Robert Yang]

CCACHE_BASEDIR: ccache removes this from file path, so that hashes will be
                the same in different build dirs.

CCACHE_TOP_DIR: Set it to a shared location for different builds.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/ccache.bbclass | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/meta/classes/ccache.bbclass b/meta/classes/ccache.bbclass
index 59e1022..f3612c5 100644
--- a/meta/classes/ccache.bbclass
+++ b/meta/classes/ccache.bbclass
@@ -8,8 +8,28 @@
 #   Add the following line to the recipe if it can't be built with ccache:
 #   CCACHE_DISABLE = '1'
 #
+# - Share ccache files between different builds
+#   Set CCACHE_TOP_DIR to a shared dir
+#   CCACHE_TOP_DIR = /path/to/shared_ccache/
+#
+# - TO debug ccahe
+#   export CCACHE_DEBUG = "1"
+#   export CCACHE_LOGFILE = "${CCACHE_DIR}/logfile.log"
+#   And also set PARALLEL_MAKE = "-j 1" to get make the log in order
+#
+
+# Set it to a shared location for different builds, so that cache files can
+# be shared between different builds.
+CCACHE_TOP_DIR ?= "${TMPDIR}/ccache"
+
+# ccahe removes CCACHE_BASEDIR from file path, so that hashes will be the same
+# in different builds.
+export CCACHE_BASEDIR ?= "${TMPDIR}"
+
+# Used for sharing cache files after compiler is rebuilt
+export CCACHE_COMPILERCHECK ?= "%compiler% -dumpspecs"
 
-export CCACHE_DIR ?= "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}"
+export CCACHE_DIR ?= "${CCACHE_TOP_DIR}/${MULTIMACH_TARGET_SYS}/${PN}"
 
 # We need to stop ccache considering the current directory or the
 # debug-prefix-map target directory to be significant when calculating
-- 
2.7.4

[PATCH V3 05/11] ccache.bbclass: Add task do_cleanccache

[Robert Yang]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/ccache.bbclass | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/meta/classes/ccache.bbclass b/meta/classes/ccache.bbclass
index f3612c5..fb3fbdf 100644
--- a/meta/classes/ccache.bbclass
+++ b/meta/classes/ccache.bbclass
@@ -48,3 +48,17 @@ python() {
         d.appendVar('DEPENDS', ' ccache-native')
         d.setVar('CCACHE', 'ccache ')
 }
+
+addtask cleanccache after do_clean
+python do_cleanccache() {
+    import shutil
+
+    ccache_dir = d.getVar('CCACHE_DIR')
+    if os.path.exists(ccache_dir):
+        bb.note("Removing %s" % ccache_dir)
+        shutil.rmtree(ccache_dir)
+    else:
+        bb.note("%s doesn't exist" % ccache_dir)
+}
+addtask cleanall after do_cleanccache
+do_cleanccache[nostamp] = "1"
-- 
2.7.4

[PATCH V3 06/11] ccache.bbclass: Set CCACHE_CONFIGPATH to fix race issues

[Robert Yang]

Fixed race issues when parallel build:
ccache: error: /path/to/ccache/i586-poky-linux/mmc-utils/ccache.conf: No such file or directory
ccache: error: /path/to/ccache/i586-poky-linux/mmc-utils/ccache.conf: No such file or directory

This is because we set CCACHE_DIR for earch recipe, and ccache will create a
ccache.conf for each CCACHE_DIR when CCACHE_CONFIGPATH is not set, but there
might be a race issue in parallel build:

ccache gcc file1.c
ccache gcc file2.c

If the two ccache processes use fopen(path, "w") to create ccache.conf at the
same time, the error would happen. Set CCACHE_CONFIGPATH to
meta/conf/ccache.conf can fix the problem, and we can add other configs to the
file when needed.

And also set cache_dir_levels to 1 (default is 2) since each recipe has a cache
dir, thus we don't have too many files in one dir.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/ccache.bbclass | 2 ++
 meta/conf/ccache.conf       | 2 ++
 2 files changed, 4 insertions(+)
 create mode 100644 meta/conf/ccache.conf

diff --git a/meta/classes/ccache.bbclass b/meta/classes/ccache.bbclass
index fb3fbdf..b545735 100644
--- a/meta/classes/ccache.bbclass
+++ b/meta/classes/ccache.bbclass
@@ -29,6 +29,8 @@ export CCACHE_BASEDIR ?= "${TMPDIR}"
 # Used for sharing cache files after compiler is rebuilt
 export CCACHE_COMPILERCHECK ?= "%compiler% -dumpspecs"
 
+export CCACHE_CONFIGPATH ?= "${COREBASE}/meta/conf/ccache.conf"
+
 export CCACHE_DIR ?= "${CCACHE_TOP_DIR}/${MULTIMACH_TARGET_SYS}/${PN}"
 
 # We need to stop ccache considering the current directory or the
diff --git a/meta/conf/ccache.conf b/meta/conf/ccache.conf
new file mode 100644
index 0000000..931012d
--- /dev/null
+++ b/meta/conf/ccache.conf
@@ -0,0 +1,2 @@
+max_size = 0
+cache_dir_levels = 1
-- 
2.7.4

[PATCH V3 07/11] bitbake.conf: Enable -fdebug-prefix-map for nativesdk

[Robert Yang]

Files like lib/.debug/libgcc_s.so.1 from nativesdk-libgcc-dbg contains
buildpath without -fdebug-prefix-map, the root cause is object files (.o)
contain buildpath, and then ccache can't use them correctly.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/conf/bitbake.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index c88d66c..2044a5d 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -558,7 +558,7 @@ export CPPFLAGS = "${TARGET_CPPFLAGS}"
 export TARGET_CPPFLAGS = ""
 
 export BUILD_CFLAGS = "${BUILD_CPPFLAGS} ${BUILD_OPTIMIZATION}"
-BUILDSDK_CFLAGS = "${BUILDSDK_CPPFLAGS} ${BUILD_OPTIMIZATION}"
+BUILDSDK_CFLAGS = "${BUILDSDK_CPPFLAGS} ${BUILD_OPTIMIZATION} ${DEBUG_PREFIX_MAP}"
 export CFLAGS = "${TARGET_CFLAGS}"
 export TARGET_CFLAGS = "${TARGET_CPPFLAGS} ${SELECTED_OPTIMIZATION}"
 
-- 
2.7.4

[PATCH V3 08/11] goarch.bbclass: Set CCACHE_DISABLE

[Robert Yang]

Go can't be built with ccache.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/goarch.bbclass | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/classes/goarch.bbclass b/meta/classes/goarch.bbclass
index b2c94fa..39fea5e 100644
--- a/meta/classes/goarch.bbclass
+++ b/meta/classes/goarch.bbclass
@@ -42,6 +42,10 @@ TUNE_CCARGS_remove = "-march=mips32r2"
 SECURITY_CFLAGS_mipsarch = "${SECURITY_NOPIE_CFLAGS}"
 SECURITY_NOPIE_CFLAGS ??= ""
 
+# go can't be built with ccache:
+# gcc: fatal error: no input files
+CCACHE_DISABLE ?= "1"
+
 def go_map_arch(a, d):
     import re
     if re.match('i.86', a):
-- 
2.7.4

[PATCH V3 09/11] cmake.bbclass: Make it work with ccache

[Robert Yang]

This can make the following recipes work with cmake:
cmake libdnf libcomps librepo createrepo-c llvm dnf libsolv assimp waffle
libjpeg-turbo taglib libproxy libical

And the following 3 recipes don't:
webkitgtk vulkan piglit
Now cmake.bbclass doesn't disble ccache any more, disable it in the recipes if
needed.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/cmake.bbclass                      | 21 +++++++++++++++------
 meta/recipes-graphics/piglit/piglit_git.bb      |  3 +++
 meta/recipes-graphics/vulkan/vulkan_1.1.73.0.bb |  3 +++
 meta/recipes-sato/webkit/webkitgtk_2.22.4.bb    |  3 +++
 4 files changed, 24 insertions(+), 6 deletions(-)

diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index f22d90a..fa7f68c 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -4,9 +4,6 @@ OECMAKE_SOURCEPATH ??= "${S}"
 DEPENDS_prepend = "cmake-native "
 B = "${WORKDIR}/build"
 
-# We need to unset CCACHE otherwise cmake gets too confused
-CCACHE = ""
-
 # What CMake generator to use.
 # The supported options are "Unix Makefiles" or "Ninja".
 OECMAKE_GENERATOR ?= "Ninja"
@@ -23,10 +20,22 @@ python() {
         d.setVarFlag("do_compile", "progress", r"outof:^\[(\d+)/(\d+)\]\s+")
     else:
         bb.fatal("Unknown CMake Generator %s" % generator)
+
+    # C/C++ Compiler (without cpu arch/tune arguments)
+    if not d.getVar('OECMAKE_C_COMPILER'):
+        cc_list = d.getVar('CC').split()
+        if cc_list[0] == 'ccache':
+            d.setVar('OECMAKE_C_COMPILER', '%s %s' % (cc_list[0], cc_list[1]))
+        else:
+            d.setVar('OECMAKE_C_COMPILER', cc_list[0])
+
+    if not d.getVar('OECMAKE_CXX_COMPILER'):
+        cxx_list = d.getVar('CXX').split()
+        if cxx_list[0] == 'ccache':
+            d.setVar('OECMAKE_CXX_COMPILER', '%s %s' % (cxx_list[0], cxx_list[1]))
+        else:
+            d.setVar('OECMAKE_CXX_COMPILER', cxx_list[0])
 }
-# C/C++ Compiler (without cpu arch/tune arguments)
-OECMAKE_C_COMPILER ?= "`echo ${CC} | sed 's/^\([^ ]*\).*/\1/'`"
-OECMAKE_CXX_COMPILER ?= "`echo ${CXX} | sed 's/^\([^ ]*\).*/\1/'`"
 OECMAKE_AR ?= "${AR}"
 
 # Compiler flags
diff --git a/meta/recipes-graphics/piglit/piglit_git.bb b/meta/recipes-graphics/piglit/piglit_git.bb
index 622e046..59d1348 100644
--- a/meta/recipes-graphics/piglit/piglit_git.bb
+++ b/meta/recipes-graphics/piglit/piglit_git.bb
@@ -52,3 +52,6 @@ RDEPENDS_${PN} = "waffle waffle-bin python3 python3-mako python3-json \
 	"
 
 INSANE_SKIP_${PN} += "dev-so already-stripped"
+
+# Can't be built with ccache
+CCACHE_DISABLE = "1"
diff --git a/meta/recipes-graphics/vulkan/vulkan_1.1.73.0.bb b/meta/recipes-graphics/vulkan/vulkan_1.1.73.0.bb
index dd3adee..02e4330 100644
--- a/meta/recipes-graphics/vulkan/vulkan_1.1.73.0.bb
+++ b/meta/recipes-graphics/vulkan/vulkan_1.1.73.0.bb
@@ -35,3 +35,6 @@ PACKAGECONFIG[wayland] = "-DBUILD_WSI_WAYLAND_SUPPORT=ON, -DBUILD_WSI_WAYLAND_SU
 
 RRECOMMENDS_${PN} = "mesa-vulkan-drivers"
 INSANE_SKIP_${PN}-dev += "dev-elf"
+
+# Can't be built with ccache
+CCACHE_DISABLE = "1"
diff --git a/meta/recipes-sato/webkit/webkitgtk_2.22.4.bb b/meta/recipes-sato/webkit/webkitgtk_2.22.4.bb
index 1961220..0f2635c 100644
--- a/meta/recipes-sato/webkit/webkitgtk_2.22.4.bb
+++ b/meta/recipes-sato/webkit/webkitgtk_2.22.4.bb
@@ -121,3 +121,6 @@ ARM_INSTRUCTION_SET_armv7ve = "thumb"
 # Segmentation fault
 GI_DATA_ENABLED_armv7a = "False"
 GI_DATA_ENABLED_armv7ve = "False"
+
+# Can't be built with ccache
+CCACHE_DISABLE = "1"
-- 
2.7.4

[PATCH V3 10/11] apr/openssl10: Enable ccache for them

[Robert Yang]

They work well now.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-connectivity/openssl/openssl10_1.0.2q.bb | 3 ---
 meta/recipes-support/apr/apr_1.6.5.bb                 | 5 -----
 2 files changed, 8 deletions(-)

diff --git a/meta/recipes-connectivity/openssl/openssl10_1.0.2q.bb b/meta/recipes-connectivity/openssl/openssl10_1.0.2q.bb
index 9d67053..809634f 100644
--- a/meta/recipes-connectivity/openssl/openssl10_1.0.2q.bb
+++ b/meta/recipes-connectivity/openssl/openssl10_1.0.2q.bb
@@ -76,9 +76,6 @@ EXTRA_OEMAKE = "${@bb.utils.contains('PACKAGECONFIG', 'manpages', '', 'OE_DISABL
 
 export OE_LDFLAGS = "${LDFLAGS}"
 
-# openssl fails with ccache: https://bugzilla.yoctoproject.org/show_bug.cgi?id=12810
-CCACHE = ""
-
 TERMIO ?= "-DTERMIO"
 TERMIO_libc-musl = "-DTERMIOS"
 EXTRA_OECONF_append_libc-musl_powerpc64 = " no-asm"
diff --git a/meta/recipes-support/apr/apr_1.6.5.bb b/meta/recipes-support/apr/apr_1.6.5.bb
index c2f5900..432e4ed 100644
--- a/meta/recipes-support/apr/apr_1.6.5.bb
+++ b/meta/recipes-support/apr/apr_1.6.5.bb
@@ -37,11 +37,6 @@ CACHED_CONFIGUREVARS += "ac_cv_header_netinet_sctp_h=no ac_cv_header_netinet_sct
 CACHED_CONFIGUREVARS += "ac_cv_sizeof_struct_iovec=yes"
 CACHED_CONFIGUREVARS += "ac_cv_file__dev_zero=yes"
 
-# Otherwise libtool fails to compile apr-utils
-# x86_64-linux-libtool: compile: unable to infer tagged configuration
-# x86_64-linux-libtool:   error: specify a tag with '--tag'
-CCACHE = ""
-
 PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)}"
 PACKAGECONFIG[ipv6] = "--enable-ipv6,--disable-ipv6,"
 
-- 
2.7.4

[PATCH V3 11/11] oeqa/selftest/cases: Update test_ccache_tool

[Robert Yang]

Now we can compile m4-native rather than target m4, this can save a lot of
build time.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/lib/oeqa/selftest/cases/buildoptions.py | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/buildoptions.py b/meta/lib/oeqa/selftest/cases/buildoptions.py
index f234bac..181e9a9 100644
--- a/meta/lib/oeqa/selftest/cases/buildoptions.py
+++ b/meta/lib/oeqa/selftest/cases/buildoptions.py
@@ -37,13 +37,13 @@ class ImageOptionsTests(OESelftestTestCase):
         p = bb_vars['SYSROOT_DESTDIR'] + bb_vars['bindir'] + "/" + "ccache"
         self.assertTrue(os.path.isfile(p), msg = "No ccache found (%s)" % p)
         self.write_config('INHERIT += "ccache"')
-        self.add_command_to_tearDown('bitbake -c clean m4')
-        bitbake("m4 -c clean")
-        bitbake("m4 -f -c compile")
-        log_compile = os.path.join(get_bb_var("WORKDIR","m4"), "temp/log.do_compile")
+        self.add_command_to_tearDown('bitbake -c clean m4-native')
+        bitbake("m4-native -c clean")
+        bitbake("m4-native -f -c compile")
+        log_compile = os.path.join(get_bb_var("WORKDIR","m4-native"), "temp/log.do_compile")
         with open(log_compile, "r") as f:
             loglines = "".join(f.readlines())
-        self.assertIn("ccache", loglines, msg="No match for ccache in m4 log.do_compile. For further details: %s" % log_compile)
+        self.assertIn("ccache", loglines, msg="No match for ccache in m4-native log.do_compile. For further details: %s" % log_compile)
 
     @OETestID(1435)
     def test_read_only_image(self):
-- 
2.7.4

✗ patchtest: failure for ccache.bbclass: Refactor it to make it more reliable (rev3)

[Patchwork]

== Series Details ==

Series: ccache.bbclass: Refactor it to make it more reliable (rev3)
Revision: 3
URL   : https://patchwork.openembedded.org/series/15276/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             LIC_FILES_CHKSUM changed on target ccache but there is no "License-Update" tag in commit message [test_lic_files_chksum_modified_not_mentioned] 
  Suggested fix    Include "License-Update: <description>" into the commit message with a brief description
  Current checksum file://LICENSE.adoc;md5=2722abeaf1750dbf175b9491112374e1
  New checksum     file://LICENSE.adoc;md5=39f34d6ff4df51ed718fd7d243fc4e51



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

Re: ✗ patchtest: failure for ccache.bbclass: Refactor it to make it more reliable (rev3)

[Robert Yang]

Fixed in the repo:

     ccache: 3.4.2 -> 3.5

     License-Update: Copyright year and zlib version are changed, others are the
     same.

     This version supports CCACHE_DEBUG which will make the debug easier.


// Robert

On 12/18/18 3:34 PM, Patchwork wrote:
> == Series Details ==
> 
> Series: ccache.bbclass: Refactor it to make it more reliable (rev3)
> Revision: 3
> URL   : https://patchwork.openembedded.org/series/15276/
> State : failure
> 
> == Summary ==
> 
> 
> Thank you for submitting this patch series to OpenEmbedded Core. This is
> an automated response. Several tests have been executed on the proposed
> series by patchtest resulting in the following failures:
> 
> 
> 
> * Issue             LIC_FILES_CHKSUM changed on target ccache but there is no "License-Update" tag in commit message [test_lic_files_chksum_modified_not_mentioned]
>    Suggested fix    Include "License-Update: <description>" into the commit message with a brief description
>    Current checksum file://LICENSE.adoc;md5=2722abeaf1750dbf175b9491112374e1
>    New checksum     file://LICENSE.adoc;md5=39f34d6ff4df51ed718fd7d243fc4e51
> 
> 
> 
> If you believe any of these test results are incorrect, please reply to the
> mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
> Otherwise we would appreciate you correcting the issues and submitting a new
> version of the patchset if applicable. Please ensure you add/increment the
> version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
> [PATCH v3] -> ...).
> 
> ---
> Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
> Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
> Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
> 
> 

Re: [PATCH V3 00/11] ccache.bbclass: Refactor it to make it more reliable

[Robert Yang]

Hi RP,

Do you have any comments on this, please? It's really very useful during
development. It can save both configure and compile time in the second run,
It can save about 8 minutes (30m -> 22m) if I rebuild core-image-minimal
totally without any sstate caches.

// Robert

On 12/18/18 3:18 PM, Robert Yang wrote:
> * V3
>    - Make cache files can be shared between different build dirs
>    - Add task do_cleanccache
>    - Make cmake work with ccache
>    - Upgrade ccache to 3.5, this version has a debug mode, which is useful for
>      figuring out why a cache file is not used.
> 
> * V2
>    - Fix comments from Martin Jansa and Martin Hundebøll.
>    - Fix RP's comments:
>      1) Add comments for ccache related code in base.bbclass
>      2) Remove all ccache-native's dependencies so that all recipes can depend on
>         and use it.
>    - Update test_ccache_tool in oeqa
> 
> * V1
>    Initial version
> 
> The following changes since commit 20aea61385e1a53ac245353899277ba20104ed2f:
> 
>    gcc: Drop 7.3 since 8.2 is working fine for us (2018-12-15 17:10:44 +0000)
> 
> are available in the git repository at:
> 
>    git://git.openembedded.org/openembedded-core-contrib rbt/ccache
>    http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/ccache
> 
> Robert Yang (11):
>    ccache-native: Remove dependencies
>    ccache: 3.4.2 -> 3.5
>    ccache.bbclass: Refactor it to make it more reliable
>    ccache.bbclass: Make it can be shared between different builds
>    ccache.bbclass: Add task do_cleanccache
>    ccache.bbclass: Set CCACHE_CONFIGPATH to fix race issues
>    bitbake.conf: Enable -fdebug-prefix-map for nativesdk
>    goarch.bbclass: Set CCACHE_DISABLE
>    cmake.bbclass: Make it work with ccache
>    apr/openssl10: Enable ccache for them
>    oeqa/selftest/cases: Update test_ccache_tool
> 
>   meta/classes/ccache.bbclass                        | 63 ++++++++++++++++++++--
>   meta/classes/cmake.bbclass                         | 21 +++++---
>   meta/classes/goarch.bbclass                        |  4 ++
>   meta/conf/bitbake.conf                             |  8 +--
>   meta/conf/ccache.conf                              |  2 +
>   meta/conf/layer.conf                               |  1 -
>   meta/lib/oe/utils.py                               |  3 ++
>   meta/lib/oeqa/selftest/cases/buildoptions.py       | 10 ++--
>   .../openssl/openssl10_1.0.2q.bb                    |  3 --
>   meta/recipes-devtools/ccache/ccache.inc            | 13 ++++-
>   meta/recipes-devtools/ccache/ccache_3.4.2.bb       | 11 ----
>   meta/recipes-devtools/ccache/ccache_3.5.bb         | 11 ++++
>   meta/recipes-graphics/piglit/piglit_git.bb         |  3 ++
>   meta/recipes-graphics/vulkan/vulkan_1.1.73.0.bb    |  3 ++
>   meta/recipes-sato/webkit/webkitgtk_2.22.4.bb       |  3 ++
>   meta/recipes-support/apr/apr_1.6.5.bb              |  5 --
>   16 files changed, 122 insertions(+), 42 deletions(-)
>   create mode 100644 meta/conf/ccache.conf
>   delete mode 100644 meta/recipes-devtools/ccache/ccache_3.4.2.bb
>   create mode 100644 meta/recipes-devtools/ccache/ccache_3.5.bb
> 

Re: [PATCH V3 03/11] ccache.bbclass: Refactor it to make it more reliable

[Randy MacLeod]

A couple of typos and grammar improvements, otherwise a good
improvement for iterative development of a package/image where
sstate isn't useful.

On 12/18/18 2:18 AM, Robert Yang wrote:
> The previous ccache.bbclass has the following problems:
> - It uses host's ccache for native recipes, but this may not work on some
>    hosts, for example, it nerver works on my Ubuntu 14.04.4, there are always
>    build failures (m4-native failed at do_configure, and others will also be
>    failed if I disable CCACHE for m4-native)
> 
> - native/nativesdk/cross/crosssdk recipes use host's ccahe, but target uses
* host's ccache
>    ccache-native, this may confuse user.
> 
> - The target recipes may use both host's ccache and ccache-native, this may
>    cause unexpected problems and hard to debug. This is because ccache-native is
* and be hard to debug
>    in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS, so ccache-native may not be present when
>    rebuild target recipes, and then it would use hosttools/ccache, but the
>    previous ccache files were generated by ccache-native.
> 
> - Target recipes can't use ccache when no ccahe is installed on the host:
* no ccache

>    CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"
> 
> After refactored:
> All types recipes (native, target and others) will use ccache-native except
> ccache-native itself, host's cache won't be used any more. It is more
> reliable now, which will work everywhere when ccache-native can be built.
> 
> And now we need use "CCACHE_DISABLE = '1'" to disable ccahe for the recipe
* disable ccache

../Randy


> rather than "CCACHE = ''" since we set CCACHE in anonymous function, and
> d.getVar('CCACHE') works after "CCACHE ??=" which is set in bitbake.conf, so we
> can't check whether CCACHE is set or not in anonymous function since it is
> always set. Use CCACHE_DISABLE to disable it would be more clear.
> 
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>   meta/classes/ccache.bbclass | 25 ++++++++++++++++++++++---
>   meta/conf/bitbake.conf      |  6 +-----
>   meta/conf/layer.conf        |  1 -
>   meta/lib/oe/utils.py        |  3 +++
>   4 files changed, 26 insertions(+), 9 deletions(-)
> 
> diff --git a/meta/classes/ccache.bbclass b/meta/classes/ccache.bbclass
> index 9609020..59e1022 100644
> --- a/meta/classes/ccache.bbclass
> +++ b/meta/classes/ccache.bbclass
> @@ -1,4 +1,14 @@
> -CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"
> +#
> +# Usage:
> +# - Enable ccache
> +#   Add the following line to a conffile such as conf/local.conf:
> +#   INHERIT += "ccache"
> +#
> +# - Disable ccache for a recipe
> +#   Add the following line to the recipe if it can't be built with ccache:
> +#   CCACHE_DISABLE = '1'
> +#
> +
>   export CCACHE_DIR ?= "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}"
>   
>   # We need to stop ccache considering the current directory or the
> @@ -7,5 +17,14 @@ export CCACHE_DIR ?= "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}"
>   # ${PV} or ${PR} change.
>   export CCACHE_NOHASHDIR ?= "1"
>   
> -DEPENDS_append_class-target = " ccache-native"
> -DEPENDS[vardepvalueexclude] = " ccache-native"
> +python() {
> +    """
> +    Enable ccache for the recipe
> +    """
> +    pn = d.getVar('PN')
> +    # quilt-native doesn't need ccache since no c files
> +    if not (pn in ('ccache-native', 'quilt-native') or
> +            bb.utils.to_boolean(d.getVar('CCACHE_DISABLE'))):
> +        d.appendVar('DEPENDS', ' ccache-native')
> +        d.setVar('CCACHE', 'ccache ')
> +}
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 6480062..c88d66c 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -492,7 +492,7 @@ HOSTTOOLS += " \
>   HOSTTOOLS += "${@'ip ping ps scp ssh stty' if (bb.utils.contains_any('IMAGE_CLASSES', 'testimage testsdk', True, False, d) or any(x in (d.getVar("BBINCLUDED") or "") for x in ["testimage.bbclass", "testsdk.bbclass"])) else ''}"
>   
>   # Link to these if present
> -HOSTTOOLS_NONFATAL += "aws ccache gcc-ar gpg ld.bfd ld.gold nc pigz sftp socat ssh sudo"
> +HOSTTOOLS_NONFATAL += "aws gcc-ar gpg ld.bfd ld.gold nc pigz sftp socat ssh sudo"
>   
>   # Temporary add few more detected in bitbake world
>   HOSTTOOLS_NONFATAL += "join nl size yes zcat"
> @@ -504,10 +504,6 @@ HOSTTOOLS_NONFATAL += "bzr"
>   HOSTTOOLS_NONFATAL += "scp"
>   
>   CCACHE ??= ""
> -# ccache < 3.1.10 will create CCACHE_DIR on startup even if disabled, and
> -# autogen sets HOME=/dev/null so in certain situations builds can fail.
> -# Explicitly export CCACHE_DIR until we can assume ccache >3.1.10 on the host.
> -export CCACHE_DIR ??= "${@os.getenv('HOME')}/.ccache"
>   
>   TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
>   
> diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
> index c67dec4..4141d6c 100644
> --- a/meta/conf/layer.conf
> +++ b/meta/conf/layer.conf
> @@ -49,7 +49,6 @@ SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS += " \
>     *->quilt-native \
>     *->subversion-native \
>     *->git-native \
> -  *->ccache-native \
>     *->icecc-create-env-native \
>     gcc-cross-${TARGET_ARCH}->virtual/${TARGET_PREFIX}libc-for-gcc \
>     gcc-cross-${TARGET_ARCH}->linux-libc-headers \
> diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
> index 29b4115..ee6f0e6 100644
> --- a/meta/lib/oe/utils.py
> +++ b/meta/lib/oe/utils.py
> @@ -363,6 +363,9 @@ def host_gcc_version(d, taskcontextonly=False):
>           return
>   
>       compiler = d.getVar("BUILD_CC")
> +    # Get rid of ccache since it is not present when parsing.
> +    if compiler.startswith('ccache '):
> +        compiler = compiler[7:]
>       try:
>           env = os.environ.copy()
>           env["PATH"] = d.getVar("PATH")
> 


-- 
# Randy MacLeod
# Wind River Linux

Re: [PATCH V3 03/11] ccache.bbclass: Refactor it to make it more reliable

[Robert Yang]

Hi Randy,

I'm sorry for the types, and thanks for the correction. I found that it has
been merged, I will be more careful.

// Robert

On 1/12/19 4:49 AM, Randy MacLeod wrote:
> A couple of typos and grammar improvements, otherwise a good
> improvement for iterative development of a package/image where
> sstate isn't useful.
> 
> On 12/18/18 2:18 AM, Robert Yang wrote:
>> The previous ccache.bbclass has the following problems:
>> - It uses host's ccache for native recipes, but this may not work on some
>>    hosts, for example, it nerver works on my Ubuntu 14.04.4, there are always
>>    build failures (m4-native failed at do_configure, and others will also be
>>    failed if I disable CCACHE for m4-native)
>>
>> - native/nativesdk/cross/crosssdk recipes use host's ccahe, but target uses
> * host's ccache
>>    ccache-native, this may confuse user.
>>
>> - The target recipes may use both host's ccache and ccache-native, this may
>>    cause unexpected problems and hard to debug. This is because ccache-native is
> * and be hard to debug
>>    in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS, so ccache-native may not be present when
>>    rebuild target recipes, and then it would use hosttools/ccache, but the
>>    previous ccache files were generated by ccache-native.
>>
>> - Target recipes can't use ccache when no ccahe is installed on the host:
> * no ccache
> 
>>    CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"
>>
>> After refactored:
>> All types recipes (native, target and others) will use ccache-native except
>> ccache-native itself, host's cache won't be used any more. It is more
>> reliable now, which will work everywhere when ccache-native can be built.
>>
>> And now we need use "CCACHE_DISABLE = '1'" to disable ccahe for the recipe
> * disable ccache
> 
> ../Randy
> 
> 
>> rather than "CCACHE = ''" since we set CCACHE in anonymous function, and
>> d.getVar('CCACHE') works after "CCACHE ??=" which is set in bitbake.conf, so we
>> can't check whether CCACHE is set or not in anonymous function since it is
>> always set. Use CCACHE_DISABLE to disable it would be more clear.
>>
>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>> ---
>>   meta/classes/ccache.bbclass | 25 ++++++++++++++++++++++---
>>   meta/conf/bitbake.conf      |  6 +-----
>>   meta/conf/layer.conf        |  1 -
>>   meta/lib/oe/utils.py        |  3 +++
>>   4 files changed, 26 insertions(+), 9 deletions(-)
>>
>> diff --git a/meta/classes/ccache.bbclass b/meta/classes/ccache.bbclass
>> index 9609020..59e1022 100644
>> --- a/meta/classes/ccache.bbclass
>> +++ b/meta/classes/ccache.bbclass
>> @@ -1,4 +1,14 @@
>> -CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"
>> +#
>> +# Usage:
>> +# - Enable ccache
>> +#   Add the following line to a conffile such as conf/local.conf:
>> +#   INHERIT += "ccache"
>> +#
>> +# - Disable ccache for a recipe
>> +#   Add the following line to the recipe if it can't be built with ccache:
>> +#   CCACHE_DISABLE = '1'
>> +#
>> +
>>   export CCACHE_DIR ?= "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}"
>>   # We need to stop ccache considering the current directory or the
>> @@ -7,5 +17,14 @@ export CCACHE_DIR ?= 
>> "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}"
>>   # ${PV} or ${PR} change.
>>   export CCACHE_NOHASHDIR ?= "1"
>> -DEPENDS_append_class-target = " ccache-native"
>> -DEPENDS[vardepvalueexclude] = " ccache-native"
>> +python() {
>> +    """
>> +    Enable ccache for the recipe
>> +    """
>> +    pn = d.getVar('PN')
>> +    # quilt-native doesn't need ccache since no c files
>> +    if not (pn in ('ccache-native', 'quilt-native') or
>> +            bb.utils.to_boolean(d.getVar('CCACHE_DISABLE'))):
>> +        d.appendVar('DEPENDS', ' ccache-native')
>> +        d.setVar('CCACHE', 'ccache ')
>> +}
>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>> index 6480062..c88d66c 100644
>> --- a/meta/conf/bitbake.conf
>> +++ b/meta/conf/bitbake.conf
>> @@ -492,7 +492,7 @@ HOSTTOOLS += " \
>>   HOSTTOOLS += "${@'ip ping ps scp ssh stty' if 
>> (bb.utils.contains_any('IMAGE_CLASSES', 'testimage testsdk', True, False, d) 
>> or any(x in (d.getVar("BBINCLUDED") or "") for x in ["testimage.bbclass", 
>> "testsdk.bbclass"])) else ''}"
>>   # Link to these if present
>> -HOSTTOOLS_NONFATAL += "aws ccache gcc-ar gpg ld.bfd ld.gold nc pigz sftp 
>> socat ssh sudo"
>> +HOSTTOOLS_NONFATAL += "aws gcc-ar gpg ld.bfd ld.gold nc pigz sftp socat ssh 
>> sudo"
>>   # Temporary add few more detected in bitbake world
>>   HOSTTOOLS_NONFATAL += "join nl size yes zcat"
>> @@ -504,10 +504,6 @@ HOSTTOOLS_NONFATAL += "bzr"
>>   HOSTTOOLS_NONFATAL += "scp"
>>   CCACHE ??= ""
>> -# ccache < 3.1.10 will create CCACHE_DIR on startup even if disabled, and
>> -# autogen sets HOME=/dev/null so in certain situations builds can fail.
>> -# Explicitly export CCACHE_DIR until we can assume ccache >3.1.10 on the host.
>> -export CCACHE_DIR ??= "${@os.getenv('HOME')}/.ccache"
>>   TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
>> diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
>> index c67dec4..4141d6c 100644
>> --- a/meta/conf/layer.conf
>> +++ b/meta/conf/layer.conf
>> @@ -49,7 +49,6 @@ SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS += " \
>>     *->quilt-native \
>>     *->subversion-native \
>>     *->git-native \
>> -  *->ccache-native \
>>     *->icecc-create-env-native \
>>     gcc-cross-${TARGET_ARCH}->virtual/${TARGET_PREFIX}libc-for-gcc \
>>     gcc-cross-${TARGET_ARCH}->linux-libc-headers \
>> diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
>> index 29b4115..ee6f0e6 100644
>> --- a/meta/lib/oe/utils.py
>> +++ b/meta/lib/oe/utils.py
>> @@ -363,6 +363,9 @@ def host_gcc_version(d, taskcontextonly=False):
>>           return
>>       compiler = d.getVar("BUILD_CC")
>> +    # Get rid of ccache since it is not present when parsing.
>> +    if compiler.startswith('ccache '):
>> +        compiler = compiler[7:]
>>       try:
>>           env = os.environ.copy()
>>           env["PATH"] = d.getVar("PATH")
>>
> 
> 

Re: [PATCH V3 03/11] ccache.bbclass: Refactor it to make it more reliable

[Richard Purdie]

On Tue, 2019-01-15 at 10:11 +0800, Robert Yang wrote:
> Hi Randy,
> 
> I'm sorry for the types, and thanks for the correction. I found that
> it has
> been merged, I will be more careful.

I did tweak the commit message!

Cheers,

Richard

Re: [PATCH V3 03/11] ccache.bbclass: Refactor it to make it more reliable

[Robert Yang]

On 1/16/19 6:40 AM, Richard Purdie wrote:
> On Tue, 2019-01-15 at 10:11 +0800, Robert Yang wrote:
>> Hi Randy,
>>
>> I'm sorry for the types, and thanks for the correction. I found that
>> it has
>> been merged, I will be more careful.
> 
> I did tweak the commit message!

Thank you very much!

// Robert

> 
> Cheers,
> 
> Richard
> 
>