[Buildroot] [pull request] Pull request for branch for-2012.05/ccache-fixes

[Buildroot] [pull request] Pull request for branch for-2012.05/ccache-fixes

[Thomas Petazzoni]

The following changes since commit b50f7a832de2801aeee33146ff5a81945f4078c2:

  barebox: fix 2012.03 typo (2012-03-07 16:26:50 +0100)

are available in the git repository at:
  git://git.free-electrons.com/users/thomas-petazzoni/buildroot.git for-2012.05/ccache-fixes

Thomas Petazzoni (2):
      ccache: Force ccache to use its internal zlib.
      ccache: set COMPILERCHECK to 'none'

 Config.in                |    7 +++++++
 package/ccache/ccache.mk |   23 +++++++++++++++++++++--
 2 files changed, 28 insertions(+), 2 deletions(-)

Thanks,
-- 
Thomas Petazzoni

[Buildroot] [PATCH 1/2] ccache: Force ccache to use its internal zlib.

[Thomas Petazzoni]

The problem is that without this, ccache would link against the zlib
of the build system, but we might build and install a different
version of zlib in $(O)/host afterwards, which ccache will pick
up. This might break if there is a version mismatch. A solution would
be to add host-zlib has a dependency of ccache, but it would require
tuning the zlib .mk file to use HOSTCC_NOCCACHE as the
compiler. Instead, we take the easy path: tell ccache to use its
internal copy of zlib, so that ccache has zero dependency besides the
C library.

Fixes bug #4808.

Thanks to Ra?l S?nchez Siles <rsanchezs@infoglobal.es> for reporting
the bug and testing the proposed solution.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/ccache/ccache.mk |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
index 9155441..393989f 100644
--- a/package/ccache/ccache.mk
+++ b/package/ccache/ccache.mk
@@ -14,6 +14,17 @@ CCACHE_SOURCE  = ccache-$(CCACHE_VERSION).tar.bz2
 HOST_CCACHE_CONF_ENV = \
 	CC="$(HOSTCC_NOCCACHE)"
 
+# Force ccache to use its internal zlib. The problem is that without
+# this, ccache would link against the zlib of the build system, but we
+# might build and install a different version of zlib in $(O)/host
+# afterwards, which ccache will pick up. This might break if there is
+# a version mismatch. A solution would be to add host-zlib has a
+# dependency of ccache, but it would require tuning the zlib .mk file
+# to use HOSTCC_NOCCACHE as the compiler. Instead, we take the easy
+# path: tell ccache to use its internal copy of zlib, so that ccache
+# has zero dependency besides the C library.
+HOST_CCACHE_CONF_OPT += ccache_cv_zlib_1_2_3=no
+
 # We directly hardcode the cache location into the binary, as it is
 # much easier to handle than passing an environment variable.
 define HOST_CCACHE_FIX_CCACHE_DIR
-- 
1.7.4.1

[Buildroot] [PATCH 1/2] ccache: Force ccache to use its internal zlib.

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> The problem is that without this, ccache would link against the zlib
 Thomas> of the build system, but we might build and install a different
 Thomas> version of zlib in $(O)/host afterwards, which ccache will pick
 Thomas> up. This might break if there is a version mismatch. A solution would
 Thomas> be to add host-zlib has a dependency of ccache, but it would require
 Thomas> tuning the zlib .mk file to use HOSTCC_NOCCACHE as the
 Thomas> compiler. Instead, we take the easy path: tell ccache to use its
 Thomas> internal copy of zlib, so that ccache has zero dependency besides the
 Thomas> C library.

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 2/2] ccache: set COMPILERCHECK to 'none'

[Thomas Petazzoni]

This allows ccache to re-use its cache contents even if the compiler
binary mtime has changed. It is the simplest approach to solve this
problem, and it works for the internal, external and crosstool-ng
toolchain backends.

Of course, it leaves the user responsible for invalidating the cache
when necessary, but there doesn't seem to be a real good solution that
allows both to: 1/ keep the cache contents accross builds and re-use
it and 2/ invalidate the cache automatically when the compiler chances
in an incompatible way.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 Config.in                |    7 +++++++
 package/ccache/ccache.mk |   12 ++++++++++--
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/Config.in b/Config.in
index 8579a63..2002996 100644
--- a/Config.in
+++ b/Config.in
@@ -191,6 +191,13 @@ config BR2_CCACHE
 	  up future builds. The cache is stored in
 	  $HOME/.buildroot-ccache.
 
+	  Note that Buildroot does not try to invalidate the cache
+	  contents when the compiler changes in an incompatible
+	  way. Therefore, if you make a change to the compiler version
+	  and/or configuration, you are responsible for purging the
+	  ccache cache by removing the $HOME/.buildroot-ccache
+	  directory.
+
 config BR2_DEPRECATED
 	bool "Show packages that are deprecated or obsolete"
 	help
diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
index 393989f..f71376e 100644
--- a/package/ccache/ccache.mk
+++ b/package/ccache/ccache.mk
@@ -25,10 +25,18 @@ HOST_CCACHE_CONF_ENV = \
 # has zero dependency besides the C library.
 HOST_CCACHE_CONF_OPT += ccache_cv_zlib_1_2_3=no
 
-# We directly hardcode the cache location into the binary, as it is
-# much easier to handle than passing an environment variable.
+# We directly hardcode configuration into the binary, as it is much
+# easier to handle than passing an environment variable. Our
+# configuration is:
+#  - the cache location
+#  - the fact that ccache shouldn't use the compiler binary mtime to
+#  - detect a change in the compiler, because in the context of
+#  - Buildroot, that completely defeats the purpose of ccache. Of
+#  - course, that leaves the user responsible for purging its cache
+#  - when the compiler changes.
 define HOST_CCACHE_FIX_CCACHE_DIR
 	sed -i 's,getenv("CCACHE_DIR"),"$(CCACHE_CACHE_DIR)",' $(@D)/ccache.c
+	sed -i 's,getenv("CCACHE_COMPILERCHECK"),"none",' $(@D)/ccache.c
 endef
 
 HOST_CCACHE_POST_CONFIGURE_HOOKS += \
-- 
1.7.4.1

[Buildroot] [PATCH 2/2] ccache: set COMPILERCHECK to 'none'

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> This allows ccache to re-use its cache contents even if the compiler
 Thomas> binary mtime has changed. It is the simplest approach to solve this
 Thomas> problem, and it works for the internal, external and crosstool-ng
 Thomas> toolchain backends.

 Thomas> Of course, it leaves the user responsible for invalidating the cache
 Thomas> when necessary, but there doesn't seem to be a real good solution that
 Thomas> allows both to: 1/ keep the cache contents accross builds and re-use
 Thomas> it and 2/ invalidate the cache automatically when the compiler chances
 Thomas> in an incompatible way.

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 2/2] ccache: set COMPILERCHECK to 'none'

[Arnout Vandecappelle]

On Wednesday 07 March 2012 20:26:50 Thomas Petazzoni wrote:
> Of course, it leaves the user responsible for invalidating the cache
> when necessary

 Maybe we should then also add a simple way to clear the cache...

 Regards,
 Arnout


-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20120310/d380e3fa/attachment-0001.html>

[Buildroot] [pull request] Pull request for branch for-2012.05/ccache-fixes

[Thomas Petazzoni]

Hello,

Le Wed,  7 Mar 2012 20:26:48 +0100,
Thomas Petazzoni <thomas.petazzoni@free-electrons.com> a ?crit :

>       ccache: set COMPILERCHECK to 'none'

Some numbers about this.

Without the patch, a build of Qt with an empty cache takes the
following time:

real    8m17.607s
user    31m30.830s
sys     1m8.990s

Without the patch, a build of Qt with a cache full takes the following
time:

real    8m12.630s
user    31m17.340s
sys     1m8.520s

So basically, no improvement.

With the patch, a build of Qt with a cache full takes the following
time:

real    4m37.559s
user    5m13.390s
sys     0m31.840s

Here, there is clearly a nice impact of ccache.

I have decided to take a simpler approach than the one explored by
Arnout. Arnout approach was, IMO, too complicated, and only applicable
to external toolchains.

My approach is simpler, but indeed requires attention from the
developer on whether the cache should be invalidated or not. However, I
am not sure it is possible to provide a solution that is both safe (in
terms of cache contents validity) *and* efficient (in terms of cache
usage). Also, the approach taken by my patch is basically the same as
what was happening with previous versions of ccache, so this is not a
new behaviour.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com