[Buildroot] [pull request v2] Pull request for branch for-2011.02/fix-ccache-support

[Buildroot] [pull request v2] Pull request for branch for-2011.02/fix-ccache-support

[Thomas Petazzoni]

Hello,

[ Sorry, previous pull request had some mistakes. ]

Here is a patch that reworks the ccache support to make it work and
make it useful. It is now a "Build options" rather than a "Toolchain
option". I have tested it with both internal toolchain and external
toolchain. The principle is simply to build ccache as early as
possible in the build process, and then use it to cache host and
target compiled files.

The ccache cache is kept in ~/.buildroot-ccache/, so that it can be
shared between different builds.

I'm not using the symlink technique that was used in the past, but
rather prefixing all invocation to host gcc and target gcc with
"$(HOST_DIR)/usr/bin/ccache".

Here are some speed-up results :

 *) First test on my laptop (dual core, 2 GB of RAM), using an ARM
    external toolchain, building Busybox and a couple of packages.

     -> ccache disabled:            5 minutes 29 seconds
     -> ccache enabled, cold cache: 5 minutes 47 seconds
     -> ccache enabled, hot cache:  3 minutes 40 seconds

    The test with "cold cache" is done after removing
    ~/.buildroot-ccache/. The test with "hot cache" is done after
    cleaning up the Buildroot build tree, but keeping the cache of the
    previous build.

    Here we see that ccache has a little cost, but that once the cache
    is hot from previous builds, the build speed-up is interesting.

 *) Second test on my build server (quad core i7, 12 GB of RAM), using
    an ARM internal toolchain, building Busybox and an ext2
    filesystem.

     -> ccache disabled:             7 minutes 10 seconds
     -> ccache enabled, cold cache:  5 minutes 43 seconds
     -> ccache enabled, hot cache:   4 minutes 24 seconds

    In this case we see that even when the cache is cold, ccache is
    useful. It's probably because gcc gets recompiled twice during the
    toolchain build process.

Regards,

Thomas

The following changes since commit 7e9c3a387820154fd1355f23c2669072c0c3a5f7:
  Peter Korsgaard (1):
        docs/news.html: add 2010.11 announce link

are available in the git repository at:

  git://git.busybox.net/~tpetazzoni/git/buildroot for-2011.02/fix-ccache-support

Thomas Petazzoni (1):
      ccache: rework ccache management

 Config.in                                |    8 ++
 Makefile                                 |   27 +++-
 docs/buildroot.html                      |   26 ++++
 package/Config.in                        |    2 +-
 package/Makefile.in                      |   10 ++-
 package/ccache/Config.in                 |    5 +
 package/ccache/ccache.mk                 |   32 +++++
 toolchain/ccache/Config.in               |   16 ---
 toolchain/ccache/Config.in.2             |   13 --
 toolchain/ccache/ccache.mk               |  196 ------------------------------
 toolchain/dependencies/dependencies.sh   |    2 +-
 toolchain/toolchain-buildroot.mk         |    1 -
 toolchain/toolchain-buildroot/Config.in  |    1 -
 toolchain/toolchain-crosstool-ng.mk      |    1 -
 toolchain/toolchain-external.mk          |    1 -
 toolchain/toolchain-external/ext-tool.mk |    2 +-
 16 files changed, 103 insertions(+), 240 deletions(-)
 create mode 100644 package/ccache/Config.in
 create mode 100644 package/ccache/ccache.mk
 delete mode 100644 toolchain/ccache/Config.in
 delete mode 100644 toolchain/ccache/Config.in.2
 delete mode 100644 toolchain/ccache/ccache.mk

Thanks,
-- 
Thomas Petazzoni

[Buildroot] [PATCH 1/1] ccache: rework ccache management

[Thomas Petazzoni]

 * ccache is now a normal package (both for the host and the target).

 * ccache option is now part of the "Build options" menu. It will
   automatically build ccache for the host before building anything,
   and will use it to cache builds for both host compilations and
   target compilations.

 * bump ccache to 3.1.3

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 Config.in                                |    8 ++
 Makefile                                 |   27 +++-
 docs/buildroot.html                      |   26 ++++
 package/Config.in                        |    2 +-
 package/Makefile.in                      |   10 ++-
 package/ccache/Config.in                 |    5 +
 package/ccache/ccache.mk                 |   32 +++++
 toolchain/ccache/Config.in               |   16 ---
 toolchain/ccache/Config.in.2             |   13 --
 toolchain/ccache/ccache.mk               |  196 ------------------------------
 toolchain/dependencies/dependencies.sh   |    2 +-
 toolchain/toolchain-buildroot.mk         |    1 -
 toolchain/toolchain-buildroot/Config.in  |    1 -
 toolchain/toolchain-crosstool-ng.mk      |    1 -
 toolchain/toolchain-external.mk          |    1 -
 toolchain/toolchain-external/ext-tool.mk |    2 +-
 16 files changed, 103 insertions(+), 240 deletions(-)
 create mode 100644 package/ccache/Config.in
 create mode 100644 package/ccache/ccache.mk
 delete mode 100644 toolchain/ccache/Config.in
 delete mode 100644 toolchain/ccache/Config.in.2
 delete mode 100644 toolchain/ccache/ccache.mk

diff --git a/Config.in b/Config.in
index a41cd3e..f6e894e 100644
--- a/Config.in
+++ b/Config.in
@@ -94,6 +94,14 @@ config BR2_JLEVEL
 	help
 	  Number of jobs to run simultaneously
 
+config BR2_CCACHE
+	bool "Enable compiler cache"
+	help
+	  This option will enable the use of ccache, a compiler
+	  cache. It will cache the result of previous builds to speed
+	  up future builds. The cache is stored in
+	  $HOME/.buildroot-ccache.
+
 config BR2_DEPRECATED
 	bool "Show packages that are deprecated or obsolete"
 	help
diff --git a/Makefile b/Makefile
index 596470f..f811cbf 100644
--- a/Makefile
+++ b/Makefile
@@ -125,10 +125,13 @@ HOSTAS:=as
 endif
 ifndef HOSTCC
 HOSTCC:=gcc
-else
+HOSTCC:=$(shell which $(HOSTCC) || type -p $(HOSTCC) || echo gcc)
+HOSTCC_NOCCACHE:=$(HOSTCC)
 endif
 ifndef HOSTCXX
 HOSTCXX:=g++
+HOSTCXX:=$(shell which $(HOSTCXX) || type -p $(HOSTCXX) || echo g++)
+HOSTCXX_NOCCACHE:=$(HOSTCXX)
 endif
 ifndef HOSTFC
 HOSTFC:=gfortran
@@ -147,15 +150,13 @@ HOSTNM:=nm
 endif
 HOSTAR:=$(shell which $(HOSTAR) || type -p $(HOSTAR) || echo ar)
 HOSTAS:=$(shell which $(HOSTAS) || type -p $(HOSTAS) || echo as)
-HOSTCC:=$(shell which $(HOSTCC) || type -p $(HOSTCC) || echo gcc)
-HOSTCXX:=$(shell which $(HOSTCXX) || type -p $(HOSTCXX) || echo g++)
 HOSTFC:=$(shell which $(HOSTLD) || type -p $(HOSTLD) || echo || which g77 || type -p g77 || echo gfortran)
 HOSTCPP:=$(shell which $(HOSTCPP) || type -p $(HOSTCPP) || echo cpp)
 HOSTLD:=$(shell which $(HOSTLD) || type -p $(HOSTLD) || echo ld)
 HOSTLN:=$(shell which $(HOSTLN) || type -p $(HOSTLN) || echo ln)
 HOSTNM:=$(shell which $(HOSTNM) || type -p $(HOSTNM) || echo nm)
 
-export HOSTAR HOSTAS HOSTCC HOSTCXX HOSTFC HOSTLD
+export HOSTAR HOSTAS HOSTCC HOSTCXX HOSTFC HOSTLD HOSTCC_NOCCACHE
 
 # bash prints the name of the directory on 'cd <dir>' if CDPATH is
 # set, so unset it here to not cause problems. Notice that the export
@@ -231,10 +232,15 @@ PREFERRED_LIB_FLAGS:=--enable-static --enable-shared
 # along with the packages to build for the target.
 #
 ##############################################################
+
+ifeq ($(BR2_CCACHE),y)
+BASE_TARGETS += host-ccache
+endif
+
 ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
-BASE_TARGETS:=uclibc-configured binutils cross_compiler uclibc-target-utils kernel-headers
+BASE_TARGETS += uclibc-configured binutils cross_compiler uclibc-target-utils kernel-headers
 else
-BASE_TARGETS:=uclibc
+BASE_TARGETS += uclibc
 endif
 TARGETS:=
 
@@ -276,6 +282,13 @@ TARGET_SKELETON=$(TOPDIR)/fs/skeleton
 
 BR2_DEPENDS_DIR=$(BUILD_DIR)/buildroot-config
 
+ifeq ($(BR2_CCACHE),y)
+CCACHE:=$(HOST_DIR)/usr/bin/ccache
+CCACHE_CACHE_DIR=$(HOME)/.buildroot-ccache
+HOSTCC  := $(CCACHE) $(HOSTCC)
+HOSTCXX := $(CCACHE) $(HOSTCXX)
+endif
+
 include toolchain/Makefile.in
 include package/Makefile.in
 
@@ -480,7 +493,7 @@ export HOSTCFLAGS
 
 $(BUILD_DIR)/buildroot-config/%onf:
 	mkdir -p $(@D)/lxdialog
-	$(MAKE) CC="$(HOSTCC)" obj=$(@D) -C $(CONFIG) -f Makefile.br $(@F)
+	$(MAKE) CC="$(HOSTCC_NOCCACHE)" HOSTCC="$(HOSTCC_NOCCACHE)" obj=$(@D) -C $(CONFIG) -f Makefile.br $(@F)
 
 COMMON_CONFIG_ENV = \
 	KCONFIG_AUTOCONFIG=$(BUILD_DIR)/buildroot-config/auto.conf \
diff --git a/docs/buildroot.html b/docs/buildroot.html
index f471962..6d7ee17 100644
--- a/docs/buildroot.html
+++ b/docs/buildroot.html
@@ -29,6 +29,7 @@
       <li><a href="#buildroot_innards">How Buildroot works</a></li>
       <li><a href="#using_toolchain">Using the uClibc toolchain outside Buildroot</a></li>
       <li><a href="#external_toolchain">Use an external toolchain</a></li>
+      <li><a href="#ccache-support">Using <code>ccache</code> in Buildroot</li>
       <li><a href="#downloaded_packages">Location of downloaded packages</a></li>
       <li><a href="#add_packages">Adding new packages to Buildroot</a></li>
       <li><a href="#board_support">Creating your own board support</a></li>
@@ -681,6 +682,31 @@ endif
     Build options -&gt; Toolchain and header file location</code> options.
     This could be useful if the toolchain must be shared with other users.</p>
 
+    <h2 id="ccache-support">Using <code>ccache</code> in Buildroot</h2>
+
+    <p><a href="http://ccache.samba.org">ccache</a> is a compiler
+    cache. It stores the object files resulting from each compilation
+    process, and is able to skip future compilation of the same source
+    file (with same compiler and same arguments) by using the
+    pre-existing object files. When doing almost identical builds from
+    scratch a number of times, it can divide the build time by a
+    factor of two.</p>
+
+    <p><code>ccache</code> support is integrated in Buildroot. You
+    just have to enable <code>Enable compiler cache</code>
+    in <code>Build options</code>. This will automatically build
+    <code>ccache</code> and use it for every host and target
+    compilation.</p>
+
+    <p>The cache is located
+    in <code>$HOME/.buildroot-ccache</code>. It is stored outside of
+    Buildroot output directory so that it can be shared by separate
+    Buildroot builds. If you want to get rid of the cache, simply
+    remove this directory.</p>
+
+    <p>You can get statistics on the cache (its size, number of hits,
+    misses, etc.) by running <code>make ccache-stats</code>.</p>
+
     <h2 id="downloaded_packages">Location of downloaded packages</h2>
 
     <p>It might be useful to know that the various tarballs that are
diff --git a/package/Config.in b/package/Config.in
index 03e580a..e551c0c 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -35,7 +35,7 @@ source "package/autoconf/Config.in"
 source "package/automake/Config.in"
 source "package/bison/Config.in"
 source "package/bsdiff/Config.in"
-source "toolchain/ccache/Config.in.2"
+source "package/ccache/Config.in"
 if BR2_PACKAGE_BUSYBOX_SHOW_OTHERS
 source "package/coreutils/Config.in"
 endif
diff --git a/package/Makefile.in b/package/Makefile.in
index 2ca8c31..d448a7e 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -142,6 +142,14 @@ TARGET_RANLIB   = $(TARGET_CROSS)ranlib
 TARGET_OBJCOPY  = $(TARGET_CROSS)objcopy
 TARGET_OBJDUMP  = $(TARGET_CROSS)objdump
 
+TARGET_CC_NOCCACHE  := $(TARGET_CC)
+TARGET_CXX_NOCCACHE := $(TARGET_CXX)
+
+ifeq ($(BR2_CCACHE),y)
+TARGET_CC  := $(CCACHE) $(TARGET_CC)
+TARGET_CXX := $(CCACHE) $(TARGET_CXX)
+endif
+
 ifeq ($(BR2_STRIP_strip),y)
 STRIP_DISCARD_ALL:=--discard-all
 STRIP_STRIP_UNNEEDED:=--strip-unneeded
@@ -175,7 +183,7 @@ HOST_LDFLAGS  += -L$(HOST_DIR)/lib -L$(HOST_DIR)/usr/lib -Wl,-rpath,$(HOST_DIR)/
 HOST_PATH=$(HOST_DIR)/bin:$(HOST_DIR)/usr/bin:$(PATH)
 
 # hostcc version as an integer - E.G. 4.3.2 => 432
-HOSTCC_VERSION:=$(shell $(HOSTCC) --version | \
+HOSTCC_VERSION:=$(shell $(HOSTCC_NOCCACHE) --version | \
 	sed -n 's/^.* \([0-9]*\)\.\([0-9]*\)\.\([0-9]*\)[ ]*.*$$/\1\2\3/p')
 
 TARGET_CONFIGURE_OPTS=PATH=$(TARGET_PATH) \
diff --git a/package/ccache/Config.in b/package/ccache/Config.in
new file mode 100644
index 0000000..a81e699
--- /dev/null
+++ b/package/ccache/Config.in
@@ -0,0 +1,5 @@
+config BR2_PACKAGE_CCACHE
+	bool "ccache"
+	help
+	  Compiler Cache
+	  http://samba.org/ftp/ccache
\ No newline at end of file
diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
new file mode 100644
index 0000000..aed62d7
--- /dev/null
+++ b/package/ccache/ccache.mk
@@ -0,0 +1,32 @@
+#############################################################
+#
+# ccache
+#
+#############################################################
+
+CCACHE_VERSION = 3.1.3
+CCACHE_SITE    = http://samba.org/ftp/ccache
+CCACHE_SOURCE  = ccache-$(CCACHE_VERSION).tar.bz2
+
+# When ccache is being built for the host, ccache is not yet
+# available, so we have to use the special C compiler without the
+# cache.
+HOST_CCACHE_CONF_ENV = \
+	CC="$(HOSTCC_NOCCACHE)"
+
+# We directly hardcode the cache location into the binary, as it is
+# much easier to handle than passing an environment variable.
+define HOST_CCACHE_FIX_CCACHE_DIR
+	sed -i 's,getenv("CCACHE_DIR"),"$(CCACHE_CACHE_DIR)",' $(@D)/ccache.c
+endef
+
+HOST_CCACHE_POST_CONFIGURE_HOOKS += \
+	HOST_CCACHE_FIX_CCACHE_DIR
+
+$(eval $(call AUTOTARGETS,package,ccache))
+$(eval $(call AUTOTARGETS,package,ccache,host))
+
+ifeq ($(BR2_CCACHE),y)
+ccache-stats: host-ccache
+	$(Q)$(CCACHE) -s
+endif
\ No newline at end of file
diff --git a/toolchain/ccache/Config.in b/toolchain/ccache/Config.in
deleted file mode 100644
index 6f36650..0000000
--- a/toolchain/ccache/Config.in
+++ /dev/null
@@ -1,16 +0,0 @@
-#
-
-comment "Ccache Options"
-
-config BR2_CCACHE
-	bool "Enable ccache support?"
-	help
-	    Enable ccache support?
-
-config BR2_CCACHE_DIR
-	string "ccache dir location?"
-	depends on BR2_CCACHE
-	default "$(TOOLCHAIN_DIR)/ccache-$(CCACHE_VER)/cache"
-	help
-	    Where ccache should store cached files.
-
diff --git a/toolchain/ccache/Config.in.2 b/toolchain/ccache/Config.in.2
deleted file mode 100644
index 6a6c789..0000000
--- a/toolchain/ccache/Config.in.2
+++ /dev/null
@@ -1,13 +0,0 @@
-#
-
-config BR2_PACKAGE_CCACHE_TARGET
-	bool "ccache"
-	depends on BR2_PACKAGE_GCC_TARGET
-	help
-	  ccache is a compiler cache. It speeds up recompilation of
-	  C/C++ code by caching previous compiles and detecting when
-	  the same compile is being done again.
-
-	  http://ccache.samba.org
-
-
diff --git a/toolchain/ccache/ccache.mk b/toolchain/ccache/ccache.mk
deleted file mode 100644
index 8e414b2..0000000
--- a/toolchain/ccache/ccache.mk
+++ /dev/null
@@ -1,196 +0,0 @@
-#############################################################
-#
-# build ccache to make recompiles faster on the build system
-#
-#############################################################
-CCACHE_VER:=2.4
-CCACHE_SITE:=http://samba.org/ftp/ccache
-CCACHE_SOURCE:=ccache-$(CCACHE_VER).tar.gz
-CCACHE_DIR1:=$(TOOLCHAIN_DIR)/ccache-$(CCACHE_VER)
-CCACHE_DIR2:=$(BUILD_DIR)/ccache-$(CCACHE_VER)
-CCACHE_CAT:=$(ZCAT)
-CCACHE_BINARY:=ccache
-CCACHE_TARGET_BINARY:=usr/bin/ccache
-
-$(DL_DIR)/$(CCACHE_SOURCE):
-	$(call DOWNLOAD,$(CCACHE_SITE),$(CCACHE_SOURCE))
-
-$(CCACHE_DIR1)/.unpacked: $(DL_DIR)/$(CCACHE_SOURCE)
-	$(CCACHE_CAT) $(DL_DIR)/$(CCACHE_SOURCE) | tar -C $(TOOLCHAIN_DIR) $(TAR_OPTIONS) -
-	touch $@
-
-$(CCACHE_DIR1)/.patched: $(CCACHE_DIR1)/.unpacked
-	# WARNING - this will break if the toolchain is moved.
-	# Should probably patch things to use a relative path.
-	$(SED) "s,getenv(\"CCACHE_PATH\"),\"$(STAGING_DIR)/usr/bin-ccache\",g" \
-		$(CCACHE_DIR1)/execute.c
-	# WARNING - this will break if the toolchain build dir is deleted
-	#	when using the default cache dir location.
-	$(SED) "s,getenv(\"CCACHE_DIR\"),\"$(BR2_CCACHE_DIR)\",g" \
-		$(CCACHE_DIR1)/ccache.c
-	mkdir -p $(CCACHE_DIR1)/cache
-	$(CONFIG_UPDATE) $(@D)
-	touch $@
-
-$(CCACHE_DIR1)/.configured: $(CCACHE_DIR1)/.patched
-	mkdir -p $(CCACHE_DIR1)
-	(cd $(CCACHE_DIR1); rm -rf config.cache; \
-		CC="$(HOSTCC)" \
-		$(CCACHE_DIR1)/configure $(QUIET) \
-		--target=$(GNU_HOST_NAME) \
-		--host=$(GNU_HOST_NAME) \
-		--build=$(GNU_HOST_NAME) \
-		--prefix=/usr \
-		$(QUIET) \
-	)
-	touch $@
-
-$(CCACHE_DIR1)/$(CCACHE_BINARY): $(CCACHE_DIR1)/.configured
-	$(MAKE) CC="$(HOSTCC)" -C $(CCACHE_DIR1)
-
-$(STAGING_DIR)/$(CCACHE_TARGET_BINARY): $(CCACHE_DIR1)/$(CCACHE_BINARY)
-	mkdir -p $(STAGING_DIR)/usr/bin
-	cp $(CCACHE_DIR1)/ccache $(STAGING_DIR)/usr/bin
-	# Keep the actual toolchain binaries in a directory at the same level.
-	# Otherwise, relative paths for include dirs break.
-	mkdir -p $(STAGING_DIR)/usr/bin-ccache
-	(cd $(STAGING_DIR)/usr/bin-ccache; \
-		ln -fs $(REAL_GNU_TARGET_NAME)-gcc $(GNU_TARGET_NAME)-gcc; \
-		ln -fs $(REAL_GNU_TARGET_NAME)-gcc $(GNU_TARGET_NAME)-cc; \
-		ln -fs $(REAL_GNU_TARGET_NAME)-gcc $(REAL_GNU_TARGET_NAME)-cc; \
-	)
-	[ -f $(STAGING_DIR)/usr/bin/$(REAL_GNU_TARGET_NAME)-gcc ] && \
-		mv $(STAGING_DIR)/usr/bin/$(REAL_GNU_TARGET_NAME)-gcc \
-			$(STAGING_DIR)/usr/bin-ccache/
-	(cd $(STAGING_DIR)/usr/bin; \
-		ln -fs ccache $(GNU_TARGET_NAME)-cc; \
-		ln -fs ccache $(GNU_TARGET_NAME)-gcc; \
-		ln -fs ccache $(REAL_GNU_TARGET_NAME)-cc; \
-		ln -fs ccache $(REAL_GNU_TARGET_NAME)-gcc; \
-	)
-ifeq ($(BR2_INSTALL_LIBSTDCPP),y)
-	[ -f $(STAGING_DIR)/usr/bin/$(REAL_GNU_TARGET_NAME)-c++ ] && \
-		mv $(STAGING_DIR)/usr/bin/$(REAL_GNU_TARGET_NAME)-c++ \
-			$(STAGING_DIR)/usr/bin-ccache/
-	[ -f $(STAGING_DIR)/usr/bin/$(REAL_GNU_TARGET_NAME)-g++ ] && \
-		mv $(STAGING_DIR)/usr/bin/$(REAL_GNU_TARGET_NAME)-g++ \
-			$(STAGING_DIR)/usr/bin-ccache/
-	(cd $(STAGING_DIR)/usr/bin; \
-		ln -fs ccache $(GNU_TARGET_NAME)-c++; \
-		ln -fs ccache $(GNU_TARGET_NAME)-g++;\
-		ln -fs ccache $(REAL_GNU_TARGET_NAME)-c++; \
-		ln -fs ccache $(REAL_GNU_TARGET_NAME)-g++; \
-	)
-	(cd $(STAGING_DIR)/usr/bin-ccache; \
-		ln -fs $(REAL_GNU_TARGET_NAME)-c++ $(GNU_TARGET_NAME)-c++; \
-		ln -fs $(REAL_GNU_TARGET_NAME)-g++ $(GNU_TARGET_NAME)-g++; \
-	)
-endif
-
-ccache: gcc $(STAGING_DIR)/$(CCACHE_TARGET_BINARY)
-
-ccache-clean:
-	rm -rf $(STAGING_DIR)/usr/bin/$(GNU_TARGET_NAME)-cc
-	rm -rf $(STAGING_DIR)/usr/bin/$(GNU_TARGET_NAME)-gcc
-	rm -rf $(STAGING_DIR)/usr/bin/$(REAL_GNU_TARGET_NAME)-cc
-	rm -rf $(STAGING_DIR)/usr/bin/$(REAL_GNU_TARGET_NAME)-gcc
-	if [ -f $(STAGING_DIR)/usr/bin-ccache/$(REAL_GNU_TARGET_NAME)-gcc ]; then \
-		mv $(STAGING_DIR)/usr/bin-ccache/$(REAL_GNU_TARGET_NAME)-gcc \
-			$(STAGING_DIR)/usr/bin/; \
-		(cd $(STAGING_DIR)/usr/bin; \
-			ln -fs $(REAL_GNU_TARGET_NAME)-gcc \
-				$(REAL_GNU_TARGET_NAME)-cc; \
-			ln -fs $(REAL_GNU_TARGET_NAME)-gcc \
-				$(GNU_TARGET_NAME)-cc; \
-			ln -fs $(REAL_GNU_TARGET_NAME)-gcc \
-				$(GNU_TARGET_NAME)-gcc; \
-		); \
-	fi
-	if [ -f $(STAGING_DIR)/usr/bin-ccache/$(REAL_GNU_TARGET_NAME)-c++ ]; then \
-		rm -f $(STAGING_DIR)/usr/bin/$(REAL_GNU_TARGET_NAME)-c++; \
-		mv $(STAGING_DIR)/usr/bin-ccache/$(REAL_GNU_TARGET_NAME)-c++ \
-			$(STAGING_DIR)/usr/bin/; \
-	fi
-	if [ -f $(STAGING_DIR)/usr/bin-ccache/$(REAL_GNU_TARGET_NAME)-g++ ]; then \
-		rm -f $(STAGING_DIR)/usr/bin/$(REAL_GNU_TARGET_NAME)-g++; \
-		mv $(STAGING_DIR)/usr/bin-ccache/$(REAL_GNU_TARGET_NAME)-g++ \
-			$(STAGING_DIR)/usr/bin/; \
-	fi
-	rm -rf $(STAGING_DIR)/usr/bin-ccache/*
-	(cd $(STAGING_DIR)/usr/bin; \
-		ln -fs $(REAL_GNU_TARGET_NAME)-g++ $(GNU_TARGET_NAME)-c++; \
-		ln -fs $(REAL_GNU_TARGET_NAME)-g++ $(GNU_TARGET_NAME)-g++; \
-		ln -fs $(REAL_GNU_TARGET_NAME)-g++ $(REAL_GNU_TARGET_NAME)-c++; \
-	)
-	-$(MAKE) -C $(CCACHE_DIR1) clean
-
-ccache-dirclean:
-	rm -rf $(CCACHE_DIR1)
-
-
-
-
-#############################################################
-#
-# build ccache for use on the target system
-#
-#############################################################
-
-$(CCACHE_DIR2)/.unpacked: $(DL_DIR)/$(CCACHE_SOURCE)
-	$(CCACHE_CAT) $(DL_DIR)/$(CCACHE_SOURCE) | tar -C $(BUILD_DIR) $(TAR_OPTIONS) -
-	touch $@
-
-$(CCACHE_DIR2)/.patched: $(CCACHE_DIR2)/.unpacked
-	touch $@
-
-$(CCACHE_DIR2)/.configured: $(CCACHE_DIR2)/.patched
-	mkdir -p $(CCACHE_DIR2)
-	(cd $(CCACHE_DIR2); rm -rf config.cache; \
-		$(TARGET_CONFIGURE_OPTS) \
-		$(CCACHE_DIR2)/configure $(QUIET) \
-		--target=$(GNU_TARGET_NAME) \
-		--host=$(GNU_TARGET_NAME) \
-		--build=$(GNU_HOST_NAME) \
-		--prefix=/usr \
-		--sysconfdir=/etc \
-		$(DISABLE_NLS) \
-		$(QUIET) \
-	)
-	touch $@
-
-$(CCACHE_DIR2)/$(CCACHE_BINARY): $(CCACHE_DIR2)/.configured
-	$(MAKE) -C $(CCACHE_DIR2) CFLAGS="$(TARGET_CFLAGS)"
-
-$(TARGET_DIR)/$(CCACHE_TARGET_BINARY): $(CCACHE_DIR2)/$(CCACHE_BINARY)
-	$(MAKE) DESTDIR=$(TARGET_DIR) -C $(CCACHE_DIR2) install
-	# put a bunch of symlinks into /bin, since that is earlier
-	# in the default PATH than /usr/bin where gcc lives
-	(cd $(TARGET_DIR)/bin; \
-		ln -fs /usr/bin/ccache cc; \
-		ln -fs /usr/bin/ccache gcc; \
-		ln -fs /usr/bin/ccache c++; \
-		ln -fs /usr/bin/ccache g++; \
-	)
-
-ccache_target: uclibc $(TARGET_DIR)/$(CCACHE_TARGET_BINARY)
-
-ccache_target-sources: $(DL_DIR)/$(CCACHE_SOURCE)
-
-ccache_target-clean:
-	rm -f $(TARGET_DIR)/$(CCACHE_TARGET_BINARY) \
-	      $(addprefix $(TARGET_DIR)/bin/,cc gcc c++ g++)
-	-$(MAKE) -C $(CCACHE_DIR2) clean
-
-ccache_target-dirclean:
-	rm -rf $(CCACHE_DIR2)
-#############################################################
-#
-# Toplevel Makefile options
-#
-#############################################################
-ifeq ($(BR2_CCACHE),y)
-TARGETS+=ccache
-endif
-ifeq ($(BR2_PACKAGE_CCACHE_TARGET),y)
-TARGETS+=ccache_target
-endif
diff --git a/toolchain/dependencies/dependencies.sh b/toolchain/dependencies/dependencies.sh
index ee21b37..1aa013f 100755
--- a/toolchain/dependencies/dependencies.sh
+++ b/toolchain/dependencies/dependencies.sh
@@ -79,7 +79,7 @@ if [ $MAKE_MAJOR -lt 3 ] || [ $MAKE_MAJOR -eq 3 -a $MAKE_MINOR -lt 81 ] ; then
 fi;
 
 # Check host gcc
-COMPILER=$(which $HOSTCC 2> /dev/null)
+COMPILER=$(which $HOSTCC_NOCCACHE 2> /dev/null)
 if [ -z "$COMPILER" ] ; then
 	COMPILER=$(which cc 2> /dev/null)
 fi;
diff --git a/toolchain/toolchain-buildroot.mk b/toolchain/toolchain-buildroot.mk
index d879697..44044d7 100644
--- a/toolchain/toolchain-buildroot.mk
+++ b/toolchain/toolchain-buildroot.mk
@@ -2,7 +2,6 @@
 
 include toolchain/dependencies/dependencies.mk
 include toolchain/binutils/binutils.mk
-include toolchain/ccache/ccache.mk
 include toolchain/elf2flt/elf2flt.mk
 include toolchain/gcc/gcc-uclibc-4.x.mk
 include toolchain/gdb/gdb.mk
diff --git a/toolchain/toolchain-buildroot/Config.in b/toolchain/toolchain-buildroot/Config.in
index a9dd192..c458251 100644
--- a/toolchain/toolchain-buildroot/Config.in
+++ b/toolchain/toolchain-buildroot/Config.in
@@ -5,5 +5,4 @@ source "toolchain/kernel-headers/Config.in"
 source "toolchain/uClibc/Config.in"
 source "toolchain/binutils/Config.in"
 source "toolchain/gcc/Config.in"
-source "toolchain/ccache/Config.in"
 endif
diff --git a/toolchain/toolchain-crosstool-ng.mk b/toolchain/toolchain-crosstool-ng.mk
index 81fce18..1c80300 100644
--- a/toolchain/toolchain-crosstool-ng.mk
+++ b/toolchain/toolchain-crosstool-ng.mk
@@ -3,7 +3,6 @@
 # Explicit ordering:
 include toolchain/helpers.mk
 include toolchain/binutils/binutils.mk
-include toolchain/ccache/ccache.mk
 include toolchain/dependencies/dependencies.mk
 include toolchain/elf2flt/elf2flt.mk
 include toolchain/gcc/gcc-uclibc-4.x.mk
diff --git a/toolchain/toolchain-external.mk b/toolchain/toolchain-external.mk
index 6f1f641..f90b6f1 100644
--- a/toolchain/toolchain-external.mk
+++ b/toolchain/toolchain-external.mk
@@ -2,7 +2,6 @@
 
 include toolchain/helpers.mk
 include toolchain/binutils/binutils.mk
-include toolchain/ccache/ccache.mk
 include toolchain/dependencies/dependencies.mk
 include toolchain/elf2flt/elf2flt.mk
 include toolchain/gcc/gcc-uclibc-4.x.mk
diff --git a/toolchain/toolchain-external/ext-tool.mk b/toolchain/toolchain-external/ext-tool.mk
index 7e4645d..53ad636 100644
--- a/toolchain/toolchain-external/ext-tool.mk
+++ b/toolchain/toolchain-external/ext-tool.mk
@@ -62,7 +62,7 @@ endif # ! no threads
 # could select a multilib variant as we want the "main" sysroot, which
 # contains all variants of the C library in the case of multilib
 # toolchains.
-TARGET_CC_NO_SYSROOT=$(filter-out --sysroot=%,$(TARGET_CC))
+TARGET_CC_NO_SYSROOT=$(filter-out --sysroot=%,$(TARGET_CC_NOCCACHE))
 SYSROOT_DIR=$(shell $(TARGET_CC_NO_SYSROOT) -print-sysroot 2>/dev/null)
 ifeq ($(SYSROOT_DIR),)
 SYSROOT_DIR=$(shell readlink -f $$(LANG=C $(TARGET_CC_NO_SYSROOT) -print-file-name=libc.a) |sed -r -e 's:usr/lib/libc\.a::;')
-- 
1.7.0.4

[Buildroot] [PATCH 1/1] ccache: rework ccache management

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

 Thomas>  * ccache is now a normal package (both for the host and the target).

Besides the general comments, I have a few specific ones as well:

 Thomas> +++ b/docs/buildroot.html
 Thomas> @@ -681,6 +682,31 @@ endif
 Thomas>      Build options -&gt; Toolchain and header file location</code> options.
 Thomas>      This could be useful if the toolchain must be shared with other users.</p>
 
 Thomas> +    <h2 id="ccache-support">Using <code>ccache</code> in Buildroot</h2>
 Thomas> +
 Thomas> +    <p><a href="http://ccache.samba.org">ccache</a> is a compiler
 Thomas> +    cache. It stores the object files resulting from each compilation
 Thomas> +    process, and is able to skip future compilation of the same source
 Thomas> +    file (with same compiler and same arguments) by using the
 Thomas> +    pre-existing object files. When doing almost identical builds from
 Thomas> +    scratch a number of times, it can divide the build time by a
 Thomas> +    factor of two.</p>

The improvement depends quite a lot on the configuration / use case, so
I think we should simpluy write something like 'it can significantly
speed up the build process.' rather than specifically saying x2.

 Thomas> +++ b/package/ccache/Config.in
 Thomas> @@ -0,0 +1,5 @@
 Thomas> +config BR2_PACKAGE_CCACHE
 Thomas> +	bool "ccache"
 Thomas> +	help
 Thomas> +	  Compiler Cache
 Thomas> +	  http://samba.org/ftp/ccache
 Thomas> \ No newline at end of file

You forgot the missing newline here and for the .mk. We normally have an
empty line before the website. The official website is afaik also
http://ccache.samba.org, rather than this download URL.

Doesn't ccache need zlib?

-- 
Bye, Peter Korsgaard

[Buildroot] [pull request v2] Pull request for branch for-2011.02/fix-ccache-support

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 Thomas> Hello,
 Thomas> [ Sorry, previous pull request had some mistakes. ]

 Thomas> Here is a patch that reworks the ccache support to make it work and
 Thomas> make it useful. It is now a "Build options" rather than a "Toolchain
 Thomas> option". I have tested it with both internal toolchain and external
 Thomas> toolchain. The principle is simply to build ccache as early as
 Thomas> possible in the build process, and then use it to cache host and
 Thomas> target compiled files.

Thanks, looks interesting.

 Thomas> The ccache cache is kept in ~/.buildroot-ccache/, so that it can be
 Thomas> shared between different builds.

Why here and not in the default ~/.ccache? Is the ~/.ccache directory
content ccache-version dependent?

 Thomas> I'm not using the symlink technique that was used in the past, but
 Thomas> rather prefixing all invocation to host gcc and target gcc with
 Thomas> "$(HOST_DIR)/usr/bin/ccache".

Is that working everywhere? I remember we had some problems back when we
added --sysroot= to TARGET_CC. The qt package in particular is stripping
the --sysroot argument because of this.

-- 
Bye, Peter Korsgaard

[Buildroot] [pull request v2] Pull request for branch for-2011.02/fix-ccache-support

[Bjørn Forsman]

Hi,

On 5 December 2010 23:40, Peter Korsgaard <jacmet@uclibc.org> wrote:
>>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

[snip]

> ?Thomas> I'm not using the symlink technique that was used in the past, but
> ?Thomas> rather prefixing all invocation to host gcc and target gcc with
> ?Thomas> "$(HOST_DIR)/usr/bin/ccache".
>
> Is that working everywhere? I remember we had some problems back when we
> added --sysroot= to TARGET_CC. The qt package in particular is stripping
> the --sysroot argument because of this.

The cdrkit package also strips off --sysroot because its build system
CMake doesn't like it when --sysroot is part of the compiler path.

Actually, I am currently working on a CMAKETARGETS macro (similar to
AUTOTARGETS) and decided to write a patch that takes --sysroot out of
TARGET_CC and move it to the {C,CXX}FLAGS variables, to make CMake
happy. I've been meaning to post the patch series but I don't feel
it's ready to be merged yet. How do the Buildroot developers feel
about

a) moving --sysroot from TARGET_CC to {C,CXX}FLAGS

b) add CMAKETARGETS infrastructure

If this sounds interesting I can post the patches for review.

Best regards,
Bj?rn Forsman

[Buildroot] [pull request v2] Pull request for branch for-2011.02/fix-ccache-support

[Peter Korsgaard]

>>>>> "Bj?rn" == Bj?rn Forsman <bjorn.forsman@gmail.com> writes:

Hi,

 Bj?rn> Actually, I am currently working on a CMAKETARGETS macro (similar to
 Bj?rn> AUTOTARGETS) and decided to write a patch that takes --sysroot out of
 Bj?rn> TARGET_CC and move it to the {C,CXX}FLAGS variables, to make CMake
 Bj?rn> happy. I've been meaning to post the patch series but I don't feel
 Bj?rn> it's ready to be merged yet. How do the Buildroot developers feel
 Bj?rn> about

 Bj?rn> a) moving --sysroot from TARGET_CC to {C,CXX}FLAGS

It afaik got moved in the first place to fix issues with external
toolchains (E.G. where sysroot is important) and various configure
scripts not using CFLAGS, so I don't think this will work.

Perhaps we should simply bite the bullet and add simple shell wrappers
in HOST_DIR / STAGING_DIR for gcc/g++/ld which add --sysroot when we're
using an external toolchain, and then get rid of --sysroot in TARGET_CC.

 Bj?rn> b) add CMAKETARGETS infrastructure

Sounds good!

 Bj?rn> If this sounds interesting I can post the patches for review.

Yes, please.

-- 
Bye, Peter Korsgaard

[Buildroot] [pull request v2] Pull request for branch for-2011.02/fix-ccache-support

[Thomas Petazzoni]

On Mon, 06 Dec 2010 16:08:17 +0100
Peter Korsgaard <jacmet@uclibc.org> wrote:

> It afaik got moved in the first place to fix issues with external
> toolchains (E.G. where sysroot is important) and various configure
> scripts not using CFLAGS, so I don't think this will work.

Yes.

> Perhaps we should simply bite the bullet and add simple shell wrappers
> in HOST_DIR / STAGING_DIR for gcc/g++/ld which add --sysroot when
> we're using an external toolchain, and then get rid of --sysroot in
> TARGET_CC.

Maybe. But for the moment, I think doing the little mungling done in
cdrkit.mk that fixes all CMake-based packages is quite reasonable.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [pull request v2] Pull request for branch for-2011.02/fix-ccache-support

[Thomas Petazzoni]

Hello,

On Mon, 6 Dec 2010 10:44:40 +0100
Bj?rn Forsman <bjorn.forsman@gmail.com> wrote:

> The cdrkit package also strips off --sysroot because its build system
> CMake doesn't like it when --sysroot is part of the compiler path.

Yes.

> Actually, I am currently working on a CMAKETARGETS macro (similar to
> AUTOTARGETS) and decided to write a patch that takes --sysroot out of
> TARGET_CC and move it to the {C,CXX}FLAGS variables, to make CMake
> happy. I've been meaning to post the patch series but I don't feel
> it's ready to be merged yet. How do the Buildroot developers feel
> about
> 
> a) moving --sysroot from TARGET_CC to {C,CXX}FLAGS

No, please don't do that :-)

There are other packages that need this. See
efb1d8d3f40281645c178c150d992601c8042c1a.

commit efb1d8d3f40281645c178c150d992601c8042c1a
Author: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Date:   Fri Jun 25 15:04:08 2010 +0200

    Cleanup TARGET_CONFIGURE_OPTS
    
    The definition of CC, LD, GCC, CPP, CXX and FC shouldn't contain the
    CFLAGS/LDFLAGS/CXXFLAGS, those should be passed through the
    appropriate variables.
    
    However, the --sysroot option is a particular case here: it needs to
    be part of the CC/LD/GCC/etc. definitions otherwise libtool strips it
    from the CFLAGS/LDFLAGS.
    
    Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

The current choice has been fairly hard to get to, we made a number of
different tries before finding a solution that seems to work relatively
fine for most packages. Therefore, I'd prefer to keep the
TARGET_CC/TARGET_CFLAGS as they are today, even if the CMAKETARGETS
infrastructure needs to do the same kind of mungling as cdrkit.mk is
doing.

If you really want to be convinced that having --sysroot in
TARGET_CFLAGS causes problems, I can probably trigger a particular
configuration to exhibit the issue.

> b) add CMAKETARGETS infrastructure

Yes, that would be great. We're likely to see more CMake-based packages
in the future.

Note that some people in the Armadeus project are working on a
different thing related to CMake and Buildroot: generate automatically
a CMake toolchain description file, to make it easy to build
CMake-based applications outside of Buildroot, using the toolchain and
staging directory of Buildroot.

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [pull request v2] Pull request for branch for-2011.02/fix-ccache-support

[Bjørn Forsman]

Hi Thomas,

2010/12/6 Thomas Petazzoni <thomas.petazzoni@free-electrons.com>:
> Hello,
>
> On Mon, 6 Dec 2010 10:44:40 +0100
> Bj?rn Forsman <bjorn.forsman@gmail.com> wrote:
>> Actually, I am currently working on a CMAKETARGETS macro (similar to
>> AUTOTARGETS) and decided to write a patch that takes --sysroot out of
>> TARGET_CC and move it to the {C,CXX}FLAGS variables, to make CMake
>> happy. I've been meaning to post the patch series but I don't feel
>> it's ready to be merged yet. How do the Buildroot developers feel
>> about
>>
>> a) moving --sysroot from TARGET_CC to {C,CXX}FLAGS
>
> No, please don't do that :-)
>
> There are other packages that need this. See
> efb1d8d3f40281645c178c150d992601c8042c1a.
>
> commit efb1d8d3f40281645c178c150d992601c8042c1a
> Author: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Date: ? Fri Jun 25 15:04:08 2010 +0200
>
> ? ?Cleanup TARGET_CONFIGURE_OPTS
>
> ? ?The definition of CC, LD, GCC, CPP, CXX and FC shouldn't contain the
> ? ?CFLAGS/LDFLAGS/CXXFLAGS, those should be passed through the
> ? ?appropriate variables.
>
> ? ?However, the --sysroot option is a particular case here: it needs to
> ? ?be part of the CC/LD/GCC/etc. definitions otherwise libtool strips it
> ? ?from the CFLAGS/LDFLAGS.
>
> ? ?Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>
> The current choice has been fairly hard to get to, we made a number of
> different tries before finding a solution that seems to work relatively
> fine for most packages. Therefore, I'd prefer to keep the
> TARGET_CC/TARGET_CFLAGS as they are today, even if the CMAKETARGETS
> infrastructure needs to do the same kind of mungling as cdrkit.mk is
> doing.
>
> If you really want to be convinced that having --sysroot in
> TARGET_CFLAGS causes problems, I can probably trigger a particular
> configuration to exhibit the issue.

Are many packages affected? If many, I'll just take your word for it and
leave TARGET_CC and xFLAGS variables be, but if there is only a few
packages affected, maybe we can fix them instead?

>> b) add CMAKETARGETS infrastructure
>
> Yes, that would be great. We're likely to see more CMake-based packages
> in the future.
>
> Note that some people in the Armadeus project are working on a
> different thing related to CMake and Buildroot: generate automatically
> a CMake toolchain description file, to make it easy to build
> CMake-based applications outside of Buildroot, using the toolchain and
> staging directory of Buildroot.

Yes, I'd like to get this CMake toolchain file feature into Buildroot as
well. How about appending a toolchain file target to $(2)_DEPENDENCIES
in Makefile.cmake.in? And place the toolchain file in output/?

Best regards,
Bj?rn Forsman

[Buildroot] [pull request v2] Pull request for branch for-2011.02/fix-ccache-support

[Thomas Petazzoni]

Hello,

On Tue, 7 Dec 2010 01:04:41 +0100
Bj?rn Forsman <bjorn.forsman@gmail.com> wrote:

> Are many packages affected?

I don't remember.

> If many, I'll just take your word for it and leave TARGET_CC and
> xFLAGS variables be, but if there is only a few packages affected,
> maybe we can fix them instead?

The problem was related to libtool. And unfortunately "fix problem" and
"libtool" in the same sentence is fairly hard to have :-)

Honestly, I've spent quite a bit of time at this moment trying to find
the solution for the --sysroot option that would work with most
packages, without requiring to do horrible things. As things are
working today, I'm not really willing to spend more time on this, but
if others want, I of course won't prevent them from doing so. I'll just
ask for some heavy testing to be done before accepting changes on
this :-)

> Yes, I'd like to get this CMake toolchain file feature into Buildroot
> as well. How about appending a toolchain file target to
> $(2)_DEPENDENCIES in Makefile.cmake.in?

Hum, why ? Do you want to use the toolchain file even to build the
CMake-based packages in Buildroot ? Or just provide it for users to
build external softwares outside of Buildroot ?

> And place the toolchain file in output/?

As it should be part of the SDK, it should probably be located in
$(HOST_DIR) somewhere. But I'm not really clear yet on what the
organization of the SDK should be. In the mean time, just assume you
can put it in $(O), moving it to some other place won't be hard.

Thanks!

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [pull request v2] Pull request for branch for-2011.02/fix-ccache-support

[Bjørn Forsman]

Hi Thomas,

2010/12/7 Thomas Petazzoni <thomas.petazzoni@free-electrons.com>:
> Hello,
>
> On Tue, 7 Dec 2010 01:04:41 +0100
> Bj?rn Forsman <bjorn.forsman@gmail.com> wrote:
>
>> Are many packages affected?
>
> I don't remember.
>
>> If many, I'll just take your word for it and leave TARGET_CC and
>> xFLAGS variables be, but if there is only a few packages affected,
>> maybe we can fix them instead?
>
> The problem was related to libtool. And unfortunately "fix problem" and
> "libtool" in the same sentence is fairly hard to have :-)
>
> Honestly, I've spent quite a bit of time at this moment trying to find
> the solution for the --sysroot option that would work with most
> packages, without requiring to do horrible things. As things are
> working today, I'm not really willing to spend more time on this, but
> if others want, I of course won't prevent them from doing so. I'll just
> ask for some heavy testing to be done before accepting changes on
> this :-)

Ok. I'll just leave --sysroot the way it is.

>> Yes, I'd like to get this CMake toolchain file feature into Buildroot
>> as well. How about appending a toolchain file target to
>> $(2)_DEPENDENCIES in Makefile.cmake.in?
>
> Hum, why ? Do you want to use the toolchain file even to build the
> CMake-based packages in Buildroot ? Or just provide it for users to
> build external softwares outside of Buildroot ?

I haven't really decided whether to use the toolchain file internally or not.
If using it internally reduces code duplication, then maybe yes :-)
And adding the target to $(2)_DEPENDENCIES in Makefile.cmake.in
was just the first place that I thought of.

I guess the toolchain file should be generated earlier, and be generated
even if the user doesn't select any CMake packages inside Buildroot,
but I don't know where the right 'hook' in Buildroot would be for that.
Maybe when the .config file is written?

>> And place the toolchain file in output/?
>
> As it should be part of the SDK, it should probably be located in
> $(HOST_DIR) somewhere. But I'm not really clear yet on what the
> organization of the SDK should be. In the mean time, just assume you
> can put it in $(O), moving it to some other place won't be hard.

Ok. So until the SDK is ready I'll just put the toolchain file in $(O) and
don't use it internally so it will be easy to change later.

I'll post the patch series once I've rebased.

Best regards,
Bj?rn Forsman

[Buildroot] [pull request v2] Pull request for branch for-2011.02/fix-ccache-support

[Thomas Petazzoni]

Hello,

On Sun, 05 Dec 2010 23:40:14 +0100
Peter Korsgaard <jacmet@uclibc.org> wrote:

>  Thomas> The ccache cache is kept in ~/.buildroot-ccache/, so that it can be
>  Thomas> shared between different builds.
> 
> Why here and not in the default ~/.ccache? Is the ~/.ccache directory
> content ccache-version dependent?

I don't know how ccache-version dependent are the contents of the cache
directory. In the previous ccache integration, the cache was inside
Buildroot build directory (which I thought was stupid since you then
couldn't share the cache between different Buildroot builds), but I
kept the idea of having a Buildroot-specific location for the cache.

I don't have strong opinion/arguments on that, so if you say we should
use the default cache location, I'll just do it.

> Is that working everywhere?

Everywhere I don't know, I obviously haven't compiled all our packages
with ccache support enabled.

> I remember we had some problems back when we added --sysroot= to TARGET_CC.
> The qt package in particular is stripping the --sysroot argument because
> of this.

I just tried Qt, and it built fine. It does not use ccache for the
parts compiled for the host (since we don't tell Qt about $(HOSTCXX)),
but it definitely uses the cache for parts compiled for the target.

Here are the results for a Busybox + Qt build, with a CodeSourcery glibc
ARM external toolchain.

Cold cache
==========

real    7m41.319s
user    37m53.620s
sys     1m31.660s

Hot cache
=========

real    3m4.738s
user    5m34.480s
sys     0m36.160s

And in the hot cache case, a quite significant time is spent rebuilding
the host tools, as ccache is not used there. So we could probably speed
this up a bit further.

I am not strongly advocating the usage of the "ccache /path/to/gcc"
solution compared to the symbolic link solutions, but if the first
solution works, I find it better because: 1) it's nicer and 2) it's
easier to implement with external toolchains.

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [pull request v2] Pull request for branch for-2011.02/fix-ccache-support

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

 >> Why here and not in the default ~/.ccache? Is the ~/.ccache
 >> directory content ccache-version dependent?

 Thomas> I don't know how ccache-version dependent are the contents of
 Thomas> the cache directory. In the previous ccache integration, the
 Thomas> cache was inside Buildroot build directory (which I thought was
 Thomas> stupid since you then couldn't share the cache between
 Thomas> different Buildroot builds), but I kept the idea of having a
 Thomas> Buildroot-specific location for the cache.

 Thomas> I don't have strong opinion/arguments on that, so if you say we
 Thomas> should use the default cache location, I'll just do it.

Ok, me neither - Just keep it where it is. It just wasn't clear to me
from the commit message if there was any deeper reason behind it.

 >> I remember we had some problems back when we added --sysroot= to TARGET_CC.
 >> The qt package in particular is stripping the --sysroot argument because
 >> of this.

 Thomas> I just tried Qt, and it built fine. It does not use ccache for the
 Thomas> parts compiled for the host (since we don't tell Qt about $(HOSTCXX)),
 Thomas> but it definitely uses the cache for parts compiled for the target.

Ok, thanks.

 Thomas> Here are the results for a Busybox + Qt build, with a
 Thomas> CodeSourcery glibc ARM external toolchain.

 Thomas> Cold cache
 Thomas> ==========

 Thomas> real    7m41.319s
 Thomas> user    37m53.620s
 Thomas> sys     1m31.660s

 Thomas> Hot cache
 Thomas> =========

 Thomas> real    3m4.738s
 Thomas> user    5m34.480s
 Thomas> sys     0m36.160s

Nice!

 Thomas> I am not strongly advocating the usage of the "ccache /path/to/gcc"
 Thomas> solution compared to the symbolic link solutions, but if the first
 Thomas> solution works, I find it better because: 1) it's nicer and 2) it's
 Thomas> easier to implement with external toolchains.

Ok, fine by me if it works.

Care to send a new pull request with the few issues I pointed out fixed,
then I'll commit it?

-- 
Bye, Peter Korsgaard