* [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support
@ 2015-10-04 12:28 Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 01/18] toolchain-external: move wrapper to toolchain directory Arnout Vandecappelle
` (18 more replies)
0 siblings, 19 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
This series introduces two features that were discussed on the list and
in the Buildroot Summer Camp: a wrapper for the internal toolchain, and
fixing ccache. See below why the two are combined in one series.
[Note: to make review easier, all the potentially contentious stuff
from the individual patches is repeated here. So for the first few
iterations, it's probably enough to reply to just this e-mail.]
The idea of a wrapper for the internal toolchain has been floating
around for a while now. It allows us to make sure that
BR2_TARGET_OPTIMIZATION and BR2_TARGET_LDFLAGS are passed down to the
compiler and linker even if the package build system doesn't use CFLAGS
and LDFLAGS. This is something which is not tested in the autobuilders
(they never set these options) so probably there are packages out there
that are not built with these flags.
A second reason for the wrapper is to be able to use the unsafe path
check also for internal toolchains without patching gcc.
More recently, and additional motivation for an internal toolchain
wrapper was discovered: it makes the support for a per-package staging
directory significantly simpler. See the discussion in [1][2].
And finally, the wrapper makes the ccache handling simpler, as
explained below.
To create the wrapper for the internal toolchain, this series refactors
as much as possible with the external toolchain. Therefore, the first
few patch moves things around a bit as an enabler of the internal
toolchain wrapper. The same patch also does some related cleanups.
The second patch adds the toolchain wrapper for the internal toolchain.
It uses the same toolchain-wrapper.c as the external toolchain. The
wrapper is created both for gcc-initial and gcc-final. It is rebuilt
for gcc-final, which is not exactly
The real gcc, g++, etc. executables are moved to gcc.real etc. and a
symlink is created to toolchain-wrapper. This requires the wrapper code
to be extended with an additional BR_CROSS_PATH_SUFFIX to find the real
executable. I could have reused the same approach of the external
toolchain, installing in /opt, but that would require copying a bunch
files around as well and overall would be much more invasive, so I
opted for this simpler change.
The creation of the wrapper symlinks is covered by a fairly complex
case-condition which is similar to the external toolchain wrapper. I
didn't refactor these two commands, because there are suble differences
that would make the refactored code very complicated.
The third patch implements the second use case: it removes the
now-redundant gcc patches that add the unsafe path check. This check is
now handled by the toolchain wrapper. Note that the patches for
binutils are still needed, since ld isn't wrapped. We actually should
also wrap cpp and ld to cover the first use case.
The first use case (passing TARGET_CFLAGS through the wrapper) isn't
implemented yet because it is a bit complicated, because some packages
do _not_ want TARGET_CFLAGS, e.g. gcc and linux. So some infrastructure
needs to be added that allows overriding TARGET_CFLAGS.
The third use case should be handled by Fabio in his per-package
staging series.
The fourth use case brings us to the ccache problem.
The problem with our current ccache is that we disable hashing of the
compiler executable itself. If we don't do that, using ccache is
pointless since we always rebuild the compiler (for the external
toolchain we rebuild the wrapper) so the cache would always miss.
But in the current situation, if a user changes the compiler
configuration (which would result in the compiler generating different
object files than before) and does 'make clean all', ccache may in fact
reuse object files from the previous run. This rarely gives problems,
because
(1) the cache expires quite quickly (it's only 1GB by default),
(2) radically changing compiler options will cause cache misses because
different header files are used,
(3) many compiler changes (e.g. changing -mtune) have little practical
effect because the resulting code is usually still compatible,
(4) we currently don't use CCACHE_BASEDIR, and almost all object files
will contain an absolute path (e.g. in debug info), so when
building in a different directory, most of it will miss,
(5) we do mostly build test, and many of the potential problems only
appear at runtime.
Still, when ccache _does_ use the wrong cached object files, the
effects are really weird and hard to debug. Also, we want reproducible
builds and obviously the above makes builds non-reproducible. So we
have a FAQ entry that warns against using ccache and tells the user to
clear the cache in case of problems.
To fix the situation, this series sets the CCACHE_COMPILERCHECK
environment variable to a hash of the compiler options. I already
analyzed the options that should be checked a while ago in [3].
Unfortunately, it's not so simple, since for the host compiler we
should not use the same hash: on the one hand, it's possible that the
host compiler has changed while the target compiler options are still
the same, and on the other hand when we change the target compiler
options, the cached host object files are still valid.
My first attempt was to create a ccache wrapper that sets the
CCACHE_COMPILERCHECK environment variable, and compiles it differently
for host and target (i.e. use ccache-host for host compilation, and
ccache-target for target compilation). However, it was very difficult
to insert the calculation of the target compiler hash correctly in the
dependency chain. The calculation of the hash really belongs to the
toolchain build, but ccache is already built before everything else.
It's much simpler, instead, to use the toolchain wrapper and insert
ccache in there. That's why these two features (internal toolchain
wrapper and ccache fix) are combined in a single series.
The ccache part of the series starts with moving the ccache handling
into the toolchain wrapper. This allows a lot of follow-up patches to
remove ccache handling in individual packages, including some patches
to handle the fact that CC has multiple words. These patches were
tested by doing a ccache-enabled build of the affected packages, then
cleaning them, then rebuilding, and checking the evolution of the
ccache stats while building. If the number of cache hits is increasing
significantly faster than the number of cache misses, all is well.
Moving the ccache handling into the toolchain wrapper has one nasty
side effect: when you change the value of BR2_CCACHE, this will have no
effect unless host-gcc-final is rebuilt. Of course, we don't really
support the incremental build scenario so it's not such a big deal.
Still, I've added a patch (marked RFC) to support this scenario, by
exporting a BR_NO_CCACHE that tells the wrapper to skip ccache. Note
that this works only to turn ccache off, but I think that's the most
important use case.
Once ccache moves to the wrapper, we can fix the compilercheck. All of
that is done in a single commit - I didn't think splitting it up was
really worthwhile. It reverts the default compilercheck to 'mtime' like
it is in upstream, and it adds an option to the toolchain wrapper to
set the CCACHE_COMPILERCHECK environment variable. The hash is
calculated based on the config options passed to gcc and on the sources
and patches.
A final cleanup step, also marked RFC, is to support changing the
output directory. Many of the build commands have a -I argument
pointing somewhere in the staging directory, so when you rebuild in
a different directory, this will mot match and the cache isn't used.
ccache has the CCACHE_BASEDIR potion to deal with that, but it's not
bullet-proof so I added a Kconfig option for it.
It would be really nice if someone else could take over this series,
since it is rather 'deep' stuff and if someone else really gets their
hands dirty on this code, there's a better chance of catching some
issues that I overlooked.
Changelog:
v2: - Rebased on master.
- Updated comments to remove the 'external' from the wrapper.
- Include BR2_GLOBAL_PATCH_DIR in the wrapper hash; also use
$(wildcard ...) for all patches.
- Remove --enable-poison-system-directories from gcc-final
configure.
- Explain in commit message why gcc-initial also needs the wrapper.
[1] http://lists.busybox.net/pipermail/buildroot/2015-March/121005.html
[2] http://lists.busybox.net/pipermail/buildroot/2015-June/131051.html
[3] http://lists.busybox.net/pipermail/buildroot/2015-April/127341.html
Cc: Fabio Porcedda <fabio.porcedda@gmail.com>
Cc: J?r?me Oufella <jerome.oufella@savoirfairelinux.com>
Cc: Danomi Manchego <danomimanchego123@gmail.com>
Cc: K?roly Kasza <kaszak@gmail.com>
Cc: Romain Naour <romain.naour@openwide.fr>
The following changes since commit 0611258aeae3b2e311b1c8fbebd337c9567830fb:
package/freerdp: needs C++ (2015-10-04 14:16:27 +0200)
are available in the git repository at:
https://github.com/arnout/buildroot.git toolchain-wrapper-2
for you to fetch changes up to 11995e8efc5b34d78a45a1ea44bcb9cbdc4e53d3:
[RFC] ccache: support changing the output directory (2015-10-04 13:23:38 +0100)
----------------------------------------------------------------
Arnout Vandecappelle (Essensium/Mind) (18):
toolchain-external: move wrapper to toolchain directory
gcc: use toolchain wrapper
gcc: remove unsafe patch check (poison system dirs) patch
infra: move ccache handling to the toolchain wrapper
perl: Remove ccache handling
imx-lib: remove now-redundant ccache handling
imx-vpu: remove now-redundant ccache handling
linux: remove now-redundant ccache handling
uboot: remove now-redundant ccache handling
barebox: remove now-redundant ccache handling
cryptodev-linux: remove now-redundant fix-ccache-compile patch
qt5base: remove now-redundant ccache handling
package-cmake: remove now-redundant target ccache support
qpid-proton: remove now-redundant ccache handling patch
Makefile.in: remove now-unused TARGET_CC/CXX_NOCCACHE
toolchain-wrapper: support change of BR2_CCACHE
ccache: use mtime for external toolchain, CONF_OPTS for internal toolchain
[RFC] ccache: support changing the output directory
Config.in | 27 +++
Makefile | 2 +
boot/barebox/barebox.mk | 3 +-
boot/uboot/uboot.mk | 2 +-
docs/manual/ccache-support.txt | 20 ++
linux/linux.mk | 2 +-
package/Makefile.in | 8 -
package/ccache/ccache.mk | 5 -
.../cryptodev-linux/0002-fix-ccache-compile.patch | 20 --
package/freescale-imx/imx-lib/imx-lib.mk | 2 +-
package/freescale-imx/imx-vpu/imx-vpu.mk | 2 +-
.../4.7.4/910-gcc-poison-system-directories.patch | 207 -------------------
.../4.8.5/910-gcc-poison-system-directories.patch | 207 -------------------
.../4.9.3/910-gcc-poison-system-directories.patch | 207 -------------------
.../5.2.0/200-gcc-poison-system-directories.patch | 207 -------------------
.../910-gcc-poison-system-directories.patch | 221 ---------------------
package/gcc/gcc-final/gcc-final.mk | 14 +-
package/gcc/gcc-initial/gcc-initial.mk | 4 +
package/gcc/gcc.mk | 65 +++++-
package/perl/perl.mk | 5 +-
package/pkg-cmake.mk | 5 +-
...fix-C-compiler-detection-with-_ARG1-_ARG2.patch | 52 -----
package/qt5/qt5base/0002-mkspecs-files.patch | 6 +-
.../qt5/qt5base/0009-fix-build-with-ccache.patch | 49 -----
package/qt5/qt5base/qt5base.mk | 1 -
support/misc/toolchainfile.cmake.in | 32 +--
toolchain/toolchain-external/toolchain-external.mk | 55 ++---
...ext-toolchain-wrapper.c => toolchain-wrapper.c} | 51 ++++-
toolchain/toolchain-wrapper.mk | 34 ++++
29 files changed, 231 insertions(+), 1284 deletions(-)
delete mode 100644 package/cryptodev-linux/0002-fix-ccache-compile.patch
delete mode 100644 package/gcc/4.7.4/910-gcc-poison-system-directories.patch
delete mode 100644 package/gcc/4.8.5/910-gcc-poison-system-directories.patch
delete mode 100644 package/gcc/4.9.3/910-gcc-poison-system-directories.patch
delete mode 100644 package/gcc/5.2.0/200-gcc-poison-system-directories.patch
delete mode 100644 package/gcc/arc-2015.06/910-gcc-poison-system-directories.patch
delete mode 100644 package/qpid-proton/0001-proton-c-fix-C-compiler-detection-with-_ARG1-_ARG2.patch
delete mode 100644 package/qt5/qt5base/0009-fix-build-with-ccache.patch
rename toolchain/{toolchain-external/ext-toolchain-wrapper.c => toolchain-wrapper.c} (81%)
create mode 100644 toolchain/toolchain-wrapper.mk
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 01/18] toolchain-external: move wrapper to toolchain directory
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 02/18] gcc: use toolchain wrapper Arnout Vandecappelle
` (17 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
The toolchain wrapper will be reused for the internal toolchain, so it
belongs in the toolchain directory. Also, the ext- prefix is removed
from it. The build commands are moved to a new toolchain-wrapper.mk.
The wrapper arguments that are also relevant for the internal toolchain
wrapper are moved to toolchain-wrapper.mk, the rest stays in
toolchain-external.mk.
While we're at it, move the building of the toolchain wrapper to the
build step of toolchain-external. There is no specific reason to do
this, other than that it fits better semantically. Also remove the
MESSAGE call, otherwise we'd see:
>>> toolchain-external undefined Building
>>> toolchain-external undefined Building toolchain wrapper
/usr/bin/gcc ...
Having an extra "Building toolchain wrapper' message is pointless.
The useless condition on $(BR2_TARGET_OPTIMIZATION) is removed. It was
always true because it wasn't qstrip'ped first, so clearly it works
without that condition as well.
Also rewrapped some comments and removed the 'external' reference.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Fabio Porcedda <fabio.porcedda@gmail.com>
Cc: J?r?me Oufella <jerome.oufella@savoirfairelinux.com>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
v2: Remove references to 'external' in comments [Romain]
Rewrap some comments [Arnout]
---
toolchain/toolchain-external/toolchain-external.mk | 55 +++++++---------------
...ext-toolchain-wrapper.c => toolchain-wrapper.c} | 6 +--
toolchain/toolchain-wrapper.mk | 26 ++++++++++
3 files changed, 47 insertions(+), 40 deletions(-)
rename toolchain/{toolchain-external/ext-toolchain-wrapper.c => toolchain-wrapper.c} (96%)
create mode 100644 toolchain/toolchain-wrapper.mk
diff --git a/toolchain/toolchain-external/toolchain-external.mk b/toolchain/toolchain-external/toolchain-external.mk
index 342f13e..6efc013 100644
--- a/toolchain/toolchain-external/toolchain-external.mk
+++ b/toolchain/toolchain-external/toolchain-external.mk
@@ -153,15 +153,14 @@ TOOLCHAIN_EXTERNAL_CROSS = $(TOOLCHAIN_EXTERNAL_BIN)/$(TOOLCHAIN_EXTERNAL_PREFIX
TOOLCHAIN_EXTERNAL_CC = $(TOOLCHAIN_EXTERNAL_CROSS)gcc
TOOLCHAIN_EXTERNAL_CXX = $(TOOLCHAIN_EXTERNAL_CROSS)g++
TOOLCHAIN_EXTERNAL_READELF = $(TOOLCHAIN_EXTERNAL_CROSS)readelf
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS = -DBR_SYSROOT='"$(STAGING_SUBDIR)"'
ifeq ($(filter $(HOST_DIR)/%,$(TOOLCHAIN_EXTERNAL_BIN)),)
# TOOLCHAIN_EXTERNAL_BIN points outside HOST_DIR => absolute path
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += \
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += \
-DBR_CROSS_PATH_ABS='"$(TOOLCHAIN_EXTERNAL_BIN)"'
else
# TOOLCHAIN_EXTERNAL_BIN points inside HOST_DIR => relative path
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += \
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += \
-DBR_CROSS_PATH_REL='"$(TOOLCHAIN_EXTERNAL_BIN:$(HOST_DIR)/%=%)"'
endif
@@ -180,59 +179,54 @@ CC_TARGET_MODE_ := $(call qstrip,$(BR2_GCC_TARGET_MODE))
# to select the right multilib variant
ifeq ($(BR2_x86_64),y)
TOOLCHAIN_EXTERNAL_CFLAGS += -m64
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_64
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_64
endif
ifneq ($(CC_TARGET_ARCH_),)
TOOLCHAIN_EXTERNAL_CFLAGS += -march=$(CC_TARGET_ARCH_)
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_ARCH='"$(CC_TARGET_ARCH_)"'
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_ARCH='"$(CC_TARGET_ARCH_)"'
endif
ifneq ($(CC_TARGET_CPU_),)
TOOLCHAIN_EXTERNAL_CFLAGS += -mcpu=$(CC_TARGET_CPU_)
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_CPU='"$(CC_TARGET_CPU_)"'
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_CPU='"$(CC_TARGET_CPU_)"'
endif
ifneq ($(CC_TARGET_ABI_),)
TOOLCHAIN_EXTERNAL_CFLAGS += -mabi=$(CC_TARGET_ABI_)
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_ABI='"$(CC_TARGET_ABI_)"'
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_ABI='"$(CC_TARGET_ABI_)"'
endif
ifneq ($(CC_TARGET_FPU_),)
TOOLCHAIN_EXTERNAL_CFLAGS += -mfpu=$(CC_TARGET_FPU_)
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_FPU='"$(CC_TARGET_FPU_)"'
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_FPU='"$(CC_TARGET_FPU_)"'
endif
ifneq ($(CC_TARGET_FLOAT_ABI_),)
TOOLCHAIN_EXTERNAL_CFLAGS += -mfloat-abi=$(CC_TARGET_FLOAT_ABI_)
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_FLOAT_ABI='"$(CC_TARGET_FLOAT_ABI_)"'
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_FLOAT_ABI='"$(CC_TARGET_FLOAT_ABI_)"'
endif
ifneq ($(CC_TARGET_MODE_),)
TOOLCHAIN_EXTERNAL_CFLAGS += -m$(CC_TARGET_MODE_)
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_MODE='"$(CC_TARGET_MODE_)"'
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_MODE='"$(CC_TARGET_MODE_)"'
endif
ifeq ($(BR2_BINFMT_FLAT),y)
TOOLCHAIN_EXTERNAL_CFLAGS += -Wl,-elf2flt
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_BINFMT_FLAT
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_BINFMT_FLAT
endif
ifeq ($(BR2_mipsel)$(BR2_mips64el),y)
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_MIPS_TARGET_LITTLE_ENDIAN
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_MIPS_TARGET_LITTLE_ENDIAN
TOOLCHAIN_EXTERNAL_CFLAGS += -EL
endif
ifeq ($(BR2_mips)$(BR2_mips64),y)
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_MIPS_TARGET_BIG_ENDIAN
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_MIPS_TARGET_BIG_ENDIAN
TOOLCHAIN_EXTERNAL_CFLAGS += -EB
endif
ifeq ($(BR2_arceb),y)
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_ARC_TARGET_BIG_ENDIAN
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_ARC_TARGET_BIG_ENDIAN
TOOLCHAIN_EXTERNAL_CFLAGS += -EB
endif
-ifneq ($(BR2_TARGET_OPTIMIZATION),)
+
TOOLCHAIN_EXTERNAL_CFLAGS += $(call qstrip,$(BR2_TARGET_OPTIMIZATION))
-# We create a list like '"-mfoo", "-mbar", "-mbarfoo"' so that each
-# flag is a separate argument when used in execv() by the external
-# toolchain wrapper.
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_ADDITIONAL_CFLAGS='$(foreach f,$(call qstrip,$(BR2_TARGET_OPTIMIZATION)),"$(f)",)'
-endif
ifeq ($(BR2_SOFT_FLOAT),y)
TOOLCHAIN_EXTERNAL_CFLAGS += -msoft-float
-TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_SOFTFLOAT=1
+TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS += -DBR_SOFTFLOAT=1
endif
# The Linaro ARMhf toolchain expects the libraries in
@@ -688,15 +682,6 @@ define TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FLAT
endef
endif
-# We use --hash-style=both to increase the compatibility of
-# the generated binary with older platforms, except for MIPS,
-# where the only acceptable hash style is 'sysv'
-ifeq ($(findstring mips,$(HOSTARCH)),mips)
-TOOLCHAIN_EXTERNAL_WRAPPER_HASH_STYLE = sysv
-else
-TOOLCHAIN_EXTERNAL_WRAPPER_HASH_STYLE = both
-endif
-
# Build toolchain wrapper for preprocessor, C and C++ compiler and setup
# symlinks for everything else. Skip gdb symlink when we are building our
# own gdb to prevent two gdb's in output/host/usr/bin.
@@ -707,8 +692,6 @@ endif
# match the *cc-* pattern. Therefore, an additional case is added for *-ar,
# *-ranlib and *-nm.
define TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER
- $(Q)$(call MESSAGE,"Building ext-toolchain wrapper")
- $(Q)mkdir -p $(HOST_DIR)/usr/bin
$(Q)cd $(HOST_DIR)/usr/bin; \
for i in $(TOOLCHAIN_EXTERNAL_CROSS)*; do \
base=$${i##*/}; \
@@ -717,7 +700,7 @@ define TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER
ln -sf $$(echo $$i | sed 's%^$(HOST_DIR)%../..%') .; \
;; \
*cc|*cc-*|*++|*++-*|*cpp) \
- ln -sf ext-toolchain-wrapper $$base; \
+ ln -sf toolchain-wrapper $$base; \
;; \
*gdb|*gdbtui) \
if test "$(BR2_PACKAGE_HOST_GDB)" != "y"; then \
@@ -729,10 +712,6 @@ define TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER
;; \
esac; \
done
- $(HOSTCC) $(HOST_CFLAGS) $(TOOLCHAIN_EXTERNAL_WRAPPER_ARGS) \
- -s -Wl,--hash-style=$(TOOLCHAIN_EXTERNAL_WRAPPER_HASH_STYLE) \
- toolchain/toolchain-external/ext-toolchain-wrapper.c \
- -o $(HOST_DIR)/usr/bin/ext-toolchain-wrapper
endef
# This sed magic is taken from Linux headers_install.sh script.
@@ -771,6 +750,8 @@ define TOOLCHAIN_EXTERNAL_FIXUP_UCLIBCNG_LDSO
fi
endef
+TOOLCHAIN_EXTERNAL_BUILD_CMDS = $(TOOLCHAIN_BUILD_WRAPPER)
+
define TOOLCHAIN_EXTERNAL_INSTALL_STAGING_CMDS
$(TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS)
$(TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER)
diff --git a/toolchain/toolchain-external/ext-toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
similarity index 96%
rename from toolchain/toolchain-external/ext-toolchain-wrapper.c
rename to toolchain/toolchain-wrapper.c
index b3b02d8..ac40dec 100644
--- a/toolchain/toolchain-external/ext-toolchain-wrapper.c
+++ b/toolchain/toolchain-wrapper.c
@@ -1,7 +1,7 @@
/**
- * Buildroot wrapper for external toolchains. This simply executes the real
- * toolchain with a number of arguments (sysroot/arch/..) hardcoded,
- * to ensure the external toolchain uses the correct configuration.
+ * Buildroot wrapper for toolchains. This simply executes the real toolchain
+ * with a number of arguments (sysroot/arch/..) hardcoded, to ensure the
+ * toolchain uses the correct configuration.
* The hardcoded path arguments are defined relative to the actual location
* of the binary.
*
diff --git a/toolchain/toolchain-wrapper.mk b/toolchain/toolchain-wrapper.mk
new file mode 100644
index 0000000..899947d
--- /dev/null
+++ b/toolchain/toolchain-wrapper.mk
@@ -0,0 +1,26 @@
+# This file contains the definition of the toolchain wrapper build commands
+
+# We use --hash-style=both to increase the compatibility of the generated
+# binary with older platforms, except for MIPS, where the only acceptable
+# hash style is 'sysv'
+ifeq ($(findstring mips,$(HOSTARCH)),mips)
+TOOLCHAIN_WRAPPER_HASH_STYLE = sysv
+else
+TOOLCHAIN_WRAPPER_HASH_STYLE = both
+endif
+
+TOOLCHAIN_WRAPPER_ARGS = $($(PKG)_TOOLCHAIN_WRAPPER_ARGS)
+TOOLCHAIN_WRAPPER_ARGS += -DBR_SYSROOT='"$(STAGING_SUBDIR)"'
+
+# We create a list like '"-mfoo", "-mbar", "-mbarfoo"' so that each flag is a
+# separate argument when used in execv() by the toolchain wrapper.
+TOOLCHAIN_WRAPPER_ARGS += -DBR_ADDITIONAL_CFLAGS='$(foreach f,$(call qstrip,$(BR2_TARGET_OPTIMIZATION)),"$(f)",)'
+
+# For simplicity, build directly into the install location
+define TOOLCHAIN_BUILD_WRAPPER
+ $(Q)mkdir -p $(HOST_DIR)/usr/bin
+ $(HOSTCC) $(HOST_CFLAGS) $(TOOLCHAIN_WRAPPER_ARGS) \
+ -s -Wl,--hash-style=$(TOOLCHAIN_WRAPPER_HASH_STYLE) \
+ toolchain/toolchain-wrapper.c \
+ -o $(HOST_DIR)/usr/bin/toolchain-wrapper
+endef
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 02/18] gcc: use toolchain wrapper
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 01/18] toolchain-external: move wrapper to toolchain directory Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 03/18] gcc: remove unsafe patch check (poison system dirs) patch Arnout Vandecappelle
` (16 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
We have a toolchain wrapper for external toolchain, but it is also
beneficial for internal toolchains, for the following reasons:
1. It can make sure that BR2_TARGET_OPTIMIZATION is passed to the
compiler even if a package's build system doesn't honor CFLAGS.
2. It allows us to do the unsafe path check (i.e. -I/usr/include)
without patching gcc.
3. It makes it simpler to implement building each package with a
separate staging directory (per-package staging).
4. It makes it simpler to implement a compiler hash check for ccache.
The wrapper is reused from the external toolchain. A third CROSS_PATH_
option is added to the wrapper: in this case, the real executable is in
the same directory, with the extension .real.
The creation of the simple symlinks is merged with the creation of the
wrapper symlinks, otherwise part of the -gcc-ar handling logic would
have to be repeated.
The complex case-condition could be refactored with the one for the
external toolchain, but then it becomes even more complex because
they each have special corner cases. For example, the internal
toolchain has to handle *.real to avoid creating an extra indirection
after host-gcc-{final,initial}-rebuild.
Instead of creating the .real files, it would also have been possible
to install the internal toolchain in $(HOST_DIR)/opt, similar to what
we do for the external toolchain. However, then we would also have to
copy things to the sysroot and do more of the magic that the external
toolchain is doing. So keeping it in $(HOST_DIR)/usr/bin is much
simpler.
Note that gcc-initial has to be wrapped as well, because it is used for
building libc and we want to apply the same magic when building libc.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Fabio Porcedda <fabio.porcedda@gmail.com>
Cc: J?r?me Oufella <jerome.oufella@savoirfairelinux.com>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
v2: - Update commit message to explain why gcc-initial has to be wrapped
[Romain]
- Remove redundant newline [Romain]
---
package/gcc/gcc-final/gcc-final.mk | 13 +++++--------
package/gcc/gcc-initial/gcc-initial.mk | 4 ++++
package/gcc/gcc.mk | 34 ++++++++++++++++++++++++++++++++++
toolchain/toolchain-wrapper.c | 4 +++-
4 files changed, 46 insertions(+), 9 deletions(-)
diff --git a/package/gcc/gcc-final/gcc-final.mk b/package/gcc/gcc-final/gcc-final.mk
index 3426ba4..2c16fdf 100644
--- a/package/gcc/gcc-final/gcc-final.mk
+++ b/package/gcc/gcc-final/gcc-final.mk
@@ -96,14 +96,11 @@ endef
HOST_GCC_FINAL_POST_INSTALL_HOOKS += HOST_GCC_FINAL_CREATE_CC_SYMLINKS
-# Create <arch>-linux-<tool> symlinks
-define HOST_GCC_FINAL_CREATE_SIMPLE_SYMLINKS
- (cd $(HOST_DIR)/usr/bin; for i in $(GNU_TARGET_NAME)-*; do \
- ln -snf $$i $(ARCH)-linux$${i##$(GNU_TARGET_NAME)}; \
- done)
-endef
-
-HOST_GCC_FINAL_POST_INSTALL_HOOKS += HOST_GCC_FINAL_CREATE_SIMPLE_SYMLINKS
+HOST_GCC_FINAL_TOOLCHAIN_WRAPPER_ARGS += $(HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS)
+HOST_GCC_FINAL_POST_BUILD_HOOKS += TOOLCHAIN_BUILD_WRAPPER
+# Note: this must be done after CREATE_CC_SYMLINKS, otherwise the
+# -cc symlink to the wrapper is not created.
+HOST_GCC_FINAL_POST_INSTALL_HOOKS += HOST_GCC_INSTALL_WRAPPER_AND_SIMPLE_SYMLINKS
# In gcc 4.7.x, the ARM EABIhf library loader path for (e)glibc was not
# correct, so we create a symbolic link to make things work
diff --git a/package/gcc/gcc-initial/gcc-initial.mk b/package/gcc/gcc-initial/gcc-initial.mk
index 6bb7997..4b03e47 100644
--- a/package/gcc/gcc-initial/gcc-initial.mk
+++ b/package/gcc/gcc-initial/gcc-initial.mk
@@ -62,4 +62,8 @@ HOST_GCC_INITIAL_MAKE_OPTS += all-target-libgcc
HOST_GCC_INITIAL_INSTALL_OPTS += install-target-libgcc
endif
+HOST_GCC_INITIAL_TOOLCHAIN_WRAPPER_ARGS += $(HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS)
+HOST_GCC_INITIAL_POST_BUILD_HOOKS += TOOLCHAIN_BUILD_WRAPPER
+HOST_GCC_INITIAL_POST_INSTALL_HOOKS += HOST_GCC_INSTALL_WRAPPER_AND_SIMPLE_SYMLINKS
+
$(eval $(host-autotools-package))
diff --git a/package/gcc/gcc.mk b/package/gcc/gcc.mk
index 501fcea..b9da398 100644
--- a/package/gcc/gcc.mk
+++ b/package/gcc/gcc.mk
@@ -235,4 +235,38 @@ HOST_GCC_COMMON_CONF_OPTS += \
--with-long-double-128
endif
+HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS += -DBR_CROSS_PATH_SUFFIX='".real"'
+
+# The LTO support in gcc creates wrappers for ar, ranlib and nm which load
+# the lto plugin. These wrappers are called *-gcc-ar, *-gcc-ranlib, and
+# *-gcc-nm and should be used instead of the real programs when -flto is
+# used. However, we should not add the toolchain wrapper for them, and they
+# match the *cc-* pattern. Therefore, an additional case is added for *-ar,
+# *-ranlib and *-nm.
+# Avoid that a .real is symlinked a second time.
+# Also create <arch>-linux-<tool> symlinks.
+define HOST_GCC_INSTALL_WRAPPER_AND_SIMPLE_SYMLINKS
+ $(Q)cd $(HOST_DIR)/usr/bin; \
+ for i in $(GNU_TARGET_NAME)-*; do \
+ case "$$i" in \
+ *.real) \
+ ;; \
+ *-ar|*-ranlib|*-nm) \
+ ln -snf $$i $(ARCH)-linux$${i##$(GNU_TARGET_NAME)}; \
+ ;; \
+ *cc|*cc-*|*++|*++-*|*cpp) \
+ rm -f $$i.real; \
+ mv $$i $$i.real; \
+ ln -sf toolchain-wrapper $$i; \
+ ln -sf toolchain-wrapper $(ARCH)-linux$${i##$(GNU_TARGET_NAME)}; \
+ ln -snf $$i.real $(ARCH)-linux$${i##$(GNU_TARGET_NAME)}.real; \
+ ;; \
+ *) \
+ ln -snf $$i $(ARCH)-linux$${i##$(GNU_TARGET_NAME)}; \
+ ;; \
+ esac; \
+ done
+
+endef
+
include $(sort $(wildcard package/gcc/*/*.mk))
diff --git a/toolchain/toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
index ac40dec..d4d25c7 100644
--- a/toolchain/toolchain-wrapper.c
+++ b/toolchain/toolchain-wrapper.c
@@ -138,8 +138,10 @@ int main(int argc, char **argv)
/* Fill in the relative paths */
#ifdef BR_CROSS_PATH_REL
ret = snprintf(path, sizeof(path), "%s/" BR_CROSS_PATH_REL "/%s", absbasedir, basename);
-#else /* BR_CROSS_PATH_ABS */
+#elif BR_CROSS_PATH_ABS
ret = snprintf(path, sizeof(path), BR_CROSS_PATH_ABS "/%s", basename);
+#else /* BR_CROSS_PATH_SUFFIX */
+ ret = snprintf(path, sizeof(path), "%s/usr/bin/%s" BR_CROSS_PATH_SUFFIX, absbasedir, basename);
#endif
if (ret >= sizeof(path)) {
perror(__FILE__ ": overflow");
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 03/18] gcc: remove unsafe patch check (poison system dirs) patch
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 01/18] toolchain-external: move wrapper to toolchain directory Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 02/18] gcc: use toolchain wrapper Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 04/18] infra: move ccache handling to the toolchain wrapper Arnout Vandecappelle
` (15 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
Now that the calls to gcc always pass through the toolchain wrapper, it
is no longer necessary to patch gcc to support poisoning.
This does have the disadvantage that there is no unsafe path check for
libc, libgcc and libstdc++ (all of these are built before the wrapper
exists). But we can assume that the toolchain components themselves
should be pretty safe.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
Verified that host-gcc-{initial,final}-patch still work for all
toolchains.
Verified that the unsafe path check still works by hacking a package
Makefile with an additional -I/usr/include
v2: Remove --enable-poison-system-directories from gcc-final configure
commands [Romain]
---
.../4.7.4/910-gcc-poison-system-directories.patch | 207 -------------------
.../4.8.5/910-gcc-poison-system-directories.patch | 207 -------------------
.../4.9.3/910-gcc-poison-system-directories.patch | 207 -------------------
.../5.2.0/200-gcc-poison-system-directories.patch | 207 -------------------
.../910-gcc-poison-system-directories.patch | 221 ---------------------
package/gcc/gcc-final/gcc-final.mk | 1 -
6 files changed, 1050 deletions(-)
delete mode 100644 package/gcc/4.7.4/910-gcc-poison-system-directories.patch
delete mode 100644 package/gcc/4.8.5/910-gcc-poison-system-directories.patch
delete mode 100644 package/gcc/4.9.3/910-gcc-poison-system-directories.patch
delete mode 100644 package/gcc/5.2.0/200-gcc-poison-system-directories.patch
delete mode 100644 package/gcc/arc-2015.06/910-gcc-poison-system-directories.patch
diff --git a/package/gcc/4.7.4/910-gcc-poison-system-directories.patch b/package/gcc/4.7.4/910-gcc-poison-system-directories.patch
deleted file mode 100644
index bc2d5c6..0000000
--- a/package/gcc/4.7.4/910-gcc-poison-system-directories.patch
+++ /dev/null
@@ -1,207 +0,0 @@
-From 160397ef3c3331099af028f1b8d3e085b07d88ad Mon Sep 17 00:00:00 2001
-From: Khem Raj <raj.khem@gmail.com>
-Date: Fri, 29 Mar 2013 08:59:00 +0400
-Subject: [PATCH 16/35] gcc: poison-system-directories
-
-Adapted to Buildroot and gcc 4.7.4 by Thomas Petazzoni, especially the
-addition of the BR_COMPILER_PARANOID_UNSAFE_PATH environment variable.
-
-Signed-off-by: Khem Raj <raj.khem@gmail.com>
-Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
-
-Upstream-Status: Inappropriate [distribution: codesourcery]
----
- gcc/Makefile.in | 2 +-
- gcc/common.opt | 4 ++++
- gcc/config.in | 6 ++++++
- gcc/configure | 20 ++++++++++++++++++--
- gcc/configure.ac | 10 ++++++++++
- gcc/doc/invoke.texi | 9 +++++++++
- gcc/gcc.c | 2 ++
- gcc/incpath.c | 19 +++++++++++++++++++
- 8 files changed, 69 insertions(+), 3 deletions(-)
-
-Index: b/gcc/common.opt
-===================================================================
---- a/gcc/common.opt
-+++ b/gcc/common.opt
-@@ -585,6 +585,10 @@
- Common Var(warn_padded) Warning
- Warn when padding is required to align structure members
-
-+Wpoison-system-directories
-+Common Var(flag_poison_system_directories) Init(1) Warning
-+Warn for -I and -L options using system directories if cross compiling
-+
- Wshadow
- Common Var(warn_shadow) Warning
- Warn when one local variable shadows another
-Index: b/gcc/config.in
-===================================================================
---- a/gcc/config.in
-+++ b/gcc/config.in
-@@ -144,6 +144,12 @@
- #endif
-
-
-+/* Define to warn for use of native system header directories */
-+#ifndef USED_FOR_TARGET
-+#undef ENABLE_POISON_SYSTEM_DIRECTORIES
-+#endif
-+
-+
- /* Define if you want all operations on RTL (the basic data structure of the
- optimizer and back end) to be checked for dynamic type safety at runtime.
- This is quite expensive. */
-Index: b/gcc/configure
-===================================================================
---- a/gcc/configure
-+++ b/gcc/configure
-@@ -918,6 +918,7 @@
- with_system_zlib
- enable_maintainer_mode
- enable_version_specific_runtime_libs
-+enable_poison_system_directories
- enable_plugin
- enable_libquadmath_support
- with_linker_hash_style
-@@ -1632,6 +1633,8 @@
- --enable-version-specific-runtime-libs
- specify that runtime libraries should be installed
- in a compiler-specific directory
-+ --enable-poison-system-directories
-+ warn for use of native system header directories
- --enable-plugin enable plugin support
- --disable-libquadmath-support
- disable libquadmath support for Fortran
-@@ -27186,6 +27189,19 @@
- fi
-
-
-+# Check whether --enable-poison-system-directories was given.
-+if test "${enable_poison_system_directories+set}" = set; then :
-+ enableval=$enable_poison_system_directories;
-+else
-+ enable_poison_system_directories=no
-+fi
-+
-+if test "x${enable_poison_system_directories}" = "xyes"; then
-+
-+$as_echo "#define ENABLE_POISON_SYSTEM_DIRECTORIES 1" >>confdefs.h
-+
-+fi
-+
- # Substitute configuration variables
-
-
-Index: b/gcc/configure.ac
-===================================================================
---- a/gcc/configure.ac
-+++ b/gcc/configure.ac
-@@ -5037,6 +5037,16 @@
- [specify that runtime libraries should be
- installed in a compiler-specific directory])])
-
-+AC_ARG_ENABLE([poison-system-directories],
-+ AS_HELP_STRING([--enable-poison-system-directories],
-+ [warn for use of native system header directories]),,
-+ [enable_poison_system_directories=no])
-+if test "x${enable_poison_system_directories}" = "xyes"; then
-+ AC_DEFINE([ENABLE_POISON_SYSTEM_DIRECTORIES],
-+ [1],
-+ [Define to warn for use of native system header directories])
-+fi
-+
- # Substitute configuration variables
- AC_SUBST(subdirs)
- AC_SUBST(srcdir)
-Index: b/gcc/doc/invoke.texi
-===================================================================
---- a/gcc/doc/invoke.texi
-+++ b/gcc/doc/invoke.texi
-@@ -260,6 +260,7 @@
- -Woverlength-strings -Wpacked -Wpacked-bitfield-compat -Wpadded @gol
- -Wparentheses -Wpedantic-ms-format -Wno-pedantic-ms-format @gol
- -Wpointer-arith -Wno-pointer-to-int-cast @gol
-+-Wno-poison-system-directories @gol
- -Wredundant-decls @gol
- -Wreturn-type -Wsequence-point -Wshadow @gol
- -Wsign-compare -Wsign-conversion -Wstack-protector @gol
-@@ -3880,6 +3881,14 @@
- for most targets, it is made up of code and thus requires the stack
- to be made executable in order for the program to work properly.
-
-+ at item -Wno-poison-system-directories
-+ at opindex Wno-poison-system-directories
-+Do not warn for @option{-I} or @option{-L} options using system
-+directories such as @file{/usr/include} when cross compiling. This
-+option is intended for use in chroot environments when such
-+directories contain the correct headers and libraries for the target
-+system rather than the host.
-+
- @item -Wfloat-equal
- @opindex Wfloat-equal
- @opindex Wno-float-equal
-Index: b/gcc/gcc.c
-===================================================================
---- a/gcc/gcc.c
-+++ b/gcc/gcc.c
-@@ -674,6 +674,8 @@
- %{flto} %{flto=*} %l " LINK_PIE_SPEC \
- "%X %{o*} %{e*} %{N} %{n} %{r}\
- %{s} %{t} %{u*} %{z} %{Z} %{!nostdlib:%{!nostartfiles:%S}}\
-+ %{Wno-poison-system-directories:--no-poison-system-directories}\
-+ %{Werror=poison-system-directories:--error-poison-system-directories}\
- %{static:} %{L*} %(mfwrap) %(link_libgcc) %o\
- %{fopenmp|ftree-parallelize-loops=*:%:include(libgomp.spec)%(link_gomp)}\
- %{fgnu-tm:%:include(libitm.spec)%(link_itm)}\
-@@ -3907,6 +3909,12 @@
- gcc_assert (!compare_debug_opt);
- }
-
-+ temp = getenv("BR_COMPILER_PARANOID_UNSAFE_PATH");
-+ if (temp && strlen(temp) > 0)
-+ {
-+ save_switch("-Werror=poison-system-directories", 0, NULL, false);
-+ }
-+
- /* Set up the search paths. We add directories that we expect to
- contain GNU Toolchain components before directories specified by
- the machine description so that we will find GNU components (like
-Index: b/gcc/incpath.c
-===================================================================
---- a/gcc/incpath.c
-+++ b/gcc/incpath.c
-@@ -30,6 +30,7 @@
- #include "intl.h"
- #include "incpath.h"
- #include "cppdefault.h"
-+#include "diagnostic-core.h"
-
- /* Microsoft Windows does not natively support inodes.
- VMS has non-numeric inodes. */
-@@ -373,6 +374,24 @@
- }
- fprintf (stderr, _("End of search list.\n"));
- }
-+
-+#ifdef ENABLE_POISON_SYSTEM_DIRECTORIES
-+ if (flag_poison_system_directories)
-+ {
-+ struct cpp_dir *p;
-+
-+ for (p = heads[QUOTE]; p; p = p->next)
-+ {
-+ if ((!strncmp (p->name, "/usr/include", 12))
-+ || (!strncmp (p->name, "/usr/local/include", 18))
-+ || (!strncmp (p->name, "/usr/X11R6/include", 18)))
-+ warning (OPT_Wpoison_system_directories,
-+ "include location \"%s\" is unsafe for "
-+ "cross-compilation",
-+ p->name);
-+ }
-+ }
-+#endif
- }
-
- /* Use given -I paths for #include "..." but not #include <...>, and
diff --git a/package/gcc/4.8.5/910-gcc-poison-system-directories.patch b/package/gcc/4.8.5/910-gcc-poison-system-directories.patch
deleted file mode 100644
index 88b2c4e..0000000
--- a/package/gcc/4.8.5/910-gcc-poison-system-directories.patch
+++ /dev/null
@@ -1,207 +0,0 @@
-From 160397ef3c3331099af028f1b8d3e085b07d88ad Mon Sep 17 00:00:00 2001
-From: Khem Raj <raj.khem@gmail.com>
-Date: Fri, 29 Mar 2013 08:59:00 +0400
-Subject: [PATCH 16/35] gcc: poison-system-directories
-
-Adapted to Buildroot and gcc 4.8.3 by Thomas Petazzoni, especially the
-addition of the BR_COMPILER_PARANOID_UNSAFE_PATH environment variable.
-
-Signed-off-by: Khem Raj <raj.khem@gmail.com>
-Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
-
-Upstream-Status: Inappropriate [distribution: codesourcery]
----
- gcc/Makefile.in | 2 +-
- gcc/common.opt | 4 ++++
- gcc/config.in | 6 ++++++
- gcc/configure | 20 ++++++++++++++++++--
- gcc/configure.ac | 10 ++++++++++
- gcc/doc/invoke.texi | 9 +++++++++
- gcc/gcc.c | 2 ++
- gcc/incpath.c | 19 +++++++++++++++++++
- 8 files changed, 69 insertions(+), 3 deletions(-)
-
-Index: b/gcc/common.opt
-===================================================================
---- a/gcc/common.opt
-+++ b/gcc/common.opt
-@@ -595,6 +595,10 @@
- Common Var(pedantic) Warning
- Issue warnings needed for strict compliance to the standard
-
-+Wpoison-system-directories
-+Common Var(flag_poison_system_directories) Init(1) Warning
-+Warn for -I and -L options using system directories if cross compiling
-+
- Wshadow
- Common Var(warn_shadow) Warning
- Warn when one local variable shadows another
-Index: b/gcc/config.in
-===================================================================
---- a/gcc/config.in
-+++ b/gcc/config.in
-@@ -138,6 +138,12 @@
- #endif
-
-
-+/* Define to warn for use of native system header directories */
-+#ifndef USED_FOR_TARGET
-+#undef ENABLE_POISON_SYSTEM_DIRECTORIES
-+#endif
-+
-+
- /* Define if you want all operations on RTL (the basic data structure of the
- optimizer and back end) to be checked for dynamic type safety at runtime.
- This is quite expensive. */
-Index: b/gcc/configure
-===================================================================
---- a/gcc/configure
-+++ b/gcc/configure
-@@ -917,6 +917,7 @@
- with_system_zlib
- enable_maintainer_mode
- enable_version_specific_runtime_libs
-+enable_poison_system_directories
- enable_plugin
- enable_libquadmath_support
- with_linker_hash_style
-@@ -1630,6 +1631,8 @@
- --enable-version-specific-runtime-libs
- specify that runtime libraries should be installed
- in a compiler-specific directory
-+ --enable-poison-system-directories
-+ warn for use of native system header directories
- --enable-plugin enable plugin support
- --disable-libquadmath-support
- disable libquadmath support for Fortran
-@@ -27195,6 +27198,19 @@
- fi
-
-
-+# Check whether --enable-poison-system-directories was given.
-+if test "${enable_poison_system_directories+set}" = set; then :
-+ enableval=$enable_poison_system_directories;
-+else
-+ enable_poison_system_directories=no
-+fi
-+
-+if test "x${enable_poison_system_directories}" = "xyes"; then
-+
-+$as_echo "#define ENABLE_POISON_SYSTEM_DIRECTORIES 1" >>confdefs.h
-+
-+fi
-+
- # Substitute configuration variables
-
-
-Index: b/gcc/configure.ac
-===================================================================
---- a/gcc/configure.ac
-+++ b/gcc/configure.ac
-@@ -5101,6 +5101,16 @@
- [specify that runtime libraries should be
- installed in a compiler-specific directory])])
-
-+AC_ARG_ENABLE([poison-system-directories],
-+ AS_HELP_STRING([--enable-poison-system-directories],
-+ [warn for use of native system header directories]),,
-+ [enable_poison_system_directories=no])
-+if test "x${enable_poison_system_directories}" = "xyes"; then
-+ AC_DEFINE([ENABLE_POISON_SYSTEM_DIRECTORIES],
-+ [1],
-+ [Define to warn for use of native system header directories])
-+fi
-+
- # Substitute configuration variables
- AC_SUBST(subdirs)
- AC_SUBST(srcdir)
-Index: b/gcc/doc/invoke.texi
-===================================================================
---- a/gcc/doc/invoke.texi
-+++ b/gcc/doc/invoke.texi
-@@ -258,6 +258,7 @@
- -Woverlength-strings -Wpacked -Wpacked-bitfield-compat -Wpadded @gol
- -Wparentheses -Wpedantic-ms-format -Wno-pedantic-ms-format @gol
- -Wpointer-arith -Wno-pointer-to-int-cast @gol
-+-Wno-poison-system-directories @gol
- -Wredundant-decls -Wno-return-local-addr @gol
- -Wreturn-type -Wsequence-point -Wshadow @gol
- -Wsign-compare -Wsign-conversion -Wsizeof-pointer-memaccess @gol
-@@ -4020,6 +4021,14 @@
- for most targets, it is made up of code and thus requires the stack
- to be made executable in order for the program to work properly.
-
-+ at item -Wno-poison-system-directories
-+ at opindex Wno-poison-system-directories
-+Do not warn for @option{-I} or @option{-L} options using system
-+directories such as @file{/usr/include} when cross compiling. This
-+option is intended for use in chroot environments when such
-+directories contain the correct headers and libraries for the target
-+system rather than the host.
-+
- @item -Wfloat-equal
- @opindex Wfloat-equal
- @opindex Wno-float-equal
-Index: b/gcc/gcc.c
-===================================================================
---- a/gcc/gcc.c
-+++ b/gcc/gcc.c
-@@ -741,6 +741,8 @@
- "%{fuse-ld=*:-fuse-ld=%*}\
- %X %{o*} %{e*} %{N} %{n} %{r}\
- %{s} %{t} %{u*} %{z} %{Z} %{!nostdlib:%{!nostartfiles:%S}}\
-+ %{Wno-poison-system-directories:--no-poison-system-directories}\
-+ %{Werror=poison-system-directories:--error-poison-system-directories}\
- %{static:} %{L*} %(mfwrap) %(link_libgcc) " SANITIZER_EARLY_SPEC " %o\
- %{fopenmp|ftree-parallelize-loops=*:%:include(libgomp.spec)%(link_gomp)}\
- %{fgnu-tm:%:include(libitm.spec)%(link_itm)}\
-@@ -3991,6 +3993,12 @@
- gcc_assert (!compare_debug_opt);
- }
-
-+ temp = getenv("BR_COMPILER_PARANOID_UNSAFE_PATH");
-+ if (temp && strlen(temp) > 0)
-+ {
-+ save_switch("-Werror=poison-system-directories", 0, NULL, false, true);
-+ }
-+
- /* Set up the search paths. We add directories that we expect to
- contain GNU Toolchain components before directories specified by
- the machine description so that we will find GNU components (like
-Index: b/gcc/incpath.c
-===================================================================
---- a/gcc/incpath.c
-+++ b/gcc/incpath.c
-@@ -28,6 +28,7 @@
- #include "intl.h"
- #include "incpath.h"
- #include "cppdefault.h"
-+#include "diagnostic-core.h"
-
- /* Microsoft Windows does not natively support inodes.
- VMS has non-numeric inodes. */
-@@ -382,6 +383,24 @@
- }
- fprintf (stderr, _("End of search list.\n"));
- }
-+
-+#ifdef ENABLE_POISON_SYSTEM_DIRECTORIES
-+ if (flag_poison_system_directories)
-+ {
-+ struct cpp_dir *p;
-+
-+ for (p = heads[QUOTE]; p; p = p->next)
-+ {
-+ if ((!strncmp (p->name, "/usr/include", 12))
-+ || (!strncmp (p->name, "/usr/local/include", 18))
-+ || (!strncmp (p->name, "/usr/X11R6/include", 18)))
-+ warning (OPT_Wpoison_system_directories,
-+ "include location \"%s\" is unsafe for "
-+ "cross-compilation",
-+ p->name);
-+ }
-+ }
-+#endif
- }
-
- /* Use given -I paths for #include "..." but not #include <...>, and
diff --git a/package/gcc/4.9.3/910-gcc-poison-system-directories.patch b/package/gcc/4.9.3/910-gcc-poison-system-directories.patch
deleted file mode 100644
index 975f01f..0000000
--- a/package/gcc/4.9.3/910-gcc-poison-system-directories.patch
+++ /dev/null
@@ -1,207 +0,0 @@
-From 160397ef3c3331099af028f1b8d3e085b07d88ad Mon Sep 17 00:00:00 2001
-From: Khem Raj <raj.khem@gmail.com>
-Date: Fri, 29 Mar 2013 08:59:00 +0400
-Subject: [PATCH 16/35] gcc: poison-system-directories
-
-Adapted to Buildroot and gcc 4.9.1 by Thomas Petazzoni, especially the
-addition of the BR_COMPILER_PARANOID_UNSAFE_PATH environment variable.
-
-Signed-off-by: Khem Raj <raj.khem@gmail.com>
-Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
-
-Upstream-Status: Inappropriate [distribution: codesourcery]
----
- gcc/Makefile.in | 2 +-
- gcc/common.opt | 4 ++++
- gcc/config.in | 6 ++++++
- gcc/configure | 20 ++++++++++++++++++--
- gcc/configure.ac | 10 ++++++++++
- gcc/doc/invoke.texi | 9 +++++++++
- gcc/gcc.c | 2 ++
- gcc/incpath.c | 19 +++++++++++++++++++
- 8 files changed, 69 insertions(+), 3 deletions(-)
-
-Index: b/gcc/common.opt
-===================================================================
---- a/gcc/common.opt
-+++ b/gcc/common.opt
-@@ -603,6 +603,10 @@
- Common Var(pedantic) Warning
- Issue warnings needed for strict compliance to the standard
-
-+Wpoison-system-directories
-+Common Var(flag_poison_system_directories) Init(1) Warning
-+Warn for -I and -L options using system directories if cross compiling
-+
- Wshadow
- Common Var(warn_shadow) Warning
- Warn when one local variable shadows another
-Index: b/gcc/config.in
-===================================================================
---- a/gcc/config.in
-+++ b/gcc/config.in
-@@ -138,6 +138,12 @@
- #endif
-
-
-+/* Define to warn for use of native system header directories */
-+#ifndef USED_FOR_TARGET
-+#undef ENABLE_POISON_SYSTEM_DIRECTORIES
-+#endif
-+
-+
- /* Define if you want all operations on RTL (the basic data structure of the
- optimizer and back end) to be checked for dynamic type safety at runtime.
- This is quite expensive. */
-Index: b/gcc/configure
-===================================================================
---- a/gcc/configure
-+++ b/gcc/configure
-@@ -929,6 +929,7 @@
- enable_maintainer_mode
- enable_link_mutex
- enable_version_specific_runtime_libs
-+enable_poison_system_directories
- enable_plugin
- enable_host_shared
- enable_libquadmath_support
-@@ -1657,6 +1658,8 @@
- --enable-version-specific-runtime-libs
- specify that runtime libraries should be installed
- in a compiler-specific directory
-+ --enable-poison-system-directories
-+ warn for use of native system header directories
- --enable-plugin enable plugin support
- --enable-host-shared build host code as shared libraries
- --disable-libquadmath-support
-@@ -27765,6 +27768,19 @@
- fi
-
-
-+# Check whether --enable-poison-system-directories was given.
-+if test "${enable_poison_system_directories+set}" = set; then :
-+ enableval=$enable_poison_system_directories;
-+else
-+ enable_poison_system_directories=no
-+fi
-+
-+if test "x${enable_poison_system_directories}" = "xyes"; then
-+
-+$as_echo "#define ENABLE_POISON_SYSTEM_DIRECTORIES 1" >>confdefs.h
-+
-+fi
-+
- # Substitute configuration variables
-
-
-Index: b/gcc/configure.ac
-===================================================================
---- a/gcc/configure.ac
-+++ b/gcc/configure.ac
-@@ -5411,6 +5411,16 @@
- [specify that runtime libraries should be
- installed in a compiler-specific directory])])
-
-+AC_ARG_ENABLE([poison-system-directories],
-+ AS_HELP_STRING([--enable-poison-system-directories],
-+ [warn for use of native system header directories]),,
-+ [enable_poison_system_directories=no])
-+if test "x${enable_poison_system_directories}" = "xyes"; then
-+ AC_DEFINE([ENABLE_POISON_SYSTEM_DIRECTORIES],
-+ [1],
-+ [Define to warn for use of native system header directories])
-+fi
-+
- # Substitute configuration variables
- AC_SUBST(subdirs)
- AC_SUBST(srcdir)
-Index: b/gcc/doc/invoke.texi
-===================================================================
---- a/gcc/doc/invoke.texi
-+++ b/gcc/doc/invoke.texi
-@@ -260,6 +260,7 @@
- -Woverlength-strings -Wpacked -Wpacked-bitfield-compat -Wpadded @gol
- -Wparentheses -Wpedantic-ms-format -Wno-pedantic-ms-format @gol
- -Wpointer-arith -Wno-pointer-to-int-cast @gol
-+-Wno-poison-system-directories @gol
- -Wredundant-decls -Wno-return-local-addr @gol
- -Wreturn-type -Wsequence-point -Wshadow @gol
- -Wsign-compare -Wsign-conversion -Wfloat-conversion @gol
-@@ -4209,6 +4210,14 @@
- for most targets, it is made up of code and thus requires the stack
- to be made executable in order for the program to work properly.
-
-+ at item -Wno-poison-system-directories
-+ at opindex Wno-poison-system-directories
-+Do not warn for @option{-I} or @option{-L} options using system
-+directories such as @file{/usr/include} when cross compiling. This
-+option is intended for use in chroot environments when such
-+directories contain the correct headers and libraries for the target
-+system rather than the host.
-+
- @item -Wfloat-equal
- @opindex Wfloat-equal
- @opindex Wno-float-equal
-Index: b/gcc/gcc.c
-===================================================================
---- a/gcc/gcc.c
-+++ b/gcc/gcc.c
-@@ -770,6 +770,8 @@
- "%{fuse-ld=*:-fuse-ld=%*}\
- %X %{o*} %{e*} %{N} %{n} %{r}\
- %{s} %{t} %{u*} %{z} %{Z} %{!nostdlib:%{!nostartfiles:%S}} " VTABLE_VERIFICATION_SPEC " \
-+ %{Wno-poison-system-directories:--no-poison-system-directories}\
-+ %{Werror=poison-system-directories:--error-poison-system-directories}\
- %{static:} %{L*} %(mfwrap) %(link_libgcc) " SANITIZER_EARLY_SPEC " %o\
- %{fopenmp|ftree-parallelize-loops=*:%:include(libgomp.spec)%(link_gomp)}\
- %{fcilkplus:%:include(libcilkrts.spec)%(link_cilkrts)}\
-@@ -4034,6 +4036,12 @@
- gcc_assert (!compare_debug_opt);
- }
-
-+ temp = getenv("BR_COMPILER_PARANOID_UNSAFE_PATH");
-+ if (temp && strlen(temp) > 0)
-+ {
-+ save_switch("-Werror=poison-system-directories", 0, NULL, false, true);
-+ }
-+
- /* Set up the search paths. We add directories that we expect to
- contain GNU Toolchain components before directories specified by
- the machine description so that we will find GNU components (like
-Index: b/gcc/incpath.c
-===================================================================
---- a/gcc/incpath.c
-+++ b/gcc/incpath.c
-@@ -28,6 +28,7 @@
- #include "intl.h"
- #include "incpath.h"
- #include "cppdefault.h"
-+#include "diagnostic-core.h"
-
- /* Microsoft Windows does not natively support inodes.
- VMS has non-numeric inodes. */
-@@ -382,6 +383,24 @@
- }
- fprintf (stderr, _("End of search list.\n"));
- }
-+
-+#ifdef ENABLE_POISON_SYSTEM_DIRECTORIES
-+ if (flag_poison_system_directories)
-+ {
-+ struct cpp_dir *p;
-+
-+ for (p = heads[QUOTE]; p; p = p->next)
-+ {
-+ if ((!strncmp (p->name, "/usr/include", 12))
-+ || (!strncmp (p->name, "/usr/local/include", 18))
-+ || (!strncmp (p->name, "/usr/X11R6/include", 18)))
-+ warning (OPT_Wpoison_system_directories,
-+ "include location \"%s\" is unsafe for "
-+ "cross-compilation",
-+ p->name);
-+ }
-+ }
-+#endif
- }
-
- /* Use given -I paths for #include "..." but not #include <...>, and
diff --git a/package/gcc/5.2.0/200-gcc-poison-system-directories.patch b/package/gcc/5.2.0/200-gcc-poison-system-directories.patch
deleted file mode 100644
index f41c0c4..0000000
--- a/package/gcc/5.2.0/200-gcc-poison-system-directories.patch
+++ /dev/null
@@ -1,207 +0,0 @@
-From 160397ef3c3331099af028f1b8d3e085b07d88ad Mon Sep 17 00:00:00 2001
-From: Khem Raj <raj.khem@gmail.com>
-Date: Fri, 29 Mar 2013 08:59:00 +0400
-Subject: [PATCH 16/35] gcc: poison-system-directories
-
-Adapted to Buildroot and gcc 4.9.1 by Thomas Petazzoni, especially the
-addition of the BR_COMPILER_PARANOID_UNSAFE_PATH environment variable.
-
-Signed-off-by: Khem Raj <raj.khem@gmail.com>
-Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
-
-Upstream-Status: Inappropriate [distribution: codesourcery]
----
- gcc/Makefile.in | 2 +-
- gcc/common.opt | 4 ++++
- gcc/config.in | 6 ++++++
- gcc/configure | 20 ++++++++++++++++++--
- gcc/configure.ac | 10 ++++++++++
- gcc/doc/invoke.texi | 9 +++++++++
- gcc/gcc.c | 2 ++
- gcc/incpath.c | 19 +++++++++++++++++++
- 8 files changed, 69 insertions(+), 3 deletions(-)
-
-Index: b/gcc/common.opt
-===================================================================
---- a/gcc/common.opt
-+++ b/gcc/common.opt
-@@ -623,6 +623,10 @@
- Common Var(warn_return_local_addr) Init(1) Warning
- Warn about returning a pointer/reference to a local or temporary variable.
-
-+Wpoison-system-directories
-+Common Var(flag_poison_system_directories) Init(1) Warning
-+Warn for -I and -L options using system directories if cross compiling
-+
- Wshadow
- Common Var(warn_shadow) Warning
- Warn when one local variable shadows another
-Index: b/gcc/config.in
-===================================================================
---- a/gcc/config.in
-+++ b/gcc/config.in
-@@ -168,6 +168,12 @@
- #endif
-
-
-+/* Define to warn for use of native system header directories */
-+#ifndef USED_FOR_TARGET
-+#undef ENABLE_POISON_SYSTEM_DIRECTORIES
-+#endif
-+
-+
- /* Define if you want all operations on RTL (the basic data structure of the
- optimizer and back end) to be checked for dynamic type safety at runtime.
- This is quite expensive. */
-Index: b/gcc/configure
-===================================================================
---- a/gcc/configure
-+++ b/gcc/configure
-@@ -932,6 +932,7 @@
- enable_maintainer_mode
- enable_link_mutex
- enable_version_specific_runtime_libs
-+enable_poison_system_directories
- enable_plugin
- enable_host_shared
- enable_libquadmath_support
-@@ -1661,6 +1662,8 @@
- --enable-version-specific-runtime-libs
- specify that runtime libraries should be installed
- in a compiler-specific directory
-+ --enable-poison-system-directories
-+ warn for use of native system header directories
- --enable-plugin enable plugin support
- --enable-host-shared build host code as shared libraries
- --disable-libquadmath-support
-@@ -28087,6 +28090,19 @@
- fi
-
-
-+# Check whether --enable-poison-system-directories was given.
-+if test "${enable_poison_system_directories+set}" = set; then :
-+ enableval=$enable_poison_system_directories;
-+else
-+ enable_poison_system_directories=no
-+fi
-+
-+if test "x${enable_poison_system_directories}" = "xyes"; then
-+
-+$as_echo "#define ENABLE_POISON_SYSTEM_DIRECTORIES 1" >>confdefs.h
-+
-+fi
-+
- # Substitute configuration variables
-
-
-Index: b/gcc/configure.ac
-===================================================================
---- a/gcc/configure.ac
-+++ b/gcc/configure.ac
-@@ -5571,6 +5571,16 @@
- [specify that runtime libraries should be
- installed in a compiler-specific directory])])
-
-+AC_ARG_ENABLE([poison-system-directories],
-+ AS_HELP_STRING([--enable-poison-system-directories],
-+ [warn for use of native system header directories]),,
-+ [enable_poison_system_directories=no])
-+if test "x${enable_poison_system_directories}" = "xyes"; then
-+ AC_DEFINE([ENABLE_POISON_SYSTEM_DIRECTORIES],
-+ [1],
-+ [Define to warn for use of native system header directories])
-+fi
-+
- # Substitute configuration variables
- AC_SUBST(subdirs)
- AC_SUBST(srcdir)
-Index: b/gcc/doc/invoke.texi
-===================================================================
---- a/gcc/doc/invoke.texi
-+++ b/gcc/doc/invoke.texi
-@@ -269,6 +269,7 @@
- -Woverlength-strings -Wpacked -Wpacked-bitfield-compat -Wpadded @gol
- -Wparentheses -Wpedantic-ms-format -Wno-pedantic-ms-format @gol
- -Wpointer-arith -Wno-pointer-to-int-cast @gol
-+-Wno-poison-system-directories @gol
- -Wredundant-decls -Wno-return-local-addr @gol
- -Wreturn-type -Wsequence-point -Wshadow -Wno-shadow-ivar @gol
- -Wshift-count-negative -Wshift-count-overflow @gol
-@@ -4432,6 +4433,14 @@
- most targets, it is made up of code and thus requires the stack to be
- made executable in order for the program to work properly.
-
-+ at item -Wno-poison-system-directories
-+ at opindex Wno-poison-system-directories
-+Do not warn for @option{-I} or @option{-L} options using system
-+directories such as @file{/usr/include} when cross compiling. This
-+option is intended for use in chroot environments when such
-+directories contain the correct headers and libraries for the target
-+system rather than the host.
-+
- @item -Wfloat-equal
- @opindex Wfloat-equal
- @opindex Wno-float-equal
-Index: b/gcc/gcc.c
-===================================================================
---- a/gcc/gcc.c
-+++ b/gcc/gcc.c
-@@ -835,6 +835,8 @@
- "%{fuse-ld=*:-fuse-ld=%*} " LINK_COMPRESS_DEBUG_SPEC \
- "%X %{o*} %{e*} %{N} %{n} %{r}\
- %{s} %{t} %{u*} %{z} %{Z} %{!nostdlib:%{!nostartfiles:%S}} " VTABLE_VERIFICATION_SPEC " \
-+ %{Wno-poison-system-directories:--no-poison-system-directories}\
-+ %{Werror=poison-system-directories:--error-poison-system-directories}\
- %{static:} %{L*} %(mfwrap) %(link_libgcc) " SANITIZER_EARLY_SPEC " %o\
- " CHKP_SPEC " \
- %{fopenacc|fopenmp|ftree-parallelize-loops=*:%:include(libgomp.spec)%(link_gomp)}\
-@@ -4244,6 +4246,12 @@
- gcc_assert (!compare_debug_opt);
- }
-
-+ temp = getenv("BR_COMPILER_PARANOID_UNSAFE_PATH");
-+ if (temp && strlen(temp) > 0)
-+ {
-+ save_switch("-Werror=poison-system-directories", 0, NULL, false, true);
-+ }
-+
- /* Set up the search paths. We add directories that we expect to
- contain GNU Toolchain components before directories specified by
- the machine description so that we will find GNU components (like
-Index: b/gcc/incpath.c
-===================================================================
---- a/gcc/incpath.c
-+++ b/gcc/incpath.c
-@@ -28,6 +28,7 @@
- #include "intl.h"
- #include "incpath.h"
- #include "cppdefault.h"
-+#include "diagnostic-core.h"
-
- /* Microsoft Windows does not natively support inodes.
- VMS has non-numeric inodes. */
-@@ -383,6 +384,24 @@
- }
- fprintf (stderr, _("End of search list.\n"));
- }
-+
-+#ifdef ENABLE_POISON_SYSTEM_DIRECTORIES
-+ if (flag_poison_system_directories)
-+ {
-+ struct cpp_dir *p;
-+
-+ for (p = heads[QUOTE]; p; p = p->next)
-+ {
-+ if ((!strncmp (p->name, "/usr/include", 12))
-+ || (!strncmp (p->name, "/usr/local/include", 18))
-+ || (!strncmp (p->name, "/usr/X11R6/include", 18)))
-+ warning (OPT_Wpoison_system_directories,
-+ "include location \"%s\" is unsafe for "
-+ "cross-compilation",
-+ p->name);
-+ }
-+ }
-+#endif
- }
-
- /* Use given -I paths for #include "..." but not #include <...>, and
diff --git a/package/gcc/arc-2015.06/910-gcc-poison-system-directories.patch b/package/gcc/arc-2015.06/910-gcc-poison-system-directories.patch
deleted file mode 100644
index 67b3799..0000000
--- a/package/gcc/arc-2015.06/910-gcc-poison-system-directories.patch
+++ /dev/null
@@ -1,221 +0,0 @@
-From 160397ef3c3331099af028f1b8d3e085b07d88ad Mon Sep 17 00:00:00 2001
-From: Khem Raj <raj.khem@gmail.com>
-Date: Fri, 29 Mar 2013 08:59:00 +0400
-Subject: [PATCH 16/35] gcc: poison-system-directories
-
-Adapted to Buildroot and gcc arc-4.8-R3 by Thomas Petazzoni,
-especially the addition of the BR_COMPILER_PARANOID_UNSAFE_PATH
-environment variable.
-
-Signed-off-by: Khem Raj <raj.khem@gmail.com>
-Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
-
-Upstream-Status: Inappropriate [distribution: codesourcery]
----
- gcc/Makefile.in | 2 +-
- gcc/common.opt | 4 ++++
- gcc/config.in | 6 ++++++
- gcc/configure | 20 ++++++++++++++++++--
- gcc/configure.ac | 10 ++++++++++
- gcc/doc/invoke.texi | 9 +++++++++
- gcc/gcc.c | 2 ++
- gcc/incpath.c | 19 +++++++++++++++++++
- 8 files changed, 69 insertions(+), 3 deletions(-)
-
-Index: b/gcc/common.opt
-===================================================================
---- a/gcc/common.opt
-+++ b/gcc/common.opt
-@@ -595,6 +595,10 @@
- Common Var(pedantic) Warning
- Issue warnings needed for strict compliance to the standard
-
-+Wpoison-system-directories
-+Common Var(flag_poison_system_directories) Init(1) Warning
-+Warn for -I and -L options using system directories if cross compiling
-+
- Wshadow
- Common Var(warn_shadow) Warning
- Warn when one local variable shadows another
-Index: b/gcc/config.in
-===================================================================
---- a/gcc/config.in
-+++ b/gcc/config.in
-@@ -138,6 +138,12 @@
- #endif
-
-
-+/* Define to warn for use of native system header directories */
-+#ifndef USED_FOR_TARGET
-+#undef ENABLE_POISON_SYSTEM_DIRECTORIES
-+#endif
-+
-+
- /* Define if you want all operations on RTL (the basic data structure of the
- optimizer and back end) to be checked for dynamic type safety at runtime.
- This is quite expensive. */
-Index: b/gcc/configure
-===================================================================
---- a/gcc/configure
-+++ b/gcc/configure
-@@ -917,6 +917,7 @@
- with_system_zlib
- enable_maintainer_mode
- enable_version_specific_runtime_libs
-+enable_poison_system_directories
- enable_plugin
- enable_libquadmath_support
- with_linker_hash_style
-@@ -1630,6 +1631,8 @@
- --enable-version-specific-runtime-libs
- specify that runtime libraries should be installed
- in a compiler-specific directory
-+ --enable-poison-system-directories
-+ warn for use of native system header directories
- --enable-plugin enable plugin support
- --disable-libquadmath-support
- disable libquadmath support for Fortran
-@@ -27103,6 +27106,19 @@
- fi
-
-
-+# Check whether --enable-poison-system-directories was given.
-+if test "${enable_poison_system_directories+set}" = set; then :
-+ enableval=$enable_poison_system_directories;
-+else
-+ enable_poison_system_directories=no
-+fi
-+
-+if test "x${enable_poison_system_directories}" = "xyes"; then
-+
-+$as_echo "#define ENABLE_POISON_SYSTEM_DIRECTORIES 1" >>confdefs.h
-+
-+fi
-+
- # Substitute configuration variables
-
-
-Index: b/gcc/configure.ac
-===================================================================
---- a/gcc/configure.ac
-+++ b/gcc/configure.ac
-@@ -5063,6 +5063,16 @@
- [specify that runtime libraries should be
- installed in a compiler-specific directory])])
-
-+AC_ARG_ENABLE([poison-system-directories],
-+ AS_HELP_STRING([--enable-poison-system-directories],
-+ [warn for use of native system header directories]),,
-+ [enable_poison_system_directories=no])
-+if test "x${enable_poison_system_directories}" = "xyes"; then
-+ AC_DEFINE([ENABLE_POISON_SYSTEM_DIRECTORIES],
-+ [1],
-+ [Define to warn for use of native system header directories])
-+fi
-+
- # Substitute configuration variables
- AC_SUBST(subdirs)
- AC_SUBST(srcdir)
-Index: b/gcc/doc/invoke.texi
-===================================================================
---- a/gcc/doc/invoke.texi
-+++ b/gcc/doc/invoke.texi
-@@ -258,6 +258,7 @@
- -Woverlength-strings -Wpacked -Wpacked-bitfield-compat -Wpadded @gol
- -Wparentheses -Wpedantic-ms-format -Wno-pedantic-ms-format @gol
- -Wpointer-arith -Wno-pointer-to-int-cast @gol
-+-Wno-poison-system-directories @gol
- -Wredundant-decls -Wno-return-local-addr @gol
- -Wreturn-type -Wsequence-point -Wshadow @gol
- -Wsign-compare -Wsign-conversion -Wsizeof-pointer-memaccess @gol
-@@ -4025,6 +4026,14 @@
- for most targets, it is made up of code and thus requires the stack
- to be made executable in order for the program to work properly.
-
-+ at item -Wno-poison-system-directories
-+ at opindex Wno-poison-system-directories
-+Do not warn for @option{-I} or @option{-L} options using system
-+directories such as @file{/usr/include} when cross compiling. This
-+option is intended for use in chroot environments when such
-+directories contain the correct headers and libraries for the target
-+system rather than the host.
-+
- @item -Wfloat-equal
- @opindex Wfloat-equal
- @opindex Wno-float-equal
-Index: b/gcc/gcc.c
-===================================================================
---- a/gcc/gcc.c
-+++ b/gcc/gcc.c
-@@ -740,6 +740,8 @@
- "%{fuse-ld=*:-fuse-ld=%*}\
- %X %{o*} %{e*} %{N} %{n} %{r}\
- %{s} %{t} %{u*} %{z} %{Z} %{!nostdlib:%{!nostartfiles:%S}}\
-+ %{Wno-poison-system-directories:--no-poison-system-directories}\
-+ %{Werror=poison-system-directories:--error-poison-system-directories}\
- %{static:} %{L*} %(mfwrap) %(link_libgcc) " SANITIZER_EARLY_SPEC " %o\
- %{fopenmp|ftree-parallelize-loops=*:%:include(libgomp.spec)%(link_gomp)}\
- %{fgnu-tm:%:include(libitm.spec)%(link_itm)}\
-@@ -3990,6 +3992,12 @@
- gcc_assert (!compare_debug_opt);
- }
-
-+ temp = getenv("BR_COMPILER_PARANOID_UNSAFE_PATH");
-+ if (temp && strlen(temp) > 0)
-+ {
-+ save_switch("-Werror=poison-system-directories", 0, NULL, false, true);
-+ }
-+
- /* Set up the search paths. We add directories that we expect to
- contain GNU Toolchain components before directories specified by
- the machine description so that we will find GNU components (like
-Index: b/gcc/incpath.c
-===================================================================
---- a/gcc/incpath.c
-+++ b/gcc/incpath.c
-@@ -28,6 +28,7 @@
- #include "intl.h"
- #include "incpath.h"
- #include "cppdefault.h"
-+#include "diagnostic-core.h"
-
- /* Microsoft Windows does not natively support inodes.
- VMS has non-numeric inodes. */
-@@ -382,6 +383,24 @@
- }
- fprintf (stderr, _("End of search list.\n"));
- }
-+
-+#ifdef ENABLE_POISON_SYSTEM_DIRECTORIES
-+ if (flag_poison_system_directories)
-+ {
-+ struct cpp_dir *p;
-+
-+ for (p = heads[QUOTE]; p; p = p->next)
-+ {
-+ if ((!strncmp (p->name, "/usr/include", 12))
-+ || (!strncmp (p->name, "/usr/local/include", 18))
-+ || (!strncmp (p->name, "/usr/X11R6/include", 18)))
-+ warning (OPT_Wpoison_system_directories,
-+ "include location \"%s\" is unsafe for "
-+ "cross-compilation",
-+ p->name);
-+ }
-+ }
-+#endif
- }
-
- /* Use given -I paths for #include "..." but not #include <...>, and
-Index: b/gcc/config/arc/arc.h
-===================================================================
---- a/gcc/config/arc/arc.h
-+++ b/gcc/config/arc/arc.h
-@@ -205,6 +205,8 @@
- %{!fsyntax-only:%{!c:%{!M:%{!MM:%{!E:%{!S:\
- %(linker) %l " LINK_PIE_SPEC "%X %{o*} %{A} %{d} %{e*} %{m} %{N} %{n} %{r}\
- %{s} %{t} %{u*} %{x} %{z} %{Z} %{!A:%{!nostdlib:%{!nostartfiles:%S}}}\
-+ %{Wno-poison-system-directories:--no-poison-system-directories}\
-+ %{Werror=poison-system-directories:--error-poison-system-directories}\
- %{static:} %{L*} %(mfwrap) %(link_libgcc) %o\
- %{fopenmp:%:include(libgomp.spec)%(link_gomp)} %(mflib)\
- %{fprofile-arcs|fprofile-generate|coverage:-lgcov}\
diff --git a/package/gcc/gcc-final/gcc-final.mk b/package/gcc/gcc-final/gcc-final.mk
index 2c16fdf..a5be3c7 100644
--- a/package/gcc/gcc-final/gcc-final.mk
+++ b/package/gcc/gcc-final/gcc-final.mk
@@ -50,7 +50,6 @@ GCC_FINAL_CROSS_LANGUAGES = $(subst $(space),$(comma),$(GCC_FINAL_CROSS_LANGUAGE
HOST_GCC_FINAL_CONF_OPTS = \
$(HOST_GCC_COMMON_CONF_OPTS) \
--enable-languages=$(GCC_FINAL_CROSS_LANGUAGES) \
- --enable-poison-system-directories \
--with-build-time-tools=$(HOST_DIR)/usr/$(GNU_TARGET_NAME)/bin
HOST_GCC_FINAL_GCC_LIB_DIR = $(HOST_DIR)/usr/$(GNU_TARGET_NAME)/lib*
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 04/18] infra: move ccache handling to the toolchain wrapper
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (2 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 03/18] gcc: remove unsafe patch check (poison system dirs) patch Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 15:07 ` Peter Korsgaard
2015-10-04 15:23 ` [Buildroot] [PATCH v3] " Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 05/18] perl: Remove ccache handling Arnout Vandecappelle
` (14 subsequent siblings)
18 siblings, 2 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
Since we always have a toolchain wrapper now, we can move the ccache
call to the toolchain wrapper.
The hostcc ccache handling obviously stays.
The global addition of ccache to TARGET_CC/CXX is removed, but many
individual packages and infras still add it. This means we have a
chain like this: ccache -> toolchain-wrapper -> ccache -> gcc
However, this is fairly harmless: for cache misses, the inner ccache
just adds overhead and for cache hits, the inner ccache is never
called. Later patches will remove these redundant ccache calls.
As a side effect, perl now supports ccache as well.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Danomi Manchego <danomimanchego123@gmail.com>
Cc: K?roly Kasza <kaszak@gmail.com>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
v2: No change
---
package/Makefile.in | 5 -----
toolchain/toolchain-wrapper.c | 5 ++++-
toolchain/toolchain-wrapper.mk | 4 ++++
3 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/package/Makefile.in b/package/Makefile.in
index 545694f..4b1ce7f 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -189,11 +189,6 @@ TARGET_OBJDUMP = $(TARGET_CROSS)objdump
TARGET_CC_NOCCACHE := $(TARGET_CC)
TARGET_CXX_NOCCACHE := $(TARGET_CXX)
-ifeq ($(BR2_CCACHE),y)
-TARGET_CC := $(CCACHE) $(TARGET_CC)
-TARGET_CXX := $(CCACHE) $(TARGET_CXX)
-endif
-
ifeq ($(BR2_STRIP_strip),y)
STRIP_STRIP_DEBUG := --strip-debug
STRIP_STRIP_UNNEEDED := --strip-unneeded
diff --git a/toolchain/toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
index d4d25c7..b5fe1d3 100644
--- a/toolchain/toolchain-wrapper.c
+++ b/toolchain/toolchain-wrapper.c
@@ -40,6 +40,9 @@ static char sysroot[PATH_MAX];
#define EXCLUSIVE_ARGS 3
static char *predef_args[] = {
+#ifdef BR_CCACHE
+ BR_CCACHE,
+#endif
path,
"--sysroot", sysroot,
#ifdef BR_ABI
@@ -251,7 +254,7 @@ int main(int argc, char **argv)
}
}
- if (execv(path, args))
+ if (execv(args[0], args))
perror(path);
free(args);
diff --git a/toolchain/toolchain-wrapper.mk b/toolchain/toolchain-wrapper.mk
index 899947d..0fd2b86 100644
--- a/toolchain/toolchain-wrapper.mk
+++ b/toolchain/toolchain-wrapper.mk
@@ -16,6 +16,10 @@ TOOLCHAIN_WRAPPER_ARGS += -DBR_SYSROOT='"$(STAGING_SUBDIR)"'
# separate argument when used in execv() by the toolchain wrapper.
TOOLCHAIN_WRAPPER_ARGS += -DBR_ADDITIONAL_CFLAGS='$(foreach f,$(call qstrip,$(BR2_TARGET_OPTIMIZATION)),"$(f)",)'
+ifeq ($(BR2_CCACHE),y)
+TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE='"$(CCACHE)"'
+endif
+
# For simplicity, build directly into the install location
define TOOLCHAIN_BUILD_WRAPPER
$(Q)mkdir -p $(HOST_DIR)/usr/bin
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 05/18] perl: Remove ccache handling
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (3 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 04/18] infra: move ccache handling to the toolchain wrapper Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 06/18] imx-lib: remove now-redundant " Arnout Vandecappelle
` (13 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
Now the ccache handling has moved to the toolchain wrapper, it is no
longer necessary to pass TARGET_CC_NOCCACHE.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
v2: No change
---
package/perl/perl.mk | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/package/perl/perl.mk b/package/perl/perl.mk
index b9f90e4..7192700 100644
--- a/package/perl/perl.mk
+++ b/package/perl/perl.mk
@@ -45,13 +45,12 @@ PERL_DEPENDENCIES += gdbm
endif
# We have to override LD, because an external multilib toolchain ld is not
-# wrapped to provide the required sysroot options. We also can't use ccache
-# because the configure script doesn't support it.
+# wrapped to provide the required sysroot options.
PERL_CONF_OPTS = \
--target=$(GNU_TARGET_NAME) \
--target-tools-prefix=$(TARGET_CROSS) \
--prefix=/usr \
- -Dld="$(TARGET_CC_NOCCACHE)" \
+ -Dld="$(TARGET_CC)" \
-Dccflags="$(TARGET_CFLAGS)" \
-Dldflags="$(TARGET_LDFLAGS) -lm" \
-Dmydomain="" \
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 06/18] imx-lib: remove now-redundant ccache handling
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (4 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 05/18] perl: Remove ccache handling Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 07/18] imx-vpu: " Arnout Vandecappelle
` (12 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
v2: No change
---
package/freescale-imx/imx-lib/imx-lib.mk | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/package/freescale-imx/imx-lib/imx-lib.mk b/package/freescale-imx/imx-lib/imx-lib.mk
index 92739a8..7046580 100644
--- a/package/freescale-imx/imx-lib/imx-lib.mk
+++ b/package/freescale-imx/imx-lib/imx-lib.mk
@@ -21,7 +21,7 @@ IMX_LIB_INCLUDE = \
IMX_LIB_MAKE_ENV = \
$(TARGET_MAKE_ENV) \
$(TARGET_CONFIGURE_OPTS) \
- CROSS_COMPILE="$(CCACHE) $(TARGET_CROSS)" \
+ CROSS_COMPILE="$(TARGET_CROSS)" \
PLATFORM=$(BR2_PACKAGE_FREESCALE_IMX_PLATFORM) \
INCLUDE="$(IMX_LIB_INCLUDE)"
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 07/18] imx-vpu: remove now-redundant ccache handling
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (5 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 06/18] imx-lib: remove now-redundant " Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 08/18] linux: " Arnout Vandecappelle
` (11 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
v2: Rebased on master
---
package/freescale-imx/imx-vpu/imx-vpu.mk | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/package/freescale-imx/imx-vpu/imx-vpu.mk b/package/freescale-imx/imx-vpu/imx-vpu.mk
index 3606857..571cc40 100644
--- a/package/freescale-imx/imx-vpu/imx-vpu.mk
+++ b/package/freescale-imx/imx-vpu/imx-vpu.mk
@@ -13,7 +13,7 @@ IMX_VPU_INSTALL_STAGING = YES
IMX_VPU_MAKE_ENV = \
$(TARGET_MAKE_ENV) \
$(TARGET_CONFIGURE_OPTS) \
- CROSS_COMPILE="$(CCACHE) $(TARGET_CROSS)" \
+ CROSS_COMPILE="$(TARGET_CROSS)" \
PLATFORM=$(BR2_PACKAGE_FREESCALE_IMX_PLATFORM)
IMX_VPU_LICENSE = Freescale Semiconductor Software License Agreement
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 08/18] linux: remove now-redundant ccache handling
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (6 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 07/18] imx-vpu: " Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 09/18] uboot: " Arnout Vandecappelle
` (10 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
v2: No change
---
linux/linux.mk | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/linux/linux.mk b/linux/linux.mk
index bbcc54b..2319576 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -67,7 +67,7 @@ LINUX_MAKE_FLAGS = \
HOSTCFLAGS="$(HOSTCFLAGS)" \
ARCH=$(KERNEL_ARCH) \
INSTALL_MOD_PATH=$(TARGET_DIR) \
- CROSS_COMPILE="$(CCACHE) $(TARGET_CROSS)" \
+ CROSS_COMPILE="$(TARGET_CROSS)" \
DEPMOD=$(HOST_DIR)/sbin/depmod
LINUX_MAKE_ENV = \
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 09/18] uboot: remove now-redundant ccache handling
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (7 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 08/18] linux: " Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 10/18] barebox: " Arnout Vandecappelle
` (9 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
v2: No change
---
boot/uboot/uboot.mk | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/boot/uboot/uboot.mk b/boot/uboot/uboot.mk
index 8f321ac..7494078 100644
--- a/boot/uboot/uboot.mk
+++ b/boot/uboot/uboot.mk
@@ -74,7 +74,7 @@ endif
UBOOT_ARCH = $(KERNEL_ARCH)
UBOOT_MAKE_OPTS += \
- CROSS_COMPILE="$(CCACHE) $(TARGET_CROSS)" \
+ CROSS_COMPILE="$(TARGET_CROSS)" \
ARCH=$(UBOOT_ARCH)
# Helper function to fill the U-Boot config.h file.
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 10/18] barebox: remove now-redundant ccache handling
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (8 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 09/18] uboot: " Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 11/18] cryptodev-linux: remove now-redundant fix-ccache-compile patch Arnout Vandecappelle
` (8 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
v2: No change
---
boot/barebox/barebox.mk | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/boot/barebox/barebox.mk b/boot/barebox/barebox.mk
index e45976d..00e1537 100644
--- a/boot/barebox/barebox.mk
+++ b/boot/barebox/barebox.mk
@@ -52,8 +52,7 @@ else
BAREBOX_ARCH = $(KERNEL_ARCH)
endif
-BAREBOX_MAKE_FLAGS = ARCH=$(BAREBOX_ARCH) CROSS_COMPILE="$(CCACHE) \
- $(TARGET_CROSS)"
+BAREBOX_MAKE_FLAGS = ARCH=$(BAREBOX_ARCH) CROSS_COMPILE="$(TARGET_CROSS)"
BAREBOX_MAKE_ENV = $(TARGET_MAKE_ENV)
ifeq ($(BR2_TARGET_BAREBOX_USE_DEFCONFIG),y)
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 11/18] cryptodev-linux: remove now-redundant fix-ccache-compile patch
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (9 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 10/18] barebox: " Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 12/18] qt5base: remove now-redundant ccache handling Arnout Vandecappelle
` (7 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
v2: No change
---
.../cryptodev-linux/0002-fix-ccache-compile.patch | 20 --------------------
1 file changed, 20 deletions(-)
delete mode 100644 package/cryptodev-linux/0002-fix-ccache-compile.patch
diff --git a/package/cryptodev-linux/0002-fix-ccache-compile.patch b/package/cryptodev-linux/0002-fix-ccache-compile.patch
deleted file mode 100644
index 058fb43..0000000
--- a/package/cryptodev-linux/0002-fix-ccache-compile.patch
+++ /dev/null
@@ -1,20 +0,0 @@
-Fix compilation with ccache enabled
-
-When ccache is enabled the CROSS_COMPILE variable contains a space, so
-it must be properly quoted.
-
-Signed-off-by: Doug Kehn <rdkehn@yahoo.com>
-
-Index: cryptodev-linux-1.7/Makefile
-===================================================================
---- cryptodev-linux-1.7.orig/Makefile
-+++ cryptodev-linux-1.7/Makefile
-@@ -18,7 +18,7 @@ ifneq (${ARCH},)
- KERNEL_MAKE_OPTS += ARCH=${ARCH}
- endif
- ifneq (${CROSS_COMPILE},)
--KERNEL_MAKE_OPTS += CROSS_COMPILE=${CROSS_COMPILE}
-+KERNEL_MAKE_OPTS += CROSS_COMPILE="${CROSS_COMPILE}"
- endif
-
- build: version.h
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 12/18] qt5base: remove now-redundant ccache handling
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (10 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 11/18] cryptodev-linux: remove now-redundant fix-ccache-compile patch Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 13/18] package-cmake: remove now-redundant target ccache support Arnout Vandecappelle
` (6 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
Note that ccache was not and still is not used for the host tools (e.g.
qmake). This commit only affects the target builds.
v2: No change
---
package/qt5/qt5base/0002-mkspecs-files.patch | 6 ++-
.../qt5/qt5base/0009-fix-build-with-ccache.patch | 49 ----------------------
package/qt5/qt5base/qt5base.mk | 1 -
3 files changed, 4 insertions(+), 52 deletions(-)
delete mode 100644 package/qt5/qt5base/0009-fix-build-with-ccache.patch
diff --git a/package/qt5/qt5base/0002-mkspecs-files.patch b/package/qt5/qt5base/0002-mkspecs-files.patch
index 09c1b67..adeabdf 100644
--- a/package/qt5/qt5base/0002-mkspecs-files.patch
+++ b/package/qt5/qt5base/0002-mkspecs-files.patch
@@ -9,6 +9,8 @@ allows us to easily pass the cross-compiler paths and flags from our
qt5.mk.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+[Arnout: remove ccache support]
+Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Index: b/mkspecs/devices/linux-buildroot-g++/qmake.conf
===================================================================
@@ -18,8 +20,8 @@ Index: b/mkspecs/devices/linux-buildroot-g++/qmake.conf
+include(../common/linux_device_pre.conf)
+
+# modifications to g++-unix.conf
-+QMAKE_CC = $${BR_CCACHE} $${CROSS_COMPILE}gcc
-+QMAKE_CXX = $${BR_CCACHE} $${CROSS_COMPILE}g++
++QMAKE_CC = $${CROSS_COMPILE}gcc
++QMAKE_CXX = $${CROSS_COMPILE}g++
+
+#modifications to gcc-base.conf
+QMAKE_CFLAGS += $${BR_COMPILER_CFLAGS}
diff --git a/package/qt5/qt5base/0009-fix-build-with-ccache.patch b/package/qt5/qt5base/0009-fix-build-with-ccache.patch
deleted file mode 100644
index 896c0ef..0000000
--- a/package/qt5/qt5base/0009-fix-build-with-ccache.patch
+++ /dev/null
@@ -1,49 +0,0 @@
-Fix QMAKE_CXX/CROSS_COMPILE verification with ccache
-
-The use of ccache leads to QMAKE_CXX definitions of the form:
-
- QMAKE_CXX = $${CCACHE} $${CROSS_COMPILE}g++
-
-The previous test required QMAKE_CXX to be a single valid (absolute or
-QMAKE_PATH_ENV-relative) path to an existing file, which was not
-compatible with definitions of QMAKE_CXX like the one above.
-
-Fix this by using only the first value in QMAKE_CXX, which usually
-points to the compiler executable, or to the ccache executable in the
-above case.
-
-Signed-off-by: Beno?t Th?baudeau <benoit@wsystem.com>
----
- mkspecs/features/device_config.prf | 9 +++++++--
- 1 file changed, 7 insertions(+), 2 deletions(-)
-
-diff --git a/mkspecs/features/device_config.prf b/mkspecs/features/device_config.prf
-index cd3a0cf..eee4ac6 100644
---- a/mkspecs/features/device_config.prf
-+++ b/mkspecs/features/device_config.prf
-@@ -19,10 +19,15 @@ defineTest(deviceSanityCheckCompiler) {
- else: \
- sfx =
-
-+ # Build the compiler filename using the first value in QMAKE_CXX in order to
-+ # support tools like ccache, which give QMAKE_CXX values of the form:
-+ # ccache <path_to_compiler>
-+ compiler = $$first(QMAKE_CXX)$$sfx
-+
- # Check if the binary exists with an absolute path. Do this check
- # before the CROSS_COMPILE empty check below to allow the mkspec
- # to derive the compiler path from other device options.
-- exists($$QMAKE_CXX$$sfx):return()
-+ exists($$compiler):return()
-
- # Check for possible reasons of failure
- # check if CROSS_COMPILE device-option is set
-@@ -31,7 +36,7 @@ defineTest(deviceSanityCheckCompiler) {
- # Check if QMAKE_CXX points to an executable.
- ensurePathEnv()
- for (dir, QMAKE_PATH_ENV) {
-- exists($$dir/$${QMAKE_CXX}$$sfx): \
-+ exists($$dir/$${compiler}): \
- return()
- }
-
diff --git a/package/qt5/qt5base/qt5base.mk b/package/qt5/qt5base/qt5base.mk
index e783b4d..570cd35 100644
--- a/package/qt5/qt5base/qt5base.mk
+++ b/package/qt5/qt5base/qt5base.mk
@@ -189,7 +189,6 @@ define QT5BASE_CONFIGURE_CMDS
-nomake tests \
-device buildroot \
-device-option CROSS_COMPILE="$(TARGET_CROSS)" \
- -device-option BR_CCACHE="$(CCACHE)" \
-device-option BR_COMPILER_CFLAGS="$(TARGET_CFLAGS) $(QT5BASE_EXTRA_CFLAGS)" \
-device-option BR_COMPILER_CXXFLAGS="$(TARGET_CXXFLAGS) $(QT5BASE_EXTRA_CFLAGS)" \
$(QT5BASE_CONFIGURE_OPTS) \
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 13/18] package-cmake: remove now-redundant target ccache support
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (11 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 12/18] qt5base: remove now-redundant ccache handling Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 14/18] qpid-proton: remove now-redundant ccache handling patch Arnout Vandecappelle
` (5 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
All the complexity with the different ways that CMAKE_C_COMPILER and
CMAKE_C_COMPILER_ARG1 can be set are no longer needed, it's all handled
by the toolchain wrapper now.
Note that it is still necessary to handle this for the host build.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
v2: Rebased on master
---
package/pkg-cmake.mk | 5 ++---
support/misc/toolchainfile.cmake.in | 32 ++------------------------------
2 files changed, 4 insertions(+), 33 deletions(-)
diff --git a/package/pkg-cmake.mk b/package/pkg-cmake.mk
index 6bc13d4..81dcfcc 100644
--- a/package/pkg-cmake.mk
+++ b/package/pkg-cmake.mk
@@ -98,7 +98,6 @@ define $(2)_CONFIGURE_CMDS
-DBUILD_TESTS=OFF \
-DBUILD_TESTING=OFF \
-DBUILD_SHARED_LIBS=$$(if $$(BR2_STATIC_LIBS),OFF,ON) \
- -DUSE_CCACHE=$$(if $$(BR2_CCACHE),ON,OFF) \
$$(CMAKE_QUIET) \
$$($$(PKG)_CONF_OPTS) \
)
@@ -245,8 +244,8 @@ $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake:
-e 's#@@TARGET_CFLAGS@@#$(call qstrip,$(TARGET_CFLAGS))#' \
-e 's#@@TARGET_CXXFLAGS@@#$(call qstrip,$(TARGET_CXXFLAGS))#' \
-e 's#@@TARGET_LDFLAGS@@#$(call qstrip,$(TARGET_LDFLAGS))#' \
- -e 's#@@TARGET_CC_NOCCACHE@@#$(subst $(HOST_DIR)/,,$(call qstrip,$(TARGET_CC_NOCCACHE)))#' \
- -e 's#@@TARGET_CXX_NOCCACHE@@#$(subst $(HOST_DIR)/,,$(call qstrip,$(TARGET_CXX_NOCCACHE)))#' \
+ -e 's#@@TARGET_CC@@#$(subst $(HOST_DIR)/,,$(call qstrip,$(TARGET_CC)))#' \
+ -e 's#@@TARGET_CXX@@#$(subst $(HOST_DIR)/,,$(call qstrip,$(TARGET_CXX)))#' \
-e 's#@@CMAKE_SYSTEM_PROCESSOR@@#$(call qstrip,$(CMAKE_SYSTEM_PROCESSOR))#' \
$(TOPDIR)/support/misc/toolchainfile.cmake.in \
> $@
diff --git a/support/misc/toolchainfile.cmake.in b/support/misc/toolchainfile.cmake.in
index cd41254..5cf381e 100644
--- a/support/misc/toolchainfile.cmake.in
+++ b/support/misc/toolchainfile.cmake.in
@@ -27,33 +27,5 @@ set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(ENV{PKG_CONFIG_SYSROOT_DIR} "${RELOCATED_HOST_DIR}/@@STAGING_SUBDIR@@")
# This toolchain file can be used both inside and outside Buildroot.
-# * When used inside Buildroot, ccache support is explicitly driven using the
-# USE_CCACHE variable.
-# * When used outside Buildroot (i.e. when USE_CCACHE is not defined), ccache
-# support is automatically enabled if the ccache program is available.
-if(DEFINED USE_CCACHE)
- if(USE_CCACHE)
- set(CMAKE_ASM_COMPILER "${RELOCATED_HOST_DIR}/@@TARGET_CC_NOCCACHE@@")
- set(CMAKE_C_COMPILER "${RELOCATED_HOST_DIR}/usr/bin/ccache")
- set(CMAKE_CXX_COMPILER "${RELOCATED_HOST_DIR}/usr/bin/ccache")
- set(CMAKE_C_COMPILER_ARG1 "${RELOCATED_HOST_DIR}/@@TARGET_CC_NOCCACHE@@")
- set(CMAKE_CXX_COMPILER_ARG1 "${RELOCATED_HOST_DIR}/@@TARGET_CXX_NOCCACHE@@")
- else()
- set(CMAKE_C_COMPILER "${RELOCATED_HOST_DIR}/@@TARGET_CC_NOCCACHE@@")
- set(CMAKE_CXX_COMPILER "${RELOCATED_HOST_DIR}/@@TARGET_CXX_NOCCACHE@@")
- endif()
-else()
- find_program(CCACHE ccache HINTS "${RELOCATED_HOST_DIR}/usr/bin")
- if(CCACHE)
- set(CMAKE_ASM_COMPILER "${RELOCATED_HOST_DIR}/@@TARGET_CC_NOCCACHE@@")
- set(CMAKE_C_COMPILER "${CCACHE}")
- set(CMAKE_CXX_COMPILER "${CCACHE}")
- set(CMAKE_C_COMPILER_ARG1 "${RELOCATED_HOST_DIR}/@@TARGET_CC_NOCCACHE@@")
- set(CMAKE_CXX_COMPILER_ARG1 "${RELOCATED_HOST_DIR}/@@TARGET_CXX_NOCCACHE@@")
- message(STATUS "ccache program has been found and will be used for the build.")
- message(STATUS " To disable ccache, add -DUSE_CCACHE=OFF on the cmake command line.")
- else()
- set(CMAKE_C_COMPILER "${RELOCATED_HOST_DIR}/@@TARGET_CC_NOCCACHE@@")
- set(CMAKE_CXX_COMPILER "${RELOCATED_HOST_DIR}/@@TARGET_CXX_NOCCACHE@@")
- endif()
-endif()
+set(CMAKE_C_COMPILER "${RELOCATED_HOST_DIR}/@@TARGET_CC@@")
+set(CMAKE_CXX_COMPILER "${RELOCATED_HOST_DIR}/@@TARGET_CXX@@")
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 14/18] qpid-proton: remove now-redundant ccache handling patch
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (12 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 13/18] package-cmake: remove now-redundant target ccache support Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 15/18] Makefile.in: remove now-unused TARGET_CC/CXX_NOCCACHE Arnout Vandecappelle
` (4 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
Although this patch is still relevant for upstream, we don't need it
anymore in buildoort because we no longer have _ARG1.
Note that it would be relevant again if we would add host-qpid-proton.
However, the 0.10 release already has the fix.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
v2: No change
---
...fix-C-compiler-detection-with-_ARG1-_ARG2.patch | 52 ----------------------
1 file changed, 52 deletions(-)
delete mode 100644 package/qpid-proton/0001-proton-c-fix-C-compiler-detection-with-_ARG1-_ARG2.patch
diff --git a/package/qpid-proton/0001-proton-c-fix-C-compiler-detection-with-_ARG1-_ARG2.patch b/package/qpid-proton/0001-proton-c-fix-C-compiler-detection-with-_ARG1-_ARG2.patch
deleted file mode 100644
index 16f4aba..0000000
--- a/package/qpid-proton/0001-proton-c-fix-C-compiler-detection-with-_ARG1-_ARG2.patch
+++ /dev/null
@@ -1,52 +0,0 @@
-From f24be9ae9ab01c7507a366e9d5529c444f0b7edd Mon Sep 17 00:00:00 2001
-From: Luca Ceresoli <luca@lucaceresoli.net>
-Date: Fri, 10 Jul 2015 10:13:47 +0200
-Subject: [PATCH] proton-c: fix C compiler detection with _ARG1/_ARG2
-
-The C compiler commandline in CMake is composed by the concatenation of
-CMAKE_C_COMPILER + CMAKE_C_COMPILER_ARG1 + CMAKE_C_COMPILER_ARG2.
-
-In most use cases the two additional argument variables are empty, thus
-CMAKE_C_COMPILER can be used without any noticeable difference.
-
-The Buildroot embedded Linux build system [0], however, optionally exploits the
-CMAKE_C_COMPILER_ARG1 variable to speed up the cross-compilation of CMake-based
-packages using ccache. It does so by setting [1]:
-
- CMAKE_C_COMPILER = /path/to/ccache
- CMAKE_C_COMPILER_ARG1 = /path/to/cross-gcc
-
-This works fine with other CMake-based packages, but proton-c's CMakeLists.txt
-calls gcc to extract the compiler version. It does so by calling
-"${CMAKE_C_COMPILER} -dumpversion", without honoring the two extra arguments.
-Within Buildroot with ccache enabled, this means calling
-"/path/to/ccache -dumpversion", which fails with the error:
-
- ccache: invalid option -- 'd'
-
-Fix the compiler check by adding the two arguments.
-
-[0] http://buildroot.net/
-[1] http://git.buildroot.net/buildroot/tree/support/misc/toolchainfile.cmake.in?id=2015.05
-
-Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
----
- proton-c/CMakeLists.txt | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/proton-c/CMakeLists.txt b/proton-c/CMakeLists.txt
-index 93449a9..8c31a89 100644
---- a/proton-c/CMakeLists.txt
-+++ b/proton-c/CMakeLists.txt
-@@ -218,7 +218,7 @@ if (CMAKE_COMPILER_IS_GNUCC)
- set (COMPILE_LANGUAGE_FLAGS "-std=c99")
- set (COMPILE_PLATFORM_FLAGS "-std=gnu99")
-
-- execute_process(COMMAND ${CMAKE_C_COMPILER} -dumpversion OUTPUT_VARIABLE GCC_VERSION
-+ execute_process(COMMAND ${CMAKE_C_COMPILER} ${CMAKE_C_COMPILER_ARG1} ${CMAKE_C_COMPILER_ARG2} -dumpversion OUTPUT_VARIABLE GCC_VERSION
- OUTPUT_STRIP_TRAILING_WHITESPACE)
- if (${GCC_VERSION} VERSION_LESS "4.3.0")
- # Only a concern if contibuting code back.
---
-1.9.1
-
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 15/18] Makefile.in: remove now-unused TARGET_CC/CXX_NOCCACHE
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (13 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 14/18] qpid-proton: remove now-redundant ccache handling patch Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 16/18] toolchain-wrapper: support change of BR2_CCACHE Arnout Vandecappelle
` (3 subsequent siblings)
18 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
---
v2: No change
---
package/Makefile.in | 3 ---
1 file changed, 3 deletions(-)
diff --git a/package/Makefile.in b/package/Makefile.in
index 4b1ce7f..7dc0eac 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -186,9 +186,6 @@ TARGET_READELF = $(TARGET_CROSS)readelf
TARGET_OBJCOPY = $(TARGET_CROSS)objcopy
TARGET_OBJDUMP = $(TARGET_CROSS)objdump
-TARGET_CC_NOCCACHE := $(TARGET_CC)
-TARGET_CXX_NOCCACHE := $(TARGET_CXX)
-
ifeq ($(BR2_STRIP_strip),y)
STRIP_STRIP_DEBUG := --strip-debug
STRIP_STRIP_UNNEEDED := --strip-unneeded
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 16/18] toolchain-wrapper: support change of BR2_CCACHE
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (14 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 15/18] Makefile.in: remove now-unused TARGET_CC/CXX_NOCCACHE Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 14:24 ` Romain Naour
2015-10-04 12:28 ` [Buildroot] [PATCH v2 17/18] ccache: use mtime for external toolchain, CONF_OPTS for internal toolchain Arnout Vandecappelle
` (2 subsequent siblings)
18 siblings, 1 reply; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
By moving the ccache call to the toolchain wrapper, the following
scenario no longer works:
make foo-dirclean all BR2_CCACHE=
That's a sometimes useful call to check if some failure is perhaps
caused by ccache.
We can enable this scenario again by exporting BR_NO_CCACHE when
BR2_CCACHE is not set, and by handling this in the toolchain wrapper.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v2: No change, but no longer RFC.
---
Makefile | 2 ++
toolchain/toolchain-wrapper.c | 15 +++++++++++----
2 files changed, 13 insertions(+), 4 deletions(-)
diff --git a/Makefile b/Makefile
index fdbca02..e8e2f02 100644
--- a/Makefile
+++ b/Makefile
@@ -375,6 +375,8 @@ BR_CACHE_DIR = $(call qstrip,$(BR2_CCACHE_DIR))
export BR_CACHE_DIR
HOSTCC := $(CCACHE) $(HOSTCC)
HOSTCXX := $(CCACHE) $(HOSTCXX)
+else
+export BR_NO_CCACHE
endif
# Scripts in support/ or post-build scripts may need to reference
diff --git a/toolchain/toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
index b5fe1d3..cf66fac 100644
--- a/toolchain/toolchain-wrapper.c
+++ b/toolchain/toolchain-wrapper.c
@@ -95,7 +95,7 @@ static void check_unsafe_path(const char *path, int paranoid)
int main(int argc, char **argv)
{
- char **args, **cur;
+ char **args, **cur, **exec_args;
char *relbasedir, *absbasedir;
char *progpath = argv[0];
char *basename;
@@ -237,6 +237,13 @@ int main(int argc, char **argv)
/* finish with NULL termination */
*cur = NULL;
+ exec_args = args;
+#ifdef BR_CCACHE
+ if (getenv("BR_NO_CCACHE"))
+ /* Skip the ccache call */
+ exec_args++;
+#endif
+
/* Debug the wrapper to see actual arguments passed to
* the compiler:
* unset, empty, or 0: do not trace
@@ -247,14 +254,14 @@ int main(int argc, char **argv)
debug = atoi(env_debug);
if (debug > 0) {
fprintf(stderr, "Toolchain wrapper executing:");
- for (i = 0; args[i]; i++)
+ for (i = 0; exec_args[i]; i++)
fprintf(stderr, "%s'%s'",
- (debug == 2) ? "\n " : " ", args[i]);
+ (debug == 2) ? "\n " : " ", exec_args[i]);
fprintf(stderr, "\n");
}
}
- if (execv(args[0], args))
+ if (execv(exec_args[0], exec_args))
perror(path);
free(args);
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 17/18] ccache: use mtime for external toolchain, CONF_OPTS for internal toolchain
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (15 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 16/18] toolchain-wrapper: support change of BR2_CCACHE Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 14:55 ` Romain Naour
2015-10-04 15:25 ` [Buildroot] [PATCH v3] " Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 18/18] [RFC] ccache: support changing the output directory Arnout Vandecappelle
2015-10-04 16:01 ` [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Peter Korsgaard
18 siblings, 2 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
Our current ccache disables hashing of the compiler executable itself,
because using the default 'mtime' doesn't work in buildroot: we always
rebuild the compiler, so the mtime is always different, so the cache
always misses.
However, in the current situation, if a user changes the compiler
configuration (which would result in the compiler generating different
object files than before) and does 'make clean all', ccache may in fact
reuse object files from the previous run. This rarely gives problems,
because
(1) the cache expires quite quickly (it's only 1GB by default),
(2) radically changing compiler options will cause cache misses because
different header files are used,
(3) many compiler changes (e.g. changing -mtune) have little practical
effect because the resulting code is usually still compatible,
(4) we currently don't use CCACHE_BASEDIR, and almost all object files
will contain an absolute path (e.g. in debug info), so when
building in a different directory, most of it will miss,
(5) we do mostly build test, and many of the potential problems only
appear at runtime.
Still, when ccache _does_ use the wrong cached object files, the
effects are really weird and hard to debug. Also, we want reproducible
builds and obviously the above makes builds non-reproducible. So we
have a FAQ entry that warns against using ccache and tells the user to
clear the cache in case of problems.
Now that ccache is called from the toolchain wrapper, it is in fact
possible to at least use the 'mtime' compiler hash for the external
toolchain and for the host-gcc. Indeed, in this case, the compiler
executable comes from a tarball so the mtime will be a good reference
for it state. Therefore, the patch (sed script) that changes the
default from 'mtime' to 'none' is removed.
For the internal toolchain, we can do better by providing a hash of
the relevant toolchain options. We are only interested in things that
affect the compiler itself, because ccache also processes the header
files and it doesn't look at libraries because it doesn't cache the
link step, just compilation. Everything that affects the compiler
itself can nicely be summarised in $(HOST_GCC_FINAL_CONF_OPTS). Of
course, also the compiler source itself is relevant, so the source
tarball and all the patches are included in the hash. For this purpose,
a new HOST_GCC_XTENSA_OVERLAY_TAR is introduced.
The following procedure tests the ccache behaviour:
Use this defconfig:
BR2_arm=y
BR2_CCACHE=y
make
readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os
-> Tag_CPU_name: "ARM926EJ-S"
Now make menuconfig, change variant into BR2_cortex_a9
make clean; make
readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os
-> Tag_CPU_name: "ARM926EJ-S"
should be "Cortex-A9"
After this commit, it is "Cortex-A9".
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Danomi Manchego <danomimanchego123@gmail.com>
Cc: K?roly Kasza <kaszak@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Romain Naour <romain.naour@openwide.fr>
---
v2: - Add BR2_GLOBAL_PATCH_DIR to the patches used to generate the hash.
[Samuel]
- Use $(wildcard ...) for the patches, so missing patch dir (e.g. for
OVERRIDE_SRCDIR) doesn't lead to an error. [Arnout]
- Remove redundant newline [Romain]
---
package/ccache/ccache.mk | 5 -----
package/gcc/gcc.mk | 29 +++++++++++++++++++++++++++--
toolchain/toolchain-wrapper.c | 12 ++++++++++++
3 files changed, 39 insertions(+), 7 deletions(-)
diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
index 3bd2803..52cf309 100644
--- a/package/ccache/ccache.mk
+++ b/package/ccache/ccache.mk
@@ -26,15 +26,10 @@ HOST_CCACHE_CONF_OPTS += --with-bundled-zlib
# is already used by autotargets for the ccache package.
# BR_CACHE_DIR is exported by Makefile based on config option
# BR2_CCACHE_DIR.
-# - ccache shouldn't use the compiler binary mtime to detect a change in
-# the compiler, because in the context of Buildroot, that completely
-# defeats the purpose of ccache. Of course, that leaves the user
-# responsible for purging its cache when the compiler changes.
# - Change hard-coded last-ditch default to match path in .config, to avoid
# the need to specify BR_CACHE_DIR when invoking ccache directly.
define HOST_CCACHE_PATCH_CONFIGURATION
sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/ccache.c
- sed -i 's,conf->compiler_check = x_strdup("mtime"),conf->compiler_check = x_strdup("none"),' $(@D)/conf.c
sed -i 's,"%s/.ccache","$(BR_CACHE_DIR)",' $(@D)/conf.c
endef
diff --git a/package/gcc/gcc.mk b/package/gcc/gcc.mk
index b9da398..103cbe1 100644
--- a/package/gcc/gcc.mk
+++ b/package/gcc/gcc.mk
@@ -23,9 +23,10 @@ GCC_SOURCE ?= gcc-$(GCC_VERSION).tar.bz2
# Xtensa special hook
#
+HOST_GCC_XTENSA_OVERLAY_TAR = $(BR2_XTENSA_OVERLAY_DIR)/xtensa_$(call qstrip,$(BR2_XTENSA_CORE_NAME)).tar
+
define HOST_GCC_XTENSA_OVERLAY_EXTRACT
- tar xf $(BR2_XTENSA_OVERLAY_DIR)/xtensa_$(call qstrip,\
- $(BR2_XTENSA_CORE_NAME)).tar -C $(@D) --strip-components=1 gcc
+ tar xf $(HOST_GCC_XTENSA_OVERLAY_TAR) -C $(@D) --strip-components=1 gcc
endef
#
@@ -237,6 +238,30 @@ endif
HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS += -DBR_CROSS_PATH_SUFFIX='".real"'
+ifeq ($(BR2_CCACHE),y)
+HOST_GCC_COMMON_CCACHE_HASH_FILES += $(DL_DIR)/$(GCC_SOURCE)
+# Cfr. PATCH_BASE_DIRS in .stamp_patched, but we catch both versioned and
+# unversioned patches unconditionally
+HOST_GCC_COMMON_CCACHE_HASH_FILES += \
+ $(wildcard \
+ package/gcc/$(GCC_VERSION)/*.patch \
+ $(addsuffix $((PKG)_RAWNAME)/$(GCC_VERSION)/*.patch,$(call qstrip,$(BR2_GLOBAL_PATCH_DIR))) \
+ $(addsuffix $((PKG)_RAWNAME)/*.patch,$(call qstrip,$(BR2_GLOBAL_PATCH_DIR))))
+ifeq ($(BR2_xtensa),y)
+HOST_GCC_COMMON_CCACHE_HASH_FILES += $(HOST_GCC_XTENSA_OVERLAY_TAR)
+endif
+ifeq ($(ARCH),powerpc)
+ifneq ($(BR2_SOFT_FLOAT),)
+HOST_GCC_COMMON_CCACHE_HASH_FILES += package/gcc/$(GCC_VERSION)/1000-powerpc-link-with-math-lib.patch.conditional
+endif
+endif
+
+HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE_HASH=\"`\
+ printf '%s' $($(PKG)_CONF_OPTS) $(GCC_VERSION) \
+ | sha256sum - $(HOST_GCC_COMMON_CCACHE_HASH_FILES) \
+ | cut -c -64 | tr -d '\n'`\"
+endif # BR2_CCACHE
+
# The LTO support in gcc creates wrappers for ar, ranlib and nm which load
# the lto plugin. These wrappers are called *-gcc-ar, *-gcc-ranlib, and
# *-gcc-nm and should be used instead of the real programs when -flto is
diff --git a/toolchain/toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
index cf66fac..7ee3028 100644
--- a/toolchain/toolchain-wrapper.c
+++ b/toolchain/toolchain-wrapper.c
@@ -254,6 +254,10 @@ int main(int argc, char **argv)
debug = atoi(env_debug);
if (debug > 0) {
fprintf(stderr, "Toolchain wrapper executing:");
+#ifdef BR_CCACHE_HASH
+ fprintf(stderr, "CCACHE_COMPILERCHECK=string:" BR_CCACHE_HASH "%s",
+ (debug == 2) ? "\n " : " ");
+#endif
for (i = 0; exec_args[i]; i++)
fprintf(stderr, "%s'%s'",
(debug == 2) ? "\n " : " ", exec_args[i]);
@@ -261,6 +265,14 @@ int main(int argc, char **argv)
}
}
+#ifdef BR_CCACHE_HASH
+ /* Allow compilercheck to be overridden through the environment */
+ if (setenv("CCACHE_COMPILERCHECK", "string:" BR_CCACHE_HASH, 0)) {
+ perror(__FILE__ ": Failed to set CCACHE_COMPILERCHECK");
+ return 3;
+ }
+#endif
+
if (execv(exec_args[0], exec_args))
perror(path);
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 18/18] [RFC] ccache: support changing the output directory
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (16 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 17/18] ccache: use mtime for external toolchain, CONF_OPTS for internal toolchain Arnout Vandecappelle
@ 2015-10-04 12:28 ` Arnout Vandecappelle
2015-10-04 15:25 ` [Buildroot] [PATCH v3] " Arnout Vandecappelle
2015-10-04 16:01 ` [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Peter Korsgaard
18 siblings, 1 reply; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 12:28 UTC (permalink / raw)
To: buildroot
When building in a different output directory than the original build,
there will currently be a lot of ccache misses because in many cases
there is some -I/... absolute path in the compilation. ccache has an
option CCACHE_BASEDIR to substitute absolute paths with relative paths,
so they wil be the same in the hash (and in the output).
Since there are some disadvantages to this path rewriting, it is made
optional as BR2_CCACHE_USE_BASEDIR. It defaults to y because the
usefulness of ccache is severely reduced without this option.
In addition to CCACHE_BASEDIR, we also substitute away the occurences
of $(HOST_DIR) in the calculation of the compiler hash. This is done
regardless of the setting of BR2_CCACHE_USE_BASEDIR because it's
quite harmless.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
Perhaps we should use $(HOST_DIR) instead of $(BASE_DIR) as
CCACHE_BASEDIR, because most of the time an absolute path will point
into the host dir. Not always, though (e.g. references to LINUX_DIR).
And most people will have $(HOST_DIR) inside $(BASE_DIR) so the latter
works well for them.
It's hard to be sure with ccache, but in my tests I still had the
impression that there were less cache hits when rebuilding in a
different output directory than when rebuilding in the same directory.
Oh well...
---
Config.in | 27 +++++++++++++++++++++++++++
docs/manual/ccache-support.txt | 20 ++++++++++++++++++++
package/gcc/gcc.mk | 4 +++-
toolchain/toolchain-wrapper.c | 11 +++++++++++
toolchain/toolchain-wrapper.mk | 4 ++++
5 files changed, 65 insertions(+), 1 deletion(-)
diff --git a/Config.in b/Config.in
index 74aeb46..d795361 100644
--- a/Config.in
+++ b/Config.in
@@ -286,6 +286,33 @@ config BR2_CCACHE_INITIAL_SETUP
These initial settings are applied after ccache has been compiled.
+config BR2_CCACHE_USE_BASEDIR
+ bool "Use relative paths"
+ default y
+ help
+ Allow ccache to convert absolute paths within the output
+ directory into relative paths.
+
+ During the build, many -I include directives are given with
+ an absolute path. These absolute paths end up in the hashes
+ that are computed by ccache. Therefore, when you build from a
+ different directory, the hash will be different and the
+ cached object will not be used.
+
+ To improve cache performance, set this option to y. This
+ allows ccache to rewrite absolute paths within the output
+ directory into relative paths. Note that only paths within
+ the output directory will be rewritten; therefore, if you
+ change BR2_HOST_DIR to point outside the output directory and
+ subsequently move it to a different location, this will lead
+ to cache misses.
+
+ This option has as a result that the debug information in the
+ object files also has only relative paths. Therefore, make
+ sure you cd to the build directory before starting gdb. See
+ the section "COMPILING IN DIFFERENT DIRECTORIES" in the
+ ccache manual for more information.
+
endif
config BR2_DEPRECATED
diff --git a/docs/manual/ccache-support.txt b/docs/manual/ccache-support.txt
index 992471d..f6746ad 100644
--- a/docs/manual/ccache-support.txt
+++ b/docs/manual/ccache-support.txt
@@ -33,3 +33,23 @@ make CCACHE_OPTIONS="--max-size=5G" ccache-options
# zero statistics counters
make CCACHE_OPTIONS="--zero-stats" ccache-options
-----------------
+
++ccache+ makes a hash of the source files and of the compiler options.
+If a compiler option is different, the cached object file will not be
+used. Many compiler options, however, contain an absolute path to the
+staging directory. Because of this, building in a different output
+directory would lead to many cache misses.
+
+To avoid this issue, buildroot has the +Use relative paths+ option
+(+BR2_CCACHE_USE_BASEDIR+). This will rewrite all absolute paths that
+point inside the output directory into relative paths. Thus, changing
+the output directory no longer leads to cache misses.
+
+A disadvantage of the relative paths is that they also end up to be
+relative paths in the object file. Therefore, for example, the debugger
+will no longer find the file, unless you cd to the output directory
+first.
+
+See https://ccache.samba.org/manual.html#_compiling_in_different_directories[the
+ccache manual's section on "Compiling in different directories"] for
+more details about this rewriting of absolute paths.
diff --git a/package/gcc/gcc.mk b/package/gcc/gcc.mk
index 103cbe1..420c4e9 100644
--- a/package/gcc/gcc.mk
+++ b/package/gcc/gcc.mk
@@ -256,8 +256,10 @@ HOST_GCC_COMMON_CCACHE_HASH_FILES += package/gcc/$(GCC_VERSION)/1000-powerpc-lin
endif
endif
+# _CONF_OPTS contains some references to the absolute path of $(HOST_DIR),
+# so substitute those away.
HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE_HASH=\"`\
- printf '%s' $($(PKG)_CONF_OPTS) $(GCC_VERSION) \
+ printf '%s' $(subst $(HOST_DIR), at HOST_DIR@,$($(PKG)_CONF_OPTS)) \
| sha256sum - $(HOST_GCC_COMMON_CCACHE_HASH_FILES) \
| cut -c -64 | tr -d '\n'`\"
endif # BR2_CCACHE
diff --git a/toolchain/toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
index 7ee3028..af5768b 100644
--- a/toolchain/toolchain-wrapper.c
+++ b/toolchain/toolchain-wrapper.c
@@ -258,6 +258,10 @@ int main(int argc, char **argv)
fprintf(stderr, "CCACHE_COMPILERCHECK=string:" BR_CCACHE_HASH "%s",
(debug == 2) ? "\n " : " ");
#endif
+#ifdef BR_CCACHE_BASEDIR
+ fprintf(stderr, "CCACHE_BASEDIR=" BR_CCACHE_BASEDIR "%s",
+ (debug == 2) ? "\n " : " ");
+#endif
for (i = 0; exec_args[i]; i++)
fprintf(stderr, "%s'%s'",
(debug == 2) ? "\n " : " ", exec_args[i]);
@@ -272,6 +276,13 @@ int main(int argc, char **argv)
return 3;
}
#endif
+#ifdef BR_CCACHE_BASEDIR
+ /* Allow compilercheck to be overridden through the environment */
+ if (setenv("CCACHE_BASEDIR", BR_CCACHE_BASEDIR, 0)) {
+ perror(__FILE__ ": Failed to set CCACHE_BASEDIR");
+ return 3;
+ }
+#endif
if (execv(exec_args[0], exec_args))
perror(path);
diff --git a/toolchain/toolchain-wrapper.mk b/toolchain/toolchain-wrapper.mk
index 0fd2b86..40d29a2 100644
--- a/toolchain/toolchain-wrapper.mk
+++ b/toolchain/toolchain-wrapper.mk
@@ -20,6 +20,10 @@ ifeq ($(BR2_CCACHE),y)
TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE='"$(CCACHE)"'
endif
+ifeq ($(BR2_CCACHE_USE_BASEDIR),y)
+TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE_BASEDIR='"$(BASE_DIR)"'
+endif
+
# For simplicity, build directly into the install location
define TOOLCHAIN_BUILD_WRAPPER
$(Q)mkdir -p $(HOST_DIR)/usr/bin
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 16/18] toolchain-wrapper: support change of BR2_CCACHE
2015-10-04 12:28 ` [Buildroot] [PATCH v2 16/18] toolchain-wrapper: support change of BR2_CCACHE Arnout Vandecappelle
@ 2015-10-04 14:24 ` Romain Naour
0 siblings, 0 replies; 26+ messages in thread
From: Romain Naour @ 2015-10-04 14:24 UTC (permalink / raw)
To: buildroot
Arnout,
Le 04/10/2015 14:28, Arnout Vandecappelle (Essensium/Mind) a ?crit :
> By moving the ccache call to the toolchain wrapper, the following
> scenario no longer works:
>
> make foo-dirclean all BR2_CCACHE=
>
> That's a sometimes useful call to check if some failure is perhaps
> caused by ccache.
>
> We can enable this scenario again by exporting BR_NO_CCACHE when
> BR2_CCACHE is not set, and by handling this in the toolchain wrapper.
>
> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
I tried this patch with the whole series applied (I removed the hash to ease the
reading)
$ make O=test/ccache/ busybox BR2_DEBUG_WRAPPER=2
Toolchain wrapper executing:CCACHE_COMPILERCHECK=string:[hash]
CCACHE_BASEDIR=/home/naourr/git/buildroot/test/ccache
'/home/naourr/git/buildroot/test/ccache/host/usr/bin/ccache'
'/home/naourr/git/buildroot/test/ccache/host/usr/bin/i686-buildroot-linux-uclibc-gcc.real'
'--sysroot'
'/home/naourr/git/buildroot/test/ccache/host/usr/i686-buildroot-linux-uclibc/sysroot'
'-E'
'-xc'
'-'
$ make O=test/ccache/ busybox BR2_DEBUG_WRAPPER=2 BR2_CCACHE=
Toolchain wrapper executing:CCACHE_COMPILERCHECK=string:[hash]
CCACHE_BASEDIR=/home/naourr/git/buildroot/test/ccache
'/home/naourr/git/buildroot/test/ccache/host/usr/bin/i686-buildroot-linux-uclibc-gcc.real'
'--sysroot'
'/home/naourr/git/buildroot/test/ccache/host/usr/i686-buildroot-linux-uclibc/sysroot'
'-E'
'-xc'
'-'
Tested-by: Romain Naour <romain.naour@openwide.fr>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
Best regards,
Romain
> ---
> v2: No change, but no longer RFC.
> ---
> Makefile | 2 ++
> toolchain/toolchain-wrapper.c | 15 +++++++++++----
> 2 files changed, 13 insertions(+), 4 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index fdbca02..e8e2f02 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -375,6 +375,8 @@ BR_CACHE_DIR = $(call qstrip,$(BR2_CCACHE_DIR))
> export BR_CACHE_DIR
> HOSTCC := $(CCACHE) $(HOSTCC)
> HOSTCXX := $(CCACHE) $(HOSTCXX)
> +else
> +export BR_NO_CCACHE
> endif
>
> # Scripts in support/ or post-build scripts may need to reference
> diff --git a/toolchain/toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
> index b5fe1d3..cf66fac 100644
> --- a/toolchain/toolchain-wrapper.c
> +++ b/toolchain/toolchain-wrapper.c
> @@ -95,7 +95,7 @@ static void check_unsafe_path(const char *path, int paranoid)
>
> int main(int argc, char **argv)
> {
> - char **args, **cur;
> + char **args, **cur, **exec_args;
> char *relbasedir, *absbasedir;
> char *progpath = argv[0];
> char *basename;
> @@ -237,6 +237,13 @@ int main(int argc, char **argv)
> /* finish with NULL termination */
> *cur = NULL;
>
> + exec_args = args;
> +#ifdef BR_CCACHE
> + if (getenv("BR_NO_CCACHE"))
> + /* Skip the ccache call */
> + exec_args++;
> +#endif
> +
> /* Debug the wrapper to see actual arguments passed to
> * the compiler:
> * unset, empty, or 0: do not trace
> @@ -247,14 +254,14 @@ int main(int argc, char **argv)
> debug = atoi(env_debug);
> if (debug > 0) {
> fprintf(stderr, "Toolchain wrapper executing:");
> - for (i = 0; args[i]; i++)
> + for (i = 0; exec_args[i]; i++)
> fprintf(stderr, "%s'%s'",
> - (debug == 2) ? "\n " : " ", args[i]);
> + (debug == 2) ? "\n " : " ", exec_args[i]);
> fprintf(stderr, "\n");
> }
> }
>
> - if (execv(args[0], args))
> + if (execv(exec_args[0], exec_args))
> perror(path);
>
> free(args);
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 17/18] ccache: use mtime for external toolchain, CONF_OPTS for internal toolchain
2015-10-04 12:28 ` [Buildroot] [PATCH v2 17/18] ccache: use mtime for external toolchain, CONF_OPTS for internal toolchain Arnout Vandecappelle
@ 2015-10-04 14:55 ` Romain Naour
2015-10-04 15:25 ` [Buildroot] [PATCH v3] " Arnout Vandecappelle
1 sibling, 0 replies; 26+ messages in thread
From: Romain Naour @ 2015-10-04 14:55 UTC (permalink / raw)
To: buildroot
Arnout,
Le 04/10/2015 14:28, Arnout Vandecappelle (Essensium/Mind) a ?crit :
> Our current ccache disables hashing of the compiler executable itself,
> because using the default 'mtime' doesn't work in buildroot: we always
> rebuild the compiler, so the mtime is always different, so the cache
> always misses.
>
> However, in the current situation, if a user changes the compiler
> configuration (which would result in the compiler generating different
> object files than before) and does 'make clean all', ccache may in fact
> reuse object files from the previous run. This rarely gives problems,
> because
> (1) the cache expires quite quickly (it's only 1GB by default),
> (2) radically changing compiler options will cause cache misses because
> different header files are used,
> (3) many compiler changes (e.g. changing -mtune) have little practical
> effect because the resulting code is usually still compatible,
> (4) we currently don't use CCACHE_BASEDIR, and almost all object files
> will contain an absolute path (e.g. in debug info), so when
> building in a different directory, most of it will miss,
> (5) we do mostly build test, and many of the potential problems only
> appear at runtime.
> Still, when ccache _does_ use the wrong cached object files, the
> effects are really weird and hard to debug. Also, we want reproducible
> builds and obviously the above makes builds non-reproducible. So we
> have a FAQ entry that warns against using ccache and tells the user to
> clear the cache in case of problems.
>
> Now that ccache is called from the toolchain wrapper, it is in fact
> possible to at least use the 'mtime' compiler hash for the external
> toolchain and for the host-gcc. Indeed, in this case, the compiler
> executable comes from a tarball so the mtime will be a good reference
> for it state. Therefore, the patch (sed script) that changes the
> default from 'mtime' to 'none' is removed.
>
> For the internal toolchain, we can do better by providing a hash of
> the relevant toolchain options. We are only interested in things that
> affect the compiler itself, because ccache also processes the header
> files and it doesn't look at libraries because it doesn't cache the
> link step, just compilation. Everything that affects the compiler
> itself can nicely be summarised in $(HOST_GCC_FINAL_CONF_OPTS). Of
> course, also the compiler source itself is relevant, so the source
> tarball and all the patches are included in the hash. For this purpose,
> a new HOST_GCC_XTENSA_OVERLAY_TAR is introduced.
>
> The following procedure tests the ccache behaviour:
>
> Use this defconfig:
> BR2_arm=y
> BR2_CCACHE=y
>
> make
> readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os
> -> Tag_CPU_name: "ARM926EJ-S"
>
> Now make menuconfig, change variant into BR2_cortex_a9
>
> make clean; make
> readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os
> -> Tag_CPU_name: "ARM926EJ-S"
> should be "Cortex-A9"
>
> After this commit, it is "Cortex-A9".
>
> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> Cc: Danomi Manchego <danomimanchego123@gmail.com>
> Cc: K?roly Kasza <kaszak@gmail.com>
> Cc: Samuel Martin <s.martin49@gmail.com>
> Cc: Romain Naour <romain.naour@openwide.fr>
> ---
> v2: - Add BR2_GLOBAL_PATCH_DIR to the patches used to generate the hash.
> [Samuel]
> - Use $(wildcard ...) for the patches, so missing patch dir (e.g. for
> OVERRIDE_SRCDIR) doesn't lead to an error. [Arnout]
> - Remove redundant newline [Romain]
> ---
> package/ccache/ccache.mk | 5 -----
> package/gcc/gcc.mk | 29 +++++++++++++++++++++++++++--
> toolchain/toolchain-wrapper.c | 12 ++++++++++++
> 3 files changed, 39 insertions(+), 7 deletions(-)
>
> diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
> index 3bd2803..52cf309 100644
> --- a/package/ccache/ccache.mk
> +++ b/package/ccache/ccache.mk
> @@ -26,15 +26,10 @@ HOST_CCACHE_CONF_OPTS += --with-bundled-zlib
> # is already used by autotargets for the ccache package.
> # BR_CACHE_DIR is exported by Makefile based on config option
> # BR2_CCACHE_DIR.
> -# - ccache shouldn't use the compiler binary mtime to detect a change in
> -# the compiler, because in the context of Buildroot, that completely
> -# defeats the purpose of ccache. Of course, that leaves the user
> -# responsible for purging its cache when the compiler changes.
> # - Change hard-coded last-ditch default to match path in .config, to avoid
> # the need to specify BR_CACHE_DIR when invoking ccache directly.
> define HOST_CCACHE_PATCH_CONFIGURATION
> sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/ccache.c
> - sed -i 's,conf->compiler_check = x_strdup("mtime"),conf->compiler_check = x_strdup("none"),' $(@D)/conf.c
> sed -i 's,"%s/.ccache","$(BR_CACHE_DIR)",' $(@D)/conf.c
> endef
>
> diff --git a/package/gcc/gcc.mk b/package/gcc/gcc.mk
> index b9da398..103cbe1 100644
> --- a/package/gcc/gcc.mk
> +++ b/package/gcc/gcc.mk
> @@ -23,9 +23,10 @@ GCC_SOURCE ?= gcc-$(GCC_VERSION).tar.bz2
> # Xtensa special hook
> #
>
> +HOST_GCC_XTENSA_OVERLAY_TAR = $(BR2_XTENSA_OVERLAY_DIR)/xtensa_$(call qstrip,$(BR2_XTENSA_CORE_NAME)).tar
> +
> define HOST_GCC_XTENSA_OVERLAY_EXTRACT
> - tar xf $(BR2_XTENSA_OVERLAY_DIR)/xtensa_$(call qstrip,\
> - $(BR2_XTENSA_CORE_NAME)).tar -C $(@D) --strip-components=1 gcc
> + tar xf $(HOST_GCC_XTENSA_OVERLAY_TAR) -C $(@D) --strip-components=1 gcc
> endef
>
> #
> @@ -237,6 +238,30 @@ endif
>
> HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS += -DBR_CROSS_PATH_SUFFIX='".real"'
>
> +ifeq ($(BR2_CCACHE),y)
> +HOST_GCC_COMMON_CCACHE_HASH_FILES += $(DL_DIR)/$(GCC_SOURCE)
> +# Cfr. PATCH_BASE_DIRS in .stamp_patched, but we catch both versioned and
> +# unversioned patches unconditionally
> +HOST_GCC_COMMON_CCACHE_HASH_FILES += \
> + $(wildcard \
> + package/gcc/$(GCC_VERSION)/*.patch \
> + $(addsuffix $((PKG)_RAWNAME)/$(GCC_VERSION)/*.patch,$(call qstrip,$(BR2_GLOBAL_PATCH_DIR))) \
> + $(addsuffix $((PKG)_RAWNAME)/*.patch,$(call qstrip,$(BR2_GLOBAL_PATCH_DIR))))
> +ifeq ($(BR2_xtensa),y)
> +HOST_GCC_COMMON_CCACHE_HASH_FILES += $(HOST_GCC_XTENSA_OVERLAY_TAR)
> +endif
> +ifeq ($(ARCH),powerpc)
> +ifneq ($(BR2_SOFT_FLOAT),)
> +HOST_GCC_COMMON_CCACHE_HASH_FILES += package/gcc/$(GCC_VERSION)/1000-powerpc-link-with-math-lib.patch.conditional
> +endif
> +endif
> +
> +HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE_HASH=\"`\
> + printf '%s' $($(PKG)_CONF_OPTS) $(GCC_VERSION) \
> + | sha256sum - $(HOST_GCC_COMMON_CCACHE_HASH_FILES) \
> + | cut -c -64 | tr -d '\n'`\"
> +endif # BR2_CCACHE
> +
> # The LTO support in gcc creates wrappers for ar, ranlib and nm which load
> # the lto plugin. These wrappers are called *-gcc-ar, *-gcc-ranlib, and
> # *-gcc-nm and should be used instead of the real programs when -flto is
> diff --git a/toolchain/toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
> index cf66fac..7ee3028 100644
> --- a/toolchain/toolchain-wrapper.c
> +++ b/toolchain/toolchain-wrapper.c
> @@ -254,6 +254,10 @@ int main(int argc, char **argv)
> debug = atoi(env_debug);
> if (debug > 0) {
> fprintf(stderr, "Toolchain wrapper executing:");
> +#ifdef BR_CCACHE_HASH
> + fprintf(stderr, "CCACHE_COMPILERCHECK=string:" BR_CCACHE_HASH "%s",
> + (debug == 2) ? "\n " : " ");
> +#endif
There is probably a formatting issue here when debug==2:
Toolchain wrapper executing:CCACHE_COMPILERCHECK=string:[hash]
CCACHE_BASEDIR=/home/naourr/git/buildroot/test/ccache
'/home/naourr/git/buildroot/test/ccache/host/usr/bin/ccache'
'/home/naourr/git/buildroot/test/ccache/host/usr/bin/i686-buildroot-linux-uclibc-gcc.real'
'--sysroot'
The fprintf line should look-like:
fprintf(stderr, "%sCCACHE_COMPILERCHECK=string:'" BR_CCACHE_HASH "'",
(debug == 2) ? "\n " : " ");
Toolchain wrapper executing:
CCACHE_COMPILERCHECK='string:[hash]'
CCACHE_BASEDIR='/home/naourr/git/buildroot/test/ccache'
'/home/naourr/git/buildroot/test/ccache/host/usr/bin/i686-buildroot-linux-uclibc-gcc.real'
'--sysroot'
'/home/naourr/git/buildroot/test/ccache/host/usr/i686-buildroot-linux-uclibc/sysroot'
'-E'
'-xc'
'-'
> for (i = 0; exec_args[i]; i++)
> fprintf(stderr, "%s'%s'",
> (debug == 2) ? "\n " : " ", exec_args[i]);
> @@ -261,6 +265,14 @@ int main(int argc, char **argv)
Same here.
Best regards,
Romain
> }
> }
>
> +#ifdef BR_CCACHE_HASH
> + /* Allow compilercheck to be overridden through the environment */
> + if (setenv("CCACHE_COMPILERCHECK", "string:" BR_CCACHE_HASH, 0)) {
> + perror(__FILE__ ": Failed to set CCACHE_COMPILERCHECK");
> + return 3;
> + }
> +#endif
> +
> if (execv(exec_args[0], exec_args))
> perror(path);
>
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 04/18] infra: move ccache handling to the toolchain wrapper
2015-10-04 12:28 ` [Buildroot] [PATCH v2 04/18] infra: move ccache handling to the toolchain wrapper Arnout Vandecappelle
@ 2015-10-04 15:07 ` Peter Korsgaard
2015-10-04 15:23 ` [Buildroot] [PATCH v3] " Arnout Vandecappelle
1 sibling, 0 replies; 26+ messages in thread
From: Peter Korsgaard @ 2015-10-04 15:07 UTC (permalink / raw)
To: buildroot
>>>>> "Arnout" == Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> writes:
> Since we always have a toolchain wrapper now, we can move the ccache
> call to the toolchain wrapper.
> The hostcc ccache handling obviously stays.
> The global addition of ccache to TARGET_CC/CXX is removed, but many
> individual packages and infras still add it. This means we have a
> chain like this: ccache -> toolchain-wrapper -> ccache -> gcc
> However, this is fairly harmless: for cache misses, the inner ccache
> just adds overhead and for cache hits, the inner ccache is never
> called. Later patches will remove these redundant ccache calls.
> As a side effect, perl now supports ccache as well.
> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> Cc: Danomi Manchego <danomimanchego123@gmail.com>
> Cc: K?roly Kasza <kaszak@gmail.com>
> Reviewed-by: Romain Naour <romain.naour@openwide.fr>
> +++ b/toolchain/toolchain-wrapper.mk
> @@ -16,6 +16,10 @@ TOOLCHAIN_WRAPPER_ARGS += -DBR_SYSROOT='"$(STAGING_SUBDIR)"'
> # separate argument when used in execv() by the toolchain wrapper.
> TOOLCHAIN_WRAPPER_ARGS += -DBR_ADDITIONAL_CFLAGS='$(foreach f,$(call qstrip,$(BR2_TARGET_OPTIMIZATION)),"$(f)",)'
> +ifeq ($(BR2_CCACHE),y)
> +TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE='"$(CCACHE)"'
> +endif
As I just mentioned IRL, CCACHE is $(HOST_DIR)/usr/bin/ccache, E.G the
absolute path to ccache - So hardcoding this in the wrapper isn't really
nice for relocatable SDK.
--
Bye, Peter Korsgaard
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v3] infra: move ccache handling to the toolchain wrapper
2015-10-04 12:28 ` [Buildroot] [PATCH v2 04/18] infra: move ccache handling to the toolchain wrapper Arnout Vandecappelle
2015-10-04 15:07 ` Peter Korsgaard
@ 2015-10-04 15:23 ` Arnout Vandecappelle
1 sibling, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 15:23 UTC (permalink / raw)
To: buildroot
Since we always have a toolchain wrapper now, we can move the ccache
call to the toolchain wrapper.
The hostcc ccache handling obviously stays.
The global addition of ccache to TARGET_CC/CXX is removed, but many
individual packages and infras still add it. This means we have a
chain like this: ccache -> toolchain-wrapper -> ccache -> gcc
However, this is fairly harmless: for cache misses, the inner ccache
just adds overhead and for cache hits, the inner ccache is never
called. Later patches will remove these redundant ccache calls.
As a side effect, perl now supports ccache as well.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Danomi Manchego <danomimanchego123@gmail.com>
Cc: K?roly Kasza <kaszak@gmail.com>
---
v3: Don't use absolute path for BR_CCACHE; instead, use the abspath
that is anyway already calculated by the wrapper. [Peter]
v2: No change
---
package/Makefile.in | 5 -----
toolchain/toolchain-wrapper.c | 15 ++++++++++++++-
toolchain/toolchain-wrapper.mk | 4 ++++
3 files changed, 18 insertions(+), 6 deletions(-)
diff --git a/package/Makefile.in b/package/Makefile.in
index 545694f..4b1ce7f 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -189,11 +189,6 @@ TARGET_OBJDUMP = $(TARGET_CROSS)objdump
TARGET_CC_NOCCACHE := $(TARGET_CC)
TARGET_CXX_NOCCACHE := $(TARGET_CXX)
-ifeq ($(BR2_CCACHE),y)
-TARGET_CC := $(CCACHE) $(TARGET_CC)
-TARGET_CXX := $(CCACHE) $(TARGET_CXX)
-endif
-
ifeq ($(BR2_STRIP_strip),y)
STRIP_STRIP_DEBUG := --strip-debug
STRIP_STRIP_UNNEEDED := --strip-unneeded
diff --git a/toolchain/toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
index d4d25c7..aee5df7 100644
--- a/toolchain/toolchain-wrapper.c
+++ b/toolchain/toolchain-wrapper.c
@@ -23,6 +23,9 @@
#include <stdlib.h>
#include <errno.h>
+#ifdef BR_CCACHE
+static char ccache_path[PATH_MAX];
+#endif
static char path[PATH_MAX];
static char sysroot[PATH_MAX];
@@ -40,6 +43,9 @@ static char sysroot[PATH_MAX];
#define EXCLUSIVE_ARGS 3
static char *predef_args[] = {
+#ifdef BR_CCACHE
+ ccache_path,
+#endif
path,
"--sysroot", sysroot,
#ifdef BR_ABI
@@ -147,6 +153,13 @@ int main(int argc, char **argv)
perror(__FILE__ ": overflow");
return 3;
}
+#ifdef BR_CCACHE
+ ret = snprintf(ccache_path, sizeof(ccache_path), "%s/usr/bin/ccache", absbasedir);
+ if (ret >= sizeof(ccache_path)) {
+ perror(__FILE__ ": overflow");
+ return 3;
+ }
+#endif
ret = snprintf(sysroot, sizeof(sysroot), "%s/" BR_SYSROOT, absbasedir);
if (ret >= sizeof(sysroot)) {
perror(__FILE__ ": overflow");
@@ -251,7 +264,7 @@ int main(int argc, char **argv)
}
}
- if (execv(path, args))
+ if (execv(args[0], args))
perror(path);
free(args);
diff --git a/toolchain/toolchain-wrapper.mk b/toolchain/toolchain-wrapper.mk
index 899947d..749afd7 100644
--- a/toolchain/toolchain-wrapper.mk
+++ b/toolchain/toolchain-wrapper.mk
@@ -16,6 +16,10 @@ TOOLCHAIN_WRAPPER_ARGS += -DBR_SYSROOT='"$(STAGING_SUBDIR)"'
# separate argument when used in execv() by the toolchain wrapper.
TOOLCHAIN_WRAPPER_ARGS += -DBR_ADDITIONAL_CFLAGS='$(foreach f,$(call qstrip,$(BR2_TARGET_OPTIMIZATION)),"$(f)",)'
+ifeq ($(BR2_CCACHE),y)
+TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE
+endif
+
# For simplicity, build directly into the install location
define TOOLCHAIN_BUILD_WRAPPER
$(Q)mkdir -p $(HOST_DIR)/usr/bin
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v3] ccache: use mtime for external toolchain, CONF_OPTS for internal toolchain
2015-10-04 12:28 ` [Buildroot] [PATCH v2 17/18] ccache: use mtime for external toolchain, CONF_OPTS for internal toolchain Arnout Vandecappelle
2015-10-04 14:55 ` Romain Naour
@ 2015-10-04 15:25 ` Arnout Vandecappelle
1 sibling, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 15:25 UTC (permalink / raw)
To: buildroot
Our current ccache disables hashing of the compiler executable itself,
because using the default 'mtime' doesn't work in buildroot: we always
rebuild the compiler, so the mtime is always different, so the cache
always misses.
However, in the current situation, if a user changes the compiler
configuration (which would result in the compiler generating different
object files than before) and does 'make clean all', ccache may in fact
reuse object files from the previous run. This rarely gives problems,
because
(1) the cache expires quite quickly (it's only 1GB by default),
(2) radically changing compiler options will cause cache misses because
different header files are used,
(3) many compiler changes (e.g. changing -mtune) have little practical
effect because the resulting code is usually still compatible,
(4) we currently don't use CCACHE_BASEDIR, and almost all object files
will contain an absolute path (e.g. in debug info), so when
building in a different directory, most of it will miss,
(5) we do mostly build test, and many of the potential problems only
appear at runtime.
Still, when ccache _does_ use the wrong cached object files, the
effects are really weird and hard to debug. Also, we want reproducible
builds and obviously the above makes builds non-reproducible. So we
have a FAQ entry that warns against using ccache and tells the user to
clear the cache in case of problems.
Now that ccache is called from the toolchain wrapper, it is in fact
possible to at least use the 'mtime' compiler hash for the external
toolchain and for the host-gcc. Indeed, in this case, the compiler
executable comes from a tarball so the mtime will be a good reference
for it state. Therefore, the patch (sed script) that changes the
default from 'mtime' to 'none' is removed.
For the internal toolchain, we can do better by providing a hash of
the relevant toolchain options. We are only interested in things that
affect the compiler itself, because ccache also processes the header
files and it doesn't look at libraries because it doesn't cache the
link step, just compilation. Everything that affects the compiler
itself can nicely be summarised in $(HOST_GCC_FINAL_CONF_OPTS). Of
course, also the compiler source itself is relevant, so the source
tarball and all the patches are included in the hash. For this purpose,
a new HOST_GCC_XTENSA_OVERLAY_TAR is introduced.
The following procedure tests the ccache behaviour:
Use this defconfig:
BR2_arm=y
BR2_CCACHE=y
make
readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os
-> Tag_CPU_name: "ARM926EJ-S"
Now make menuconfig, change variant into BR2_cortex_a9
make clean; make
readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os
-> Tag_CPU_name: "ARM926EJ-S"
should be "Cortex-A9"
After this commit, it is "Cortex-A9".
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Danomi Manchego <danomimanchego123@gmail.com>
Cc: K?roly Kasza <kaszak@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Romain Naour <romain.naour@openwide.fr>
---
v3: - Fix format string of BR2_DEBUG_WRAPPER. [Romain]
v2: - Add BR2_GLOBAL_PATCH_DIR to the patches used to generate the hash.
[Samuel]
- Use $(wildcard ...) for the patches, so missing patch dir (e.g. for
OVERRIDE_SRCDIR) doesn't lead to an error. [Arnout]
- Remove redundant newline [Romain]
---
package/ccache/ccache.mk | 5 -----
package/gcc/gcc.mk | 29 +++++++++++++++++++++++++++--
toolchain/toolchain-wrapper.c | 12 ++++++++++++
3 files changed, 39 insertions(+), 7 deletions(-)
diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
index 3bd2803..52cf309 100644
--- a/package/ccache/ccache.mk
+++ b/package/ccache/ccache.mk
@@ -26,15 +26,10 @@ HOST_CCACHE_CONF_OPTS += --with-bundled-zlib
# is already used by autotargets for the ccache package.
# BR_CACHE_DIR is exported by Makefile based on config option
# BR2_CCACHE_DIR.
-# - ccache shouldn't use the compiler binary mtime to detect a change in
-# the compiler, because in the context of Buildroot, that completely
-# defeats the purpose of ccache. Of course, that leaves the user
-# responsible for purging its cache when the compiler changes.
# - Change hard-coded last-ditch default to match path in .config, to avoid
# the need to specify BR_CACHE_DIR when invoking ccache directly.
define HOST_CCACHE_PATCH_CONFIGURATION
sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/ccache.c
- sed -i 's,conf->compiler_check = x_strdup("mtime"),conf->compiler_check = x_strdup("none"),' $(@D)/conf.c
sed -i 's,"%s/.ccache","$(BR_CACHE_DIR)",' $(@D)/conf.c
endef
diff --git a/package/gcc/gcc.mk b/package/gcc/gcc.mk
index b9da398..103cbe1 100644
--- a/package/gcc/gcc.mk
+++ b/package/gcc/gcc.mk
@@ -23,9 +23,10 @@ GCC_SOURCE ?= gcc-$(GCC_VERSION).tar.bz2
# Xtensa special hook
#
+HOST_GCC_XTENSA_OVERLAY_TAR = $(BR2_XTENSA_OVERLAY_DIR)/xtensa_$(call qstrip,$(BR2_XTENSA_CORE_NAME)).tar
+
define HOST_GCC_XTENSA_OVERLAY_EXTRACT
- tar xf $(BR2_XTENSA_OVERLAY_DIR)/xtensa_$(call qstrip,\
- $(BR2_XTENSA_CORE_NAME)).tar -C $(@D) --strip-components=1 gcc
+ tar xf $(HOST_GCC_XTENSA_OVERLAY_TAR) -C $(@D) --strip-components=1 gcc
endef
#
@@ -237,6 +238,30 @@ endif
HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS += -DBR_CROSS_PATH_SUFFIX='".real"'
+ifeq ($(BR2_CCACHE),y)
+HOST_GCC_COMMON_CCACHE_HASH_FILES += $(DL_DIR)/$(GCC_SOURCE)
+# Cfr. PATCH_BASE_DIRS in .stamp_patched, but we catch both versioned and
+# unversioned patches unconditionally
+HOST_GCC_COMMON_CCACHE_HASH_FILES += \
+ $(wildcard \
+ package/gcc/$(GCC_VERSION)/*.patch \
+ $(addsuffix $((PKG)_RAWNAME)/$(GCC_VERSION)/*.patch,$(call qstrip,$(BR2_GLOBAL_PATCH_DIR))) \
+ $(addsuffix $((PKG)_RAWNAME)/*.patch,$(call qstrip,$(BR2_GLOBAL_PATCH_DIR))))
+ifeq ($(BR2_xtensa),y)
+HOST_GCC_COMMON_CCACHE_HASH_FILES += $(HOST_GCC_XTENSA_OVERLAY_TAR)
+endif
+ifeq ($(ARCH),powerpc)
+ifneq ($(BR2_SOFT_FLOAT),)
+HOST_GCC_COMMON_CCACHE_HASH_FILES += package/gcc/$(GCC_VERSION)/1000-powerpc-link-with-math-lib.patch.conditional
+endif
+endif
+
+HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE_HASH=\"`\
+ printf '%s' $($(PKG)_CONF_OPTS) $(GCC_VERSION) \
+ | sha256sum - $(HOST_GCC_COMMON_CCACHE_HASH_FILES) \
+ | cut -c -64 | tr -d '\n'`\"
+endif # BR2_CCACHE
+
# The LTO support in gcc creates wrappers for ar, ranlib and nm which load
# the lto plugin. These wrappers are called *-gcc-ar, *-gcc-ranlib, and
# *-gcc-nm and should be used instead of the real programs when -flto is
diff --git a/toolchain/toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
index 0c8f1d8..d52115f 100644
--- a/toolchain/toolchain-wrapper.c
+++ b/toolchain/toolchain-wrapper.c
@@ -264,6 +264,10 @@ int main(int argc, char **argv)
debug = atoi(env_debug);
if (debug > 0) {
fprintf(stderr, "Toolchain wrapper executing:");
+#ifdef BR_CCACHE_HASH
+ fprintf(stderr, "%sCCACHE_COMPILERCHECK='string:" BR_CCACHE_HASH "'",
+ (debug == 2) ? "\n " : " ");
+#endif
for (i = 0; exec_args[i]; i++)
fprintf(stderr, "%s'%s'",
(debug == 2) ? "\n " : " ", exec_args[i]);
@@ -271,6 +275,14 @@ int main(int argc, char **argv)
}
}
+#ifdef BR_CCACHE_HASH
+ /* Allow compilercheck to be overridden through the environment */
+ if (setenv("CCACHE_COMPILERCHECK", "string:" BR_CCACHE_HASH, 0)) {
+ perror(__FILE__ ": Failed to set CCACHE_COMPILERCHECK");
+ return 3;
+ }
+#endif
+
if (execv(exec_args[0], exec_args))
perror(path);
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v3] [RFC] ccache: support changing the output directory
2015-10-04 12:28 ` [Buildroot] [PATCH v2 18/18] [RFC] ccache: support changing the output directory Arnout Vandecappelle
@ 2015-10-04 15:25 ` Arnout Vandecappelle
0 siblings, 0 replies; 26+ messages in thread
From: Arnout Vandecappelle @ 2015-10-04 15:25 UTC (permalink / raw)
To: buildroot
When building in a different output directory than the original build,
there will currently be a lot of ccache misses because in many cases
there is some -I/... absolute path in the compilation. ccache has an
option CCACHE_BASEDIR to substitute absolute paths with relative paths,
so they wil be the same in the hash (and in the output).
Since there are some disadvantages to this path rewriting, it is made
optional as BR2_CCACHE_USE_BASEDIR. It defaults to y because the
usefulness of ccache is severely reduced without this option.
In addition to CCACHE_BASEDIR, we also substitute away the occurences
of $(HOST_DIR) in the calculation of the compiler hash. This is done
regardless of the setting of BR2_CCACHE_USE_BASEDIR because it's
quite harmless.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
Perhaps we should use $(HOST_DIR) instead of $(BASE_DIR) as
CCACHE_BASEDIR, because most of the time an absolute path will point
into the host dir. Not always, though (e.g. references to LINUX_DIR).
And most people will have $(HOST_DIR) inside $(BASE_DIR) so the latter
works well for them.
It's hard to be sure with ccache, but in my tests I still had the
impression that there were less cache hits when rebuilding in a
different output directory than when rebuilding in the same directory.
Oh well...
v3: - Fix format string of BR2_DEBUG_WRAPPER. [Romain]
---
Config.in | 27 +++++++++++++++++++++++++++
docs/manual/ccache-support.txt | 20 ++++++++++++++++++++
package/gcc/gcc.mk | 4 +++-
toolchain/toolchain-wrapper.c | 11 +++++++++++
toolchain/toolchain-wrapper.mk | 4 ++++
5 files changed, 65 insertions(+), 1 deletion(-)
diff --git a/Config.in b/Config.in
index 74aeb46..d795361 100644
--- a/Config.in
+++ b/Config.in
@@ -286,6 +286,33 @@ config BR2_CCACHE_INITIAL_SETUP
These initial settings are applied after ccache has been compiled.
+config BR2_CCACHE_USE_BASEDIR
+ bool "Use relative paths"
+ default y
+ help
+ Allow ccache to convert absolute paths within the output
+ directory into relative paths.
+
+ During the build, many -I include directives are given with
+ an absolute path. These absolute paths end up in the hashes
+ that are computed by ccache. Therefore, when you build from a
+ different directory, the hash will be different and the
+ cached object will not be used.
+
+ To improve cache performance, set this option to y. This
+ allows ccache to rewrite absolute paths within the output
+ directory into relative paths. Note that only paths within
+ the output directory will be rewritten; therefore, if you
+ change BR2_HOST_DIR to point outside the output directory and
+ subsequently move it to a different location, this will lead
+ to cache misses.
+
+ This option has as a result that the debug information in the
+ object files also has only relative paths. Therefore, make
+ sure you cd to the build directory before starting gdb. See
+ the section "COMPILING IN DIFFERENT DIRECTORIES" in the
+ ccache manual for more information.
+
endif
config BR2_DEPRECATED
diff --git a/docs/manual/ccache-support.txt b/docs/manual/ccache-support.txt
index 992471d..f6746ad 100644
--- a/docs/manual/ccache-support.txt
+++ b/docs/manual/ccache-support.txt
@@ -33,3 +33,23 @@ make CCACHE_OPTIONS="--max-size=5G" ccache-options
# zero statistics counters
make CCACHE_OPTIONS="--zero-stats" ccache-options
-----------------
+
++ccache+ makes a hash of the source files and of the compiler options.
+If a compiler option is different, the cached object file will not be
+used. Many compiler options, however, contain an absolute path to the
+staging directory. Because of this, building in a different output
+directory would lead to many cache misses.
+
+To avoid this issue, buildroot has the +Use relative paths+ option
+(+BR2_CCACHE_USE_BASEDIR+). This will rewrite all absolute paths that
+point inside the output directory into relative paths. Thus, changing
+the output directory no longer leads to cache misses.
+
+A disadvantage of the relative paths is that they also end up to be
+relative paths in the object file. Therefore, for example, the debugger
+will no longer find the file, unless you cd to the output directory
+first.
+
+See https://ccache.samba.org/manual.html#_compiling_in_different_directories[the
+ccache manual's section on "Compiling in different directories"] for
+more details about this rewriting of absolute paths.
diff --git a/package/gcc/gcc.mk b/package/gcc/gcc.mk
index 103cbe1..420c4e9 100644
--- a/package/gcc/gcc.mk
+++ b/package/gcc/gcc.mk
@@ -256,8 +256,10 @@ HOST_GCC_COMMON_CCACHE_HASH_FILES += package/gcc/$(GCC_VERSION)/1000-powerpc-lin
endif
endif
+# _CONF_OPTS contains some references to the absolute path of $(HOST_DIR),
+# so substitute those away.
HOST_GCC_COMMON_TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE_HASH=\"`\
- printf '%s' $($(PKG)_CONF_OPTS) $(GCC_VERSION) \
+ printf '%s' $(subst $(HOST_DIR), at HOST_DIR@,$($(PKG)_CONF_OPTS)) \
| sha256sum - $(HOST_GCC_COMMON_CCACHE_HASH_FILES) \
| cut -c -64 | tr -d '\n'`\"
endif # BR2_CCACHE
diff --git a/toolchain/toolchain-wrapper.c b/toolchain/toolchain-wrapper.c
index d52115f..11f93f0 100644
--- a/toolchain/toolchain-wrapper.c
+++ b/toolchain/toolchain-wrapper.c
@@ -268,6 +268,10 @@ int main(int argc, char **argv)
fprintf(stderr, "%sCCACHE_COMPILERCHECK='string:" BR_CCACHE_HASH "'",
(debug == 2) ? "\n " : " ");
#endif
+#ifdef BR_CCACHE_BASEDIR
+ fprintf(stderr, "%sCCACHE_BASEDIR='" BR_CCACHE_BASEDIR "'",
+ (debug == 2) ? "\n " : " ");
+#endif
for (i = 0; exec_args[i]; i++)
fprintf(stderr, "%s'%s'",
(debug == 2) ? "\n " : " ", exec_args[i]);
@@ -282,6 +286,13 @@ int main(int argc, char **argv)
return 3;
}
#endif
+#ifdef BR_CCACHE_BASEDIR
+ /* Allow compilercheck to be overridden through the environment */
+ if (setenv("CCACHE_BASEDIR", BR_CCACHE_BASEDIR, 0)) {
+ perror(__FILE__ ": Failed to set CCACHE_BASEDIR");
+ return 3;
+ }
+#endif
if (execv(exec_args[0], exec_args))
perror(path);
diff --git a/toolchain/toolchain-wrapper.mk b/toolchain/toolchain-wrapper.mk
index 749afd7..c78363a 100644
--- a/toolchain/toolchain-wrapper.mk
+++ b/toolchain/toolchain-wrapper.mk
@@ -20,6 +20,10 @@ ifeq ($(BR2_CCACHE),y)
TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE
endif
+ifeq ($(BR2_CCACHE_USE_BASEDIR),y)
+TOOLCHAIN_WRAPPER_ARGS += -DBR_CCACHE_BASEDIR='"$(BASE_DIR)"'
+endif
+
# For simplicity, build directly into the install location
define TOOLCHAIN_BUILD_WRAPPER
$(Q)mkdir -p $(HOST_DIR)/usr/bin
--
2.5.3
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
` (17 preceding siblings ...)
2015-10-04 12:28 ` [Buildroot] [PATCH v2 18/18] [RFC] ccache: support changing the output directory Arnout Vandecappelle
@ 2015-10-04 16:01 ` Peter Korsgaard
18 siblings, 0 replies; 26+ messages in thread
From: Peter Korsgaard @ 2015-10-04 16:01 UTC (permalink / raw)
To: buildroot
>>>>> "Arnout" == Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> writes:
> This series introduces two features that were discussed on the list and
> in the Buildroot Summer Camp: a wrapper for the internal toolchain, and
> fixing ccache. See below why the two are combined in one series.
> [Note: to make review easier, all the potentially contentious stuff
> from the individual patches is repeated here. So for the first few
> iterations, it's probably enough to reply to just this e-mail.]
Committed entire series, including the v3 versions of the 3 patches,
thanks!
--
Bye, Peter Korsgaard
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2015-10-04 16:01 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-10-04 12:28 [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 01/18] toolchain-external: move wrapper to toolchain directory Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 02/18] gcc: use toolchain wrapper Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 03/18] gcc: remove unsafe patch check (poison system dirs) patch Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 04/18] infra: move ccache handling to the toolchain wrapper Arnout Vandecappelle
2015-10-04 15:07 ` Peter Korsgaard
2015-10-04 15:23 ` [Buildroot] [PATCH v3] " Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 05/18] perl: Remove ccache handling Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 06/18] imx-lib: remove now-redundant " Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 07/18] imx-vpu: " Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 08/18] linux: " Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 09/18] uboot: " Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 10/18] barebox: " Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 11/18] cryptodev-linux: remove now-redundant fix-ccache-compile patch Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 12/18] qt5base: remove now-redundant ccache handling Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 13/18] package-cmake: remove now-redundant target ccache support Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 14/18] qpid-proton: remove now-redundant ccache handling patch Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 15/18] Makefile.in: remove now-unused TARGET_CC/CXX_NOCCACHE Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 16/18] toolchain-wrapper: support change of BR2_CCACHE Arnout Vandecappelle
2015-10-04 14:24 ` Romain Naour
2015-10-04 12:28 ` [Buildroot] [PATCH v2 17/18] ccache: use mtime for external toolchain, CONF_OPTS for internal toolchain Arnout Vandecappelle
2015-10-04 14:55 ` Romain Naour
2015-10-04 15:25 ` [Buildroot] [PATCH v3] " Arnout Vandecappelle
2015-10-04 12:28 ` [Buildroot] [PATCH v2 18/18] [RFC] ccache: support changing the output directory Arnout Vandecappelle
2015-10-04 15:25 ` [Buildroot] [PATCH v3] " Arnout Vandecappelle
2015-10-04 16:01 ` [Buildroot] [PATCH v2 00/18] Internal toolchain wrapper & ccache support Peter Korsgaard
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox