[Buildroot] [PATCH 0 of 4] ccache dependencies

[Buildroot] [PATCH 0 of 4] ccache dependencies

[Thomas De Schampheleire]

As discussed, this series includes some changes related to ccache dependencies.
Important: see note in the second patch.
I included Samuel's patches here for convenience.

Best regards, 
Thomas

[Buildroot] [PATCH 1 of 4] dependencies: build without ccache

[Thomas De Schampheleire]

This patch forces the plain host compiler to be used during the building of
dependencies, without ccache as it is not yet built.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---
 support/dependencies/dependencies.mk |  2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/support/dependencies/dependencies.mk b/support/dependencies/dependencies.mk
--- a/support/dependencies/dependencies.mk
+++ b/support/dependencies/dependencies.mk
@@ -25,6 +25,8 @@ core-dependencies:
 		DL_TOOLS="$(sort $(DL_TOOLS_DEPENDENCIES))" \
 		$(TOPDIR)/support/dependencies/dependencies.sh
 
+dependencies: HOSTCC=$(HOSTCC_NOCCACHE)
+dependencies: HOSTCXX=$(HOSTCXX_NOCCACHE)
 dependencies: core-dependencies $(DEPENDENCIES_HOST_PREREQ)
 
 dependencies-source:

[Buildroot] [PATCH 2 of 4] host-ccache: turn into a proper dependency

[Thomas De Schampheleire]

This patch moves the host-ccache build target from BASE_TARGETS in Makefile
to an actual host prerequisite in support/dependencies. This causes
host-ccache to be built as part of the dependencies, before any real package
is built.
Since the dependencies are built without ccache anyway, there is no need to
set HOST_CCACHE_CONF_ENV anymore.

Suggested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---

 Makefile                             |  4 ----
 package/ccache/ccache.mk             |  6 ------
 support/dependencies/dependencies.mk |  4 ++++
 3 files changed, 4 insertions(+), 10 deletions(-)


Note: this in itself works, when the dependencies are built (regular 'make'
for example, or explicit 'make dependencies'). However, if we want to
supersede patch http://patchwork.ozlabs.org/patch/257372/, that wants
the following to work:

$ make clean menuconfig
$ make {barebox,busybox,ctng,linux,uclibc}-menuconfig

then I think something additional is necessary. In the above flow, the
dependencies target is never used, so host-ccache doesn't get a chance to
build.

diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -222,10 +222,6 @@ GNU_HOST_NAME:=$(shell support/gnuconfig
 #
 ################################################################################
 
-ifeq ($(BR2_CCACHE),y)
-BASE_TARGETS += host-ccache
-endif
-
 ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
 BASE_TARGETS += toolchain-buildroot
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
--- a/package/ccache/ccache.mk
+++ b/package/ccache/ccache.mk
@@ -10,12 +10,6 @@ CCACHE_SOURCE  = ccache-$(CCACHE_VERSION
 CCACHE_LICENSE = GPLv3+, others
 CCACHE_LICENSE_FILES = LICENSE.txt GPL-3.0.txt
 
-# When ccache is being built for the host, ccache is not yet
-# available, so we have to use the special C compiler without the
-# cache.
-HOST_CCACHE_CONF_ENV = \
-	CC="$(HOSTCC_NOCCACHE)"
-
 # Force ccache to use its internal zlib. The problem is that without
 # this, ccache would link against the zlib of the build system, but we
 # might build and install a different version of zlib in $(O)/host
diff --git a/support/dependencies/dependencies.mk b/support/dependencies/dependencies.mk
--- a/support/dependencies/dependencies.mk
+++ b/support/dependencies/dependencies.mk
@@ -20,6 +20,10 @@ ifeq ($(BR2_STRIP_sstrip),y)
 DEPENDENCIES_HOST_PREREQ+=host-sstrip
 endif
 
+ifeq ($(BR2_CCACHE),y)
+DEPENDENCIES_HOST_PREREQ += host-ccache
+endif
+
 core-dependencies:
 	@HOSTCC="$(firstword $(HOSTCC))" MAKE="$(MAKE)" \
 		DL_TOOLS="$(sort $(DL_TOOLS_DEPENDENCIES))" \

[Buildroot] [PATCH 2 of 4] host-ccache: turn into a proper dependency

[Samuel Martin]

Hi Thomas (both), all,

2013/8/2 Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com>:
> This patch moves the host-ccache build target from BASE_TARGETS in Makefile
> to an actual host prerequisite in support/dependencies. This causes
> host-ccache to be built as part of the dependencies, before any real package
> is built.
> Since the dependencies are built without ccache anyway, there is no need to
> set HOST_CCACHE_CONF_ENV anymore.
>
> Suggested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>

>
> ---
>
>  Makefile                             |  4 ----
>  package/ccache/ccache.mk             |  6 ------
>  support/dependencies/dependencies.mk |  4 ++++
>  3 files changed, 4 insertions(+), 10 deletions(-)
>
>
> Note: this in itself works, when the dependencies are built (regular 'make'
> for example, or explicit 'make dependencies'). However, if we want to
> supersede patch http://patchwork.ozlabs.org/patch/257372/, that wants
> the following to work:
>
> $ make clean menuconfig
> $ make {barebox,busybox,ctng,linux,uclibc}-menuconfig
>
> then I think something additional is necessary. In the above flow, the
> dependencies target is never used, so host-ccache doesn't get a chance to
> build.
Humm... I don't think so. This patch should be enough to fix the bug #5678.
I mean, to run *-menuconfig, the package need to be extracted beforehand.
I expect the 'dependencies' target is part of dependency list of the
*-extract target (because of host-tar).
If it's done so, then it's ok tm.

>
> diff --git a/Makefile b/Makefile
> --- a/Makefile
> +++ b/Makefile
> @@ -222,10 +222,6 @@ GNU_HOST_NAME:=$(shell support/gnuconfig
>  #
>  ################################################################################
>
> -ifeq ($(BR2_CCACHE),y)
> -BASE_TARGETS += host-ccache
> -endif
> -
>  ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
>  BASE_TARGETS += toolchain-buildroot
>  else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
> diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
> --- a/package/ccache/ccache.mk
> +++ b/package/ccache/ccache.mk
> @@ -10,12 +10,6 @@ CCACHE_SOURCE  = ccache-$(CCACHE_VERSION
>  CCACHE_LICENSE = GPLv3+, others
>  CCACHE_LICENSE_FILES = LICENSE.txt GPL-3.0.txt
>
> -# When ccache is being built for the host, ccache is not yet
> -# available, so we have to use the special C compiler without the
> -# cache.
> -HOST_CCACHE_CONF_ENV = \
> -       CC="$(HOSTCC_NOCCACHE)"
> -
>  # Force ccache to use its internal zlib. The problem is that without
>  # this, ccache would link against the zlib of the build system, but we
>  # might build and install a different version of zlib in $(O)/host
> diff --git a/support/dependencies/dependencies.mk b/support/dependencies/dependencies.mk
> --- a/support/dependencies/dependencies.mk
> +++ b/support/dependencies/dependencies.mk
> @@ -20,6 +20,10 @@ ifeq ($(BR2_STRIP_sstrip),y)
>  DEPENDENCIES_HOST_PREREQ+=host-sstrip
>  endif
>
> +ifeq ($(BR2_CCACHE),y)
> +DEPENDENCIES_HOST_PREREQ += host-ccache
> +endif
> +
>  core-dependencies:
>         @HOSTCC="$(firstword $(HOSTCC))" MAKE="$(MAKE)" \
>                 DL_TOOLS="$(sort $(DL_TOOLS_DEPENDENCIES))" \

Regards,

-- 
Samuel

[Buildroot] [PATCH 2 of 4] host-ccache: turn into a proper dependency

[Thomas De Schampheleire]

Hi Samuel,

On Fri, Aug 2, 2013 at 10:27 AM, Samuel Martin <s.martin49@gmail.com> wrote:
> Hi Thomas (both), all,
>
> 2013/8/2 Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com>:
>> This patch moves the host-ccache build target from BASE_TARGETS in Makefile
>> to an actual host prerequisite in support/dependencies. This causes
>> host-ccache to be built as part of the dependencies, before any real package
>> is built.
>> Since the dependencies are built without ccache anyway, there is no need to
>> set HOST_CCACHE_CONF_ENV anymore.
>>
>> Suggested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> Acked-by: Samuel Martin <s.martin49@gmail.com>
>
>>
>> ---
>>
>>  Makefile                             |  4 ----
>>  package/ccache/ccache.mk             |  6 ------
>>  support/dependencies/dependencies.mk |  4 ++++
>>  3 files changed, 4 insertions(+), 10 deletions(-)
>>
>>
>> Note: this in itself works, when the dependencies are built (regular 'make'
>> for example, or explicit 'make dependencies'). However, if we want to
>> supersede patch http://patchwork.ozlabs.org/patch/257372/, that wants
>> the following to work:
>>
>> $ make clean menuconfig
>> $ make {barebox,busybox,ctng,linux,uclibc}-menuconfig
>>
>> then I think something additional is necessary. In the above flow, the
>> dependencies target is never used, so host-ccache doesn't get a chance to
>> build.
> Humm... I don't think so. This patch should be enough to fix the bug #5678.
> I mean, to run *-menuconfig, the package need to be extracted beforehand.
> I expect the 'dependencies' target is part of dependency list of the
> *-extract target (because of host-tar).
> If it's done so, then it's ok tm.

I may be missing something, but I don't see a dependency of *-extract
on 'dependencies'.
Also, I tried the scenario and it doesn't work. The compiler used to
do the linux-menuconfig is with ccache, and it isn't built
automatically, unless you specify 'dependencies' manually.

[Buildroot] [PATCH 2 of 4] host-ccache: turn into a proper dependency

[Samuel Martin]

2013/8/2 Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com>:
> Hi Samuel,
>
> On Fri, Aug 2, 2013 at 10:27 AM, Samuel Martin <s.martin49@gmail.com> wrote:
>> Hi Thomas (both), all,
>>
>> 2013/8/2 Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com>:
>>> This patch moves the host-ccache build target from BASE_TARGETS in Makefile
>>> to an actual host prerequisite in support/dependencies. This causes
>>> host-ccache to be built as part of the dependencies, before any real package
>>> is built.
>>> Since the dependencies are built without ccache anyway, there is no need to
>>> set HOST_CCACHE_CONF_ENV anymore.
>>>
>>> Suggested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>>> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
>> Acked-by: Samuel Martin <s.martin49@gmail.com>
>>
>>>
>>> ---
>>>
>>>  Makefile                             |  4 ----
>>>  package/ccache/ccache.mk             |  6 ------
>>>  support/dependencies/dependencies.mk |  4 ++++
>>>  3 files changed, 4 insertions(+), 10 deletions(-)
>>>
>>>
>>> Note: this in itself works, when the dependencies are built (regular 'make'
>>> for example, or explicit 'make dependencies'). However, if we want to
>>> supersede patch http://patchwork.ozlabs.org/patch/257372/, that wants
>>> the following to work:
>>>
>>> $ make clean menuconfig
>>> $ make {barebox,busybox,ctng,linux,uclibc}-menuconfig
>>>
>>> then I think something additional is necessary. In the above flow, the
>>> dependencies target is never used, so host-ccache doesn't get a chance to
>>> build.
>> Humm... I don't think so. This patch should be enough to fix the bug #5678.
>> I mean, to run *-menuconfig, the package need to be extracted beforehand.
>> I expect the 'dependencies' target is part of dependency list of the
>> *-extract target (because of host-tar).
>> If it's done so, then it's ok tm.
>
> I may be missing something, but I don't see a dependency of *-extract
> on 'dependencies'.
I didn't say it was, but how I expect it should be. ;)

> Also, I tried the scenario and it doesn't work. The compiler used to
> do the linux-menuconfig is with ccache, and it isn't built
> automatically, unless you specify 'dependencies' manually.
Arf... Maybe more work is needed to fix that bug.


-- 
Samuel

[Buildroot] [PATCH 2 of 4] host-ccache: turn into a proper dependency

[Thomas De Schampheleire]

Hi Thomas,

On Fri, Aug 2, 2013 at 10:54 AM, Samuel Martin <s.martin49@gmail.com> wrote:
> 2013/8/2 Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com>:
>> Hi Samuel,
>>
>> On Fri, Aug 2, 2013 at 10:27 AM, Samuel Martin <s.martin49@gmail.com> wrote:
>>> Hi Thomas (both), all,
>>>
>>> 2013/8/2 Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com>:
>>>> This patch moves the host-ccache build target from BASE_TARGETS in Makefile
>>>> to an actual host prerequisite in support/dependencies. This causes
>>>> host-ccache to be built as part of the dependencies, before any real package
>>>> is built.
>>>> Since the dependencies are built without ccache anyway, there is no need to
>>>> set HOST_CCACHE_CONF_ENV anymore.
>>>>
>>>> Suggested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
>>>> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
>>> Acked-by: Samuel Martin <s.martin49@gmail.com>
>>>
>>>>
>>>> ---
>>>>
>>>>  Makefile                             |  4 ----
>>>>  package/ccache/ccache.mk             |  6 ------
>>>>  support/dependencies/dependencies.mk |  4 ++++
>>>>  3 files changed, 4 insertions(+), 10 deletions(-)
>>>>
>>>>
>>>> Note: this in itself works, when the dependencies are built (regular 'make'
>>>> for example, or explicit 'make dependencies'). However, if we want to
>>>> supersede patch http://patchwork.ozlabs.org/patch/257372/, that wants
>>>> the following to work:
>>>>
>>>> $ make clean menuconfig
>>>> $ make {barebox,busybox,ctng,linux,uclibc}-menuconfig
>>>>
>>>> then I think something additional is necessary. In the above flow, the
>>>> dependencies target is never used, so host-ccache doesn't get a chance to
>>>> build.
>>> Humm... I don't think so. This patch should be enough to fix the bug #5678.
>>> I mean, to run *-menuconfig, the package need to be extracted beforehand.
>>> I expect the 'dependencies' target is part of dependency list of the
>>> *-extract target (because of host-tar).
>>> If it's done so, then it's ok tm.
>>
>> I may be missing something, but I don't see a dependency of *-extract
>> on 'dependencies'.
> I didn't say it was, but how I expect it should be. ;)
>
>> Also, I tried the scenario and it doesn't work. The compiler used to
>> do the linux-menuconfig is with ccache, and it isn't built
>> automatically, unless you specify 'dependencies' manually.
> Arf... Maybe more work is needed to fix that bug.

What should we do with this series?
And how to proceed with the menuconfig thing?

Thanks,
Thomas

[Buildroot] [PATCH 2 of 4] host-ccache: turn into a proper dependency

[Thomas Petazzoni]

Dear Samuel Martin,

On Fri, 2 Aug 2013 10:27:49 +0200, Samuel Martin wrote:

> > Note: this in itself works, when the dependencies are built (regular 'make'
> > for example, or explicit 'make dependencies'). However, if we want to
> > supersede patch http://patchwork.ozlabs.org/patch/257372/, that wants
> > the following to work:
> >
> > $ make clean menuconfig
> > $ make {barebox,busybox,ctng,linux,uclibc}-menuconfig
> >
> > then I think something additional is necessary. In the above flow, the
> > dependencies target is never used, so host-ccache doesn't get a chance to
> > build.
> Humm... I don't think so. This patch should be enough to fix the bug #5678.
> I mean, to run *-menuconfig, the package need to be extracted beforehand.
> I expect the 'dependencies' target is part of dependency list of the
> *-extract target (because of host-tar).

But it's not the case as far as I can see.

in busybox.mk:

busybox-menuconfig busybox-xconfig busybox-gconfig: busybox-patch

in pkg-generic.mk

$(1)-patch:             $(1)-extract $$($(2)_TARGET_PATCH)

$(1)-extract:           $(1)-source \
                        $$($(2)_TARGET_EXTRACT)

$(1)-source:            $$($(2)_TARGET_SOURCE)

and that's it. Nowhere it pulls 'dependencies' as far as I can see.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 3 of 4] crosstool-ng: remove unneed explicit ccache dependency

[Thomas De Schampheleire]

This ccache dependency was here to enforce build order.
It is not needed anymore.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---
package/crosstool-ng/crosstool-ng.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/crosstool-ng/crosstool-ng.mk b/package/crosstool-ng/crosstool-ng.mk
--- a/package/crosstool-ng/crosstool-ng.mk
+++ b/package/crosstool-ng/crosstool-ng.mk
@@ -11,7 +11,7 @@ CROSSTOOL_NG_INSTALL_TARGET    = NO
 CROSSTOOL_NG_MAKE              = $(MAKE1)
 
 HOST_CROSSTOOL_NG_DEPENDENCIES = \
-	$(if $(BR2_CCACHE),host-ccache) host-gawk \
+	host-gawk \
 	host-automake host-gperf host-bison host-flex \
 	host-texinfo
 

[Buildroot] [PATCH 4 of 4] sstrip: remove unneed explicit ccache dependency

[Thomas De Schampheleire]

This ccache dependency was here to enforce the build order.
It is not needed anymore.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---
package/sstrip/sstrip.mk | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/package/sstrip/sstrip.mk b/package/sstrip/sstrip.mk
--- a/package/sstrip/sstrip.mk
+++ b/package/sstrip/sstrip.mk
@@ -8,11 +8,6 @@ SSTRIP_SITE = svn://dev.openwrt.org/open
 SSTRIP_VERSION = 20154
 HOST_SSTRIP_BINARY = $(GNU_TARGET_NAME)-sstrip
 
-# This is a kludge to get host-ccache built before us or it fails
-ifeq ($(BR2_CCACHE),y)
-HOST_SSTRIP_DEPENDENCIES = host-ccache
-endif
-
 define SSTRIP_BUILD_CMDS
 	cd $(@D) ; \
 	$(TARGET_CC) $(TARGET_CFLAGS) -include endian.h -include byteswap.h \