[Buildroot] [PATCH 1/3] infra: make possible to run 'make *-menuconfig' from a clean output dir

[Buildroot] [PATCH 1/3] infra: make possible to run 'make *-menuconfig' from a clean output dir

[Samuel Martin]

Fixes #5678

This patch allows to run:

$ make clean menuconfig
$ make {barebox,busybox,ctng,linux,uclibc}-menuconfig

with ccache enabled or not.

This patch introduce a new make variable BASE_PKGS for packages like
host-ccache that need to be handled beforehand.

These packages are added to the dependency list of any packages, except
for those from the BASE_PKGS list.

This also implies that if there are some dependency relationships between
two packages from the BASE_PKGS list, then these dependencies should be
manually handled.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
 Makefile               | 6 ++++--
 package/pkg-generic.mk | 3 +++
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 955e684..015fbdf 100644
--- a/Makefile
+++ b/Makefile
@@ -220,7 +220,7 @@ GNU_HOST_NAME:=$(shell support/gnuconfig/config.guess)
 ################################################################################
 
 ifeq ($(BR2_CCACHE),y)
-BASE_TARGETS += host-ccache
+BASE_PKGS += host-ccache
 endif
 
 ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
@@ -361,7 +361,7 @@ include fs/common.mk
 TARGETS+=target-post-image
 
 TARGETS_CLEAN:=$(patsubst %,%-clean,$(TARGETS))
-TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS))
+TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS) $(BASE_PKGS))
 TARGETS_DIRCLEAN:=$(patsubst %,%-dirclean,$(TARGETS))
 TARGETS_ALL:=$(patsubst %,__real_tgt_%,$(TARGETS))
 
@@ -397,6 +397,8 @@ $(BASE_TARGETS): dirs $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
 $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
 	$(MAKE) $(EXTRAMAKEARGS) HOSTCC="$(HOSTCC_NOCCACHE)" HOSTCXX="$(HOSTCXX_NOCCACHE)" silentoldconfig
 
+BASE_TARGETS += $(BASE_PKGS)
+
 prepare: $(BUILD_DIR)/buildroot-config/auto.conf
 
 toolchain: prepare dirs dependencies $(BASE_TARGETS)
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 668f011..282ce99 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -303,6 +303,9 @@ $(2)_REDISTRIBUTE		?= YES
 
 
 $(2)_DEPENDENCIES ?= $(filter-out $(1),$(patsubst host-host-%,host-%,$(addprefix host-,$($(3)_DEPENDENCIES))))
+ifeq ($(filter $(1),$(BASE_PKGS)),)
+$(2)_DEPENDENCIES += $(BASE_PKGS)
+endif
 
 $(2)_INSTALL_STAGING		?= NO
 $(2)_INSTALL_IMAGES		?= NO
-- 
1.8.3.2

[Buildroot] [PATCH 2/3] crosstool-ng: remove unneed explicit ccache dependency

[Samuel Martin]

This ccache dependency was here to enforce build order.
It is not needed anymore.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
 package/crosstool-ng/crosstool-ng.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/crosstool-ng/crosstool-ng.mk b/package/crosstool-ng/crosstool-ng.mk
index d3ee5f8..8d98db4 100644
--- a/package/crosstool-ng/crosstool-ng.mk
+++ b/package/crosstool-ng/crosstool-ng.mk
@@ -11,7 +11,7 @@ CROSSTOOL_NG_INSTALL_TARGET    = NO
 CROSSTOOL_NG_MAKE              = $(MAKE1)
 
 HOST_CROSSTOOL_NG_DEPENDENCIES = \
-	$(if $(BR2_CCACHE),host-ccache) host-gawk \
+	host-gawk \
 	host-automake host-gperf host-bison host-flex \
 	host-texinfo
 
-- 
1.8.3.2

[Buildroot] [PATCH 3/3] sstrip: remove unneed explicit ccache dependency

[Samuel Martin]

This ccache dependency was here to enforce the build order.
It is not needed anymore.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
 package/sstrip/sstrip.mk | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/package/sstrip/sstrip.mk b/package/sstrip/sstrip.mk
index be8c563..c79a501 100644
--- a/package/sstrip/sstrip.mk
+++ b/package/sstrip/sstrip.mk
@@ -8,11 +8,6 @@ SSTRIP_SITE = svn://dev.openwrt.org/openwrt/trunk/tools/sstrip
 SSTRIP_VERSION = 20154
 HOST_SSTRIP_BINARY = $(GNU_TARGET_NAME)-sstrip
 
-# This is a kludge to get host-ccache built before us or it fails
-ifeq ($(BR2_CCACHE),y)
-HOST_SSTRIP_DEPENDENCIES = host-ccache
-endif
-
 define SSTRIP_BUILD_CMDS
 	cd $(@D) ; \
 	$(TARGET_CC) $(TARGET_CFLAGS) -include endian.h -include byteswap.h \
-- 
1.8.3.2

[Buildroot] [PATCH 1/3] infra: make possible to run 'make *-menuconfig' from a clean output dir

[Thomas Petazzoni]

Dear Samuel Martin,

On Sun,  7 Jul 2013 20:43:43 +0200, Samuel Martin wrote:

>  ifeq ($(BR2_CCACHE),y)
> -BASE_TARGETS += host-ccache
> +BASE_PKGS += host-ccache
>  endif
>  
>  ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
> @@ -361,7 +361,7 @@ include fs/common.mk
>  TARGETS+=target-post-image
>  
>  TARGETS_CLEAN:=$(patsubst %,%-clean,$(TARGETS))
> -TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS))
> +TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS) $(BASE_PKGS))
>  TARGETS_DIRCLEAN:=$(patsubst %,%-dirclean,$(TARGETS))
>  TARGETS_ALL:=$(patsubst %,__real_tgt_%,$(TARGETS))
>  
> @@ -397,6 +397,8 @@ $(BASE_TARGETS): dirs $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
>  $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
>  	$(MAKE) $(EXTRAMAKEARGS) HOSTCC="$(HOSTCC_NOCCACHE)" HOSTCXX="$(HOSTCXX_NOCCACHE)" silentoldconfig
>  
> +BASE_TARGETS += $(BASE_PKGS)
> +

Is this really the correct fix? Currently BASE_TARGETS contains
host-ccache if enabled, and the toolchain-<backend> target. So,
normally, all packages should depend on both, and there should be no
need to introduce a separate $(BASE_PKGS).

>  prepare: $(BUILD_DIR)/buildroot-config/auto.conf
>  
>  toolchain: prepare dirs dependencies $(BASE_TARGETS)
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index 668f011..282ce99 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -303,6 +303,9 @@ $(2)_REDISTRIBUTE		?= YES
>  
>  
>  $(2)_DEPENDENCIES ?= $(filter-out $(1),$(patsubst host-host-%,host-%,$(addprefix host-,$($(3)_DEPENDENCIES))))
> +ifeq ($(filter $(1),$(BASE_PKGS)),)
> +$(2)_DEPENDENCIES += $(BASE_PKGS)
> +endif

I.e, wouldn't

$(2)_DEPENDENCIES += $(BASE_TARGETS)

work?

Hum, the issue would of course be a recursive dependency on the
Crosstool-NG package, because toolchain-crosstool-ng depends on
crosstool-ng, and crosstool-ng being a package, it would in turn depend
on toolchain-crosstool-ng. Ditto for some other packages like
gcc-{initial,intermediate,final}, binutils and so on. So maybe what I'm
proposing in fact doesn't work, unless such packages are identified
specifically, like <pkg>_TOOLCHAIN_COMPONENT = YES, which would prevent
$(BASE_TARGETS) from being added to the dependencies of such packages.

Hum, now that I think of it, technically speaking, only the target
packages do need to have the dependency on the toolchain, host packages
do not. But both need the dependency on host-ccache.

Ok, the problem is complex, but I would really prefer to see a global
reflexion on how to name all those BASE_TARGETS, BASE_PKGS, TARGETS
variables rather than adding yet another hack into this mess.

Note that I am also currently fiddling in this area, since I have
issues getting 'make source' and 'make external-deps' to work properly
since the conversion of the internal backend to packages.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH 1/3] infra: make possible to run 'make *-menuconfig' from a clean output dir

[Samuel Martin]

Hi Thomas, all,

2013/7/7 Thomas Petazzoni <thomas.petazzoni@free-electrons.com>:
> Dear Samuel Martin,
>
> On Sun,  7 Jul 2013 20:43:43 +0200, Samuel Martin wrote:
>
>>  ifeq ($(BR2_CCACHE),y)
>> -BASE_TARGETS += host-ccache
>> +BASE_PKGS += host-ccache
>>  endif
>>
>>  ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
>> @@ -361,7 +361,7 @@ include fs/common.mk
>>  TARGETS+=target-post-image
>>
>>  TARGETS_CLEAN:=$(patsubst %,%-clean,$(TARGETS))
>> -TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS))
>> +TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS) $(BASE_PKGS))
>>  TARGETS_DIRCLEAN:=$(patsubst %,%-dirclean,$(TARGETS))
>>  TARGETS_ALL:=$(patsubst %,__real_tgt_%,$(TARGETS))
>>
>> @@ -397,6 +397,8 @@ $(BASE_TARGETS): dirs $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
>>  $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
>>       $(MAKE) $(EXTRAMAKEARGS) HOSTCC="$(HOSTCC_NOCCACHE)" HOSTCXX="$(HOSTCXX_NOCCACHE)" silentoldconfig
>>
>> +BASE_TARGETS += $(BASE_PKGS)
>> +
>
> Is this really the correct fix? Currently BASE_TARGETS contains
> host-ccache if enabled, and the toolchain-<backend> target. So,
> normally, all packages should depend on both, and there should be no
> need to introduce a separate $(BASE_PKGS).
ahem. I knew this patch would be controversial ;)
There are also few point I'm not entirely happy wit them.

>
>>  prepare: $(BUILD_DIR)/buildroot-config/auto.conf
>>
>>  toolchain: prepare dirs dependencies $(BASE_TARGETS)
>> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
>> index 668f011..282ce99 100644
>> --- a/package/pkg-generic.mk
>> +++ b/package/pkg-generic.mk
>> @@ -303,6 +303,9 @@ $(2)_REDISTRIBUTE         ?= YES
>>
>>
>>  $(2)_DEPENDENCIES ?= $(filter-out $(1),$(patsubst host-host-%,host-%,$(addprefix host-,$($(3)_DEPENDENCIES))))
>> +ifeq ($(filter $(1),$(BASE_PKGS)),)
>> +$(2)_DEPENDENCIES += $(BASE_PKGS)
>> +endif
>
> I.e, wouldn't
>
> $(2)_DEPENDENCIES += $(BASE_TARGETS)
>
> work?
This was my first (naive) though, but it ended up like this:
---
make: *** No rule to make target `host-toolchain-buildroot-source',
needed by `source'.  Stop.
---
On the very first attemp, I even got the same error for
'dirs-source'... which a non-sense!
That's why I split dependencies that are packages from the rest of the
ones in BASE_TARGETS.

>
> Hum, the issue would of course be a recursive dependency on the
> Crosstool-NG package, because toolchain-crosstool-ng depends on
> crosstool-ng, and crosstool-ng being a package, it would in turn depend
> on toolchain-crosstool-ng. Ditto for some other packages like
> gcc-{initial,intermediate,final}, binutils and so on. So maybe what I'm
> proposing in fact doesn't work, unless such packages are identified
> specifically, like <pkg>_TOOLCHAIN_COMPONENT = YES, which would prevent
> $(BASE_TARGETS) from being added to the dependencies of such packages.
>
> Hum, now that I think of it, technically speaking, only the target
> packages do need to have the dependency on the toolchain, host packages
> do not. But both need the dependency on host-ccache.
>
> Ok, the problem is complex, but I would really prefer to see a global
> reflexion on how to name all those BASE_TARGETS, BASE_PKGS, TARGETS
> variables rather than adding yet another hack into this mess.
Agree, that's why I prefer post this RFC.
I may not have all thing in mind... and I'm not a make guru...

>
> Note that I am also currently fiddling in this area, since I have
> issues getting 'make source' and 'make external-deps' to work properly
> since the conversion of the internal backend to packages.
>
Ok, let me know about your progress

Regards,

-- 
Samuel