[Buildroot] [PATCHv4 00/18] Package based 'source', 'legal-info', 'source-check' and 'external-deps'

[Buildroot] [PATCHv4 00/18] Package based 'source', 'legal-info', 'source-check' and 'external-deps'

[Thomas Petazzoni]

Hello,

This is a new iteration of the "Package based 'source', 'legal-info',
'source-check' and 'external-deps'" series.

Changes since v3:
=================

 - On patch "fs: only add non rootfs- targets to PACKAGES" fix typo on
   commit log and add Reviewed-by from Yann.

 - Rework "pkg-generic: take into account patch dependencies in
   source, external-deps and legal-info" to define and use a
   <pkg>_FINAL_ALL_DEPENDENCIES variable in order to simplify the
   code. Suggested by Yann E. Morin.

 - Remove superfluous SoB lines.

 - Added Acked-by from Yann on "pkg-generic: remove the
   .stamp_rsync_sourced fake stamp file".

 - Added Reviewed-by from Yann on "pkg-generic: don't use DL_MODE in
   .stamp_downloaded".

 - Added Acked-by from Yann on "pkg-download: get rid of DL_MODE"

 - Added Acked-by from Yann on "pkg-download: fix indentation for
   SOURCE_CHECK_* macros".

 - Added Reviewed-by from Yann on "pkg-generic: introduce a
   <pkg>_ALL_DOWNLOADS variable and factorize code".

 - Added Acked-by and Tested-by from Yann on "Makefile: implement the
   'source' target using the package infrastructure" and "Makefile:
   remove unneeded variables".

 - Added Acked-by from Yann on "Makefile: add BR_BUILDING variable".

 - Fixed xloader test with BR_BUILDING in "packages: refactor checks
   using BR_BUILDING".

Changes since v2:
=================

 - Remove all patches that have been applied to master, and rebase on
   top of the latest master, fixing the conflicts along the way.

 - Add a patch 'fs: only add non rootfs- targets to PACKAGES' to
   remove the circular dependency that can be introduced in some
   configurations by the previously applied 'fs: add rootfs
   dependencies to PACKAGES' patch.

 - Add patch 'pkg-generic: take into account patch dependencies in
   source, external-deps and legal-info' to take care of
   <pkg>_PATCH_DEPENDENCIES, and also change the implementation of
   source-check to take care of <pkg>_PATCH_DEPENDENCIES. Long term, a
   better refactoring would be useful to have a single variable
   listing all dependencies, but we didn't want to do this right now,
   as we're close to -rc1 and we want to get this patch series finally
   merged.

 - Add three patches to ensure that 'make source-check' can still be
   executed even if the various checks done by packages filtering on
   MAKECMDGOALS return false. Instead of just adding source-check
   everywhere, we refactor the code in an hopefully much nicer
   way. The patches are intentionally put at the end of the series, so
   that the rest of the series can be applied regardless of the review
   on those patches. If those patches are not applied, only the
   source-check target would have a regression: it would no longer be
   possible to execute this target if one of the checks in
   linux/barebox/uboot fires up an error.

      Makefile: add a few more targets to nobuild_targets
      packages: refactor checks using BR_BUILDING
      Makefile: add BR_BUILDING variable

 - Added Reviewed-by from Yann and Arnout on 'pkg-download: remove
   support for the SHOW_EXTERNAL_DEPS DL_MODE'

 - Added Reviewed-by and Tested-by from Yann, and Reviewed-by from
   Arnout on 'Makefile: move source-check outside of
   noconfig_targets'. Also reworded the commit log to be more correct
   about why we're moving source-check outside of noconfig_targets,
   following the comments made by Arnout.

 - Added Reviewed-by and Tested-by from Yann and Reviewed-by from
   Arnout on 'pkg-download: extend DOWNLOAD_INNER, add a SOURCE_CHECK
   macro'.

 - In patch 'pkg-generic: implement source-check targets', fix the
   missing $$(sep) as noticed by Yann. The problem was not visible
   with the full series applied because the code introduced in this
   patch gets refactored later in a correct way. But it isn't good if
   the series is not bisectable, so the issue was fixed anyway.

 - Added Acked-by + Tested-by from Yann and Reviewed-by from Arnout on
   'Makefile: implement a package based source-check target'.

 - Added Reviewed-by + Tested-by from Arnout on 'pkg-generic: remove
   the .stamp_rsync_sourced fake stamp file'.

 - Added Acked-by + Tested-by from Arnout on 'pkg-generic: don't use
   DL_MODE in .stamp_downloaded'.

 - Added Acked-by + Tested-by from Arnout on 'pkg-download: get rid of
   DL_MODE'.

 - Added Acked-by + Tested-by from Arnout on 'pkg-download: fix
   indentation for SOURCE_CHECK_* macros'.

 - Remove patch 'pkg-generic: propagate <pkg>_EXTRA_DOWNLOADS from
   target to host package' which was not strictly needed. Suggested by
   Arnout.

 - Added Acked-by + Tested-by from Arnout on 'Makefile: implement the
   'source' target using the package infrastructure' and 'Makefile:
   remove unneeded variables'.


Changes since v1:
=================

 - Remove all the patches that have been applied to master, and rebase
   on top of the latest master, fixing the conflicts along the way.

 - Fix a bug in the implementation of $(1)-source-check: the code was
   using $(p) to reference the loop iterator, while it should have
   been using $$(p) instead.

 - Add missing phony targets in pkg-kconfig: $(1)-savedefconfig and
   $(1)-update-defconfig. Noticed by Yann E. Morin.

 - Mark rootfs-ext2-symlink as a phony target. Noticed by Yann
   E. Morin.

 - Add Reviewed-by from Yann on "Makefile: targets are now declared
   phony by the appropriate infrastructures" since the problem he
   pointed out in the review has been fixed in previous patches.

 - In patch "Makefile: rename TARGETS to PACKAGES", fix typo in commit
   log (should be *listed*), and also rename TARGETS to PACKAGES in
   package/matchbox/matchbox.mk as well as
   support/scripts/graph-depends, as mentionned by Yann E. Morin.

 - Added Acked-by from Yann on "fs: add rootfs dependencies to
   PACKAGES".

 - Extend the commit log of "Makefile: use <pkg>-all-legal-info to
   implement the legal-info target" to explain why the 'make
   legal-info' output before and after this patch may be different,
   and why it is actually more correct after the patch. Suggested by
   Yann.

 - Add Reviewed-by from Yann on patch "Makefile: simplify
   show-targets".

 - Added Tested-by and Reviewed-by from Yann on patch "Makefile: use
   the package infra based external-deps".

Original introduction of the series
===================================

(Note: this is the original introduction of the series, matching the
v1. Since some patches have been merged, the patch numbers below do
not match this current version of the series.)

The initial goal of this series was to change the way we recursively
go through host dependencies for the implementation of the 'source',
'external-deps', 'legal-info' and 'source-check' targets.

The implementation of such targets currently do not rely on the
package infrastructure, and the main Makefile tries to iterate through
the dependencies of all packages using the TARGETS, TARGETS_HOST_DEPS
and HOST_DEPS variable, but it does so with only a two-level
recursion, which does not guarantee that all dependencies will have
been taken into account. It is also not at all in the spirit of the
rest of the package infrastructure.

So our final goal is simply that all those targets are implemented
using per-package make targets that recursively go through their own
dependencies.

Along the way, a number of other related changes or cleanups have been
made, and we try to describe below the overall logic of the patch
series:

 - Currently, some packages do directly use the 'DOWNLOAD' macro to
   download files. This has the unfortunate effect that the package
   infrastructure is not aware that those files are being downloaded,
   which is bad.

   To fix this, patches 1 to 7 fixes the two problematic packages: the
   'linux' and 'perl' packages. In the case of the 'perl' package, an
   addition was needed to the package infrastructure: supporting full
   URLs in <pkg>_EXTRA_DOWNLOADS. Some related cleanups and
   improvements are done as part of these changes.

   Compared to the v1 of those patches: I've fixed the typos found by
   Baruch and Romain, added the Reviewed-by tags that were given,
   fixed the manual line-break issue reported by Arnout, and adjusted
   the order of the variables in perl.mk as suggested by Arnout.

 - As part of the review of the first version of this series (which
   only included patches 1 to 7), a comment was made that packages
   like Linux were only applying patches named linux-*.patch, while we
   have moved for all other packages to a convention that consists in
   applying *.patch. Therefore, patches 8 and 9 make the Linux,
   U-Boot, Barebox, AT91Bootstrap and AT91Bootstrap3 follow this
   convention, and fix the user manual accordingly.

 - Patches 10 and 11 start extending the package infrastructure by
   adding the <pkg>-external-deps, <pkg>-all-external-deps,
   <pkg>-all-source and <pkg>-all-legal-info targets.

 - Patch 12 is doing some silly cleanup of the main Makefile.

 - Patches 13 to 17 clarify and fix the way we declare the PHONY make
   targets, by making the package infrastructure responsible for doing
   that for package targets.

 - Patches 18 and 19 make some not very important cleanups (variable
   renaming, etc.)

 - Patch 20 switches the 'legal-info' target to fully use the package
   infrastructure, thanks to the <pkg>-all-legal-info target added
   previously.

 - Patch 21 simplifies show-targets a bit.

 - Patch 22, like patch 20, switches the global 'external-deps' target
   to fully use the package infrastructure, thanks to the
   <pkg>-all-external-deps target added previously.

 - Patch 23 cleans up the download infra by removing the support for
   the SHOW_EXTERNAL_DEPS DL_MODE, unneeded now that the package infra
   is in charge of the 'external-deps' feature.

 - Patches 24, 25, 26, 27, 28, 29 and 30 take care of the global
   'source-check' target, also moved to use the package infrastructure
   through newly introduced <pkg>-source-check and
   <pkg>-all-source-check targets.

 - Patch 31 does some indentation cleanup.

 - Patch 32 and 33 make some improvements to the package
   infrastructure in terms of downloading source, patches and extra
   downloads.

 - Patch 34 implements the global 'source' target using the package
   infrastructrure, thanks to <pkg>-source and <pkg>-all-source.

 - Patch 35 removes the now unneeded variables from the main Makefile,
   which was our original goal.

Thomas

Thomas Petazzoni (18):
  fs: only add non rootfs- targets to PACKAGES
  pkg-generic: take into account patch dependencies in source,
    external-deps and legal-info
  Makefile: use the package infra based external-deps
  pkg-download: remove support for the SHOW_EXTERNAL_DEPS DL_MODE
  Makefile: move source-check outside of noconfig_targets
  pkg-download: extend DOWNLOAD_INNER, add a SOURCE_CHECK macro
  pkg-generic: implement source-check targets
  Makefile: implement a package based source-check target
  pkg-generic: remove the .stamp_rsync_sourced fake stamp file
  pkg-generic: don't use DL_MODE in .stamp_downloaded
  pkg-download: get rid of DL_MODE
  pkg-download: fix indentation for SOURCE_CHECK_* macros
  pkg-generic: introduce a <pkg>_ALL_DOWNLOADS variable and factorize
    code
  Makefile: implement the 'source' target using the package
    infrastructure
  Makefile: remove unneeded variables
  Makefile: add BR_BUILDING variable
  packages: refactor checks using BR_BUILDING
  Makefile: add a few more targets to nobuild_targets

 Makefile                              | 53 ++++++++++------------
 boot/at91bootstrap/at91bootstrap.mk   |  5 +--
 boot/at91bootstrap3/at91bootstrap3.mk |  2 +-
 boot/barebox/barebox.mk               |  5 +--
 boot/uboot/uboot.mk                   | 13 ++----
 boot/xloader/xloader.mk               |  6 +--
 fs/common.mk                          |  2 +-
 linux/linux.mk                        |  2 +-
 package/pkg-download.mk               | 83 +++++++++--------------------------
 package/pkg-generic.mk                | 67 ++++++++++++----------------
 package/sunxi-boards/sunxi-boards.mk  |  4 +-
 11 files changed, 85 insertions(+), 157 deletions(-)

-- 
2.1.0

[Buildroot] [PATCHv4 01/18] fs: only add non rootfs- targets to PACKAGES

[Thomas Petazzoni]

Commit 4570dbcb14b394d8f3c97baf4c9e5f4f225d1d37
("4570dbcb14b394d8f3c97baf4c9e5f4f225d1d37") added the dependencies of
rootfs building logic to PACKAGES, only thinking about the case where
rootfs building logic actually depends on real, normal packages
(host-mtd, host-genext2fs, etc.). But some file systems (rootfs-ubi)
depend on other filesystem images (rootfs-ubifs). And such targets
should not be added to PACKAGES, otherwise an incorrect circular
dependency is created.

This commit fixes that by only adding the rootfs building logic
dependencies that do *not* start with rootfs- to the global PACKAGES
variable.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
 fs/common.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/common.mk b/fs/common.mk
index 41ee86d..4387d06 100644
--- a/fs/common.mk
+++ b/fs/common.mk
@@ -106,7 +106,7 @@ rootfs-$(1): $$(BINARIES_DIR)/rootfs.$(1) $$(ROOTFS_$(2)_POST_TARGETS)
 
 ifeq ($$(BR2_TARGET_ROOTFS_$(2)),y)
 TARGETS_ROOTFS += rootfs-$(1)
-PACKAGES += $$(ROOTFS_$(2)_DEPENDENCIES)
+PACKAGES += $$(filter-out rootfs-%,$$(ROOTFS_$(2)_DEPENDENCIES))
 endif
 endef
 
-- 
2.1.0

[Buildroot] [PATCHv4 02/18] pkg-generic: take into account patch dependencies in source, external-deps and legal-info

[Thomas Petazzoni]

The $(1)-all-{source,external-deps,legal-info} targets currently only
take care of the dependencies in <pkg>_DEPENDENCIES, but not
<pkg>_PATCH_DEPENDENCIES. This patch fixes that by introducing a
<pkg>_FINAL_ALL_DEPENDENCIES. It also reworks the $(1)-show-depends
target to make use of <pkg>_FINALL_ALL_DEPENDENCIES.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/pkg-generic.mk | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 455bdf1..dd8c4c9 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -431,6 +431,7 @@ endif
 # Eliminate duplicates in dependencies
 $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES))
 $(2)_FINAL_PATCH_DEPENDENCIES = $$(sort $$($(2)_PATCH_DEPENDENCIES))
+$(2)_FINAL_ALL_DEPENDENCIES = $$(sort $$($(2)_FINAL_DEPENDENCIES) $$($(2)_FINAL_PATCH_DEPENDENCIES))
 
 $(2)_INSTALL_STAGING		?= NO
 $(2)_INSTALL_IMAGES		?= NO
@@ -583,7 +584,7 @@ $(1)-show-version:
 			@echo $$($(2)_VERSION)
 
 $(1)-show-depends:
-			@echo $$(sort $$($(2)_FINAL_DEPENDENCIES) $$($(2)_FINAL_PATCH_DEPENDENCIES))
+			@echo $$($(2)_FINAL_ALL_DEPENDENCIES)
 
 $(1)-graph-depends: graph-depends-requirements
 			@$$(INSTALL) -d $$(GRAPHS_DIR)
@@ -592,11 +593,14 @@ $(1)-graph-depends: graph-depends-requirements
 			|tee $$(GRAPHS_DIR)/$$(@).dot \
 			|dot $$(BR2_GRAPH_DOT_OPTS) -T$$(BR_GRAPH_OUT) -o $$(GRAPHS_DIR)/$$(@).$$(BR_GRAPH_OUT)
 
-$(1)-all-source:       $$(foreach p,$$($(2)_FINAL_DEPENDENCIES),$$(p)-all-source) $(1)-source
+$(1)-all-source:	$(1)-source
+$(1)-all-source:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-source)
 
-$(1)-all-external-deps:        $$(foreach p,$$($(2)_FINAL_DEPENDENCIES),$$(p)-all-external-deps) $(1)-external-deps
+$(1)-all-external-deps:	$(1)-external-deps
+$(1)-all-external-deps:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-external-deps)
 
-$(1)-all-legal-info:   $$(foreach p,$$($(2)_FINAL_DEPENDENCIES),$$(p)-all-legal-info) $(1)-legal-info
+$(1)-all-legal-info:	$(1)-legal-info
+$(1)-all-legal-info:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-legal-info)
 
 $(1)-dirclean:		$$($(2)_TARGET_DIRCLEAN)
 
-- 
2.1.0

[Buildroot] [PATCHv4 03/18] Makefile: use the package infra based external-deps

[Thomas Petazzoni]

This commit changes the global 'external-deps' target to use the newly
introduced per-package <pkg>-all-external-deps, instead of relying on
the 'source' target with a custom DL_MODE.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
 Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 6a7581a..7649e4c 100644
--- a/Makefile
+++ b/Makefile
@@ -614,8 +614,9 @@ target-post-image: $(TARGETS_ROOTFS) target-finalize
 
 source: $(PACKAGES_SOURCE) $(HOST_SOURCE)
 
+_external-deps: $(foreach p,$(PACKAGES),$(p)-all-external-deps)
 external-deps:
-	@$(MAKE1) -Bs DL_MODE=SHOW_EXTERNAL_DEPS $(EXTRAMAKEARGS) source | sort -u
+	@$(MAKE1) -Bs $(EXTRAMAKEARGS) _external-deps | sort -u
 
 legal-info-clean:
 	@rm -fr $(LEGAL_INFO_DIR)
-- 
2.1.0

[Buildroot] [PATCHv4 04/18] pkg-download: remove support for the SHOW_EXTERNAL_DEPS DL_MODE

[Thomas Petazzoni]

Now that the external-deps implementation relies on the per-package
<pkg>-all-external-deps and <pkg>-external-deps targets and no longer
on the 'source' target with a custom DL_MODE, we can get rid of the
support for the SHOW_EXTERNAL_DEPS DL_MODE value in the pkg-download
logic.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 package/pkg-download.mk | 45 ++-------------------------------------------
 package/pkg-generic.mk  |  5 +----
 2 files changed, 3 insertions(+), 47 deletions(-)

diff --git a/package/pkg-download.mk b/package/pkg-download.mk
index 1fc23c4..eee9e59 100644
--- a/package/pkg-download.mk
+++ b/package/pkg-download.mk
@@ -20,9 +20,8 @@ export LOCALFILES := $(call qstrip,$(BR2_LOCALFILES))
 
 DL_WRAPPER = support/download/dl-wrapper
 
-# Default spider mode is 'DOWNLOAD'. Other possible values are 'SOURCE_CHECK'
-# used by the _source-check target and 'SHOW_EXTERNAL_DEPS', used by the
-# external-deps target.
+# Default spider mode is 'DOWNLOAD'. Other possible value is
+# 'SOURCE_CHECK' used by the _source-check target.
 DL_MODE = DOWNLOAD
 
 # DL_DIR may have been set already from the environment
@@ -74,11 +73,6 @@ export BR_NO_CHECK_HASH_FOR
 # The SOURCE_CHECK_* helpers are in charge of simply checking that the source
 # is available for download. This can be used to make sure one will be able
 # to get all the sources needed for one's build configuration.
-#
-# The SHOW_EXTERNAL_DEPS_* helpers simply output to the console the names
-# of the files that will be downloaded, or path and revision of the
-# source repositories, producing a list of all the "external dependencies"
-# of a given build configuration.
 ################################################################################
 
 # Try a shallow clone - but that only works if the version is a ref (tag or
@@ -103,11 +97,6 @@ define SOURCE_CHECK_GIT
   $(GIT) ls-remote --heads $($(PKG)_SITE) > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_GIT
-	echo $($(PKG)_SOURCE)
-endef
-
-
 define DOWNLOAD_BZR
 	$(EXTRA_ENV) $(DL_WRAPPER) -b bzr \
 		-o $(DL_DIR)/$($(PKG)_SOURCE) \
@@ -122,10 +111,6 @@ define SOURCE_CHECK_BZR
 	$(BZR) ls --quiet $($(PKG)_SITE) > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_BZR
-	echo $($(PKG)_SOURCE)
-endef
-
 define DOWNLOAD_CVS
 	$(EXTRA_ENV) $(DL_WRAPPER) -b cvs \
 		-o $(DL_DIR)/$($(PKG)_SOURCE) \
@@ -142,10 +127,6 @@ define SOURCE_CHECK_CVS
 	$(CVS) -d:pserver:anonymous:@$(call stripurischeme,$(call qstrip,$($(PKG)_SITE))) login
 endef
 
-define SHOW_EXTERNAL_DEPS_CVS
-	echo $($(PKG)_SOURCE)
-endef
-
 define DOWNLOAD_SVN
 	$(EXTRA_ENV) $(DL_WRAPPER) -b svn \
 		-o $(DL_DIR)/$($(PKG)_SOURCE) \
@@ -160,10 +141,6 @@ define SOURCE_CHECK_SVN
   $(SVN) ls $($(PKG)_SITE)@$($(PKG)_DL_VERSION) > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_SVN
-  echo $($(PKG)_SOURCE)
-endef
-
 # SCP URIs should be of the form scp://[user@]host:filepath
 # Note that filepath is relative to the user's home directory, so you may want
 # to prepend the path with a slash: scp://[user@]host:/absolutepath
@@ -180,11 +157,6 @@ define SOURCE_CHECK_SCP
 	$(SSH) $(call domain,$(1),:) ls '$(call notdomain,$(1),:)' > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_SCP
-	echo $(2)
-endef
-
-
 define DOWNLOAD_HG
 	$(EXTRA_ENV) $(DL_WRAPPER) -b hg \
 		-o $(DL_DIR)/$($(PKG)_SOURCE) \
@@ -201,11 +173,6 @@ define SOURCE_CHECK_HG
   $(HG) incoming --force -l1 $($(PKG)_SITE) > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_HG
-  echo $($(PKG)_SOURCE)
-endef
-
-
 define DOWNLOAD_WGET
 	$(EXTRA_ENV) $(DL_WRAPPER) -b wget \
 		-o $(DL_DIR)/$(2) \
@@ -219,10 +186,6 @@ define SOURCE_CHECK_WGET
   $(WGET) --spider '$(call qstrip,$(1))'
 endef
 
-define SHOW_EXTERNAL_DEPS_WGET
-  echo $(2)
-endef
-
 define DOWNLOAD_LOCALFILES
 	$(EXTRA_ENV) $(DL_WRAPPER) -b cp \
 		-o $(DL_DIR)/$(2) \
@@ -236,10 +199,6 @@ define SOURCE_CHECK_LOCALFILES
   test -e $(call stripurischeme,$(call qstrip,$(1)))
 endef
 
-define SHOW_EXTERNAL_DEPS_LOCALFILES
-  echo $(2)
-endef
-
 ################################################################################
 # DOWNLOAD -- Download helper. Will try to download source from:
 # 1) BR2_PRIMARY_SITE if enabled
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index dd8c4c9..d7e3ca9 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -122,13 +122,10 @@ $(BUILD_DIR)/%/.stamp_rsynced:
 	$(foreach hook,$($(PKG)_POST_RSYNC_HOOKS),$(call $(hook))$(sep))
 	$(Q)touch $@
 
-# Handle the SOURCE_CHECK and SHOW_EXTERNAL_DEPS cases for rsynced
-# packages
+# Handle the SOURCE_CHECK case for rsynced packages
 $(BUILD_DIR)/%/.stamp_rsync_sourced:
 ifeq ($(DL_MODE),SOURCE_CHECK)
 	test -d $(SRCDIR)
-else ifeq ($(DL_MODE),SHOW_EXTERNAL_DEPS)
-	echo "file://$(SRCDIR)"
 else
 	@true # Nothing to do to source a local package
 endif
-- 
2.1.0

[Buildroot] [PATCHv4 05/18] Makefile: move source-check outside of noconfig_targets

[Thomas Petazzoni]

make source-check is here to check whether the remote sources for the
current selection of packages are still available. In its current
implementation, since it simply calls recursively a sub-make with the
source target, it can be a noconfig_targets. However, a follow-up
change will make source-check not use a sub-make, which will require
it to no longer be a noconfig_targets.

Therefore, as a preparation, this commit moves source-check outside of
noconfig_targets.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 Makefile | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index 7649e4c..a442325 100644
--- a/Makefile
+++ b/Makefile
@@ -75,7 +75,7 @@ export BR2_VERSION_FULL := $(BR2_VERSION)$(shell $(TOPDIR)/support/scripts/setlo
 noconfig_targets := menuconfig nconfig gconfig xconfig config oldconfig randconfig \
 	defconfig %_defconfig allyesconfig allnoconfig silentoldconfig release \
 	randpackageconfig allyespackageconfig allnopackageconfig \
-	source-check print-version olddefconfig
+	print-version olddefconfig
 
 # Strip quotes and then whitespaces
 qstrip = $(strip $(subst ",,$(1)))
@@ -422,7 +422,7 @@ world: target-post-image
 
 .PHONY: all world toolchain dirs clean distclean source outputmakefile \
 	legal-info legal-info-prepare legal-info-clean printvars help \
-	list-defconfigs target-finalize target-post-image
+	list-defconfigs target-finalize target-post-image source-check
 
 ################################################################################
 #
@@ -618,6 +618,10 @@ _external-deps: $(foreach p,$(PACKAGES),$(p)-all-external-deps)
 external-deps:
 	@$(MAKE1) -Bs $(EXTRAMAKEARGS) _external-deps | sort -u
 
+# check if download URLs are outdated
+source-check:
+	$(MAKE1) DL_MODE=SOURCE_CHECK $(EXTRAMAKEARGS) source
+
 legal-info-clean:
 	@rm -fr $(LEGAL_INFO_DIR)
 
@@ -778,10 +782,6 @@ savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 		--savedefconfig=$(if $(DEFCONFIG),$(DEFCONFIG),$(CONFIG_DIR)/defconfig) \
 		$(CONFIG_CONFIG_IN)
 
-# check if download URLs are outdated
-source-check:
-	$(MAKE1) DL_MODE=SOURCE_CHECK $(EXTRAMAKEARGS) source
-
 .PHONY: defconfig savedefconfig
 
 ################################################################################
-- 
2.1.0

[Buildroot] [PATCHv4 06/18] pkg-download: extend DOWNLOAD_INNER, add a SOURCE_CHECK macro

[Thomas Petazzoni]

As part of moving to a package infrastructure based source-check
implementation, we are going to move away from the global DL_MODE
variable to select the behavior of the DOWNLOAD_INNER macro.

As a preparation to this, this commit makes the DOWNLOAD_INNER macro
take a third argument, which is the action to be done: either DOWNLOAD
or SOURCE_CHECK. For now, the DOWNLOAD macro passes $(DL_MODE) as this
third argument, in order to keep the existing behavior.

In addition, a SOURCE_CHECK macro is added, which calls DOWNLOAD_INNER
with the appropriate action. This macro will be used in the upcoming
package infra based implementation of source-check.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[tested by doing a "make source" on a randpackageconfig]
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 package/pkg-download.mk | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/package/pkg-download.mk b/package/pkg-download.mk
index eee9e59..76afced 100644
--- a/package/pkg-download.mk
+++ b/package/pkg-download.mk
@@ -212,14 +212,18 @@ endef
 ################################################################################
 
 define DOWNLOAD
-	$(call DOWNLOAD_INNER,$(1),$(notdir $(1)))
+	$(call DOWNLOAD_INNER,$(1),$(notdir $(1)),$(DL_MODE))
+endef
+
+define SOURCE_CHECK
+	$(call DOWNLOAD_INNER,$(1),$(notdir $(1)),SOURCE_CHECK)
 endef
 
 define DOWNLOAD_INNER
 	$(Q)if test -n "$(call qstrip,$(BR2_PRIMARY_SITE))" ; then \
 		case "$(call geturischeme,$(BR2_PRIMARY_SITE))" in \
-			scp) $(call $(DL_MODE)_SCP,$(BR2_PRIMARY_SITE)/$(2),$(2)) && exit ;; \
-			*) $(call $(DL_MODE)_WGET,$(BR2_PRIMARY_SITE)/$(2),$(2)) && exit ;; \
+			scp) $(call $(3)_SCP,$(BR2_PRIMARY_SITE)/$(2),$(2)) && exit ;; \
+			*) $(call $(3)_WGET,$(BR2_PRIMARY_SITE)/$(2),$(2)) && exit ;; \
 		esac ; \
 	fi ; \
 	if test "$(BR2_PRIMARY_SITE_ONLY)" = "y" ; then \
@@ -232,18 +236,18 @@ define DOWNLOAD_INNER
 			scheme="$($(PKG)_SITE_METHOD)" ; \
 		fi ; \
 		case "$$scheme" in \
-			git) $($(DL_MODE)_GIT) && exit ;; \
-			svn) $($(DL_MODE)_SVN) && exit ;; \
-			cvs) $($(DL_MODE)_CVS) && exit ;; \
-			bzr) $($(DL_MODE)_BZR) && exit ;; \
-			file) $($(DL_MODE)_LOCALFILES) && exit ;; \
-			scp) $($(DL_MODE)_SCP) && exit ;; \
-			hg) $($(DL_MODE)_HG) && exit ;; \
-			*) $(call $(DL_MODE)_WGET,$(1),$(2)) && exit ;; \
+			git) $($(3)_GIT) && exit ;; \
+			svn) $($(3)_SVN) && exit ;; \
+			cvs) $($(3)_CVS) && exit ;; \
+			bzr) $($(3)_BZR) && exit ;; \
+			file) $($(3)_LOCALFILES) && exit ;; \
+			scp) $($(3)_SCP) && exit ;; \
+			hg) $($(3)_HG) && exit ;; \
+			*) $(call $(3)_WGET,$(1),$(2)) && exit ;; \
 		esac ; \
 	fi ; \
 	if test -n "$(call qstrip,$(BR2_BACKUP_SITE))" ; then \
-		$(call $(DL_MODE)_WGET,$(BR2_BACKUP_SITE)/$(2),$(2)) && exit ; \
+		$(call $(3)_WGET,$(BR2_BACKUP_SITE)/$(2),$(2)) && exit ; \
 	fi ; \
 	exit 1
 endef
-- 
2.1.0

[Buildroot] [PATCHv4 07/18] pkg-generic: implement source-check targets

[Thomas Petazzoni]

This commits extends the common package infrastructure with the
$(1)-source-check and $(1)-all-source-check targets.

The $(1)-source-check target simply calls the newly added
SOURCE_CHECK macro on all items to be downloaded.

The $(1)-all-source-check target will depend on the
$(1)-all-source-check targets of all dependent packages and the
$(1)-source-check target of the current package, which allows to do a
recursive source-check in the dependency tree.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/pkg-generic.mk | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index d7e3ca9..2176771 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -580,6 +580,16 @@ endif
 $(1)-show-version:
 			@echo $$($(2)_VERSION)
 
+$(1)-source-check:
+ifeq ($$($(2)_OVERRIDE_SRCDIR),)
+	$$(foreach p,$$($(2)_SOURCE) $$($(2)_EXTRA_DOWNLOADS) $$($(2)_PATCH),\
+		$$(if $$(findstring ://,$$(p)),\
+			$$(call SOURCE_CHECK,$$(p)),\
+			$$(call SOURCE_CHECK,$$($(2)_SITE:/=)/$$(p)))$$(sep))
+else
+	test -d $$($(2)_OVERRIDE_SRCDIR)
+endif
+
 $(1)-show-depends:
 			@echo $$($(2)_FINAL_ALL_DEPENDENCIES)
 
@@ -593,6 +603,9 @@ $(1)-graph-depends: graph-depends-requirements
 $(1)-all-source:	$(1)-source
 $(1)-all-source:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-source)
 
+$(1)-all-source-check:	$(1)-source-check
+$(1)-all-source-check:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-source-check)
+
 $(1)-all-external-deps:	$(1)-external-deps
 $(1)-all-external-deps:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-external-deps)
 
@@ -788,6 +801,7 @@ endif
 	$(1)-all-external-deps \
 	$(1)-all-legal-info \
 	$(1)-all-source \
+	$(1)-all-source-check \
 	$(1)-build \
 	$(1)-clean-for-rebuild \
 	$(1)-clean-for-reconfigure \
@@ -811,7 +825,8 @@ endif
 	$(1)-rsync \
 	$(1)-show-depends \
 	$(1)-show-version \
-	$(1)-source
+	$(1)-source \
+	$(1)-source-check
 
 endif # $(2)_KCONFIG_VAR
 endef # inner-generic-package
-- 
2.1.0

[Buildroot] [PATCHv4 08/18] Makefile: implement a package based source-check target

[Thomas Petazzoni]

This commit switches the implementation of the global source-check
target to use a package infrastructure based mechanism, using the
$(1)-all-source-check target added in the previous commit.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 Makefile | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index a442325..f5ff574 100644
--- a/Makefile
+++ b/Makefile
@@ -619,8 +619,7 @@ external-deps:
 	@$(MAKE1) -Bs $(EXTRAMAKEARGS) _external-deps | sort -u
 
 # check if download URLs are outdated
-source-check:
-	$(MAKE1) DL_MODE=SOURCE_CHECK $(EXTRAMAKEARGS) source
+source-check: $(foreach p,$(PACKAGES),$(p)-all-source-check)
 
 legal-info-clean:
 	@rm -fr $(LEGAL_INFO_DIR)
-- 
2.1.0

[Buildroot] [PATCHv4 09/18] pkg-generic: remove the .stamp_rsync_sourced fake stamp file

[Thomas Petazzoni]

The only reason for the .stamp_rsync_sourced fake stamp file target to
exist was to handle the SOURCE_CHECK operation on packages using the
OVERRIDE_SRCDIR mechanism. Now that this is handled directly inside
$(1)-source-check, there is no longer any need for this part of the
code.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
 package/pkg-generic.mk | 13 +------------
 1 file changed, 1 insertion(+), 12 deletions(-)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 2176771..6537d6a 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -122,14 +122,6 @@ $(BUILD_DIR)/%/.stamp_rsynced:
 	$(foreach hook,$($(PKG)_POST_RSYNC_HOOKS),$(call $(hook))$(sep))
 	$(Q)touch $@
 
-# Handle the SOURCE_CHECK case for rsynced packages
-$(BUILD_DIR)/%/.stamp_rsync_sourced:
-ifeq ($(DL_MODE),SOURCE_CHECK)
-	test -d $(SRCDIR)
-else
-	@true # Nothing to do to source a local package
-endif
-
 # Patch
 #
 # The RAWNAME variable is the lowercased package name, which allows to
@@ -442,7 +434,6 @@ $(2)_TARGET_INSTALL_HOST =      $$($(2)_DIR)/.stamp_host_installed
 $(2)_TARGET_BUILD =		$$($(2)_DIR)/.stamp_built
 $(2)_TARGET_CONFIGURE =		$$($(2)_DIR)/.stamp_configured
 $(2)_TARGET_RSYNC =	        $$($(2)_DIR)/.stamp_rsynced
-$(2)_TARGET_RSYNC_SOURCE =      $$($(2)_DIR)/.stamp_rsync_sourced
 $(2)_TARGET_PATCH =		$$($(2)_DIR)/.stamp_patched
 $(2)_TARGET_EXTRACT =		$$($(2)_DIR)/.stamp_extracted
 $(2)_TARGET_SOURCE =		$$($(2)_DIR)/.stamp_downloaded
@@ -571,7 +562,7 @@ $(1)-extract:		$(1)-rsync
 
 $(1)-rsync:		$$($(2)_TARGET_RSYNC)
 
-$(1)-source:		$$($(2)_TARGET_RSYNC_SOURCE)
+$(1)-source:
 
 $(1)-external-deps:
 	@echo "file://$$($(2)_OVERRIDE_SRCDIR)"
@@ -645,8 +636,6 @@ $$($(2)_TARGET_BUILD):			PKG=$(2)
 $$($(2)_TARGET_CONFIGURE):		PKG=$(2)
 $$($(2)_TARGET_RSYNC):                  SRCDIR=$$($(2)_OVERRIDE_SRCDIR)
 $$($(2)_TARGET_RSYNC):                  PKG=$(2)
-$$($(2)_TARGET_RSYNC_SOURCE):		SRCDIR=$$($(2)_OVERRIDE_SRCDIR)
-$$($(2)_TARGET_RSYNC_SOURCE):		PKG=$(2)
 $$($(2)_TARGET_PATCH):			PKG=$(2)
 $$($(2)_TARGET_PATCH):			RAWNAME=$$(patsubst host-%,%,$(1))
 $$($(2)_TARGET_PATCH):			PKGDIR=$(pkgdir)
-- 
2.1.0

[Buildroot] [PATCHv4 10/18] pkg-generic: don't use DL_MODE in .stamp_downloaded

[Thomas Petazzoni]

The .stamp_downloaded target is now only being used to really
download, and no longer for other activities like "source check" or
"external deps". So the check on DL_MODE being equal to DOWNLOAD is no
longer necessary.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
 package/pkg-generic.mk | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 6537d6a..6279fb1 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -71,7 +71,6 @@ endif
 # Retrieve the archive
 $(BUILD_DIR)/%/.stamp_downloaded:
 	$(foreach hook,$($(PKG)_PRE_DOWNLOAD_HOOKS),$(call $(hook))$(sep))
-ifeq ($(DL_MODE),DOWNLOAD)
 # Only show the download message if it isn't already downloaded
 	$(Q)for p in $($(PKG)_SOURCE) $($(PKG)_PATCH) $($(PKG)_EXTRA_DOWNLOADS) ; do \
 		if test ! -e $(DL_DIR)/`basename $$p` ; then \
@@ -79,7 +78,6 @@ ifeq ($(DL_MODE),DOWNLOAD)
 			break ; \
 		fi ; \
 	done
-endif
 	$(if $($(PKG)_SOURCE),$(call DOWNLOAD,$($(PKG)_SITE:/=)/$($(PKG)_SOURCE)))
 	$(foreach p,$($(PKG)_EXTRA_DOWNLOADS),\
 		$(if $(findstring ://,$(p)),\
@@ -94,10 +92,8 @@ endif
 		)\
 	$(sep))
 	$(foreach hook,$($(PKG)_POST_DOWNLOAD_HOOKS),$(call $(hook))$(sep))
-ifeq ($(DL_MODE),DOWNLOAD)
 	$(Q)mkdir -p $(@D)
 	$(Q)touch $@
-endif
 
 # Unpack the archive
 $(BUILD_DIR)/%/.stamp_extracted:
-- 
2.1.0

[Buildroot] [PATCHv4 11/18] pkg-download: get rid of DL_MODE

[Thomas Petazzoni]

The DL_MODE variable is now no longer used with any other value than
"DOWNLOAD", so it no longer makes sense to have this variable at
all. Therefore, this commit gets rid of it.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
 package/pkg-download.mk | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/package/pkg-download.mk b/package/pkg-download.mk
index 76afced..a145011 100644
--- a/package/pkg-download.mk
+++ b/package/pkg-download.mk
@@ -20,10 +20,6 @@ export LOCALFILES := $(call qstrip,$(BR2_LOCALFILES))
 
 DL_WRAPPER = support/download/dl-wrapper
 
-# Default spider mode is 'DOWNLOAD'. Other possible value is
-# 'SOURCE_CHECK' used by the _source-check target.
-DL_MODE = DOWNLOAD
-
 # DL_DIR may have been set already from the environment
 ifeq ($(origin DL_DIR),undefined)
 DL_DIR ?= $(call qstrip,$(BR2_DL_DIR))
@@ -212,7 +208,7 @@ endef
 ################################################################################
 
 define DOWNLOAD
-	$(call DOWNLOAD_INNER,$(1),$(notdir $(1)),$(DL_MODE))
+	$(call DOWNLOAD_INNER,$(1),$(notdir $(1)),DOWNLOAD)
 endef
 
 define SOURCE_CHECK
-- 
2.1.0

[Buildroot] [PATCHv4 12/18] pkg-download: fix indentation for SOURCE_CHECK_* macros

[Thomas Petazzoni]

Some of the SOURCE_CHECK_* macros are using a non-standard two-spaces
indentation. This commit switches them to use a single tab based
indentation, like in the rest of Buildroot.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[with git show -w]
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
 package/pkg-download.mk | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/package/pkg-download.mk b/package/pkg-download.mk
index a145011..a38062e 100644
--- a/package/pkg-download.mk
+++ b/package/pkg-download.mk
@@ -90,7 +90,7 @@ endef
 # TODO: improve to check that the given PKG_DL_VERSION exists on the remote
 # repository
 define SOURCE_CHECK_GIT
-  $(GIT) ls-remote --heads $($(PKG)_SITE) > /dev/null
+	$(GIT) ls-remote --heads $($(PKG)_SITE) > /dev/null
 endef
 
 define DOWNLOAD_BZR
@@ -134,7 +134,7 @@ define DOWNLOAD_SVN
 endef
 
 define SOURCE_CHECK_SVN
-  $(SVN) ls $($(PKG)_SITE)@$($(PKG)_DL_VERSION) > /dev/null
+	$(SVN) ls $($(PKG)_SITE)@$($(PKG)_DL_VERSION) > /dev/null
 endef
 
 # SCP URIs should be of the form scp://[user@]host:filepath
@@ -166,7 +166,7 @@ endef
 # TODO: improve to check that the given PKG_DL_VERSION exists on the remote
 # repository
 define SOURCE_CHECK_HG
-  $(HG) incoming --force -l1 $($(PKG)_SITE) > /dev/null
+	$(HG) incoming --force -l1 $($(PKG)_SITE) > /dev/null
 endef
 
 define DOWNLOAD_WGET
@@ -179,7 +179,7 @@ define DOWNLOAD_WGET
 endef
 
 define SOURCE_CHECK_WGET
-  $(WGET) --spider '$(call qstrip,$(1))'
+	$(WGET) --spider '$(call qstrip,$(1))'
 endef
 
 define DOWNLOAD_LOCALFILES
@@ -192,7 +192,7 @@ define DOWNLOAD_LOCALFILES
 endef
 
 define SOURCE_CHECK_LOCALFILES
-  test -e $(call stripurischeme,$(call qstrip,$(1)))
+	test -e $(call stripurischeme,$(call qstrip,$(1)))
 endef
 
 ################################################################################
-- 
2.1.0

[Buildroot] [PATCHv4 13/18] pkg-generic: introduce a <pkg>_ALL_DOWNLOADS variable and factorize code

[Thomas Petazzoni]

Now, both the download and source-check code are iterating over
<pkg>_SOURCE, <pkg>_PATCH and <pkg>_EXTRA_DOWNLOADS elements, figuring
out whether they contain full URLs or not. Instead of doing this
repeatdly, this patch introduces an internal <pkg>_ALL_DOWNLOADS
variable, which contains the list of everything that needs to be
downloaded, with URLs already expanded to take into account <pkg>_SITE
if needed.

This allows to simplify quite significantly the .stamp_download and
source-check implementation.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
 package/pkg-generic.mk | 26 ++++++++------------------
 1 file changed, 8 insertions(+), 18 deletions(-)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 6279fb1..248ca77 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -72,25 +72,13 @@ endif
 $(BUILD_DIR)/%/.stamp_downloaded:
 	$(foreach hook,$($(PKG)_PRE_DOWNLOAD_HOOKS),$(call $(hook))$(sep))
 # Only show the download message if it isn't already downloaded
-	$(Q)for p in $($(PKG)_SOURCE) $($(PKG)_PATCH) $($(PKG)_EXTRA_DOWNLOADS) ; do \
+	$(Q)for p in $($(PKG)_ALL_DOWNLOADS); do \
 		if test ! -e $(DL_DIR)/`basename $$p` ; then \
 			$(call MESSAGE,"Downloading") ; \
 			break ; \
 		fi ; \
 	done
-	$(if $($(PKG)_SOURCE),$(call DOWNLOAD,$($(PKG)_SITE:/=)/$($(PKG)_SOURCE)))
-	$(foreach p,$($(PKG)_EXTRA_DOWNLOADS),\
-		$(if $(findstring ://,$(p)),\
-			$(call DOWNLOAD,$(p)),\
-			$(call DOWNLOAD,$($(PKG)_SITE:/=)/$(p))\
-		)\
-	$(sep))
-	$(foreach p,$($(PKG)_PATCH),\
-		$(if $(findstring ://,$(p)),\
-			$(call DOWNLOAD,$(p)),\
-			$(call DOWNLOAD,$($(PKG)_SITE:/=)/$(p))\
-		)\
-	$(sep))
+	$(foreach p,$($(PKG)_ALL_DOWNLOADS),$(call DOWNLOAD,$(p))$(sep))
 	$(foreach hook,$($(PKG)_POST_DOWNLOAD_HOOKS),$(call $(hook))$(sep))
 	$(Q)mkdir -p $(@D)
 	$(Q)touch $@
@@ -355,6 +343,11 @@ ifndef $(2)_PATCH
  endif
 endif
 
+$(2)_ALL_DOWNLOADS = \
+	$$(foreach p,$$($(2)_SOURCE) $$($(2)_PATCH) $$($(2)_EXTRA_DOWNLOADS),\
+		$$(if $$(findstring ://,$$(p)),$$(p),\
+			$$($(2)_SITE:/=)/$$(p)))
+
 ifndef $(2)_SITE
  ifdef $(3)_SITE
   $(2)_SITE = $$($(3)_SITE)
@@ -569,10 +562,7 @@ $(1)-show-version:
 
 $(1)-source-check:
 ifeq ($$($(2)_OVERRIDE_SRCDIR),)
-	$$(foreach p,$$($(2)_SOURCE) $$($(2)_EXTRA_DOWNLOADS) $$($(2)_PATCH),\
-		$$(if $$(findstring ://,$$(p)),\
-			$$(call SOURCE_CHECK,$$(p)),\
-			$$(call SOURCE_CHECK,$$($(2)_SITE:/=)/$$(p)))$$(sep))
+	$$(foreach p,$$($(2)_ALL_DOWNLOADS),$$(call SOURCE_CHECK,$$(p))$$(sep))
 else
 	test -d $$($(2)_OVERRIDE_SRCDIR)
 endif
-- 
2.1.0

[Buildroot] [PATCHv4 14/18] Makefile: implement the 'source' target using the package infrastructure

[Thomas Petazzoni]

Now that all the bits are in place, switch the global 'source' target
to use the package infrastructure logic.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[with 'make source' (actually together with the next patch).]
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[tested with a randpackageconfig]
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index f5ff574..e8a694b 100644
--- a/Makefile
+++ b/Makefile
@@ -612,7 +612,7 @@ target-post-image: $(TARGETS_ROOTFS) target-finalize
 		$(call MESSAGE,"Executing post-image script $(s)"); \
 		$(EXTRA_ENV) $(s) $(BINARIES_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep))
 
-source: $(PACKAGES_SOURCE) $(HOST_SOURCE)
+source: $(foreach p,$(PACKAGES),$(p)-all-source)
 
 _external-deps: $(foreach p,$(PACKAGES),$(p)-all-external-deps)
 external-deps:
-- 
2.1.0

[Buildroot] [PATCHv4 15/18] Makefile: remove unneeded variables

[Thomas Petazzoni]

Now that all the external-deps, source-check and source targets are
properly implemented based on the package infrastructure, the
PACKAGES_SOURCE, TARGET_HOST_DEPS, HOST_DEPS and HOST_SOURCE variables
are no longer needed. This is a good thing since they were anyway
incorrect, as they were only doing a two level recursion in the
dependencies of host packages.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[tested with a randpackageconfig]
---
 Makefile | 21 ---------------------
 1 file changed, 21 deletions(-)

diff --git a/Makefile b/Makefile
index e8a694b..0ba341b 100644
--- a/Makefile
+++ b/Makefile
@@ -389,27 +389,6 @@ include fs/common.mk
 
 include $(BR2_EXTERNAL)/external.mk
 
-PACKAGES_SOURCE := $(patsubst %,%-source,$(PACKAGES))
-
-# host-* dependencies have to be handled specially, as those aren't
-# visible in Kconfig and hence not added to a variable like PACKAGES.
-# instead, find all the host-* targets listed in each <PKG>_DEPENDENCIES
-# variable for each enabled target.
-# Notice: this only works for newstyle gentargets/autotargets packages
-TARGETS_HOST_DEPS = $(sort $(filter host-%,$(foreach dep,\
-		$(addsuffix _DEPENDENCIES,\
-			$(call UPPERCASE,$(PACKAGES) $(TARGETS_ROOTFS))),\
-		$($(dep)))))
-# Host packages can in turn have their own dependencies. Likewise find
-# all the package names listed in the HOST_<PKG>_DEPENDENCIES for each
-# host package found above. Ideally this should be done recursively until
-# no more packages are found, but that's hard to do in make, so limit to
-# 1 level for now.
-HOST_DEPS = $(sort $(foreach dep,\
-		$(addsuffix _DEPENDENCIES,$(call UPPERCASE,$(TARGETS_HOST_DEPS))),\
-		$($(dep))))
-HOST_SOURCE += $(addsuffix -source,$(sort $(TARGETS_HOST_DEPS) $(HOST_DEPS)))
-
 dirs: $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
 	$(HOST_DIR) $(BINARIES_DIR)
 
-- 
2.1.0

[Buildroot] [PATCHv4 16/18] Makefile: add BR_BUILDING variable

[Thomas Petazzoni]

Some packages do some sanity checks on their configuration, for
example linux checks that the defconfig string is not empty when a
defconfig is used. Such checks are currently always performed, except
when the 'source' target is part of make goals.

This is problematic for two reasons:

 - Other targets such as 'source-check', 'external-deps' or
   'legal-info', that do not consist in doing a build, cannot be
   executed in such situations.

 - The current code removes the check as soon as one of the targets is
   source. But if there are other non-source targets called at the
   same time, the checks are ignored.

This commit therefore introduces an internal variable called
BR_BUILDING, which tells packages if we are actually building or
not. A variable nobuild_targets indicates the targets that we do not
consider as being build targets.

For the moment, nobuild_targets only contains 'source', to be
completely iso-functional.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
 Makefile | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/Makefile b/Makefile
index 0ba341b..e0be636 100644
--- a/Makefile
+++ b/Makefile
@@ -77,6 +77,22 @@ noconfig_targets := menuconfig nconfig gconfig xconfig config oldconfig randconf
 	randpackageconfig allyespackageconfig allnopackageconfig \
 	print-version olddefconfig
 
+# Some global targets do not trigger a build, but are used to collect
+# metadata, or do various checks. When such targets are triggered,
+# some packages should not do their configuration sanity
+# checks. Provide them a BR_BUILDING variable set to 'y' when we're
+# actually building and they should do their sanity checks.
+#
+# We're building in two situations: when MAKECMDGOALS is empty
+# (default target is to build), or when MAKECMDGOALS contains
+# something else than one of the nobuild_targets.
+nobuild_targets := source
+ifeq ($(MAKECMDGOALS),)
+BR_BUILDING = y
+else ifneq ($(filter-out $(nobuild_targets),$(MAKECMDGOALS)),)
+BR_BUILDING = y
+endif
+
 # Strip quotes and then whitespaces
 qstrip = $(strip $(subst ",,$(1)))
 #"))
-- 
2.1.0

[Buildroot] [PATCHv4 17/18] packages: refactor checks using BR_BUILDING

[Thomas Petazzoni]

Instead of manually testing MAKECMDGOALS, use the newly introduced
BR_BUILDING variable to know if we're building or not.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 boot/at91bootstrap/at91bootstrap.mk   |  5 +----
 boot/at91bootstrap3/at91bootstrap3.mk |  2 +-
 boot/barebox/barebox.mk               |  5 +----
 boot/uboot/uboot.mk                   | 13 ++++---------
 boot/xloader/xloader.mk               |  6 +-----
 linux/linux.mk                        |  2 +-
 package/sunxi-boards/sunxi-boards.mk  |  4 +---
 7 files changed, 10 insertions(+), 27 deletions(-)

diff --git a/boot/at91bootstrap/at91bootstrap.mk b/boot/at91bootstrap/at91bootstrap.mk
index 802e85e..f655058 100644
--- a/boot/at91bootstrap/at91bootstrap.mk
+++ b/boot/at91bootstrap/at91bootstrap.mk
@@ -40,11 +40,8 @@ endef
 
 $(eval $(generic-package))
 
-ifeq ($(BR2_TARGET_AT91BOOTSTRAP),y)
-# we NEED a board name unless we're at make source
-ifeq ($(filter source,$(MAKECMDGOALS)),)
+ifeq ($(BR2_TARGET_AT91BOOTSTRAP)$(BR_BUILDING),yy)
 ifeq ($(AT91BOOTSTRAP_BOARD),)
 $(error No AT91Bootstrap board name set. Check your BR2_TARGET_AT91BOOTSTRAP_BOARD setting)
 endif
 endif
-endif
diff --git a/boot/at91bootstrap3/at91bootstrap3.mk b/boot/at91bootstrap3/at91bootstrap3.mk
index 3e3b78e..ff49e14 100644
--- a/boot/at91bootstrap3/at91bootstrap3.mk
+++ b/boot/at91bootstrap3/at91bootstrap3.mk
@@ -43,7 +43,7 @@ AT91BOOTSTRAP3_KCONFIG_OPTS = $(AT91BOOTSTRAP3_MAKE_OPTS)
 $(eval $(kconfig-package))
 
 # Checks to give errors that the user can understand
-ifeq ($(filter source,$(MAKECMDGOALS)),)
+ifeq ($(BR_BUILDING),y)
 ifeq ($(BR2_TARGET_AT91BOOTSTRAP3_USE_DEFCONFIG),y)
 ifeq ($(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_DEFCONFIG)),)
 $(error No at91bootstrap3 defconfig name specified, check your BR2_TARGET_AT91BOOTSTRAP3_DEFCONFIG setting)
diff --git a/boot/barebox/barebox.mk b/boot/barebox/barebox.mk
index 98cb04e..54f52bb 100644
--- a/boot/barebox/barebox.mk
+++ b/boot/barebox/barebox.mk
@@ -106,11 +106,8 @@ endif
 
 $(eval $(kconfig-package))
 
-ifeq ($(BR2_TARGET_BAREBOX),y)
-# we NEED a board defconfig file unless we're at make source
-ifeq ($(filter source,$(MAKECMDGOALS)),)
+ifeq ($(BR2_TARGET_BAREBOX)$(BR_BUILDING),yy)
 ifeq ($(BAREBOX_SOURCE_CONFIG),)
 $(error No Barebox config file. Check your BR2_TARGET_BAREBOX_BOARD_DEFCONFIG or BR2_TARGET_BAREBOX_CUSTOM_CONFIG_FILE settings)
 endif
 endif
-endif
diff --git a/boot/uboot/uboot.mk b/boot/uboot/uboot.mk
index 1526177..01aaf2c 100644
--- a/boot/uboot/uboot.mk
+++ b/boot/uboot/uboot.mk
@@ -164,8 +164,7 @@ define UBOOT_INSTALL_OMAP_IFT_IMAGE
 endef
 
 ifeq ($(BR2_TARGET_UBOOT_OMAP_IFT),y)
-# we NEED a config file unless we're at make source
-ifeq ($(filter source,$(MAKECMDGOALS)),)
+ifeq ($(BR_BUILDING),y)
 ifeq ($(call qstrip,$(BR2_TARGET_UBOOT_OMAP_IFT_CONFIG)),)
 $(error No gpsign config file. Check your BR2_TARGET_UBOOT_OMAP_IFT_CONFIG setting)
 endif
@@ -179,8 +178,7 @@ UBOOT_POST_INSTALL_IMAGES_HOOKS += UBOOT_INSTALL_OMAP_IFT_IMAGE
 endif
 
 ifeq ($(BR2_TARGET_UBOOT_ENVIMAGE),y)
-# we NEED a environment settings unless we're at make source
-ifeq ($(filter source,$(MAKECMDGOALS)),)
+ifeq ($(BR_BUILDING),y)
 ifeq ($(call qstrip,$(BR2_TARGET_UBOOT_ENVIMAGE_SOURCE)),)
 $(error Please define a source file for Uboot environment (BR2_TARGET_UBOOT_ENVIMAGE_SOURCE setting))
 endif
@@ -193,9 +191,7 @@ endif
 
 $(eval $(generic-package))
 
-ifeq ($(BR2_TARGET_UBOOT),y)
-# we NEED a board name unless we're at make source
-ifeq ($(filter source,$(MAKECMDGOALS)),)
+ifeq ($(BR2_TARGET_UBOOT)$(BR_BUILDING),yy)
 ifeq ($(UBOOT_BOARD_NAME),)
 $(error No U-Boot board name set. Check your BR2_TARGET_UBOOT_BOARDNAME setting)
 endif
@@ -221,5 +217,4 @@ $(error No custom U-Boot repository URL specified. Check your BR2_TARGET_UBOOT_C
 endif # qstrip BR2_TARGET_UBOOT_CUSTOM_CUSTOM_REPO_VERSION
 endif # BR2_TARGET_UBOOT_CUSTOM_GIT || BR2_TARGET_UBOOT_CUSTOM_HG
 
-endif # filter source
-endif # BR2_TARGET_UBOOT
+endif # BR2_TARGET_UBOOT && BR_BUILDING
diff --git a/boot/xloader/xloader.mk b/boot/xloader/xloader.mk
index 90d5fa2..c68dc81 100644
--- a/boot/xloader/xloader.mk
+++ b/boot/xloader/xloader.mk
@@ -25,12 +25,8 @@ endef
 
 $(eval $(generic-package))
 
-ifeq ($(BR2_TARGET_XLOADER),y)
-# we NEED a board name unless we're at make source
-ifeq ($(filter source,$(MAKECMDGOALS)),)
+ifeq ($(BR2_TARGET_XLOADER)$(BR_BUILDING),yy)
 ifeq ($(XLOADER_BOARD_NAME),)
 $(error NO x-loader board name set. Check your BR2_BOOT_XLOADER_BOARDNAME setting)
 endif
 endif
-
-endif
diff --git a/linux/linux.mk b/linux/linux.mk
index deed073..0c348da 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -354,7 +354,7 @@ $(LINUX_DIR)/.stamp_initramfs_rebuilt: $(LINUX_DIR)/.stamp_target_installed $(LI
 linux-rebuild-with-initramfs: $(LINUX_DIR)/.stamp_initramfs_rebuilt
 
 # Checks to give errors that the user can understand
-ifeq ($(filter source,$(MAKECMDGOALS)),)
+ifeq ($(BR_BUILDING),y)
 ifeq ($(BR2_LINUX_KERNEL_USE_DEFCONFIG),y)
 ifeq ($(call qstrip,$(BR2_LINUX_KERNEL_DEFCONFIG)),)
 $(error No kernel defconfig name specified, check your BR2_LINUX_KERNEL_DEFCONFIG setting)
diff --git a/package/sunxi-boards/sunxi-boards.mk b/package/sunxi-boards/sunxi-boards.mk
index 1a2bb4c..cacf824 100644
--- a/package/sunxi-boards/sunxi-boards.mk
+++ b/package/sunxi-boards/sunxi-boards.mk
@@ -16,13 +16,11 @@ define SUNXI_BOARDS_INSTALL_IMAGES_CMDS
 		$(BINARIES_DIR)/script.bin
 endef
 
-ifeq ($(BR2_PACKAGE_SUNXI_BOARDS),y)
+ifeq ($(BR2_PACKAGE_SUNXI_BOARDS)$(BR_BUILDING),yy)
 # we NEED a board name
-ifeq ($(filter source,$(MAKECMDGOALS)),)
 ifeq ($(SUNXI_BOARDS_FEX_FILE),)
 $(error No sunxi .fex file specified. Check your BR2_PACKAGE_SUNXI_BOARDS_FEX_FILE settings)
 endif
 endif
-endif
 
 $(eval $(generic-package))
-- 
2.1.0

[Buildroot] [PATCHv4 18/18] Makefile: add a few more targets to nobuild_targets

[Thomas Petazzoni]

With this commit, one can now execute the source-check, external-deps
and legal-info targets regardless of the checks normally being done by
packages on the configuration.

Note that we intentionally do not go down the road of adding %-source,
%-legal-info, and the miryad of other targets that could work in such
situations. We only whitelist a few targets that are really useful to
have as nobuild_targets.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index e0be636..2d7c05a 100644
--- a/Makefile
+++ b/Makefile
@@ -86,7 +86,7 @@ noconfig_targets := menuconfig nconfig gconfig xconfig config oldconfig randconf
 # We're building in two situations: when MAKECMDGOALS is empty
 # (default target is to build), or when MAKECMDGOALS contains
 # something else than one of the nobuild_targets.
-nobuild_targets := source
+nobuild_targets := source source-check legal-info external-deps _external-deps
 ifeq ($(MAKECMDGOALS),)
 BR_BUILDING = y
 else ifneq ($(filter-out $(nobuild_targets),$(MAKECMDGOALS)),)
-- 
2.1.0

[Buildroot] [PATCHv4 01/18] fs: only add non rootfs- targets to PACKAGES

[Thomas Petazzoni]

Hello,

On Sun, 26 Apr 2015 11:50:59 +0200, Thomas Petazzoni wrote:
> Commit 4570dbcb14b394d8f3c97baf4c9e5f4f225d1d37
> ("4570dbcb14b394d8f3c97baf4c9e5f4f225d1d37") added the dependencies of
> rootfs building logic to PACKAGES, only thinking about the case where
> rootfs building logic actually depends on real, normal packages
> (host-mtd, host-genext2fs, etc.). But some file systems (rootfs-ubi)
> depend on other filesystem images (rootfs-ubifs). And such targets
> should not be added to PACKAGES, otherwise an incorrect circular
> dependency is created.
> 
> This commit fixes that by only adding the rootfs building logic
> dependencies that do *not* start with rootfs- to the global PACKAGES
> variable.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Patch applied.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv4 02/18] pkg-generic: take into account patch dependencies in source, external-deps and legal-info

[Yann E. MORIN]

Thomas, All,

On 2015-04-26 11:51 +0200, Thomas Petazzoni spake thusly:
> The $(1)-all-{source,external-deps,legal-info} targets currently only
> take care of the dependencies in <pkg>_DEPENDENCIES, but not
> <pkg>_PATCH_DEPENDENCIES. This patch fixes that by introducing a
> <pkg>_FINAL_ALL_DEPENDENCIES. It also reworks the $(1)-show-depends
> target to make use of <pkg>_FINALL_ALL_DEPENDENCIES.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[tested a "make source" and "make external-deps" and "make legal-info"
 on a randpackageconfig]
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Regards,
Yann E. MORIN.

> ---
>  package/pkg-generic.mk | 12 ++++++++----
>  1 file changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index 455bdf1..dd8c4c9 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -431,6 +431,7 @@ endif
>  # Eliminate duplicates in dependencies
>  $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES))
>  $(2)_FINAL_PATCH_DEPENDENCIES = $$(sort $$($(2)_PATCH_DEPENDENCIES))
> +$(2)_FINAL_ALL_DEPENDENCIES = $$(sort $$($(2)_FINAL_DEPENDENCIES) $$($(2)_FINAL_PATCH_DEPENDENCIES))
>  
>  $(2)_INSTALL_STAGING		?= NO
>  $(2)_INSTALL_IMAGES		?= NO
> @@ -583,7 +584,7 @@ $(1)-show-version:
>  			@echo $$($(2)_VERSION)
>  
>  $(1)-show-depends:
> -			@echo $$(sort $$($(2)_FINAL_DEPENDENCIES) $$($(2)_FINAL_PATCH_DEPENDENCIES))
> +			@echo $$($(2)_FINAL_ALL_DEPENDENCIES)
>  
>  $(1)-graph-depends: graph-depends-requirements
>  			@$$(INSTALL) -d $$(GRAPHS_DIR)
> @@ -592,11 +593,14 @@ $(1)-graph-depends: graph-depends-requirements
>  			|tee $$(GRAPHS_DIR)/$$(@).dot \
>  			|dot $$(BR2_GRAPH_DOT_OPTS) -T$$(BR_GRAPH_OUT) -o $$(GRAPHS_DIR)/$$(@).$$(BR_GRAPH_OUT)
>  
> -$(1)-all-source:       $$(foreach p,$$($(2)_FINAL_DEPENDENCIES),$$(p)-all-source) $(1)-source
> +$(1)-all-source:	$(1)-source
> +$(1)-all-source:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-source)
>  
> -$(1)-all-external-deps:        $$(foreach p,$$($(2)_FINAL_DEPENDENCIES),$$(p)-all-external-deps) $(1)-external-deps
> +$(1)-all-external-deps:	$(1)-external-deps
> +$(1)-all-external-deps:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-external-deps)
>  
> -$(1)-all-legal-info:   $$(foreach p,$$($(2)_FINAL_DEPENDENCIES),$$(p)-all-legal-info) $(1)-legal-info
> +$(1)-all-legal-info:	$(1)-legal-info
> +$(1)-all-legal-info:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-legal-info)
>  
>  $(1)-dirclean:		$$($(2)_TARGET_DIRCLEAN)
>  
> -- 
> 2.1.0
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCHv4 07/18] pkg-generic: implement source-check targets

[Yann E. MORIN]

Thomas, All,

On 2015-04-26 11:51 +0200, Thomas Petazzoni spake thusly:
> This commits extends the common package infrastructure with the
> $(1)-source-check and $(1)-all-source-check targets.
> 
> The $(1)-source-check target simply calls the newly added
> SOURCE_CHECK macro on all items to be downloaded.
> 
> The $(1)-all-source-check target will depend on the
> $(1)-all-source-check targets of all dependent packages and the
> $(1)-source-check target of the current package, which allows to do a
> recursive source-check in the dependency tree.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  package/pkg-generic.mk | 17 ++++++++++++++++-
>  1 file changed, 16 insertions(+), 1 deletion(-)
> 
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index d7e3ca9..2176771 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -580,6 +580,16 @@ endif
>  $(1)-show-version:
>  			@echo $$($(2)_VERSION)
>  
> +$(1)-source-check:
> +ifeq ($$($(2)_OVERRIDE_SRCDIR),)
> +	$$(foreach p,$$($(2)_SOURCE) $$($(2)_EXTRA_DOWNLOADS) $$($(2)_PATCH),\
> +		$$(if $$(findstring ://,$$(p)),\
> +			$$(call SOURCE_CHECK,$$(p)),\
> +			$$(call SOURCE_CHECK,$$($(2)_SITE:/=)/$$(p)))$$(sep))
> +else
> +	test -d $$($(2)_OVERRIDE_SRCDIR)
> +endif

Why don;t you split this inside the OVERRIDE_SRCDIR condition, just
above, along with the current $(1)-source ?

Regards,
Yann E. MORIN.

>  $(1)-show-depends:
>  			@echo $$($(2)_FINAL_ALL_DEPENDENCIES)
>  
> @@ -593,6 +603,9 @@ $(1)-graph-depends: graph-depends-requirements
>  $(1)-all-source:	$(1)-source
>  $(1)-all-source:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-source)
>  
> +$(1)-all-source-check:	$(1)-source-check
> +$(1)-all-source-check:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-source-check)
> +
>  $(1)-all-external-deps:	$(1)-external-deps
>  $(1)-all-external-deps:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-external-deps)
>  
> @@ -788,6 +801,7 @@ endif
>  	$(1)-all-external-deps \
>  	$(1)-all-legal-info \
>  	$(1)-all-source \
> +	$(1)-all-source-check \
>  	$(1)-build \
>  	$(1)-clean-for-rebuild \
>  	$(1)-clean-for-reconfigure \
> @@ -811,7 +825,8 @@ endif
>  	$(1)-rsync \
>  	$(1)-show-depends \
>  	$(1)-show-version \
> -	$(1)-source
> +	$(1)-source \
> +	$(1)-source-check
>  
>  endif # $(2)_KCONFIG_VAR
>  endef # inner-generic-package
> -- 
> 2.1.0
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCHv4 17/18] packages: refactor checks using BR_BUILDING

[Yann E. MORIN]

Thomas, All,

On 2015-04-26 11:51 +0200, Thomas Petazzoni spake thusly:
> Instead of manually testing MAKECMDGOALS, use the newly introduced
> BR_BUILDING variable to know if we're building or not.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[tested with a randpackageconfig + xloader enabled without a board name
set]

Regards,
Yann E. MORIN.

> ---
>  boot/at91bootstrap/at91bootstrap.mk   |  5 +----
>  boot/at91bootstrap3/at91bootstrap3.mk |  2 +-
>  boot/barebox/barebox.mk               |  5 +----
>  boot/uboot/uboot.mk                   | 13 ++++---------
>  boot/xloader/xloader.mk               |  6 +-----
>  linux/linux.mk                        |  2 +-
>  package/sunxi-boards/sunxi-boards.mk  |  4 +---
>  7 files changed, 10 insertions(+), 27 deletions(-)
> 
> diff --git a/boot/at91bootstrap/at91bootstrap.mk b/boot/at91bootstrap/at91bootstrap.mk
> index 802e85e..f655058 100644
> --- a/boot/at91bootstrap/at91bootstrap.mk
> +++ b/boot/at91bootstrap/at91bootstrap.mk
> @@ -40,11 +40,8 @@ endef
>  
>  $(eval $(generic-package))
>  
> -ifeq ($(BR2_TARGET_AT91BOOTSTRAP),y)
> -# we NEED a board name unless we're at make source
> -ifeq ($(filter source,$(MAKECMDGOALS)),)
> +ifeq ($(BR2_TARGET_AT91BOOTSTRAP)$(BR_BUILDING),yy)
>  ifeq ($(AT91BOOTSTRAP_BOARD),)
>  $(error No AT91Bootstrap board name set. Check your BR2_TARGET_AT91BOOTSTRAP_BOARD setting)
>  endif
>  endif
> -endif
> diff --git a/boot/at91bootstrap3/at91bootstrap3.mk b/boot/at91bootstrap3/at91bootstrap3.mk
> index 3e3b78e..ff49e14 100644
> --- a/boot/at91bootstrap3/at91bootstrap3.mk
> +++ b/boot/at91bootstrap3/at91bootstrap3.mk
> @@ -43,7 +43,7 @@ AT91BOOTSTRAP3_KCONFIG_OPTS = $(AT91BOOTSTRAP3_MAKE_OPTS)
>  $(eval $(kconfig-package))
>  
>  # Checks to give errors that the user can understand
> -ifeq ($(filter source,$(MAKECMDGOALS)),)
> +ifeq ($(BR_BUILDING),y)
>  ifeq ($(BR2_TARGET_AT91BOOTSTRAP3_USE_DEFCONFIG),y)
>  ifeq ($(call qstrip,$(BR2_TARGET_AT91BOOTSTRAP3_DEFCONFIG)),)
>  $(error No at91bootstrap3 defconfig name specified, check your BR2_TARGET_AT91BOOTSTRAP3_DEFCONFIG setting)
> diff --git a/boot/barebox/barebox.mk b/boot/barebox/barebox.mk
> index 98cb04e..54f52bb 100644
> --- a/boot/barebox/barebox.mk
> +++ b/boot/barebox/barebox.mk
> @@ -106,11 +106,8 @@ endif
>  
>  $(eval $(kconfig-package))
>  
> -ifeq ($(BR2_TARGET_BAREBOX),y)
> -# we NEED a board defconfig file unless we're at make source
> -ifeq ($(filter source,$(MAKECMDGOALS)),)
> +ifeq ($(BR2_TARGET_BAREBOX)$(BR_BUILDING),yy)
>  ifeq ($(BAREBOX_SOURCE_CONFIG),)
>  $(error No Barebox config file. Check your BR2_TARGET_BAREBOX_BOARD_DEFCONFIG or BR2_TARGET_BAREBOX_CUSTOM_CONFIG_FILE settings)
>  endif
>  endif
> -endif
> diff --git a/boot/uboot/uboot.mk b/boot/uboot/uboot.mk
> index 1526177..01aaf2c 100644
> --- a/boot/uboot/uboot.mk
> +++ b/boot/uboot/uboot.mk
> @@ -164,8 +164,7 @@ define UBOOT_INSTALL_OMAP_IFT_IMAGE
>  endef
>  
>  ifeq ($(BR2_TARGET_UBOOT_OMAP_IFT),y)
> -# we NEED a config file unless we're at make source
> -ifeq ($(filter source,$(MAKECMDGOALS)),)
> +ifeq ($(BR_BUILDING),y)
>  ifeq ($(call qstrip,$(BR2_TARGET_UBOOT_OMAP_IFT_CONFIG)),)
>  $(error No gpsign config file. Check your BR2_TARGET_UBOOT_OMAP_IFT_CONFIG setting)
>  endif
> @@ -179,8 +178,7 @@ UBOOT_POST_INSTALL_IMAGES_HOOKS += UBOOT_INSTALL_OMAP_IFT_IMAGE
>  endif
>  
>  ifeq ($(BR2_TARGET_UBOOT_ENVIMAGE),y)
> -# we NEED a environment settings unless we're at make source
> -ifeq ($(filter source,$(MAKECMDGOALS)),)
> +ifeq ($(BR_BUILDING),y)
>  ifeq ($(call qstrip,$(BR2_TARGET_UBOOT_ENVIMAGE_SOURCE)),)
>  $(error Please define a source file for Uboot environment (BR2_TARGET_UBOOT_ENVIMAGE_SOURCE setting))
>  endif
> @@ -193,9 +191,7 @@ endif
>  
>  $(eval $(generic-package))
>  
> -ifeq ($(BR2_TARGET_UBOOT),y)
> -# we NEED a board name unless we're at make source
> -ifeq ($(filter source,$(MAKECMDGOALS)),)
> +ifeq ($(BR2_TARGET_UBOOT)$(BR_BUILDING),yy)
>  ifeq ($(UBOOT_BOARD_NAME),)
>  $(error No U-Boot board name set. Check your BR2_TARGET_UBOOT_BOARDNAME setting)
>  endif
> @@ -221,5 +217,4 @@ $(error No custom U-Boot repository URL specified. Check your BR2_TARGET_UBOOT_C
>  endif # qstrip BR2_TARGET_UBOOT_CUSTOM_CUSTOM_REPO_VERSION
>  endif # BR2_TARGET_UBOOT_CUSTOM_GIT || BR2_TARGET_UBOOT_CUSTOM_HG
>  
> -endif # filter source
> -endif # BR2_TARGET_UBOOT
> +endif # BR2_TARGET_UBOOT && BR_BUILDING
> diff --git a/boot/xloader/xloader.mk b/boot/xloader/xloader.mk
> index 90d5fa2..c68dc81 100644
> --- a/boot/xloader/xloader.mk
> +++ b/boot/xloader/xloader.mk
> @@ -25,12 +25,8 @@ endef
>  
>  $(eval $(generic-package))
>  
> -ifeq ($(BR2_TARGET_XLOADER),y)
> -# we NEED a board name unless we're at make source
> -ifeq ($(filter source,$(MAKECMDGOALS)),)
> +ifeq ($(BR2_TARGET_XLOADER)$(BR_BUILDING),yy)
>  ifeq ($(XLOADER_BOARD_NAME),)
>  $(error NO x-loader board name set. Check your BR2_BOOT_XLOADER_BOARDNAME setting)
>  endif
>  endif
> -
> -endif
> diff --git a/linux/linux.mk b/linux/linux.mk
> index deed073..0c348da 100644
> --- a/linux/linux.mk
> +++ b/linux/linux.mk
> @@ -354,7 +354,7 @@ $(LINUX_DIR)/.stamp_initramfs_rebuilt: $(LINUX_DIR)/.stamp_target_installed $(LI
>  linux-rebuild-with-initramfs: $(LINUX_DIR)/.stamp_initramfs_rebuilt
>  
>  # Checks to give errors that the user can understand
> -ifeq ($(filter source,$(MAKECMDGOALS)),)
> +ifeq ($(BR_BUILDING),y)
>  ifeq ($(BR2_LINUX_KERNEL_USE_DEFCONFIG),y)
>  ifeq ($(call qstrip,$(BR2_LINUX_KERNEL_DEFCONFIG)),)
>  $(error No kernel defconfig name specified, check your BR2_LINUX_KERNEL_DEFCONFIG setting)
> diff --git a/package/sunxi-boards/sunxi-boards.mk b/package/sunxi-boards/sunxi-boards.mk
> index 1a2bb4c..cacf824 100644
> --- a/package/sunxi-boards/sunxi-boards.mk
> +++ b/package/sunxi-boards/sunxi-boards.mk
> @@ -16,13 +16,11 @@ define SUNXI_BOARDS_INSTALL_IMAGES_CMDS
>  		$(BINARIES_DIR)/script.bin
>  endef
>  
> -ifeq ($(BR2_PACKAGE_SUNXI_BOARDS),y)
> +ifeq ($(BR2_PACKAGE_SUNXI_BOARDS)$(BR_BUILDING),yy)
>  # we NEED a board name
> -ifeq ($(filter source,$(MAKECMDGOALS)),)
>  ifeq ($(SUNXI_BOARDS_FEX_FILE),)
>  $(error No sunxi .fex file specified. Check your BR2_PACKAGE_SUNXI_BOARDS_FEX_FILE settings)
>  endif
>  endif
> -endif
>  
>  $(eval $(generic-package))
> -- 
> 2.1.0
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCHv4 18/18] Makefile: add a few more targets to nobuild_targets

[Yann E. MORIN]

On 2015-04-26 11:51 +0200, Thomas Petazzoni spake thusly:
> With this commit, one can now execute the source-check, external-deps
> and legal-info targets regardless of the checks normally being done by
> packages on the configuration.
> 
> Note that we intentionally do not go down the road of adding %-source,
> %-legal-info, and the miryad of other targets that could work in such
> situations. We only whitelist a few targets that are really useful to
> have as nobuild_targets.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/Makefile b/Makefile
> index e0be636..2d7c05a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -86,7 +86,7 @@ noconfig_targets := menuconfig nconfig gconfig xconfig config oldconfig randconf
>  # We're building in two situations: when MAKECMDGOALS is empty
>  # (default target is to build), or when MAKECMDGOALS contains
>  # something else than one of the nobuild_targets.
> -nobuild_targets := source
> +nobuild_targets := source source-check legal-info external-deps _external-deps

You may also want to add those:
    clean distclean

Because otherwise, it is not possible to clean or distclean a directory
when the .config has one or more unset board names.

Regards,
Yann E. MORIN.

>  ifeq ($(MAKECMDGOALS),)
>  BR_BUILDING = y
>  else ifneq ($(filter-out $(nobuild_targets),$(MAKECMDGOALS)),)
> -- 
> 2.1.0
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCHv4 07/18] pkg-generic: implement source-check targets

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Sun, 26 Apr 2015 17:20:28 +0200, Yann E. MORIN wrote:

> Why don;t you split this inside the OVERRIDE_SRCDIR condition, just
> above, along with the current $(1)-source ?

Good idea. I've changed that before applying. Thanks for the suggestion!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv4 18/18] Makefile: add a few more targets to nobuild_targets

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Sun, 26 Apr 2015 17:40:52 +0200, Yann E. MORIN wrote:

> You may also want to add those:
>     clean distclean
> 
> Because otherwise, it is not possible to clean or distclean a directory
> when the .config has one or more unset board names.

Done! I've added clean and distclean to nobuild_targets before applying.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCHv4 00/18] Package based 'source', 'legal-info', 'source-check' and 'external-deps'

[Thomas Petazzoni]

Hello all,

On Sun, 26 Apr 2015 11:50:58 +0200, Thomas Petazzoni wrote:

> Thomas Petazzoni (18):
>   fs: only add non rootfs- targets to PACKAGES
>   pkg-generic: take into account patch dependencies in source,
>     external-deps and legal-info
>   Makefile: use the package infra based external-deps
>   pkg-download: remove support for the SHOW_EXTERNAL_DEPS DL_MODE
>   Makefile: move source-check outside of noconfig_targets
>   pkg-download: extend DOWNLOAD_INNER, add a SOURCE_CHECK macro
>   pkg-generic: implement source-check targets
>   Makefile: implement a package based source-check target
>   pkg-generic: remove the .stamp_rsync_sourced fake stamp file
>   pkg-generic: don't use DL_MODE in .stamp_downloaded
>   pkg-download: get rid of DL_MODE
>   pkg-download: fix indentation for SOURCE_CHECK_* macros
>   pkg-generic: introduce a <pkg>_ALL_DOWNLOADS variable and factorize
>     code
>   Makefile: implement the 'source' target using the package
>     infrastructure
>   Makefile: remove unneeded variables
>   Makefile: add BR_BUILDING variable
>   packages: refactor checks using BR_BUILDING
>   Makefile: add a few more targets to nobuild_targets

This patch series is now fully applied. Thanks a lot to Yann and Arnout
for helping with their in-depth review and testing!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com