[Buildroot] [PATCH v4 0/9] Preparation for per-package host/target directories

[Buildroot] [PATCH v4 0/9] Preparation for per-package host/target directories

[Thomas Petazzoni]

Hello,

Here is a fourth iteration of the per-package host and target
directory work, but without the actual per-package host and directory
patches: it's only some preparation patches.

If you're interested in testing it, you can find the patch series at:

  http://git.free-electrons.com/users/thomas-petazzoni/buildroot/log/?h=ppsh-v4-preparation

I'm keeping the full explanation about per-package SDK below, but
obviously this is only to give some context, because this series by
itself does not implement it.

Changes since v3
================

 - Dropped patches that have been merged upstream:
   pkgconf: use relative path to  STAGING_DIR instead of absolute path
   toolchain: post-pone evaluation of  TOOLCHAIN_EXTERNAL_BIN

 - For now, removed the most controversial/complicated patches
   (changing the rpath strategy, and switching to per-package
   host/target directories). The goal for now is to merge only the
   preparation patches.

 - Fixed "Makefile, skeleton: move the host skeleton logic to
   host-skeleton package" to use $(Q) instead of @ as suggested by
   Yann.

 - Reviewed-by from Yann added on:
   Makefile, skeleton: move the host skeleton logic to host-skeleton package
   pkg-cmake: install CMake files as part of a package
   package/pkg-generic: add the concept of extract dependency
   package/pkg-generic: handle host-tar as an extract dependency
   package/pkg-generic: handle host-xz as an extract dependency
   package/pkg-generic: handle host-lzip as an extract dependency
   package/pkg-generic: handle host-fakedate as a regular dependency

 - Acked-by from Yann added on:
   core: kill DEPENDENCIES_HOST_PREREQ

 - Fix typo in _EXTRACT_DEPENDENCIES documentation.

 - Add BR2_TAR_HOST_DEPENDENCY as a dependency of the tar filesystem
   format, as suggested by Yann.

 - Use = instead of += when defining BR2_TAR_HOST_DEPENDENCY.

 - Make sure host-ccache is not added as a dependency of itself.

 - Make sure host-fakedate is not added as a dependency of itself.

Changes since v2
================

 - Solved the problem of all DEPENDENCIES_HOST_PREREQ targets:
   host-ccache, host-tar, host-lzip, host-xz, host-fakedate.

   To achieve this, introduced <pkg>_EXTRACT_DEPENDENCIES and used
   that to handle the dependencies on host-tar, host-lzip and host-xz.

   Used regular dependencies for host-ccache and host-fakedate.

   See below for more details.

Changes since v1
================

 - Rebased on the latest Buildroot next branch

 - The pkg-config changes are reworked according to Arnout's comments:
   only @STAGING_SUBDIR@ is replaced in pkg-config.in to generate
   pkg-config, the rest of the logic is internal to the script.

 - New patch to move the "host skeleton" logic into a proper
   host-skeleton package.

 - New patch to have the CMake related files generated as part of the
   toolchain package.

 - New patch to add a new "install" step that depends on the different
   install steps (target, staging, images, host). This is needed to
   later add some logic that is executed once all install steps of a
   package have been done.

 - Fix the approach to solve the RPATH logic: instead of using
   LD_LIBRARY_PATH, we actually fix with patchelf the RPATH of host
   binaries right after their installation.

 - Misc other improvements in the per-package SDK implementation.

What is per-package SDK and target?
===================================

Currently, output/host (SDK) and output/target are global directories:
all packages use files installed by other packages in those global
directories, and in turn into those global directories.

The idea of per-package SDK and target is that instead of having such
global directories, we would have one SDK directory and one target
directory per-package, containing just what this package needs
(dependencies), and in which the package installs its files.

Why do we want per-package SDK and target?
==========================================

There are two main motivations for per-package SDK and target
directories, which are related:

 1. Since the SDK directory is global, a package can currently see
    libraries/headers that it has not explicitly expressed a
    dependency on. So package A may not have a dependency on package
    B, but if package B happens to have been installed before, and
    package A "configure" script automaticatically detects a library
    installed by package B, it will use it. We have added tons of
    conditional dependencies in Buildroot to make such situations less
    problematic, but it is an endless fight, as upstream developers
    constantly add more optional dependencies to their software.

    Using per-package SDK, a given package uses as its SDK a directory
    that only contains the libraries/headers from packages explicitly
    listed in its dependencies. So it cannot mistakenly see a
    library/header installed by another package.

 2. Top-level parallel build (building multiple packages in parallel)
    is currently not enabled because we have global SDK and target
    directories.

    Let's imagine package A configure script auto-detects a library
    provided by package B, but Buildroot does not encode this
    dependency in package A's .mk file. Package A and package B are
    therefore totally independent from a build dependency point of
    view.

    Without top-level parallel build (current situation), you have a
    guarantee on the build order: either A is always built before B,
    or B is always built before A. So at least, every build gives the
    same result: A is built with B's functionality or without it, but
    it is consistent accross rebuilds.

    If you enable top-level parallel build, because A and B are
    totally independent, you can get A built before B or B built
    before A depending on the build scheduling. This can change at
    every build! This means that for a given configuration, A will
    sometimes be built with B's functionality, sometimes not. Totally
    unacceptable.

    And of course, this extends to case where package B gets installed
    while package A is being configured. So package A configure script
    will see some parts of B being installed, other parts not
    installed, causing all sort of weird and difficult to debug race
    conditions.

    By having per-package SDK directories, we ensure that package A
    will only see the dependencies its has explicitly expressed, and
    that no other package gets installed in parallel in the same SDK
    directory.

How is it implemented?
======================

The basic idea is pretty simple:

 - For each package, we have:

   output/per-package/<package>/host, which is the specific SDK
   directory for <package>.

   output/per-package/<package>/target, which is the specific target
   directory for <package>.

 - Before <package> is configured, we copy into its specific SDK and
   target directories the SDK and target directories from its direct
   dependencies. This basically populates its SDK with the
   cross-compiler and all libraries that <package> needs. This copy is
   done using rsync with hard links, so it is very fast and cheap from
   a storage consumption point of view.

 - While <package> is being built, HOST_DIR, STAGING_DIR and
   TARGET_DIR are tweaked to point to the current <package> specific
   SDK and target directories. I.e, HOST_DIR no longer points to
   output/host, but to output/per-package/<package>/host/.

And this is basically enough to get things working!

There are of course a few additional things to do:

 - At the end of the build, we still want a global target directory
   (to generate the filesystem images) and a global SDK. Therefore, in
   target-finalize, we assemble such global directories by copying the
   per-package SDK and target directories from all packages listed in
   the $(PACKAGES) variable.

 - We need to fix our pkg-config wrapper script so that it uses
   relative path instead of absolute paths. This allows the wrapper
   script to work regardless of which per-package SDK directory it is
   installed in.

 - We need to move away from using absolute RPATH for host binaries,
   and to achieve that we fix the RPATH to $ORIGIN/../lib in host
   binaries right after the installation of each package.

 - We need to get away from the DEPENDENCIES_HOST_PREREQ mechanism,
   used for host-ccache, host-tar, host-lzip, host-xz and
   host-fakedate. The problem with this mechanism is that because
   those dependencies are not accounted in each package
   <pkg>_DEPENDENCIES variable, the per-package host/target folders
   produced by those packages are not rsync'ed when building other
   packages. Due to this, ccache/tar/lzip/xz/date cannot be found in
   the host directory of the package being built.

   To solve this, we basically move all those packages to be proper
   dependencies of all other packages. This involves adding a few
   exceptions, but more importantly, it requires adding the concept of
   "extract dependency", i.e a dependency that is ready before the
   extract step of a package. This is used for host-tar, host-lzip and
   host-xz. host-ccache and host-fakedate are regular dependencies.

What remains to be done?
========================

 - Completely get rid of the global HOST_DIR, TARGET_DIR and
   STAGING_DIR definitions, so that nothing uses them, and they really
   become per-package variables.

 - While the toolchain sysroot, pkg-config paths and host RPATHs are
   properly handled, there are quite certainly a lot of other places
   that have absolute paths that won't work well with per-package SDK,
   and that need to be adjusted.

 - Perhaps use a temporary per-package SDK and target directory when
   building a package, and rename it to the final name once the
   package has finished building. This would allow to detect
   situations where the per-package SDK directory of package A
   contains absolute paths referring to the per-package SDK directory
   of package B. Such absolute paths are wrong, but we currently don't
   see them because the per-package SDK directory for package B still
   exists.

 - Think about what needs to be done about the output/images/
   folder. Should it also be per-package like we do for output/target?

 - Many other issues that I'm sure people will report.

Comparison with Gustavo's patch series
======================================

Gustavo Zacarias did a previous implementation of this, available in
http://repo.or.cz/buildroot-gz.git/shortlog/refs/heads/pps3-next. Compared
to Gustavo's patch series, this patch series:

 - Implement per-package SDK and target directories, while Gustavo was
   only doing per-package staging.

 - Gustavo had several patches to pass BR_TOOLCHAIN_SYSROOT to the
   toolchain wrapper to pass the location of the current sysroot.

   This is no longer needed, because 1/ we're doing a full per-package
   SDK directory rather than per-package staging, which means the
   sysroot is always at the same relative location compared to the
   cross-compiler binary and 2/ the toolchain wrapper derives the
   sysroot location relatively to the wrapper location. Thanks to
   this, the following patches from Gustavo are (I believe) not
   needed:

   http://repo.or.cz/buildroot-gz.git/commitdiff/1ef6e798064649726d396bcb581ddbe4573c2460
   http://repo.or.cz/buildroot-gz.git/commitdiff/d15f6c6917b555bbbbbf97c292fad4db9d4007d4
   http://repo.or.cz/buildroot-gz.git/commitdiff/95900b65c29a010d85a769c9c6374cf3835f2169
   http://repo.or.cz/buildroot-gz.git/commitdiff/7ed53a5437ab09b85bf6d1953f6ff6049dfcc114

 - Gustavo had a patch to make our pkg-config wrapper script look at
   BR_TOOLCHAIN_SYSROOT, in order to find the
   /usr/{lib,share}/pkgconfig folders of the current sysroot. Just
   like for the toolchain-wrapper, such a change is no longer needed,
   and a simple change to our pkg-config script to use relative paths
   is sufficient.

 - I haven't integrated Gustavo patches that move from $(shell ...) to
   using backticks to delay the evaluation of
   STAGING_DIR/HOST_DIR/TARGET_DIR:

   http://repo.or.cz/buildroot-gz.git/commitdiff/0c11c60865f1445830643bb404f92c20d563260c
   http://repo.or.cz/buildroot-gz.git/commitdiff/207d2248ec8542293b6201b5c86f971021a3a591
   http://repo.or.cz/buildroot-gz.git/commitdiff/7f6a69819fbb176e29a1c3a53b4a76efe87dad15
   http://repo.or.cz/buildroot-gz.git/commitdiff/3328a9745eee555f5e5ef7df835afc26612ecd7b
   http://repo.or.cz/buildroot-gz.git/commitdiff/45a9d8c2aa6f3c0d2d8ed989d843890025faa3e8

 - I haven't looked at Qt specific issues, such as the ones fixed by
   Gustavo in:

   http://repo.or.cz/buildroot-gz.git/commitdiff/643e982e635f4386ccefcda9da4b84536af84d04

 - I am not doing all the .la files, *-config, *.prl, etc. tweaks that
   Gustavo is doing in:

   http://repo.or.cz/buildroot-gz.git/commitdiff/3f7b227b4c8e4514df6e5d54f88fcfb3a3a4bae0

   It is part of the "remaining absolute paths that need to be fixed"
   item that I mentionned above.

Best regards,

Thomas


Thomas Petazzoni (9):
  Makefile, skeleton: move the host skeleton logic to host-skeleton
    package
  pkg-cmake: install CMake files as part of a package
  package/pkg-generic: add the concept of extract dependency
  package/pkg-generic: handle host-tar as an extract dependency
  package/pkg-generic: handle host-xz as an extract dependency
  package/pkg-generic: handle host-lzip as an extract dependency
  package/pkg-generic: handle host-ccache as a regular dependency
  package/pkg-generic: handle host-fakedate as a regular dependency
  core: kill DEPENDENCIES_HOST_PREREQ

 Makefile                                 | 16 +-------------
 docs/manual/adding-packages-generic.txt  |  7 ++++++
 fs/tar/tar.mk                            |  2 ++
 package/ccache/ccache.mk                 |  5 +++++
 package/lzip/lzip.mk                     |  2 +-
 package/pkg-cmake.mk                     | 12 +++++-----
 package/pkg-generic.mk                   | 38 +++++++++++++++++++++++++++++---
 package/skeleton/skeleton.mk             | 12 ++++++++++
 package/tar/tar.mk                       |  5 +++++
 package/xz/xz.mk                         |  5 +++++
 support/dependencies/check-host-lzip.mk  |  2 +-
 support/dependencies/check-host-tar.mk   |  2 +-
 support/dependencies/check-host-xzcat.mk |  2 +-
 support/dependencies/dependencies.mk     | 14 ++----------
 toolchain/toolchain/toolchain.mk         |  3 ---
 15 files changed, 85 insertions(+), 42 deletions(-)

-- 
2.14.3

[Buildroot] [PATCH v4 1/9] Makefile, skeleton: move the host skeleton logic to host-skeleton package

[Thomas Petazzoni]

As part of the per-package SDK work, we want to avoid having logic
that installs files to the global HOST_DIR, and instead do it inside
packages. One thing that gets installed to the global HOST_DIR is the
minimal "skeleton" that we create in host:

 - the "usr" symbolic link for backward compatibility

 - the "lib" directory, and its lib64 or lib32 symbolic links

This commit moves this logic to a new host-skeleton package, and makes
all packages (except itself) depend on it. We also make sure that this
host-skeleton package doesn't depend on host-patchelf, because
host-patchelf depends on host-skeleton.

While at it, use $(Q) instead of @ in the HOST_SKELETON_INSTALL_CMDS.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
Changes since v3:
 - Use $(Q) instead of @ in the HOST_SKELETON_INSTALL_CMDS
 - Add Reviewed-by from Yann
Changes since v2:
 - None
Changes since v1:
 - New patch
---
 Makefile                     | 13 +------------
 package/pkg-generic.mk       |  4 ++++
 package/skeleton/skeleton.mk | 12 ++++++++++++
 3 files changed, 17 insertions(+), 12 deletions(-)

diff --git a/Makefile b/Makefile
index 1b34818fc8..33c87b5ebb 100644
--- a/Makefile
+++ b/Makefile
@@ -560,7 +560,7 @@ endif
 
 .PHONY: dirs
 dirs: $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
-	$(HOST_DIR) $(HOST_DIR)/usr $(HOST_DIR)/lib $(BINARIES_DIR)
+	$(HOST_DIR) $(BINARIES_DIR)
 
 $(BUILD_DIR)/buildroot-config/auto.conf: $(BR2_CONFIG)
 	$(MAKE1) $(EXTRAMAKEARGS) HOSTCC="$(HOSTCC_NOCCACHE)" HOSTCXX="$(HOSTCXX_NOCCACHE)" silentoldconfig
@@ -579,17 +579,6 @@ sdk: world
 	$(INSTALL) -m 755 $(TOPDIR)/support/misc/relocate-sdk.sh $(HOST_DIR)/relocate-sdk.sh
 	echo $(HOST_DIR) > $(HOST_DIR)/share/buildroot/sdk-location
 
-# Compatibility symlink in case a post-build script still uses $(HOST_DIR)/usr
-$(HOST_DIR)/usr: $(HOST_DIR)
-	@ln -snf . $@
-
-$(HOST_DIR)/lib: $(HOST_DIR)
-	@mkdir -p $@
-	@case $(HOSTARCH) in \
-		(*64) ln -snf lib $(@D)/lib64;; \
-		(*)   ln -snf lib $(@D)/lib32;; \
-	esac
-
 # Populating the staging with the base directories is handled by the skeleton package
 $(STAGING_DIR):
 	@mkdir -p $(STAGING_DIR)
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 5edb4b0838..3cadb28595 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -551,6 +551,10 @@ $(2)_DEPENDENCIES += toolchain
 endif
 endif
 
+ifneq ($(1),host-skeleton)
+$(2)_DEPENDENCIES += host-skeleton
+endif
+
 # Eliminate duplicates in dependencies
 $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES))
 $(2)_FINAL_PATCH_DEPENDENCIES = $$(sort $$($(2)_PATCH_DEPENDENCIES))
diff --git a/package/skeleton/skeleton.mk b/package/skeleton/skeleton.mk
index d380f41649..efcf420d72 100644
--- a/package/skeleton/skeleton.mk
+++ b/package/skeleton/skeleton.mk
@@ -11,4 +11,16 @@
 SKELETON_ADD_TOOLCHAIN_DEPENDENCY = NO
 SKELETON_ADD_SKELETON_DEPENDENCY = NO
 
+# We create a compatibility symlink in case a post-build script still
+# uses $(HOST_DIR)/usr
+define HOST_SKELETON_INSTALL_CMDS
+	$(Q)ln -snf . $(HOST_DIR)/usr
+	$(Q)mkdir -p $(HOST_DIR)/lib
+	$(Q)case $(HOSTARCH) in \
+		(*64) ln -snf lib $(HOST_DIR)/lib64;; \
+		(*)   ln -snf lib $(HOST_DIR)/lib32;; \
+	esac
+endef
+
 $(eval $(virtual-package))
+$(eval $(host-generic-package))
-- 
2.14.3

[Buildroot] [PATCH v4 2/9] pkg-cmake: install CMake files as part of a package

[Thomas Petazzoni]

Currently, the toolchainfile.cmake and Buildroot.cmake files are
installed outside of any package, just triggered by the toolchain
target.

As part of the per-package SDK effort, we are trying to avoid anything
that installs to the global $(HOST_DIR), and this is one of the
remaining files installed in $(HOST_DIR) outside of any package. We
fix this by installing such files as part of the toolchain package
post-install staging hooks.

Yes, a post-install staging hook to install things to $(HOST_DIR) is a
bit weird, but the toolchain infrastructure is made of target packages
only, and they all install a lot of stuff to $(HOST_DIR) already.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
Changes since v3:
 - Added Reviewed-by from Yann
Changes since v2:
 - None
Changes since v1:
 - New patch
---
 package/pkg-cmake.mk             | 12 +++++++-----
 toolchain/toolchain/toolchain.mk |  3 ---
 2 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/package/pkg-cmake.mk b/package/pkg-cmake.mk
index 9b07798a1e..34b49bdbe5 100644
--- a/package/pkg-cmake.mk
+++ b/package/pkg-cmake.mk
@@ -253,8 +253,8 @@ endif
 # based on the toolchainfile.cmake file's location: $(HOST_DIR)/share/buildroot
 # In all the other variables, HOST_DIR will be replaced by RELOCATED_HOST_DIR,
 # so we have to strip "$(HOST_DIR)/" from the paths that contain it.
-$(HOST_DIR)/share/buildroot/toolchainfile.cmake:
-	@mkdir -p $(@D)
+define TOOLCHAIN_CMAKE_INSTALL_FILES
+	@mkdir -p $(HOST_DIR)/share/buildroot
 	sed \
 		-e 's#@@STAGING_SUBDIR@@#$(call qstrip,$(STAGING_SUBDIR))#' \
 		-e 's#@@TARGET_CFLAGS@@#$(call qstrip,$(TARGET_CFLAGS))#' \
@@ -268,7 +268,9 @@ $(HOST_DIR)/share/buildroot/toolchainfile.cmake:
 		-e 's#@@TOOLCHAIN_HAS_FORTRAN@@#$(if $(BR2_TOOLCHAIN_HAS_FORTRAN),1,0)#' \
 		-e 's#@@CMAKE_BUILD_TYPE@@#$(if $(BR2_ENABLE_DEBUG),Debug,Release)#' \
 		$(TOPDIR)/support/misc/toolchainfile.cmake.in \
-		> $@
+		> $(HOST_DIR)/share/buildroot/toolchainfile.cmake
+	$(Q)$(INSTALL) -D -m 0644 support/misc/Buildroot.cmake \
+		$(HOST_DIR)/share/buildroot/Platform/Buildroot.cmake
+endef
 
-$(HOST_DIR)/share/buildroot/Platform/Buildroot.cmake:
-	$(Q)$(INSTALL) -D -m 0644 support/misc/Buildroot.cmake $(@)
+TOOLCHAIN_POST_INSTALL_STAGING_HOOKS += TOOLCHAIN_CMAKE_INSTALL_FILES
diff --git a/toolchain/toolchain/toolchain.mk b/toolchain/toolchain/toolchain.mk
index 179c09edf1..91c9ca2eff 100644
--- a/toolchain/toolchain/toolchain.mk
+++ b/toolchain/toolchain/toolchain.mk
@@ -38,6 +38,3 @@ TOOLCHAIN_INSTALL_STAGING = YES
 endif
 
 $(eval $(virtual-package))
-
-toolchain: $(HOST_DIR)/share/buildroot/toolchainfile.cmake
-toolchain: $(HOST_DIR)/share/buildroot/Platform/Buildroot.cmake
-- 
2.14.3

[Buildroot] [PATCH v4 3/9] package/pkg-generic: add the concept of extract dependency

[Thomas Petazzoni]

Extract dependencies are dependencies that must be ready before the
extract step of a package, i.e for tools that are needed to extract
packages themselves. Current examples of such tools are host-tar,
host-lzip and host-xz.

They are currently handled through DEPENDENCIES_HOST_PREREQ. However,
this mechanism has a number of drawbacks:

 - First and foremost, because host-tar/host-lzip/host-xz are not
   listed in the dependencies of packages, the package infrastructure
   does not know it should rsync them in the context of per-package
   SDK.

 - Second, there is no dependency handling *between* them. I.e, we
   have no mechanism that says host-tar should be built before
   host-lzip, while it is in fact the case: if you need to build
   host-lzip, you need to extract a tarball, so you may need host-tar
   if your system tarball is not capable enough.

For those reasons, it makes sense to add explicit support for "extract
dependencies" in the package infrastructure, through the
<pkg>_EXTRACT_DEPENDENCIES variable. It is unlikely this variable will
ever be used by a package .mk file, but it will be used internally by
the package infrastructure.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
Changes since v3:
 - Fix typo in documentation noticed by Yann (use -> used)
 - Add Reviewed-by from Yann
Changes since v2:
 - New patch
---
 docs/manual/adding-packages-generic.txt | 7 +++++++
 package/pkg-generic.mk                  | 8 +++++++-
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
index 521d6d50c7..194da00e67 100644
--- a/docs/manual/adding-packages-generic.txt
+++ b/docs/manual/adding-packages-generic.txt
@@ -347,6 +347,13 @@ information is (assuming the package name is +libfoo+) :
   a similar way, +HOST_LIBFOO_DEPENDENCIES+ lists the dependencies for
   the current host package.
 
+* +LIBFOO_EXTRACT_DEPENDENCIES+ lists the dependencies (in terms of
+  package name) that are required for the current target package to be
+  extracted. These depnedencies are guaranteed to be compiled and
+  installed before the extract step of the current package
+  starts. This is only used internally by the package infrastructure,
+  and should typically not be used directly by packages.
+
 * +LIBFOO_PATCH_DEPENDENCIES+ lists the dependencies (in terms of
   package name) that are required for the current package to be
   patched. These dependencies are guaranteed to be extracted and
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 3cadb28595..0a404040b0 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -557,8 +557,13 @@ endif
 
 # Eliminate duplicates in dependencies
 $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES))
+$(2)_FINAL_EXTRACT_DEPENDENCIES = $$(sort $$($(2)_EXTRACT_DEPENDENCIES))
 $(2)_FINAL_PATCH_DEPENDENCIES = $$(sort $$($(2)_PATCH_DEPENDENCIES))
-$(2)_FINAL_ALL_DEPENDENCIES = $$(sort $$($(2)_FINAL_DEPENDENCIES) $$($(2)_FINAL_PATCH_DEPENDENCIES))
+$(2)_FINAL_ALL_DEPENDENCIES = \
+	$$(sort \
+		$$($(2)_FINAL_DEPENDENCIES) \
+		$$($(2)_FINAL_EXTRACT_DEPENDENCIES) \
+		$$($(2)_FINAL_PATCH_DEPENDENCIES))
 
 $(2)_INSTALL_STAGING		?= NO
 $(2)_INSTALL_IMAGES		?= NO
@@ -681,6 +686,7 @@ $$($(2)_TARGET_PATCH):  | $$(patsubst %,%-patch,$$($(2)_FINAL_PATCH_DEPENDENCIES
 
 $(1)-extract:			$$($(2)_TARGET_EXTRACT)
 $$($(2)_TARGET_EXTRACT):	$$($(2)_TARGET_SOURCE)
+$$($(2)_TARGET_EXTRACT): | $$($(2)_FINAL_EXTRACT_DEPENDENCIES)
 
 $(1)-depends:		$$($(2)_FINAL_DEPENDENCIES)
 
-- 
2.14.3

[Buildroot] [PATCH v4 4/9] package/pkg-generic: handle host-tar as an extract dependency

[Thomas Petazzoni]

This moves the host-tar dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.

To achieve that, check-host-tar.mk fills in the
BR2_TAR_HOST_DEPENDENCY variable with host-tar if building a host-tar
is needed. The name BR2_TAR_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.

The BR2_TAR_HOST_DEPENDENCY is added to all packages, except host-tar
itself (obviously) and host-skeleton, because we depend on
host-skeleton to install host-tar properly in HOST_DIR.

In addition, we modify tar.mk to explicitly build host-tar without
ccache: since ccache source code is available as a tarball, ccache
will obviously depend on host-tar if the system tar is insufficient.

Finally, to make things really clean, we also add
$(BR2_TAR_HOST_DEPENDENCY) to the dependencies of the tar filesystem
format, since it requires tar, so we'd better make sure we have a
suitable tar.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
Changes since v3:
 - Add dependency on BR2_TAR_HOST_DEPENDENCY in the tar filesystem
 - Use = instead of += when defining BR2_TAR_HOST_DEPENDENCY
 - Fix typo in commit log
 - Added Reviewed-by from Yann.

Changes since v2:
 - New patch
---
 fs/tar/tar.mk                          | 2 ++
 package/pkg-generic.mk                 | 4 ++++
 package/tar/tar.mk                     | 5 +++++
 support/dependencies/check-host-tar.mk | 2 +-
 4 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/fs/tar/tar.mk b/fs/tar/tar.mk
index e39c2fdbf1..68149e9eb7 100644
--- a/fs/tar/tar.mk
+++ b/fs/tar/tar.mk
@@ -6,6 +6,8 @@
 
 TAR_OPTS := $(call qstrip,$(BR2_TARGET_ROOTFS_TAR_OPTIONS))
 
+ROOTFS_TAR_DEPENDENCIES = $(BR2_TAR_HOST_DEPENDENCY)
+
 define ROOTFS_TAR_CMD
 	(cd $(TARGET_DIR); find -print0 | LC_ALL=C sort -z | \
 		tar $(TAR_OPTS) -cf $@ --null --no-recursion -T - --numeric-owner)
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 0a404040b0..ae50c5f284 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -555,6 +555,10 @@ ifneq ($(1),host-skeleton)
 $(2)_DEPENDENCIES += host-skeleton
 endif
 
+ifeq ($(filter host-tar host-skeleton,$(1)),)
+$(2)_EXTRACT_DEPENDENCIES += $(BR2_TAR_HOST_DEPENDENCY)
+endif
+
 # Eliminate duplicates in dependencies
 $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES))
 $(2)_FINAL_EXTRACT_DEPENDENCIES = $$(sort $$($(2)_EXTRACT_DEPENDENCIES))
diff --git a/package/tar/tar.mk b/package/tar/tar.mk
index a8a15d9518..92b6e9eaa5 100644
--- a/package/tar/tar.mk
+++ b/package/tar/tar.mk
@@ -47,4 +47,9 @@ endef
 
 HOST_TAR_CONF_OPTS = --without-selinux
 
+# we are built before ccache
+HOST_TAR_CONF_ENV = \
+	CC="$(HOSTCC_NOCCACHE)" \
+	CXX="$(HOSTCXX_NOCCACHE)"
+
 $(eval $(host-autotools-package))
diff --git a/support/dependencies/check-host-tar.mk b/support/dependencies/check-host-tar.mk
index ad0b32e277..07d02fd1b1 100644
--- a/support/dependencies/check-host-tar.mk
+++ b/support/dependencies/check-host-tar.mk
@@ -1,6 +1,6 @@
 TAR ?= tar
 
 ifeq (,$(call suitable-host-package,tar,$(TAR)))
-DEPENDENCIES_HOST_PREREQ += host-tar
 TAR = $(HOST_DIR)/bin/tar
+BR2_TAR_HOST_DEPENDENCY = host-tar
 endif
-- 
2.14.3

[Buildroot] [PATCH v4 5/9] package/pkg-generic: handle host-xz as an extract dependency

[Thomas Petazzoni]

This moves the host-xz dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.

To achieve that, check-host-xz.mk fills in the
BR2_XZ_HOST_DEPENDENCY with host-tar if building a host-tar is
needed. The name BR2_XZ_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.

The BR2_XZ_HOST_DEPENDENCY is added to all packages, except:

 - host-xz, because we would otherwise depend on ourself.

 - host-tar, because xz itself is delivered as a tarball, so we need
   to have host-xz depend on host-tar, and not host-tar depend on
   host-xz

 - host-skeleton, because we need to have host-xz depend on
   host-skeleton, and not the opposite.

In addition, we modify xz.mk to explicitly build host-xz without
ccache. We generally took the approach of building host-ccache *after*
all the extractors have been built.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
Changes in v3:
 - Added Reviewed-by from Yann.

Changes since v2:
 - New patch
---
 package/pkg-generic.mk                   | 4 ++++
 package/xz/xz.mk                         | 5 +++++
 support/dependencies/check-host-xzcat.mk | 2 +-
 3 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index ae50c5f284..b03d7263b1 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -559,6 +559,10 @@ ifeq ($(filter host-tar host-skeleton,$(1)),)
 $(2)_EXTRACT_DEPENDENCIES += $(BR2_TAR_HOST_DEPENDENCY)
 endif
 
+ifeq ($(filter host-tar host-skeleton host-xz,$(1)),)
+$(2)_EXTRACT_DEPENDENCIES += $(BR2_XZCAT_HOST_DEPENDENCY)
+endif
+
 # Eliminate duplicates in dependencies
 $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES))
 $(2)_FINAL_EXTRACT_DEPENDENCIES = $$(sort $$($(2)_EXTRACT_DEPENDENCIES))
diff --git a/package/xz/xz.mk b/package/xz/xz.mk
index e8116f24ae..bcdac13ee3 100644
--- a/package/xz/xz.mk
+++ b/package/xz/xz.mk
@@ -18,5 +18,10 @@ else
 XZ_CONF_OPTS = --disable-threads
 endif
 
+# we are built before ccache
+HOST_XZ_CONF_ENV = \
+	CC="$(HOSTCC_NOCCACHE)" \
+	CXX="$(HOSTCXX_NOCCACHE)"
+
 $(eval $(autotools-package))
 $(eval $(host-autotools-package))
diff --git a/support/dependencies/check-host-xzcat.mk b/support/dependencies/check-host-xzcat.mk
index 9be75c7311..e5d72f95dd 100644
--- a/support/dependencies/check-host-xzcat.mk
+++ b/support/dependencies/check-host-xzcat.mk
@@ -2,7 +2,7 @@
 # If it is not present, build our own host-xzcat
 
 ifeq (,$(call suitable-host-package,xzcat,$(XZCAT)))
-DEPENDENCIES_HOST_PREREQ += host-xz
+BR2_XZCAT_HOST_DEPENDENCY = host-xz
 EXTRACTOR_DEPENDENCY_PRECHECKED_EXTENSIONS += .xz .lzma
 XZCAT = $(HOST_DIR)/bin/xzcat
 endif
-- 
2.14.3

[Buildroot] [PATCH v4 6/9] package/pkg-generic: handle host-lzip as an extract dependency

[Thomas Petazzoni]

This moves the host-xz dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.

To achieve that, check-host-xz.mk fills in the
BR2_LZIP_HOST_DEPENDENCY with host-tar if building a host-tar is
needed. The name BR2_LZIP_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.

The BR2_LZIP_HOST_DEPENDENCY is added to all packages, except:

 - host-lzip, because we would otherwise depend on ourself.

 - host-tar, because lzip itself is delivered as a tarball, so we need
   to have host-lzip depend on host-tar, and not host-tar depend on
   host-lzip

 - host-skeleton, because we need to have host-lzip depend on
   host-skeleton, and not the opposite.

We also mutually exclude host-lzip and host-xz from dependending on
each other, to avoid a circular dependency.

In addition, we modify lzip.mk to explicitly build host-lzip without
ccache. We generally took the approach of building host-ccache *after*
all the extractors have been built.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
Changes in v3:
 - Added Reviewed-by from Yann

Changes since v2:
 - New patch
---
 package/lzip/lzip.mk                    | 2 +-
 package/pkg-generic.mk                  | 6 +++++-
 support/dependencies/check-host-lzip.mk | 2 +-
 3 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/package/lzip/lzip.mk b/package/lzip/lzip.mk
index 4088662de2..72742240a5 100644
--- a/package/lzip/lzip.mk
+++ b/package/lzip/lzip.mk
@@ -16,7 +16,7 @@ endef
 
 define HOST_LZIP_CONFIGURE_CMDS
 	(cd $(@D); $(HOST_MAKE_ENV) ./configure --prefix=$(HOST_DIR) \
-		$(HOST_CONFIGURE_OPTS) )
+		$(HOST_CONFIGURE_OPTS) CC="$(HOSTCC_NOCCACHE)" CXX="$(HOSTCXX_NOCCACHE)")
 endef
 
 define LZIP_BUILD_CMDS
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index b03d7263b1..f03f1d930f 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -559,10 +559,14 @@ ifeq ($(filter host-tar host-skeleton,$(1)),)
 $(2)_EXTRACT_DEPENDENCIES += $(BR2_TAR_HOST_DEPENDENCY)
 endif
 
-ifeq ($(filter host-tar host-skeleton host-xz,$(1)),)
+ifeq ($(filter host-tar host-skeleton host-xz host-lzip,$(1)),)
 $(2)_EXTRACT_DEPENDENCIES += $(BR2_XZCAT_HOST_DEPENDENCY)
 endif
 
+ifeq ($(filter host-tar host-skeleton host-xz host-lzip,$(1)),)
+$(2)_EXTRACT_DEPENDENCIES += $(BR2_LZIP_HOST_DEPENDENCY)
+endif
+
 # Eliminate duplicates in dependencies
 $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES))
 $(2)_FINAL_EXTRACT_DEPENDENCIES = $$(sort $$($(2)_EXTRACT_DEPENDENCIES))
diff --git a/support/dependencies/check-host-lzip.mk b/support/dependencies/check-host-lzip.mk
index 00cdd0a236..cdd784058c 100644
--- a/support/dependencies/check-host-lzip.mk
+++ b/support/dependencies/check-host-lzip.mk
@@ -1,5 +1,5 @@
 ifeq (,$(call suitable-host-package,lzip,$(LZCAT)))
-DEPENDENCIES_HOST_PREREQ += host-lzip
+BR2_LZIP_HOST_DEPENDENCY = host-lzip
 EXTRACTOR_DEPENDENCY_PRECHECKED_EXTENSIONS += .lz
 LZCAT = $(HOST_DIR)/bin/lzip -d -c
 endif
-- 
2.14.3

[Buildroot] [PATCH v4 7/9] package/pkg-generic: handle host-ccache as a regular dependency

[Thomas Petazzoni]

This moves the host-ccache dependency handling from
DEPENDENCIES_HOST_PREREQ to a proper package dependency. When
BR2_CCACHE=y, we add host-ccache as a regular dependency of all
packages except:

 - The extractor packages host-tar, host-xz and host-lzip

 - host-ccache itself

 - host-skeleton, because all packages depend on it

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
Changes since v3:
 - Prevent host-ccache from being a dependency of itself, as noticed
   by Yann.

Changes since v2:
 - New patch
---
 package/ccache/ccache.mk             | 5 +++++
 package/pkg-generic.mk               | 6 ++++++
 support/dependencies/dependencies.mk | 4 ----
 3 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
index 552a611f42..9a11d46d30 100644
--- a/package/ccache/ccache.mk
+++ b/package/ccache/ccache.mk
@@ -21,6 +21,11 @@ CCACHE_LICENSE_FILES = LICENSE.txt GPL-3.0.txt
 # has zero dependency besides the C library.
 HOST_CCACHE_CONF_OPTS += --with-bundled-zlib
 
+# We are ccache, so we can't use ccache
+HOST_CCACHE_CONF_ENV = \
+	CC="$(HOSTCC_NOCCACHE)" \
+	CXX="$(HOSTCXX_NOCCACHE)"
+
 # Patch host-ccache as follows:
 #  - Use BR_CACHE_DIR instead of CCACHE_DIR, because CCACHE_DIR
 #    is already used by autotargets for the ccache package.
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index f03f1d930f..bc4882235f 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -567,6 +567,12 @@ ifeq ($(filter host-tar host-skeleton host-xz host-lzip,$(1)),)
 $(2)_EXTRACT_DEPENDENCIES += $(BR2_LZIP_HOST_DEPENDENCY)
 endif
 
+ifeq ($(BR2_CCACHE),y)
+ifeq ($(filter host-tar host-skeleton host-xz host-lzip host-ccache,$(1)),)
+$(2)_DEPENDENCIES += host-ccache
+endif
+endif
+
 # Eliminate duplicates in dependencies
 $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES))
 $(2)_FINAL_EXTRACT_DEPENDENCIES = $$(sort $$($(2)_EXTRACT_DEPENDENCIES))
diff --git a/support/dependencies/dependencies.mk b/support/dependencies/dependencies.mk
index 3fc235863c..bab064319f 100644
--- a/support/dependencies/dependencies.mk
+++ b/support/dependencies/dependencies.mk
@@ -17,10 +17,6 @@ endef
 include support/dependencies/check-host-tar.mk
 -include $(sort $(filter-out %-tar.mk,$(wildcard support/dependencies/check-host-*.mk)))
 
-ifeq ($(BR2_CCACHE),y)
-DEPENDENCIES_HOST_PREREQ += host-ccache
-endif
-
 core-dependencies:
 	@MAKE="$(MAKE)" DL_TOOLS="$(sort $(DL_TOOLS_DEPENDENCIES))" \
 		$(TOPDIR)/support/dependencies/dependencies.sh
-- 
2.14.3

[Buildroot] [PATCH v4 8/9] package/pkg-generic: handle host-fakedate as a regular dependency

[Thomas Petazzoni]

This commit moves the host-fakedate dependency handling from
DEPENDENCIES_HOST_PREREQ to a proper regular dependency handled by the
package infrastructure.

host-fakedate is added as dependency to all packages, except
host-skeleton, because we depend on it.

In addition, we make sure that host-fakedate does not grow a
dependency on host-{tar,xz,lzip,ccache} to avoid circular
dependencies. host-fakedate does not need any extraction tool and does
not need to build C/C++ code (the source code is just a shell script
available in Buildroot).

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
Changes since v3:
 - Make sure we don't introduce a host-fakedate -> host-fakedate
   circular dependency.
 - Added Reviewed-by from Yann.

Changes since v2:
 - New patch
---
 Makefile               |  1 -
 package/pkg-generic.mk | 14 ++++++++++----
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
index 33c87b5ebb..925d6e4aec 100644
--- a/Makefile
+++ b/Makefile
@@ -255,7 +255,6 @@ export LC_ALL = C
 export GZIP = -n
 BR2_VERSION_GIT_EPOCH = $(shell GIT_DIR=$(TOPDIR)/.git $(GIT) log -1 --format=%at)
 export SOURCE_DATE_EPOCH ?= $(if $(wildcard $(TOPDIR)/.git),$(BR2_VERSION_GIT_EPOCH),$(BR2_VERSION_EPOCH))
-DEPENDENCIES_HOST_PREREQ += host-fakedate
 endif
 
 # To put more focus on warnings, be less verbose as default
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index bc4882235f..6d1f267328 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -555,24 +555,30 @@ ifneq ($(1),host-skeleton)
 $(2)_DEPENDENCIES += host-skeleton
 endif
 
-ifeq ($(filter host-tar host-skeleton,$(1)),)
+ifeq ($(filter host-tar host-skeleton host-fakedate,$(1)),)
 $(2)_EXTRACT_DEPENDENCIES += $(BR2_TAR_HOST_DEPENDENCY)
 endif
 
-ifeq ($(filter host-tar host-skeleton host-xz host-lzip,$(1)),)
+ifeq ($(filter host-tar host-skeleton host-xz host-lzip host-fakedate,$(1)),)
 $(2)_EXTRACT_DEPENDENCIES += $(BR2_XZCAT_HOST_DEPENDENCY)
 endif
 
-ifeq ($(filter host-tar host-skeleton host-xz host-lzip,$(1)),)
+ifeq ($(filter host-tar host-skeleton host-xz host-lzip host-fakedate,$(1)),)
 $(2)_EXTRACT_DEPENDENCIES += $(BR2_LZIP_HOST_DEPENDENCY)
 endif
 
 ifeq ($(BR2_CCACHE),y)
-ifeq ($(filter host-tar host-skeleton host-xz host-lzip host-ccache,$(1)),)
+ifeq ($(filter host-tar host-skeleton host-xz host-lzip host-fakedate host-ccache,$(1)),)
 $(2)_DEPENDENCIES += host-ccache
 endif
 endif
 
+ifeq ($(BR2_REPRODUCIBLE),y)
+ifeq ($(filter host-skeleton host-fakedate,$(1)),)
+$(2)_DEPENDENCIES += host-fakedate
+endif
+endif
+
 # Eliminate duplicates in dependencies
 $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES))
 $(2)_FINAL_EXTRACT_DEPENDENCIES = $$(sort $$($(2)_EXTRACT_DEPENDENCIES))
-- 
2.14.3

[Buildroot] [PATCH v4 9/9] core: kill DEPENDENCIES_HOST_PREREQ

[Thomas Petazzoni]

Now that DEPENDENCIES_HOST_PREREQ is no longer used anywhere, we can
kill it.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
Changes since v3:
 - Added Acked-by from Yann.

Changes since v2:
 - New patch
---
 Makefile                             |  2 --
 package/pkg-generic.mk               |  2 --
 support/dependencies/dependencies.mk | 10 ++--------
 3 files changed, 2 insertions(+), 12 deletions(-)

diff --git a/Makefile b/Makefile
index 925d6e4aec..a3a9d290d9 100644
--- a/Makefile
+++ b/Makefile
@@ -501,8 +501,6 @@ include package/Makefile.in
 -include $(sort $(wildcard arch/arch.mk.*))
 include support/dependencies/dependencies.mk
 
-PACKAGES += $(DEPENDENCIES_HOST_PREREQ)
-
 include $(sort $(wildcard toolchain/*.mk))
 include $(sort $(wildcard toolchain/*/*.mk))
 
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 6d1f267328..6d82f7027e 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -690,9 +690,7 @@ $(1)-configure:			$$($(2)_TARGET_CONFIGURE)
 $$($(2)_TARGET_CONFIGURE):	| $$($(2)_FINAL_DEPENDENCIES)
 
 $$($(2)_TARGET_SOURCE) $$($(2)_TARGET_RSYNC): | dirs prepare
-ifeq ($$(filter $(1),$$(DEPENDENCIES_HOST_PREREQ)),)
 $$($(2)_TARGET_SOURCE) $$($(2)_TARGET_RSYNC): | dependencies
-endif
 
 ifeq ($$($(2)_OVERRIDE_SRCDIR),)
 # In the normal case (no package override), the sequence of steps is
diff --git a/support/dependencies/dependencies.mk b/support/dependencies/dependencies.mk
index bab064319f..563574d6ad 100644
--- a/support/dependencies/dependencies.mk
+++ b/support/dependencies/dependencies.mk
@@ -17,19 +17,13 @@ endef
 include support/dependencies/check-host-tar.mk
 -include $(sort $(filter-out %-tar.mk,$(wildcard support/dependencies/check-host-*.mk)))
 
-core-dependencies:
+dependencies:
 	@MAKE="$(MAKE)" DL_TOOLS="$(sort $(DL_TOOLS_DEPENDENCIES))" \
 		$(TOPDIR)/support/dependencies/dependencies.sh
 
-$(DEPENDENCIES_HOST_PREREQ): HOSTCC=$(HOSTCC_NOCCACHE)
-$(DEPENDENCIES_HOST_PREREQ): HOSTCXX=$(HOSTCXX_NOCCACHE)
-$(DEPENDENCIES_HOST_PREREQ): core-dependencies
-
-dependencies: core-dependencies $(DEPENDENCIES_HOST_PREREQ)
-
 ################################################################################
 #
 # Toplevel Makefile options
 #
 ################################################################################
-.PHONY: dependencies core-dependencies
+.PHONY: dependencies
-- 
2.14.3

[Buildroot] [PATCH v4 1/9] Makefile, skeleton: move the host skeleton logic to host-skeleton package

[Yann E. MORIN]

Thomas, All,

On 2018-03-24 15:20 +0100, Thomas Petazzoni spake thusly:
> As part of the per-package SDK work, we want to avoid having logic
> that installs files to the global HOST_DIR, and instead do it inside
> packages. One thing that gets installed to the global HOST_DIR is the
> minimal "skeleton" that we create in host:
> 
>  - the "usr" symbolic link for backward compatibility
> 
>  - the "lib" directory, and its lib64 or lib32 symbolic links
> 
> This commit moves this logic to a new host-skeleton package, and makes
> all packages (except itself) depend on it. We also make sure that this
> host-skeleton package doesn't depend on host-patchelf, because
> host-patchelf depends on host-skeleton.
> 
> While at it, use $(Q) instead of @ in the HOST_SKELETON_INSTALL_CMDS.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[--SNIP--]
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index 5edb4b0838..3cadb28595 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -551,6 +551,10 @@ $(2)_DEPENDENCIES += toolchain
>  endif
>  endif
>  
> +ifneq ($(1),host-skeleton)
> +$(2)_DEPENDENCIES += host-skeleton
> +endif

In fact, I'm not too sure I should have given my Reviewed-by tag,
because now the graph-depends are all borked: all packages have a
dependency arrow toward host-skeleton.

What about adding host-skeleton as a dependency to skeleton-init-common
and skeleton-custom only?

Or we can also just add host-skeleton as a special case in
support/scripts/graph-depends...

Regards,
Yann E. MORIN.

> +
>  # Eliminate duplicates in dependencies
>  $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES))
>  $(2)_FINAL_PATCH_DEPENDENCIES = $$(sort $$($(2)_PATCH_DEPENDENCIES))
> diff --git a/package/skeleton/skeleton.mk b/package/skeleton/skeleton.mk
> index d380f41649..efcf420d72 100644
> --- a/package/skeleton/skeleton.mk
> +++ b/package/skeleton/skeleton.mk
> @@ -11,4 +11,16 @@
>  SKELETON_ADD_TOOLCHAIN_DEPENDENCY = NO
>  SKELETON_ADD_SKELETON_DEPENDENCY = NO
>  
> +# We create a compatibility symlink in case a post-build script still
> +# uses $(HOST_DIR)/usr
> +define HOST_SKELETON_INSTALL_CMDS
> +	$(Q)ln -snf . $(HOST_DIR)/usr
> +	$(Q)mkdir -p $(HOST_DIR)/lib
> +	$(Q)case $(HOSTARCH) in \
> +		(*64) ln -snf lib $(HOST_DIR)/lib64;; \
> +		(*)   ln -snf lib $(HOST_DIR)/lib32;; \
> +	esac
> +endef
> +
>  $(eval $(virtual-package))
> +$(eval $(host-generic-package))
> -- 
> 2.14.3
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v4 1/9] Makefile, skeleton: move the host skeleton logic to host-skeleton package

[Thomas Petazzoni]

Hello,

On Sat, 24 Mar 2018 16:28:52 +0100, Yann E. MORIN wrote:

> > +ifneq ($(1),host-skeleton)
> > +$(2)_DEPENDENCIES += host-skeleton
> > +endif  
> 
> In fact, I'm not too sure I should have given my Reviewed-by tag,
> because now the graph-depends are all borked: all packages have a
> dependency arrow toward host-skeleton.

It is not borked, that's completely correct. Maybe not pretty and
convenient, but definitely not "borked".

> What about adding host-skeleton as a dependency to skeleton-init-common
> and skeleton-custom only?

Really feels wrong to me: there is absolutely no reason for
skeleton-init-common and skeleton-custom to depend on host-skeleton.

> Or we can also just add host-skeleton as a special case in
> support/scripts/graph-depends...

That feels like a much better solution to me. We already do that for
the skeleton anyway. Would that be OK for you?

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v4 7/9] package/pkg-generic: handle host-ccache as a regular dependency

[Yann E. MORIN]

Thomas, All,

On 2018-03-24 15:20 +0100, Thomas Petazzoni spake thusly:
> This moves the host-ccache dependency handling from
> DEPENDENCIES_HOST_PREREQ to a proper package dependency. When
> BR2_CCACHE=y, we add host-ccache as a regular dependency of all
> packages except:
> 
>  - The extractor packages host-tar, host-xz and host-lzip
> 
>  - host-ccache itself
> 
>  - host-skeleton, because all packages depend on it
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>

Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

Regards,
Yann E. MORIN.

> ---
> Changes since v3:
>  - Prevent host-ccache from being a dependency of itself, as noticed
>    by Yann.
> 
> Changes since v2:
>  - New patch
> ---
>  package/ccache/ccache.mk             | 5 +++++
>  package/pkg-generic.mk               | 6 ++++++
>  support/dependencies/dependencies.mk | 4 ----
>  3 files changed, 11 insertions(+), 4 deletions(-)
> 
> diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
> index 552a611f42..9a11d46d30 100644
> --- a/package/ccache/ccache.mk
> +++ b/package/ccache/ccache.mk
> @@ -21,6 +21,11 @@ CCACHE_LICENSE_FILES = LICENSE.txt GPL-3.0.txt
>  # has zero dependency besides the C library.
>  HOST_CCACHE_CONF_OPTS += --with-bundled-zlib
>  
> +# We are ccache, so we can't use ccache
> +HOST_CCACHE_CONF_ENV = \
> +	CC="$(HOSTCC_NOCCACHE)" \
> +	CXX="$(HOSTCXX_NOCCACHE)"
> +
>  # Patch host-ccache as follows:
>  #  - Use BR_CACHE_DIR instead of CCACHE_DIR, because CCACHE_DIR
>  #    is already used by autotargets for the ccache package.
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index f03f1d930f..bc4882235f 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -567,6 +567,12 @@ ifeq ($(filter host-tar host-skeleton host-xz host-lzip,$(1)),)
>  $(2)_EXTRACT_DEPENDENCIES += $(BR2_LZIP_HOST_DEPENDENCY)
>  endif
>  
> +ifeq ($(BR2_CCACHE),y)
> +ifeq ($(filter host-tar host-skeleton host-xz host-lzip host-ccache,$(1)),)
> +$(2)_DEPENDENCIES += host-ccache
> +endif
> +endif
> +
>  # Eliminate duplicates in dependencies
>  $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES))
>  $(2)_FINAL_EXTRACT_DEPENDENCIES = $$(sort $$($(2)_EXTRACT_DEPENDENCIES))
> diff --git a/support/dependencies/dependencies.mk b/support/dependencies/dependencies.mk
> index 3fc235863c..bab064319f 100644
> --- a/support/dependencies/dependencies.mk
> +++ b/support/dependencies/dependencies.mk
> @@ -17,10 +17,6 @@ endef
>  include support/dependencies/check-host-tar.mk
>  -include $(sort $(filter-out %-tar.mk,$(wildcard support/dependencies/check-host-*.mk)))
>  
> -ifeq ($(BR2_CCACHE),y)
> -DEPENDENCIES_HOST_PREREQ += host-ccache
> -endif
> -
>  core-dependencies:
>  	@MAKE="$(MAKE)" DL_TOOLS="$(sort $(DL_TOOLS_DEPENDENCIES))" \
>  		$(TOPDIR)/support/dependencies/dependencies.sh
> -- 
> 2.14.3
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v4 1/9] Makefile, skeleton: move the host skeleton logic to host-skeleton package

[Yann E. MORIN]

Thomas, All,

On 2018-03-24 16:37 +0100, Thomas Petazzoni spake thusly:
> On Sat, 24 Mar 2018 16:28:52 +0100, Yann E. MORIN wrote:
> > > +ifneq ($(1),host-skeleton)
> > > +$(2)_DEPENDENCIES += host-skeleton
> > > +endif  
> > 
> > In fact, I'm not too sure I should have given my Reviewed-by tag,
> > because now the graph-depends are all borked: all packages have a
> > dependency arrow toward host-skeleton.
> 
> It is not borked, that's completely correct. Maybe not pretty and
> convenient, but definitely not "borked".

Yeah, not broken, just not "nice-looking"... You got what I meant. ;-)

> > Or we can also just add host-skeleton as a special case in
> > support/scripts/graph-depends...
> 
> That feels like a much better solution to me. We already do that for
> the skeleton anyway. Would that be OK for you?

Yeah, but we should be cautious not to add too much exceptions in that
script, because we'll have to be sure to sync it with the infra...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v4 7/9] package/pkg-generic: handle host-ccache as a regular dependency

[Matthew Weber]

Thomas,

On Sat, Mar 24, 2018 at 9:20 AM, Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
> This moves the host-ccache dependency handling from
> DEPENDENCIES_HOST_PREREQ to a proper package dependency. When
> BR2_CCACHE=y, we add host-ccache as a regular dependency of all
> packages except:
>
>  - The extractor packages host-tar, host-xz and host-lzip
>
>  - host-ccache itself
>
>  - host-skeleton, because all packages depend on it

Verified that host-ccache built after host-tar/xz/lzip and verified a
build without ccache enabled worked.

Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>

[Buildroot] [PATCH v4 4/9] package/pkg-generic: handle host-tar as an extract dependency

[Matthew Weber]

Thomas,

On Sat, Mar 24, 2018 at 9:20 AM, Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
> This moves the host-tar dependency handling from
> DEPENDENCY_HOST_PREREQ to an extract dependency.
>
> To achieve that, check-host-tar.mk fills in the
> BR2_TAR_HOST_DEPENDENCY variable with host-tar if building a host-tar
> is needed. The name BR2_TAR_HOST_DEPENDENCY has been chosen because it
> matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
> check-host-cmake.mk.
>
> The BR2_TAR_HOST_DEPENDENCY is added to all packages, except host-tar
> itself (obviously) and host-skeleton, because we depend on
> host-skeleton to install host-tar properly in HOST_DIR.
>
> In addition, we modify tar.mk to explicitly build host-tar without
> ccache: since ccache source code is available as a tarball, ccache
> will obviously depend on host-tar if the system tar is insufficient.
>
> Finally, to make things really clean, we also add
> $(BR2_TAR_HOST_DEPENDENCY) to the dependencies of the tar filesystem
> format, since it requires tar, so we'd better make sure we have a
> suitable tar.

Forced my configuration to build host-tar and verified it always built
after host-skeleton. (Did a non-ccache and ccache build)

Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>

[Buildroot] [PATCH v4 3/9] package/pkg-generic: add the concept of extract dependency

[Matthew Weber]

Thomas,

On Sat, Mar 24, 2018 at 9:20 AM, Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
> Extract dependencies are dependencies that must be ready before the
> extract step of a package, i.e for tools that are needed to extract
> packages themselves. Current examples of such tools are host-tar,
> host-lzip and host-xz.
>
> They are currently handled through DEPENDENCIES_HOST_PREREQ. However,
> this mechanism has a number of drawbacks:
>
>  - First and foremost, because host-tar/host-lzip/host-xz are not
>    listed in the dependencies of packages, the package infrastructure
>    does not know it should rsync them in the context of per-package
>    SDK.
>
>  - Second, there is no dependency handling *between* them. I.e, we
>    have no mechanism that says host-tar should be built before
>    host-lzip, while it is in fact the case: if you need to build
>    host-lzip, you need to extract a tarball, so you may need host-tar
>    if your system tarball is not capable enough.
>

Forced host-tar to build in my configuration and verified the new
extract dependency built the expected pre-reqs.  Then did a make
toolchain and make busybox to test an explicit package build.

Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>

>
> +* +LIBFOO_EXTRACT_DEPENDENCIES+ lists the dependencies (in terms of
> +  package name) that are required for the current target package to be
> +  extracted. These depnedencies are guaranteed to be compiled and

depnedencies -> dependencies

Matt

[Buildroot] [PATCH v4 1/9] Makefile, skeleton: move the host skeleton logic to host-skeleton package

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > As part of the per-package SDK work, we want to avoid having logic
 > that installs files to the global HOST_DIR, and instead do it inside
 > packages. One thing that gets installed to the global HOST_DIR is the
 > minimal "skeleton" that we create in host:

 >  - the "usr" symbolic link for backward compatibility

 >  - the "lib" directory, and its lib64 or lib32 symbolic links

 > This commit moves this logic to a new host-skeleton package, and makes
 > all packages (except itself) depend on it. We also make sure that this
 > host-skeleton package doesn't depend on host-patchelf, because
 > host-patchelf depends on host-skeleton.

Ehh, where is that done? Afaik nothing depends on (in the make sense) on
host-patchelf, it is just forcibly enabled in all configs.

Thinking about that, why don't we just make the sdk target depend on
host-patchelf instead of always building it just in case somebody will
ever run 'make sdk'?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v4 1/9] Makefile, skeleton: move the host skeleton logic to host-skeleton package

[Yann E. MORIN]

Peter, Thomas, All,

On 2018-03-25 15:00 +0200, Peter Korsgaard spake thusly:
> >>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:
> 
>  > As part of the per-package SDK work, we want to avoid having logic
>  > that installs files to the global HOST_DIR, and instead do it inside
>  > packages. One thing that gets installed to the global HOST_DIR is the
>  > minimal "skeleton" that we create in host:
> 
>  >  - the "usr" symbolic link for backward compatibility
> 
>  >  - the "lib" directory, and its lib64 or lib32 symbolic links
> 
>  > This commit moves this logic to a new host-skeleton package, and makes
>  > all packages (except itself) depend on it. We also make sure that this
>  > host-skeleton package doesn't depend on host-patchelf, because
>  > host-patchelf depends on host-skeleton.
> 
> Ehh, where is that done? Afaik nothing depends on (in the make sense) on
> host-patchelf, it is just forcibly enabled in all configs.

That's because this is a leftover from a previous non-itreration:

    http://lists.busybox.net/pipermail/buildroot/2017-November/207795.html

I guess Thomas just forgot to amend the commit-log. I won;t blame him on
that one, it happens all the time to me as well! ;-)

> Thinking about that, why don't we just make the sdk target depend on
> host-patchelf instead of always building it just in case somebody will
> ever run 'make sdk'?

That's probably good for another, unrelated patch, right?

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v4 1/9] Makefile, skeleton: move the host skeleton logic to host-skeleton package

[Thomas Petazzoni]

Hello,

On Sun, 25 Mar 2018 15:00:55 +0200, Peter Korsgaard wrote:

> Thinking about that, why don't we just make the sdk target depend on
> host-patchelf instead of always building it just in case somebody will
> ever run 'make sdk'?

We use patchelf in target-finalize to fix rpath in the target folder,
so it's not only used for "make sdk".

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v4 1/9] Makefile, skeleton: move the host skeleton logic to host-skeleton package

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > Hello,
 > On Sun, 25 Mar 2018 15:00:55 +0200, Peter Korsgaard wrote:

 >> Thinking about that, why don't we just make the sdk target depend on
 >> host-patchelf instead of always building it just in case somebody will
 >> ever run 'make sdk'?

 > We use patchelf in target-finalize to fix rpath in the target folder,
 > so it's not only used for "make sdk".

Ahh yes, correct.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v4 1/9] Makefile, skeleton: move the host skeleton logic to host-skeleton package

[Peter Korsgaard]

>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:

Hi,

 >> Ehh, where is that done? Afaik nothing depends on (in the make sense) on
 >> host-patchelf, it is just forcibly enabled in all configs.

 > That's because this is a leftover from a previous non-itreration:

 >     http://lists.busybox.net/pipermail/buildroot/2017-November/207795.html

 > I guess Thomas just forgot to amend the commit-log. I won;t blame him on
 > that one, it happens all the time to me as well! ;-)

Ok, I'll drop that when committing.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v4 1/9] Makefile, skeleton: move the host skeleton logic to host-skeleton package

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > As part of the per-package SDK work, we want to avoid having logic
 > that installs files to the global HOST_DIR, and instead do it inside
 > packages. One thing that gets installed to the global HOST_DIR is the
 > minimal "skeleton" that we create in host:

 >  - the "usr" symbolic link for backward compatibility

 >  - the "lib" directory, and its lib64 or lib32 symbolic links

 > This commit moves this logic to a new host-skeleton package, and makes
 > all packages (except itself) depend on it. We also make sure that this
 > host-skeleton package doesn't depend on host-patchelf, because
 > host-patchelf depends on host-skeleton.

 > While at it, use $(Q) instead of @ in the HOST_SKELETON_INSTALL_CMDS.

 > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
 > Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
 > ---
 > Changes since v3:
 >  - Use $(Q) instead of @ in the HOST_SKELETON_INSTALL_CMDS
 >  - Add Reviewed-by from Yann
 > Changes since v2:
 >  - None
 > Changes since v1:
 >  - New patch

Committed after dropping the host-patchelf reference, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v4 2/9] pkg-cmake: install CMake files as part of a package

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > Currently, the toolchainfile.cmake and Buildroot.cmake files are
 > installed outside of any package, just triggered by the toolchain
 > target.

 > As part of the per-package SDK effort, we are trying to avoid anything
 > that installs to the global $(HOST_DIR), and this is one of the
 > remaining files installed in $(HOST_DIR) outside of any package. We
 > fix this by installing such files as part of the toolchain package
 > post-install staging hooks.

 > Yes, a post-install staging hook to install things to $(HOST_DIR) is a
 > bit weird, but the toolchain infrastructure is made of target packages
 > only, and they all install a lot of stuff to $(HOST_DIR) already.

 > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
 > Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
 > ---
 > Changes since v3:
 >  - Added Reviewed-by from Yann
 > Changes since v2:
 >  - None
 > Changes since v1:
 >  - New patch

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v4 3/9] package/pkg-generic: add the concept of extract dependency

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > Extract dependencies are dependencies that must be ready before the
 > extract step of a package, i.e for tools that are needed to extract
 > packages themselves. Current examples of such tools are host-tar,
 > host-lzip and host-xz.

 > They are currently handled through DEPENDENCIES_HOST_PREREQ. However,
 > this mechanism has a number of drawbacks:

 >  - First and foremost, because host-tar/host-lzip/host-xz are not
 >    listed in the dependencies of packages, the package infrastructure
 >    does not know it should rsync them in the context of per-package
 >    SDK.

 >  - Second, there is no dependency handling *between* them. I.e, we
 >    have no mechanism that says host-tar should be built before
 >    host-lzip, while it is in fact the case: if you need to build
 >    host-lzip, you need to extract a tarball, so you may need host-tar
 >    if your system tarball is not capable enough.

 > For those reasons, it makes sense to add explicit support for "extract
 > dependencies" in the package infrastructure, through the
 > <pkg>_EXTRACT_DEPENDENCIES variable. It is unlikely this variable will
 > ever be used by a package .mk file, but it will be used internally by
 > the package infrastructure.

 > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
 > Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
 > ---
 > Changes since v3:
 >  - Fix typo in documentation noticed by Yann (use -> used)
 >  - Add Reviewed-by from Yann
 > Changes since v2:
 >  - New patch
 > ---
 >  docs/manual/adding-packages-generic.txt | 7 +++++++
 >  package/pkg-generic.mk                  | 8 +++++++-
 >  2 files changed, 14 insertions(+), 1 deletion(-)

 > diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
 > index 521d6d50c7..194da00e67 100644
 > --- a/docs/manual/adding-packages-generic.txt
 > +++ b/docs/manual/adding-packages-generic.txt
 > @@ -347,6 +347,13 @@ information is (assuming the package name is +libfoo+) :
 >    a similar way, +HOST_LIBFOO_DEPENDENCIES+ lists the dependencies for
 >    the current host package.
 
 > +* +LIBFOO_EXTRACT_DEPENDENCIES+ lists the dependencies (in terms of
 > +  package name) that are required for the current target package to be
 > +  extracted. These depnedencies are guaranteed to be compiled and

s/depnedencies/dependencies/

Committed with that fixed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v4 4/9] package/pkg-generic: handle host-tar as an extract dependency

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > This moves the host-tar dependency handling from
 > DEPENDENCY_HOST_PREREQ to an extract dependency.

 > To achieve that, check-host-tar.mk fills in the
 > BR2_TAR_HOST_DEPENDENCY variable with host-tar if building a host-tar
 > is needed. The name BR2_TAR_HOST_DEPENDENCY has been chosen because it
 > matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
 > check-host-cmake.mk.

 > The BR2_TAR_HOST_DEPENDENCY is added to all packages, except host-tar
 > itself (obviously) and host-skeleton, because we depend on
 > host-skeleton to install host-tar properly in HOST_DIR.

 > In addition, we modify tar.mk to explicitly build host-tar without
 > ccache: since ccache source code is available as a tarball, ccache
 > will obviously depend on host-tar if the system tar is insufficient.

 > Finally, to make things really clean, we also add
 > $(BR2_TAR_HOST_DEPENDENCY) to the dependencies of the tar filesystem
 > format, since it requires tar, so we'd better make sure we have a
 > suitable tar.

 > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
 > Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
 > ---
 > Changes since v3:
 >  - Add dependency on BR2_TAR_HOST_DEPENDENCY in the tar filesystem
 >  - Use = instead of += when defining BR2_TAR_HOST_DEPENDENCY
 >  - Fix typo in commit log
 >  - Added Reviewed-by from Yann.

 > Changes since v2:
 >  - New patch

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v4 5/9] package/pkg-generic: handle host-xz as an extract dependency

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > This moves the host-xz dependency handling from
 > DEPENDENCY_HOST_PREREQ to an extract dependency.

 > To achieve that, check-host-xz.mk fills in the
 > BR2_XZ_HOST_DEPENDENCY with host-tar if building a host-tar is

Cut'n'paste? ;)

Both of these host-tar references should be host-xz.

Committed with that fixed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v4 6/9] package/pkg-generic: handle host-lzip as an extract dependency

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > This moves the host-xz dependency handling from
 > DEPENDENCY_HOST_PREREQ to an extract dependency.

 > To achieve that, check-host-xz.mk fills in the

This should be check-host-lzip.mk.

> BR2_LZIP_HOST_DEPENDENCY with host-tar if building a host-tar is

and host-lzip.

Committed with that fixed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v4 7/9] package/pkg-generic: handle host-ccache as a regular dependency

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > This moves the host-ccache dependency handling from
 > DEPENDENCIES_HOST_PREREQ to a proper package dependency. When
 > BR2_CCACHE=y, we add host-ccache as a regular dependency of all
 > packages except:

 >  - The extractor packages host-tar, host-xz and host-lzip

 >  - host-ccache itself

 >  - host-skeleton, because all packages depend on it

 > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
 > ---
 > Changes since v3:
 >  - Prevent host-ccache from being a dependency of itself, as noticed
 >    by Yann.

 > Changes since v2:
 >  - New patch

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v4 8/9] package/pkg-generic: handle host-fakedate as a regular dependency

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > This commit moves the host-fakedate dependency handling from
 > DEPENDENCIES_HOST_PREREQ to a proper regular dependency handled by the
 > package infrastructure.

 > host-fakedate is added as dependency to all packages, except
 > host-skeleton, because we depend on it.

 > In addition, we make sure that host-fakedate does not grow a
 > dependency on host-{tar,xz,lzip,ccache} to avoid circular
 > dependencies. host-fakedate does not need any extraction tool and does
 > not need to build C/C++ code (the source code is just a shell script
 > available in Buildroot).

 > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
 > Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
 > ---
 > Changes since v3:
 >  - Make sure we don't introduce a host-fakedate -> host-fakedate
 >    circular dependency.
 >  - Added Reviewed-by from Yann.

 > Changes since v2:
 >  - New patch

Committed, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v4 9/9] core: kill DEPENDENCIES_HOST_PREREQ

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > Now that DEPENDENCIES_HOST_PREREQ is no longer used anywhere, we can
 > kill it.

 > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
 > Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
 > ---
 > Changes since v3:
 >  - Added Acked-by from Yann.

 > Changes since v2:
 >  - New patch
 > ---
 >  Makefile                             |  2 --
 >  package/pkg-generic.mk               |  2 --
 >  support/dependencies/dependencies.mk | 10 ++--------
 >  3 files changed, 2 insertions(+), 12 deletions(-)

 > diff --git a/Makefile b/Makefile
 > index 925d6e4aec..a3a9d290d9 100644
 > --- a/Makefile
 > +++ b/Makefile
 > @@ -501,8 +501,6 @@ include package/Makefile.in
 >  -include $(sort $(wildcard arch/arch.mk.*))
 >  include support/dependencies/dependencies.mk
 
 > -PACKAGES += $(DEPENDENCIES_HOST_PREREQ)
 > -
 >  include $(sort $(wildcard toolchain/*.mk))
 >  include $(sort $(wildcard toolchain/*/*.mk))
 
 > diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
 > index 6d1f267328..6d82f7027e 100644
 > --- a/package/pkg-generic.mk
 > +++ b/package/pkg-generic.mk
 > @@ -690,9 +690,7 @@ $(1)-configure:			$$($(2)_TARGET_CONFIGURE)
 >  $$($(2)_TARGET_CONFIGURE):	| $$($(2)_FINAL_DEPENDENCIES)
 
 >  $$($(2)_TARGET_SOURCE) $$($(2)_TARGET_RSYNC): | dirs prepare
 > -ifeq ($$(filter $(1),$$(DEPENDENCIES_HOST_PREREQ)),)
 >  $$($(2)_TARGET_SOURCE) $$($(2)_TARGET_RSYNC): | dependencies
 > -endif

These two lines can now be simplified, but that can be done later.

Committed, thanks.

-- 
Bye, Peter Korsgaard