Buildroot Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: Herve Codina <herve.codina@bootlin.com>
To: buildroot@busybox.net
Subject: [Buildroot] [PATCH 13/15] Makefile: Rsync global {TARGET, HOST}_DIR using exclusion file list
Date: Mon, 21 Jun 2021 16:11:28 +0200	[thread overview]
Message-ID: <20210621141130.48654-14-herve.codina@bootlin.com> (raw)
In-Reply-To: <20210621141130.48654-1-herve.codina@bootlin.com>

On a per-package build, rsync final {TARGET,HOST}_DIR using
exclusion file list computed for each packages.
As we rsync only files generated by each packages, we need to
to do this rsync recursively on each dependencies to collect
all needed files. This is done based on existing
<PKG>_FINAL_RECURSIVE_DEPENDENCIES

Signed-off-by: Herve Codina <herve.codina@bootlin.com>
---
 Makefile | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 472af5a318..50bc6b4503 100644
--- a/Makefile
+++ b/Makefile
@@ -740,10 +740,28 @@ TARGET_DIR_FILES_LISTS = $(sort $(wildcard $(BUILD_DIR)/*/.files-list.txt))
 HOST_DIR_FILES_LISTS = $(sort $(wildcard $(BUILD_DIR)/*/.files-list-host.txt))
 STAGING_DIR_FILES_LISTS = $(sort $(wildcard $(BUILD_DIR)/*/.files-list-staging.txt))
 
+ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y)
+# rsync the contents of per-package directories
+# $1: space-separated list of packages to rsync from
+# $2: 'host' or 'target'
+# $3: destination directory
+# $4: exclude file list to use
+define per-package-rsync-delta
+	mkdir -p $(3)
+	$(foreach pkg,$(1),\
+		rsync -a --link-dest=$(PER_PACKAGE_DIR)/$(pkg)/$(strip $(2))/ \
+		--filter='merge $($(call UPPERCASE,$(pkg))_DIR)/$(strip $(4))' \
+		$(PER_PACKAGE_DIR)/$(pkg)/$(strip $(2))/ \
+		$(3)$(sep))
+endef
+endif
+
 .PHONY: host-finalize
 host-finalize: $(PACKAGES) $(HOST_DIR) $(HOST_DIR_SYMLINK)
 	@$(call MESSAGE,"Finalizing host directory")
-	$(call per-package-rsync,$(sort $(PACKAGES)),host,$(HOST_DIR))
+	$(call per-package-rsync-delta, $(sort $(foreach pkg, $(PACKAGES), \
+		$(pkg) $($(call UPPERCASE,$(pkg))_FINAL_RECURSIVE_DEPENDENCIES))), \
+		host,$(HOST_DIR),.files-final-rsync-host.exclude_rsync)
 
 .PHONY: staging-finalize
 staging-finalize: $(STAGING_DIR_SYMLINK)
@@ -751,7 +769,9 @@ staging-finalize: $(STAGING_DIR_SYMLINK)
 .PHONY: target-finalize
 target-finalize: $(PACKAGES) $(TARGET_DIR) host-finalize
 	@$(call MESSAGE,"Finalizing target directory")
-	$(call per-package-rsync,$(sort $(PACKAGES)),target,$(TARGET_DIR))
+	$(call per-package-rsync-delta, $(sort $(foreach pkg, $(PACKAGES), \
+		$(pkg) $($(call UPPERCASE,$(pkg))_FINAL_RECURSIVE_DEPENDENCIES))), \
+		target,$(TARGET_DIR),.files-final-rsync.exclude_rsync)
 	$(foreach hook,$(TARGET_FINALIZE_HOOKS),$($(hook))$(sep))
 	rm -rf $(TARGET_DIR)/usr/include $(TARGET_DIR)/usr/share/aclocal \
 		$(TARGET_DIR)/usr/lib/pkgconfig $(TARGET_DIR)/usr/share/pkgconfig \
-- 
2.31.1

  parent reply	other threads:[~2021-06-21 14:11 UTC|newest]

Thread overview: 83+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-21 14:11 [Buildroot] [PATCH 00/15] Overwritten file detection and fixes, one more step to TLP build Herve Codina
2021-06-21 14:11 ` [Buildroot] [PATCH 01/15] package/pkg-generic.mk: detect files overwritten in TARGET_DIR and HOST_DIR Herve Codina
2021-06-21 21:31   ` Yann E. MORIN
2021-06-22  7:40     ` Herve Codina
2021-06-22  9:30       ` Thomas Petazzoni
2021-06-22  9:57         ` Nicolas Cavallari
2021-06-22 10:24           ` Yann E. MORIN
2021-06-24 14:09             ` Herve Codina
2021-06-24 16:18               ` Yann E. MORIN
2021-06-21 14:11 ` [Buildroot] [PATCH 02/15] package/e2fsprogs: fix fsck overwrite in HOST_DIR Herve Codina
2021-06-21 20:52   ` Thomas Petazzoni
2021-06-22 16:26     ` Herve Codina
2021-06-22 19:40   ` Yann E. MORIN
2021-06-24 14:13     ` Herve Codina
2021-06-21 14:11 ` [Buildroot] [PATCH 03/15] package/pkg-generic.mk: Remove Info documents dir entry Herve Codina
2021-06-21 20:51   ` Thomas Petazzoni
2021-06-22  8:43     ` Herve Codina
2021-06-22  9:34       ` Thomas Petazzoni
2021-06-22 20:18         ` Yann E. MORIN
2021-06-24 15:03           ` Herve Codina
2021-06-21 14:11 ` [Buildroot] [PATCH 04/15] package/pkg-generic.mk: Fix .la files overwrite detection Herve Codina
2021-06-21 20:54   ` Thomas Petazzoni
2021-06-22 18:01     ` Herve Codina
2021-06-21 21:42   ` Yann E. MORIN
2021-06-22  9:31     ` Herve Codina
2021-06-22  9:56       ` Yann E. MORIN
2021-06-22 10:12         ` Thomas Petazzoni
2021-06-22 10:30           ` Yann E. MORIN
2021-06-24 15:44             ` Herve Codina
2021-06-24 16:22               ` Yann E. MORIN
2021-06-21 14:11 ` [Buildroot] [PATCH 05/15] package/pkg-generic.mk: Perform .la files fixup in per-package HOST_DIR Herve Codina
2021-06-21 20:48   ` Thomas Petazzoni
2021-06-22  9:38     ` Herve Codina
2021-06-22 10:12   ` Yann E. MORIN
2021-06-24 16:20     ` Herve Codina
2021-06-21 14:11 ` [Buildroot] [PATCH 06/15] package/pkg-generic.mk: Introduce <PKG>_PER_PACKAGE_TWEAK_HOOKS Herve Codina
2021-06-21 15:10   ` Thomas Petazzoni
2021-06-22 20:39   ` Yann E. MORIN
2021-06-23 12:40     ` Thomas Petazzoni
2021-06-25  7:15       ` Herve Codina
2021-06-25  7:21     ` Herve Codina
2021-07-02  7:18       ` Herve Codina
2021-07-03  6:21         ` Yann E. MORIN
2021-06-21 14:11 ` [Buildroot] [PATCH 07/15] package/apr-util: Use <PKG>_PER_PACKAGE_TWEAK_HOOKS Herve Codina
2021-06-21 20:56   ` Thomas Petazzoni
2021-06-22  9:47     ` Herve Codina
2021-06-22 20:42       ` Yann E. MORIN
2021-06-25  7:30         ` Herve Codina
2021-06-21 14:11 ` [Buildroot] [PATCH 08/15] package/apache: Move APACHE_FIXUP_APR_LIBTOOL to <PKG>_PER_PACKAGE_TWEAK_HOOKS Herve Codina
2021-06-22 20:43   ` Yann E. MORIN
2021-06-21 14:11 ` [Buildroot] [PATCH 09/15] package/pkg-python: Remove _sysconfigdata*.pyc files when _sysconfigdata*.py are changed Herve Codina
2021-06-21 15:12   ` Thomas Petazzoni
2021-06-22 17:52     ` Herve Codina
2021-06-22 20:50       ` Yann E. MORIN
2021-06-25  8:04         ` Herve Codina
2021-06-21 14:11 ` [Buildroot] [PATCH 10/15] package/pkg-generic.mk: Move python fixup to generic package infrastructure Herve Codina
2021-06-22 21:01   ` Yann E. MORIN
2021-06-25  8:22     ` Herve Codina
2021-06-25  9:27       ` Yann E. MORIN
2021-06-21 14:11 ` [Buildroot] [PATCH 11/15] package/owfs: Remove Python sysconfigdata fixup Herve Codina
2021-06-22 21:02   ` Yann E. MORIN
2021-06-25  8:25     ` Herve Codina
2021-06-21 14:11 ` [Buildroot] [PATCH 12/15] package/pkg-generic.mk: Generate final rsync exclude file list Herve Codina
2021-06-22 21:15   ` Yann E. MORIN
2021-06-25  9:05     ` Herve Codina
2021-06-25  9:32       ` Yann E. MORIN
2021-06-21 14:11 ` Herve Codina [this message]
2021-06-24 20:20   ` [Buildroot] [PATCH 13/15] Makefile: Rsync global {TARGET, HOST}_DIR using exclusion " Yann E. MORIN
2021-06-24 20:34     ` Yann E. MORIN
2021-06-25 11:59       ` Herve Codina
2021-06-25 12:50         ` Yann E. MORIN
2021-06-25 12:00     ` Herve Codina
2021-06-21 14:11 ` [Buildroot] [PATCH 14/15] Makefile: Breaks hardlinks in global {TARGET, HOST}_DIR on per-package build Herve Codina
2021-06-24 20:22   ` Yann E. MORIN
2021-06-21 14:11 ` [Buildroot] [PATCH 15/15] package/pkg-generic.mk: Fix per-package <pkg>-{reconfigure, rebuild, reinstall} Herve Codina
2021-06-24 20:44   ` Yann E. MORIN
2021-06-25 14:00     ` Herve Codina
2021-06-21 20:42 ` [Buildroot] [PATCH 00/15] Overwritten file detection and fixes, one more step to TLP build Arnout Vandecappelle
2021-07-06 14:15   ` Herve Codina
2021-06-24 20:53 ` Yann E. MORIN
2021-06-25  9:08 ` Andreas Naumann
2021-06-25 13:13   ` Herve Codina
2021-06-25 14:55     ` Andreas Naumann

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210621141130.48654-14-herve.codina@bootlin.com \
    --to=herve.codina@bootlin.com \
    --cc=buildroot@busybox.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox