* [Buildroot] [PATCH 1 of 4] infra: Add POST_RSYNC_HOOKS support
2013-11-01 16:06 [Buildroot] [PATCH 0 of 4] Add POST_RSYNC hook Thomas De Schampheleire
@ 2013-11-01 16:06 ` Thomas De Schampheleire
2013-11-01 16:06 ` [Buildroot] [PATCH 2 of 4] infra: centralize rsync exclude list for VCS files Thomas De Schampheleire
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Thomas De Schampheleire @ 2013-11-01 16:06 UTC (permalink / raw)
To: buildroot
One of the use cases is for the 'local packages' to restore
the SCM info. Some packages use this information to generate
version info during build time. In this case, the local package
can have this hook to restore it by symbolic link for example.
[Thomas: update commit title]
Signed-off-by: Tzu-Jung Lee <tjlee@ambarella.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
Include the comments against v1 from Thomas De Schampheleire:
Update manual for the new hook.
Set the default hook to empty
docs/manual/adding-packages-generic.txt | 1 +
package/pkg-generic.mk | 2 ++
2 files changed, 3 insertions(+)
diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
--- a/docs/manual/adding-packages-generic.txt
+++ b/docs/manual/adding-packages-generic.txt
@@ -467,6 +467,7 @@ The following hook points are available:
* +LIBFOO_POST_DOWNLOAD_HOOKS+
* +LIBFOO_POST_EXTRACT_HOOKS+
+* +LIBFOO_POST_RSYNC_HOOKS+
* +LIBFOO_PRE_PATCH_HOOKS+
* +LIBFOO_POST_PATCH_HOOKS+
* +LIBFOO_PRE_CONFIGURE_HOOKS+
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -69,6 +69,7 @@ endif
@$(call MESSAGE,"Syncing from source dir $(SRCDIR)")
@test -d $(SRCDIR) || (echo "ERROR: $(SRCDIR) does not exist" ; exit 1)
rsync -au --cvs-exclude --include core $(SRCDIR)/ $(@D)
+ $(foreach hook,$($(PKG)_POST_RSYNC_HOOKS),$(call $(hook))$(sep))
$(Q)touch $@
# Handle the SOURCE_CHECK and SHOW_EXTERNAL_DEPS cases for rsynced
@@ -339,6 +340,7 @@ endif
# post-steps hooks
$(2)_POST_DOWNLOAD_HOOKS ?=
$(2)_POST_EXTRACT_HOOKS ?=
+$(2)_POST_RSYNC_HOOKS ?=
$(2)_PRE_PATCH_HOOKS ?=
$(2)_POST_PATCH_HOOKS ?=
$(2)_PRE_CONFIGURE_HOOKS ?=
^ permalink raw reply [flat|nested] 6+ messages in thread* [Buildroot] [PATCH 2 of 4] infra: centralize rsync exclude list for VCS files
2013-11-01 16:06 [Buildroot] [PATCH 0 of 4] Add POST_RSYNC hook Thomas De Schampheleire
2013-11-01 16:06 ` [Buildroot] [PATCH 1 of 4] infra: Add POST_RSYNC_HOOKS support Thomas De Schampheleire
@ 2013-11-01 16:06 ` Thomas De Schampheleire
2013-11-01 16:06 ` [Buildroot] [PATCH 3 of 4] manual: split info on hooks to a separate section/file Thomas De Schampheleire
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Thomas De Schampheleire @ 2013-11-01 16:06 UTC (permalink / raw)
To: buildroot
Buildroot has three places where rsync is used:
1. to copy the target skeleton
2. to copy the rootfs overlay(s)
3. to copy overridden package sources
In all of these cases, we want to exclude version control files by default.
Place 1 and 2 used an identical set of explicit --exclude options, while
place 3 used the option --cvs-exclude. This last option, however, not only
excludes version control files, but also binary files (.o, .so) and any file
or directory named 'core' (a problem for the linux kernel that has several
directories with this name). Moreover, the exact list of excluded files when
using --cvs-exclude depends on the version of rsync.
This patch creates one global variable RSYNC_VCS_EXCLUSIONS that can be used
by the various rsync commands. It excludes the version control files of
svn, git, hg, cvs and bzr.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
Makefile | 14 ++++++++------
package/pkg-generic.mk | 2 +-
2 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -426,11 +426,14 @@ ifeq ($(BR2_ROOTFS_SKELETON_CUSTOM),y)
TARGET_SKELETON=$(BR2_ROOTFS_SKELETON_CUSTOM_PATH)
endif
+RSYNC_VCS_EXCLUSIONS = \
+ --exclude .svn --exclude .git --exclude .hg --exclude .bzr \
+ --exclude CVS
+
$(BUILD_DIR)/.root:
mkdir -p $(TARGET_DIR)
- rsync -a \
- --exclude .empty --exclude .svn --exclude .git \
- --exclude .hg --exclude=CVS --exclude '*~' \
+ rsync -a $(RSYNC_VCS_EXCLUSIONS) \
+ --exclude .empty --exclude '*~' \
$(TARGET_SKELETON)/ $(TARGET_DIR)/
cp support/misc/target-dir-warning.txt $(TARGET_DIR_WARNING_FILE)
@ln -snf lib $(TARGET_DIR)/$(LIB_SYMLINK)
@@ -504,9 +507,8 @@ endif
@$(foreach d, $(call qstrip,$(BR2_ROOTFS_OVERLAY)), \
$(call MESSAGE,"Copying overlay $(d)"); \
- rsync -a \
- --exclude .empty --exclude .svn --exclude .git \
- --exclude .hg --exclude=CVS --exclude '*~' \
+ rsync -a $(RSYNC_VCS_EXCLUSIONS) \
+ --exclude .empty --exclude '*~' \
$(d)/ $(TARGET_DIR)$(sep))
@$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_BUILD_SCRIPT)), \
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -68,7 +68,7 @@ endif
$(BUILD_DIR)/%/.stamp_rsynced:
@$(call MESSAGE,"Syncing from source dir $(SRCDIR)")
@test -d $(SRCDIR) || (echo "ERROR: $(SRCDIR) does not exist" ; exit 1)
- rsync -au --cvs-exclude --include core $(SRCDIR)/ $(@D)
+ rsync -au $(RSYNC_VCS_EXCLUSIONS) $(SRCDIR)/ $(@D)
$(foreach hook,$($(PKG)_POST_RSYNC_HOOKS),$(call $(hook))$(sep))
$(Q)touch $@
^ permalink raw reply [flat|nested] 6+ messages in thread* [Buildroot] [PATCH 3 of 4] manual: split info on hooks to a separate section/file
2013-11-01 16:06 [Buildroot] [PATCH 0 of 4] Add POST_RSYNC hook Thomas De Schampheleire
2013-11-01 16:06 ` [Buildroot] [PATCH 1 of 4] infra: Add POST_RSYNC_HOOKS support Thomas De Schampheleire
2013-11-01 16:06 ` [Buildroot] [PATCH 2 of 4] infra: centralize rsync exclude list for VCS files Thomas De Schampheleire
@ 2013-11-01 16:06 ` Thomas De Schampheleire
2013-11-01 16:06 ` [Buildroot] [PATCH 4 of 4] manual: add some info on the POST_RSYNC hook Thomas De Schampheleire
2013-11-01 21:08 ` [Buildroot] [PATCH 0 of 4] Add " Samuel Martin
4 siblings, 0 replies; 6+ messages in thread
From: Thomas De Schampheleire @ 2013-11-01 16:06 UTC (permalink / raw)
To: buildroot
Split out the information on hooks to a separate section (and source file).
Not only because the hooks are useful for all infrastructures (and thus
don't really fit specifically in the generic infrastructure section), but
also for clarity when the info on hooks will be expanded in later patches.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
docs/manual/adding-packages-autotools.txt | 3 +-
docs/manual/adding-packages-cmake.txt | 3 +-
docs/manual/adding-packages-generic.txt | 40 +-------------------------
docs/manual/adding-packages-hooks.txt | 41 +++++++++++++++++++++++++++
docs/manual/adding-packages.txt | 2 +
5 files changed, 46 insertions(+), 43 deletions(-)
diff --git a/docs/manual/adding-packages-autotools.txt b/docs/manual/adding-packages-autotools.txt
--- a/docs/manual/adding-packages-autotools.txt
+++ b/docs/manual/adding-packages-autotools.txt
@@ -162,8 +162,7 @@ well for most autotools-based packages.
still possible to customize what is done in any particular step:
* By adding a post-operation hook (after extract, patch, configure,
- build or install). See the reference documentation of the generic
- infrastructure for details.
+ build or install). See xref:hooks[] for details.
* By overriding one of the steps. For example, even if the autotools
infrastructure is used, if the package +.mk+ file defines its
diff --git a/docs/manual/adding-packages-cmake.txt b/docs/manual/adding-packages-cmake.txt
--- a/docs/manual/adding-packages-cmake.txt
+++ b/docs/manual/adding-packages-cmake.txt
@@ -135,8 +135,7 @@ for most CMake-based packages. However,
possible to customize what is done in any particular step:
* By adding a post-operation hook (after extract, patch, configure,
- build or install). See the reference documentation of the generic
- infrastructure for details.
+ build or install). See xref:hooks[] for details.
* By overriding one of the steps. For example, even if the CMake
infrastructure is used, if the package +.mk+ file defines its own
diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
--- a/docs/manual/adding-packages-generic.txt
+++ b/docs/manual/adding-packages-generic.txt
@@ -451,42 +451,4 @@ In the action definitions, you can use t
* Of course the +$(HOST_DIR)+, +$(STAGING_DIR)+ and +$(TARGET_DIR)+
variables to install the packages properly.
-The last feature of the generic infrastructure is the ability to add
-hooks. These define further actions to perform after existing steps.
-Most hooks aren't really useful for generic packages, since the +.mk+
-file already has full control over the actions performed in each step
-of the package construction. The hooks are more useful for packages
-using the autotools infrastructure described below. However, since
-they are provided by the generic infrastructure, they are documented
-here. The exception is +LIBFOO_POST_PATCH_HOOKS+. Patching the
-package and producing legal info are not user definable, so
-+LIBFOO_POST_PATCH_HOOKS+ and +LIBFOO_POST_LEGAL_INFO_HOOKS+ are
-useful for generic packages.
-
-The following hook points are available:
-
-* +LIBFOO_POST_DOWNLOAD_HOOKS+
-* +LIBFOO_POST_EXTRACT_HOOKS+
-* +LIBFOO_POST_RSYNC_HOOKS+
-* +LIBFOO_PRE_PATCH_HOOKS+
-* +LIBFOO_POST_PATCH_HOOKS+
-* +LIBFOO_PRE_CONFIGURE_HOOKS+
-* +LIBFOO_POST_CONFIGURE_HOOKS+
-* +LIBFOO_POST_BUILD_HOOKS+
-* +LIBFOO_POST_INSTALL_HOOKS+ (for host packages only)
-* +LIBFOO_POST_INSTALL_STAGING_HOOKS+ (for target packages only)
-* +LIBFOO_POST_INSTALL_TARGET_HOOKS+ (for target packages only)
-* +LIBFOO_POST_LEGAL_INFO_HOOKS+
-
-These variables are 'lists' of variable names containing actions to be
-performed at this hook point. This allows several hooks to be
-registered at a given hook point. Here is an example:
-
-----------------------
-define LIBFOO_POST_PATCH_FIXUP
- action1
- action2
-endef
-
-LIBFOO_POST_PATCH_HOOKS += LIBFOO_POST_PATCH_FIXUP
-----------------------
+Finally, you can also use hooks. See xref:hooks[] for more information.
diff --git a/docs/manual/adding-packages-hooks.txt b/docs/manual/adding-packages-hooks.txt
new file mode 100644
--- /dev/null
+++ b/docs/manual/adding-packages-hooks.txt
@@ -0,0 +1,41 @@
+// -*- mode:doc; -*-
+// vim: set syntax=asciidoc:
+
+[[hooks]]
+Hooks available in the various build steps
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The generic infrastructure (and as a result also the derived autotools
+and cmake infrastructures) allow packages to specify hooks.
+These define further actions to perform after existing steps.
+Most hooks aren't really useful for generic packages, since the +.mk+
+file already has full control over the actions performed in each step
+of the package construction.
+
+The following hook points are available:
+
+* +LIBFOO_POST_DOWNLOAD_HOOKS+
+* +LIBFOO_POST_EXTRACT_HOOKS+
+* +LIBFOO_POST_RSYNC_HOOKS+
+* +LIBFOO_PRE_PATCH_HOOKS+
+* +LIBFOO_POST_PATCH_HOOKS+
+* +LIBFOO_PRE_CONFIGURE_HOOKS+
+* +LIBFOO_POST_CONFIGURE_HOOKS+
+* +LIBFOO_POST_BUILD_HOOKS+
+* +LIBFOO_POST_INSTALL_HOOKS+ (for host packages only)
+* +LIBFOO_POST_INSTALL_STAGING_HOOKS+ (for target packages only)
+* +LIBFOO_POST_INSTALL_TARGET_HOOKS+ (for target packages only)
+* +LIBFOO_POST_LEGAL_INFO_HOOKS+
+
+These variables are 'lists' of variable names containing actions to be
+performed at this hook point. This allows several hooks to be
+registered at a given hook point. Here is an example:
+
+----------------------
+define LIBFOO_POST_PATCH_FIXUP
+ action1
+ action2
+endef
+
+LIBFOO_POST_PATCH_HOOKS += LIBFOO_POST_PATCH_FIXUP
+----------------------
diff --git a/docs/manual/adding-packages.txt b/docs/manual/adding-packages.txt
--- a/docs/manual/adding-packages.txt
+++ b/docs/manual/adding-packages.txt
@@ -18,6 +18,8 @@ include::adding-packages-autotools.txt[]
include::adding-packages-cmake.txt[]
+include::adding-packages-hooks.txt[]
+
include::adding-packages-gettext.txt[]
include::adding-packages-tips.txt[]
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] [PATCH 4 of 4] manual: add some info on the POST_RSYNC hook
2013-11-01 16:06 [Buildroot] [PATCH 0 of 4] Add POST_RSYNC hook Thomas De Schampheleire
` (2 preceding siblings ...)
2013-11-01 16:06 ` [Buildroot] [PATCH 3 of 4] manual: split info on hooks to a separate section/file Thomas De Schampheleire
@ 2013-11-01 16:06 ` Thomas De Schampheleire
2013-11-01 21:08 ` [Buildroot] [PATCH 0 of 4] Add " Samuel Martin
4 siblings, 0 replies; 6+ messages in thread
From: Thomas De Schampheleire @ 2013-11-01 16:06 UTC (permalink / raw)
To: buildroot
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
docs/manual/adding-packages-hooks.txt | 19 +++++++++++++++++++
1 files changed, 19 insertions(+), 0 deletions(-)
diff --git a/docs/manual/adding-packages-hooks.txt b/docs/manual/adding-packages-hooks.txt
--- a/docs/manual/adding-packages-hooks.txt
+++ b/docs/manual/adding-packages-hooks.txt
@@ -39,3 +39,22 @@ endef
LIBFOO_POST_PATCH_HOOKS += LIBFOO_POST_PATCH_FIXUP
----------------------
+
+Using the +POST_RSYNC+ hook
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The +POST_RSYNC+ hook is run only for packages that have a local source
+override, using the +OVERRIDE_SRCDIR+ mechanism. In that case, the
+package sources are copied using +rsync+ from a local location into the
+buildroot build directory. The +rsync+ command does not copy all files
+from the source directory, though. Files belonging to a version control
+system, like the directories +.git+, +.hg+, etc. are not copied. For
+most packages this is sufficient, but a given package can perform
+additional actions using the +POST_RSYNC+ hook.
+
+In principle, the hook can contain any command you want. One specific
+use case, though, is the intentional copying of the version control
+directory using +rsync+. The +rsync+ command you use in the hook can, among
+others, use the following variables:
+
+* +$(SRCDIR)+: the path to the overridden source directory
+* +$(@D)+: the path to the build directory
^ permalink raw reply [flat|nested] 6+ messages in thread* [Buildroot] [PATCH 0 of 4] Add POST_RSYNC hook
2013-11-01 16:06 [Buildroot] [PATCH 0 of 4] Add POST_RSYNC hook Thomas De Schampheleire
` (3 preceding siblings ...)
2013-11-01 16:06 ` [Buildroot] [PATCH 4 of 4] manual: add some info on the POST_RSYNC hook Thomas De Schampheleire
@ 2013-11-01 21:08 ` Samuel Martin
4 siblings, 0 replies; 6+ messages in thread
From: Samuel Martin @ 2013-11-01 21:08 UTC (permalink / raw)
To: buildroot
Hi Thomas, all,
2013/11/1 Thomas De Schampheleire <patrickdepinguin@gmail.com>
> This patch series adopts the POST_RSYNC hook patch by Tzu-Jung Lee and
> changes the default rsync exclusion list, as discussed on the Buildroot
> developer days.
> The other rsync-related patch submitted by Tzu-Jung Lee (custom rsync
> command) should not be needed anymore as its use case is covered by
> the second patch in this series.
>
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
>
> ---
> Makefile | 14 +++--
> docs/manual/adding-packages-autotools.txt | 3 +-
> docs/manual/adding-packages-cmake.txt | 3 +-
> docs/manual/adding-packages-generic.txt | 39 +-----------------
> docs/manual/adding-packages-hooks.txt | 60
> +++++++++++++++++++++++++++
> docs/manual/adding-packages.txt | 2 +
> package/pkg-generic.mk | 4 +-
> 7 files changed, 76 insertions(+), 49 deletions(-)
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>
For the whole series:
Acked-by: Samuel Martin <s.martin49@gmail.com>
Regards,
--
Samuel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20131101/ef0689dd/attachment.html>
^ permalink raw reply [flat|nested] 6+ messages in thread