[Buildroot] [PATCH 0 of 2 v4-resend] Add find utility functions / exclude files from strip

[Buildroot] [PATCH 0 of 2 v4-resend] Add find utility functions / exclude files from strip

[Thomas De Schampheleire]

This patch series adds a few utility functions to Makefile.package.in that
help in composing find commands, and uses these functions to exclude
user-configured files and directories from being stripped in target-finalize.

Thanks to input from Arnout, the find functions are now much simpler and
should be clearly understandable by everyone.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---
v4: simplify find functions even further; whitespace changes
v3: update based on review comments, remove Python changes for now
v2: simplify find functions

 Config.in                   |  19 +++++++++++++++++++
 Makefile                    |  10 ++++++++--
 package/Makefile.package.in |  10 ++++++++++
 3 files changed, 37 insertions(+), 2 deletions(-)

[Buildroot] [PATCH 1 of 2 v4-resend] Makefile.package.in: add utility functions find*clauses and notfirstword

[Thomas De Schampheleire]

This patch adds a few utility functions to Makefile.package.in.

Functions finddirclauses and findfileclauses help in building a find command
that skips a set of directories and performs operations on a set of files.
This pattern can for example be used to keep certain files or directories from
being stripped, or to remove certain files from a package installation.

The notfirstword function is the inverse of the 'firstword' function in make:
it returns all but the first word.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

---
v4: simplify find*clauses even further. Thanks, Arnout!
v3: use -path instead of -wholename; fix a bug in finddirclauses. Thanks, Cam.
v2: integrate _cont variants in main find*clauses functions. Thanks, Arnout.

 package/Makefile.package.in |  10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/package/Makefile.package.in b/package/Makefile.package.in
--- a/package/Makefile.package.in
+++ b/package/Makefile.package.in
@@ -87,6 +87,16 @@ MESSAGE = echo "$(TERM_BOLD)>>> $($(PKG)
 TERM_BOLD := $(shell tput smso)
 TERM_RESET := $(shell tput rmso)
 
+# Utility functions for 'find'
+# findfileclauses(filelist) => -name 'X' -o -name 'Y'
+findfileclauses = $(call notfirstword,$(patsubst %,-o -name '%',$(1)))
+# finddirclauses(base, dirlist) => -path 'base/dirX' -o -path 'base/dirY'
+finddirclauses = $(call notfirstword,$(patsubst %,-o -path '$(1)/%',$(2)))
+
+# Miscellaneous utility functions
+# notfirstword(wordlist): returns all but the first word in wordlist
+notfirstword = $(wordlist 2,$(words $(1)),$(1))
+
 # Download method commands
 WGET:=$(call qstrip,$(BR2_WGET)) $(QUIET)
 SVN:=$(call qstrip,$(BR2_SVN))

[Buildroot] [PATCH 2 of 2 v4-resend] build: add option to exclude executables/dirs from being stripped

[Thomas De Schampheleire]

Sometimes it may be desirable to keep debug symbols for some binaries and
libraries on the target. This commit introduces the config option
BR2_STRIP_EXCLUDE_FILES, which is interpreted as a list of such binaries
and libraries, and the option BR2_STRIP_EXCLUDE_DIRS, which indicates
directories excluded from stripping entirely.
These exclusions are passed to the find command in the target-finalize step.

Signed-off-by: Sven Neumann <s.neumann@raumfeld.com>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

---
v4: whitespace changes
v3: merge with redundant patch 'stripping: use findfileclauses utility function'
v2: add the possibility to exclude entire directories

 Config.in |  19 +++++++++++++++++++
 Makefile  |  10 ++++++++--
 2 files changed, 27 insertions(+), 2 deletions(-)

diff --git a/Config.in b/Config.in
--- a/Config.in
+++ b/Config.in
@@ -269,6 +269,25 @@ config BR2_STRIP_none
 	  none    do not strip (only for debugging!)
 endchoice
 
+config BR2_STRIP_EXCLUDE_FILES
+	string "executables that should not be stripped"
+	depends on !BR2_STRIP_none
+	default ""
+	help
+	  You may specify a space-separated list of binaries and libraries
+	  here that should not be stripped on the target.
+
+config BR2_STRIP_EXCLUDE_DIRS
+	string "directories that should be skipped when stripping"
+	depends on !BR2_STRIP_none
+	default ""
+	help
+	  You may specify a space-separated list of directories that should
+	  be skipped when stripping. Binaries and libraries in these
+	  directories will not be touched.
+	  The directories should be specified relative to the target directory,
+	  without leading slash.
+
 choice
 	prompt "gcc optimization level"
 	default BR2_OPTIMIZE_S
diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -438,6 +438,13 @@ endif
 erase-fakeroots:
 	rm -f $(BUILD_DIR)/.fakeroot*
 
+STRIP_FIND_CMD = find $(TARGET_DIR)
+ifneq (,$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS)))
+STRIP_FIND_CMD += \( $(call finddirclauses,$(TARGET_DIR),$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS))) \) -prune -o
+endif
+STRIP_FIND_CMD += -type f -perm +111
+STRIP_FIND_CMD += -not \( $(call findfileclauses,libthread_db*.so* $(call qstrip,$(BR2_STRIP_EXCLUDE_FILES))) \) -print
+
 target-finalize:
 ifeq ($(BR2_HAVE_DEVFILES),y)
 	( support/scripts/copy.sh $(STAGING_DIR) $(TARGET_DIR) )
@@ -462,8 +469,7 @@ endif
 ifeq ($(BR2_PACKAGE_PYTHON_PYC_ONLY),y)
 	find $(TARGET_DIR)/usr/lib/ -name '*.py' -print0 | xargs -0 rm -f
 endif
-	find $(TARGET_DIR) -type f -perm +111 '!' -name 'libthread_db*.so*' | \
-		xargs $(STRIPCMD) 2>/dev/null || true
+	$(STRIP_FIND_CMD) | xargs $(STRIPCMD) 2>/dev/null || true
 	find $(TARGET_DIR)/lib/modules -type f -name '*.ko' | \
 		xargs -r $(KSTRIPCMD) || true
 

[Buildroot] [PATCH 0 of 2 v4-resend] Add find utility functions / exclude files from strip

[Thomas De Schampheleire]

On Wed, Mar 14, 2012 at 5:45 PM, Thomas De Schampheleire
<patrickdepinguin+buildroot@gmail.com> wrote:
> This patch series adds a few utility functions to Makefile.package.in that
> help in composing find commands, and uses these functions to exclude
> user-configured files and directories from being stripped in target-finalize.
>
> Thanks to input from Arnout, the find functions are now much simpler and
> should be clearly understandable by everyone.
>
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
>
> ---
> v4: simplify find functions even further; whitespace changes
> v3: update based on review comments, remove Python changes for now
> v2: simplify find functions
>
> ?Config.in ? ? ? ? ? ? ? ? ? | ?19 +++++++++++++++++++
> ?Makefile ? ? ? ? ? ? ? ? ? ?| ?10 ++++++++--
> ?package/Makefile.package.in | ?10 ++++++++++
> ?3 files changed, 37 insertions(+), 2 deletions(-)

bump

[Buildroot] [PATCH 1 of 2 v4-resend] Makefile.package.in: add utility functions find*clauses and notfirstword

[Thomas De Schampheleire]

On Wed, Mar 14, 2012 at 5:45 PM, Thomas De Schampheleire
<patrickdepinguin+buildroot@gmail.com> wrote:
> This patch adds a few utility functions to Makefile.package.in.
>
> Functions finddirclauses and findfileclauses help in building a find command
> that skips a set of directories and performs operations on a set of files.
> This pattern can for example be used to keep certain files or directories from
> being stripped, or to remove certain files from a package installation.
>
> The notfirstword function is the inverse of the 'firstword' function in make:
> it returns all but the first word.
>
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
>
> ---
> v4: simplify find*clauses even further. Thanks, Arnout!
> v3: use -path instead of -wholename; fix a bug in finddirclauses. Thanks, Cam.
> v2: integrate _cont variants in main find*clauses functions. Thanks, Arnout.
>
> ?package/Makefile.package.in | ?10 ++++++++++
> ?1 files changed, 10 insertions(+), 0 deletions(-)
>
> diff --git a/package/Makefile.package.in b/package/Makefile.package.in
> --- a/package/Makefile.package.in
> +++ b/package/Makefile.package.in
> @@ -87,6 +87,16 @@ MESSAGE = echo "$(TERM_BOLD)>>> $($(PKG)
> ?TERM_BOLD := $(shell tput smso)
> ?TERM_RESET := $(shell tput rmso)
>
> +# Utility functions for 'find'
> +# findfileclauses(filelist) => -name 'X' -o -name 'Y'
> +findfileclauses = $(call notfirstword,$(patsubst %,-o -name '%',$(1)))
> +# finddirclauses(base, dirlist) => -path 'base/dirX' -o -path 'base/dirY'
> +finddirclauses = $(call notfirstword,$(patsubst %,-o -path '$(1)/%',$(2)))
> +
> +# Miscellaneous utility functions
> +# notfirstword(wordlist): returns all but the first word in wordlist
> +notfirstword = $(wordlist 2,$(words $(1)),$(1))
> +
> ?# Download method commands
> ?WGET:=$(call qstrip,$(BR2_WGET)) $(QUIET)
> ?SVN:=$(call qstrip,$(BR2_SVN))

bump

[Buildroot] [PATCH 2 of 2 v4-resend] build: add option to exclude executables/dirs from being stripped

[Thomas De Schampheleire]

On Wed, Mar 14, 2012 at 5:45 PM, Thomas De Schampheleire
<patrickdepinguin+buildroot@gmail.com> wrote:
> Sometimes it may be desirable to keep debug symbols for some binaries and
> libraries on the target. This commit introduces the config option
> BR2_STRIP_EXCLUDE_FILES, which is interpreted as a list of such binaries
> and libraries, and the option BR2_STRIP_EXCLUDE_DIRS, which indicates
> directories excluded from stripping entirely.
> These exclusions are passed to the find command in the target-finalize step.
>
> Signed-off-by: Sven Neumann <s.neumann@raumfeld.com>
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
>
> ---
> v4: whitespace changes
> v3: merge with redundant patch 'stripping: use findfileclauses utility function'
> v2: add the possibility to exclude entire directories
>
> ?Config.in | ?19 +++++++++++++++++++
> ?Makefile ?| ?10 ++++++++--
> ?2 files changed, 27 insertions(+), 2 deletions(-)
>
> diff --git a/Config.in b/Config.in
> --- a/Config.in
> +++ b/Config.in
> @@ -269,6 +269,25 @@ config BR2_STRIP_none
> ? ? ? ? ?none ? ?do not strip (only for debugging!)
> ?endchoice
>
> +config BR2_STRIP_EXCLUDE_FILES
> + ? ? ? string "executables that should not be stripped"
> + ? ? ? depends on !BR2_STRIP_none
> + ? ? ? default ""
> + ? ? ? help
> + ? ? ? ? You may specify a space-separated list of binaries and libraries
> + ? ? ? ? here that should not be stripped on the target.
> +
> +config BR2_STRIP_EXCLUDE_DIRS
> + ? ? ? string "directories that should be skipped when stripping"
> + ? ? ? depends on !BR2_STRIP_none
> + ? ? ? default ""
> + ? ? ? help
> + ? ? ? ? You may specify a space-separated list of directories that should
> + ? ? ? ? be skipped when stripping. Binaries and libraries in these
> + ? ? ? ? directories will not be touched.
> + ? ? ? ? The directories should be specified relative to the target directory,
> + ? ? ? ? without leading slash.
> +
> ?choice
> ? ? ? ?prompt "gcc optimization level"
> ? ? ? ?default BR2_OPTIMIZE_S
> diff --git a/Makefile b/Makefile
> --- a/Makefile
> +++ b/Makefile
> @@ -438,6 +438,13 @@ endif
> ?erase-fakeroots:
> ? ? ? ?rm -f $(BUILD_DIR)/.fakeroot*
>
> +STRIP_FIND_CMD = find $(TARGET_DIR)
> +ifneq (,$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS)))
> +STRIP_FIND_CMD += \( $(call finddirclauses,$(TARGET_DIR),$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS))) \) -prune -o
> +endif
> +STRIP_FIND_CMD += -type f -perm +111
> +STRIP_FIND_CMD += -not \( $(call findfileclauses,libthread_db*.so* $(call qstrip,$(BR2_STRIP_EXCLUDE_FILES))) \) -print
> +
> ?target-finalize:
> ?ifeq ($(BR2_HAVE_DEVFILES),y)
> ? ? ? ?( support/scripts/copy.sh $(STAGING_DIR) $(TARGET_DIR) )
> @@ -462,8 +469,7 @@ endif
> ?ifeq ($(BR2_PACKAGE_PYTHON_PYC_ONLY),y)
> ? ? ? ?find $(TARGET_DIR)/usr/lib/ -name '*.py' -print0 | xargs -0 rm -f
> ?endif
> - ? ? ? find $(TARGET_DIR) -type f -perm +111 '!' -name 'libthread_db*.so*' | \
> - ? ? ? ? ? ? ? xargs $(STRIPCMD) 2>/dev/null || true
> + ? ? ? $(STRIP_FIND_CMD) | xargs $(STRIPCMD) 2>/dev/null || true
> ? ? ? ?find $(TARGET_DIR)/lib/modules -type f -name '*.ko' | \
> ? ? ? ? ? ? ? ?xargs -r $(KSTRIPCMD) || true
>

bump

[Buildroot] [PATCH 2 of 2 v4-resend] build: add option to exclude executables/dirs from being stripped

[Thomas De Schampheleire]

On Fri, Mar 23, 2012 at 3:53 PM, Thomas De Schampheleire
<patrickdepinguin+buildroot@gmail.com> wrote:
> On Wed, Mar 14, 2012 at 5:45 PM, Thomas De Schampheleire
> <patrickdepinguin+buildroot@gmail.com> wrote:
>> Sometimes it may be desirable to keep debug symbols for some binaries and
>> libraries on the target. This commit introduces the config option
>> BR2_STRIP_EXCLUDE_FILES, which is interpreted as a list of such binaries
>> and libraries, and the option BR2_STRIP_EXCLUDE_DIRS, which indicates
>> directories excluded from stripping entirely.
>> These exclusions are passed to the find command in the target-finalize step.
>>
>> Signed-off-by: Sven Neumann <s.neumann@raumfeld.com>
>> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
>> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
>>
>> ---
>> v4: whitespace changes
>> v3: merge with redundant patch 'stripping: use findfileclauses utility function'
>> v2: add the possibility to exclude entire directories
>>
>> ?Config.in | ?19 +++++++++++++++++++
>> ?Makefile ?| ?10 ++++++++--
>> ?2 files changed, 27 insertions(+), 2 deletions(-)
>>
>> diff --git a/Config.in b/Config.in
>> --- a/Config.in
>> +++ b/Config.in
>> @@ -269,6 +269,25 @@ config BR2_STRIP_none
>> ? ? ? ? ?none ? ?do not strip (only for debugging!)
>> ?endchoice
>>
>> +config BR2_STRIP_EXCLUDE_FILES
>> + ? ? ? string "executables that should not be stripped"
>> + ? ? ? depends on !BR2_STRIP_none
>> + ? ? ? default ""
>> + ? ? ? help
>> + ? ? ? ? You may specify a space-separated list of binaries and libraries
>> + ? ? ? ? here that should not be stripped on the target.
>> +
>> +config BR2_STRIP_EXCLUDE_DIRS
>> + ? ? ? string "directories that should be skipped when stripping"
>> + ? ? ? depends on !BR2_STRIP_none
>> + ? ? ? default ""
>> + ? ? ? help
>> + ? ? ? ? You may specify a space-separated list of directories that should
>> + ? ? ? ? be skipped when stripping. Binaries and libraries in these
>> + ? ? ? ? directories will not be touched.
>> + ? ? ? ? The directories should be specified relative to the target directory,
>> + ? ? ? ? without leading slash.
>> +
>> ?choice
>> ? ? ? ?prompt "gcc optimization level"
>> ? ? ? ?default BR2_OPTIMIZE_S
>> diff --git a/Makefile b/Makefile
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -438,6 +438,13 @@ endif
>> ?erase-fakeroots:
>> ? ? ? ?rm -f $(BUILD_DIR)/.fakeroot*
>>
>> +STRIP_FIND_CMD = find $(TARGET_DIR)
>> +ifneq (,$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS)))
>> +STRIP_FIND_CMD += \( $(call finddirclauses,$(TARGET_DIR),$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS))) \) -prune -o
>> +endif
>> +STRIP_FIND_CMD += -type f -perm +111
>> +STRIP_FIND_CMD += -not \( $(call findfileclauses,libthread_db*.so* $(call qstrip,$(BR2_STRIP_EXCLUDE_FILES))) \) -print
>> +
>> ?target-finalize:
>> ?ifeq ($(BR2_HAVE_DEVFILES),y)
>> ? ? ? ?( support/scripts/copy.sh $(STAGING_DIR) $(TARGET_DIR) )
>> @@ -462,8 +469,7 @@ endif
>> ?ifeq ($(BR2_PACKAGE_PYTHON_PYC_ONLY),y)
>> ? ? ? ?find $(TARGET_DIR)/usr/lib/ -name '*.py' -print0 | xargs -0 rm -f
>> ?endif
>> - ? ? ? find $(TARGET_DIR) -type f -perm +111 '!' -name 'libthread_db*.so*' | \
>> - ? ? ? ? ? ? ? xargs $(STRIPCMD) 2>/dev/null || true
>> + ? ? ? $(STRIP_FIND_CMD) | xargs $(STRIPCMD) 2>/dev/null || true
>> ? ? ? ?find $(TARGET_DIR)/lib/modules -type f -name '*.ko' | \
>> ? ? ? ? ? ? ? ?xargs -r $(KSTRIPCMD) || true
>>
>
> bump

bump again.

[Buildroot] [PATCH 1 of 2 v4-resend] Makefile.package.in: add utility functions find*clauses and notfirstword

[Thomas De Schampheleire]

On Fri, Mar 23, 2012 at 3:53 PM, Thomas De Schampheleire
<patrickdepinguin+buildroot@gmail.com> wrote:
> On Wed, Mar 14, 2012 at 5:45 PM, Thomas De Schampheleire
> <patrickdepinguin+buildroot@gmail.com> wrote:
>> This patch adds a few utility functions to Makefile.package.in.
>>
>> Functions finddirclauses and findfileclauses help in building a find command
>> that skips a set of directories and performs operations on a set of files.
>> This pattern can for example be used to keep certain files or directories from
>> being stripped, or to remove certain files from a package installation.
>>
>> The notfirstword function is the inverse of the 'firstword' function in make:
>> it returns all but the first word.
>>
>> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
>> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
>>
>> ---
>> v4: simplify find*clauses even further. Thanks, Arnout!
>> v3: use -path instead of -wholename; fix a bug in finddirclauses. Thanks, Cam.
>> v2: integrate _cont variants in main find*clauses functions. Thanks, Arnout.
>>
>> ?package/Makefile.package.in | ?10 ++++++++++
>> ?1 files changed, 10 insertions(+), 0 deletions(-)
>>
>> diff --git a/package/Makefile.package.in b/package/Makefile.package.in
>> --- a/package/Makefile.package.in
>> +++ b/package/Makefile.package.in
>> @@ -87,6 +87,16 @@ MESSAGE = echo "$(TERM_BOLD)>>> $($(PKG)
>> ?TERM_BOLD := $(shell tput smso)
>> ?TERM_RESET := $(shell tput rmso)
>>
>> +# Utility functions for 'find'
>> +# findfileclauses(filelist) => -name 'X' -o -name 'Y'
>> +findfileclauses = $(call notfirstword,$(patsubst %,-o -name '%',$(1)))
>> +# finddirclauses(base, dirlist) => -path 'base/dirX' -o -path 'base/dirY'
>> +finddirclauses = $(call notfirstword,$(patsubst %,-o -path '$(1)/%',$(2)))
>> +
>> +# Miscellaneous utility functions
>> +# notfirstword(wordlist): returns all but the first word in wordlist
>> +notfirstword = $(wordlist 2,$(words $(1)),$(1))
>> +
>> ?# Download method commands
>> ?WGET:=$(call qstrip,$(BR2_WGET)) $(QUIET)
>> ?SVN:=$(call qstrip,$(BR2_SVN))
>
> bump

bump again

[Buildroot] [PATCH 1 of 2 v4-resend] Makefile.package.in: add utility functions find*clauses and notfirstword

[Arnout Vandecappelle]

>>> diff --git a/package/Makefile.package.in b/package/Makefile.package.in

  Unfortunately, Makefile.package.in no longer exists...

  Regards,
  Arnout


-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 1 of 2 v4-resend] Makefile.package.in: add utility functions find*clauses and notfirstword

[Thomas De Schampheleire]

Op 4 mei 2012 22:28 schreef "Arnout Vandecappelle" <arnout@mind.be> het
volgende:
>>>>
>>>> diff --git a/package/Makefile.package.in b/package/Makefile.package.in
>
>
>  Unfortunately, Makefile.package.in no longer exists...
>

Dammit, frickin people that reorganize the code all the time! ;)

I should have checked that, will repost...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20120505/5efbb6c5/attachment.html>