[yocto 2.1] [PATCH 0/1] toolchain-shar-relocate.sh: make it faster

[yocto 2.1] [PATCH 0/1] toolchain-shar-relocate.sh: make it faster

[Robert Yang]

The following changes since commit d5cf21179d9f8d3c053316b0864d72fc609f5423:

  gdk-pixbuf: Avoid rebuild failures (2015-09-24 17:53:25 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib rbt/sdk_time
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/sdk_time

Robert Yang (1):
  toolchain-shar-relocate.sh: make it faster

 meta/files/toolchain-shar-relocate.sh |   24 ++++++++++--------------
 1 file changed, 10 insertions(+), 14 deletions(-)

-- 
1.7.9.5

[PATCH 1/1] toolchain-shar-relocate.sh: make it faster

[Robert Yang]

Make the extrating faster by:
* Merge the two heavy t"for .. find" loops into one
* Move the commands out of for loop rather than inside, this can reduce the
  forking amount.

As a result, when install:
* buildtools-nativesdk-standalone: 14s -> 7s (50% saved)
* core-image-minimal-core2-64-toolchain: 56s -> 47s (17% saved)

[YOCTO #8404]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/files/toolchain-shar-relocate.sh |   24 ++++++++++--------------
 1 file changed, 10 insertions(+), 14 deletions(-)

diff --git a/meta/files/toolchain-shar-relocate.sh b/meta/files/toolchain-shar-relocate.sh
index dfb8e16..4ef2927 100644
--- a/meta/files/toolchain-shar-relocate.sh
+++ b/meta/files/toolchain-shar-relocate.sh
@@ -26,25 +26,21 @@ if [ $relocate = 1 ] ; then
 	fi
 fi
 
-# replace @SDKPATH@ with the new prefix in all text files: configs/scripts/etc
+# replace @SDKPATH@ with the new prefix in all text files: configs/scripts/etc.
+# replace the host perl with SDK perl.
 for replace in "$target_sdk_dir -maxdepth 1" "$native_sysroot"; do
-	$SUDO_EXEC find $replace -type f -exec file '{}' \; | \
-		grep ":.*\(ASCII\|script\|source\).*text" | \
-		awk -F':' '{printf "\"%s\"\n", $1}' | \
-		grep -v "$target_sdk_dir/environment-setup-*" | \
-		$SUDO_EXEC xargs -n32 sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
-done
+	$SUDO_EXEC find $replace -type f
+done | xargs -n100 file | grep ":.*\(ASCII\|script\|source\).*text" | \
+    awk -F':' '{printf "\"%s\"\n", $1}' | \
+    grep -v "$target_sdk_dir/environment-setup-*" | \
+    xargs -n100 $SUDO_EXEC sed -i \
+        -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" \
+        -e "s:^#! */usr/bin/perl.*:#! /usr/bin/env perl:g" \
+        -e "s: /usr/bin/perl: /usr/bin/env perl:g"
 
 # change all symlinks pointing to @SDKPATH@
 for l in $($SUDO_EXEC find $native_sysroot -type l); do
 	$SUDO_EXEC ln -sfn $(readlink $l|$SUDO_EXEC sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:") $l
 done
 
-# find out all perl scripts in $native_sysroot and modify them replacing the
-# host perl with SDK perl.
-for perl_script in $($SUDO_EXEC find $native_sysroot -type f -exec grep -l "^#!.*perl" '{}' \;); do
-	$SUDO_EXEC sed -i -e "s:^#! */usr/bin/perl.*:#! /usr/bin/env perl:g" -e \
-		"s: /usr/bin/perl: /usr/bin/env perl:g" $perl_script
-done
-
 echo done
-- 
1.7.9.5

Re: [PATCH 1/1] toolchain-shar-relocate.sh: make it faster

[Robert Yang]

On 09/28/2015 03:09 PM, Robert Yang wrote:
> Make the extrating faster by:
> * Merge the two heavy t"for .. find" loops into one

Sorry, a typo, should be no "t", updated in the repo.

// Robert

> * Move the commands out of for loop rather than inside, this can reduce the
>    forking amount.
>
> As a result, when install:
> * buildtools-nativesdk-standalone: 14s -> 7s (50% saved)
> * core-image-minimal-core2-64-toolchain: 56s -> 47s (17% saved)
>
> [YOCTO #8404]
>
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>   meta/files/toolchain-shar-relocate.sh |   24 ++++++++++--------------
>   1 file changed, 10 insertions(+), 14 deletions(-)
>
> diff --git a/meta/files/toolchain-shar-relocate.sh b/meta/files/toolchain-shar-relocate.sh
> index dfb8e16..4ef2927 100644
> --- a/meta/files/toolchain-shar-relocate.sh
> +++ b/meta/files/toolchain-shar-relocate.sh
> @@ -26,25 +26,21 @@ if [ $relocate = 1 ] ; then
>   	fi
>   fi
>
> -# replace @SDKPATH@ with the new prefix in all text files: configs/scripts/etc
> +# replace @SDKPATH@ with the new prefix in all text files: configs/scripts/etc.
> +# replace the host perl with SDK perl.
>   for replace in "$target_sdk_dir -maxdepth 1" "$native_sysroot"; do
> -	$SUDO_EXEC find $replace -type f -exec file '{}' \; | \
> -		grep ":.*\(ASCII\|script\|source\).*text" | \
> -		awk -F':' '{printf "\"%s\"\n", $1}' | \
> -		grep -v "$target_sdk_dir/environment-setup-*" | \
> -		$SUDO_EXEC xargs -n32 sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
> -done
> +	$SUDO_EXEC find $replace -type f
> +done | xargs -n100 file | grep ":.*\(ASCII\|script\|source\).*text" | \
> +    awk -F':' '{printf "\"%s\"\n", $1}' | \
> +    grep -v "$target_sdk_dir/environment-setup-*" | \
> +    xargs -n100 $SUDO_EXEC sed -i \
> +        -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" \
> +        -e "s:^#! */usr/bin/perl.*:#! /usr/bin/env perl:g" \
> +        -e "s: /usr/bin/perl: /usr/bin/env perl:g"
>
>   # change all symlinks pointing to @SDKPATH@
>   for l in $($SUDO_EXEC find $native_sysroot -type l); do
>   	$SUDO_EXEC ln -sfn $(readlink $l|$SUDO_EXEC sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:") $l
>   done
>
> -# find out all perl scripts in $native_sysroot and modify them replacing the
> -# host perl with SDK perl.
> -for perl_script in $($SUDO_EXEC find $native_sysroot -type f -exec grep -l "^#!.*perl" '{}' \;); do
> -	$SUDO_EXEC sed -i -e "s:^#! */usr/bin/perl.*:#! /usr/bin/env perl:g" -e \
> -		"s: /usr/bin/perl: /usr/bin/env perl:g" $perl_script
> -done
> -
>   echo done
>