[PATCH 0/4] scripts: rss fixes for oe-find-sysroot and oe-run-native

[PATCH 0/4] scripts: rss fixes for oe-find-sysroot and oe-run-native

[brian avery]

This patch set does a couple of things:

1) revert brian's runqemu-extract-sdk patch and add in Robert's more generically useful patch.
2) Correct the suggestion oe-find-native gives if bitbake meta-ide-support has not been run
3) fix a /bin/sh to bin/bash since the script used bashisms and was failing on my dash system and worked

Partial fix for bugs 11170 && 11119, Ed's patch set required to finish off the bugs.

-brian

The following changes since commit 94467d5087ca3b13e7baac670af107550701b333:

  dev-manual, ref-manual: Edits to the "Incrementing Binary Package" section. (2017-04-12 19:35:00 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib bavery/ed/oe-core/rm_work-runqemu-bavery
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=bavery/ed/oe-core/rm_work-runqemu-bavery

Ed Bartosh (1):
  Revert "scripts: change way we find native tools (pseudo)"

Robert Yang (1):
  oe-find-native-sysroot: work with RSS

brian avery (2):
  oe-find-native-sysroot: add appopriate suggestion
  oe-run-native: explicitly use bash

 scripts/oe-find-native-sysroot | 59 ++++++++++++++++++++++++++++--------------
 scripts/oe-run-native          | 17 +++++++-----
 scripts/runqemu-export-rootfs  |  2 +-
 scripts/runqemu-extract-sdk    |  2 +-
 4 files changed, 53 insertions(+), 27 deletions(-)

--
1.9.1

[PATCH 1/4] Revert "scripts: change way we find native tools (pseudo)"

[brian avery]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

This reverts commit f200f37699031cd98f4594b2992e6b0d8f753440.

This reverts the patch that fixed runqemu-extract-sdk. It failed
to fix other issues in the script/tools that were introduced by RSS. The
following patch from Robert Yang fixes both.  Therefore, reverting this
patch in favor of his.

Signed-off-by: brian avery <brian.avery@intel.com>
---
 scripts/oe-find-native-sysroot | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/oe-find-native-sysroot b/scripts/oe-find-native-sysroot
index 3f4c29d..13a5c46 100755
--- a/scripts/oe-find-native-sysroot
+++ b/scripts/oe-find-native-sysroot
@@ -2,7 +2,7 @@
 #
 # Find a native sysroot to use - either from an in-tree OE build or
 # from a toolchain installation. It then ensures the variable
-# $OECORE_NATIVE_SYSROOT is set to the sysroot's base directory, and sets
+# $OECORE_NATIVE_SYSROOT is set to the sysroot's base directory, and sets 
 # $PSEUDO to the path of the pseudo binary.
 #
 # This script is intended to be run within other scripts by source'ing
@@ -54,10 +54,10 @@ if [ "x$OECORE_NATIVE_SYSROOT" = "x" ]; then
                 exit 1
             fi
             touch conf/sanity.conf
-            OECORE_NATIVE_SYSROOT=`bitbake -e meta-ide-support | grep ^RECIPE_SYSROOT_NATIVE | cut -d '"' -f2`
+            OECORE_NATIVE_SYSROOT=`bitbake -e | grep ^STAGING_DIR_NATIVE | cut -d '"' -f2`
             rm -f conf/sanity.conf
         else
-            OECORE_NATIVE_SYSROOT=`bitbake -e meta-ide-support | grep ^RECIPE_SYSROOT_NATIVE | cut -d '"' -f2`
+            OECORE_NATIVE_SYSROOT=`bitbake -e | grep ^STAGING_DIR_NATIVE | cut -d '"' -f2`
         fi
     else
         echo "Error: Unable to locate bitbake command."
@@ -72,7 +72,7 @@ fi
 if [ "x$OECORE_NATIVE_SYSROOT" = "x" ]; then
     # This indicates that there was an error running bitbake -e that
     # the user needs to be informed of
-    echo "There was an error running bitbake to determine RECIPE_SYSROOT_NATIVE"
+    echo "There was an error running bitbake to determine STAGING_DIR_NATIVE"
     echo "Here is the output from bitbake -e"
     bitbake -e
     exit 1
-- 
1.9.1

[PATCH 2/4] oe-find-native-sysroot: work with RSS

[brian avery]

From: Robert Yang <liezhi.yang@windriver.com>

The generic STAGING_DIR_NATIVE is gone since RSS, so when find
OECORE_NATIVE_SYSROOT, the user has to specify which recipe's
STAGING_DIR_NATIVE will be used as OECORE_NATIVE_SYSROOT.

* The usage is changed from ". oe-find-native-sysroot" to
  ". oe-find-native-sysroot <recipe>".
* The oe-run-native's usage has changed from
  "oe-run-native tool" to "oe-run-native native-recipe tool".

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 scripts/oe-find-native-sysroot | 56 ++++++++++++++++++++++++++++--------------
 scripts/oe-run-native          | 15 +++++++----
 scripts/runqemu-export-rootfs  |  2 +-
 scripts/runqemu-extract-sdk    |  2 +-
 4 files changed, 49 insertions(+), 26 deletions(-)

diff --git a/scripts/oe-find-native-sysroot b/scripts/oe-find-native-sysroot
index 13a5c46..ad7b2d1 100755
--- a/scripts/oe-find-native-sysroot
+++ b/scripts/oe-find-native-sysroot
@@ -9,7 +9,7 @@
 # it, e.g:
 #
 # SYSROOT_SETUP_SCRIPT=`which oe-find-native-sysroot`
-# . $SYSROOT_SETUP_SCRIPT
+# . $SYSROOT_SETUP_SCRIPT <recipe>
 #
 # This script will terminate execution of your calling program unless
 # you set a variable $SKIP_STRICT_SYSROOT_CHECK to a non-empty string
@@ -30,8 +30,8 @@
 # with this program; if not, write to the Free Software Foundation, Inc.,
 # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
 
-if [ "$1" = '--help' -o "$1" = '-h' ] ; then
-    echo 'Usage: oe-find-native-sysroot [-h|--help]'
+if [ "$1" = '--help' -o "$1" = '-h' -o $# -ne 1 ] ; then
+    echo 'Usage: oe-find-native-sysroot <recipe> [-h|--help]'
     echo ''
     echo 'OpenEmbedded find-native-sysroot - helper script to set'
     echo 'environment variables OECORE_NATIVE_SYSROOT and PSEUDO'
@@ -39,11 +39,31 @@ if [ "$1" = '--help' -o "$1" = '-h' ] ; then
     echo 'executable binary'
     echo ''
     echo 'options:'
+    echo '  recipe              its STAGING_DIR_NATIVE is used as native sysroot'
     echo '  -h, --help          show this help message and exit'
     echo ''
     exit 2
 fi
 
+# Global vars
+BITBAKE_E=""
+OECORE_NATIVE_SYSROOT=""
+
+set_oe_native_sysroot(){
+    echo "Running bitbake -e $1"
+    BITBAKE_E="`bitbake -e $1`"
+    OECORE_NATIVE_SYSROOT=`echo "$BITBAKE_E" | grep ^STAGING_DIR_NATIVE | cut -d '"' -f2`
+
+    if [ "x$OECORE_NATIVE_SYSROOT" = "x" ]; then
+        # This indicates that there was an error running bitbake -e that
+        # the user needs to be informed of
+        echo "There was an error running bitbake to determine STAGING_DIR_NATIVE"
+        echo "Here is the output from bitbake -e $1"
+        echo $BITBAKE_E
+        exit 1
+    fi
+}
+
 if [ "x$OECORE_NATIVE_SYSROOT" = "x" ]; then
     BITBAKE=`which bitbake 2> /dev/null`
     if [ "x$BITBAKE" != "x" ]; then
@@ -54,10 +74,10 @@ if [ "x$OECORE_NATIVE_SYSROOT" = "x" ]; then
                 exit 1
             fi
             touch conf/sanity.conf
-            OECORE_NATIVE_SYSROOT=`bitbake -e | grep ^STAGING_DIR_NATIVE | cut -d '"' -f2`
+            set_oe_native_sysroot $1
             rm -f conf/sanity.conf
         else
-            OECORE_NATIVE_SYSROOT=`bitbake -e | grep ^STAGING_DIR_NATIVE | cut -d '"' -f2`
+            set_oe_native_sysroot $1
         fi
     else
         echo "Error: Unable to locate bitbake command."
@@ -69,21 +89,11 @@ if [ "x$OECORE_NATIVE_SYSROOT" = "x" ]; then
     fi
 fi
 
-if [ "x$OECORE_NATIVE_SYSROOT" = "x" ]; then
-    # This indicates that there was an error running bitbake -e that
-    # the user needs to be informed of
-    echo "There was an error running bitbake to determine STAGING_DIR_NATIVE"
-    echo "Here is the output from bitbake -e"
-    bitbake -e
-    exit 1
-fi
-
-# Set up pseudo command
-if [ ! -e "$OECORE_NATIVE_SYSROOT/usr/bin/pseudo" ]; then
-    echo "Error: Unable to find pseudo binary in $OECORE_NATIVE_SYSROOT/usr/bin/"
+if [ ! -e "$OECORE_NATIVE_SYSROOT/" ]; then
+    echo "Error: $OECORE_NATIVE_SYSROOT doesn't exist."
 
     if [ "x$OECORE_DISTRO_VERSION" = "x" ]; then
-        echo "Have you run 'bitbake meta-ide-support'?"
+        echo "Have you run 'bitbake $1 -caddto_recipe_sysroot'?"
     else
         echo "This shouldn't happen - something is wrong with your toolchain installation"
     fi
@@ -92,4 +102,12 @@ if [ ! -e "$OECORE_NATIVE_SYSROOT/usr/bin/pseudo" ]; then
         exit 1
     fi
 fi
-PSEUDO="$OECORE_NATIVE_SYSROOT/usr/bin/pseudo"
+
+# Set up pseudo command
+pseudo="$OECORE_NATIVE_SYSROOT/usr/bin/pseudo"
+if [ -e "$pseudo" ]; then
+    echo "PSEUDO=$pseudo"
+    PSEUDO="$pseudo"
+else
+    echo "PSEUDO $pseudo is not found."
+fi
diff --git a/scripts/oe-run-native b/scripts/oe-run-native
index f2453e7..de3c02e 100755
--- a/scripts/oe-run-native
+++ b/scripts/oe-run-native
@@ -22,34 +22,39 @@
 #
 
 if [ $# -lt 1 -o "$1" = '--help' -o "$1" = '-h' ] ; then
-    echo 'oe-run-native: error: the following arguments are required: <native tool>'
-    echo 'Usage: oe-run-native tool [parameters]'
+    echo 'oe-run-native: error: the following arguments are required: <native recipe> <native tool>'
+    echo 'Usage: oe-run-native native-recipe tool [parameters]'
     echo ''
     echo 'OpenEmbedded run-native - runs native tools'
     echo ''
     echo 'arguments:'
+    echo '  native-recipe       The recipe which provoides tool'
     echo '  tool                Native tool to run'
     echo ''
     exit 2
 fi
 
+native_recipe="$1"
+tool="$2"
+shift
+
 SYSROOT_SETUP_SCRIPT=`which oe-find-native-sysroot 2> /dev/null`
 if [ -z "$SYSROOT_SETUP_SCRIPT" ]; then
         echo "Error: Unable to find oe-find-native-sysroot script"
         exit 1
 fi
-. $SYSROOT_SETUP_SCRIPT
+. $SYSROOT_SETUP_SCRIPT $native_recipe
 
 OLD_PATH=$PATH
 
 # look for a tool only in native sysroot
 PATH=$OECORE_NATIVE_SYSROOT/usr/bin:$OECORE_NATIVE_SYSROOT/bin:$OECORE_NATIVE_SYSROOT/usr/sbin:$OECORE_NATIVE_SYSROOT/sbin
-tool=`/usr/bin/which $1 2>/dev/null`
+tool=`/usr/bin/which $tool 2>/dev/null`
 
 if [ -n "$tool" ] ; then
     # add old path to allow usage of host tools
     PATH=$PATH:$OLD_PATH $@
 else
-    echo "Error: Unable to find '$1' in native sysroot"
+    echo "Error: Unable to find '$tool' in $PATH"
     exit 1
 fi
diff --git a/scripts/runqemu-export-rootfs b/scripts/runqemu-export-rootfs
index 7ebc071..c7992d8 100755
--- a/scripts/runqemu-export-rootfs
+++ b/scripts/runqemu-export-rootfs
@@ -44,7 +44,7 @@ if [ -z "$SYSROOT_SETUP_SCRIPT" ]; then
 	echo "Did you forget to source your build environment setup script?"
 	exit 1
 fi
-. $SYSROOT_SETUP_SCRIPT
+. $SYSROOT_SETUP_SCRIPT meta-ide-support
 
 if [ ! -e "$OECORE_NATIVE_SYSROOT/usr/bin/unfsd" ]; then
 	echo "Error: Unable to find unfsd binary in $OECORE_NATIVE_SYSROOT/usr/bin/"
diff --git a/scripts/runqemu-extract-sdk b/scripts/runqemu-extract-sdk
index 32ddd48..2a0dd50 100755
--- a/scripts/runqemu-extract-sdk
+++ b/scripts/runqemu-extract-sdk
@@ -35,7 +35,7 @@ if [ -z "$SYSROOT_SETUP_SCRIPT" ]; then
     echo "Did you forget to source your build system environment setup script?"
     exit 1
 fi
-. $SYSROOT_SETUP_SCRIPT
+. $SYSROOT_SETUP_SCRIPT meta-ide-support
 PSEUDO_OPTS="-P $OECORE_NATIVE_SYSROOT/usr"
 
 ROOTFS_TARBALL=$1
-- 
1.9.1

[PATCH 3/4] oe-find-native-sysroot: add appopriate suggestion

[brian avery]

right now, if it fails, the script tells the user to run bitbake foo
-caddto_recipe_sysroot. This works for native recipes but not things
like meta-ide. This patch checks whether the recipe is native and gives
out the appopriate warning.

Signed-off-by: brian avery <brian.avery@intel.com>
---
 scripts/oe-find-native-sysroot | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/scripts/oe-find-native-sysroot b/scripts/oe-find-native-sysroot
index ad7b2d1..5c8681e 100755
--- a/scripts/oe-find-native-sysroot
+++ b/scripts/oe-find-native-sysroot
@@ -2,7 +2,7 @@
 #
 # Find a native sysroot to use - either from an in-tree OE build or
 # from a toolchain installation. It then ensures the variable
-# $OECORE_NATIVE_SYSROOT is set to the sysroot's base directory, and sets 
+# $OECORE_NATIVE_SYSROOT is set to the sysroot's base directory, and sets
 # $PSEUDO to the path of the pseudo binary.
 #
 # This script is intended to be run within other scripts by source'ing
@@ -48,7 +48,6 @@ fi
 # Global vars
 BITBAKE_E=""
 OECORE_NATIVE_SYSROOT=""
-
 set_oe_native_sysroot(){
     echo "Running bitbake -e $1"
     BITBAKE_E="`bitbake -e $1`"
@@ -93,7 +92,11 @@ if [ ! -e "$OECORE_NATIVE_SYSROOT/" ]; then
     echo "Error: $OECORE_NATIVE_SYSROOT doesn't exist."
 
     if [ "x$OECORE_DISTRO_VERSION" = "x" ]; then
-        echo "Have you run 'bitbake $1 -caddto_recipe_sysroot'?"
+        if [[ $1 =~ .*native.* ]]; then
+            echo "Have you run 'bitbake $1 -caddto_recipe_sysroot'?"
+        else
+            echo "Have you run 'bitbake $1 '?"
+        fi
     else
         echo "This shouldn't happen - something is wrong with your toolchain installation"
     fi
-- 
1.9.1

[PATCH 4/4] oe-run-native: explicitly use bash

[brian avery]

This script sources another script (oe-find-native-sysroot) with
arguments.  It was using /bin/sh. Sourcing with arguments works only in
bash so it was failing in dash.  This commit makes it dash proof.

Signed-off-by: brian avery <brian.avery@intel.com>
---
 scripts/oe-run-native | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/oe-run-native b/scripts/oe-run-native
index de3c02e..12f7ce6 100755
--- a/scripts/oe-run-native
+++ b/scripts/oe-run-native
@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/bin/bash
 #
 # Copyright (c) 2016,  Intel Corporation.
 # All Rights Reserved
-- 
1.9.1