[PATCH v2 0/3] Add support for relocatable SDK to ADT installer and some fixes

[PATCH v2 0/3] Add support for relocatable SDK to ADT installer and some fixes

[Laurentiu Palcu]

Changes in V2:
	- addressed Richard's comments and restored the spaces back to tabs in do_populate_adt
	  shell function;

Hi,

This patchset adds support for relocating SDK to ADT installer and a couple of
fixes.

Thanks,
Laurentiu

The following changes since commit af847d36375aa53f0c1ee2a00c97ba9f38837d1b:

  bitbake: bitbake: build.py: Add stampdir argument to cached_mtime_noerror (2012-08-16 12:27:41 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/relocatable_adt
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/relocatable_adt

Laurentiu Palcu (3):
  populate_sdk_base.bbclass: fix SDK relocation issues
  package.bbclass: change RPATHs for cross-canadian binaries
  adt-installer: add support for relocatable SDK

 meta/classes/package.bbclass                       |    2 +-
 meta/classes/populate_sdk_base.bbclass             |    4 +-
 .../installer/adt-installer/adt_installer          |   11 ++
 .../adt-installer/scripts/adt_installer_internal   |  111 ++++++++++++++++----
 .../installer/adt-installer_1.0.bb                 |    7 +-
 5 files changed, 111 insertions(+), 24 deletions(-)

-- 
1.7.9.5

[PATCH v2 1/3] populate_sdk_base.bbclass: fix SDK relocation issues

[Laurentiu Palcu]

The problem appears if multiple setup environment scripts are found. In
order to find only the script we're interested in, I removed globbing in
matching pattern with ${REAL_MULTIMACH_TARGET_SYS} that will be expanded
to the correct string.

Also, fix a problem when changing the scripts/configs. The grep pattern
matched also files that contained "text" in their name.

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 meta/classes/populate_sdk_base.bbclass |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index 0d19741..a1cb12a 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -154,7 +154,7 @@ echo "done"
 
 echo -n "Setting it up..."
 # fix environment paths
-env_setup_script=$(find $target_sdk_dir -name "environment-setup*")
+env_setup_script=$(find $target_sdk_dir -name "environment-setup-${REAL_MULTIMACH_TARGET_SYS}")
 sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
 
 # fix dynamic loader paths in all ELF SDK binaries
@@ -168,7 +168,7 @@ if [ $? -ne 0 ]; then
 fi
 
 # replace ${SDKPATH} with the new prefix in all text files: configs/scripts/etc
-find $native_sysroot -type f -exec file '{}' \;|grep text|cut -d':' -f1|xargs sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
+find $native_sysroot -type f -exec file '{}' \;|grep ":.*ASCII.*text"|cut -d':' -f1|xargs sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
 
 echo done
 
-- 
1.7.9.5

[PATCH v2 2/3] package.bbclass: change RPATHs for cross-canadian binaries

[Laurentiu Palcu]

When building the meta-toolchain, the binaries didn't get relocatable
RPATHs. They were hardcoded to the default path. Hence, if one had
already installed one SDK in the default path and one in another
location, the later toolchain's binaries would search and load libraries
from the first location, ending in a "Segmentation Fault".

[YOCTO #2927]

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 meta/classes/package.bbclass |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index d122cd9..278f0f0 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -448,7 +448,7 @@ python perform_packagecopy () {
     subprocess.call('tar -cf - -C %s -ps . | tar -xf - -C %s' % (dest, dvar), shell=True)
 
     # replace RPATHs for the nativesdk binaries, to make them relocatable
-    if bb.data.inherits_class('nativesdk', d):
+    if bb.data.inherits_class('nativesdk', d) or bb.data.inherits_class('cross-canadian', d):
         rpath_replace (dvar, d)
 }
 
-- 
1.7.9.5

[PATCH v2 3/3] adt-installer: add support for relocatable SDK

[Laurentiu Palcu]

Since we made the SDK relocatable, we have to add this functionality to
adt-installer too.

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 .../installer/adt-installer/adt_installer          |   11 ++
 .../adt-installer/scripts/adt_installer_internal   |  111 ++++++++++++++++----
 .../installer/adt-installer_1.0.bb                 |    7 +-
 3 files changed, 108 insertions(+), 21 deletions(-)

diff --git a/meta/recipes-devtools/installer/adt-installer/adt_installer b/meta/recipes-devtools/installer/adt-installer/adt_installer
index 1a53eb9..2d252b7 100755
--- a/meta/recipes-devtools/installer/adt-installer/adt_installer
+++ b/meta/recipes-devtools/installer/adt-installer/adt_installer
@@ -332,6 +332,17 @@ if [ -f "$YOCTOADT_INSTALL_LOG_FILE" ]; then
   rm $YOCTOADT_INSTALL_LOG_FILE
 fi
 
+echo -n "Please enter the install location (default: $DEFAULT_INSTALL_FOLDER): "
+read INSTALL_FOLDER
+
+if [ "$INSTALL_FOLDER" = "" ]; then
+    INSTALL_FOLDER=$DEFAULT_INSTALL_FOLDER
+fi
+
+eval INSTALL_FOLDER=$INSTALL_FOLDER
+export INSTALL_FOLDER=$(readlink -m $INSTALL_FOLDER)
+
+clear
 
 usage
 
diff --git a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
index 6201095..a540c0d 100755
--- a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
+++ b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
@@ -50,8 +50,8 @@ install_native_sdk()
 
 echo_info "\nStart installing selected native ADT for archs: $YOCTOADT_TARGETS..."
 
-# where the packages are installed. Currently only / is supported
-NATIVE_INSTALL_DIR="/"
+# where the packages are installed.
+NATIVE_INSTALL_DIR=$INSTALL_FOLDER
 
 if [ -d "$INSTALL_FOLDER" ]; then
   echo_info "\nNative ADT installation directory \"$INSTALL_FOLDER\" already exists! Continue installation will override its contents!"
@@ -59,28 +59,51 @@ if [ -d "$INSTALL_FOLDER" ]; then
 fi
 
 #Now begin to install native sdk and extract qemu rootfs which needs privilege rights
-echo_info "#######################################################################"
-echo_info "Please note from this point on installation requires sudo password ..."
-echo_info "#######################################################################"
-username='id -nu'
+#depending on the install location
+username=$(id -nu)
+
+# find the owner of the parent
+dir=$NATIVE_INSTALL_DIR
+while [ 1 ]; do
+    if [ -d $dir ]; then
+        owner=$(stat -c %U $dir)
+        break
+   else
+        dir=$(dirname $dir)
+   fi
+done
+
+if [ "$owner" = "$username" ]; then
+    SUDO=""
+else
+    echo_info "#######################################################################"
+    echo_info "Please note from this point on installation requires sudo password ..."
+    echo_info "#######################################################################"
+    SUDO=sudo
+fi
+
 #we need to make this directory firstly since opkg need to use it.
 OPKG_LOCK_DIR="$NATIVE_INSTALL_DIR/$OPKG_LIBDIR/opkg"
 if [ ! -d "$OPKG_LOCK_DIR" ]; then
-  sudo mkdir -p $OPKG_LOCK_DIR
+  $SUDO mkdir -p $OPKG_LOCK_DIR
   echo_info "Successfully create directory $OPKG_LOCK_DIR. "
 #if user delete /opt/xxx, while dangling folders there, report error
 elif [ ! -d "$INSTALL_FOLDER" ]; then
   echo_info "\nDangling opkg cache folder $OPKG_LOCK_DIR detected. Continue installation will remove the folder!"
   confirm_install $1
-  sudo rm -rf $OPKG_LOCK_DIR
-  sudo mkdir -p $OPKG_LOCK_DIR
+  $SUDO rm -rf $OPKG_LOCK_DIR
+  $SUDO mkdir -p $OPKG_LOCK_DIR
 #if user are updating installing, just let him/her go, give her/him prompt
 else
   echo_info "ADT has already been installed. Will update its contents..."
 fi
 
 #first update repository
-OPKG_CMD="sudo -E $LOCAL_OPKG_LOC/bin/opkg-cl"
+if [ "x$SUDO" = "x" ]; then
+    OPKG_CMD="$LOCAL_OPKG_LOC/bin/opkg-cl"
+else
+    OPKG_CMD="sudo -E $LOCAL_OPKG_LOC/bin/opkg-cl"
+fi
 
 echo_info "Updating opkg..."
 $OPKG_CMD -f $OPKG_CONFIG_FILE -o $NATIVE_INSTALL_DIR update &>> $YOCTOADT_INSTALL_LOG_FILE
@@ -117,13 +140,6 @@ for native_target_type in $YOCTOADT_TARGETS; do
 
 done
 
-# Link the ld.so.cache file into the hosts filesystem
-if [ ! -f "$OECORE_NATIVE_SYSROOT/etc/ld.so.cache" ]; then 
-echo_info "Link the ld.so.cache file into the host filesystem"
-sudo ln -s /etc/ld.so.cache $OECORE_NATIVE_SYSROOT/etc/ld.so.cache
-check_result
-fi
-
 if [ "$YOCTOADT_QEMU" == "Y" ] || [ "$YOCTOADT_QEMU" = "y" ]; then
   echo_info "\nInstalling qemu native ..."
   $OPKG_INSTALL_NATIVE_CMD qemu-nativesdk &>> $YOCTOADT_INSTALL_LOG_FILE
@@ -138,6 +154,65 @@ if [ "$YOCTOADT_NFS_UTIL" == "Y" ] || [ "$YOCTOADT_NFS_UTIL" == "y" ]; then
   check_result
 fi
 
+# Lose the ./opt/${DISTRO}/${SDK_VERSION} part, we don't really need to keep
+# the entire directory structure. We could patch opkg to do that but it's far
+# simpler to do that here and achieve the same result.
+# This is done in two steps:
+# Step 1: copy ./opt/${DISTRO}/${SDK_VERSION} contents to $NATIVE_INSTALL_DIR.
+# We cannot use move if $NATIVE_INSTALL_DIR is not empty (for example: contains
+# another SDK)
+$SUDO cp -r $NATIVE_INSTALL_DIR/$DEFAULT_INSTALL_FOLDER/* $NATIVE_INSTALL_DIR
+
+# delete the source directory now
+$SUDO rm -rf $NATIVE_INSTALL_DIR/$DEFAULT_INSTALL_FOLDER/*
+
+# Step 2: Delete the ./opt/${DISTRO}/${SDK_VERSION} directories too, they should be empty
+dir=$NATIVE_INSTALL_DIR/$DEFAULT_INSTALL_FOLDER
+while [ "$dir" != "$NATIVE_INSTALL_DIR" ]; do
+    # if the user chose / as the install folder, then we should leave /opt in place
+    if [ "$dir" = "/opt" ]; then
+        break
+    fi
+
+    # try to delete the directory, only if it's empty
+    $SUDO rmdir $dir
+    if [ $? -ne 0 ]; then
+        break
+    fi
+
+    # go to the next directory
+    dir=$(dirname $dir)
+done
+
+# Link the ld.so.cache file into the hosts filesystem
+if [ ! -f "$OECORE_NATIVE_SYSROOT/etc/ld.so.cache" ]; then
+echo_info "Link the ld.so.cache file into the host filesystem"
+$SUDO ln -s /etc/ld.so.cache $OECORE_NATIVE_SYSROOT/etc/ld.so.cache
+check_result
+fi
+
+# relocate binaries
+echo_info "\nRelocating binaries ..."
+escaped_sdkpath=$(echo $DEFAULT_INSTALL_FOLDER |sed -e "s:[\+\.]:\\\\\\\\\0:g")
+
+# We don't change the script in-place since we may want the user to re-run
+# adt-installer script
+$SUDO sed -e "s:##DEFAULT_INSTALL_DIR##:$escaped_sdkpath:" scripts/relocate_sdk.py > scripts/relocate_sdk_tmp.py
+$SUDO chmod +x scripts/relocate_sdk_tmp.py
+
+dl_path=$(find $OECORE_NATIVE_SYSROOT/lib -name "ld-linux*")
+executable_files=$(find $OECORE_NATIVE_SYSROOT -type f -perm +111)
+
+$SUDO scripts/relocate_sdk_tmp.py $INSTALL_FOLDER $dl_path $executable_files
+check_result
+
+# replace /opt/${DISTRO}/${SDK_VERSION} with the install folder in all configs
+env_setup_script=$(find $NATIVE_INSTALL_DIR -name "environment-setup-*")
+$SUDO sed -i -e "s:$DEFAULT_INSTALL_FOLDER:$NATIVE_INSTALL_DIR:g" $env_setup_script
+
+find $OECORE_NATIVE_SYSROOT -type f -exec file '{}' \;|grep ":.*ASCII.*text"|cut -d':' -f1|\
+    xargs $SUDO sed -i -e "s:$DEFAULT_INSTALL_FOLDER:$NATIVE_INSTALL_DIR:g"
+
 echo_info "\nSuccessfully installed selected native ADT!"
 }
 
@@ -180,7 +255,7 @@ else
 fi
 
 if [ ! -z "$env_filename" ]; then
-  sudo sed -i -e "s%##SDKTARGETSYSROOT##%$target_sysroot%g" $env_filename
+  $SUDO sed -i -e "s%##SDKTARGETSYSROOT##%$target_sysroot%g" $env_filename
 else
   echo_info "[ADT_INST] Error: Failed to find environment script for arch: $1"
   return 1 
diff --git a/meta/recipes-devtools/installer/adt-installer_1.0.bb b/meta/recipes-devtools/installer/adt-installer_1.0.bb
index e1edf2f..becdef6 100644
--- a/meta/recipes-devtools/installer/adt-installer_1.0.bb
+++ b/meta/recipes-devtools/installer/adt-installer_1.0.bb
@@ -30,7 +30,7 @@ ALLOW_EMPTY = "1"
 
 PACKAGES = ""
 
-PR = "r9"
+PR = "r10"
 
 ADT_DEPLOY = "${TMPDIR}/deploy/sdk/"
 ADT_DIR = "${WORKDIR}/adt-installer/"
@@ -69,8 +69,9 @@ fakeroot do_populate_adt () {
 	cp adt_installer ${ADT_DIR}
 	cp adt_installer.conf ${ADT_DIR}
 	sed -i -e 's#YOCTOADT_VERSION#${SDK_VERSION}#' ${ADT_DIR}/adt_installer.conf
-        echo 'SDK_VENDOR=${SDK_VENDOR}' >> ${ADT_DIR}/scripts/data_define
-        echo 'INSTALL_FOLDER=${SDKPATH}' >> ${ADT_DIR}/scripts/data_define
+	echo 'SDK_VENDOR=${SDK_VENDOR}' >> ${ADT_DIR}/scripts/data_define
+	echo 'DEFAULT_INSTALL_FOLDER=${SDKPATH}' >> ${ADT_DIR}/scripts/data_define
+	cp ${COREBASE}/scripts/relocate_sdk.py ${ADT_DIR}/scripts/
 	tar cfj adt_installer.tar.bz2 adt-installer
 	cp ${WORKDIR}/adt_installer.tar.bz2 ${ADT_DEPLOY}
 }
-- 
1.7.9.5

Re: [PATCH v2 0/3] Add support for relocatable SDK to ADT installer and some fixes

[Richard Purdie]

On Fri, 2012-08-17 at 13:38 +0300, Laurentiu Palcu wrote:
> Changes in V2:
> 	- addressed Richard's comments and restored the spaces back to tabs in do_populate_adt
> 	  shell function;
> 
> Hi,
> 
> This patchset adds support for relocating SDK to ADT installer and a couple of
> fixes.
> 
> Thanks,
> Laurentiu
> 
> The following changes since commit af847d36375aa53f0c1ee2a00c97ba9f38837d1b:
> 
>   bitbake: bitbake: build.py: Add stampdir argument to cached_mtime_noerror (2012-08-16 12:27:41 +0100)
> 
> are available in the git repository at:
> 
>   git://git.yoctoproject.org/poky-contrib lpalcu/relocatable_adt
>   http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/relocatable_adt
> 
> Laurentiu Palcu (3):
>   populate_sdk_base.bbclass: fix SDK relocation issues
>   package.bbclass: change RPATHs for cross-canadian binaries
>   adt-installer: add support for relocatable SDK

Merged to master, thanks.

Richard