[PATCH v2 0/6] SDK and adt_installer fixes

[PATCH v2 0/6] SDK and adt_installer fixes

[Laurentiu Palcu]

Changes in v2:
Since the previous patchset was not merged I took the liberty of adding
another 2 fixes and send another patchset so that they're merged in the
right order.

Changes in v1:

Hi,

This patchset contains some SDK/adt-installer fixes.

Thanks,
Laurentiu

The following changes since commit 913944d904266bf90af0cad94b4f0fb3652bd29d:

  upstream_tracking: update lsb and ltp (2012-09-14 17:12:52 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/adt_sdk_fixes
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/adt_sdk_fixes

Laurentiu Palcu (6):
  adt-installer: fix package installation issue
  SDK: fix installation into symlinked directories
  adt-installer: ensure directory exists before copying/removing
  SDK: remove references to Poky distro from tarball installer
  SDK: relocate symlinks too
  SDK: allow toolchain installation from another directory

 meta/classes/populate_sdk_base.bbclass             |   21 +++++--
 .../installer/adt-installer/adt_installer          |    6 +-
 .../adt-installer/scripts/adt_installer_internal   |   61 +++++++++++---------
 3 files changed, 54 insertions(+), 34 deletions(-)

-- 
1.7.9.5

[PATCH v2 1/6] adt-installer: fix package installation issue

[Laurentiu Palcu]

When the cross canadian toolchains are installed, for different
architectures, they might contain common files. This leads to
installation failures since the opkg, by default, does not overwrite
files. This issue happens, for example, for binutils packages (that
contain the same locale files) or gdb (which installs some syscalls xml
files). The locale files could be removed from the binutils
cross-canadian package but we cannot do the same for the syscalls GDB
files which are used by GDB to display user friendly names for the
syscall numbers. Hence, the best solution is to force opkg to overwrite
these files.

[YOCTO #3109]

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 .../adt-installer/scripts/adt_installer_internal   |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
index 78ea6d0..f113aa4 100755
--- a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
+++ b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
@@ -112,7 +112,7 @@ check_result
 
 #install below must sdk-host packages
 OPKG_INSTALL_CMD="$OPKG_CMD "
-OPKG_INSTALL_NATIVE_CMD="$OPKG_INSTALL_CMD  -f $OPKG_CONFIG_FILE -o $NATIVE_INSTALL_DIR install"
+OPKG_INSTALL_NATIVE_CMD="$OPKG_INSTALL_CMD  --force-overwrite -f $OPKG_CONFIG_FILE -o $NATIVE_INSTALL_DIR install"
 
 BASE_HOSTSDK_PKGNAMES="pseudo opkg pkgconfig libtool autoconf automake"
 for pkg in $BASE_HOSTSDK_PKGNAMES; do
-- 
1.7.9.5

[PATCH v2 2/6] SDK: fix installation into symlinked directories

[Laurentiu Palcu]

The SDK installation scripts should not canonicalize symlinked
directories because the entire relocation would be done to the directory
to which the symlink points. Instead, if the installation is a symlink,
use that path to relocate the binaries.

For example, if we have the following symlink: /opt/sdk -> ~/my/test/sdk
the binaries will be relocated to /opt/sdk not ~/my/test/sdk as it is
done now.

[YOCTO #3102]

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 meta/classes/populate_sdk_base.bbclass             |    8 ++++++--
 .../installer/adt-installer/adt_installer          |    6 +++++-
 .../adt-installer/scripts/adt_installer_internal   |    2 +-
 3 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index 1c151d7..78669cf 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -126,7 +126,11 @@ if [ "$target_sdk_dir" = "" ]; then
 fi
 
 eval target_sdk_dir=$target_sdk_dir
-target_sdk_dir=$(readlink -m $target_sdk_dir)
+if [ -d $target_sdk_dir ]; then
+	target_sdk_dir=$(cd $target_sdk_dir; pwd)
+else
+	target_sdk_dir=$(readlink -m $target_sdk_dir)
+fi
 
 echo -n "You are about to install Poky SDK to \"$target_sdk_dir\". Proceed[Y/n]?"
 read answer
@@ -154,7 +158,7 @@ echo "done"
 
 echo -n "Setting it up..."
 # fix environment paths
-env_setup_script=$(find $target_sdk_dir -name "environment-setup-${REAL_MULTIMACH_TARGET_SYS}")
+env_setup_script=$(find $target_sdk_dir/ -name "environment-setup-${REAL_MULTIMACH_TARGET_SYS}")
 sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
 
 # fix dynamic loader paths in all ELF SDK binaries
diff --git a/meta/recipes-devtools/installer/adt-installer/adt_installer b/meta/recipes-devtools/installer/adt-installer/adt_installer
index 2d252b7..0be5a1d 100755
--- a/meta/recipes-devtools/installer/adt-installer/adt_installer
+++ b/meta/recipes-devtools/installer/adt-installer/adt_installer
@@ -340,7 +340,11 @@ if [ "$INSTALL_FOLDER" = "" ]; then
 fi
 
 eval INSTALL_FOLDER=$INSTALL_FOLDER
-export INSTALL_FOLDER=$(readlink -m $INSTALL_FOLDER)
+if [ -d $INSTALL_FOLDER ]; then
+	export INSTALL_FOLDER=$(cd $INSTALL_FOLDER; pwd)
+else
+	export INSTALL_FOLDER=$(readlink -m $INSTALL_FOLDER)
+fi
 
 clear
 
diff --git a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
index f113aa4..76acaa7 100755
--- a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
+++ b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
@@ -207,7 +207,7 @@ $SUDO scripts/relocate_sdk_tmp.py $INSTALL_FOLDER $dl_path $executable_files
 check_result
 
 # replace /opt/${DISTRO}/${SDK_VERSION} with the install folder in all configs
-env_setup_script=$(find $NATIVE_INSTALL_DIR -name "environment-setup-*")
+env_setup_script=$(find $NATIVE_INSTALL_DIR/ -name "environment-setup-*")
 $SUDO sed -i -e "s:$DEFAULT_INSTALL_FOLDER:$NATIVE_INSTALL_DIR:g" $env_setup_script
 
 find $OECORE_NATIVE_SYSROOT -type f -exec file '{}' \;|grep ":.*ASCII.*text"|cut -d':' -f1|\
-- 
1.7.9.5

[PATCH v2 3/6] adt-installer: ensure directory exists before copying/removing

[Laurentiu Palcu]

If the installation is done in a directory which already contains a
valid installation, opkg will not install anything and the moving the
contents of /install/dir/opt/poky/1.2 (for example) to /install/dir will
throw some errors. However, the install directory will not be affected.
This patch will ensure that the /install/dir/opt/poky/1.2 exists.

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 .../adt-installer/scripts/adt_installer_internal   |   52 ++++++++++----------
 1 file changed, 27 insertions(+), 25 deletions(-)

diff --git a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
index 76acaa7..fbcd2ce 100755
--- a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
+++ b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
@@ -158,31 +158,33 @@ fi
 # the entire directory structure. We could patch opkg to do that but it's far
 # simpler to do that here and achieve the same result.
 # This is done in two steps:
-# Step 1: copy ./opt/${DISTRO}/${SDK_VERSION} contents to $NATIVE_INSTALL_DIR.
-# We cannot use move if $NATIVE_INSTALL_DIR is not empty (for example: contains
-# another SDK)
-$SUDO cp -r $NATIVE_INSTALL_DIR/$DEFAULT_INSTALL_FOLDER/* $NATIVE_INSTALL_DIR
-
-# delete the source directory now
-$SUDO rm -rf $NATIVE_INSTALL_DIR/$DEFAULT_INSTALL_FOLDER/*
-
-# Step 2: Delete the ./opt/${DISTRO}/${SDK_VERSION} directories too, they should be empty
-dir=$NATIVE_INSTALL_DIR/$DEFAULT_INSTALL_FOLDER
-while [ "$dir" != "$NATIVE_INSTALL_DIR" ]; do
-    # if the user chose / as the install folder, then we should leave /opt in place
-    if [ "$dir" = "/opt" ]; then
-        break
-    fi
-
-    # try to delete the directory, only if it's empty
-    $SUDO rmdir $dir
-    if [ $? -ne 0 ]; then
-        break
-    fi
-
-    # go to the next directory
-    dir=$(dirname $dir)
-done
+if [ -d $NATIVE_INSTALL_DIR/$DEFAULT_INSTALL_FOLDER ]; then
+    # Step 1: copy ./opt/${DISTRO}/${SDK_VERSION} contents to $NATIVE_INSTALL_DIR.
+    # We cannot use move if $NATIVE_INSTALL_DIR is not empty (for example: contains
+    # another SDK)
+    $SUDO cp -r $NATIVE_INSTALL_DIR/$DEFAULT_INSTALL_FOLDER/* $NATIVE_INSTALL_DIR
+
+    # delete the source directory now
+    $SUDO rm -rf $NATIVE_INSTALL_DIR/$DEFAULT_INSTALL_FOLDER/*
+
+    # Step 2: Delete the ./opt/${DISTRO}/${SDK_VERSION} directories too, they should be empty
+    dir=$NATIVE_INSTALL_DIR/$DEFAULT_INSTALL_FOLDER
+    while [ "$dir" != "$NATIVE_INSTALL_DIR" ]; do
+        # if the user chose / as the install folder, then we should leave /opt in place
+        if [ "$dir" = "/opt" ]; then
+            break
+        fi
+
+        # try to delete the directory, only if it's empty
+        $SUDO rmdir $dir
+        if [ $? -ne 0 ]; then
+            break
+        fi
+
+        # go to the next directory
+        dir=$(dirname $dir)
+    done
+fi
 
 # Link the ld.so.cache file into the hosts filesystem
 if [ ! -f "$OECORE_NATIVE_SYSROOT/etc/ld.so.cache" ]; then
-- 
1.7.9.5

[PATCH v2 4/6] SDK: remove references to Poky distro from tarball installer

[Laurentiu Palcu]

The installer should be generic.

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 meta/classes/populate_sdk_base.bbclass |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index 78669cf..f20a6ac 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -118,7 +118,7 @@ fakeroot create_shar() {
 
 DEFAULT_INSTALL_DIR="${SDKPATH}"
 
-echo -n "Enter target directory for Poky SDK (default: $DEFAULT_INSTALL_DIR): "
+echo -n "Enter target directory for SDK (default: $DEFAULT_INSTALL_DIR): "
 read target_sdk_dir
 
 if [ "$target_sdk_dir" = "" ]; then
@@ -132,7 +132,7 @@ else
 	target_sdk_dir=$(readlink -m $target_sdk_dir)
 fi
 
-echo -n "You are about to install Poky SDK to \"$target_sdk_dir\". Proceed[Y/n]?"
+echo -n "You are about to install the SDK to \"$target_sdk_dir\". Proceed[Y/n]?"
 read answer
 
 if [ "$answer" = "" ]; then
-- 
1.7.9.5

[PATCH v2 5/6] SDK: relocate symlinks too

[Laurentiu Palcu]

The directory usr/libexec/ in the SDK sysroot contains the default
symlinks to the toolchain binaries and these, too, need to point to the
correct toolchain path.

[YOCTO #3090]

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 meta/classes/populate_sdk_base.bbclass             |    5 +++++
 .../adt-installer/scripts/adt_installer_internal   |    5 +++++
 2 files changed, 10 insertions(+)

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index f20a6ac..52bae53 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -174,6 +174,11 @@ fi
 # replace ${SDKPATH} with the new prefix in all text files: configs/scripts/etc
 find $native_sysroot -type f -exec file '{}' \;|grep ":.*ASCII.*text"|cut -d':' -f1|xargs sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
 
+# change all symlinks pointing to ${SDKPATH}
+for l in $(find $native_sysroot -type l); do
+	ln -sf $(readlink $l|sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:") $l
+done
+
 echo done
 
 # delete the relocating script, so that user is forced to re-run the installer
diff --git a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
index fbcd2ce..63fcf60 100755
--- a/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
+++ b/meta/recipes-devtools/installer/adt-installer/scripts/adt_installer_internal
@@ -215,6 +215,11 @@ $SUDO sed -i -e "s:$DEFAULT_INSTALL_FOLDER:$NATIVE_INSTALL_DIR:g" $env_setup_scr
 find $OECORE_NATIVE_SYSROOT -type f -exec file '{}' \;|grep ":.*ASCII.*text"|cut -d':' -f1|\
     xargs $SUDO sed -i -e "s:$DEFAULT_INSTALL_FOLDER:$NATIVE_INSTALL_DIR:g"
 
+# change all symlinks pointing to /opt/${DISTRO}/${SDK_VERSION}
+for l in $(find $NATIVE_INSTALL_DIR -type l); do
+	ln -sf $(readlink $l|sed -e "s:$DEFAULT_INSTALL_FOLDER:$NATIVE_INSTALL_DIR:") $l
+done
+
 echo_info "\nSuccessfully installed selected native ADT!"
 }
 
-- 
1.7.9.5

[PATCH v2 6/6] SDK: allow toolchain installation from another directory

[Laurentiu Palcu]

This patch will allow one to run the installer from another directory
than the one where it's actually located.

Suppose the installer is in /home/user/test/my/sdk and the current
directory is in a different place. With this patch, one can run the
installer like this:

$ sh ~/test/my/sdk/poky-eglibc-x86_64-arm-toolchain-gmae-1.2+snapshot-20120920.sh

[YOCTO #3135]

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 meta/classes/populate_sdk_base.bbclass |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index 52bae53..9f51562 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -150,10 +150,10 @@ if [ $? -ne 0 ]; then
 	exit 1
 fi
 
-payload_offset=$(($(grep -na -m1 "^MARKER:$" $(basename $0)|cut -d':' -f1) + 1))
+payload_offset=$(($(grep -na -m1 "^MARKER:$" $0|cut -d':' -f1) + 1))
 
 echo -n "Extracting SDK..."
-tail -n +$payload_offset $(basename $0) | tar xj --strip-components=4 -C $target_sdk_dir
+tail -n +$payload_offset $0| tar xj --strip-components=4 -C $target_sdk_dir
 echo "done"
 
 echo -n "Setting it up..."
-- 
1.7.9.5

Re: [PATCH v2 5/6] SDK: relocate symlinks too

[Richard Purdie]

On Thu, 2012-09-20 at 16:48 +0300, Laurentiu Palcu wrote:
> The directory usr/libexec/ in the SDK sysroot contains the default
> symlinks to the toolchain binaries and these, too, need to point to the
> correct toolchain path.
> 
> [YOCTO #3090]
> 
> Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
> ---
>  meta/classes/populate_sdk_base.bbclass             |    5 +++++
>  .../adt-installer/scripts/adt_installer_internal   |    5 +++++
>  2 files changed, 10 insertions(+)

I've merged this since it definitely improves the situation but I am
left wondering if we shouldn't generate relative symlinks in the first
place for the toolchain...

Cheers,

Richard

Re: [PATCH v2 5/6] SDK: relocate symlinks too

[Laurentiu Palcu]

On 09/21/2012 01:22 PM, Richard Purdie wrote:
> On Thu, 2012-09-20 at 16:48 +0300, Laurentiu Palcu wrote:
>> The directory usr/libexec/ in the SDK sysroot contains the default
>> symlinks to the toolchain binaries and these, too, need to point to the
>> correct toolchain path.
>>
>> [YOCTO #3090]
>>
>> Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
>> ---
>>  meta/classes/populate_sdk_base.bbclass             |    5 +++++
>>  .../adt-installer/scripts/adt_installer_internal   |    5 +++++
>>  2 files changed, 10 insertions(+)
> 
> I've merged this since it definitely improves the situation but I am
> left wondering if we shouldn't generate relative symlinks in the first
> place for the toolchain...
I suppose we can do that too and leave this patch as a safety in case
absolute symlinks appear (by mistake) anywhere in the SDK rootfs. It's
up to you.

Thanks,
Laurentiu
> 
> Cheers,
> 
> Richard
> 
> 
> 

Re: [PATCH v2 0/6] SDK and adt_installer fixes

[Saul Wold]

On 09/20/2012 06:47 AM, Laurentiu Palcu wrote:
> Changes in v2:
> Since the previous patchset was not merged I took the liberty of adding
> another 2 fixes and send another patchset so that they're merged in the
> right order.
>
> Changes in v1:
>
> Hi,
>
> This patchset contains some SDK/adt-installer fixes.
>
> Thanks,
> Laurentiu
>
> The following changes since commit 913944d904266bf90af0cad94b4f0fb3652bd29d:
>
>    upstream_tracking: update lsb and ltp (2012-09-14 17:12:52 +0100)
>
> are available in the git repository at:
>
>    git://git.yoctoproject.org/poky-contrib lpalcu/adt_sdk_fixes
>    http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/adt_sdk_fixes
>
> Laurentiu Palcu (6):
>    adt-installer: fix package installation issue
>    SDK: fix installation into symlinked directories
>    adt-installer: ensure directory exists before copying/removing
>    SDK: remove references to Poky distro from tarball installer
>    SDK: relocate symlinks too
>    SDK: allow toolchain installation from another directory
>
>   meta/classes/populate_sdk_base.bbclass             |   21 +++++--
>   .../installer/adt-installer/adt_installer          |    6 +-
>   .../adt-installer/scripts/adt_installer_internal   |   61 +++++++++++---------
>   3 files changed, 54 insertions(+), 34 deletions(-)
>

Merged into OE-Core

Thanks
	Sau!