[PATCH 0/6] relocatable SDK

[PATCH 0/6] relocatable SDK

[Laurentiu Palcu]

Hi,

This patchset adds relocatable SDK functionality. Instead of a tarball, the
output of "bitbake meta-toolchain/meta-toolchain-sdk" will be a self extracting
archive.

The user will then execute the .sh script and give it the target
directory for SDK installation (default is /opt/poky). The installer will then
extract the embedded tarball to the user provided location and will set up the
SDK: change the paths in the environment script, change the dynamic loader path
in all binaries and, also, change the ls.so.cache path in the dynamic loader
itself, together with the SYSDIR paths/lengths.

With that, no more root privileges are needed in order to install the SDK.

Thanks,
Laurentiu

The following changes since commit 651b223c5fc93c7504e304e954b9ae4640ed47c6:

  usbutils: avoid dependency on bash (2012-07-26 18:52:00 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/relocatable_sdk
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/relocatable_sdk

Laurentiu Palcu (6):
  eglibc: relocatable SDK changes
  binutils: relocatable SDK: change PT_INTERP section size
  scripts: add script for relocating the SDK
  populate_sdk_base.bbclass: create self-extracting archive
  relocatable.bbclass: split it up, to reuse code
  package.bbclass: change RPATHs for nativesdk packages

 meta/classes/chrpath.bbclass                       |   89 +++++++++
 meta/classes/package.bbclass                       |    5 +
 meta/classes/populate_sdk_base.bbclass             |   82 ++++++++
 meta/classes/relocatable.bbclass                   |   91 +--------
 .../eglibc/eglibc-2.15/relocatable_sdk.patch       |   70 +++++++
 meta/recipes-core/eglibc/eglibc_2.15.bb            |    6 +-
 .../binutils/binutils-crosssdk_2.22.bb             |    5 +
 .../binutils/binutils/relocatable_sdk.patch        |   13 ++
 scripts/relocate_sdk.py                            |  200 ++++++++++++++++++++
 9 files changed, 470 insertions(+), 91 deletions(-)
 create mode 100644 meta/classes/chrpath.bbclass
 create mode 100644 meta/recipes-core/eglibc/eglibc-2.15/relocatable_sdk.patch
 create mode 100644 meta/recipes-devtools/binutils/binutils/relocatable_sdk.patch
 create mode 100755 scripts/relocate_sdk.py

-- 
1.7.9.5

[PATCH 1/6] eglibc: relocatable SDK changes

[Laurentiu Palcu]

Modifying the dynamic loader path in all binaries by the SDK installer
would not be possible because we cannot know in advance the SDK target
location. Hence, the PT_INTERP section size has been set to 4096 (which
is the maximum path lengh in Linux).

Also, for the dynamic loader SYSDIRS and ld.so.cache paths, special
4096 bytes sections were allocated so that we can easily parse the ELF
binary at install time and modify the paths.

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 .../eglibc/eglibc-2.15/relocatable_sdk.patch       |   70 ++++++++++++++++++++
 meta/recipes-core/eglibc/eglibc_2.15.bb            |    6 +-
 2 files changed, 74 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-core/eglibc/eglibc-2.15/relocatable_sdk.patch

diff --git a/meta/recipes-core/eglibc/eglibc-2.15/relocatable_sdk.patch b/meta/recipes-core/eglibc/eglibc-2.15/relocatable_sdk.patch
new file mode 100644
index 0000000..6c6f601
--- /dev/null
+++ b/meta/recipes-core/eglibc/eglibc-2.15/relocatable_sdk.patch
@@ -0,0 +1,70 @@
+Index: libc/elf/interp.c
+===================================================================
+--- libc.orig/elf/interp.c
++++ libc/elf/interp.c
+@@ -17,5 +17,5 @@
+    Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
+    02111-1307 USA.  */
+ 
+-const char __invoke_dynamic_linker__[] __attribute__ ((section (".interp")))
++const char __invoke_dynamic_linker__[4096] __attribute__ ((section (".interp")))
+   = RUNTIME_LINKER;
+Index: libc/elf/dl-load.c
+===================================================================
+--- libc.orig/elf/dl-load.c
++++ libc/elf/dl-load.c
+@@ -145,8 +145,8 @@ static size_t max_capstrlen attribute_re
+ /* Get the generated information about the trusted directories.  */
+ #include "trusted-dirs.h"
+ 
+-static const char system_dirs[] = SYSTEM_DIRS;
+-static const size_t system_dirs_len[] =
++static const char system_dirs[4096] __attribute__ ((section (".sysdirs"))) = SYSTEM_DIRS;
++volatile static const size_t system_dirs_len[] __attribute__ ((section (".sysdirslen"))) =
+ {
+   SYSTEM_DIRS_LEN
+ };
+Index: libc/elf/dl-cache.c
+===================================================================
+--- libc.orig/elf/dl-cache.c
++++ libc/elf/dl-cache.c
+@@ -134,6 +134,10 @@ do									      \
+ while (0)
+ 
+ 
++const char LD_SO_CACHE[4096] __attribute__ ((section (".ldsocache"))) =
++		SYSCONFDIR "/ld.so.cache";
++
++
+ int
+ internal_function
+ _dl_cache_libcmp (const char *p1, const char *p2)
+Index: libc/elf/ldconfig.c
+===================================================================
+--- libc.orig/elf/ldconfig.c
++++ libc/elf/ldconfig.c
+@@ -167,6 +167,9 @@ static struct argp argp =
+   options, parse_opt, NULL, doc, NULL, more_help, NULL
+ };
+ 
++
++extern const char LD_SO_CACHE[4096] __attribute__ ((section (".ldsocache")));
++
+ /* Check if string corresponds to an important hardware capability or
+    a platform.  */
+ static int
+Index: libc/sysdeps/generic/dl-cache.h
+===================================================================
+--- libc.orig/sysdeps/generic/dl-cache.h
++++ libc/sysdeps/generic/dl-cache.h
+@@ -28,10 +28,6 @@
+   ((flags) == 1 || (flags) == _DL_CACHE_DEFAULT_ID)
+ #endif
+ 
+-#ifndef LD_SO_CACHE
+-# define LD_SO_CACHE SYSCONFDIR "/ld.so.cache"
+-#endif
+-
+ #ifndef add_system_dir
+ # define add_system_dir(dir) add_dir (dir)
+ #endif
diff --git a/meta/recipes-core/eglibc/eglibc_2.15.bb b/meta/recipes-core/eglibc/eglibc_2.15.bb
index d9cb048..69db5c9 100644
--- a/meta/recipes-core/eglibc/eglibc_2.15.bb
+++ b/meta/recipes-core/eglibc/eglibc_2.15.bb
@@ -3,7 +3,7 @@ require eglibc.inc
 SRCREV = "19294"
 
 DEPENDS += "gperf-native"
-PR = "r12"
+PR = "r13"
 PR_append = "+svnr${SRCPV}"
 
 EGLIBC_BRANCH="eglibc-2_15"
@@ -33,7 +33,9 @@ LIC_FILES_CHKSUM = "file://LICENSES;md5=98a1128c4b58120182cbea3b1752d8b9 \
       file://posix/rxspencer/COPYRIGHT;md5=dc5485bb394a13b2332ec1c785f5d83a \
       file://COPYING.LIB;md5=bbb461211a33b134d42ed5ee802b37ff "
 
-SRC_URI_append_virtclass-nativesdk = " file://ld-search-order.patch"
+SRC_URI_append_virtclass-nativesdk = " file://ld-search-order.patch \
+            file://relocatable_sdk.patch \
+            "
 S = "${WORKDIR}/${EGLIBC_BRANCH}/libc"
 B = "${WORKDIR}/build-${TARGET_SYS}"
 
-- 
1.7.9.5

[PATCH 2/6] binutils: relocatable SDK: change PT_INTERP section size

[Laurentiu Palcu]

This patch is needed so that all SDK binaries have the PT_INTERP section
size set to 4096 (max path size in Linux) in order to be able to parse
the binaries later and change the interpreter to the path we want.

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 .../binutils/binutils-crosssdk_2.22.bb             |    5 +++++
 .../binutils/binutils/relocatable_sdk.patch        |   13 +++++++++++++
 2 files changed, 18 insertions(+)
 create mode 100644 meta/recipes-devtools/binutils/binutils/relocatable_sdk.patch

diff --git a/meta/recipes-devtools/binutils/binutils-crosssdk_2.22.bb b/meta/recipes-devtools/binutils/binutils-crosssdk_2.22.bb
index 0eb5684..c936549 100644
--- a/meta/recipes-devtools/binutils/binutils-crosssdk_2.22.bb
+++ b/meta/recipes-devtools/binutils/binutils-crosssdk_2.22.bb
@@ -2,8 +2,13 @@ require binutils-cross_${PV}.bb
 
 inherit crosssdk
 
+PR = "r1"
+
 PROVIDES = "virtual/${TARGET_PREFIX}binutils-crosssdk"
 
+
+SRC_URI += "file://relocatable_sdk.patch"
+
 do_configure_prepend () {
 	sed -i 's#/usr/local/lib /lib /usr/lib#${SDKPATHNATIVE}/lib ${SDKPATHNATIVE}/usr/lib /usr/local/lib /lib /usr/lib#' ${S}/ld/configure.tgt
 }
diff --git a/meta/recipes-devtools/binutils/binutils/relocatable_sdk.patch b/meta/recipes-devtools/binutils/binutils/relocatable_sdk.patch
new file mode 100644
index 0000000..09cb925
--- /dev/null
+++ b/meta/recipes-devtools/binutils/binutils/relocatable_sdk.patch
@@ -0,0 +1,13 @@
+Index: binutils-2.22/ld/scripttempl/elf.sc
+===================================================================
+--- binutils-2.22.orig/ld/scripttempl/elf.sc
++++ binutils-2.22/ld/scripttempl/elf.sc
+@@ -116,7 +116,7 @@ if test -n "${COMMONPAGESIZE}"; then
+   DATA_SEGMENT_RELRO_END=". = DATA_SEGMENT_RELRO_END (${SEPARATE_GOTPLT-0}, .);"
+ fi
+ if test -z "${INITIAL_READONLY_SECTIONS}${CREATE_SHLIB}"; then
+-  INITIAL_READONLY_SECTIONS=".interp       ${RELOCATING-0} : { *(.interp) }"
++  INITIAL_READONLY_SECTIONS=".interp       ${RELOCATING-0} : { *(.interp); . = 0x1000; }"
+ fi
+ if test -z "$PLT"; then
+   IPLT=".iplt         ${RELOCATING-0} : { *(.iplt) }"
-- 
1.7.9.5

[PATCH 3/6] scripts: add script for relocating the SDK

[Laurentiu Palcu]

This script will be embedded in the SDK tarball and will be called by
the SDK installer. It replaces the interpreter path in all binaries and
it also changes the ld.so.cache and SYSDIRS in dynamic loader.

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 scripts/relocate_sdk.py |  200 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 200 insertions(+)
 create mode 100755 scripts/relocate_sdk.py

diff --git a/scripts/relocate_sdk.py b/scripts/relocate_sdk.py
new file mode 100755
index 0000000..0668600
--- /dev/null
+++ b/scripts/relocate_sdk.py
@@ -0,0 +1,200 @@
+#!/usr/bin/env python
+#
+# Copyright (c) 2012 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+# See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+#
+# DESCRIPTION
+# This script is called by the SDK installer script. It replaces the dynamic
+# loader path in all binaries and also fixes the SYSDIR paths/lengths and the
+# location of ld.so.cache in the dynamic loader binary
+#
+# AUTHORS
+# Laurentiu Palcu <laurentiu.palcu@intel.com>
+#
+
+import struct
+import sys
+import stat
+import os
+import re
+
+old_prefix = re.compile("\/opt\/poky")
+
+def get_arch():
+    f.seek(0)
+    e_ident =f.read(16)
+    ei_mag0,ei_mag1_3,ei_class = struct.unpack("<B3sB11x", e_ident)
+
+    if (ei_mag0 != 0x7f and ei_mag1_3 != "ELF") or ei_class == 0:
+        return 0
+
+    if ei_class == 1:
+        return 32
+    elif ei_class == 2:
+        return 64
+
+def parse_elf_header():
+    global e_type, e_machine, e_version, e_entry, e_phoff, e_shoff, e_flags,\
+           e_ehsize, e_phentsize, e_phnum, e_shentsize, e_shnum, e_shstrndx
+
+    f.seek(0)
+    elf_header = f.read(64)
+
+    if arch == 32:
+        # 32bit
+        hdr_struct = struct.Struct("<HHILLLIHHHHHH")
+        hdr_size = 52
+    else:
+        # 64bit
+        hdr_struct = struct.Struct("<HHIQQQIHHHHHH")
+        hdr_size = 64
+
+    e_type, e_machine, e_version, e_entry, e_phoff, e_shoff, e_flags,\
+    e_ehsize, e_phentsize, e_phnum, e_shentsize, e_shnum, e_shstrndx =\
+        hdr_struct.unpack(elf_header[16:hdr_size])
+
+def change_interpreter():
+    if arch == 32:
+        ph_struct = struct.Struct("<IIIIIIII")
+    else:
+        ph_struct = struct.Struct("<IIQQQQQQ")
+
+    """ look for PT_INTERP section """
+    for i in range(0,e_phnum):
+        f.seek(e_phoff + i * e_phentsize)
+        ph_hdr = f.read(e_phentsize)
+        if arch == 32:
+            # 32bit
+            p_type, p_offset, p_vaddr, p_paddr, p_filesz,\
+                p_memsz, p_flags, p_align = ph_struct.unpack(ph_hdr)
+        else:
+            # 64bit
+            p_type, p_flags, p_offset, p_vaddr, p_paddr, \
+            p_filesz, p_memsz, p_align = ph_struct.unpack(ph_hdr)
+
+        """ change interpreter """
+        if p_type == 3:
+            # PT_INTERP section
+            f.seek(p_offset)
+            dl_path = new_dl_path + "\0" * (e_phentsize - len(new_dl_path))
+            f.write(new_dl_path)
+            break
+
+def change_dl_sysdirs():
+    if arch == 32:
+        sh_struct = struct.Struct("<IIIIIIIIII")
+    else:
+        sh_struct = struct.Struct("<IIQQQQIIQQ")
+
+    """ read section string table """
+    f.seek(e_shoff + e_shstrndx * e_shentsize)
+    sh_hdr = f.read(e_shentsize)
+    if arch == 32:
+        sh_offset, sh_size = struct.unpack("<16xII16x", sh_hdr)
+    else:
+        sh_offset, sh_size = struct.unpack("<24xQQ24x", sh_hdr)
+
+    f.seek(sh_offset)
+    sh_strtab = f.read(sh_size)
+
+    sysdirs = sysdirs_len = ""
+
+    """ change ld.so.cache path and default libs path for dynamic loader """
+    for i in range(0,e_shnum):
+        f.seek(e_shoff + i * e_shentsize)
+        sh_hdr = f.read(e_shentsize)
+
+        sh_name, sh_type, sh_flags, sh_addr, sh_offset, sh_size, sh_link,\
+            sh_info, sh_addralign, sh_entsize = sh_struct.unpack(sh_hdr)
+
+        name = sh_strtab[sh_name:sh_strtab.find("\0", sh_name)]
+
+        """ look only into SHT_PROGBITS sections """
+        if sh_type == 1:
+            f.seek(sh_offset)
+            """ default library paths cannot be changed on the fly because  """
+            """ the string lengths have to be changed too.                  """
+            if name == ".sysdirs":
+                sysdirs = f.read(sh_size)
+                sysdirs_off = sh_offset
+                sysdirs_sect_size = sh_size
+            elif name == ".sysdirslen":
+                sysdirslen = f.read(sh_size)
+                sysdirslen_off = sh_offset
+            elif name == ".ldsocache":
+                ldsocache_path = f.read(sh_size)
+                new_ldsocache_path = old_prefix.sub(new_prefix, ldsocache_path)
+                # pad with zeros
+                new_ldsocache_path += "\0" * (sh_size - len(new_ldsocache_path))
+                # write it back
+                f.seek(sh_offset)
+                f.write(new_ldsocache_path)
+
+    if sysdirs != "" and sysdirslen != "":
+        paths = sysdirs.split("\0")
+        sysdirs = ""
+        sysdirslen = ""
+        for path in paths:
+            """ exit the loop when we encounter first empty string """
+            if path == "":
+                break
+
+            new_path = old_prefix.sub(new_prefix, path)
+            sysdirs += new_path + "\0"
+
+            if arch == 32:
+                sysdirslen += struct.pack("<L", len(new_path))
+            else:
+                sysdirslen += struct.pack("<Q", len(new_path))
+
+        """ pad with zeros """
+        sysdirs += "\0" * (sysdirs_sect_size - len(sysdirs))
+
+        """ write the sections back """
+        f.seek(sysdirs_off)
+        f.write(sysdirs)
+        f.seek(sysdirslen_off)
+        f.write(sysdirslen)
+
+
+# MAIN
+if len(sys.argv) < 4:
+    exit(1)
+
+new_prefix = sys.argv[1]
+new_dl_path = sys.argv[2]
+executables_list = sys.argv[3:]
+
+for e in executables_list:
+    perms = os.stat(e)[stat.ST_MODE]
+    if os.access(e, os.W_OK|os.R_OK):
+        perms = None
+    else:
+        os.chmod(e, perms|stat.S_IRWXU)
+
+    f = open(e, "r+b")
+
+    arch = get_arch()
+    if arch:
+        parse_elf_header()
+        change_interpreter()
+        change_dl_sysdirs()
+
+    """ change permissions back """
+    if perms:
+        os.chmod(e, perms)
+
+    f.close()
+
-- 
1.7.9.5

[PATCH 4/6] populate_sdk_base.bbclass: create self-extracting archive

[Laurentiu Palcu]

In order for the SDK to be relocatable, the user would need to call a
setup script to change the binaries acordingly. Having an auto-extracting
archive has the advantage of being more user friendly and the user does
not have to call the setup script separately after extracting the SDK.
It is called automatically.

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 meta/classes/populate_sdk_base.bbclass |   82 ++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index 9483e93..4464cdd 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -55,6 +55,8 @@ fakeroot python do_populate_sdk() {
         bb.build.exec_func("create_sdk_files", localdata)
 
     bb.build.exec_func("tar_sdk", d)
+
+    bb.build.exec_func("create_shar", d)
 }
 
 fakeroot populate_sdk_image() {
@@ -94,6 +96,8 @@ fakeroot create_sdk_files() {
 
 	# Add version information
 	toolchain_create_sdk_version ${SDK_OUTPUT}/${SDKPATH}/version-${REAL_MULTIMACH_TARGET_SYS}
+
+	cp ${COREBASE}/scripts/relocate_sdk.py ${SDK_OUTPUT}/${SDKPATH}/
 }
 
 fakeroot tar_sdk() {
@@ -103,6 +107,84 @@ fakeroot tar_sdk() {
 	tar --owner=root --group=root -cj --file=${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.tar.bz2 .
 }
 
+fakeroot create_shar() {
+	cat << "EOF" > ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh
+#!/bin/bash
+
+DEFAULT_INSTALL_DIR="/opt/poky"
+
+echo -n "Enter target directory for Poky SDK (default: $DEFAULT_INSTALL_DIR): "
+read target_sdk_dir
+
+if [ "$target_sdk_dir" = "" ]; then
+	target_sdk_dir=$DEFAULT_INSTALL_DIR
+fi
+
+eval target_sdk_dir=$target_sdk_dir
+target_sdk_dir=$(readlink -m $target_sdk_dir)
+
+echo -n "You are about to install Poky SDK to \"$target_sdk_dir\". Proceed[Y/n]?"
+read answer
+
+if [ "$answer" = "" ]; then
+	answer="y"
+fi
+
+if [ "$answer" != "Y" -a "$answer" != "y" ]; then
+	echo "Installation aborted!"
+	exit 1
+fi
+
+mkdir -p $target_sdk_dir >/dev/null 2>&1
+if [ $? -ne 0 ]; then
+	echo "Error: Unable to create target directory. Do you have permissions?"
+	exit 1
+fi
+
+target_sdk_dir_absoulte_path=$(readlink -f $target_sdk_dir)
+
+payload_offset=$(($(grep -na -m1 "^MARKER:$" $(basename $0)|cut -d':' -f1) + 1))
+
+echo -n "Extracting SDK..."
+tail -n +$payload_offset $(basename $0) | tar xj --strip-components=3 -C $target_sdk_dir
+echo "done"
+
+echo -n "Setting it up..."
+# fix environment paths
+env_setup_script=$(find $target_sdk_dir_absoulte_path -name "environment-setup*")
+sed -e "s:/opt/poky:$target_sdk_dir_absoulte_path:g" -i $env_setup_script
+
+# fix dynamic loader paths in all ELF SDK binaries
+native_sysroot=$(cat $env_setup_script |grep OECORE_NATIVE_SYSROOT|cut -d'=' -f2|tr -d '"')
+dl_path=$(find $native_sysroot/lib -name "ld-linux*")
+executable_files=$(find $native_sysroot -type f -perm +111)
+${env_setup_script%/*}/relocate_sdk.py $target_sdk_dir_absoulte_path $dl_path $executable_files
+if [ $? -ne 0 ]; then
+	echo "SDK could not be set up. Relocate script failed. Abort!"
+	exit 1
+fi
+
+# replace /opt/poky with the new prefix in all text files: configs/scripts/etc
+find $native_sysroot -type f -exec file '{}' \;|grep text|cut -d':' -f1|xargs sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
+
+echo done
+
+# delete the relocating script, so that user is forced to re-run the installer
+# if he/she wants another location for the sdk
+rm ${env_setup_script%/*}/relocate_sdk.py
+
+echo "SDK has been successfully set up and is ready to be used."
+
+exit 0
+
+MARKER:
+EOF
+	cat ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.tar.bz2 >> ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh
+
+	# delete the old tarball, we don't need it anymore
+	rm ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.tar.bz2
+}
+
 populate_sdk_log_check() {
 	for target in $*
 	do
-- 
1.7.9.5

[PATCH 5/6] relocatable.bbclass: split it up, to reuse code

[Laurentiu Palcu]

Most of the code in relocatable.bbclass will be used for relocating the
SDK binaries. So, create another class chrpath.bbclass that will contain
the core of the relocatable.bbclass, so we can reuse it.

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 meta/classes/chrpath.bbclass     |   89 +++++++++++++++++++++++++++++++++++++
 meta/classes/relocatable.bbclass |   91 +-------------------------------------
 2 files changed, 91 insertions(+), 89 deletions(-)
 create mode 100644 meta/classes/chrpath.bbclass

diff --git a/meta/classes/chrpath.bbclass b/meta/classes/chrpath.bbclass
new file mode 100644
index 0000000..10b5ca0
--- /dev/null
+++ b/meta/classes/chrpath.bbclass
@@ -0,0 +1,89 @@
+CHRPATH_BIN ?= "chrpath"
+PREPROCESS_RELOCATE_DIRS ?= ""
+
+def process_dir (directory, d):
+    import subprocess as sub
+    import stat
+
+    cmd = d.expand('${CHRPATH_BIN}')
+    tmpdir = d.getVar('TMPDIR')
+    basedir = d.expand('${base_prefix}')
+
+    #bb.debug("Checking %s for binaries to process" % directory)
+    if not os.path.exists(directory):
+        return
+
+    dirs = os.listdir(directory)
+    for file in dirs:
+        fpath = directory + "/" + file
+        fpath = os.path.normpath(fpath)
+        if os.path.islink(fpath):
+            # Skip symlinks
+            continue
+
+        if os.path.isdir(fpath):
+            process_dir(fpath, d)
+        else:
+            #bb.note("Testing %s for relocatability" % fpath)
+
+            # We need read and write permissions for chrpath, if we don't have
+            # them then set them temporarily. Take a copy of the files
+            # permissions so that we can restore them afterwards.
+            perms = os.stat(fpath)[stat.ST_MODE]
+            if os.access(fpath, os.W_OK|os.R_OK):
+                perms = None
+            else:
+                # Temporarily make the file writeable so we can chrpath it
+                os.chmod(fpath, perms|stat.S_IRWXU)
+
+            p = sub.Popen([cmd, '-l', fpath],stdout=sub.PIPE,stderr=sub.PIPE)
+            err, out = p.communicate()
+            # If returned succesfully, process stderr for results
+            if p.returncode != 0:
+                continue
+
+            # Throw away everything other than the rpath list
+            curr_rpath = err.partition("RPATH=")[2]
+            #bb.note("Current rpath for %s is %s" % (fpath, curr_rpath.strip()))
+            rpaths = curr_rpath.split(":")
+            new_rpaths = []
+            for rpath in rpaths:
+                # If rpath is already dynamic continue
+                if rpath.find("$ORIGIN") != -1:
+                    continue
+                # If the rpath shares a root with base_prefix determine a new dynamic rpath from the
+                # base_prefix shared root
+                if rpath.find(basedir) != -1:
+                    depth = fpath.partition(basedir)[2].count('/')
+                    libpath = rpath.partition(basedir)[2].strip()
+                # otherwise (i.e. cross packages) determine a shared root based on the TMPDIR
+                # NOTE: This will not work reliably for cross packages, particularly in the case
+                # where your TMPDIR is a short path (i.e. /usr/poky) as chrpath cannot insert an
+                # rpath longer than that which is already set.
+                else:
+                    depth = fpath.rpartition(tmpdir)[2].count('/')
+                    libpath = rpath.partition(tmpdir)[2].strip()
+
+                base = "$ORIGIN"
+                while depth > 1:
+                    base += "/.."
+                    depth-=1
+                new_rpaths.append("%s%s" % (base, libpath))
+
+            # if we have modified some rpaths call chrpath to update the binary
+            if len(new_rpaths):
+                args = ":".join(new_rpaths)
+                #bb.note("Setting rpath for %s to %s" %(fpath, args))
+                sub.call([cmd, '-r', args, fpath])
+
+            if perms:
+                os.chmod(fpath, perms)
+
+def rpath_replace (path, d):
+    bindirs = d.expand("${bindir} ${sbindir} ${base_sbindir} ${base_bindir} ${libdir} ${base_libdir} ${libexecdir} ${PREPROCESS_RELOCATE_DIRS}").split()
+
+    for bindir in bindirs:
+        #bb.note ("Processing directory " + bindir)
+        directory = path + "/" + bindir
+        process_dir (directory, d)
+
diff --git a/meta/classes/relocatable.bbclass b/meta/classes/relocatable.bbclass
index 072f533..4ca9981 100644
--- a/meta/classes/relocatable.bbclass
+++ b/meta/classes/relocatable.bbclass
@@ -1,93 +1,6 @@
-SYSROOT_PREPROCESS_FUNCS += "relocatable_binaries_preprocess"
-
-CHRPATH_BIN ?= "chrpath"
-PREPROCESS_RELOCATE_DIRS ?= ""
-
-def process_dir (directory, d):
-    import subprocess as sub
-    import stat
-
-    cmd = d.expand('${CHRPATH_BIN}')
-    tmpdir = d.getVar('TMPDIR')
-    basedir = d.expand('${base_prefix}')
-
-    #bb.debug("Checking %s for binaries to process" % directory)
-    if not os.path.exists(directory):
-        return
-
-    dirs = os.listdir(directory)
-    for file in dirs:
-        fpath = directory + "/" + file
-        fpath = os.path.normpath(fpath)
-        if os.path.islink(fpath):
-            # Skip symlinks
-            continue
-
-        if os.path.isdir(fpath):
-            process_dir(fpath, d)
-        else:
-            #bb.note("Testing %s for relocatability" % fpath)
-
-            # We need read and write permissions for chrpath, if we don't have
-            # them then set them temporarily. Take a copy of the files
-            # permissions so that we can restore them afterwards.
-            perms = os.stat(fpath)[stat.ST_MODE]
-            if os.access(fpath, os.W_OK|os.R_OK):
-                perms = None
-            else:
-                # Temporarily make the file writeable so we can chrpath it
-                os.chmod(fpath, perms|stat.S_IRWXU)
+inherit chrpath
 
-            p = sub.Popen([cmd, '-l', fpath],stdout=sub.PIPE,stderr=sub.PIPE)
-            err, out = p.communicate()
-            # If returned succesfully, process stderr for results
-            if p.returncode != 0:
-                continue
-
-            # Throw away everything other than the rpath list
-            curr_rpath = err.partition("RPATH=")[2]
-            #bb.note("Current rpath for %s is %s" % (fpath, curr_rpath.strip()))
-            rpaths = curr_rpath.split(":")
-            new_rpaths = []
-            for rpath in rpaths:
-                # If rpath is already dynamic continue
-                if rpath.find("$ORIGIN") != -1:
-                    continue
-                # If the rpath shares a root with base_prefix determine a new dynamic rpath from the
-                # base_prefix shared root
-                if rpath.find(basedir) != -1:
-                    depth = fpath.partition(basedir)[2].count('/')
-                    libpath = rpath.partition(basedir)[2].strip()
-                # otherwise (i.e. cross packages) determine a shared root based on the TMPDIR
-                # NOTE: This will not work reliably for cross packages, particularly in the case
-                # where your TMPDIR is a short path (i.e. /usr/poky) as chrpath cannot insert an
-                # rpath longer than that which is already set.
-                else:
-                    depth = fpath.rpartition(tmpdir)[2].count('/')
-                    libpath = rpath.partition(tmpdir)[2].strip()
-
-                base = "$ORIGIN"
-                while depth > 1:
-                    base += "/.."
-                    depth-=1
-                new_rpaths.append("%s%s" % (base, libpath))
-
-            # if we have modified some rpaths call chrpath to update the binary
-            if len(new_rpaths):
-                args = ":".join(new_rpaths)
-                #bb.note("Setting rpath for %s to %s" %(fpath, args))
-                sub.call([cmd, '-r', args, fpath])
-
-            if perms:
-                os.chmod(fpath, perms)
-
-def rpath_replace (path, d):
-    bindirs = d.expand("${bindir} ${sbindir} ${base_sbindir} ${base_bindir} ${libdir} ${base_libdir} ${libexecdir} ${PREPROCESS_RELOCATE_DIRS}").split()
-
-    for bindir in bindirs:
-        #bb.note ("Processing directory " + bindir)
-        directory = path + "/" + bindir
-        process_dir (directory, d)
+SYSROOT_PREPROCESS_FUNCS += "relocatable_binaries_preprocess"
 
 python relocatable_binaries_preprocess() {
     rpath_replace(d.expand('${SYSROOT_DESTDIR}'), d)
-- 
1.7.9.5

[PATCH 6/6] package.bbclass: change RPATHs for nativesdk packages

[Laurentiu Palcu]

Change binaries RPATHs, to include $ORIGIN, to make them relocatable.

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
---
 meta/classes/package.bbclass |    5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 962abcd..52986e1 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -40,6 +40,7 @@
 
 inherit packagedata
 inherit prserv
+inherit chrpath
 
 PKGD    = "${WORKDIR}/package"
 PKGDEST = "${WORKDIR}/packages-split"
@@ -438,6 +439,10 @@ python perform_packagecopy () {
     subprocess.call('rm -rf %s/*' % (dvar), shell=True)
     # Preserve sparse files and hard links
     subprocess.call('tar -cf - -C %s -ps . | tar -xf - -C %s' % (dest, dvar), shell=True)
+
+    # replace RPATHs for the nativesdk binaries, to make them relocatable
+    if bb.data.inherits_class('nativesdk', d):
+        rpath_replace (dvar, d)
 }
 
 # We generate a master list of directories to process, we start by
-- 
1.7.9.5

Re: [PATCH 0/6] relocatable SDK

[Saul Wold]

On 07/27/2012 04:21 AM, Laurentiu Palcu wrote:
> Hi,
>
> This patchset adds relocatable SDK functionality. Instead of a tarball, the
> output of "bitbake meta-toolchain/meta-toolchain-sdk" will be a self extracting
> archive.
>
> The user will then execute the .sh script and give it the target
> directory for SDK installation (default is /opt/poky). The installer will then
> extract the embedded tarball to the user provided location and will set up the
> SDK: change the paths in the environment script, change the dynamic loader path
> in all binaries and, also, change the ls.so.cache path in the dynamic loader
> itself, together with the SYSDIR paths/lengths.
>
> With that, no more root privileges are needed in order to install the SDK.
>
> Thanks,
> Laurentiu
>
> The following changes since commit 651b223c5fc93c7504e304e954b9ae4640ed47c6:
>
>    usbutils: avoid dependency on bash (2012-07-26 18:52:00 +0100)
>
> are available in the git repository at:
>
>    git://git.yoctoproject.org/poky-contrib lpalcu/relocatable_sdk
>    http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/relocatable_sdk
>
> Laurentiu Palcu (6):
>    eglibc: relocatable SDK changes
>    binutils: relocatable SDK: change PT_INTERP section size
>    scripts: add script for relocating the SDK
>    populate_sdk_base.bbclass: create self-extracting archive
>    relocatable.bbclass: split it up, to reuse code
>    package.bbclass: change RPATHs for nativesdk packages
>
>   meta/classes/chrpath.bbclass                       |   89 +++++++++
>   meta/classes/package.bbclass                       |    5 +
>   meta/classes/populate_sdk_base.bbclass             |   82 ++++++++
>   meta/classes/relocatable.bbclass                   |   91 +--------
>   .../eglibc/eglibc-2.15/relocatable_sdk.patch       |   70 +++++++
>   meta/recipes-core/eglibc/eglibc_2.15.bb            |    6 +-
>   .../binutils/binutils-crosssdk_2.22.bb             |    5 +
>   .../binutils/binutils/relocatable_sdk.patch        |   13 ++
>   scripts/relocate_sdk.py                            |  200 ++++++++++++++++++++
>   9 files changed, 470 insertions(+), 91 deletions(-)
>   create mode 100644 meta/classes/chrpath.bbclass
>   create mode 100644 meta/recipes-core/eglibc/eglibc-2.15/relocatable_sdk.patch
>   create mode 100644 meta/recipes-devtools/binutils/binutils/relocatable_sdk.patch
>   create mode 100755 scripts/relocate_sdk.py
>
Your patches need to have Upstream-Status and Signed-off-by: added along 
with a header explaining why the patch is needed.

Just a quick first pass review, there may be more comments as I review 
deeper.

Sau!

Re: [PATCH 0/6] relocatable SDK

[Philip Balister]

On 07/27/2012 02:05 PM, Saul Wold wrote:
> On 07/27/2012 04:21 AM, Laurentiu Palcu wrote:
>> Hi,
>>
>> This patchset adds relocatable SDK functionality. Instead of a
>> tarball, the
>> output of "bitbake meta-toolchain/meta-toolchain-sdk" will be a self
>> extracting
>> archive.
>>
>> The user will then execute the .sh script and give it the target
>> directory for SDK installation (default is /opt/poky). The installer

For oe-core the default should be something like /opt/oe-core, not 
/opt/some-distro.

Philip

>> will then
>> extract the embedded tarball to the user provided location and will
>> set up the
>> SDK: change the paths in the environment script, change the dynamic
>> loader path
>> in all binaries and, also, change the ls.so.cache path in the dynamic
>> loader
>> itself, together with the SYSDIR paths/lengths.
>>
>> With that, no more root privileges are needed in order to install the
>> SDK.
>>
>> Thanks,
>> Laurentiu
>>
>> The following changes since commit
>> 651b223c5fc93c7504e304e954b9ae4640ed47c6:
>>
>>    usbutils: avoid dependency on bash (2012-07-26 18:52:00 +0100)
>>
>> are available in the git repository at:
>>
>>    git://git.yoctoproject.org/poky-contrib lpalcu/relocatable_sdk
>>
>> http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/relocatable_sdk
>>
>>
>> Laurentiu Palcu (6):
>>    eglibc: relocatable SDK changes
>>    binutils: relocatable SDK: change PT_INTERP section size
>>    scripts: add script for relocating the SDK
>>    populate_sdk_base.bbclass: create self-extracting archive
>>    relocatable.bbclass: split it up, to reuse code
>>    package.bbclass: change RPATHs for nativesdk packages
>>
>>   meta/classes/chrpath.bbclass                       |   89 +++++++++
>>   meta/classes/package.bbclass                       |    5 +
>>   meta/classes/populate_sdk_base.bbclass             |   82 ++++++++
>>   meta/classes/relocatable.bbclass                   |   91 +--------
>>   .../eglibc/eglibc-2.15/relocatable_sdk.patch       |   70 +++++++
>>   meta/recipes-core/eglibc/eglibc_2.15.bb            |    6 +-
>>   .../binutils/binutils-crosssdk_2.22.bb             |    5 +
>>   .../binutils/binutils/relocatable_sdk.patch        |   13 ++
>>   scripts/relocate_sdk.py                            |  200
>> ++++++++++++++++++++
>>   9 files changed, 470 insertions(+), 91 deletions(-)
>>   create mode 100644 meta/classes/chrpath.bbclass
>>   create mode 100644
>> meta/recipes-core/eglibc/eglibc-2.15/relocatable_sdk.patch
>>   create mode 100644
>> meta/recipes-devtools/binutils/binutils/relocatable_sdk.patch
>>   create mode 100755 scripts/relocate_sdk.py
>>
> Your patches need to have Upstream-Status and Signed-off-by: added along
> with a header explaining why the patch is needed.
>
> Just a quick first pass review, there may be more comments as I review
> deeper.
>
> Sau!
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>

Re: [PATCH 0/6] relocatable SDK

[Laurentiu Palcu]

Hi Philip,

>>> The user will then execute the .sh script and give it the target
>>> directory for SDK installation (default is /opt/poky). The installer
> 
> For oe-core the default should be something like /opt/oe-core, not 
> /opt/some-distro.
This a good observation.

SDKPATH = "/opt/${DISTRO}/${SDK_VERSION}"

I will change the installer/setup scripts so that the default SDK
installation directory takes into account the DISTRO variable.

Thanks,
Laurentiu

> 
> Philip

Re: [PATCH 0/6] relocatable SDK

[Richard Purdie]

On Mon, 2012-07-30 at 11:50 +0300, Laurentiu Palcu wrote:
> Hi Philip,
> 
> >>> The user will then execute the .sh script and give it the target
> >>> directory for SDK installation (default is /opt/poky). The installer
> > 
> > For oe-core the default should be something like /opt/oe-core, not 
> > /opt/some-distro.
> This a good observation.
> 
> SDKPATH = "/opt/${DISTRO}/${SDK_VERSION}"
> 
> I will change the installer/setup scripts so that the default SDK
> installation directory takes into account the DISTRO variable.

SDKPATH is the right variable to use for this. The above is from
poky.conf and is a valid distro configuration option. The OE-Core
default doesn't use distro and installs into /usr/local/oecore/

Cheers,

Richard