[PATCH v3 0/3] relocate_sdk.py: improvements

[PATCH v3 0/3] relocate_sdk.py: improvements

[Jason Wessel]

New in v3 are all the fixes recommended by Laurentiu Palcu
  * Fix sudo exec problem
  * Preserve padding in the .interp section

---

Now that I have had to debug the SDK relocator on multiple occasions
I figure it might be nice to get the patches upstreamed.  This last
time through fixing the python 2.4.x compatibility was very easy
using the prior two patches to allow me to do so.

The idea here is to increase the portability of the relocator because
once the SDK is installed/relocated, generally speaking it is working
well on old and new hosts.

Thanks,
Jason.

[PATCH v3 1/3] relocate_sdk.py: Fix corruption of sdk binaries

[Jason Wessel]

There are two cases of corruption that the relocate_sdk.py was not correctly
dealing with.

1) SDK Extras should be left alone
   Extra external binaries included in an SDK that were linked against the
   host's version of /usr/lib/ld-so.so should not get a relocation applied.
   In the case that was discovered these were LSB compliant binaries that
   already worked on many hosts.

2) If the interp section is too small generate an error
   In the case of the qemu user code, it was using its own .ld file
   to link the executables which overrides the default in the nativesdk
   binutils.  This generated host executables which had a interp section
   that was too small to relocate.

   Now the relocate_sdk.py will print an error and continue on such that
   the error can be fixed by a developer without having to do the
   difficult task of debugging why it is crashing or not loading correctly.

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
 scripts/relocate_sdk.py |   13 +++++++++++--
 1 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/scripts/relocate_sdk.py b/scripts/relocate_sdk.py
index 74bb7a5..45d2c24 100755
--- a/scripts/relocate_sdk.py
+++ b/scripts/relocate_sdk.py
@@ -66,7 +66,7 @@ def parse_elf_header():
     e_ehsize, e_phentsize, e_phnum, e_shentsize, e_shnum, e_shstrndx =\
         hdr_struct.unpack(elf_header[16:hdr_size])
 
-def change_interpreter():
+def change_interpreter(elf_file_name):
     if arch == 32:
         ph_struct = struct.Struct("<IIIIIIII")
     else:
@@ -89,7 +89,16 @@ def change_interpreter():
         if p_type == 3:
             # PT_INTERP section
             f.seek(p_offset)
+            # External SDKs with mixed pre-compiled binaries should not get
+            # relocated so look for some variant of /lib
+            fname = f.read(11)
+            if fname.startswith("/lib/") or fname.startswith("/lib64/") or fname.startswith("/lib32/") or fname.startswith("/usr/lib32/") or fname.startswith("/usr/lib32/") or fname.startswith("/usr/lib64/"):
+                break
+            if (len(new_dl_path) >= p_filesz):
+                print "ERROR: could not relocate %s, interp size = %i and %i is needed." % (elf_file_name, p_memsz, len(new_dl_path) + 1)
+                break
             dl_path = new_dl_path + "\0" * (p_filesz - len(new_dl_path))
+            f.seek(p_offset)
             f.write(dl_path)
             break
 
@@ -199,7 +208,7 @@ for e in executables_list:
     arch = get_arch()
     if arch:
         parse_elf_header()
-        change_interpreter()
+        change_interpreter(e)
         change_dl_sysdirs()
 
     """ change permissions back """
-- 
1.7.1

[PATCH v3 2/3] populate_sdk_base.bbclass: Improve debugging capabilities for SDK installer

[Jason Wessel]

After having to debug the SDK installer a few times in
addition to the relocation code the following patch was created
to improve the capabilities around debugging the SDK installer.

1) Add a verbose mode -D which set a set -x to see what
   the SDK installer is doing.

2) Add a mode -S to save the relocation scripts for the purpose
   of debugging them in conjunction with -D

3) Add a mode -R to not execute the relocation scripts for the
   purpose of debugging the relocations.

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
 meta/classes/populate_sdk_base.bbclass |   48 ++++++++++++++++++++++++++++----
 1 files changed, 42 insertions(+), 6 deletions(-)

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index c025d40..923f925 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -136,7 +136,10 @@ DEFAULT_INSTALL_DIR="${SDKPATH}"
 SUDO_EXEC=""
 target_sdk_dir=""
 answer=""
-while getopts ":yd:" OPT; do
+relocate=1
+savescripts=0
+verbose=0
+while getopts ":yd:DRS" OPT; do
 	case $OPT in
 	y)
 		answer="Y"
@@ -145,15 +148,33 @@ while getopts ":yd:" OPT; do
 	d)
 		target_sdk_dir=$OPTARG
 		;;
+	D)
+		verbose=1
+		;;
+	R)
+		relocate=0
+		savescripts=1
+		;;
+	S)
+		savescripts=1
+		;;
 	*)
 		echo "Usage: $(basename $0) [-y] [-d <dir>]"
 		echo "  -y         Automatic yes to all prompts"
 		echo "  -d <dir>   Install the SDK to <dir>"
+		echo "======== Advanced DEBUGGING ONLY OPTIONS ========"
+		echo "  -S         Save relocation scripts"
+		echo "  -R         Do not relocate executables"
+		echo "  -D         use set -x to see what is going on"
 		exit 1
 		;;
 	esac
 done
 
+if [ $verbose = 1 ] ; then
+	set -x
+fi
+
 printf "Enter target directory for SDK (default: $DEFAULT_INSTALL_DIR): "
 if [ "$target_sdk_dir" = "" ]; then
 	read target_sdk_dir
@@ -231,10 +252,23 @@ if [ "$dl_path" = "" ] ; then
 	exit 1
 fi
 executable_files=$($SUDO_EXEC find $native_sysroot -type f -perm +111)
-$SUDO_EXEC ${env_setup_script%/*}/relocate_sdk.py $target_sdk_dir $dl_path $executable_files
-if [ $? -ne 0 ]; then
-	echo "SDK could not be set up. Relocate script failed. Abort!"
-	exit 1
+
+tdir=`mktemp -d`
+if [ x$tdir = x ] ; then
+   echo "SDK relocate failed, could not create a temporary directory"
+   exit 1
+fi
+echo "#!/bin/bash" > $tdir/relocate_sdk.sh
+echo exec ${env_setup_script%/*}/relocate_sdk.py $target_sdk_dir $dl_path $executable_files >> $tdir/relocate_sdk.sh
+$SUDO_EXEC mv $tdir/relocate_sdk.sh ${env_setup_script%/*}/relocate_sdk.sh
+$SUDO_EXEC chmod 755 ${env_setup_script%/*}/relocate_sdk.sh
+rm -rf $tdir
+if [ $relocate = 1 ] ; then
+	$SUDO_EXEC ${env_setup_script%/*}/relocate_sdk.sh
+	if [ $? -ne 0 ]; then
+		echo "SDK could not be set up. Relocate script failed. Abort!"
+		exit 1
+	fi
 fi
 
 # replace ${SDKPATH} with the new prefix in all text files: configs/scripts/etc
@@ -249,7 +283,9 @@ echo done
 
 # delete the relocating script, so that user is forced to re-run the installer
 # if he/she wants another location for the sdk
-$SUDO_EXEC rm ${env_setup_script%/*}/relocate_sdk.py
+if [ $savescripts = 0 ] ; then
+	$SUDO_EXEC rm ${env_setup_script%/*}/relocate_sdk.py ${env_setup_script%/*}/relocate_sdk.sh
+fi
 
 echo "SDK has been successfully set up and is ready to be used."
 
-- 
1.7.1

[PATCH v3 3/3] relocate_sdk.py: allow relocate_sdk.py to work with python 2.4.x

[Jason Wessel]

Avoid the chicken / egg problem of an SDK that provides a working
python but requires that version of python to extract itself.  The
RHEL 5.x systems and some other enterprise Linux systems ship with
python 2.4.x as the default python.  We need to at least be able to
extract work executables even if we never use the the host provided
python again.

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
 scripts/relocate_sdk.py |   26 +++++++++++++-------------
 1 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/scripts/relocate_sdk.py b/scripts/relocate_sdk.py
index 45d2c24..1d7bbb3 100755
--- a/scripts/relocate_sdk.py
+++ b/scripts/relocate_sdk.py
@@ -55,22 +55,22 @@ def parse_elf_header():
 
     if arch == 32:
         # 32bit
-        hdr_struct = struct.Struct("<HHILLLIHHHHHH")
+        hdr_fmt = "<HHILLLIHHHHHH"
         hdr_size = 52
     else:
         # 64bit
-        hdr_struct = struct.Struct("<HHIQQQIHHHHHH")
+        hdr_fmt = "<HHIQQQIHHHHHH"
         hdr_size = 64
 
     e_type, e_machine, e_version, e_entry, e_phoff, e_shoff, e_flags,\
     e_ehsize, e_phentsize, e_phnum, e_shentsize, e_shnum, e_shstrndx =\
-        hdr_struct.unpack(elf_header[16:hdr_size])
+        struct.unpack(hdr_fmt, elf_header[16:hdr_size])
 
 def change_interpreter(elf_file_name):
     if arch == 32:
-        ph_struct = struct.Struct("<IIIIIIII")
+        ph_fmt = "<IIIIIIII"
     else:
-        ph_struct = struct.Struct("<IIQQQQQQ")
+        ph_fmt = "<IIQQQQQQ"
 
     """ look for PT_INTERP section """
     for i in range(0,e_phnum):
@@ -79,11 +79,11 @@ def change_interpreter(elf_file_name):
         if arch == 32:
             # 32bit
             p_type, p_offset, p_vaddr, p_paddr, p_filesz,\
-                p_memsz, p_flags, p_align = ph_struct.unpack(ph_hdr)
+                p_memsz, p_flags, p_align = struct.unpack(ph_fmt, ph_hdr)
         else:
             # 64bit
             p_type, p_flags, p_offset, p_vaddr, p_paddr, \
-            p_filesz, p_memsz, p_align = ph_struct.unpack(ph_hdr)
+            p_filesz, p_memsz, p_align = struct.unpack(ph_fmt, ph_hdr)
 
         """ change interpreter """
         if p_type == 3:
@@ -104,9 +104,9 @@ def change_interpreter(elf_file_name):
 
 def change_dl_sysdirs():
     if arch == 32:
-        sh_struct = struct.Struct("<IIIIIIIIII")
+        sh_fmt = "<IIIIIIIIII"
     else:
-        sh_struct = struct.Struct("<IIQQQQIIQQ")
+        sh_fmt = "<IIQQQQIIQQ"
 
     """ read section string table """
     f.seek(e_shoff + e_shstrndx * e_shentsize)
@@ -127,7 +127,7 @@ def change_dl_sysdirs():
         sh_hdr = f.read(e_shentsize)
 
         sh_name, sh_type, sh_flags, sh_addr, sh_offset, sh_size, sh_link,\
-            sh_info, sh_addralign, sh_entsize = sh_struct.unpack(sh_hdr)
+            sh_info, sh_addralign, sh_entsize = struct.unpack(sh_fmt, sh_hdr)
 
         name = sh_strtab[sh_name:sh_strtab.find("\0", sh_name)]
 
@@ -181,7 +181,7 @@ def change_dl_sysdirs():
 
 # MAIN
 if len(sys.argv) < 4:
-    exit(-1)
+    sys.exit(-1)
 
 new_prefix = sys.argv[1]
 new_dl_path = sys.argv[2]
@@ -196,14 +196,14 @@ for e in executables_list:
 
     try:
         f = open(e, "r+b")
-    except IOError as ioex:
+    except IOError, ioex:
         if ioex.errno == errno.ETXTBSY:
             print("Could not open %s. File used by another process.\nPlease "\
                   "make sure you exit all processes that might use any SDK "\
                   "binaries." % e)
         else:
             print("Could not open %s: %s(%d)" % (e, ioex.strerror, ioex.errno))
-        exit(-1)
+        sys.exit(-1)
 
     arch = get_arch()
     if arch:
-- 
1.7.1