* [Patch 0/1] Update relocate_sdk.py to work with Python 2.4 @ 2013-10-11 15:30 Konrad Scherer 2013-10-11 15:30 ` [PATCH] Allow script to work with Python 2.4 and 3 Konrad Scherer 0 siblings, 1 reply; 5+ messages in thread From: Konrad Scherer @ 2013-10-11 15:30 UTC (permalink / raw) To: openembedded-core As a bootstrap script, relocate_sdk.py needs to support as many versions of python as possible. Since python 2.4.3 is default python on RedHat/CentOS 5.x, the recent updates to support python 3 broke these platforms. This is the cleanest way I could find to support all pythons 2.4 to 3. I have tested the patch on 2.4.3, 2.6.2, 2.7.3 and 3.2.3. --- Konrad Scherer, MTS, Linux Products Group, Wind River ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH] Allow script to work with Python 2.4 and 3. 2013-10-11 15:30 [Patch 0/1] Update relocate_sdk.py to work with Python 2.4 Konrad Scherer @ 2013-10-11 15:30 ` Konrad Scherer 2013-10-14 18:03 ` Saul Wold 0 siblings, 1 reply; 5+ messages in thread From: Konrad Scherer @ 2013-10-11 15:30 UTC (permalink / raw) To: openembedded-core From: Konrad Scherer <Konrad.Scherer@windriver.com> Python 2.4 does not support the 'b' string literal or the keyword 'as' in exception handling. Python 3 does not accept the old method of exception handling and defaults to unicode. The b() function converts strings to bytes on Python 3 and using sys.exc_info() avoids the exception handling syntax. Signed-off-by: Konrad Scherer <Konrad.Scherer@windriver.com> --- scripts/relocate_sdk.py | 43 ++++++++++++++++++++++++++----------------- 1 file changed, 26 insertions(+), 17 deletions(-) diff --git a/scripts/relocate_sdk.py b/scripts/relocate_sdk.py index fe6e4e0..0971e63 100755 --- a/scripts/relocate_sdk.py +++ b/scripts/relocate_sdk.py @@ -31,7 +31,15 @@ import os import re import errno -old_prefix = re.compile(b"##DEFAULT_INSTALL_DIR##") +if sys.version < '3': + def b(x): + return x +else: + import codecs + def b(x): + return codecs.latin_1_encode(x)[0] + +old_prefix = re.compile(b("##DEFAULT_INSTALL_DIR##")) def get_arch(): f.seek(0) @@ -92,15 +100,15 @@ def change_interpreter(elf_file_name): # External SDKs with mixed pre-compiled binaries should not get # relocated so look for some variant of /lib fname = f.read(11) - if fname.startswith(b"/lib/") or fname.startswith(b"/lib64/") or \ - fname.startswith(b"/lib32/") or fname.startswith(b"/usr/lib32/") or \ - fname.startswith(b"/usr/lib32/") or fname.startswith(b"/usr/lib64/"): + if fname.startswith(b("/lib/")) or fname.startswith(b("/lib64/")) or \ + fname.startswith(b("/lib32/")) or fname.startswith(b("/usr/lib32/")) or \ + fname.startswith(b("/usr/lib32/")) or fname.startswith(b("/usr/lib64/")): break if (len(new_dl_path) >= p_filesz): print("ERROR: could not relocate %s, interp size = %i and %i is needed." \ % (elf_file_name, p_memsz, len(new_dl_path) + 1)) break - dl_path = new_dl_path + b"\0" * (p_filesz - len(new_dl_path)) + dl_path = new_dl_path + b("\0") * (p_filesz - len(new_dl_path)) f.seek(p_offset) f.write(dl_path) break @@ -132,40 +140,40 @@ def change_dl_sysdirs(): sh_name, sh_type, sh_flags, sh_addr, sh_offset, sh_size, sh_link,\ sh_info, sh_addralign, sh_entsize = struct.unpack(sh_fmt, sh_hdr) - name = sh_strtab[sh_name:sh_strtab.find(b"\0", sh_name)] + name = sh_strtab[sh_name:sh_strtab.find(b("\0"), sh_name)] """ look only into SHT_PROGBITS sections """ if sh_type == 1: f.seek(sh_offset) """ default library paths cannot be changed on the fly because """ """ the string lengths have to be changed too. """ - if name == b".sysdirs": + if name == b(".sysdirs"): sysdirs = f.read(sh_size) sysdirs_off = sh_offset sysdirs_sect_size = sh_size - elif name == b".sysdirslen": + elif name == b(".sysdirslen"): sysdirslen = f.read(sh_size) sysdirslen_off = sh_offset - elif name == b".ldsocache": + elif name == b(".ldsocache"): ldsocache_path = f.read(sh_size) new_ldsocache_path = old_prefix.sub(new_prefix, ldsocache_path) # pad with zeros - new_ldsocache_path += b"\0" * (sh_size - len(new_ldsocache_path)) + new_ldsocache_path += b("\0") * (sh_size - len(new_ldsocache_path)) # write it back f.seek(sh_offset) f.write(new_ldsocache_path) if sysdirs != "" and sysdirslen != "": - paths = sysdirs.split(b"\0") - sysdirs = b"" - sysdirslen = b"" + paths = sysdirs.split(b("\0")) + sysdirs = b("") + sysdirslen = b("") for path in paths: """ exit the loop when we encounter first empty string """ - if path == b"": + if path == b(""): break new_path = old_prefix.sub(new_prefix, path) - sysdirs += new_path + b"\0" + sysdirs += new_path + b("\0") if arch == 32: sysdirslen += struct.pack("<L", len(new_path)) @@ -173,7 +181,7 @@ def change_dl_sysdirs(): sysdirslen += struct.pack("<Q", len(new_path)) """ pad with zeros """ - sysdirs += b"\0" * (sysdirs_sect_size - len(sysdirs)) + sysdirs += b("\0") * (sysdirs_sect_size - len(sysdirs)) """ write the sections back """ f.seek(sysdirs_off) @@ -205,7 +213,8 @@ for e in executables_list: try: f = open(e, "r+b") - except IOError as ioex: + except IOError: + exctype, ioex = sys.exc_info()[:2] if ioex.errno == errno.ETXTBSY: print("Could not open %s. File used by another process.\nPlease "\ "make sure you exit all processes that might use any SDK "\ -- 1.8.2.1 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH] Allow script to work with Python 2.4 and 3. 2013-10-11 15:30 ` [PATCH] Allow script to work with Python 2.4 and 3 Konrad Scherer @ 2013-10-14 18:03 ` Saul Wold 2013-10-15 5:22 ` Laurentiu Palcu 0 siblings, 1 reply; 5+ messages in thread From: Saul Wold @ 2013-10-14 18:03 UTC (permalink / raw) To: Konrad Scherer, Laurentiu Palcu; +Cc: openembedded-core On 10/11/2013 08:30 AM, Konrad Scherer wrote: > From: Konrad Scherer <Konrad.Scherer@windriver.com> > > Python 2.4 does not support the 'b' string literal or the > keyword 'as' in exception handling. Python 3 does not accept > the old method of exception handling and defaults to unicode. > The b() function converts strings to bytes on Python 3 and > using sys.exc_info() avoids the exception handling syntax. > I would like to get Laurentiu's opinion on this. Also the patch synopsis should be in the format of <file/recipe>: synopsis In this case it should: relocate_sdk.py: Allow script to work with Python 2.4 and 3 Thanks Sau! > Signed-off-by: Konrad Scherer <Konrad.Scherer@windriver.com> > --- > scripts/relocate_sdk.py | 43 ++++++++++++++++++++++++++----------------- > 1 file changed, 26 insertions(+), 17 deletions(-) > > diff --git a/scripts/relocate_sdk.py b/scripts/relocate_sdk.py > index fe6e4e0..0971e63 100755 > --- a/scripts/relocate_sdk.py > +++ b/scripts/relocate_sdk.py > @@ -31,7 +31,15 @@ import os > import re > import errno > > -old_prefix = re.compile(b"##DEFAULT_INSTALL_DIR##") > +if sys.version < '3': > + def b(x): > + return x > +else: > + import codecs > + def b(x): > + return codecs.latin_1_encode(x)[0] > + > +old_prefix = re.compile(b("##DEFAULT_INSTALL_DIR##")) > > def get_arch(): > f.seek(0) > @@ -92,15 +100,15 @@ def change_interpreter(elf_file_name): > # External SDKs with mixed pre-compiled binaries should not get > # relocated so look for some variant of /lib > fname = f.read(11) > - if fname.startswith(b"/lib/") or fname.startswith(b"/lib64/") or \ > - fname.startswith(b"/lib32/") or fname.startswith(b"/usr/lib32/") or \ > - fname.startswith(b"/usr/lib32/") or fname.startswith(b"/usr/lib64/"): > + if fname.startswith(b("/lib/")) or fname.startswith(b("/lib64/")) or \ > + fname.startswith(b("/lib32/")) or fname.startswith(b("/usr/lib32/")) or \ > + fname.startswith(b("/usr/lib32/")) or fname.startswith(b("/usr/lib64/")): > break > if (len(new_dl_path) >= p_filesz): > print("ERROR: could not relocate %s, interp size = %i and %i is needed." \ > % (elf_file_name, p_memsz, len(new_dl_path) + 1)) > break > - dl_path = new_dl_path + b"\0" * (p_filesz - len(new_dl_path)) > + dl_path = new_dl_path + b("\0") * (p_filesz - len(new_dl_path)) > f.seek(p_offset) > f.write(dl_path) > break > @@ -132,40 +140,40 @@ def change_dl_sysdirs(): > sh_name, sh_type, sh_flags, sh_addr, sh_offset, sh_size, sh_link,\ > sh_info, sh_addralign, sh_entsize = struct.unpack(sh_fmt, sh_hdr) > > - name = sh_strtab[sh_name:sh_strtab.find(b"\0", sh_name)] > + name = sh_strtab[sh_name:sh_strtab.find(b("\0"), sh_name)] > > """ look only into SHT_PROGBITS sections """ > if sh_type == 1: > f.seek(sh_offset) > """ default library paths cannot be changed on the fly because """ > """ the string lengths have to be changed too. """ > - if name == b".sysdirs": > + if name == b(".sysdirs"): > sysdirs = f.read(sh_size) > sysdirs_off = sh_offset > sysdirs_sect_size = sh_size > - elif name == b".sysdirslen": > + elif name == b(".sysdirslen"): > sysdirslen = f.read(sh_size) > sysdirslen_off = sh_offset > - elif name == b".ldsocache": > + elif name == b(".ldsocache"): > ldsocache_path = f.read(sh_size) > new_ldsocache_path = old_prefix.sub(new_prefix, ldsocache_path) > # pad with zeros > - new_ldsocache_path += b"\0" * (sh_size - len(new_ldsocache_path)) > + new_ldsocache_path += b("\0") * (sh_size - len(new_ldsocache_path)) > # write it back > f.seek(sh_offset) > f.write(new_ldsocache_path) > > if sysdirs != "" and sysdirslen != "": > - paths = sysdirs.split(b"\0") > - sysdirs = b"" > - sysdirslen = b"" > + paths = sysdirs.split(b("\0")) > + sysdirs = b("") > + sysdirslen = b("") > for path in paths: > """ exit the loop when we encounter first empty string """ > - if path == b"": > + if path == b(""): > break > > new_path = old_prefix.sub(new_prefix, path) > - sysdirs += new_path + b"\0" > + sysdirs += new_path + b("\0") > > if arch == 32: > sysdirslen += struct.pack("<L", len(new_path)) > @@ -173,7 +181,7 @@ def change_dl_sysdirs(): > sysdirslen += struct.pack("<Q", len(new_path)) > > """ pad with zeros """ > - sysdirs += b"\0" * (sysdirs_sect_size - len(sysdirs)) > + sysdirs += b("\0") * (sysdirs_sect_size - len(sysdirs)) > > """ write the sections back """ > f.seek(sysdirs_off) > @@ -205,7 +213,8 @@ for e in executables_list: > > try: > f = open(e, "r+b") > - except IOError as ioex: > + except IOError: > + exctype, ioex = sys.exc_info()[:2] > if ioex.errno == errno.ETXTBSY: > print("Could not open %s. File used by another process.\nPlease "\ > "make sure you exit all processes that might use any SDK "\ > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] Allow script to work with Python 2.4 and 3. 2013-10-14 18:03 ` Saul Wold @ 2013-10-15 5:22 ` Laurentiu Palcu 2013-10-15 9:30 ` Burton, Ross 0 siblings, 1 reply; 5+ messages in thread From: Laurentiu Palcu @ 2013-10-15 5:22 UTC (permalink / raw) To: Saul Wold; +Cc: openembedded-core Hi, Somehow I missed this patch... See my comment below. On Mon, Oct 14, 2013 at 11:03:09AM -0700, Saul Wold wrote: > On 10/11/2013 08:30 AM, Konrad Scherer wrote: > >From: Konrad Scherer <Konrad.Scherer@windriver.com> > > > >Python 2.4 does not support the 'b' string literal or the > >keyword 'as' in exception handling. Python 3 does not accept > >the old method of exception handling and defaults to unicode. > >The b() function converts strings to bytes on Python 3 and > >using sys.exc_info() avoids the exception handling syntax. > > > I would like to get Laurentiu's opinion on this. > > Also the patch synopsis should be in the format of > <file/recipe>: synopsis > > In this case it should: > > relocate_sdk.py: Allow script to work with Python 2.4 and 3 > > Thanks > > Sau! > > >Signed-off-by: Konrad Scherer <Konrad.Scherer@windriver.com> > >--- > > scripts/relocate_sdk.py | 43 ++++++++++++++++++++++++++----------------- > > 1 file changed, 26 insertions(+), 17 deletions(-) > > > >diff --git a/scripts/relocate_sdk.py b/scripts/relocate_sdk.py > >index fe6e4e0..0971e63 100755 > >--- a/scripts/relocate_sdk.py > >+++ b/scripts/relocate_sdk.py > >@@ -31,7 +31,15 @@ import os > > import re > > import errno > > > >-old_prefix = re.compile(b"##DEFAULT_INSTALL_DIR##") > >+if sys.version < '3': > >+ def b(x): > >+ return x > >+else: > >+ import codecs > >+ def b(x): > >+ return codecs.latin_1_encode(x)[0] Is it wise to encode the strings with latin_1 by default? Would it work if the filename uses cyrillic characters? Or chinese? Python3, by default, uses the system's default encoding. But, if we want to be sure that the system encoding matches filesystem's, I think it would suffice to write b(x) like below: def b(x): return x.encode(sys.getfilesystemencoding()) What do you say? Would this work for you? Other than that, looks good to me. Thanks, Laurentiu > >+ > >+old_prefix = re.compile(b("##DEFAULT_INSTALL_DIR##")) > > > > def get_arch(): > > f.seek(0) > >@@ -92,15 +100,15 @@ def change_interpreter(elf_file_name): > > # External SDKs with mixed pre-compiled binaries should not get > > # relocated so look for some variant of /lib > > fname = f.read(11) > >- if fname.startswith(b"/lib/") or fname.startswith(b"/lib64/") or \ > >- fname.startswith(b"/lib32/") or fname.startswith(b"/usr/lib32/") or \ > >- fname.startswith(b"/usr/lib32/") or fname.startswith(b"/usr/lib64/"): > >+ if fname.startswith(b("/lib/")) or fname.startswith(b("/lib64/")) or \ > >+ fname.startswith(b("/lib32/")) or fname.startswith(b("/usr/lib32/")) or \ > >+ fname.startswith(b("/usr/lib32/")) or fname.startswith(b("/usr/lib64/")): > > break > > if (len(new_dl_path) >= p_filesz): > > print("ERROR: could not relocate %s, interp size = %i and %i is needed." \ > > % (elf_file_name, p_memsz, len(new_dl_path) + 1)) > > break > >- dl_path = new_dl_path + b"\0" * (p_filesz - len(new_dl_path)) > >+ dl_path = new_dl_path + b("\0") * (p_filesz - len(new_dl_path)) > > f.seek(p_offset) > > f.write(dl_path) > > break > >@@ -132,40 +140,40 @@ def change_dl_sysdirs(): > > sh_name, sh_type, sh_flags, sh_addr, sh_offset, sh_size, sh_link,\ > > sh_info, sh_addralign, sh_entsize = struct.unpack(sh_fmt, sh_hdr) > > > >- name = sh_strtab[sh_name:sh_strtab.find(b"\0", sh_name)] > >+ name = sh_strtab[sh_name:sh_strtab.find(b("\0"), sh_name)] > > > > """ look only into SHT_PROGBITS sections """ > > if sh_type == 1: > > f.seek(sh_offset) > > """ default library paths cannot be changed on the fly because """ > > """ the string lengths have to be changed too. """ > >- if name == b".sysdirs": > >+ if name == b(".sysdirs"): > > sysdirs = f.read(sh_size) > > sysdirs_off = sh_offset > > sysdirs_sect_size = sh_size > >- elif name == b".sysdirslen": > >+ elif name == b(".sysdirslen"): > > sysdirslen = f.read(sh_size) > > sysdirslen_off = sh_offset > >- elif name == b".ldsocache": > >+ elif name == b(".ldsocache"): > > ldsocache_path = f.read(sh_size) > > new_ldsocache_path = old_prefix.sub(new_prefix, ldsocache_path) > > # pad with zeros > >- new_ldsocache_path += b"\0" * (sh_size - len(new_ldsocache_path)) > >+ new_ldsocache_path += b("\0") * (sh_size - len(new_ldsocache_path)) > > # write it back > > f.seek(sh_offset) > > f.write(new_ldsocache_path) > > > > if sysdirs != "" and sysdirslen != "": > >- paths = sysdirs.split(b"\0") > >- sysdirs = b"" > >- sysdirslen = b"" > >+ paths = sysdirs.split(b("\0")) > >+ sysdirs = b("") > >+ sysdirslen = b("") > > for path in paths: > > """ exit the loop when we encounter first empty string """ > >- if path == b"": > >+ if path == b(""): > > break > > > > new_path = old_prefix.sub(new_prefix, path) > >- sysdirs += new_path + b"\0" > >+ sysdirs += new_path + b("\0") > > > > if arch == 32: > > sysdirslen += struct.pack("<L", len(new_path)) > >@@ -173,7 +181,7 @@ def change_dl_sysdirs(): > > sysdirslen += struct.pack("<Q", len(new_path)) > > > > """ pad with zeros """ > >- sysdirs += b"\0" * (sysdirs_sect_size - len(sysdirs)) > >+ sysdirs += b("\0") * (sysdirs_sect_size - len(sysdirs)) > > > > """ write the sections back """ > > f.seek(sysdirs_off) > >@@ -205,7 +213,8 @@ for e in executables_list: > > > > try: > > f = open(e, "r+b") > >- except IOError as ioex: > >+ except IOError: > >+ exctype, ioex = sys.exc_info()[:2] > > if ioex.errno == errno.ETXTBSY: > > print("Could not open %s. File used by another process.\nPlease "\ > > "make sure you exit all processes that might use any SDK "\ > > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] Allow script to work with Python 2.4 and 3. 2013-10-15 5:22 ` Laurentiu Palcu @ 2013-10-15 9:30 ` Burton, Ross 0 siblings, 0 replies; 5+ messages in thread From: Burton, Ross @ 2013-10-15 9:30 UTC (permalink / raw) To: Laurentiu Palcu; +Cc: OE-core On 15 October 2013 06:22, Laurentiu Palcu <laurentiu.palcu@intel.com> wrote: > Is it wise to encode the strings with latin_1 by default? Would it work > if the filename uses cyrillic characters? Or chinese? Especially not considering every modern distro defaults to UTF-8 locales now. Using the system encoding makes sense. Ross ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2013-10-15 9:30 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-10-11 15:30 [Patch 0/1] Update relocate_sdk.py to work with Python 2.4 Konrad Scherer 2013-10-11 15:30 ` [PATCH] Allow script to work with Python 2.4 and 3 Konrad Scherer 2013-10-14 18:03 ` Saul Wold 2013-10-15 5:22 ` Laurentiu Palcu 2013-10-15 9:30 ` Burton, Ross
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.