* [Buildroot] [PATCH v3 1/4] support/scripts: add fix_rpaths
@ 2015-07-13 10:17 Samuel Martin
2015-07-13 10:17 ` [Buildroot] [PATCH v3 2/4] Makefile: add HOST_FIX_RPATH_HOOK to TARGET_FINALIZE_HOOKS Samuel Martin
` (3 more replies)
0 siblings, 4 replies; 7+ messages in thread
From: Samuel Martin @ 2015-07-13 10:17 UTC (permalink / raw)
To: buildroot
This pyhton script leverages patchelf program to fix the RPATH of binaries.
It offers 2 actions:
- clear the RPATH;
- set the RPATH using relative paths between every single binary and the
libraries directories.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v2->v3:
- no change
---
support/scripts/fix_rpaths | 302 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 302 insertions(+)
create mode 100755 support/scripts/fix_rpaths
diff --git a/support/scripts/fix_rpaths b/support/scripts/fix_rpaths
new file mode 100755
index 0000000..1c840e0
--- /dev/null
+++ b/support/scripts/fix_rpaths
@@ -0,0 +1,302 @@
+#!/usr/bin/env python
+##
+## Author(s):
+## - Samuel Martin <s.martin49@gmail.com>
+##
+## Copyright (C) 2013 Samuel Martin
+##
+## This program is free software; you can redistribute it and/or modify
+## it under the terms of the GNU General Public License as published by
+## the Free Software Foundation; either version 2 of the License, or
+## (at your option) any later version.
+##
+## This program is distributed in the hope that it will be useful,
+## but WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+## GNU General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with this program; if not, write to the Free Software
+## Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+##
+""" This script scans a direcotry for EFL files and fix their RPATH, making
+them relative.
+"""
+
+from __future__ import print_function, with_statement
+import os
+import stat
+import subprocess
+import sys
+
+PATCHELF_PROGRAM = "patchelf"
+
+
+# pylint: disable=too-few-public-methods
+class PreservedTime(object):
+ """ With-class ensuring the file's times are preserved
+
+ :param path: File path
+ """
+ # pylint: disable=redefined-outer-name
+ def __init__(self, path):
+ self.path = path
+ self.times = None
+
+ # pylint: disable=invalid-name
+ def __enter__(self):
+ st = os.lstat(self.path)
+ self.times = (st.st_atime, st.st_mtime)
+ return self.path
+
+ # pylint: disable=redefined-builtin
+ def __exit__(self, type, value, traceback):
+ os.utime(self.path, self.times)
+
+
+# pylint: disable=redefined-outer-name
+def is_elf_binary(path):
+ """ Return True if path points to a valid ELF file.
+
+ :param path: File path
+ """
+ if not stat.S_ISREG(os.lstat(path).st_mode):
+ return False
+ with PreservedTime(path):
+ # pylint: disable=invalid-name
+ with open(path, "rb") as fp:
+ data = fp.read(4)
+ return data == b"\x7fELF"
+
+
+def has_rpath(elf_file, patchelf_bin=PATCHELF_PROGRAM):
+ """ Return True if the given ELF file accept a RPATH.
+
+ :param elf_file: ELF file path
+ :param patchelf_bin: patchelf program path
+ """
+ cmd = [patchelf_bin, "--print-rpath", elf_file]
+ with PreservedTime(elf_file):
+ try:
+ subprocess.check_call(cmd, stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE)
+ elf_with_rpath = True
+ except subprocess.CalledProcessError as _:
+ elf_with_rpath = False
+ return elf_with_rpath
+
+
+def compute_rpath(binary, libdirs):
+ """ Return the RPATH value (with relative paths to the given libdirs).
+
+ :param binary: ELF binary path
+ :param libdirs: List of library directory paths
+ """
+ bindir = os.path.dirname(binary)
+ relpaths = [os.path.relpath(libdir, bindir) for libdir in libdirs]
+ # reduce the list, but keep its original order
+ # pylint: disable=unnecessary-lambda
+ sorted(set(relpaths), key=lambda x: relpaths.index(x))
+ rpaths = [os.path.join("$ORIGIN", relpath) for relpath in relpaths]
+ rpath = ":".join(rpaths)
+ return rpath
+
+
+def fix_rpath(elf_file, rpath, patchelf_bin=PATCHELF_PROGRAM):
+ """ Fix the ELF file RPATH.
+
+ :param elf_file: ELF file patch
+ :param rpath: New RPATH value
+ :param patchelf_bin: patchelf program path
+ """
+ cmd = [patchelf_bin, "--set-rpath", rpath, elf_file]
+ with PreservedTime(elf_file):
+ subprocess.check_call(cmd, stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE)
+
+
+def shrink_rpath(elf_file, patchelf_bin=PATCHELF_PROGRAM):
+ """ Shrink the ELF file's RPATH.
+
+ :param elf_file: ELF file patch
+ :param patchelf_bin: patchelf program path
+ """
+ cmd = [patchelf_bin, "--shrink-rpath", elf_file]
+ with PreservedTime(elf_file):
+ subprocess.check_call(cmd, stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE)
+ print("RPATH cleared: %s" % elf_file)
+
+
+def dump_rpath(elf_file, patchelf_bin=PATCHELF_PROGRAM):
+ """ Return the ELF file's RPATH.
+
+ :param elf_file: ELF file patch
+ :param patchelf_bin: patchelf program path
+ """
+ cmd = [patchelf_bin, "--print-rpath", elf_file]
+ with PreservedTime(elf_file):
+ proc = subprocess.Popen(cmd, stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE)
+ proc.wait()
+ rpath = proc.communicate()[0].strip()
+ if sys.version_info.major >= 3:
+ def _decode(txt):
+ """ Decode function """
+ return txt.decode()
+ else:
+ def _decode(txt):
+ """ Decode function """
+ return txt
+ return _decode(rpath)
+
+
+def update_rpath(elf_file, libdirs, patchelf_bin=PATCHELF_PROGRAM):
+ """ Return the ELF file's RPATH.
+
+ :param elf_file: ELF file patch
+ :param libdirs: List of library directory paths
+ :param patchelf_bin: patchelf program path
+ """
+ rpath = compute_rpath(elf_file, libdirs)
+ fix_rpath(elf_file, rpath, patchelf_bin=patchelf_bin)
+ print("RPATH set: %s \tRPATH='%s'" % (elf_file,
+ dump_rpath(elf_file, patchelf_bin)))
+
+
+def find_files(root, file_filter_func=None, exclude_dirs=None):
+ """ Generator returning files from the root location.
+
+ :param root: Root path to be scan
+ :param file_filter_func: Filter function returning a boolean whether the
+ file path should be yielded or not
+ :param exclude_dirs: List of directories to be prune from the scan
+ """
+ def dummy_filter(_):
+ """ Dummy filter function. Always return True.
+ """
+ return True
+ if not file_filter_func:
+ file_filter_func = dummy_filter
+ for parent, dirs, files in os.walk(root):
+ for xdir in exclude_dirs:
+ if xdir in dirs:
+ del dirs[dirs.index(xdir)]
+ continue
+ for idx, a_dir in enumerate(dirs):
+ if os.path.join(parent, a_dir).endswith(xdir):
+ del dirs[idx]
+ continue
+ for a_file in files:
+ full_path = os.path.join(parent, a_file)
+ if not file_filter_func(full_path):
+ continue
+ yield full_path
+
+
+def scan_and_apply(root, rpath_func, exclude_dirs=None,
+ patchelf_bin=PATCHELF_PROGRAM):
+ """ Scan and update RPATH on ELF files under the root location.
+
+ The new RPATH value is computed from the binaries's and the libraries
+ directories.
+
+ :param root: Root path to be scan
+ :param libdirs: List of library directory paths
+ :param patchelf_bin: patchelf program path
+ """
+ def file_filter(path):
+ """ Return True if the path points to a valid ELF file accepting RPATH.
+ """
+ # check for valid file (discard non-ELF files and broken symlinks)
+ if not is_elf_binary(path):
+ return False
+ return has_rpath(path)
+ exclude_dirs = exclude_dirs if exclude_dirs else list()
+ for filepath in find_files(root, file_filter_func=file_filter,
+ exclude_dirs=exclude_dirs):
+ rpath_func(filepath, patchelf_bin=patchelf_bin)
+
+
+def main():
+ """ Main function
+ """
+ import argparse
+ parser = argparse.ArgumentParser(description="""\
+ Update the RPATH in all EFL files in ROOT.
+
+ It can perform 2 types of actions on the EFL files, preserving
+ their times:
+ 1) 'set' the RPATH, with relative paths between ELF files and
+ the library directories;
+ 2) or 'clear' the RPATH.
+
+ """)
+ parser.add_argument("action", choices=["set", "clear"],
+ help="""\
+ Action processed on RPATH.
+ 'set' updates the RPATH with relative path between each binary and
+ library directories passed via the required --libdirs option.
+ 'clear' empties the RPATH of the binaries
+ """)
+ parser.add_argument("rootdir", metavar="ROOT",
+ help="Root path to scan for RPATH fixup")
+ parser.add_argument("--libdirs", nargs="+", default=list(),
+ help="""\
+ List of library directory paths (must be sub-location of ROOT)""")
+ parser.add_argument("--exclude-dirs", nargs="+", default=list(),
+ help="List of directories skipped for RPATH update")
+ parser.add_argument("--patchelf-program", dest="patchelf_bin",
+ default=PATCHELF_PROGRAM,
+ help="Path to patchelf program to be used")
+ args = parser.parse_args()
+ # sanitizing arguments
+ action = args.action
+ root = os.path.abspath(args.rootdir)
+ libdirs = [os.path.abspath(l) for l in args.libdirs if os.path.isdir(l)]
+ exclude_dirs = [x for x in args.exclude_dirs]
+ patchelf_bin = os.path.abspath(args.patchelf_bin)
+ # sanity checks
+ if action == "set" and not libdirs:
+ msg = "\nERROR: Setting RPATH requires non-empty --libdirs option\n\n"
+ msg += parser.format_help()
+ raise Exception(msg)
+ if not os.path.exists(root):
+ msg = "\nERROR: ROOT must be an existing path.\n"
+ msg += "\troot: %s\n\n" % root
+ msg += parser.format_help()
+ raise Exception(msg)
+ for libdir in libdirs:
+ if not libdir.startswith(root):
+ msg = "\nERROR: each libdirs must be under the root location.\n"
+ msg += "\troot: %s\n" % root
+ msg += "\tfaulty libdir: %s\n\n" % libdir
+ msg += parser.format_help()
+ raise Exception(msg)
+ if not os.path.exists(patchelf_bin):
+ patchelf_found = False
+ for path in os.environ.get("PATH", "").split(":"):
+ if not path:
+ continue
+ if PATCHELF_PROGRAM in os.listdir(path):
+ patchelf_found = True
+ break
+ if not patchelf_found:
+ msg = "\nERROR: no '%s' program found on the host system.\n\n" % \
+ PATCHELF_PROGRAM
+ msg += parser.format_help()
+ raise Exception(msg)
+ if args.action == "set":
+ def set_rpath(elf_file, patchelf_bin=PATCHELF_PROGRAM):
+ """ Set RPATH
+ """
+ return update_rpath(elf_file, libdirs, patchelf_bin=patchelf_bin)
+ action = set_rpath
+ elif args.action == "clear":
+ action = shrink_rpath
+ scan_and_apply(root, action, exclude_dirs=exclude_dirs,
+ patchelf_bin=args.patchelf_bin)
+
+
+if __name__ == "__main__":
+ main()
--
2.4.5
^ permalink raw reply related [flat|nested] 7+ messages in thread* [Buildroot] [PATCH v3 2/4] Makefile: add HOST_FIX_RPATH_HOOK to TARGET_FINALIZE_HOOKS
2015-07-13 10:17 [Buildroot] [PATCH v3 1/4] support/scripts: add fix_rpaths Samuel Martin
@ 2015-07-13 10:17 ` Samuel Martin
2015-07-13 10:28 ` Baruch Siach
2015-07-13 10:17 ` [Buildroot] [PATCH v3 3/4] Makefile: add TARGET_CLEAR_RPATH_HOOK " Samuel Martin
` (2 subsequent siblings)
3 siblings, 1 reply; 7+ messages in thread
From: Samuel Martin @ 2015-07-13 10:17 UTC (permalink / raw)
To: buildroot
This patch adds host-patchelf as a target-finalize dependency, and
introduces the HOST_FIX_RPATH_HOOK hook fixing the ELF files' RPATH of
the host tree (excluding the sysroot).
After running this hook, the RPATH from any host ELF files is relative to
the binary location itself.
Note that, we avoid to fix RPATH in the sysroot, and in the external
toolcahin installation location.
As a first step toward a fully relocatable SDK, this change allows to get
the toolchain relocatable, but not yet the whole SDK.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v2->v3:
- move hook in Makefile (Baruch)
---
Makefile | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/Makefile b/Makefile
index daf692e..ac387ea 100644
--- a/Makefile
+++ b/Makefile
@@ -544,6 +544,18 @@ endif
$(TARGETS_ROOTFS): target-finalize
+target-finalize: host-patchelf
+
+define HOST_FIX_RPATH_HOOK
+ $(TOPDIR)/support/scripts/fix_rpaths \
+ set $(HOST_DIR) \
+ --patchelf-program $(HOST_DIR)/usr/bin/patchelf \
+ --libdirs $(HOST_DIR)/usr/lib \
+ --exclude-dirs sysroot opt/ext-toolchain
+endef
+
+TARGET_FINALIZE_HOOKS += HOST_FIX_RPATH_HOOK
+
target-finalize: $(PACKAGES)
@$(call MESSAGE,"Finalizing target directory")
$(foreach hook,$(TARGET_FINALIZE_HOOKS),$($(hook))$(sep))
--
2.4.5
^ permalink raw reply related [flat|nested] 7+ messages in thread* [Buildroot] [PATCH v3 2/4] Makefile: add HOST_FIX_RPATH_HOOK to TARGET_FINALIZE_HOOKS
2015-07-13 10:17 ` [Buildroot] [PATCH v3 2/4] Makefile: add HOST_FIX_RPATH_HOOK to TARGET_FINALIZE_HOOKS Samuel Martin
@ 2015-07-13 10:28 ` Baruch Siach
0 siblings, 0 replies; 7+ messages in thread
From: Baruch Siach @ 2015-07-13 10:28 UTC (permalink / raw)
To: buildroot
Hi Samuel,
On Mon, Jul 13, 2015 at 12:17:55PM +0200, Samuel Martin wrote:
> This patch adds host-patchelf as a target-finalize dependency, and
> introduces the HOST_FIX_RPATH_HOOK hook fixing the ELF files' RPATH of
> the host tree (excluding the sysroot).
>
> After running this hook, the RPATH from any host ELF files is relative to
> the binary location itself.
>
> Note that, we avoid to fix RPATH in the sysroot, and in the external
> toolcahin installation location.
>
> As a first step toward a fully relocatable SDK, this change allows to get
> the toolchain relocatable, but not yet the whole SDK.
>
> Signed-off-by: Samuel Martin <s.martin49@gmail.com>
>
> ---
> changes v2->v3:
> - move hook in Makefile (Baruch)
> ---
> Makefile | 12 ++++++++++++
> 1 file changed, 12 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index daf692e..ac387ea 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -544,6 +544,18 @@ endif
>
> $(TARGETS_ROOTFS): target-finalize
>
> +target-finalize: host-patchelf
Instead of duplicating the target-finalize target, why not add host-patchelf
to $(PACKAGES) like we already do for host-localedef?
> +
> +define HOST_FIX_RPATH_HOOK
> + $(TOPDIR)/support/scripts/fix_rpaths \
> + set $(HOST_DIR) \
> + --patchelf-program $(HOST_DIR)/usr/bin/patchelf \
> + --libdirs $(HOST_DIR)/usr/lib \
> + --exclude-dirs sysroot opt/ext-toolchain
> +endef
> +
> +TARGET_FINALIZE_HOOKS += HOST_FIX_RPATH_HOOK
> +
> target-finalize: $(PACKAGES)
> @$(call MESSAGE,"Finalizing target directory")
> $(foreach hook,$(TARGET_FINALIZE_HOOKS),$($(hook))$(sep))
--
http://baruch.siach.name/blog/ ~. .~ Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
- baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Buildroot] [PATCH v3 3/4] Makefile: add TARGET_CLEAR_RPATH_HOOK to TARGET_FINALIZE_HOOKS
2015-07-13 10:17 [Buildroot] [PATCH v3 1/4] support/scripts: add fix_rpaths Samuel Martin
2015-07-13 10:17 ` [Buildroot] [PATCH v3 2/4] Makefile: add HOST_FIX_RPATH_HOOK to TARGET_FINALIZE_HOOKS Samuel Martin
@ 2015-07-13 10:17 ` Samuel Martin
2015-07-13 10:17 ` [Buildroot] [PATCH v3 4/4] Makefile: staging symlink uses a relative path Samuel Martin
2015-07-13 10:24 ` [Buildroot] [PATCH v3 1/4] support/scripts: add fix_rpaths Baruch Siach
3 siblings, 0 replies; 7+ messages in thread
From: Samuel Martin @ 2015-07-13 10:17 UTC (permalink / raw)
To: buildroot
This patch introduces the TARGET_CLEAR_RPATH_HOOK hook fixing the ELF
files' RPATH of the host tree (excluding the sysroot).
After running this hook, the RPATH from any target ELF files is empty.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v2->v3:
- move hook in Makefile (Baruch)
---
Makefile | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/Makefile b/Makefile
index ac387ea..9ae9afb 100644
--- a/Makefile
+++ b/Makefile
@@ -556,6 +556,14 @@ endef
TARGET_FINALIZE_HOOKS += HOST_FIX_RPATH_HOOK
+define TARGET_CLEAR_RPATH_HOOK
+ $(TOPDIR)/support/scripts/fix_rpaths \
+ clear $(TARGET_DIR) \
+ --patchelf-program $(HOST_DIR)/usr/bin/patchelf
+endef
+
+TARGET_FINALIZE_HOOKS += TARGET_CLEAR_RPATH_HOOK
+
target-finalize: $(PACKAGES)
@$(call MESSAGE,"Finalizing target directory")
$(foreach hook,$(TARGET_FINALIZE_HOOKS),$($(hook))$(sep))
--
2.4.5
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [Buildroot] [PATCH v3 4/4] Makefile: staging symlink uses a relative path
2015-07-13 10:17 [Buildroot] [PATCH v3 1/4] support/scripts: add fix_rpaths Samuel Martin
2015-07-13 10:17 ` [Buildroot] [PATCH v3 2/4] Makefile: add HOST_FIX_RPATH_HOOK to TARGET_FINALIZE_HOOKS Samuel Martin
2015-07-13 10:17 ` [Buildroot] [PATCH v3 3/4] Makefile: add TARGET_CLEAR_RPATH_HOOK " Samuel Martin
@ 2015-07-13 10:17 ` Samuel Martin
2015-07-13 10:24 ` [Buildroot] [PATCH v3 1/4] support/scripts: add fix_rpaths Baruch Siach
3 siblings, 0 replies; 7+ messages in thread
From: Samuel Martin @ 2015-07-13 10:17 UTC (permalink / raw)
To: buildroot
A step forward toward a relocatable SDK.
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
---
changes v2-v3:
- no change
---
Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Makefile b/Makefile
index 9ae9afb..2ca0ef2 100644
--- a/Makefile
+++ b/Makefile
@@ -447,7 +447,7 @@ $(STAGING_DIR):
@ln -snf lib $(STAGING_DIR)/usr/$(LIB_SYMLINK)
@mkdir -p $(STAGING_DIR)/usr/include
@mkdir -p $(STAGING_DIR)/usr/bin
- @ln -snf $(STAGING_DIR) $(BASE_DIR)/staging
+ @ln -snf $(subst $(BASE_DIR)/,,$(STAGING_DIR)) $(BASE_DIR)/staging
ifeq ($(BR2_ROOTFS_SKELETON_CUSTOM),y)
TARGET_SKELETON = $(BR2_ROOTFS_SKELETON_CUSTOM_PATH)
--
2.4.5
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [Buildroot] [PATCH v3 1/4] support/scripts: add fix_rpaths
2015-07-13 10:17 [Buildroot] [PATCH v3 1/4] support/scripts: add fix_rpaths Samuel Martin
` (2 preceding siblings ...)
2015-07-13 10:17 ` [Buildroot] [PATCH v3 4/4] Makefile: staging symlink uses a relative path Samuel Martin
@ 2015-07-13 10:24 ` Baruch Siach
2015-07-13 10:31 ` Samuel Martin
3 siblings, 1 reply; 7+ messages in thread
From: Baruch Siach @ 2015-07-13 10:24 UTC (permalink / raw)
To: buildroot
Hi Samuel,
On Mon, Jul 13, 2015 at 12:17:54PM +0200, Samuel Martin wrote:
> This pyhton script leverages patchelf program to fix the RPATH of binaries.
>
> It offers 2 actions:
> - clear the RPATH;
> - set the RPATH using relative paths between every single binary and the
> libraries directories.
>
> Signed-off-by: Samuel Martin <s.martin49@gmail.com>
>
> ---
> changes v2->v3:
> - no change
Have you seen my comments on v2 at
http://lists.busybox.net/pipermail/buildroot/2015-July/132446.html?
baruch
--
http://baruch.siach.name/blog/ ~. .~ Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
- baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2015-07-13 10:31 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-07-13 10:17 [Buildroot] [PATCH v3 1/4] support/scripts: add fix_rpaths Samuel Martin
2015-07-13 10:17 ` [Buildroot] [PATCH v3 2/4] Makefile: add HOST_FIX_RPATH_HOOK to TARGET_FINALIZE_HOOKS Samuel Martin
2015-07-13 10:28 ` Baruch Siach
2015-07-13 10:17 ` [Buildroot] [PATCH v3 3/4] Makefile: add TARGET_CLEAR_RPATH_HOOK " Samuel Martin
2015-07-13 10:17 ` [Buildroot] [PATCH v3 4/4] Makefile: staging symlink uses a relative path Samuel Martin
2015-07-13 10:24 ` [Buildroot] [PATCH v3 1/4] support/scripts: add fix_rpaths Baruch Siach
2015-07-13 10:31 ` Samuel Martin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox