[Buildroot] [PATCH v6 00/10] Make the SDK relocatable

[Buildroot] [PATCH v6 00/10] Make the SDK relocatable

[Wolfgang Grandegger]

Hello,

this is v6 of my patch series to make the buildroot SDK (HOST_DIR)
relocatable. It sanitizes the RPATH of all ELF files in the "target",
"staging" and "host" tree using "patchelf --make-rpath-relative". We
now use patchelf v0.9 to still support old Debian and RHEL systems.

v5 did RPATH sanitization per package after package installation into
the host, staging or target tree using GLOBAL_INSTRUMENTATION_HOOKS.
This approach got more and more complex and inefficient. Therefore
it was abandoned in favor of global sanititation at the end of the
host, staging and target build (see changes since v5).

Furthermore this patch creates the script "relocate-sdk.sh" in the top
directory of the "host" tree allowing to relocate the SDK after it has
been moved to a new location. It replaces the old path with the new
one in all text files identified by "file --mime-type". The location
is stored in "usr/share/buildroot/sdk-location".

Unfortunately, "qmake" uses hard-coded pathes compiled into the QT5
libraries. To overcome this problem, "qt5pase" now creates "qt.conf".

Hope I have not too much pending issues.

Wolfgang.

Things not yet addressed:

- "make toolchain" creates a toolchain tree which still has references
  to the build system (in ELF and text files).

Changes since v5:
- switch back to v4
- patchelf patches are now based on v0.9
- patchelf now calculates minimal relative path to the ELF file
  (and not to the rootdir as before).
- patchelf: neededLibFound is now passed to libFoundInRPath
- Makefile: the "host-finalize" target has been added for SDK relocation
- fix-rpath now uses variables to define the list of dirs to be pruned
- fix-rpath: the staging tree is sanitized like the target tree
- various minor fixes (typos)

Changes since v4:

- RPATH sanitation is now done per package and installation step
  in "pkg-generic.mk".
- After the installation step, the list of the installed files is
  stored in "<package-build-dir>/.br_[host|staging|target]_filelist".
- The GLOBAL_INSTRUMENTATION_HOOKS "step_sanitize_rpath" then calls
  "fix-rpath" to do the sanitation.
- DEPENDENCIES_HOST_PREREQ += host-patchelf is set early in the
  Makefile as we need it for RPATH sanitation. As soon it's available
  sanitation can start (currently missing some packages).
- The patchelf "file busy" issue is now worked-around differently.

Changes since v3:

- The patchelf patch implementing " --make-rpath-relative" now supports
  the option "--relative-to-file" instructing to use "$ORIGIN" in
  RPATHs. Otherwise an absolute path relative to the root directory will
  be used.
- The staging tree is now sanitized as well using the options
  "--relative-to-file" and "--no-standard-libs".
- For the "target" tree, relative RPATHs do not use "$ORIGIN" any
  longer. An absolute path relative to the root directory is used
  instead.

Changes since v2:

- provide "qt.conf" to make "qmake" relocatable
- sed now uses the separator "\" to substitute the directory path.
  It's one of the few characters not allowed in file names. To
  avoid interpreting it as escape character, the "read -r" is used.
- The paranoia substituion check is done before doing the real
  substituion.

Changes since v1:

- The name SDK has been chosen for the relocatabed "HOST_DIR" (instead
  of toolchanin).
- The patchelf version bump and patching are now done by 2 patches
- No more helper functions are used in the Makefile to call "fix-rpath"
  but added directly.
- The staging tree is not touched any more... until we have a good
  reason to do so. 
- The sanitation is now performed by an optimized "fix-rpath" script.
- The relocate-sdk script is now copied for support/misc to the
  top directory of the host tree.

Samuel Martin (1):
  support/scripts: add fix-rpath script to sanitize the rpath

Wolfgang Grandegger (9):
  support/scripts: check-host-rpath now handles $ORIGIN/../lib
  package/patchelf: add patch for rpath sanitization under a root
    directory
  core: sanitize RPATH in staging tree at the end of target finalization
  core: sanitize RPATH in target tree at the end of target finalization
  core: sanitize RPATH in host tree at the very end of the build
  support/scripts: add relocate-sdk.sh script for SDK relocation
  core: install relocation script and location at the end of the build
  external-toolchain: check if a buildroot SDK has already been
    relocated
  package/qt5base: provide "qt.conf" to make "qmake" relocatable

 Makefile                                           |  17 +-
 ...move-apparently-incorrect-usage-of-static.patch |  49 +++
 ...unction-for-splitting-a-colon-separated-s.patch |  58 +++
 ...to-make-the-rpath-relative-under-a-specif.patch | 419 +++++++++++++++++++++
 package/qt5/qt5base/qt.conf.in                     |   6 +
 package/qt5/qt5base/qt5base.mk                     |   8 +
 support/misc/relocate-sdk.sh                       |  47 +++
 support/scripts/check-host-rpath                   |   2 +-
 support/scripts/fix-rpath                          | 122 ++++++
 toolchain/helpers.mk                               |  18 +
 .../toolchain-external/pkg-toolchain-external.mk   |   1 +
 11 files changed, 745 insertions(+), 2 deletions(-)
 create mode 100644 package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch
 create mode 100644 package/patchelf/0002-Extract-a-function-for-splitting-a-colon-separated-s.patch
 create mode 100644 package/patchelf/0003-Add-option-to-make-the-rpath-relative-under-a-specif.patch
 create mode 100644 package/qt5/qt5base/qt.conf.in
 create mode 100755 support/misc/relocate-sdk.sh
 create mode 100755 support/scripts/fix-rpath

-- 
2.7.4

[Buildroot] [PATCH v6 01/10] support/scripts: check-host-rpath now handles $ORIGIN/../lib

[Wolfgang Grandegger]

The new RPath sanitization uses now the relative path
"$ORIGIN/../lib" for binaries in "/usr/bin".

Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
---
 support/scripts/check-host-rpath | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/support/scripts/check-host-rpath b/support/scripts/check-host-rpath
index 020c123..e79560b 100755
--- a/support/scripts/check-host-rpath
+++ b/support/scripts/check-host-rpath
@@ -58,7 +58,7 @@ check_elf_has_rpath() {
         for dir in ${rpath//:/ }; do
             # Remove duplicate and trailing '/' for proper match
             dir="$( sed -r -e 's:/+:/:g; s:/$::;' <<<"${dir}" )"
-            [ "${dir}" = "${hostdir}/usr/lib" -o "${dir}" = "\$ORIGIN/../../usr/lib" ] && return 0
+            [ "${dir}" = "${hostdir}/usr/lib" -o "${dir}" = "\$ORIGIN/../lib" ] && return 0
         done
     done < <( readelf -d "${file}"                                              \
               |sed -r -e '/.* \(R(UN)?PATH\) +Library r(un)?path: \[(.+)\]$/!d' \
-- 
2.7.4

[Buildroot] [PATCH v6 02/10] package/patchelf: add patch for rpath sanitization under a root directory

[Wolfgang Grandegger]

The patch allows to use patchelf to sanitize the rpath of the buildroot
libraries and binaries using the option "--make-rpath-relative <rootdir>".
Recent versions of patchelf will not built on old Debian and RHEL systems
due to C++11 constructs. Therefore we stick with v0.9 for the time being.

Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
---
 ...move-apparently-incorrect-usage-of-static.patch |  49 +++
 ...unction-for-splitting-a-colon-separated-s.patch |  58 +++
 ...to-make-the-rpath-relative-under-a-specif.patch | 419 +++++++++++++++++++++
 3 files changed, 526 insertions(+)
 create mode 100644 package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch
 create mode 100644 package/patchelf/0002-Extract-a-function-for-splitting-a-colon-separated-s.patch
 create mode 100644 package/patchelf/0003-Add-option-to-make-the-rpath-relative-under-a-specif.patch

diff --git a/package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch b/package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch
new file mode 100644
index 0000000..46fd123
--- /dev/null
+++ b/package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch
@@ -0,0 +1,49 @@
+From a365bcb7d7025da51b33165ef7ebc7180199a05e Mon Sep 17 00:00:00 2001
+From: Eelco Dolstra <eelco.dolstra@logicblox.com>
+Date: Mon, 19 Sep 2016 17:31:37 +0200
+Subject: [PATCH 14/30] Remove apparently incorrect usage of "static"
+
+---
+ src/patchelf.cc | 8 +++-----
+ 1 file changed, 3 insertions(+), 5 deletions(-)
+
+Index: patchelf-0.9.old/src/patchelf.cc
+===================================================================
+--- patchelf-0.9.old.orig/src/patchelf.cc	2017-07-03 09:06:12.292069400 +0200
++++ patchelf-0.9.old/src/patchelf.cc	2017-07-03 09:20:57.000000000 +0200
+@@ -941,7 +941,6 @@
+     assert(strTabAddr == rdi(shdrDynStr.sh_addr));
+ 
+     /* Walk through the dynamic section, look for the DT_SONAME entry. */
+-    static vector<string> neededLibs;
+     dyn = (Elf_Dyn *) (contents + rdi(shdrDynamic.sh_offset));
+     Elf_Dyn * dynSoname = 0;
+     char * soname = 0;
+@@ -949,8 +948,7 @@
+         if (rdi(dyn->d_tag) == DT_SONAME) {
+             dynSoname = dyn;
+             soname = strTab + rdi(dyn->d_un.d_val);
+-        } else if (rdi(dyn->d_tag) == DT_INIT)
+-            neededLibs.push_back(string(strTab + rdi(dyn->d_un.d_val)));
++        }
+     }
+ 
+     if (op == printSoname) {
+@@ -1058,7 +1056,7 @@
+        unless you use its `--enable-new-dtag' option, in which case it
+        generates a DT_RPATH and DT_RUNPATH pointing at the same
+        string. */
+-    static vector<string> neededLibs;
++    vector<string> neededLibs;
+     dyn = (Elf_Dyn *) (contents + rdi(shdrDynamic.sh_offset));
+     Elf_Dyn * dynRPath = 0, * dynRunPath = 0;
+     char * rpath = 0;
+@@ -1091,7 +1089,7 @@
+     /* For each directory in the RPATH, check if it contains any
+        needed library. */
+     if (op == rpShrink) {
+-        static vector<bool> neededLibFound(neededLibs.size(), false);
++        vector<bool> neededLibFound(neededLibs.size(), false);
+ 
+         newRPath = "";
+ 
diff --git a/package/patchelf/0002-Extract-a-function-for-splitting-a-colon-separated-s.patch b/package/patchelf/0002-Extract-a-function-for-splitting-a-colon-separated-s.patch
new file mode 100644
index 0000000..63f3bce
--- /dev/null
+++ b/package/patchelf/0002-Extract-a-function-for-splitting-a-colon-separated-s.patch
@@ -0,0 +1,58 @@
+From 2e3fdc2030c75c19df6fc2924083cfad53856562 Mon Sep 17 00:00:00 2001
+From: Tuomas Tynkkynen <tuomas@tuxera.com>
+Date: Fri, 3 Jun 2016 23:03:51 +0300
+Subject: [PATCH 08/30] Extract a function for splitting a colon-separated
+ string
+
+We're going to need this logic in another place, so make a function of
+this.
+---
+ src/patchelf.cc | 28 +++++++++++++++++++---------
+ 1 file changed, 19 insertions(+), 9 deletions(-)
+
+Index: patchelf-0.9/src/patchelf.cc
+===================================================================
+--- patchelf-0.9.orig/src/patchelf.cc	2017-07-03 14:04:36.988281130 +0200
++++ patchelf-0.9/src/patchelf.cc	2017-07-03 14:04:36.988281130 +0200
+@@ -57,6 +57,22 @@
+ #define ElfFileParamNames Elf_Ehdr, Elf_Phdr, Elf_Shdr, Elf_Addr, Elf_Off, Elf_Dyn, Elf_Sym
+ 
+ 
++static vector<string> splitColonDelimitedString(const char * s){
++    vector<string> parts;
++    const char * pos = s;
++    while (*pos) {
++        const char * end = strchr(pos, ':');
++        if (!end) end = strchr(pos, 0);
++
++        parts.push_back(string(pos, end - pos));
++        if (*end == ':') ++end;
++        pos = end;
++    }
++
++    return parts;
++}
++
++
+ static unsigned int getPageSize(){
+     return pageSize;
+ }
+@@ -1093,15 +1109,9 @@
+ 
+         newRPath = "";
+ 
+-        char * pos = rpath;
+-        while (*pos) {
+-            char * end = strchr(pos, ':');
+-            if (!end) end = strchr(pos, 0);
+-
+-            /* Get the name of the directory. */
+-            string dirName(pos, end - pos);
+-            if (*end == ':') ++end;
+-            pos = end;
++        vector<string> rpathDirs = splitColonDelimitedString(rpath);
++        for (vector<string>::iterator it = rpathDirs.begin(); it != rpathDirs.end(); ++it) {
++            const string & dirName = *it;
+ 
+             /* Non-absolute entries are allowed (e.g., the special
+                "$ORIGIN" hack). */
diff --git a/package/patchelf/0003-Add-option-to-make-the-rpath-relative-under-a-specif.patch b/package/patchelf/0003-Add-option-to-make-the-rpath-relative-under-a-specif.patch
new file mode 100644
index 0000000..4f700d9
--- /dev/null
+++ b/package/patchelf/0003-Add-option-to-make-the-rpath-relative-under-a-specif.patch
@@ -0,0 +1,419 @@
+From af8d4a24a0ef613bdb47f0b1c3d962d59c53a4be Mon Sep 17 00:00:00 2001
+From: Wolfgang Grandegger <wg@grandegger.com>
+Date: Mon, 20 Feb 2017 16:29:24 +0100
+Subject: [PATCH] Add option to make the rpath relative under a specified root
+ directory
+
+Running "patchelf" with the option "--make-rpath-relative ROOTDIR" will
+modify or delete the RPATHDIRs according the following rules
+similar to Martin's patches [1] making the Buildroot toolchaing/SDK
+relocatable.
+
+RPATHDIR starts with "$ORIGIN":
+    The original build-system already took care of setting a relative
+    RPATH, resolve it and test if it's valid (does exist)
+
+RPATHDIR starts with ROOTDIR:
+    The original build-system added some absolute RPATH (absolute on
+    the build machine). Test if it's valid (does exist).
+
+ROOTDIR/RPATHDIR exists:
+    The original build-system already took care of setting an absolute
+    RPATH (absolute in the final rootfs), resolve it and test if it's
+    valid (does exist).
+
+RPATHDIR points somewhere else:
+    (can be anywhere: build trees, staging tree, host location,
+    non-existing location, etc.). Just discard such a path.
+
+The option "--no-standard-libs" will discard RPATHDIRs ROOTDIR/lib and
+ROOTDIR/usr/lib. Like "--shrink-rpath", RPATHDIRs are also discarded
+if the directories do not contain a library referenced by the
+DT_NEEDED fields.
+If the option "--relative-to-file" is given, the rpath will start
+with "$ORIGIN" making it relative to the ELF file, otherwise an
+absolute path relative to ROOTDIR will be used.
+
+[1] http://lists.busybox.net/pipermail/buildroot/2016-April/159422.html
+
+Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
+---
+ src/patchelf.cc | 187 ++++++++++++++++++++++++++++++++++++++++++++++++--------
+ 1 file changed, 161 insertions(+), 26 deletions(-)
+
+Index: patchelf-0.9/src/patchelf.cc
+===================================================================
+--- patchelf-0.9.orig/src/patchelf.cc	2017-07-03 14:05:07.196281487 +0200
++++ patchelf-0.9/src/patchelf.cc	2017-07-03 16:13:38.590374530 +0200
+@@ -46,13 +46,16 @@
+ 
+ static bool forceRPath = false;
+ 
++static bool noStandardLibDirs = false;
++
++static bool relativeToFile = false;
++
+ static string fileName;
+ static int pageSize = PAGESIZE;
+ 
+ off_t fileSize, maxSize;
+ unsigned char * contents = 0;
+ 
+-
+ #define ElfFileParams class Elf_Ehdr, class Elf_Phdr, class Elf_Shdr, class Elf_Addr, class Elf_Off, class Elf_Dyn, class Elf_Sym
+ #define ElfFileParamNames Elf_Ehdr, Elf_Phdr, Elf_Shdr, Elf_Addr, Elf_Off, Elf_Dyn, Elf_Sym
+ 
+@@ -77,6 +80,49 @@
+     return pageSize;
+ }
+ 
++static bool absolutePathExists(const string & path, string & canonicalPath)
++{
++    char *cpath = realpath(path.c_str(), NULL);
++    if (cpath) {
++        canonicalPath = cpath;
++        free(cpath);
++        return true;
++    } else {
++        return false;
++    }
++}
++
++static string makePathRelative(const string & path,
++    const string & refPath)
++{
++    string relPath = "$ORIGIN";
++    string p = path, refP = refPath;
++    size_t pos;
++
++    /* Strip the common part of path and refPath */
++    while (true) {
++        pos = p.find_first_of('/', 1);
++	if (refP.find_first_of('/', 1) != pos)
++	    break;
++	if (p.substr(0, pos) != refP.substr(0, pos))
++            break;
++	if (pos == string::npos)
++	    break;
++	p = p.substr(pos);
++	refP = refP.substr(pos);
++    }
++    /* Check if both pathes are equal */
++    if (p != refP) {
++	pos = 0;
++	while (pos != string::npos) {
++	    pos =refP.find_first_of('/', pos + 1);
++	    relPath.append("/..");
++	}
++	relPath.append(p);
++    }
++
++    return relPath;
++}
+ 
+ template<ElfFileParams>
+ class ElfFile
+@@ -183,14 +229,18 @@
+ 
+     void setInterpreter(const string & newInterpreter);
+ 
+-    typedef enum { rpPrint, rpShrink, rpSet, rpRemove } RPathOp;
++    typedef enum { rpPrint, rpShrink, rpMakeRelative, rpSet, rpRemove} RPathOp;
+ 
+-    void modifyRPath(RPathOp op, string newRPath);
++    bool libFoundInRPath(const string & dirName,
++                         const vector<string> neededLibs,
++			 vector<bool> & neededLibFound);
++
++    void modifyRPath(RPathOp op, string rootDir, string newRPath);
+ 
+     void addNeeded(set<string> libs);
+ 
+     void removeNeeded(set<string> libs);
+-    
++
+     void replaceNeeded(map<string, string>& libs);
+ 
+     void printNeededLibs();
+@@ -1041,7 +1091,27 @@
+ 
+ 
+ template<ElfFileParams>
+-void ElfFile<ElfFileParamNames>::modifyRPath(RPathOp op, string newRPath)
++bool ElfFile<ElfFileParamNames>::libFoundInRPath(const string & dirName,
++    const vector<string> neededLibs, vector<bool> & neededLibFound)
++{
++    /* For each library that we haven't found yet, see if it
++       exists in this directory. */
++    bool libFound = false;
++    for (unsigned int j = 0; j < neededLibs.size(); ++j)
++        if (!neededLibFound[j]) {
++            string libName = dirName + "/" + neededLibs[j];
++            struct stat st;
++            if (stat(libName.c_str(), &st) == 0) {
++                neededLibFound[j] = true;
++                libFound = true;
++            }
++        }
++    return libFound;
++}
++
++
++template<ElfFileParams>
++void ElfFile<ElfFileParamNames>::modifyRPath(RPathOp op, string rootDir, string newRPath)
+ {
+     Elf_Shdr & shdrDynamic = findSection(".dynamic");
+ 
+@@ -1096,6 +1166,11 @@
+         return;
+     }
+ 
++    if (op == rpMakeRelative && !rpath) {
++        debug("no RPATH to make relative\n");
++        return;
++    }
++
+     if (op == rpShrink && !rpath) {
+         debug("no RPATH to shrink\n");
+         return;
+@@ -1120,26 +1195,86 @@
+                 continue;
+             }
+ 
+-            /* For each library that we haven't found yet, see if it
+-               exists in this directory. */
+-            bool libFound = false;
+-            for (unsigned int j = 0; j < neededLibs.size(); ++j)
+-                if (!neededLibFound[j]) {
+-                    string libName = dirName + "/" + neededLibs[j];
+-                    struct stat st;
+-                    if (stat(libName.c_str(), &st) == 0) {
+-                        neededLibFound[j] = true;
+-                        libFound = true;
+-                    }
+-                }
+-
+-            if (!libFound)
++            if (!libFoundInRPath(dirName, neededLibs, neededLibFound))
+                 debug("removing directory `%s' from RPATH\n", dirName.c_str());
+             else
+                 concatToRPath(newRPath, dirName);
+         }
+     }
+ 
++    /* Make the the RPATH relative to the specified path */
++    if (op == rpMakeRelative) {
++        vector<bool> neededLibFound(neededLibs.size(), false);
++        string fileDir = fileName.substr(0, fileName.find_last_of("/"));
++
++        newRPath = "";
++
++        vector<string> rpathDirs = splitColonDelimitedString(rpath);
++        for (vector<string>::iterator it = rpathDirs.begin(); it != rpathDirs.end(); ++it) {
++            const string & dirName = *it;
++
++            string canonicalPath;
++
++            /* Figure out if we should keep or discard the path. There are several
++               cases to be handled:
++               "dirName" starts with "$ORIGIN":
++                   The original build-system already took care of setting a relative
++                   RPATH. Resolve it and test if it's valid (does exist).
++               "dirName" start with "rootDir":
++                   The original build-system added some absolute RPATH (absolute on
++                   the build machine). Test if it's valid (does exist).
++               "rootDir"/"dirName" exists:
++                    The original build-system already took care of setting an absolute
++                    RPATH (absolute in the final rootfs). Resolve it and test if it's
++                    valid (does exist).
++               "dirName" points somewhere else:
++                    (can be anywhere: build trees, staging tree, host location,
++                    non-existing location, etc.). Just discard such a path. */
++            if (!dirName.compare(0, 7, "$ORIGIN")) {
++                string path = fileDir + dirName.substr(7);
++                if (!absolutePathExists(path, canonicalPath)) {
++                    debug("removing directory '%s' from RPATH because '%s' doesn't exist\n",
++                          dirName.c_str(), path.c_str());
++                    continue;
++                }
++            } else if (!dirName.compare(0, rootDir.length(), rootDir)) {
++                if (!absolutePathExists(dirName, canonicalPath)) {
++                    debug("removing directory '%s' from RPATH because it doesn't exist\n", dirName.c_str());
++                    continue;
++                }
++            } else {
++                string path = rootDir + dirName;
++                if (!absolutePathExists(path, canonicalPath)) {
++                    debug("removing directory '%s' from RPATH because it's not in rootdir\n",
++                          dirName.c_str());
++                    continue;
++                }
++            }
++
++            if (noStandardLibDirs) {
++                if (!canonicalPath.compare(rootDir + "/lib") ||
++                    !canonicalPath.compare(rootDir + "/usr/lib")) {
++                    debug("removing directory '%s' from RPATH because it's a standard library directory\n",
++                         dirName.c_str());
++                    continue;
++                }
++            }
++
++            if (!libFoundInRPath(canonicalPath, neededLibs, neededLibFound)) {
++                debug("removing directory '%s' from RPATH because it does not contain needed libs\n",
++                      dirName.c_str());
++                continue;
++            }
++
++            /* Finally make "canonicalPath" relative to "filedir" in "rootDir" */
++            if (relativeToFile)
++                concatToRPath(newRPath, makePathRelative(canonicalPath, fileDir));
++            else
++                concatToRPath(newRPath, canonicalPath.substr(rootDir.length()));
++            debug("keeping relative path of %s\n", canonicalPath.c_str());
++        }
++    }
++
+     if (op == rpRemove) {
+         if (!rpath) {
+             debug("no RPATH to delete\n");
+@@ -1261,35 +1396,35 @@
+ void ElfFile<ElfFileParamNames>::replaceNeeded(map<string, string>& libs)
+ {
+     if (libs.empty()) return;
+-    
++
+     Elf_Shdr & shdrDynamic = findSection(".dynamic");
+     Elf_Shdr & shdrDynStr = findSection(".dynstr");
+     char * strTab = (char *) contents + rdi(shdrDynStr.sh_offset);
+ 
+     Elf_Dyn * dyn = (Elf_Dyn *) (contents + rdi(shdrDynamic.sh_offset));
+-    
++
+     unsigned int dynStrAddedBytes = 0;
+-    
++
+     for ( ; rdi(dyn->d_tag) != DT_NULL; dyn++) {
+         if (rdi(dyn->d_tag) == DT_NEEDED) {
+             char * name = strTab + rdi(dyn->d_un.d_val);
+             if (libs.find(name) != libs.end()) {
+                 const string & replacement = libs[name];
+-                
++
+                 debug("replacing DT_NEEDED entry `%s' with `%s'\n", name, replacement.c_str());
+-                
++
+                 // technically, the string referred by d_val could be used otherwise, too (although unlikely)
+                 // we'll therefore add a new string
+                 debug("resizing .dynstr ...");
+-                
++
+                 string & newDynStr = replaceSection(".dynstr",
+                     rdi(shdrDynStr.sh_size) + replacement.size() + 1 + dynStrAddedBytes);
+                 setSubstr(newDynStr, rdi(shdrDynStr.sh_size) + dynStrAddedBytes, replacement + '\0');
+-                
++
+                 dyn->d_un.d_val = shdrDynStr.sh_size + dynStrAddedBytes;
+-                
++
+                 dynStrAddedBytes += replacement.size() + 1;
+-                
++
+                 changed = true;
+             } else {
+                 debug("keeping DT_NEEDED entry `%s'\n", name);
+@@ -1311,7 +1446,7 @@
+     for (set<string>::iterator it = libs.begin(); it != libs.end(); it++) {
+         length += it->size() + 1;
+     }
+-    
++
+     string & newDynStr = replaceSection(".dynstr",
+         rdi(shdrDynStr.sh_size) + length + 1);
+     set<Elf64_Xword> libStrings;
+@@ -1321,7 +1456,7 @@
+         libStrings.insert(rdi(shdrDynStr.sh_size) + pos);
+         pos += it->size() + 1;
+     }
+-    
++
+     /* add all new needed entries to the dynamic section */
+     string & newDynamic = replaceSection(".dynamic",
+         rdi(shdrDynamic.sh_size) + sizeof(Elf_Dyn) * libs.size());
+@@ -1342,7 +1477,7 @@
+         wri(newDyn.d_un.d_val, *it);
+         setSubstr(newDynamic, i * sizeof(Elf_Dyn), string((char *) &newDyn, sizeof(Elf_Dyn)));
+     }
+-    
++
+     changed = true;
+ }
+ 
+@@ -1413,7 +1548,9 @@
+ static bool removeRPath = false;
+ static bool setRPath = false;
+ static bool printRPath = false;
++static bool makeRPathRelative = false;
+ static string newRPath;
++static string rootDir;
+ static set<string> neededLibsToRemove;
+ static map<string, string> neededLibsToReplace;
+ static set<string> neededLibsToAdd;
+@@ -1438,14 +1575,16 @@
+         elfFile.setInterpreter(newInterpreter);
+ 
+     if (printRPath)
+-        elfFile.modifyRPath(elfFile.rpPrint, "");
++        elfFile.modifyRPath(elfFile.rpPrint, "", "");
+ 
+     if (shrinkRPath)
+-        elfFile.modifyRPath(elfFile.rpShrink, "");
++        elfFile.modifyRPath(elfFile.rpShrink, "", "");
+     else if (removeRPath)
+-        elfFile.modifyRPath(elfFile.rpRemove, "");
++        elfFile.modifyRPath(elfFile.rpRemove, "", "");
+     else if (setRPath)
+-        elfFile.modifyRPath(elfFile.rpSet, newRPath);
++        elfFile.modifyRPath(elfFile.rpSet, "", newRPath);
++    else if (makeRPathRelative)
++        elfFile.modifyRPath(elfFile.rpMakeRelative, rootDir, "");
+ 
+     if (printNeeded) elfFile.printNeededLibs();
+ 
+@@ -1508,6 +1647,9 @@
+   [--set-rpath RPATH]\n\
+   [--remove-rpath]\n\
+   [--shrink-rpath]\n\
++  [--make-rpath-relative ROOTDIR]\n\
++  [--no-standard-lib-dirs]\n\
++  [--relative-to-file]\n\
+   [--print-rpath]\n\
+   [--force-rpath]\n\
+   [--add-needed LIBRARY]\n\
+@@ -1541,7 +1683,7 @@
+             if (++i == argc) error("missing argument");
+             pageSize = atoi(argv[i]);
+             if (pageSize <= 0) error("invalid argument to --page-size");
+-	}
++        }
+         else if (arg == "--print-interpreter") {
+             printInterpreter = true;
+         }
+@@ -1564,6 +1706,17 @@
+             setRPath = true;
+             newRPath = argv[i];
+         }
++        else if (arg == "--make-rpath-relative") {
++            if (++i == argc) error("missing argument to --make-rpath-relative");
++            makeRPathRelative = true;
++            rootDir = argv[i];
++        }
++        else if (arg == "--no-standard-lib-dirs") {
++            noStandardLibDirs = true;
++        }
++        else if (arg == "--relative-to-file") {
++            relativeToFile = true;
++        }
+         else if (arg == "--print-rpath") {
+             printRPath = true;
+         }
-- 
2.7.4

[Buildroot] [PATCH v6 03/10] support/scripts: add fix-rpath script to sanitize the rpath

[Wolfgang Grandegger]

From: Samuel Martin <s.martin49@gmail.com>

This commit introduces the script "fix-rpath" able to scan a tree,
detect ELF files, check their RPATH and fix it in a proper way.
The RPATH fixup is done by the patchelf utility using the option
"--make-rpath-relative <root-directory>".

Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
---
 support/scripts/fix-rpath | 122 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 122 insertions(+)
 create mode 100755 support/scripts/fix-rpath

diff --git a/support/scripts/fix-rpath b/support/scripts/fix-rpath
new file mode 100755
index 0000000..6a81193
--- /dev/null
+++ b/support/scripts/fix-rpath
@@ -0,0 +1,122 @@
+#!/usr/bin/env bash
+
+# Copyright (C) 2016 Samuel Martin <s.martin49@gmail.com>
+# Copyright (C) 2017 Wolfgang Grandegger <wg@grandegger.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+usage() {
+  cat <<EOF >&2
+Usage:	${0} TREE_KIND
+
+Description:
+
+    This script scans a tree and sanitize ELF files' RPATH found in there.
+
+    Sanitization behaves the same whatever the kind of the processed tree,
+    but the resulting RPATH differs. The rpath sanitization is done using
+    "patchelf --make-rpath-relazive".
+
+Arguments:
+
+    TREE_KIND	Kind of tree to be processed.
+		Allowed values: host, target, staging
+
+Environment:
+
+    PATCHELF	patchelf program to use
+		(default: HOST_DIR/usr/bin/patchelf)
+EOF
+}
+
+: ${PATCHELF:=${HOST_DIR}/usr/bin/patchelf}
+
+# ELF files should not be in these sub-directories
+HOST_EXCLUDEPATHS="/usr/share/terminfo"
+STAGING_EXCLUDEPATHS="/usr/include /usr/share/terminfo"
+
+main() {
+    local rootdir
+    local tree="${1}"
+    local find_args=( )
+    local sanitize_extra_args=( )
+
+    case "${tree}" in
+	host)
+	    rootdir="${HOST_DIR}"
+
+	    # do not process the sysroot (only contains target binaries)
+	    find_args+=( "-path" "${STAGING_DIR}" "-prune" "-o" )
+
+	    # do not process the external toolchain installation directory to
+	    # avoid breaking it.
+	    test "${TOOLCHAIN_EXTERNAL_DOWNLOAD_INSTALL_DIR}" != "" && \
+		find_args+=( "-path" "${TOOLCHAIN_EXTERNAL_DOWNLOAD_INSTALL_DIR}" "-prune" "-o" )
+
+	    for excludepath in ${HOST_EXCLUDEPATHS}; do
+		find_args+=( "-path" "${HOST_DIR}""${excludepath}" "-prune" "-o" );
+	    done
+
+	    # do not process the patchelf binary but a copy to work-around "file in use"
+	    find_args+=( "-path" "${PATCHELF}" "-prune" "-o" )
+	    cp "${PATCHELF}" "${PATCHELF}.__to_be_patched"
+
+	    sanitize_extra_args+=( "--relative-to-file" )
+	    ;;
+
+	staging)
+	    rootdir="${STAGING_DIR}"
+
+	    # ELF files should not be in these sub-directories
+	    for excludepath in ${STAGING_EXCLUDEPATHS}; do
+		find_args+=( "-path" "${STAGING_DIR}""${excludepath}" "-prune" "-o" )
+	    done
+
+	    sanitize_extra_args+=( "--no-standard-lib-dirs" "--relative-to-file" )
+	    ;;
+
+	target)
+	    rootdir="${TARGET_DIR}"
+	    sanitize_extra_args+=( "--no-standard-lib-dirs" "--relative-to-file" )
+	    ;;
+
+	*)
+	    usage
+	    exit 1
+	    ;;
+    esac
+
+    find_args+=( "-type" "f" "-print" )
+
+    while read file ; do
+	# check if it's an ELF file
+	if ${PATCHELF} --print-rpath "${file}" > /dev/null 2>&1; then
+	    # make files writable if necessary
+	    changed=$(chmod -c u+w "${file}")
+	    # call patchelf to sanitize the rpath
+	    ${PATCHELF} --make-rpath-relative "${rootdir}" ${sanitize_extra_args[@]} "${file}"
+	    # restore the original permission
+	    test "${changed}" != "" && chmod u-w "${file}"
+	fi
+    done < <(find "${rootdir}" ${find_args[@]})
+
+    # Restore patched patchelf utility
+    test "${tree}" = "host" && mv "${PATCHELF}.__to_be_patched" "${PATCHELF}"
+
+    # ignore errors
+    return 0
+}
+
+main ${@}
-- 
2.7.4

[Buildroot] [PATCH v6 04/10] core: sanitize RPATH in staging tree at the end of target finalization

[Wolfgang Grandegger]

Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
---
 Makefile | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Makefile b/Makefile
index b37171f..1d07673 100644
--- a/Makefile
+++ b/Makefile
@@ -715,6 +715,9 @@ endif
 		$(call MESSAGE,"Executing post-build script $(s)"); \
 		$(EXTRA_ENV) $(s) $(TARGET_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep))
 
+	@$(call MESSAGE,"Sanitizing RPATH in staging tree")
+	$(TOPDIR)/support/scripts/fix-rpath staging
+
 .PHONY: target-post-image
 target-post-image: $(TARGETS_ROOTFS) target-finalize
 	@$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_IMAGE_SCRIPT)), \
-- 
2.7.4

[Buildroot] [PATCH v6 05/10] core: sanitize RPATH in target tree at the end of target finalization

[Wolfgang Grandegger]

Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
---
 Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Makefile b/Makefile
index 1d07673..d8db3f4 100644
--- a/Makefile
+++ b/Makefile
@@ -717,6 +717,8 @@ endif
 
 	@$(call MESSAGE,"Sanitizing RPATH in staging tree")
 	$(TOPDIR)/support/scripts/fix-rpath staging
+	@$(call MESSAGE,"Sanitizing RPATH in target tree")
+	$(TOPDIR)/support/scripts/fix-rpath target
 
 .PHONY: target-post-image
 target-post-image: $(TARGETS_ROOTFS) target-finalize
-- 
2.7.4

[Buildroot] [PATCH v6 06/10] core: sanitize RPATH in host tree at the very end of the build

[Wolfgang Grandegger]

We introduce the "finalize-host" target for that purpose.
We need the patchelf host utility to check and fix the rpath.

Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
---
 Makefile | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index d8db3f4..f5646ad 100644
--- a/Makefile
+++ b/Makefile
@@ -550,7 +550,15 @@ $(BUILD_DIR)/buildroot-config/auto.conf: $(BR2_CONFIG)
 prepare: $(BUILD_DIR)/buildroot-config/auto.conf
 
 .PHONY: world
-world: target-post-image
+world: target-post-image host-finalize
+
+.PHONY: host-finalize
+host-finalize:
+	@$(call MESSAGE,"Rendering the SDK relocatable")
+	$(TOPDIR)/support/scripts/fix-rpath host
+
+# We need patchelf for RPATH sanitization
+PACKAGES += host-patchelf
 
 # Populating the staging with the base directories is handled by the skeleton package
 $(STAGING_DIR):
-- 
2.7.4

[Buildroot] [PATCH v6 07/10] support/scripts: add relocate-sdk.sh script for SDK relocation

[Wolfgang Grandegger]

It will install the script "relocate-sdk.sh" in the HOST_DIR
allowing to adjust the path to the SDK directory in all text
files after it has been moved to a new location.

Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
---
 support/misc/relocate-sdk.sh | 47 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
 create mode 100755 support/misc/relocate-sdk.sh

diff --git a/support/misc/relocate-sdk.sh b/support/misc/relocate-sdk.sh
new file mode 100755
index 0000000..4e3801c
--- /dev/null
+++ b/support/misc/relocate-sdk.sh
@@ -0,0 +1,47 @@
+#!/bin/bash
+#
+if [ "$#" -ne 0 ]; then
+    echo "Run this script to relocate the buildroot SDK at that location"
+    exit 1
+fi
+
+LOCFILE="usr/share/buildroot/sdk-location"
+FILEPATH="$(readlink -f "$0")"
+NEWPATH="$(dirname "${FILEPATH}")"
+
+cd "${NEWPATH}"
+if [ ! -r "${LOCFILE}" ]; then
+    echo "Previous location of the buildroot SDK not found!"
+    exit 1
+fi
+OLDPATH="$(cat "${LOCFILE}")"
+
+if [ "${NEWPATH}" = "${OLDPATH}" ]; then
+    echo "This buildroot SDK has already been relocated!"
+    exit 0
+fi
+
+# Check if the path substitution does work properly, e.g.
+# a tree "/a/b/c" copied into "/a/b/c/" would not be allowed.
+newpath="$(sed -e "s\\${OLDPATH}\\${NEWPATH}\\g" "${LOCFILE}")"
+if [ "${NEWPATH}" != "${newpath}" ]; then
+    echo "Something went wrong with substituting the path!"
+    echo "Please choose another location for your SDK!"
+    exit 1
+fi
+
+echo "Relocating the buildroot SDK from ${OLDPATH} to ${NEWPATH} ..."
+
+# Make sure file uses the right language
+export LC_ALL=C
+# Replace the old path with the new one in all text files
+while read -r FILE ; do
+    if file -b --mime-type "${FILE}" | grep -q '^text/' && [ "${FILE}" != "${LOCFILE}" ]
+    then
+	sed -i "s\\${OLDPATH}\\${NEWPATH}\\g" "${FILE}"
+    fi;
+done < <(grep -lr "${OLDPATH}" .)
+
+# At the very end, we update the location file to not break the
+# SDK if this script gets interruted.
+sed -i "s\\${OLDPATH}\\${NEWPATH}\\g" ${LOCFILE}
-- 
2.7.4

[Buildroot] [PATCH v6 08/10] core: install relocation script and location at the end of the build

[Wolfgang Grandegger]

The script "relocate-sdk.sh" is installed into the top directory of
the SDK (HOST_DIR) and the SDK location path is stored in the file
"HOST_DIR/usr/share/buildroot/sdk-location"-

Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
---
 Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Makefile b/Makefile
index f5646ad..7e92428 100644
--- a/Makefile
+++ b/Makefile
@@ -556,6 +556,8 @@ world: target-post-image host-finalize
 host-finalize:
 	@$(call MESSAGE,"Rendering the SDK relocatable")
 	$(TOPDIR)/support/scripts/fix-rpath host
+	install $(TOPDIR)/support/misc/relocate-sdk.sh $(HOST_DIR)
+	echo $(HOST_DIR) > $(HOST_DIR)/usr/share/buildroot/sdk-location
 
 # We need patchelf for RPATH sanitization
 PACKAGES += host-patchelf
-- 
2.7.4

[Buildroot] [PATCH v6 09/10] external-toolchain: check if a buildroot SDK has already been relocated

[Wolfgang Grandegger]

The location of the buildroot SDK is stored in the file "sdk-location"
in "usr/share/buildroot". If it's content does not match the current
SDK location, ask the user to run the script "relocate-sdk.sh" in the
top directory once. The external toolchain may be a pre-installed one
in a directory that is not writeable by us. Therefore, we can't run
the script directly.

Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
---
 toolchain/helpers.mk                                   | 18 ++++++++++++++++++
 toolchain/toolchain-external/pkg-toolchain-external.mk |  1 +
 2 files changed, 19 insertions(+)

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index 90834f4..806968b 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -479,3 +479,21 @@ define simplify_symlink
 	ln -sf "$${DOTS}$${REL_DEST}" "$${FULL_SRC}" ; \
 )
 endef
+
+#
+# Check if it's a buildroot toolchain and if it's already relocatable by
+# reading and testing the toolchain location file
+#
+# $1: toolchain installation directory
+#
+define check_buildroot_sdk_relocated
+( \
+	if [ -r $(1)/share/buildroot/sdk-location ]; then \
+		sdkroot=`dirname "$(1)"`; \
+		if [ "`cat $(1)/share/buildroot/sdk-location`" != "$${sdkroot}" ]; then \
+			echo "Please relocate the buildroot SDK by executing \"$${sdkroot}/relocate-sdk.sh\" once!" ; \
+			exit 1 ; \
+		fi \
+	fi \
+)
+endef
diff --git a/toolchain/toolchain-external/pkg-toolchain-external.mk b/toolchain/toolchain-external/pkg-toolchain-external.mk
index 8460e37..3dd7185 100644
--- a/toolchain/toolchain-external/pkg-toolchain-external.mk
+++ b/toolchain/toolchain-external/pkg-toolchain-external.mk
@@ -545,6 +545,7 @@ endif
 # matches the configuration provided in Buildroot: ABI, C++ support,
 # kernel headers version, type of C library and all C library features.
 define $(2)_CONFIGURE_CMDS
+	$$(Q)$$(call check_buildroot_sdk_relocated,$$(TOOLCHAIN_EXTERNAL_INSTALL_DIR))
 	$$(Q)$$(call check_cross_compiler_exists,$$(TOOLCHAIN_EXTERNAL_CC))
 	$$(Q)$$(call check_unusable_toolchain,$$(TOOLCHAIN_EXTERNAL_CC))
 	$$(Q)SYSROOT_DIR="$$(call toolchain_find_sysroot,$$(TOOLCHAIN_EXTERNAL_CC))" ; \
-- 
2.7.4

[Buildroot] [PATCH v6 10/10] package/qt5base: provide "qt.conf" to make "qmake" relocatable

[Wolfgang Grandegger]

The file "qt.conf" can be used to override the hard-coded paths that are
compiled into the Qt library. We need it to make "qmake" relocatable.

CC: Julien Corjon <corjon.j@ecagroup.com>
CC: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
---
 package/qt5/qt5base/qt.conf.in | 6 ++++++
 package/qt5/qt5base/qt5base.mk | 8 ++++++++
 2 files changed, 14 insertions(+)
 create mode 100644 package/qt5/qt5base/qt.conf.in

diff --git a/package/qt5/qt5base/qt.conf.in b/package/qt5/qt5base/qt.conf.in
new file mode 100644
index 0000000..48e4b92
--- /dev/null
+++ b/package/qt5/qt5base/qt.conf.in
@@ -0,0 +1,6 @@
+[Paths]
+Prefix=@@HOST_DIR@@/usr
+Sysroot=@@STAGING_DIR@@
+Headers=/usr/include/qt5
+Plugins=/usr/lib/qt/plugins
+Examples=/usr/lib/qt/examples
diff --git a/package/qt5/qt5base/qt5base.mk b/package/qt5/qt5base/qt5base.mk
index 5fe8bb8..883850b 100644
--- a/package/qt5/qt5base/qt5base.mk
+++ b/package/qt5/qt5base/qt5base.mk
@@ -258,9 +258,17 @@ define QT5BASE_BUILD_CMDS
 	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)
 endef
 
+# The file "qt.conf" can be used to override the hard-coded paths that are
+# compiled into the Qt library. We need it to make "qmake" relocatable.
+define QT5BASE_INSTALL_QT_CONF
+	sed -e "s\\@@HOST_DIR@@\\$(HOST_DIR)\\" -e "s\\@@STAGING_DIR@@\\$(STAGING_DIR)\\" \
+		$(QT5BASE_PKGDIR)/qt.conf.in > $(HOST_DIR)/usr/bin/qt.conf
+endef
+
 define QT5BASE_INSTALL_STAGING_CMDS
 	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) install
 	$(QT5_LA_PRL_FILES_FIXUP)
+	$(QT5BASE_INSTALL_QT_CONF)
 endef
 
 define QT5BASE_INSTALL_TARGET_LIBS
-- 
2.7.4

[Buildroot] [PATCH v6 00/10] Make the SDK relocatable

[Arnout Vandecappelle]

 Hi Wolfgang,

 I've already marked patch 7 and 10 as superseded in patchwork because they were
already applied by Thomas. Can you check if there is a significant difference
with what was applied?

On 04-07-17 18:22, Wolfgang Grandegger wrote:
> Hello,
> 
> this is v6 of my patch series to make the buildroot SDK (HOST_DIR)
> relocatable. It sanitizes the RPATH of all ELF files in the "target",
> "staging" and "host" tree using "patchelf --make-rpath-relative". We
> now use patchelf v0.9 to still support old Debian and RHEL systems.
> 
> v5 did RPATH sanitization per package after package installation into
> the host, staging or target tree using GLOBAL_INSTRUMENTATION_HOOKS.
> This approach got more and more complex and inefficient. Therefore
> it was abandoned in favor of global sanititation at the end of the
> host, staging and target build (see changes since v5).
> 
> Furthermore this patch creates the script "relocate-sdk.sh" in the top
> directory of the "host" tree allowing to relocate the SDK after it has
> been moved to a new location. It replaces the old path with the new
> one in all text files identified by "file --mime-type". The location
> is stored in "usr/share/buildroot/sdk-location".
> 
> Unfortunately, "qmake" uses hard-coded pathes compiled into the QT5
> libraries. To overcome this problem, "qt5pase" now creates "qt.conf".
> 
> Hope I have not too much pending issues.

 We all do :-) But this is really complicated infrastructural work, so it is to
be expected that it takes time. Just 7 revisions would already be pretty good!

> 
> Wolfgang.
> 
> Things not yet addressed:
> 
> - "make toolchain" creates a toolchain tree which still has references
>   to the build system (in ELF and text files).
> 
> Changes since v5:
> - switch back to v4
> - patchelf patches are now based on v0.9
> - patchelf now calculates minimal relative path to the ELF file
>   (and not to the rootdir as before).
> - patchelf: neededLibFound is now passed to libFoundInRPath

 If any more issues are found in the rpath fixup, they can be handled in
follow-up patches.


 Regards,
 Arnout

> - Makefile: the "host-finalize" target has been added for SDK relocation
> - fix-rpath now uses variables to define the list of dirs to be pruned
> - fix-rpath: the staging tree is sanitized like the target tree
> - various minor fixes (typos)
> 
> Changes since v4:
> 
> - RPATH sanitation is now done per package and installation step
>   in "pkg-generic.mk".
> - After the installation step, the list of the installed files is
>   stored in "<package-build-dir>/.br_[host|staging|target]_filelist".
> - The GLOBAL_INSTRUMENTATION_HOOKS "step_sanitize_rpath" then calls
>   "fix-rpath" to do the sanitation.
> - DEPENDENCIES_HOST_PREREQ += host-patchelf is set early in the
>   Makefile as we need it for RPATH sanitation. As soon it's available
>   sanitation can start (currently missing some packages).
> - The patchelf "file busy" issue is now worked-around differently.
> 
> Changes since v3:
> 
> - The patchelf patch implementing " --make-rpath-relative" now supports
>   the option "--relative-to-file" instructing to use "$ORIGIN" in
>   RPATHs. Otherwise an absolute path relative to the root directory will
>   be used.
> - The staging tree is now sanitized as well using the options
>   "--relative-to-file" and "--no-standard-libs".
> - For the "target" tree, relative RPATHs do not use "$ORIGIN" any
>   longer. An absolute path relative to the root directory is used
>   instead.
> 
> Changes since v2:
> 
> - provide "qt.conf" to make "qmake" relocatable
> - sed now uses the separator "\" to substitute the directory path.
>   It's one of the few characters not allowed in file names. To
>   avoid interpreting it as escape character, the "read -r" is used.
> - The paranoia substituion check is done before doing the real
>   substituion.
> 
> Changes since v1:
> 
> - The name SDK has been chosen for the relocatabed "HOST_DIR" (instead
>   of toolchanin).
> - The patchelf version bump and patching are now done by 2 patches
> - No more helper functions are used in the Makefile to call "fix-rpath"
>   but added directly.
> - The staging tree is not touched any more... until we have a good
>   reason to do so. 
> - The sanitation is now performed by an optimized "fix-rpath" script.
> - The relocate-sdk script is now copied for support/misc to the
>   top directory of the host tree.
> 
> Samuel Martin (1):
>   support/scripts: add fix-rpath script to sanitize the rpath
> 
> Wolfgang Grandegger (9):
>   support/scripts: check-host-rpath now handles $ORIGIN/../lib
>   package/patchelf: add patch for rpath sanitization under a root
>     directory
>   core: sanitize RPATH in staging tree at the end of target finalization
>   core: sanitize RPATH in target tree at the end of target finalization
>   core: sanitize RPATH in host tree at the very end of the build
>   support/scripts: add relocate-sdk.sh script for SDK relocation
>   core: install relocation script and location at the end of the build
>   external-toolchain: check if a buildroot SDK has already been
>     relocated
>   package/qt5base: provide "qt.conf" to make "qmake" relocatable
> 
>  Makefile                                           |  17 +-
>  ...move-apparently-incorrect-usage-of-static.patch |  49 +++
>  ...unction-for-splitting-a-colon-separated-s.patch |  58 +++
>  ...to-make-the-rpath-relative-under-a-specif.patch | 419 +++++++++++++++++++++
>  package/qt5/qt5base/qt.conf.in                     |   6 +
>  package/qt5/qt5base/qt5base.mk                     |   8 +
>  support/misc/relocate-sdk.sh                       |  47 +++
>  support/scripts/check-host-rpath                   |   2 +-
>  support/scripts/fix-rpath                          | 122 ++++++
>  toolchain/helpers.mk                               |  18 +
>  .../toolchain-external/pkg-toolchain-external.mk   |   1 +
>  11 files changed, 745 insertions(+), 2 deletions(-)
>  create mode 100644 package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch
>  create mode 100644 package/patchelf/0002-Extract-a-function-for-splitting-a-colon-separated-s.patch
>  create mode 100644 package/patchelf/0003-Add-option-to-make-the-rpath-relative-under-a-specif.patch
>  create mode 100644 package/qt5/qt5base/qt.conf.in
>  create mode 100755 support/misc/relocate-sdk.sh
>  create mode 100755 support/scripts/fix-rpath
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH v6 00/10] Make the SDK relocatable

[Wolfgang Grandegger]

Hello Arnout,

Am 04.07.2017 um 19:55 schrieb Arnout Vandecappelle:
>   Hi Wolfgang,
> 
>   I've already marked patch 7 and 10 as superseded in patchwork because they were
> already applied by Thomas. Can you check if there is a significant difference
> with what was applied?
> 
> On 04-07-17 18:22, Wolfgang Grandegger wrote:
>> Hello,
>>
>> this is v6 of my patch series to make the buildroot SDK (HOST_DIR)
>> relocatable. It sanitizes the RPATH of all ELF files in the "target",
>> "staging" and "host" tree using "patchelf --make-rpath-relative". We
>> now use patchelf v0.9 to still support old Debian and RHEL systems.
>>
>> v5 did RPATH sanitization per package after package installation into
>> the host, staging or target tree using GLOBAL_INSTRUMENTATION_HOOKS.
>> This approach got more and more complex and inefficient. Therefore
>> it was abandoned in favor of global sanititation at the end of the
>> host, staging and target build (see changes since v5).
>>
>> Furthermore this patch creates the script "relocate-sdk.sh" in the top
>> directory of the "host" tree allowing to relocate the SDK after it has
>> been moved to a new location. It replaces the old path with the new
>> one in all text files identified by "file --mime-type". The location
>> is stored in "usr/share/buildroot/sdk-location".
>>
>> Unfortunately, "qmake" uses hard-coded pathes compiled into the QT5
>> libraries. To overcome this problem, "qt5pase" now creates "qt.conf".
>>
>> Hope I have not too much pending issues.
> 
>   We all do :-) But this is really complicated infrastructural work, so it is to
> be expected that it takes time. Just 7 revisions would already be pretty good!

It's going forward... that's really good news!
>>
>> Wolfgang.
>>
>> Things not yet addressed:
>>
>> - "make toolchain" creates a toolchain tree which still has references
>>    to the build system (in ELF and text files).
>>
>> Changes since v5:
>> - switch back to v4
>> - patchelf patches are now based on v0.9
>> - patchelf now calculates minimal relative path to the ELF file
>>    (and not to the rootdir as before).
>> - patchelf: neededLibFound is now passed to libFoundInRPath
> 
>   If any more issues are found in the rpath fixup, they can be handled in
> follow-up patches.

Great!

BTW, I have another little patch for patchelf speeding up the open path 
significantly. Currently, the complete file will be read in regardless 
if it's an ELF file or not. Then it starts checking size, magic number, 
etc. Now it just reads the necessary data (ELF Header) for the checks 
and only if it's an ELF file, the rest of the data will be read as well. 
More soon...

Wolfgang.

[Buildroot] [PATCH v6 10/10] package/qt5base: provide "qt.conf" to make "qmake" relocatable

[Arnout Vandecappelle]

 Hi Wolfgang,

On 04-07-17 18:22, Wolfgang Grandegger wrote:
> The file "qt.conf" can be used to override the hard-coded paths that are
> compiled into the Qt library. We need it to make "qmake" relocatable.

 All qt5 packages are failing because of this, e.g.
http://autobuild.buildroot.net/results/33e/33eb7b7ef7d232bed2432d1d44fd80a4fc8c4e2d

qt5serialport: installs files in
/accts/mlweber1/instance-2/output/host/usr/x86_64-buildroot-linux-uclibc/sysroot//accts/mlweber1/instance-2/output


 Can you have a look?

 Regards,
 Arnout

> 
> CC: Julien Corjon <corjon.j@ecagroup.com>
> CC: Peter Seiderer <ps.report@gmx.net>
> Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
> ---
>  package/qt5/qt5base/qt.conf.in | 6 ++++++
>  package/qt5/qt5base/qt5base.mk | 8 ++++++++
>  2 files changed, 14 insertions(+)
>  create mode 100644 package/qt5/qt5base/qt.conf.in
> 
> diff --git a/package/qt5/qt5base/qt.conf.in b/package/qt5/qt5base/qt.conf.in
> new file mode 100644
> index 0000000..48e4b92
> --- /dev/null
> +++ b/package/qt5/qt5base/qt.conf.in
> @@ -0,0 +1,6 @@
> +[Paths]
> +Prefix=@@HOST_DIR@@/usr
> +Sysroot=@@STAGING_DIR@@
> +Headers=/usr/include/qt5
> +Plugins=/usr/lib/qt/plugins
> +Examples=/usr/lib/qt/examples
> diff --git a/package/qt5/qt5base/qt5base.mk b/package/qt5/qt5base/qt5base.mk
> index 5fe8bb8..883850b 100644
> --- a/package/qt5/qt5base/qt5base.mk
> +++ b/package/qt5/qt5base/qt5base.mk
> @@ -258,9 +258,17 @@ define QT5BASE_BUILD_CMDS
>  	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)
>  endef
>  
> +# The file "qt.conf" can be used to override the hard-coded paths that are
> +# compiled into the Qt library. We need it to make "qmake" relocatable.
> +define QT5BASE_INSTALL_QT_CONF
> +	sed -e "s\\@@HOST_DIR@@\\$(HOST_DIR)\\" -e "s\\@@STAGING_DIR@@\\$(STAGING_DIR)\\" \
> +		$(QT5BASE_PKGDIR)/qt.conf.in > $(HOST_DIR)/usr/bin/qt.conf
> +endef
> +
>  define QT5BASE_INSTALL_STAGING_CMDS
>  	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) install
>  	$(QT5_LA_PRL_FILES_FIXUP)
> +	$(QT5BASE_INSTALL_QT_CONF)
>  endef
>  
>  define QT5BASE_INSTALL_TARGET_LIBS
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH v6 10/10] package/qt5base: provide "qt.conf" to make "qmake" relocatable

[Wolfgang Grandegger]

Hello Arnout,

Am 05.07.2017 um 10:59 schrieb Arnout Vandecappelle:
>   Hi Wolfgang,
> 
> On 04-07-17 18:22, Wolfgang Grandegger wrote:
>> The file "qt.conf" can be used to override the hard-coded paths that are
>> compiled into the Qt library. We need it to make "qmake" relocatable.
> 
>   All qt5 packages are failing because of this, e.g.
> http://autobuild.buildroot.net/results/33e/33eb7b7ef7d232bed2432d1d44fd80a4fc8c4e2d
> 
> qt5serialport: installs files in
> /accts/mlweber1/instance-2/output/host/usr/x86_64-buildroot-linux-uclibc/sysroot//accts/mlweber1/instance-2/output
> 
> 
>   Can you have a look?

I know the problem! The patched used an old version (from an old series) 
of "qt.conf.in"..., sorry!

Wolfgang.

> 
>   Regards,
>   Arnout
> 
>>
>> CC: Julien Corjon <corjon.j@ecagroup.com>
>> CC: Peter Seiderer <ps.report@gmx.net>
>> Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
>> ---
>>   package/qt5/qt5base/qt.conf.in | 6 ++++++
>>   package/qt5/qt5base/qt5base.mk | 8 ++++++++
>>   2 files changed, 14 insertions(+)
>>   create mode 100644 package/qt5/qt5base/qt.conf.in
>>
>> diff --git a/package/qt5/qt5base/qt.conf.in b/package/qt5/qt5base/qt.conf.in
>> new file mode 100644
>> index 0000000..48e4b92
>> --- /dev/null
>> +++ b/package/qt5/qt5base/qt.conf.in
>> @@ -0,0 +1,6 @@
>> +[Paths]
>> +Prefix=@@HOST_DIR@@/usr
>> +Sysroot=@@STAGING_DIR@@
>> +Headers=/usr/include/qt5
>> +Plugins=/usr/lib/qt/plugins
>> +Examples=/usr/lib/qt/examples
>> diff --git a/package/qt5/qt5base/qt5base.mk b/package/qt5/qt5base/qt5base.mk
>> index 5fe8bb8..883850b 100644
>> --- a/package/qt5/qt5base/qt5base.mk
>> +++ b/package/qt5/qt5base/qt5base.mk
>> @@ -258,9 +258,17 @@ define QT5BASE_BUILD_CMDS
>>   	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)
>>   endef
>>   
>> +# The file "qt.conf" can be used to override the hard-coded paths that are
>> +# compiled into the Qt library. We need it to make "qmake" relocatable.
>> +define QT5BASE_INSTALL_QT_CONF
>> +	sed -e "s\\@@HOST_DIR@@\\$(HOST_DIR)\\" -e "s\\@@STAGING_DIR@@\\$(STAGING_DIR)\\" \
>> +		$(QT5BASE_PKGDIR)/qt.conf.in > $(HOST_DIR)/usr/bin/qt.conf
>> +endef
>> +
>>   define QT5BASE_INSTALL_STAGING_CMDS
>>   	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) install
>>   	$(QT5_LA_PRL_FILES_FIXUP)
>> +	$(QT5BASE_INSTALL_QT_CONF)
>>   endef
>>   
>>   define QT5BASE_INSTALL_TARGET_LIBS
>>
> 

[Buildroot] [PATCH v6 10/10] package/qt5base: provide "qt.conf" to make "qmake" relocatable

[Thomas Petazzoni]

Hello,

On Wed, 5 Jul 2017 10:59:40 +0200, Arnout Vandecappelle wrote:

> On 04-07-17 18:22, Wolfgang Grandegger wrote:
> > The file "qt.conf" can be used to override the hard-coded paths that are
> > compiled into the Qt library. We need it to make "qmake" relocatable.  
> 
>  All qt5 packages are failing because of this, e.g.
> http://autobuild.buildroot.net/results/33e/33eb7b7ef7d232bed2432d1d44fd80a4fc8c4e2d
> 
> qt5serialport: installs files in
> /accts/mlweber1/instance-2/output/host/usr/x86_64-buildroot-linux-uclibc/sysroot//accts/mlweber1/instance-2/output

Due to the numerous build failures caused by this patch, I reverted the
commit.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH v6 10/10] package/qt5base: provide "qt.conf" to make "qmake" relocatable

[Wolfgang Grandegger]

Hello Thomas,

Am 05.07.2017 um 11:30 schrieb Thomas Petazzoni:
> Hello,
> 
> On Wed, 5 Jul 2017 10:59:40 +0200, Arnout Vandecappelle wrote:
> 
>> On 04-07-17 18:22, Wolfgang Grandegger wrote:
>>> The file "qt.conf" can be used to override the hard-coded paths that are
>>> compiled into the Qt library. We need it to make "qmake" relocatable.
>>
>>   All qt5 packages are failing because of this, e.g.
>> http://autobuild.buildroot.net/results/33e/33eb7b7ef7d232bed2432d1d44fd80a4fc8c4e2d
>>
>> qt5serialport: installs files in
>> /accts/mlweber1/instance-2/output/host/usr/x86_64-buildroot-linux-uclibc/sysroot//accts/mlweber1/instance-2/output
> 
> Due to the numerous build failures caused by this patch, I reverted the
> commit.

I will resend then a proper patch for QT5 with the next release of my 
"Make SDK relocatable" patch series.

Wolfgang

[Buildroot] [PATCH v6 10/10] package/qt5base: provide "qt.conf" to make "qmake" relocatable

[Wolfgang Grandegger]

Am 05.07.2017 um 11:30 schrieb Wolfgang Grandegger:
> Hello Arnout,
> 
> Am 05.07.2017 um 10:59 schrieb Arnout Vandecappelle:
>>   Hi Wolfgang,
>>
>> On 04-07-17 18:22, Wolfgang Grandegger wrote:
>>> The file "qt.conf" can be used to override the hard-coded paths that are
>>> compiled into the Qt library. We need it to make "qmake" relocatable.
>>
>>   All qt5 packages are failing because of this, e.g.
>> http://autobuild.buildroot.net/results/33e/33eb7b7ef7d232bed2432d1d44fd80a4fc8c4e2d 
>>
>>
>> qt5serialport: installs files in
>> /accts/mlweber1/instance-2/output/host/usr/x86_64-buildroot-linux-uclibc/sysroot//accts/mlweber1/instance-2/output 
>>
>>
>>
>>   Can you have a look?
> 
> I know the problem! The patched used an old version (from an old series) 
> of "qt.conf.in"..., sorry!

The patch is reverted now. So just for the records, the patch below
fixes the problem. I'm going to provide a proper patch with my next
series. This also shows, that if "qt.conf" is in place, it's active and
will always overwrite the pathes compiled into qmake... which will break
the idea of "make sdk" somehow.

[Buildroot] [PATCH v6 01/10] support/scripts: check-host-rpath now handles $ORIGIN/../lib

[Thomas Petazzoni]

Hello,

On Tue,  4 Jul 2017 18:22:30 +0200, Wolfgang Grandegger wrote:
> The new RPath sanitization uses now the relative path
> "$ORIGIN/../lib" for binaries in "/usr/bin".
> 
> Signed-off-by: Wolfgang Grandegger <wg@grandegger.com>
> ---
>  support/scripts/check-host-rpath | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/support/scripts/check-host-rpath b/support/scripts/check-host-rpath
> index 020c123..e79560b 100755
> --- a/support/scripts/check-host-rpath
> +++ b/support/scripts/check-host-rpath
> @@ -58,7 +58,7 @@ check_elf_has_rpath() {
>          for dir in ${rpath//:/ }; do
>              # Remove duplicate and trailing '/' for proper match
>              dir="$( sed -r -e 's:/+:/:g; s:/$::;' <<<"${dir}" )"
> -            [ "${dir}" = "${hostdir}/usr/lib" -o "${dir}" = "\$ORIGIN/../../usr/lib" ] && return 0
> +            [ "${dir}" = "${hostdir}/usr/lib" -o "${dir}" = "\$ORIGIN/../lib" ] && return 0

I guess this needs to be revisited with the removal of $(HOST_DIR)/usr:
now everything is under $(HOST_DIR) directly.

Could you rebase your series on the latest master, in which I just
merged earlier today Arnout patches removing $(HOST_DIR)/usr ?

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH v6 02/10] package/patchelf: add patch for rpath sanitization under a root directory

[Thomas Petazzoni]

Hello,

On Tue,  4 Jul 2017 18:22:31 +0200, Wolfgang Grandegger wrote:

> diff --git a/package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch b/package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch
> new file mode 100644
> index 0000000..46fd123
> --- /dev/null
> +++ b/package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch
> @@ -0,0 +1,49 @@
> +From a365bcb7d7025da51b33165ef7ebc7180199a05e Mon Sep 17 00:00:00 2001
> +From: Eelco Dolstra <eelco.dolstra@logicblox.com>
> +Date: Mon, 19 Sep 2016 17:31:37 +0200
> +Subject: [PATCH 14/30] Remove apparently incorrect usage of "static"

The 14/30 should be removed, just make it [PATCH]

> +

Please add here:

[Upstream commit: URL of the upstream commit.]
Signed-off-by: ...


> diff --git a/package/patchelf/0002-Extract-a-function-for-splitting-a-colon-separated-s.patch b/package/patchelf/0002-Extract-a-function-for-splitting-a-colon-separated-s.patch
> new file mode 100644
> index 0000000..63f3bce
> --- /dev/null
> +++ b/package/patchelf/0002-Extract-a-function-for-splitting-a-colon-separated-s.patch
> @@ -0,0 +1,58 @@
> +From 2e3fdc2030c75c19df6fc2924083cfad53856562 Mon Sep 17 00:00:00 2001
> +From: Tuomas Tynkkynen <tuomas@tuxera.com>
> +Date: Fri, 3 Jun 2016 23:03:51 +0300
> +Subject: [PATCH 08/30] Extract a function for splitting a colon-separated
> + string
> +
> +We're going to need this logic in another place, so make a function of
> +this.

Same comments here.


> diff --git a/package/patchelf/0003-Add-option-to-make-the-rpath-relative-under-a-specif.patch b/package/patchelf/0003-Add-option-to-make-the-rpath-relative-under-a-specif.patch
> new file mode 100644
> index 0000000..4f700d9
> --- /dev/null
> +++ b/package/patchelf/0003-Add-option-to-make-the-rpath-relative-under-a-specif.patch
> @@ -0,0 +1,419 @@
> +From af8d4a24a0ef613bdb47f0b1c3d962d59c53a4be Mon Sep 17 00:00:00 2001
> +From: Wolfgang Grandegger <wg@grandegger.com>
> +Date: Mon, 20 Feb 2017 16:29:24 +0100
> +Subject: [PATCH] Add option to make the rpath relative under a specified root
> + directory
> +
> +Running "patchelf" with the option "--make-rpath-relative ROOTDIR" will
> +modify or delete the RPATHDIRs according the following rules
> +similar to Martin's patches [1] making the Buildroot toolchaing/SDK
> +relocatable.
> +
> +RPATHDIR starts with "$ORIGIN":
> +    The original build-system already took care of setting a relative
> +    RPATH, resolve it and test if it's valid (does exist)
> +
> +RPATHDIR starts with ROOTDIR:
> +    The original build-system added some absolute RPATH (absolute on
> +    the build machine). Test if it's valid (does exist).
> +
> +ROOTDIR/RPATHDIR exists:
> +    The original build-system already took care of setting an absolute
> +    RPATH (absolute in the final rootfs), resolve it and test if it's
> +    valid (does exist).
> +
> +RPATHDIR points somewhere else:
> +    (can be anywhere: build trees, staging tree, host location,
> +    non-existing location, etc.). Just discard such a path.
> +
> +The option "--no-standard-libs" will discard RPATHDIRs ROOTDIR/lib and
> +ROOTDIR/usr/lib. Like "--shrink-rpath", RPATHDIRs are also discarded
> +if the directories do not contain a library referenced by the
> +DT_NEEDED fields.
> +If the option "--relative-to-file" is given, the rpath will start
> +with "$ORIGIN" making it relative to the ELF file, otherwise an
> +absolute path relative to ROOTDIR will be used.
> +
> +[1] http://lists.busybox.net/pipermail/buildroot/2016-April/159422.html

If this has been accepted upstream, add the URL to the upstream commit
here as well.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH v6 02/10] package/patchelf: add patch for rpath sanitization under a root directory

[Wolfgang Grandegger]

Hello,

Am 05.07.2017 um 13:16 schrieb Thomas Petazzoni:
> Hello,
> 
> On Tue,  4 Jul 2017 18:22:31 +0200, Wolfgang Grandegger wrote:
> 
>> diff --git a/package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch b/package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch
>> new file mode 100644
>> index 0000000..46fd123
>> --- /dev/null
>> +++ b/package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch
>> @@ -0,0 +1,49 @@
>> +From a365bcb7d7025da51b33165ef7ebc7180199a05e Mon Sep 17 00:00:00 2001
>> +From: Eelco Dolstra <eelco.dolstra@logicblox.com>
>> +Date: Mon, 19 Sep 2016 17:31:37 +0200
>> +Subject: [PATCH 14/30] Remove apparently incorrect usage of "static"
> 
> The 14/30 should be removed, just make it [PATCH]
> 
>> +
> 
> Please add here:
> 
> [Upstream commit: URL of the upstream commit.]
> Signed-off-by: ...

They (patchelf developers) don't add "Signed-off-by". Should I add mine 
anyway?

> 
>> diff --git a/package/patchelf/0002-Extract-a-function-for-splitting-a-colon-separated-s.patch b/package/patchelf/0002-Extract-a-function-for-splitting-a-colon-separated-s.patch
>> new file mode 100644
>> index 0000000..63f3bce
>> --- /dev/null
>> +++ b/package/patchelf/0002-Extract-a-function-for-splitting-a-colon-separated-s.patch
>> @@ -0,0 +1,58 @@
>> +From 2e3fdc2030c75c19df6fc2924083cfad53856562 Mon Sep 17 00:00:00 2001
>> +From: Tuomas Tynkkynen <tuomas@tuxera.com>
>> +Date: Fri, 3 Jun 2016 23:03:51 +0300
>> +Subject: [PATCH 08/30] Extract a function for splitting a colon-separated
>> + string
>> +
>> +We're going to need this logic in another place, so make a function of
>> +this.
> 
> Same comments here.
> 
> 
>> diff --git a/package/patchelf/0003-Add-option-to-make-the-rpath-relative-under-a-specif.patch b/package/patchelf/0003-Add-option-to-make-the-rpath-relative-under-a-specif.patch
>> new file mode 100644
>> index 0000000..4f700d9
>> --- /dev/null
>> +++ b/package/patchelf/0003-Add-option-to-make-the-rpath-relative-under-a-specif.patch
>> @@ -0,0 +1,419 @@
>> +From af8d4a24a0ef613bdb47f0b1c3d962d59c53a4be Mon Sep 17 00:00:00 2001
>> +From: Wolfgang Grandegger <wg@grandegger.com>
>> +Date: Mon, 20 Feb 2017 16:29:24 +0100
>> +Subject: [PATCH] Add option to make the rpath relative under a specified root
>> + directory
>> +
>> +Running "patchelf" with the option "--make-rpath-relative ROOTDIR" will
>> +modify or delete the RPATHDIRs according the following rules
>> +similar to Martin's patches [1] making the Buildroot toolchaing/SDK
>> +relocatable.
>> +
>> +RPATHDIR starts with "$ORIGIN":
>> +    The original build-system already took care of setting a relative
>> +    RPATH, resolve it and test if it's valid (does exist)
>> +
>> +RPATHDIR starts with ROOTDIR:
>> +    The original build-system added some absolute RPATH (absolute on
>> +    the build machine). Test if it's valid (does exist).
>> +
>> +ROOTDIR/RPATHDIR exists:
>> +    The original build-system already took care of setting an absolute
>> +    RPATH (absolute in the final rootfs), resolve it and test if it's
>> +    valid (does exist).
>> +
>> +RPATHDIR points somewhere else:
>> +    (can be anywhere: build trees, staging tree, host location,
>> +    non-existing location, etc.). Just discard such a path.
>> +
>> +The option "--no-standard-libs" will discard RPATHDIRs ROOTDIR/lib and
>> +ROOTDIR/usr/lib. Like "--shrink-rpath", RPATHDIRs are also discarded
>> +if the directories do not contain a library referenced by the
>> +DT_NEEDED fields.
>> +If the option "--relative-to-file" is given, the rpath will start
>> +with "$ORIGIN" making it relative to the ELF file, otherwise an
>> +absolute path relative to ROOTDIR will be used.
>> +
>> +[1] http://lists.busybox.net/pipermail/buildroot/2016-April/159422.html
> 
> If this has been accepted upstream, add the URL to the upstream commit
> here as well.

So far there was now response to my GitHub-Merge-Request. And it's now 
based on v0.9 for the known reasons.

Wolfgang.

[Buildroot] [PATCH v6 02/10] package/patchelf: add patch for rpath sanitization under a root directory

[Thomas Petazzoni]

Hello,

On Wed, 5 Jul 2017 14:35:34 +0200, Wolfgang Grandegger wrote:

> > [Upstream commit: URL of the upstream commit.]
> > Signed-off-by: ...  
> 
> They (patchelf developers) don't add "Signed-off-by". Should I add mine 
> anyway?

Yes, add yours. It's a way of describing how the patch has been added
to Buildroot.

> > If this has been accepted upstream, add the URL to the upstream commit
> > here as well.  
> 
> So far there was now response to my GitHub-Merge-Request.

OK.

> And it's now based on v0.9 for the known reasons.

Sure, but that doesn't prevent from indicating in this patch that it is
available upstream (if it's the case).

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH v6 10/10] package/qt5base: provide "qt.conf" to make "qmake" relocatable

[Wolfgang Grandegger]

Hello,

Am 05.07.2017 um 12:28 schrieb Wolfgang Grandegger:
> Am 05.07.2017 um 11:30 schrieb Wolfgang Grandegger:
>> Hello Arnout,
>>
>> Am 05.07.2017 um 10:59 schrieb Arnout Vandecappelle:
>>>    Hi Wolfgang,
>>>
>>> On 04-07-17 18:22, Wolfgang Grandegger wrote:
>>>> The file "qt.conf" can be used to override the hard-coded paths that are
>>>> compiled into the Qt library. We need it to make "qmake" relocatable.
>>>
>>>    All qt5 packages are failing because of this, e.g.
>>> http://autobuild.buildroot.net/results/33e/33eb7b7ef7d232bed2432d1d44fd80a4fc8c4e2d
>>>
>>>
>>> qt5serialport: installs files in
>>> /accts/mlweber1/instance-2/output/host/usr/x86_64-buildroot-linux-uclibc/sysroot//accts/mlweber1/instance-2/output
>>>
>>>
>>>
>>>    Can you have a look?
>>
>> I know the problem! The patched used an old version (from an old series)
>> of "qt.conf.in"..., sorry!
> 
> The patch is reverted now. So just for the records, the patch below
> fixes the problem. I'm going to provide a proper patch with my next
> series. This also shows, that if "qt.conf" is in place, it's active and
> will always overwrite the pathes compiled into qmake... which will break
> the idea of "make sdk" somehow.

What about making the relocation stuff configurable as shown in the patch below.
The advantage is that it could also control the generation of the "qt.conf",
for example.

Wolfgang.

[Buildroot] [PATCH v6 10/10] package/qt5base: provide "qt.conf" to make "qmake" relocatable

[Thomas Petazzoni]

Hello,

On Wed, 5 Jul 2017 15:48:22 +0200, Wolfgang Grandegger wrote:

> What about making the relocation stuff configurable as shown in the patch below.
> The advantage is that it could also control the generation of the "qt.conf",
> for example.

We don't want to make it configurable, through a Config.in option.
However, we would like to have a separate make target that prepares the
SDK, creates the tarball, etc.

Basically, a normal "make" would not run all the rpath sanitization
logic, absolute path replacement, etc. There should be a separate "make
sdk" target that does this.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH v6 02/10] package/patchelf: add patch for rpath sanitization under a root directory

[Arnout Vandecappelle]

On 05-07-17 14:35, Wolfgang Grandegger wrote:
> Hello,
> 
> Am 05.07.2017 um 13:16 schrieb Thomas Petazzoni:
>> Hello,
>>
>> On Tue,  4 Jul 2017 18:22:31 +0200, Wolfgang Grandegger wrote:
>>
>>> diff --git
>>> a/package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch
>>> b/package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch
>>> new file mode 100644
>>> index 0000000..46fd123
>>> --- /dev/null
>>> +++ b/package/patchelf/0001-Remove-apparently-incorrect-usage-of-static.patch
>>> @@ -0,0 +1,49 @@
>>> +From a365bcb7d7025da51b33165ef7ebc7180199a05e Mon Sep 17 00:00:00 2001
>>> +From: Eelco Dolstra <eelco.dolstra@logicblox.com>
>>> +Date: Mon, 19 Sep 2016 17:31:37 +0200
>>> +Subject: [PATCH 14/30] Remove apparently incorrect usage of "static"
>>
>> The 14/30 should be removed, just make it [PATCH]
>>
>>> +
>>
>> Please add here:
>>
>> [Upstream commit: URL of the upstream commit.]
>> Signed-off-by: ...
> 
> They (patchelf developers) don't add "Signed-off-by". Should I add mine anyway?

 You should add yours in the Buildroot patch, yes, as well as a reference to the
upstream commit. See e.g.
package/trinity/0002-fix-build-when_USE_BPF-is-not-defined.patch

[snip]
>> If this has been accepted upstream, add the URL to the upstream commit
>> here as well.
> 
> So far there was now response to my GitHub-Merge-Request. And it's now based on
> v0.9 for the known reasons.

 Even if the patch differs from the upstream patch, you should still point to
the upstream pull request so people can check it out when they want to bump
patchelf.

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH v6 10/10] package/qt5base: provide "qt.conf" to make "qmake" relocatable

[Wolfgang Grandegger]

Am 05.07.2017 um 16:13 schrieb Thomas Petazzoni:
> Hello,
> 
> On Wed, 5 Jul 2017 15:48:22 +0200, Wolfgang Grandegger wrote:
> 
>> What about making the relocation stuff configurable as shown in the patch below.
>> The advantage is that it could also control the generation of the "qt.conf",
>> for example.
> 
> We don't want to make it configurable, through a Config.in option.
> However, we would like to have a separate make target that prepares the
> SDK, creates the tarball, etc.
> 
> Basically, a normal "make" would not run all the rpath sanitization
> logic, absolute path replacement, etc. There should be a separate "make
> sdk" target that does this.

Yes I understand, but how do you want to handle static things like the 
creation of "qt.conf" required for the relocatable SDK?

Wolfgang.

[Buildroot] [PATCH v6 10/10] package/qt5base: provide "qt.conf" to make "qmake" relocatable

[Arnout Vandecappelle]

On 05-07-17 15:48, Wolfgang Grandegger wrote:
> Hello,
> 
> Am 05.07.2017 um 12:28 schrieb Wolfgang Grandegger:
>> Am 05.07.2017 um 11:30 schrieb Wolfgang Grandegger:
>>> Hello Arnout,
>>>
>>> Am 05.07.2017 um 10:59 schrieb Arnout Vandecappelle:
>>>>    Hi Wolfgang,
>>>>
>>>> On 04-07-17 18:22, Wolfgang Grandegger wrote:
>>>>> The file "qt.conf" can be used to override the hard-coded paths that are
>>>>> compiled into the Qt library. We need it to make "qmake" relocatable.
>>>>
>>>>    All qt5 packages are failing because of this, e.g.
>>>> http://autobuild.buildroot.net/results/33e/33eb7b7ef7d232bed2432d1d44fd80a4fc8c4e2d
>>>>
>>>>
>>>> qt5serialport: installs files in
>>>> /accts/mlweber1/instance-2/output/host/usr/x86_64-buildroot-linux-uclibc/sysroot//accts/mlweber1/instance-2/output
>>>>
>>>>
>>>>
>>>>    Can you have a look?
>>>
>>> I know the problem! The patched used an old version (from an old series)
>>> of "qt.conf.in"..., sorry!
>>
>> The patch is reverted now. So just for the records, the patch below
>> fixes the problem. I'm going to provide a proper patch with my next
>> series. This also shows, that if "qt.conf" is in place, it's active and
>> will always overwrite the pathes compiled into qmake... which will break
>> the idea of "make sdk" somehow.
> 
> What about making the relocation stuff configurable as shown in the patch below.
> The advantage is that it could also control the generation of the "qt.conf",
> for example.

 I don't really think this is worth it. Much simpler to always have the
possibility to sdk-ize it by running 'make sdk'.

> 
> Wolfgang.
> 
>>From 43ef9954f1b5b92e97b5774081a263f2ce068c5f Mon Sep 17 00:00:00 2001
> From: Wolfgang Grandegger <wg@grandegger.com>
> Date: Wed, 5 Jul 2017 15:36:21 +0200
> Subject: [RFC PATCH] core: add config option to make SDK (host dir) relocatable
> 
> ---
>  Config.in | 18 ++++++++++++++++++
>  Makefile  |  5 +++++
>  2 files changed, 23 insertions(+)
> 
> diff --git a/Config.in b/Config.in
> index 72ceadf..57f899c 100644
> --- a/Config.in
> +++ b/Config.in
> @@ -721,6 +721,24 @@ config BR2_REPRODUCIBLE
>  	  This is labeled as an experimental feature, as not all
>  	  packages behave properly to ensure reproducibility.
>  
> +config BR2_RELOCATABLE
> +	bool "Make a relocatable SDK (experimental)"
> +	help
> +	  This option will build a relocatable SDK (host dir). That
> +	  is the directory to store all the binary files that are
> +	  built for the host but also the cross compilation toolchain
> +	  when building the internal buildroot toolchain. It also
> +	  includes the "sysroot" tree required for cross compilation.
> +	  The relocatable host directory can then be moved to a new
> +	  location as shown below:
> +
> +	    $ cd <buildroot-build-directory>
> +	    $ cp -r host <the-new-directory>/sdk
> +	    $ <the-new-directory>/sdk/relocate-sdk.sh
> +
> +	  After executing the script "relocate-sdk.sh" once, it can
> +	  be used for external builds.

 This text is very useful but should be in the manual instead.

> +
>  endmenu
>  
>  endmenu
> diff --git a/Makefile b/Makefile
> index 57191d1..962f80a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -460,6 +460,7 @@ endif
>  # these locations, so export them so it is easier to use
>  export BR2_CONFIG
>  export BR2_REPRODUCIBLE
> +export BR2_RELOCATABLE
>  export TARGET_DIR
>  export STAGING_DIR
>  export HOST_DIR
> @@ -554,10 +555,12 @@ world: target-post-image host-finalize
>  
>  .PHONY: host-finalize
>  host-finalize:

 Make this 'sdk' instead of host-finalize, and add it to help: as well.

> +ifeq ($(BR2_RELOCATABLE),y)
>  	@$(call MESSAGE,"Rendering the SDK relocatable")
>  	$(TOPDIR)/support/scripts/fix-rpath host
>  	install $(TOPDIR)/support/misc/relocate-sdk.sh $(HOST_DIR)
>  	echo $(HOST_DIR) > $(HOST_DIR)/share/buildroot/sdk-location
> +endif
>  
>  # We need patchelf for RPATH sanitization
>  PACKAGES += host-patchelf
> @@ -730,10 +733,12 @@ endif
>  		$(call MESSAGE,"Executing post-build script $(s)"); \
>  		$(EXTRA_ENV) $(s) $(TARGET_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep))
>  
> +ifeq ($(BR2_RELOCATABLE),y)
>  	@$(call MESSAGE,"Sanitizing RPATH in staging tree")
>  	$(TOPDIR)/support/scripts/fix-rpath staging
>  	@$(call MESSAGE,"Sanitizing RPATH in target tree")
>  	$(TOPDIR)/support/scripts/fix-rpath target

 This has nothing to do with making it relocatable, it is always necessary to
sanitize the rpaths in target. Well, it's not really necessary, things work as
they are now, but it would be an improvement.


 Regards,
 Arnout


> +endif
>  
>  .PHONY: target-post-image
>  target-post-image: $(TARGETS_ROOTFS) target-finalize
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH v6 10/10] package/qt5base: provide "qt.conf" to make "qmake" relocatable

[Thomas Petazzoni]

Hello,

On Wed, 5 Jul 2017 16:33:35 +0200, Wolfgang Grandegger wrote:

> > We don't want to make it configurable, through a Config.in option.
> > However, we would like to have a separate make target that prepares the
> > SDK, creates the tarball, etc.
> > 
> > Basically, a normal "make" would not run all the rpath sanitization
> > logic, absolute path replacement, etc. There should be a separate "make
> > sdk" target that does this.  
> 
> Yes I understand, but how do you want to handle static things like the 
> creation of "qt.conf" required for the relocatable SDK?

In order to make sure that the SDK situation is as close to a normal
Buildroot usage, I think the qt.conf file should always be created and
used by qmake. So basically, the Qt package in Buildroot should create
and install this file.

Am I missing something here?

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH v6 10/10] package/qt5base: provide "qt.conf" to make "qmake" relocatable

[Arnout Vandecappelle]

On 05-07-17 16:33, Wolfgang Grandegger wrote:
> 
> 
> Am 05.07.2017 um 16:13 schrieb Thomas Petazzoni:
>> Hello,
>>
>> On Wed, 5 Jul 2017 15:48:22 +0200, Wolfgang Grandegger wrote:
>>
>>> What about making the relocation stuff configurable as shown in the patch below.
>>> The advantage is that it could also control the generation of the "qt.conf",
>>> for example.
>>
>> We don't want to make it configurable, through a Config.in option.
>> However, we would like to have a separate make target that prepares the
>> SDK, creates the tarball, etc.
>>
>> Basically, a normal "make" would not run all the rpath sanitization
>> logic, absolute path replacement, etc. There should be a separate "make
>> sdk" target that does this.
> 
> Yes I understand, but how do you want to handle static things like the creation
> of "qt.conf" required for the relocatable SDK?

 It should still work if you have the relocatable SDK option selected. So
basically we want the option to always be selected and things to still work.

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH v6 10/10] package/qt5base: provide "qt.conf" to make "qmake" relocatable

[Wolfgang Grandegger]

Hello,

Am 05.07.2017 um 16:48 schrieb Thomas Petazzoni:
> Hello,
> 
> On Wed, 5 Jul 2017 16:33:35 +0200, Wolfgang Grandegger wrote:
> 
>>> We don't want to make it configurable, through a Config.in option.
>>> However, we would like to have a separate make target that prepares the
>>> SDK, creates the tarball, etc.
>>>
>>> Basically, a normal "make" would not run all the rpath sanitization
>>> logic, absolute path replacement, etc. There should be a separate "make
>>> sdk" target that does this.
>>
>> Yes I understand, but how do you want to handle static things like the
>> creation of "qt.conf" required for the relocatable SDK?
> 
> In order to make sure that the SDK situation is as close to a normal
> Buildroot usage, I think the qt.conf file should always be created and
> used by qmake. So basically, the Qt package in Buildroot should create
> and install this file.
> 
> Am I missing something here?

It works here (with my usecase)! But the qmake internals are not really 
transparent (not documented) and may break if new path variables are 
added etc.(by Qt). But well, then we just need to fix it. Let see how 
good qt.conf works...

Wolfgang.