[Buildroot] [PATCH v3 3/3] support/scripts/check-host-bins: add new check on host binaries/libs

[Thomas Petazzoni via buildroot <buildroot@buildroot.org>]

One frequent issue in Buildroot is that when building host libraries
or applications, the build system of the package detects some
libraries provided by the system, and happily links to them, without
Buildroot knowing. Sometimes this doesn't cause any problem, but
sometimes this causes issues, and we're regularly eliminating such
mis-detection by forcing those packages to not detect the system
libraries that have not been built by Buildroot.

Based on a suggestion from Yann E. Morin, this commit extends
check-host-bins to verify that all binaries and libraries in
$(HOST_DIR) only have shared library dependencies on libraries that
are in Buildroot $(HOST_DIR), to the exception of the C library (and
friends), for which we of course use the system C library.

For example, if the binary output/host/bin/plop is linked against
libpng, but libpng was not built and installed by Buildroot, the build
will now fail with:

*** ERROR: package host-gdb uses libs not in HOST_DIR:
  - liblzma.so.5
  - libxxhash.so.0

The script includes an allowlist of libraries provided by the C
library. It is potentially possible that this list might need to be
extended to cover all systems/distributions/C libraries, but only
wider testing of this script will help detect such cases.

Co-developed-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
It would be very useful if a few people could apply this patch to
their local tree, run their usual build, and see how it behaves. This
way, I can get some feedback to address the most obvious issues before
it gets merged and starts causing build failures in the autobuilders.

Changes since v2:
- Implement the check in the existing check-host-bins script, as
  suggested by Yann E. Morin.

Changes since v1:
- Replaced the per-file file --mime-type checks with direct ELF magic
  detection using Bash read -N 4, significantly reducing scan time.
- Switched file traversal to NUL-delimited find -print0 output, safely
  handling paths containing whitespace.
- Suppressed harmless readelf errors for valid ELF object files
  without a dynamic section, such as the Go test fixtures mentioned
  during review.
- Added librt.so*, libutil.so*, and libresolv.so* to the
  system-library allowlist.
- Quoted file and host-directory paths where appropriate.
- Fixed shellcheck issues

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 support/scripts/check-host-bins | 48 ++++++++++++++++++++++++++++-----
 1 file changed, 42 insertions(+), 6 deletions(-)

diff --git a/support/scripts/check-host-bins b/support/scripts/check-host-bins
index 7b48af42ca..098e35e592 100755
--- a/support/scripts/check-host-bins
+++ b/support/scripts/check-host-bins
@@ -21,6 +21,14 @@ main() {
     while read -r file; do
         is_elf "${file}" || continue
         elf_needs_rpath "${file}" "${hostdir}" || continue
+	missing_libs="$(get_missing_libs "${file}" "${hostdir}")"
+        if [ "${missing_libs}" ]; then
+            ret=1
+            printf "***\n"
+            printf "*** ERROR: package %s uses libs not in HOST_DIR:\n" "${pkg}"
+            # shellcheck disable=SC2086 # we need the word splitting
+            printf '  - %s\n' ${missing_libs}
+        fi
         check_elf_has_rpath "${file}" "${hostdir}" "${perpackagedir}" && continue
         if [ ${ret} -eq 0 ]; then
             ret=1
@@ -57,16 +65,44 @@ is_elf() {
 elf_needs_rpath() {
     local file="${1}"
     local hostdir="${2}"
+
+    [ -n "$(get_libs "${file}" "${hostdir}")" ]
+}
+
+# This function returns all non-toolchain libs that are not in HOST_DIR
+get_missing_libs() {
+    local file="${1}"
+    local hostdir="${2}"
+    local lib
+
+    get_libs "${file}" "${hostdir}" \
+    | while read -r lib; do
+        if [ ! -e "${hostdir}/lib/${lib}" ]; then
+            printf '%s\n' "${lib}"
+        fi
+    done
+}
+
+# This function returns the list of non-toolchain libraries that an
+# ELF file has as DT_NEEDED
+get_libs() {
+    local file="${1}"
+    local hostdir="${2}"
     local lib
 
     while read -r lib; do
-        [ -e "${hostdir}/lib/${lib}" ] && return 0
-    done < <( readelf -d "${file}" 2>/dev/null                             \
-              |sed -r -e '/^.* \(NEEDED\) .*Shared library: \[(.+)\]$/!d;' \
-                     -e 's//\1/;'                                          \
+        case "${lib}" in
+        libc.so*|libm.so*|libstdc++.so*|libpthread.so*|libgcc_s.so*|libdl.so*|ld-*|libgomp.so*|libcrypt.so*|libatomic.so*|librt.so*|libutil.so*|libresolv.so*)
+            continue
+            ;;
+        *)
+            printf '%s\n' "${lib}"
+            ;;
+        esac
+    done < <( readelf -d "${file}" 2>/dev/null                                 \
+                  |sed -r -e '/^.* \(NEEDED\) .*Shared library: \[(.+)\]$/!d;' \
+                  -e 's//\1/;'                                                 \
             )
-
-    return 1
 }
 
 # This function checks whether at least one of the RPATH of the given
-- 
2.55.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot