[Buildroot] [PATCH v3 0/3] Improve checking of host binaries

[Buildroot] [PATCH v3 0/3] Improve checking of host binaries

[Thomas Petazzoni via buildroot]

Hello,

This is v3 of my "check-host-libs" work, completely reworked following
the feedback from Yann E. Morin. Now the check is done directly as
part of check-host-rpath, which we have renamed check-host-bins. The
implementation is largely taken from Yann's proposal, with some minor
fix.

Note: do NOT apply this on master. At best on "next", or perhaps once
there is consensus that we want this (if we reach such a consensus),
then I can do a big build with lots of host packages on a system
"polluted" with lots of system-installed packages, to see how bad the
damage will be. Alternatively, getting some feedback from people
testing this on their configuration/system would be very useful.

Thanks!

Thomas

Thomas Petazzoni (3):
  support/scripts/check-host-rpath: fix shellcheck issues
  support/scripts/check-host-rpath: rename to check-host-bins
  support/scripts/check-host-bins: add new check on host binaries/libs

 .checkpackageignore                           |  1 -
 package/pkg-generic.mk                        | 10 ++--
 .../{check-host-rpath => check-host-bins}     | 54 +++++++++++++++----
 3 files changed, 50 insertions(+), 15 deletions(-)
 rename support/scripts/{check-host-rpath => check-host-bins} (73%)

-- 
2.55.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v3 1/3] support/scripts/check-host-rpath: fix shellcheck issues

[Thomas Petazzoni via buildroot]

Fixes:

In ./support/scripts/check-host-rpath line 21:
    while read file; do
          ^--^ SC2162 (info): read without -r will mangle backslashes.

In ./support/scripts/check-host-rpath line 62:
    while read lib; do
          ^--^ SC2162 (info): read without -r will mangle backslashes.

In ./support/scripts/check-host-rpath line 90:
    while read rpath; do
          ^--^ SC2162 (info): read without -r will mangle backslashes.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
Changes since v2:
- New in v3
---
 .checkpackageignore              | 1 -
 support/scripts/check-host-rpath | 6 +++---
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/.checkpackageignore b/.checkpackageignore
index c3ec943075..3cf8cabece 100644
--- a/.checkpackageignore
+++ b/.checkpackageignore
@@ -997,7 +997,6 @@ support/libtool/buildroot-libtool-v2.4.patch lib_patch.ApplyOrder lib_patch.Sob
 support/misc/relocate-sdk.sh Shellcheck
 support/scripts/apply-patches.sh Shellcheck
 support/scripts/check-bin-arch Shellcheck
-support/scripts/check-host-rpath Shellcheck
 support/scripts/expunge-gconv-modules Shellcheck
 support/scripts/fix-configure-powerpc64.sh lib_shellscript.EmptyLastLine
 support/scripts/generate-gitlab-ci-yml Shellcheck
diff --git a/support/scripts/check-host-rpath b/support/scripts/check-host-rpath
index 41aa0aa1ed..7b48af42ca 100755
--- a/support/scripts/check-host-rpath
+++ b/support/scripts/check-host-rpath
@@ -18,7 +18,7 @@ main() {
     hostdir="$( sed -r -e 's:/+:/:g; s:/$::;' <<<"${hostdir}" )"
 
     ret=0
-    while read file; do
+    while read -r file; do
         is_elf "${file}" || continue
         elf_needs_rpath "${file}" "${hostdir}" || continue
         check_elf_has_rpath "${file}" "${hostdir}" "${perpackagedir}" && continue
@@ -59,7 +59,7 @@ elf_needs_rpath() {
     local hostdir="${2}"
     local lib
 
-    while read lib; do
+    while read -r lib; do
         [ -e "${hostdir}/lib/${lib}" ] && return 0
     done < <( readelf -d "${file}" 2>/dev/null                             \
               |sed -r -e '/^.* \(NEEDED\) .*Shared library: \[(.+)\]$/!d;' \
@@ -87,7 +87,7 @@ check_elf_has_rpath() {
     local perpackagedir="${3}"
     local rpath dir
 
-    while read rpath; do
+    while read -r rpath; do
         for dir in ${rpath//:/ }; do
             # Remove duplicate and trailing '/' for proper match
             dir="$( sed -r -e 's:/+:/:g; s:/$::;' <<<"${dir}" )"
-- 
2.55.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v3 2/3] support/scripts/check-host-rpath: rename to check-host-bins

[Thomas Petazzoni via buildroot]

As we are about to extend check-host-rpaths to check more than the
RPATH of host binaries, let's rename it check-host-bins to indicate
that it runs various checks on binaries installed by host packages.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
Changes since v2:
- New in v3
---
 package/pkg-generic.mk                                | 10 +++++-----
 support/scripts/{check-host-rpath => check-host-bins} |  0
 2 files changed, 5 insertions(+), 5 deletions(-)
 rename support/scripts/{check-host-rpath => check-host-bins} (100%)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 09d6e5b20b..34e24f5aa9 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -55,13 +55,13 @@ define step_time
 endef
 GLOBAL_INSTRUMENTATION_HOOKS += step_time
 
-# This hook checks that host packages that need libraries that we build
-# have a proper DT_RPATH or DT_RUNPATH tag
-define check_host_rpath
+# This hook runs various checks on the binaries installed by host
+# packages.
+define check_host_bins
 	$(if $(filter install-host,$(2)),\
-		$(if $(filter end,$(1)),support/scripts/check-host-rpath $(3) $(HOST_DIR) $(PER_PACKAGE_DIR)))
+		$(if $(filter end,$(1)),support/scripts/check-host-bins $(3) $(HOST_DIR) $(PER_PACKAGE_DIR)))
 endef
-GLOBAL_INSTRUMENTATION_HOOKS += check_host_rpath
+GLOBAL_INSTRUMENTATION_HOOKS += check_host_bins
 
 define step_check_build_dir_one
 	if [ -d $(2) ]; then \
diff --git a/support/scripts/check-host-rpath b/support/scripts/check-host-bins
similarity index 100%
rename from support/scripts/check-host-rpath
rename to support/scripts/check-host-bins
-- 
2.55.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v3 3/3] support/scripts/check-host-bins: add new check on host binaries/libs

[Thomas Petazzoni via buildroot]

One frequent issue in Buildroot is that when building host libraries
or applications, the build system of the package detects some
libraries provided by the system, and happily links to them, without
Buildroot knowing. Sometimes this doesn't cause any problem, but
sometimes this causes issues, and we're regularly eliminating such
mis-detection by forcing those packages to not detect the system
libraries that have not been built by Buildroot.

Based on a suggestion from Yann E. Morin, this commit extends
check-host-bins to verify that all binaries and libraries in
$(HOST_DIR) only have shared library dependencies on libraries that
are in Buildroot $(HOST_DIR), to the exception of the C library (and
friends), for which we of course use the system C library.

For example, if the binary output/host/bin/plop is linked against
libpng, but libpng was not built and installed by Buildroot, the build
will now fail with:

*** ERROR: package host-gdb uses libs not in HOST_DIR:
  - liblzma.so.5
  - libxxhash.so.0

The script includes an allowlist of libraries provided by the C
library. It is potentially possible that this list might need to be
extended to cover all systems/distributions/C libraries, but only
wider testing of this script will help detect such cases.

Co-developed-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
It would be very useful if a few people could apply this patch to
their local tree, run their usual build, and see how it behaves. This
way, I can get some feedback to address the most obvious issues before
it gets merged and starts causing build failures in the autobuilders.

Changes since v2:
- Implement the check in the existing check-host-bins script, as
  suggested by Yann E. Morin.

Changes since v1:
- Replaced the per-file file --mime-type checks with direct ELF magic
  detection using Bash read -N 4, significantly reducing scan time.
- Switched file traversal to NUL-delimited find -print0 output, safely
  handling paths containing whitespace.
- Suppressed harmless readelf errors for valid ELF object files
  without a dynamic section, such as the Go test fixtures mentioned
  during review.
- Added librt.so*, libutil.so*, and libresolv.so* to the
  system-library allowlist.
- Quoted file and host-directory paths where appropriate.
- Fixed shellcheck issues

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 support/scripts/check-host-bins | 48 ++++++++++++++++++++++++++++-----
 1 file changed, 42 insertions(+), 6 deletions(-)

diff --git a/support/scripts/check-host-bins b/support/scripts/check-host-bins
index 7b48af42ca..098e35e592 100755
--- a/support/scripts/check-host-bins
+++ b/support/scripts/check-host-bins
@@ -21,6 +21,14 @@ main() {
     while read -r file; do
         is_elf "${file}" || continue
         elf_needs_rpath "${file}" "${hostdir}" || continue
+	missing_libs="$(get_missing_libs "${file}" "${hostdir}")"
+        if [ "${missing_libs}" ]; then
+            ret=1
+            printf "***\n"
+            printf "*** ERROR: package %s uses libs not in HOST_DIR:\n" "${pkg}"
+            # shellcheck disable=SC2086 # we need the word splitting
+            printf '  - %s\n' ${missing_libs}
+        fi
         check_elf_has_rpath "${file}" "${hostdir}" "${perpackagedir}" && continue
         if [ ${ret} -eq 0 ]; then
             ret=1
@@ -57,16 +65,44 @@ is_elf() {
 elf_needs_rpath() {
     local file="${1}"
     local hostdir="${2}"
+
+    [ -n "$(get_libs "${file}" "${hostdir}")" ]
+}
+
+# This function returns all non-toolchain libs that are not in HOST_DIR
+get_missing_libs() {
+    local file="${1}"
+    local hostdir="${2}"
+    local lib
+
+    get_libs "${file}" "${hostdir}" \
+    | while read -r lib; do
+        if [ ! -e "${hostdir}/lib/${lib}" ]; then
+            printf '%s\n' "${lib}"
+        fi
+    done
+}
+
+# This function returns the list of non-toolchain libraries that an
+# ELF file has as DT_NEEDED
+get_libs() {
+    local file="${1}"
+    local hostdir="${2}"
     local lib
 
     while read -r lib; do
-        [ -e "${hostdir}/lib/${lib}" ] && return 0
-    done < <( readelf -d "${file}" 2>/dev/null                             \
-              |sed -r -e '/^.* \(NEEDED\) .*Shared library: \[(.+)\]$/!d;' \
-                     -e 's//\1/;'                                          \
+        case "${lib}" in
+        libc.so*|libm.so*|libstdc++.so*|libpthread.so*|libgcc_s.so*|libdl.so*|ld-*|libgomp.so*|libcrypt.so*|libatomic.so*|librt.so*|libutil.so*|libresolv.so*)
+            continue
+            ;;
+        *)
+            printf '%s\n' "${lib}"
+            ;;
+        esac
+    done < <( readelf -d "${file}" 2>/dev/null                                 \
+                  |sed -r -e '/^.* \(NEEDED\) .*Shared library: \[(.+)\]$/!d;' \
+                  -e 's//\1/;'                                                 \
             )
-
-    return 1
 }
 
 # This function checks whether at least one of the RPATH of the given
-- 
2.55.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 3/3] support/scripts/check-host-bins: add new check on host binaries/libs

[Matthew Weber]

Thomas/Yann,

On Tue, Aug 25, 2026 at 4:42 PM Thomas Petazzoni via buildroot
<buildroot@buildroot.org> wrote:
>
> One frequent issue in Buildroot is that when building host libraries
> or applications, the build system of the package detects some
> libraries provided by the system, and happily links to them, without
> Buildroot knowing. Sometimes this doesn't cause any problem, but
> sometimes this causes issues, and we're regularly eliminating such
> mis-detection by forcing those packages to not detect the system
> libraries that have not been built by Buildroot.
>
> Based on a suggestion from Yann E. Morin, this commit extends
> check-host-bins to verify that all binaries and libraries in
> $(HOST_DIR) only have shared library dependencies on libraries that
> are in Buildroot $(HOST_DIR), to the exception of the C library (and
> friends), for which we of course use the system C library.
>
> For example, if the binary output/host/bin/plop is linked against
> libpng, but libpng was not built and installed by Buildroot, the build
> will now fail with:
>
> *** ERROR: package host-gdb uses libs not in HOST_DIR:
>   - liblzma.so.5
>   - libxxhash.so.0
>
> The script includes an allowlist of libraries provided by the C
> library. It is potentially possible that this list might need to be
> extended to cover all systems/distributions/C libraries, but only
> wider testing of this script will help detect such cases.
>
> Co-developed-by: Yann E. MORIN <yann.morin.1998@free.fr>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
> It would be very useful if a few people could apply this patch to
> their local tree, run their usual build, and see how it behaves. This
> way, I can get some feedback to address the most obvious issues before
> it gets merged and starts causing build failures in the autobuilders.

I did some testing and found the musl library needs added to the
whitelist for Alpine.  I also added libraries to the scan which added
more changes then what I could easily include here.  Here's my mocked
up check-host-bins and test script.  The test case 5 was modeled after
the distro tests that xen does to target different containers to find
whitelist gaps.

https://gist.github.com/matthew-l-weber/34fae15a4319a81462961eadf0e31164

Some of this was AI generated, does Buildroot have any rules around AI use?

Best Regards,
Matt
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 3/3] support/scripts/check-host-bins: add new check on host binaries/libs

[Thomas Petazzoni via buildroot]

Hello Matt,

On Wed, Aug 26, 2026 at 11:19:37PM -0500, Matthew Weber wrote:

> I did some testing and found the musl library needs added to the
> whitelist for Alpine.  I also added libraries to the scan which added
> more changes then what I could easily include here.  Here's my mocked
> up check-host-bins and test script.  The test case 5 was modeled after
> the distro tests that xen does to target different containers to find
> whitelist gaps.

Thanks for the testing. Instead of having a full new version of the
script with all fixes mixed together, do you think you could spit out
each fix as an independent patch, so that the changes can more easily
be reviewed? Especially some fixes don't seem necessarily related to
the new library check, but would be applicable to the existing
check-host-rpath code.

> https://gist.github.com/matthew-l-weber/34fae15a4319a81462961eadf0e31164
> 
> Some of this was AI generated, does Buildroot have any rules around AI use?

As far as I'm aware, we don't have rules around AI use. I'm also using
AI to help in some of my recent Buildroot work.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot