Re: [Buildroot] [PATCH] support/scripts/check-host-libs: add new check on host binaries/libs

[Thomas Petazzoni <thomas.petazzoni@bootlin.com>]

Hello,

Thanks for the quick feedback!

On Tue, 20 Sep 2022 09:46:24 +0200
<yann.morin@orange.com> wrote:


> I'm afraid this is going to be too big a hammer, at least if that's a
> hard error.
> 
> Indeed, when the build environment is reproducible (like, it is a
> container image), and there are no package in Buildroot, and the host
> directory will not be distributed (i.e. one does not care about doing an
> SDK), then it is usually good-enough to use the system-installed
> libraries, rather than add a Buildroot package.

True.

> > The script includes an allowlist of libraries provided by the C
> > library. It is potentially possible that this list might need to be
> > extended to cover all systems/distributions/C libraries, but only
> > wider testing of this script will help detect such cases.  
> 
> There are at least missing entries already:
>     librt.so.1
>     libutil.so.1
>     libresolv.so.2

Thanks, I'll add them.

> Also, there are error messages for some go stuff:
>     ==> HOST_DIR/lib/go/src/debug/elf/testdata/go-relocation-test-gcc930-ranges-no-rela-x86-64  
>     readelf: Error: no .dynamic section in the dynamic segment
> 
>     ==> HOST_DIR/lib/go/src/debug/elf/testdata/go-relocation-test-gcc930-ranges-with-rela-x86-64  
>     readelf: Error: no .dynamic section in the dynamic segment

I guess this can be resolved by simplying sending the readelf error
output to oblivion.

> Finally, this script takes more than a minute to run on our build, about
> a 10% increase from ~12min. This is not nice at all.
> 
>     $ find host/*bin host/lib* -type f |wc -l
>     16186
> 
>     $ find host/*bin host/lib* -type f |(keep just libs + execs) |wc -l
>     339
> 
> That last command, though, is what takes time: there is a huge ton of go
> junk installed in HOST_DIR, and this takes ages to filter-out.

Wow, 1 extra minute, this seems a lot. However, I don't understand what
you mean by "That last command". What exactly takes time? The fact that
we run "file" on zillion of files to filter out files that are not
executable/shared libraries? Or the readelf on the filtered files?

> So, although I understand the rationale, this should probably be opt-in.
> 
> Or it should only be ran when doing an SDK?

I am fine with this being opt-in, but I would not tie it to the SDK,
but rather to CI in the autobuilders. Indeed, while locally for your
own projects where you might control the build environment (using
containers, as you mentioned), in the general situation, Buildroot
tries to not use system libraries other than the C library. So having
this in the autobuilders would not impose the extra time on users, but
would allow us to detect a number of undetected spurious host
dependencies.

So, it would be a Config.in option, disabled by default. Autobuilders
would enable it, and users who are interested by the extra checks could
also enable it.

Thoughts?

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot