Re: [Buildroot] [PATCH] support/scripts/check-host-libs: add new check on host binaries/libs

[Thomas Petazzoni via buildroot <buildroot@buildroot.org>]

Howdy,

On Tue, 20 Sep 2022 11:11:43 +0200
<yann.morin@orange.com> wrote:

> Sorry, I was not explicit enough. Yes, the filtering only, i.e. only
> running 'file' on all those 16k+ files takes more than a minute:
> 
>     $ date +%s; find host/*bin host/lib* -type f |while read f; do
>         mime=$(file -b --mime-type ${f})
>         if test "${mime}" != "application/x-sharedlib" -a
>                 "${mime}" != "application/x-executable" ; then
>             continue
>         fi
>         printf '%s\n' "${f}"
>     done |wc -l; date +%s
> 
>     1663659605
>     339
>     1663659681
> 
> I.e. 76 seconds just to identify the files to actually look at.

ACK.

> Or we can see at optimising it. Overall, I am not a fan of "if it's too
> slow, don't do it", but I prefer "if it's too slow, make it faster".

Sure :-)

> So, we can start by spawning less processes, then use sed to filter out
> the result:
> 
>     find ${HOST_DIR}/*bin ${HOST_DIR}/lib* -type f -print 0 \
>     |xargs -0 -r file --mime-type \
>     |sed -r -e '/^(.+): application/(x-executable|x-sharedlib)$/!d; s//\1/' \
>     |while read f; do
>         readelf blabla...
>     done

If it's really running "file" on a zillion files that is slow, I'm not
sure how this can really improve the performance: it still runs "file"
on zillion files.

Also, I initially had some constructs with a ... | while read f, but
the variables in the sub-shell where not visible outside, so the
"bailout" variable didn't work.

An alternative would be to try to do this in Python and use
parallelization.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot