From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2A17C0018A for ; Wed, 1 Nov 2023 15:21:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 5FDA384BEA; Wed, 1 Nov 2023 15:21:33 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 5FDA384BEA X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TK7DJbH-kCBN; Wed, 1 Nov 2023 15:21:30 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 758F184BE8; Wed, 1 Nov 2023 15:21:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 758F184BE8 Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id C36AC1BF35B for ; Wed, 1 Nov 2023 15:21:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 9C1394EC69 for ; Wed, 1 Nov 2023 15:21:27 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 9C1394EC69 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nZi_SKmv-rV0 for ; Wed, 1 Nov 2023 15:21:23 +0000 (UTC) Received: from smtp4-g21.free.fr (smtp4-g21.free.fr [212.27.42.4]) by smtp4.osuosl.org (Postfix) with ESMTPS id D3C564EC43 for ; Wed, 1 Nov 2023 15:21:22 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org D3C564EC43 Received: from ymorin.is-a-geek.org (unknown [IPv6:2a01:cb19:8290:3800:67d2:c63c:7c2a:c31]) (Authenticated sender: yann.morin.1998@free.fr) by smtp4-g21.free.fr (Postfix) with ESMTPSA id 9603719F748; Wed, 1 Nov 2023 16:21:14 +0100 (CET) Received: by ymorin.is-a-geek.org (sSMTP sendmail emulation); Wed, 01 Nov 2023 16:21:14 +0100 Date: Wed, 1 Nov 2023 16:21:14 +0100 From: "Yann E. MORIN" To: Giulio Benetti Message-ID: <20231101152114.GE114892@scaer> References: <20231031120713.678783-1-giulio.benetti@benettiengineering.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20231031120713.678783-1-giulio.benetti@benettiengineering.com> User-Agent: Mutt/1.5.22 (2013-10-16) X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=free.fr; s=smtp-20201208; t=1698852079; bh=KOrowVNh8QsMhHFxal9Bv2Y5+gPdFfGYdDKdqo0YOb8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=AkN9EUymzgIcyB6YMZRyrylEpfrs+Sr2DzkdVa0dF0rdk0KKxyPo7lcBJjjXPEG1A Iejl1M1qqLr+iQgX7X8ymQM1mnaW3rmJ8JwlTC7eY4gla85Ct9GxhXi+jhDZPQQQUj lwtvHxOTDSohs4glCF1KumAvlv7BhCgp5HTS4WjhHVMZlN5lMn+lf28SaYJXlthY8W yCMQLy/lME1aseOQdCKzZP11gPH6wY9UjTAmJCYrkv/zcJbbraV6F6zF5FmHLSUMac ThBv2gnZ3JTmBfIxnb7Lg70GMtab2Tqd3QbR3J6R5ROur8nd8Qrnp5Zzfz9/f/XgzR Zyt4XH1Kd1NIQ== X-Mailman-Original-Authentication-Results: smtp4.osuosl.org; dkim=pass (2048-bit key) header.d=free.fr header.i=@free.fr header.a=rsa-sha256 header.s=smtp-20201208 header.b=AkN9EUym Subject: Re: [Buildroot] [PATCH] package/nodejs/nodejs-src: remove .node files with different architecture X-BeenThere: buildroot@buildroot.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Martin Bark , Daniel Price , Thomas Petazzoni , buildroot@buildroot.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: buildroot-bounces@buildroot.org Sender: "buildroot" Giulio, All, On 2023-10-31 13:07 +0100, Giulio Benetti spake thusly: > Actually nodejs-src fails to build when additional modules with prebuilt > .node files are added. This is due to Buildroot's check-bin-arch that > checks for executable files and libraries that have different architecture > from the one we're building for. So let's go through all .node files in > $(TARGET_DIR)/usr/lib/node_modules and check if the architecture is > different or not found and in case delete the .node file. Unfortunately, this is not going to work correctly. For example, the module failing in #15823 is reported to be serialport, but really the failing one is the scoped @serialport/bindings-cpp module. That module has "prebuilds" binaries for various systems, here's the list: $ find prebuilds/ -type f prebuilds/android-arm/node.napi.armv7.node prebuilds/darwin-x64+arm64/node.napi.node prebuilds/android-arm64/node.napi.armv8.node prebuilds/win32-ia32/node.napi.node prebuilds/linux-arm/node.napi.armv7.node prebuilds/linux-arm/node.napi.armv6.node prebuilds/linux-x64/node.napi.musl.node prebuilds/linux-x64/node.napi.glibc.node prebuilds/linux-arm64/node.napi.armv8.node prebuilds/win32-x64/node.napi.node and running readelf -h on each will report many modules actually matching the current machine. For example, those two will both match AArch64, while obviously we only want to keep the latter: prebuilds/android-arm64/node.napi.armv8.node prebuilds/linux-arm64/node.napi.armv8.node Same goes for arm, and similar goes for x64 musl vs. glibc. Those last two are also very interesting: it means that modules need to be built properly against the correct C library, not just the target CPU. Which means that, even a module matching the current arch is not necessarily applicable to said arch if it was built for another C library... Which leads to the question: what happens for such a module when there is no prebuilt binary for the current arch? Is it going to be compiled by npm when it installs the module? Thanks to Adam, I could inspect the tarball of @serial/bindings-cpp@12.0.1, and I could spot .cpp and h files in the src/ subidr: $ wget 'https://registry.npmjs.org/@serialport/bindings-cpp/-/bindings-cpp-12.0.1.tgz' $ tar tzf bindings-cpp-12.0.1.tgz |grep src/ package/src/darwin_list.cpp package/src/poller.cpp package/src/serialport_linux.cpp package/src/serialport_unix.cpp package/src/serialport_win.cpp package/src/serialport.cpp package/src/darwin_list.h package/src/poller.h package/src/serialport_linux.h package/src/serialport_unix.h package/src/serialport_win.h package/src/serialport.h There's also binding.gyp that looks like it lists what to build under which condition. In which case, how does npm decide that it can use one such prebuild, or whether it should compile from the source files? Can we force it to never use prebuilds and always compile? I'm afraid the proposed patch just covers up a bigger issue by sweeping the dust under the rug... It will indeed make check-bin-arch happy because no binary will indeed match another arch, indeed, but that does not mean the module will be actually correctly installed... So yes, npm is such a mess that maybe sweeping the dust under the rug is the only solution we can implement, but it is disappointing... Maybe we can just also exclude the node modules from check-bin-arch altogether and be done with it: NODEJS_SRC_BIN_ARCH_EXCLUDE = /usr/lib/node_modules/ Regards, Yann E. MORIN. > Fixes: > https://bugs.busybox.net/show_bug.cgi?id=15823 > > Signed-off-by: Giulio Benetti > --- > package/nodejs/nodejs-src/nodejs-src.mk | 12 ++++++++++++ > 1 file changed, 12 insertions(+) > > diff --git a/package/nodejs/nodejs-src/nodejs-src.mk b/package/nodejs/nodejs-src/nodejs-src.mk > index 3452c93728..bdbf0709c6 100644 > --- a/package/nodejs/nodejs-src/nodejs-src.mk > +++ b/package/nodejs/nodejs-src/nodejs-src.mk > @@ -247,6 +247,18 @@ define NODEJS_SRC_INSTALL_MODULES > # npm install call below and setting npm_config_rollback=false can both > # help in diagnosing the problem. > $(NPM) install -g $(NODEJS_SRC_MODULES_LIST) > + > + # Remove prebuilt files which are not compatible with the architecture > + # and OS(Linux) we're building for. NOTE: .node files that don't have a > + # readelf output have different ABI(i.e. Windows, Darwin etc.) > + for f in $$(find $(TARGET_DIR)/usr/lib/node_modules -type f -name "*.node"); do \ > + echo $$f; \ > + arch=`$(TARGET_READELF) -h "$$f" 2>&1 | \ > + sed -r -e '/^ Machine: +(.+)/!d; s//\1/;' | head -1`; \ > + if [ "$$arch" != "$(BR2_READELF_ARCH_NAME)" ]; then \ > + rm -f $$f; \ > + fi \ > + done > endef > endif > > -- > 2.34.1 > > _______________________________________________ > buildroot mailing list > buildroot@buildroot.org > https://lists.buildroot.org/mailman/listinfo/buildroot -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' _______________________________________________ buildroot mailing list buildroot@buildroot.org https://lists.buildroot.org/mailman/listinfo/buildroot