npm based packages omit dependencies since Yocto 5.1

npm based packages omit dependencies since Yocto 5.1

[Böszörményi Zoltán]

Hi,

I have a minimalistic recipe for pm2 (https://www.npmjs.com/package/pm2):
==================================
node-pm2_5.3.1.bb
==================================
SUMMARY = "Production process manager for Node.JS applications with a built-in load balancer."
HOMEPAGE = "http://pm2.keymetrics.io/"
LICENSE = "AGPL-3.0-or-later"
LIC_FILES_CHKSUM = "file://GNU-AGPL-3.0.txt;md5=9f9fa01c099265d62f73735e404ff8bb"

SRC_URI = " \
     npm://registry.npmjs.org/;package=pm2;version=${PV} \
     npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \
     "

S = "${UNPACKDIR}/npm"

inherit npm

do_install:append () {
         rm -rf ${D}${libdir}/node_modules/pm2/lib/templates/init-scripts/openrc.tpl
}

RDEPENDS:${PN} = "bash"
==================================

Previously this recipe has built and worked correctly under
Yocto 4.3 and 5.0, as in the package included its own internal
dependencies in /usr/lib/node_modules/pm2/node_modules.
Yocto 4.3 ships nodejs 20.8.1, Yocto 5.0 ships nodejs 20.12.2.

With Yocto 5.1 (node 20.18.0) and master (22.12.0), the package
only has the pm2 module contents in /usr/lib/node_modules/pm2
but no dependencies.

It seems that at some point, nodejs 20.x LTS upgraded npm
to a version with a breaking change which now flattens the
dependencies into a single level in /usr/lib/node_modules
from the previously accustomed dependency tree in
/usr/lib/node_modules/pm2/node_modules.

Is there an incantation of EXTRA_OENPM or the npm:// SRC_URI
that would allow either to unflatten the dependency tree, or
include the dependencies as well as the main node module?

Thanks in advance,
Zoltán Böszörményi

Re: [OE-core] npm based packages omit dependencies since Yocto 5.1

[Alexander Kanavin]

There have been recent npm fetcher fixes in master, can you try that please?

Alex

On Tue, 21 Jan 2025 at 11:31, Zoltan Boszormenyi via
lists.openembedded.org <zboszor=gmail.com@lists.openembedded.org>
wrote:
>
> Hi,
>
> I have a minimalistic recipe for pm2 (https://www.npmjs.com/package/pm2):
> ==================================
> node-pm2_5.3.1.bb
> ==================================
> SUMMARY = "Production process manager for Node.JS applications with a built-in load balancer."
> HOMEPAGE = "http://pm2.keymetrics.io/"
> LICENSE = "AGPL-3.0-or-later"
> LIC_FILES_CHKSUM = "file://GNU-AGPL-3.0.txt;md5=9f9fa01c099265d62f73735e404ff8bb"
>
> SRC_URI = " \
>      npm://registry.npmjs.org/;package=pm2;version=${PV} \
>      npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \
>      "
>
> S = "${UNPACKDIR}/npm"
>
> inherit npm
>
> do_install:append () {
>          rm -rf ${D}${libdir}/node_modules/pm2/lib/templates/init-scripts/openrc.tpl
> }
>
> RDEPENDS:${PN} = "bash"
> ==================================
>
> Previously this recipe has built and worked correctly under
> Yocto 4.3 and 5.0, as in the package included its own internal
> dependencies in /usr/lib/node_modules/pm2/node_modules.
> Yocto 4.3 ships nodejs 20.8.1, Yocto 5.0 ships nodejs 20.12.2.
>
> With Yocto 5.1 (node 20.18.0) and master (22.12.0), the package
> only has the pm2 module contents in /usr/lib/node_modules/pm2
> but no dependencies.
>
> It seems that at some point, nodejs 20.x LTS upgraded npm
> to a version with a breaking change which now flattens the
> dependencies into a single level in /usr/lib/node_modules
> from the previously accustomed dependency tree in
> /usr/lib/node_modules/pm2/node_modules.
>
> Is there an incantation of EXTRA_OENPM or the npm:// SRC_URI
> that would allow either to unflatten the dependency tree, or
> include the dependencies as well as the main node module?
>
> Thanks in advance,
> Zoltán Böszörményi
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#210084): https://lists.openembedded.org/g/openembedded-core/message/210084
> Mute This Topic: https://lists.openembedded.org/mt/110731121/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] npm based packages omit dependencies since Yocto 5.1

[Martin Jansa]

I believe it's the side effect of UNPACKDIR changes, I had to add
;destsuffix=npm in all npmsw:// entries (or ;destsuffix=git where S is
set to WORKDIR/git) for dependencies in node_modules to be unpacked
where they used to be before.

On Tue, Jan 21, 2025 at 11:31 AM Zoltan Boszormenyi via
lists.openembedded.org <zboszor=gmail.com@lists.openembedded.org>
wrote:
>
> Hi,
>
> I have a minimalistic recipe for pm2 (https://www.npmjs.com/package/pm2):
> ==================================
> node-pm2_5.3.1.bb
> ==================================
> SUMMARY = "Production process manager for Node.JS applications with a built-in load balancer."
> HOMEPAGE = "http://pm2.keymetrics.io/"
> LICENSE = "AGPL-3.0-or-later"
> LIC_FILES_CHKSUM = "file://GNU-AGPL-3.0.txt;md5=9f9fa01c099265d62f73735e404ff8bb"
>
> SRC_URI = " \
>      npm://registry.npmjs.org/;package=pm2;version=${PV} \
>      npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \
>      "
>
> S = "${UNPACKDIR}/npm"
>
> inherit npm
>
> do_install:append () {
>          rm -rf ${D}${libdir}/node_modules/pm2/lib/templates/init-scripts/openrc.tpl
> }
>
> RDEPENDS:${PN} = "bash"
> ==================================
>
> Previously this recipe has built and worked correctly under
> Yocto 4.3 and 5.0, as in the package included its own internal
> dependencies in /usr/lib/node_modules/pm2/node_modules.
> Yocto 4.3 ships nodejs 20.8.1, Yocto 5.0 ships nodejs 20.12.2.
>
> With Yocto 5.1 (node 20.18.0) and master (22.12.0), the package
> only has the pm2 module contents in /usr/lib/node_modules/pm2
> but no dependencies.
>
> It seems that at some point, nodejs 20.x LTS upgraded npm
> to a version with a breaking change which now flattens the
> dependencies into a single level in /usr/lib/node_modules
> from the previously accustomed dependency tree in
> /usr/lib/node_modules/pm2/node_modules.
>
> Is there an incantation of EXTRA_OENPM or the npm:// SRC_URI
> that would allow either to unflatten the dependency tree, or
> include the dependencies as well as the main node module?
>
> Thanks in advance,
> Zoltán Böszörményi
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#210084): https://lists.openembedded.org/g/openembedded-core/message/210084
> Mute This Topic: https://lists.openembedded.org/mt/110731121/3617156
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [martin.jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] npm based packages omit dependencies since Yocto 5.1

[Böszörményi Zoltán]

That's it! ;destsuffix=npm helped with Yocto 5.1.
Thank you very much!

I will re-test with Yocto master and report back.

2025. 01. 21. 11:53 keltezéssel, Martin Jansa írta:
> I believe it's the side effect of UNPACKDIR changes, I had to add
> ;destsuffix=npm in all npmsw:// entries (or ;destsuffix=git where S is
> set to WORKDIR/git) for dependencies in node_modules to be unpacked
> where they used to be before.
>
> On Tue, Jan 21, 2025 at 11:31 AM Zoltan Boszormenyi via
> lists.openembedded.org <zboszor=gmail.com@lists.openembedded.org>
> wrote:
>> Hi,
>>
>> I have a minimalistic recipe for pm2 (https://www.npmjs.com/package/pm2):
>> ==================================
>> node-pm2_5.3.1.bb
>> ==================================
>> SUMMARY = "Production process manager for Node.JS applications with a built-in load balancer."
>> HOMEPAGE = "http://pm2.keymetrics.io/"
>> LICENSE = "AGPL-3.0-or-later"
>> LIC_FILES_CHKSUM = "file://GNU-AGPL-3.0.txt;md5=9f9fa01c099265d62f73735e404ff8bb"
>>
>> SRC_URI = " \
>>       npm://registry.npmjs.org/;package=pm2;version=${PV} \
>>       npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \
>>       "
>>
>> S = "${UNPACKDIR}/npm"
>>
>> inherit npm
>>
>> do_install:append () {
>>           rm -rf ${D}${libdir}/node_modules/pm2/lib/templates/init-scripts/openrc.tpl
>> }
>>
>> RDEPENDS:${PN} = "bash"
>> ==================================
>>
>> Previously this recipe has built and worked correctly under
>> Yocto 4.3 and 5.0, as in the package included its own internal
>> dependencies in /usr/lib/node_modules/pm2/node_modules.
>> Yocto 4.3 ships nodejs 20.8.1, Yocto 5.0 ships nodejs 20.12.2.
>>
>> With Yocto 5.1 (node 20.18.0) and master (22.12.0), the package
>> only has the pm2 module contents in /usr/lib/node_modules/pm2
>> but no dependencies.
>>
>> It seems that at some point, nodejs 20.x LTS upgraded npm
>> to a version with a breaking change which now flattens the
>> dependencies into a single level in /usr/lib/node_modules
>> from the previously accustomed dependency tree in
>> /usr/lib/node_modules/pm2/node_modules.
>>
>> Is there an incantation of EXTRA_OENPM or the npm:// SRC_URI
>> that would allow either to unflatten the dependency tree, or
>> include the dependencies as well as the main node module?
>>
>> Thanks in advance,
>> Zoltán Böszörményi
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#210084): https://lists.openembedded.org/g/openembedded-core/message/210084
>> Mute This Topic: https://lists.openembedded.org/mt/110731121/3617156
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [martin.jansa@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>

Re: [OE-core] npm based packages omit dependencies since Yocto 5.1

[Böszörményi Zoltán]

The recent changes in Yocto master with the unmodified recipe didn't help.
But npmsw://...;destsuffix=npm in the recipe did.

Thanks again!

2025. 01. 21. 12:27 keltezéssel, Zoltan Boszormenyi via lists.openembedded.org írta:
> That's it! ;destsuffix=npm helped with Yocto 5.1.
> Thank you very much!
>
> I will re-test with Yocto master and report back.
>
> 2025. 01. 21. 11:53 keltezéssel, Martin Jansa írta:
>> I believe it's the side effect of UNPACKDIR changes, I had to add
>> ;destsuffix=npm in all npmsw:// entries (or ;destsuffix=git where S is
>> set to WORKDIR/git) for dependencies in node_modules to be unpacked
>> where they used to be before.
>>
>> On Tue, Jan 21, 2025 at 11:31 AM Zoltan Boszormenyi via
>> lists.openembedded.org <zboszor=gmail.com@lists.openembedded.org>
>> wrote:
>>> Hi,
>>>
>>> I have a minimalistic recipe for pm2 (https://www.npmjs.com/package/pm2):
>>> ==================================
>>> node-pm2_5.3.1.bb
>>> ==================================
>>> SUMMARY = "Production process manager for Node.JS applications with a built-in load 
>>> balancer."
>>> HOMEPAGE = "http://pm2.keymetrics.io/"
>>> LICENSE = "AGPL-3.0-or-later"
>>> LIC_FILES_CHKSUM = "file://GNU-AGPL-3.0.txt;md5=9f9fa01c099265d62f73735e404ff8bb"
>>>
>>> SRC_URI = " \
>>>       npm://registry.npmjs.org/;package=pm2;version=${PV} \
>>>       npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \
>>>       "
>>>
>>> S = "${UNPACKDIR}/npm"
>>>
>>> inherit npm
>>>
>>> do_install:append () {
>>>           rm -rf ${D}${libdir}/node_modules/pm2/lib/templates/init-scripts/openrc.tpl
>>> }
>>>
>>> RDEPENDS:${PN} = "bash"
>>> ==================================
>>>
>>> Previously this recipe has built and worked correctly under
>>> Yocto 4.3 and 5.0, as in the package included its own internal
>>> dependencies in /usr/lib/node_modules/pm2/node_modules.
>>> Yocto 4.3 ships nodejs 20.8.1, Yocto 5.0 ships nodejs 20.12.2.
>>>
>>> With Yocto 5.1 (node 20.18.0) and master (22.12.0), the package
>>> only has the pm2 module contents in /usr/lib/node_modules/pm2
>>> but no dependencies.
>>>
>>> It seems that at some point, nodejs 20.x LTS upgraded npm
>>> to a version with a breaking change which now flattens the
>>> dependencies into a single level in /usr/lib/node_modules
>>> from the previously accustomed dependency tree in
>>> /usr/lib/node_modules/pm2/node_modules.
>>>
>>> Is there an incantation of EXTRA_OENPM or the npm:// SRC_URI
>>> that would allow either to unflatten the dependency tree, or
>>> include the dependencies as well as the main node module?
>>>
>>> Thanks in advance,
>>> Zoltán Böszörményi
>>>
>>>
>>>
>>>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#210090): https://lists.openembedded.org/g/openembedded-core/message/210090
> Mute This Topic: https://lists.openembedded.org/mt/110731121/3617728
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [zboszor@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] npm based packages omit dependencies since Yocto 5.1

[Martin Jansa]

Please try it with the changes from
https://lists.openembedded.org/g/bitbake-devel/message/16920 as well.
I believe the destsuffix=npm doesn't work with them anymore, it
doesn't for me, but unfortunately I haven't had the time yet to figure
out why and provide proper feedback on that RFC.

On Tue, Jan 21, 2025 at 5:26 PM Böszörményi Zoltán <zboszor@gmail.com> wrote:
>
> The recent changes in Yocto master with the unmodified recipe didn't help.
> But npmsw://...;destsuffix=npm in the recipe did.
>
> Thanks again!
>
> 2025. 01. 21. 12:27 keltezéssel, Zoltan Boszormenyi via lists.openembedded.org írta:
> > That's it! ;destsuffix=npm helped with Yocto 5.1.
> > Thank you very much!
> >
> > I will re-test with Yocto master and report back.
> >
> > 2025. 01. 21. 11:53 keltezéssel, Martin Jansa írta:
> >> I believe it's the side effect of UNPACKDIR changes, I had to add
> >> ;destsuffix=npm in all npmsw:// entries (or ;destsuffix=git where S is
> >> set to WORKDIR/git) for dependencies in node_modules to be unpacked
> >> where they used to be before.
> >>
> >> On Tue, Jan 21, 2025 at 11:31 AM Zoltan Boszormenyi via
> >> lists.openembedded.org <zboszor=gmail.com@lists.openembedded.org>
> >> wrote:
> >>> Hi,
> >>>
> >>> I have a minimalistic recipe for pm2 (https://www.npmjs.com/package/pm2):
> >>> ==================================
> >>> node-pm2_5.3.1.bb
> >>> ==================================
> >>> SUMMARY = "Production process manager for Node.JS applications with a built-in load
> >>> balancer."
> >>> HOMEPAGE = "http://pm2.keymetrics.io/"
> >>> LICENSE = "AGPL-3.0-or-later"
> >>> LIC_FILES_CHKSUM = "file://GNU-AGPL-3.0.txt;md5=9f9fa01c099265d62f73735e404ff8bb"
> >>>
> >>> SRC_URI = " \
> >>>       npm://registry.npmjs.org/;package=pm2;version=${PV} \
> >>>       npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \
> >>>       "
> >>>
> >>> S = "${UNPACKDIR}/npm"
> >>>
> >>> inherit npm
> >>>
> >>> do_install:append () {
> >>>           rm -rf ${D}${libdir}/node_modules/pm2/lib/templates/init-scripts/openrc.tpl
> >>> }
> >>>
> >>> RDEPENDS:${PN} = "bash"
> >>> ==================================
> >>>
> >>> Previously this recipe has built and worked correctly under
> >>> Yocto 4.3 and 5.0, as in the package included its own internal
> >>> dependencies in /usr/lib/node_modules/pm2/node_modules.
> >>> Yocto 4.3 ships nodejs 20.8.1, Yocto 5.0 ships nodejs 20.12.2.
> >>>
> >>> With Yocto 5.1 (node 20.18.0) and master (22.12.0), the package
> >>> only has the pm2 module contents in /usr/lib/node_modules/pm2
> >>> but no dependencies.
> >>>
> >>> It seems that at some point, nodejs 20.x LTS upgraded npm
> >>> to a version with a breaking change which now flattens the
> >>> dependencies into a single level in /usr/lib/node_modules
> >>> from the previously accustomed dependency tree in
> >>> /usr/lib/node_modules/pm2/node_modules.
> >>>
> >>> Is there an incantation of EXTRA_OENPM or the npm:// SRC_URI
> >>> that would allow either to unflatten the dependency tree, or
> >>> include the dependencies as well as the main node module?
> >>>
> >>> Thanks in advance,
> >>> Zoltán Böszörményi
> >>>
> >>>
> >>>
> >>>
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#210090): https://lists.openembedded.org/g/openembedded-core/message/210090
> > Mute This Topic: https://lists.openembedded.org/mt/110731121/3617728
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [zboszor@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>