[master][PATCH] kernel-devsrc: fix missing RDEPENDS gawk and make

[master][PATCH] kernel-devsrc: fix missing RDEPENDS gawk and make

[Claus Stovgaard]

gawk is also needed for x86-64. See scripts/check-sysctl-docs
Since version 6.6 the package rules has been split up into a rules file
resulting in the need for make. See scripts/package/debian/rules

For reproducing the issue on a normal qemux86-64 machine, change to the
linux-yocto-dev kernel, add the kernel-devsrc to the toolchain target
task and disable ptest in distro features. (Notice ptest pulls gawk and
make in as RDEPENDS for other packages hiding the issue)

when populating sdk you will see dnf failing.

 Problem: conflicting requests
  - nothing provides /usr/bin/gawk needed by kernel-devsrc-1.0-r0.qemux86_64 from oe-repo
  - nothing provides /usr/bin/make needed by kernel-devsrc-1.0-r0.qemux86_64 from oe-repo

Signed-off-by: Claus Stovgaard <claus.stovgaard@gmail.com>
---
 meta/recipes-kernel/linux/kernel-devsrc.bb | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb b/meta/recipes-kernel/linux/kernel-devsrc.bb
index ce5777fbe9..222f6cf8e1 100644
--- a/meta/recipes-kernel/linux/kernel-devsrc.bb
+++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
@@ -395,7 +395,9 @@ RDEPENDS:${PN} += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-dev', '', d)}"
 RDEPENDS:${PN} += "${@bb.utils.contains('ARCH', 'powerpc', 'elfutils-dev', '', d)}"
 # 5.8+ needs gcc-plugins libmpc-dev
 RDEPENDS:${PN} += "gcc-plugins libmpc-dev"
-# 5.13+ needs awk for arm64
-RDEPENDS:${PN}:append:aarch64 = " gawk"
+# 5.13+ needs awk
+RDEPENDS:${PN} += "gawk"
 # 5.13+ needs grep for powerpc
 RDEPENDS:${PN}:append:powerpc = " grep"
+# 6.6+ needs make for debian package rules
+RDEPENDS:${PN} += "make"
-- 
2.43.0

Re: [OE-core] [master][PATCH] kernel-devsrc: fix missing RDEPENDS gawk and make

[Bruce Ashfield]

On Thu, Feb 1, 2024 at 6:54 AM Claus Stovgaard
<claus.stovgaard@gmail.com> wrote:
>
> gawk is also needed for x86-64. See scripts/check-sysctl-docs
> Since version 6.6 the package rules has been split up into a rules file
> resulting in the need for make. See scripts/package/debian/rules

When I looked, the rules aren't actually used when we are building an out
of tree module (and via make scripts prepare) indirectly.

In a situation like this the approach I've taken to exclude the source from
devsrc to keep it minimal.

If you are seeing gawk required as part of building a module or make
scripts prepare, then we definitely do need to add the dependency.

>
> For reproducing the issue on a normal qemux86-64 machine, change to the
> linux-yocto-dev kernel, add the kernel-devsrc to the toolchain target
> task and disable ptest in distro features. (Notice ptest pulls gawk and
> make in as RDEPENDS for other packages hiding the issue)
>
> when populating sdk you will see dnf failing.
>
>  Problem: conflicting requests
>   - nothing provides /usr/bin/gawk needed by kernel-devsrc-1.0-r0.qemux86_64 from oe-repo
>   - nothing provides /usr/bin/make needed by kernel-devsrc-1.0-r0.qemux86_64 from oe-repo
>
> Signed-off-by: Claus Stovgaard <claus.stovgaard@gmail.com>
> ---
>  meta/recipes-kernel/linux/kernel-devsrc.bb | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb b/meta/recipes-kernel/linux/kernel-devsrc.bb
> index ce5777fbe9..222f6cf8e1 100644
> --- a/meta/recipes-kernel/linux/kernel-devsrc.bb
> +++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
> @@ -395,7 +395,9 @@ RDEPENDS:${PN} += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-dev', '', d)}"
>  RDEPENDS:${PN} += "${@bb.utils.contains('ARCH', 'powerpc', 'elfutils-dev', '', d)}"
>  # 5.8+ needs gcc-plugins libmpc-dev
>  RDEPENDS:${PN} += "gcc-plugins libmpc-dev"
> -# 5.13+ needs awk for arm64
> -RDEPENDS:${PN}:append:aarch64 = " gawk"
> +# 5.13+ needs awk
> +RDEPENDS:${PN} += "gawk"

The commit message should really say "all architectures require gawk", if this
is the change we are making to the actual RDEPENDS itself.

>  # 5.13+ needs grep for powerpc
>  RDEPENDS:${PN}:append:powerpc = " grep"
> +# 6.6+ needs make for debian package rules
> +RDEPENDS:${PN} += "make"

Similarly, we should just be removing the rules from devsrc if they
aren't required
and we can avoid this dependency.

Bruce

> --
> 2.43.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#194584): https://lists.openembedded.org/g/openembedded-core/message/194584
> Mute This Topic: https://lists.openembedded.org/mt/104095126/1050810
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [master][PATCH] kernel-devsrc: fix missing RDEPENDS gawk and make

[claus.stovgaard]

Thanks for your quick reply Bruce

On Thu, 2024-02-01 at 07:45 -0500, Bruce Ashfield wrote:
> On Thu, Feb 1, 2024 at 6:54 AM Claus Stovgaard
> <claus.stovgaard@gmail.com> wrote:
> > 
> > gawk is also needed for x86-64. See scripts/check-sysctl-docs
> > Since version 6.6 the package rules has been split up into a rules
> > file
> > resulting in the need for make. See scripts/package/debian/rules
> 
> When I looked, the rules aren't actually used when we are building an
> out
> of tree module (and via make scripts prepare) indirectly.
> 
> In a situation like this the approach I've taken to exclude the
> source from
> devsrc to keep it minimal.

Okay. I will patch the do_install instead and remove the debian rules
file. I don't see the usage either so it is fine with me.

> 
> If you are seeing gawk required as part of building a module or make
> scripts prepare, then we definitely do need to add the dependency.
> 

I can see we are including the scripts/check-sysctl-docs and objdump-
func needing gawk - no matter the architecture.

Alternative we should properly remove those also.

Will split it up in two commits, and start with the "make" fix

/Claus


> > 
> > For reproducing the issue on a normal qemux86-64 machine, change to
> > the
> > linux-yocto-dev kernel, add the kernel-devsrc to the toolchain
> > target
> > task and disable ptest in distro features. (Notice ptest pulls gawk
> > and
> > make in as RDEPENDS for other packages hiding the issue)
> > 
> > when populating sdk you will see dnf failing.
> > 
> >  Problem: conflicting requests
> >   - nothing provides /usr/bin/gawk needed by kernel-devsrc-1.0-
> > r0.qemux86_64 from oe-repo
> >   - nothing provides /usr/bin/make needed by kernel-devsrc-1.0-
> > r0.qemux86_64 from oe-repo
> > 
> > Signed-off-by: Claus Stovgaard <claus.stovgaard@gmail.com>
> > ---
> >  meta/recipes-kernel/linux/kernel-devsrc.bb | 6 ++++--
> >  1 file changed, 4 insertions(+), 2 deletions(-)
> > 
> > diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb
> > b/meta/recipes-kernel/linux/kernel-devsrc.bb
> > index ce5777fbe9..222f6cf8e1 100644
> > --- a/meta/recipes-kernel/linux/kernel-devsrc.bb
> > +++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
> > @@ -395,7 +395,9 @@ RDEPENDS:${PN} += "${@bb.utils.contains('ARCH',
> > 'x86', 'elfutils-dev', '', d)}"
> >  RDEPENDS:${PN} += "${@bb.utils.contains('ARCH', 'powerpc',
> > 'elfutils-dev', '', d)}"
> >  # 5.8+ needs gcc-plugins libmpc-dev
> >  RDEPENDS:${PN} += "gcc-plugins libmpc-dev"
> > -# 5.13+ needs awk for arm64
> > -RDEPENDS:${PN}:append:aarch64 = " gawk"
> > +# 5.13+ needs awk
> > +RDEPENDS:${PN} += "gawk"
> 
> The commit message should really say "all architectures require
> gawk", if this
> is the change we are making to the actual RDEPENDS itself.
> 
> >  # 5.13+ needs grep for powerpc
> >  RDEPENDS:${PN}:append:powerpc = " grep"
> > +# 6.6+ needs make for debian package rules
> > +RDEPENDS:${PN} += "make"
> 
> Similarly, we should just be removing the rules from devsrc if they
> aren't required
> and we can avoid this dependency.
> 
> Bruce
> 
> > --
> > 2.43.0
> > 
> > 
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#194584):
> > https://lists.openembedded.org/g/openembedded-core/message/194584
> > Mute This Topic:
> > https://lists.openembedded.org/mt/104095126/1050810
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe:
> > https://lists.openembedded.org/g/openembedded-core/unsub [
> > bruce.ashfield@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> > 
> 
> 
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [master][PATCH] kernel-devsrc: fix missing RDEPENDS gawk and make

[Bruce Ashfield]

On Thu, Feb 1, 2024 at 8:31 AM <claus.stovgaard@gmail.com> wrote:
>
>
> Thanks for your quick reply Bruce
>
> On Thu, 2024-02-01 at 07:45 -0500, Bruce Ashfield wrote:
> > On Thu, Feb 1, 2024 at 6:54 AM Claus Stovgaard
> > <claus.stovgaard@gmail.com> wrote:
> > >
> > > gawk is also needed for x86-64. See scripts/check-sysctl-docs
> > > Since version 6.6 the package rules has been split up into a rules
> > > file
> > > resulting in the need for make. See scripts/package/debian/rules
> >
> > When I looked, the rules aren't actually used when we are building an
> > out
> > of tree module (and via make scripts prepare) indirectly.
> >
> > In a situation like this the approach I've taken to exclude the
> > source from
> > devsrc to keep it minimal.
>
> Okay. I will patch the do_install instead and remove the debian rules
> file. I don't see the usage either so it is fine with me.

Just remove it in kernel-devsrc's copy of things. We can let the
kernel continue to install them, we just don't need them in kernel-devsrc
itself.

>
> >
> > If you are seeing gawk required as part of building a module or make
> > scripts prepare, then we definitely do need to add the dependency.
> >
>
> I can see we are including the scripts/check-sysctl-docs and objdump-
> func needing gawk - no matter the architecture.
>
> Alternative we should properly remove those also.

We definitely can, if they are bringing in something we don't need to
prepare the source tree for building modules.

The ones I've added over time have all been related to that.

Bruce

>
> Will split it up in two commits, and start with the "make" fix
>
> /Claus
>
>
> > >
> > > For reproducing the issue on a normal qemux86-64 machine, change to
> > > the
> > > linux-yocto-dev kernel, add the kernel-devsrc to the toolchain
> > > target
> > > task and disable ptest in distro features. (Notice ptest pulls gawk
> > > and
> > > make in as RDEPENDS for other packages hiding the issue)
> > >
> > > when populating sdk you will see dnf failing.
> > >
> > >  Problem: conflicting requests
> > >   - nothing provides /usr/bin/gawk needed by kernel-devsrc-1.0-
> > > r0.qemux86_64 from oe-repo
> > >   - nothing provides /usr/bin/make needed by kernel-devsrc-1.0-
> > > r0.qemux86_64 from oe-repo
> > >
> > > Signed-off-by: Claus Stovgaard <claus.stovgaard@gmail.com>
> > > ---
> > >  meta/recipes-kernel/linux/kernel-devsrc.bb | 6 ++++--
> > >  1 file changed, 4 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb
> > > b/meta/recipes-kernel/linux/kernel-devsrc.bb
> > > index ce5777fbe9..222f6cf8e1 100644
> > > --- a/meta/recipes-kernel/linux/kernel-devsrc.bb
> > > +++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
> > > @@ -395,7 +395,9 @@ RDEPENDS:${PN} += "${@bb.utils.contains('ARCH',
> > > 'x86', 'elfutils-dev', '', d)}"
> > >  RDEPENDS:${PN} += "${@bb.utils.contains('ARCH', 'powerpc',
> > > 'elfutils-dev', '', d)}"
> > >  # 5.8+ needs gcc-plugins libmpc-dev
> > >  RDEPENDS:${PN} += "gcc-plugins libmpc-dev"
> > > -# 5.13+ needs awk for arm64
> > > -RDEPENDS:${PN}:append:aarch64 = " gawk"
> > > +# 5.13+ needs awk
> > > +RDEPENDS:${PN} += "gawk"
> >
> > The commit message should really say "all architectures require
> > gawk", if this
> > is the change we are making to the actual RDEPENDS itself.
> >
> > >  # 5.13+ needs grep for powerpc
> > >  RDEPENDS:${PN}:append:powerpc = " grep"
> > > +# 6.6+ needs make for debian package rules
> > > +RDEPENDS:${PN} += "make"
> >
> > Similarly, we should just be removing the rules from devsrc if they
> > aren't required
> > and we can avoid this dependency.
> >
> > Bruce
> >
> > > --
> > > 2.43.0
> > >
> > >
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > > Links: You receive all messages sent to this group.
> > > View/Reply Online (#194584):
> > > https://lists.openembedded.org/g/openembedded-core/message/194584
> > > Mute This Topic:
> > > https://lists.openembedded.org/mt/104095126/1050810
> > > Group Owner: openembedded-core+owner@lists.openembedded.org
> > > Unsubscribe:
> > > https://lists.openembedded.org/g/openembedded-core/unsub [
> > > bruce.ashfield@gmail.com]
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > >
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [master][PATCH] kernel-devsrc: fix missing RDEPENDS gawk and make

[claus.stovgaard]

On Thu, 2024-02-01 at 08:41 -0500, Bruce Ashfield wrote:
> On Thu, Feb 1, 2024 at 8:31 AM <claus.stovgaard@gmail.com> wrote:
> > 
> > 
> > Thanks for your quick reply Bruce
> > 
> > On Thu, 2024-02-01 at 07:45 -0500, Bruce Ashfield wrote:
> > > On Thu, Feb 1, 2024 at 6:54 AM Claus Stovgaard
> > > <claus.stovgaard@gmail.com> wrote:
> > > > 
> > > > gawk is also needed for x86-64. See scripts/check-sysctl-docs
> > > > Since version 6.6 the package rules has been split up into a
> > > > rules
> > > > file
> > > > resulting in the need for make. See
> > > > scripts/package/debian/rules
> > > 
> > > When I looked, the rules aren't actually used when we are
> > > building an
> > > out
> > > of tree module (and via make scripts prepare) indirectly.
> > > 
> > > In a situation like this the approach I've taken to exclude the
> > > source from
> > > devsrc to keep it minimal.
> > 
> > Okay. I will patch the do_install instead and remove the debian
> > rules
> > file. I don't see the usage either so it is fine with me.
> 
> Just remove it in kernel-devsrc's copy of things. We can let the
> kernel continue to install them, we just don't need them in kernel-
> devsrc
> itself.
> 

I have sent a new patch to rm the rules file - removing the need for
make, as you suggested.

> > 
> > > 
> > > If you are seeing gawk required as part of building a module or
> > > make
> > > scripts prepare, then we definitely do need to add the
> > > dependency.
> > > 
> > 
> > I can see we are including the scripts/check-sysctl-docs and
> > objdump-
> > func needing gawk - no matter the architecture.
> > 
> > Alternative we should properly remove those also.
> 
> We definitely can, if they are bringing in something we don't need to
> prepare the source tree for building modules.
> 
> The ones I've added over time have all been related to that.
> 
> Bruce
> 

I looked a bit more into it, and I could remove the check-sysctl-docs
script, but it will still end up needing an awk.
(Also check-sysctl-docs has been there since 5.10)

A quick look revealed the folloing files.

usr/src/kernel/arch/x86/tools/gen-insn-attr-x86.awk:#!/usr/bin/awk -f
usr/src/kernel/scripts/ver_linux:#!/usr/bin/awk -f
usr/src/kernel/tools/arch/x86/tools/gen-insn-attr-86.awk:#!/usr/bin/awk
-f

My guess is that for all normal SDK's a awk/gawk has always been
present, hiding the issue. So I would suggest that we just change the
RDEPEND to cover all architectures, and not depend on some other
package include gawk.

(I only noticed the gawk issue, because I had a SDK with had the "make"
issue, and created the most basic setup to reproduce it with poky only.
For all normal SDK's I would expect it to include gawk)

What do you think Bruce?

/Claus



> > 
> > Will split it up in two commits, and start with the "make" fix
> > 
> > /Claus
> > 
> > 
> > > 

Re: [OE-core] [master][PATCH] kernel-devsrc: fix missing RDEPENDS gawk and make

[Bruce Ashfield]

On Thu, Feb 1, 2024 at 9:37 AM <claus.stovgaard@gmail.com> wrote:
>
> On Thu, 2024-02-01 at 08:41 -0500, Bruce Ashfield wrote:
> > On Thu, Feb 1, 2024 at 8:31 AM <claus.stovgaard@gmail.com> wrote:
> > >
> > >
> > > Thanks for your quick reply Bruce
> > >
> > > On Thu, 2024-02-01 at 07:45 -0500, Bruce Ashfield wrote:
> > > > On Thu, Feb 1, 2024 at 6:54 AM Claus Stovgaard
> > > > <claus.stovgaard@gmail.com> wrote:
> > > > >
> > > > > gawk is also needed for x86-64. See scripts/check-sysctl-docs
> > > > > Since version 6.6 the package rules has been split up into a
> > > > > rules
> > > > > file
> > > > > resulting in the need for make. See
> > > > > scripts/package/debian/rules
> > > >
> > > > When I looked, the rules aren't actually used when we are
> > > > building an
> > > > out
> > > > of tree module (and via make scripts prepare) indirectly.
> > > >
> > > > In a situation like this the approach I've taken to exclude the
> > > > source from
> > > > devsrc to keep it minimal.
> > >
> > > Okay. I will patch the do_install instead and remove the debian
> > > rules
> > > file. I don't see the usage either so it is fine with me.
> >
> > Just remove it in kernel-devsrc's copy of things. We can let the
> > kernel continue to install them, we just don't need them in kernel-
> > devsrc
> > itself.
> >
>
> I have sent a new patch to rm the rules file - removing the need for
> make, as you suggested.
>
> > >
> > > >
> > > > If you are seeing gawk required as part of building a module or
> > > > make
> > > > scripts prepare, then we definitely do need to add the
> > > > dependency.
> > > >
> > >
> > > I can see we are including the scripts/check-sysctl-docs and
> > > objdump-
> > > func needing gawk - no matter the architecture.
> > >
> > > Alternative we should properly remove those also.
> >
> > We definitely can, if they are bringing in something we don't need to
> > prepare the source tree for building modules.
> >
> > The ones I've added over time have all been related to that.
> >
> > Bruce
> >
>
> I looked a bit more into it, and I could remove the check-sysctl-docs
> script, but it will still end up needing an awk.
> (Also check-sysctl-docs has been there since 5.10)
>
> A quick look revealed the folloing files.
>
> usr/src/kernel/arch/x86/tools/gen-insn-attr-x86.awk:#!/usr/bin/awk -f
> usr/src/kernel/scripts/ver_linux:#!/usr/bin/awk -f
> usr/src/kernel/tools/arch/x86/tools/gen-insn-attr-86.awk:#!/usr/bin/awk
> -f
>
> My guess is that for all normal SDK's a awk/gawk has always been
> present, hiding the issue. So I would suggest that we just change the
> RDEPEND to cover all architectures, and not depend on some other
> package include gawk.

Definitely. We don't want any dependencies being fulfilled transitively.

In particular since gawk has already been required by arm64 for a while,
it isn't a completely unreasonable thing to add.

>
> (I only noticed the gawk issue, because I had a SDK with had the "make"
> issue, and created the most basic setup to reproduce it with poky only.
> For all normal SDK's I would expect it to include gawk)
>
> What do you think Bruce?

Seems reasonable to me.

Bruce

>
> /Claus
>
>
>
> > >
> > > Will split it up in two commits, and start with the "make" fix
> > >
> > > /Claus
> > >
> > >
> > > >
>

--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [master][PATCH] kernel-devsrc: fix missing RDEPENDS gawk and make

[claus.stovgaard]

On Thu, 2024-02-01 at 09:52 -0500, Bruce Ashfield wrote:
> On Thu, Feb 1, 2024 at 9:37 AM <claus.stovgaard@gmail.com> wrote:
> > 
> > On Thu, 2024-02-01 at 08:41 -0500, Bruce Ashfield wrote:
> > > On Thu, Feb 1, 2024 at 8:31 AM <claus.stovgaard@gmail.com> wrote:
> > > 
> > 
> > I looked a bit more into it, and I could remove the check-sysctl-
> > docs
> > script, but it will still end up needing an awk.
> > (Also check-sysctl-docs has been there since 5.10)
> > 
> > A quick look revealed the folloing files.
> > 
> > usr/src/kernel/arch/x86/tools/gen-insn-attr-x86.awk:#!/usr/bin/awk
> > -f
> > usr/src/kernel/scripts/ver_linux:#!/usr/bin/awk -f
> > usr/src/kernel/tools/arch/x86/tools/gen-insn-attr-
> > 86.awk:#!/usr/bin/awk
> > -f
> > 
> > My guess is that for all normal SDK's a awk/gawk has always been
> > present, hiding the issue. So I would suggest that we just change
> > the
> > RDEPEND to cover all architectures, and not depend on some other
> > package include gawk.
> 
> Definitely. We don't want any dependencies being fulfilled
> transitively.
> 
> In particular since gawk has already been required by arm64 for a
> while,
> it isn't a completely unreasonable thing to add.
> 
> > 
> > (I only noticed the gawk issue, because I had a SDK with had the
> > "make"
> > issue, and created the most basic setup to reproduce it with poky
> > only.
> > For all normal SDK's I would expect it to include gawk)
> > 
> > What do you think Bruce?
> 
> Seems reasonable to me.
> 
> Bruce
> > 

Sounds good. I have made a patch for this

Regards
Claus

> >