[PATCH] linux-yocto: add entries for machine features

[PATCH] linux-yocto: add entries for machine features

[Jon Mason]

Most machine features have corresponding kernel config fragments, but
only a few have an entry in the kernel recipe to be automatically added.
Adding the majority of them here.

Signed-off-by: Jon Mason <jdmason@kudzu.us>
---
 meta/recipes-kernel/linux/linux-yocto.inc | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/recipes-kernel/linux/linux-yocto.inc b/meta/recipes-kernel/linux/linux-yocto.inc
index 7ea661e138d9..23ae0b4260c2 100644
--- a/meta/recipes-kernel/linux/linux-yocto.inc
+++ b/meta/recipes-kernel/linux/linux-yocto.inc
@@ -33,7 +33,12 @@ KERNEL_LD:append:arc = " ${TOOLCHAIN_OPTIONS}"
 
 KERNEL_FEATURES:append:qemuall=" features/debug/printk.scc"
 
+KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'bluetooth', 'features/bluetooth/bluetooth.scc', '', d)}"
+KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'efi', 'cfg/efi.scc', '', d)}"
+KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'ext2', 'cfg/fs/ext2.scc', '', d)}"
 KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'numa', 'features/numa/numa.scc', '', d)}"
+KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'pci', 'features/pci/pci.scc', '', d)}"
+KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'rtc', 'cfg/timer/rtc.scc', '', d)}"
 KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'vfat', 'cfg/fs/vfat.scc', '', d)}"
 
 # A KMACHINE is the mapping of a yocto $MACHINE to what is built
-- 
2.30.2

Re: [OE-core] [PATCH] linux-yocto: add entries for machine features

[Bruce Ashfield]

On Thu, Sep 29, 2022 at 11:23 PM Jon Mason <jdmason@kudzu.us> wrote:
>
> Most machine features have corresponding kernel config fragments, but
> only a few have an entry in the kernel recipe to be automatically added.
> Adding the majority of them here.
>
> Signed-off-by: Jon Mason <jdmason@kudzu.us>
> ---
>  meta/recipes-kernel/linux/linux-yocto.inc | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/meta/recipes-kernel/linux/linux-yocto.inc b/meta/recipes-kernel/linux/linux-yocto.inc
> index 7ea661e138d9..23ae0b4260c2 100644
> --- a/meta/recipes-kernel/linux/linux-yocto.inc
> +++ b/meta/recipes-kernel/linux/linux-yocto.inc
> @@ -33,7 +33,12 @@ KERNEL_LD:append:arc = " ${TOOLCHAIN_OPTIONS}"
>
>  KERNEL_FEATURES:append:qemuall=" features/debug/printk.scc"
>
> +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'bluetooth', 'features/bluetooth/bluetooth.scc', '', d)}"
> +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'efi', 'cfg/efi.scc', '', d)}"
> +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'ext2', 'cfg/fs/ext2.scc', '', d)}"
>  KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'numa', 'features/numa/numa.scc', '', d)}"
> +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'pci', 'features/pci/pci.scc', '', d)}"
> +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'rtc', 'cfg/timer/rtc.scc', '', d)}"
>  KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'vfat', 'cfg/fs/vfat.scc', '', d)}"

The concept of this is ok, but there are some non-obvious details that
need to be considered.

There's a really old enhancement in bugzilla about doing a better job
of integration between machine, kernel and distro features:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=2267

One issue is that this should generate warnings if the config values
don't make it into the final .config. otherwise they aren't really a
useful mapping of machine -> kernel features (we don't want
"just-in-case" configuration). Obviously the linux-yocto config audit
can warn on things like this, but the fragments need to be tagged as
required (hardware) for that to happen.  Which they won't currently be
if specified this way. I'd need to do some tooling work to make that
happen.

Another detail is that KERNEL_FEATURES are applied last. Which means
they can override / enable things that a BSP configuration doesn't
want and could have even explicitly disabled, used different values,
etc.

A proper BSP configuration is already a definition with a curated set
of options, and in a linux-yocto flow, that is typically the first and
last word on the values. doing a lot of BSP configuration via machine
features spreads the logic out to multiple places, which can be a
challenge to track down later. The fact that these map to the
centralized fragments is really good (since while the enablement might
be distributed and spread over recipe space, the actual options are
centralized), but it makes jumping off from a kernel-cache BSP or one
of the qemu references harder, as these will keep showing up .. when
you might not want them.

Yes, we can argue that machine's shouldn't set features that they
don't want to support, but that doesn't cover all of the complications
and potential issues.

If we want to start dabbling with this, these should be a little bit
more opt-in, and be specific to the qemu machines we define (versus
just appending to KERNEL_FEATURES). As for opt-in, I'd suggest
something like an image or machine feature in the spirit of
"DYNAMIC_CONFIG" or "ENABLE_MACHINE_FEATURES", and when that is set,
we do the mapping.

Bruce


>
>  # A KMACHINE is the mapping of a yocto $MACHINE to what is built
> --
> 2.30.2
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#171203): https://lists.openembedded.org/g/openembedded-core/message/171203
> Mute This Topic: https://lists.openembedded.org/mt/94009766/1050810
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [PATCH] linux-yocto: add entries for machine features

[Jon Mason]

On Fri, Sep 30, 2022 at 09:29:24AM -0400, Bruce Ashfield wrote:
> On Thu, Sep 29, 2022 at 11:23 PM Jon Mason <jdmason@kudzu.us> wrote:
> >
> > Most machine features have corresponding kernel config fragments, but
> > only a few have an entry in the kernel recipe to be automatically added.
> > Adding the majority of them here.
> >
> > Signed-off-by: Jon Mason <jdmason@kudzu.us>
> > ---
> >  meta/recipes-kernel/linux/linux-yocto.inc | 5 +++++
> >  1 file changed, 5 insertions(+)
> >
> > diff --git a/meta/recipes-kernel/linux/linux-yocto.inc b/meta/recipes-kernel/linux/linux-yocto.inc
> > index 7ea661e138d9..23ae0b4260c2 100644
> > --- a/meta/recipes-kernel/linux/linux-yocto.inc
> > +++ b/meta/recipes-kernel/linux/linux-yocto.inc
> > @@ -33,7 +33,12 @@ KERNEL_LD:append:arc = " ${TOOLCHAIN_OPTIONS}"
> >
> >  KERNEL_FEATURES:append:qemuall=" features/debug/printk.scc"
> >
> > +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'bluetooth', 'features/bluetooth/bluetooth.scc', '', d)}"
> > +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'efi', 'cfg/efi.scc', '', d)}"
> > +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'ext2', 'cfg/fs/ext2.scc', '', d)}"
> >  KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'numa', 'features/numa/numa.scc', '', d)}"
> > +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'pci', 'features/pci/pci.scc', '', d)}"
> > +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'rtc', 'cfg/timer/rtc.scc', '', d)}"
> >  KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'vfat', 'cfg/fs/vfat.scc', '', d)}"
> 
> The concept of this is ok, but there are some non-obvious details that
> need to be considered.
> 
> There's a really old enhancement in bugzilla about doing a better job
> of integration between machine, kernel and distro features:
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=2267
> 
> One issue is that this should generate warnings if the config values
> don't make it into the final .config. otherwise they aren't really a
> useful mapping of machine -> kernel features (we don't want
> "just-in-case" configuration). Obviously the linux-yocto config audit
> can warn on things like this, but the fragments need to be tagged as
> required (hardware) for that to happen.  Which they won't currently be
> if specified this way. I'd need to do some tooling work to make that
> happen.
> 
> Another detail is that KERNEL_FEATURES are applied last. Which means
> they can override / enable things that a BSP configuration doesn't
> want and could have even explicitly disabled, used different values,
> etc.
> 
> A proper BSP configuration is already a definition with a curated set
> of options, and in a linux-yocto flow, that is typically the first and
> last word on the values. doing a lot of BSP configuration via machine
> features spreads the logic out to multiple places, which can be a
> challenge to track down later. The fact that these map to the
> centralized fragments is really good (since while the enablement might
> be distributed and spread over recipe space, the actual options are
> centralized), but it makes jumping off from a kernel-cache BSP or one
> of the qemu references harder, as these will keep showing up .. when
> you might not want them.
> 
> Yes, we can argue that machine's shouldn't set features that they
> don't want to support, but that doesn't cover all of the complications
> and potential issues.
> 
> If we want to start dabbling with this, these should be a little bit
> more opt-in, and be specific to the qemu machines we define (versus
> just appending to KERNEL_FEATURES). As for opt-in, I'd suggest
> something like an image or machine feature in the spirit of
> "DYNAMIC_CONFIG" or "ENABLE_MACHINE_FEATURES", and when that is set,
> we do the mapping.
> 
> Bruce

Perhaps I got a little overzealous. :)

My issue really is EFI.  EFI has some kernel features needed to boot
and is fairly compartmentalized.  Adding the kernel fragement in the
edk2 recipe doesn't really make sense.  Also, there a some machines
that can do u-boot and edk2, and adding it by default for both might not
be what we want.  Adding this makes it easier to switch between the two
in CI.

Let me know if that is acceptable and I'll hack the patch down to just
efi.

Thanks,
Jon

> 
> 
> >
> >  # A KMACHINE is the mapping of a yocto $MACHINE to what is built
> > --
> > 2.30.2
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#171203): https://lists.openembedded.org/g/openembedded-core/message/171203
> > Mute This Topic: https://lists.openembedded.org/mt/94009766/1050810
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
> 
> 
> -- 
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
> 

Re: [OE-core] [PATCH] linux-yocto: add entries for machine features

[Bruce Ashfield]

On Fri, Sep 30, 2022 at 3:15 PM Jon Mason <jdmason@kudzu.us> wrote:
>
> On Fri, Sep 30, 2022 at 09:29:24AM -0400, Bruce Ashfield wrote:
> > On Thu, Sep 29, 2022 at 11:23 PM Jon Mason <jdmason@kudzu.us> wrote:
> > >
> > > Most machine features have corresponding kernel config fragments, but
> > > only a few have an entry in the kernel recipe to be automatically added.
> > > Adding the majority of them here.
> > >
> > > Signed-off-by: Jon Mason <jdmason@kudzu.us>
> > > ---
> > >  meta/recipes-kernel/linux/linux-yocto.inc | 5 +++++
> > >  1 file changed, 5 insertions(+)
> > >
> > > diff --git a/meta/recipes-kernel/linux/linux-yocto.inc b/meta/recipes-kernel/linux/linux-yocto.inc
> > > index 7ea661e138d9..23ae0b4260c2 100644
> > > --- a/meta/recipes-kernel/linux/linux-yocto.inc
> > > +++ b/meta/recipes-kernel/linux/linux-yocto.inc
> > > @@ -33,7 +33,12 @@ KERNEL_LD:append:arc = " ${TOOLCHAIN_OPTIONS}"
> > >
> > >  KERNEL_FEATURES:append:qemuall=" features/debug/printk.scc"
> > >
> > > +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'bluetooth', 'features/bluetooth/bluetooth.scc', '', d)}"
> > > +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'efi', 'cfg/efi.scc', '', d)}"
> > > +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'ext2', 'cfg/fs/ext2.scc', '', d)}"
> > >  KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'numa', 'features/numa/numa.scc', '', d)}"
> > > +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'pci', 'features/pci/pci.scc', '', d)}"
> > > +KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'rtc', 'cfg/timer/rtc.scc', '', d)}"
> > >  KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'vfat', 'cfg/fs/vfat.scc', '', d)}"
> >
> > The concept of this is ok, but there are some non-obvious details that
> > need to be considered.
> >
> > There's a really old enhancement in bugzilla about doing a better job
> > of integration between machine, kernel and distro features:
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=2267
> >
> > One issue is that this should generate warnings if the config values
> > don't make it into the final .config. otherwise they aren't really a
> > useful mapping of machine -> kernel features (we don't want
> > "just-in-case" configuration). Obviously the linux-yocto config audit
> > can warn on things like this, but the fragments need to be tagged as
> > required (hardware) for that to happen.  Which they won't currently be
> > if specified this way. I'd need to do some tooling work to make that
> > happen.
> >
> > Another detail is that KERNEL_FEATURES are applied last. Which means
> > they can override / enable things that a BSP configuration doesn't
> > want and could have even explicitly disabled, used different values,
> > etc.
> >
> > A proper BSP configuration is already a definition with a curated set
> > of options, and in a linux-yocto flow, that is typically the first and
> > last word on the values. doing a lot of BSP configuration via machine
> > features spreads the logic out to multiple places, which can be a
> > challenge to track down later. The fact that these map to the
> > centralized fragments is really good (since while the enablement might
> > be distributed and spread over recipe space, the actual options are
> > centralized), but it makes jumping off from a kernel-cache BSP or one
> > of the qemu references harder, as these will keep showing up .. when
> > you might not want them.
> >
> > Yes, we can argue that machine's shouldn't set features that they
> > don't want to support, but that doesn't cover all of the complications
> > and potential issues.
> >
> > If we want to start dabbling with this, these should be a little bit
> > more opt-in, and be specific to the qemu machines we define (versus
> > just appending to KERNEL_FEATURES). As for opt-in, I'd suggest
> > something like an image or machine feature in the spirit of
> > "DYNAMIC_CONFIG" or "ENABLE_MACHINE_FEATURES", and when that is set,
> > we do the mapping.
> >
> > Bruce
>
> Perhaps I got a little overzealous. :)
>
> My issue really is EFI.  EFI has some kernel features needed to boot
> and is fairly compartmentalized.  Adding the kernel fragement in the
> edk2 recipe doesn't really make sense.  Also, there a some machines
> that can do u-boot and edk2, and adding it by default for both might not
> be what we want.  Adding this makes it easier to switch between the two
> in CI.
>
> Let me know if that is acceptable and I'll hack the patch down to just
> efi.

That makes sense to me. Let's start with that, and see how the pattern expands!

Bruce

>
> Thanks,
> Jon
>
> >
> >
> > >
> > >  # A KMACHINE is the mapping of a yocto $MACHINE to what is built
> > > --
> > > 2.30.2
> > >
> > >
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > > Links: You receive all messages sent to this group.
> > > View/Reply Online (#171203): https://lists.openembedded.org/g/openembedded-core/message/171203
> > > Mute This Topic: https://lists.openembedded.org/mt/94009766/1050810
> > > Group Owner: openembedded-core+owner@lists.openembedded.org
> > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > >
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II
> >



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II