Multiple images on the same hardware

Multiple images on the same hardware

[Aleksandar Nikolic]

[-- Attachment #1: Type: text/plain, Size: 1336 bytes --]

Hi,

I know similar questions have been posted before, but still I have a few things to clarify.

We are using the same hardware for multiple projects, which means some projects support some interfaces (e.g. bluetooth - Y, WiFi - N), while others some other interfaces (e.g. bluetooth - N, WiFi - Y). Of course I don't want support for bluetooth in case we don't need it.

AFAIK there are (at least) two ways to address this: [1] with multiple MACHINE configs and [2] with multiple IMAGE recipes.

F or MACHINE_FEATURES the Yocto documentation states the following: Specifies the list of hardware features the MACHINE ( https://docs.yoctoproject.org/4.0.15/ref-manual/variables.html?highlight=machine_features#term-MACHINE ) is capable of supporting. I am reading as: [1] there is only one MACHINE conf per target and [2] MACHINE_FEATURES does not change. This leaves us with the second option: we should have multiple image recipes and each recipe should define for itself what package groups it will include (e.g. bluetooth support, wifi support, etc.). However, this doesn't play nicely with e.g. kernel fragments. If the product does not need to support bluetooth, I also don't want the software stack compiled in my kernel, however, I don't have a variable to use in my kernel recipe. Any thoughts to this?

Aleksandar

[-- Attachment #2: Type: text/html, Size: 1803 bytes --]

Re: [yocto] Multiple images on the same hardware

[Alexander Kanavin]

Multiple machines has a heavy cost, as you effectively double the
amount of builds (and build times) just so you can enable or not some
features in the kernel on the exact same hardware. I would suggest to
keep the same machine and same distro, and just make multiple images
(maybe even in the same bitbake invocation) that include (or not)
needed kernel modules and userspace packages.

Alex

On Thu, 25 Jan 2024 at 10:38, Aleksandar Nikolic <an010@live.com> wrote:
>
> Hi,
>
> I know similar questions have been posted before, but still I have a few things to clarify.
>
> We are using the same hardware for multiple projects, which means some projects support some interfaces (e.g. bluetooth - Y, WiFi - N), while others some other interfaces (e.g. bluetooth - N, WiFi - Y). Of course I don't want support for bluetooth in case we don't need it.
>
> AFAIK there are (at least) two ways to address this: [1] with multiple MACHINE configs and [2] with multiple IMAGE recipes.
>
> For MACHINE_FEATURES the Yocto documentation states the following: Specifies the list of hardware features the MACHINE is capable of supporting. I am reading as: [1] there is only one MACHINE conf per target and [2] MACHINE_FEATURES does not change. This leaves us with the second option: we should have multiple image recipes and each recipe should define for itself what package groups it will include (e.g. bluetooth support, wifi support, etc.). However, this doesn't play nicely with e.g. kernel fragments. If the product does not need to support bluetooth, I also don't want the software stack compiled in my kernel, however, I don't have a variable to use in my kernel recipe. Any thoughts to this?
>
> Aleksandar
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#62283): https://lists.yoctoproject.org/g/yocto/message/62283
> Mute This Topic: https://lists.yoctoproject.org/mt/103951016/1686489
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [yocto] Multiple images on the same hardware

[Aleksandar Nikolic]

[-- Attachment #1: Type: text/plain, Size: 539 bytes --]

I thought that could be an issue with multiple machines, thanks for the confirmation.

How am I supposed to decide which kernel config fragments are included in which image though, since they go into SRC_URI? The same problem arises if the same userspace package needs to be built slightly differently for different products
That's not something I can easily control in the image recipe.

Multiple recipes for multiple products is one way to solve this, but I don't find that as an elegant way of solving this problem.

Aleksandar

[-- Attachment #2: Type: text/html, Size: 568 bytes --]

Re: [yocto] Multiple images on the same hardware

[Alexander Kanavin]

On the image level you can only control which packages go into it.
Kernel modules are all split into their own packages, so you can
install (or not) what you want.

Building the same userspace recipe slightly differently for different
images isn't possible. You need to accept that and find some other way
to differentiate between images (e.g. by installing (or not) the
package that the recipe produces to begin with, or by supplying
different runtime configurations).

The alternative is indeed making two (or more) slightly different
distros or slightly different machines, and running separate builds
for them, which greatly increases complexity.

Alex

On Thu, 25 Jan 2024 at 13:18, Aleksandar Nikolic <an010@live.com> wrote:
>
> I thought that could be an issue with multiple machines, thanks for the confirmation.
>
> How am I supposed to decide which kernel config fragments are included in which image though, since they go into SRC_URI? The same problem arises if the same userspace package needs to be built slightly differently for different products
> That's not something I can easily control in the image recipe.
>
> Multiple recipes for multiple products is one way to solve this, but I don't find that as an elegant way of solving this problem.
>
> Aleksandar
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#62290): https://lists.yoctoproject.org/g/yocto/message/62290
> Mute This Topic: https://lists.yoctoproject.org/mt/103951016/1686489
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [yocto] Multiple images on the same hardware

[Aleksandar Nikolic]

[-- Attachment #1: Type: text/plain, Size: 141 bytes --]

As I thought, thanks. I guess I'll try bbappend-ing the recipe in the respective product layers and see where that brings me.

Aleksandar

[-- Attachment #2: Type: text/html, Size: 149 bytes --]

Re: [yocto] Multiple images on the same hardware

[Mike Looijmans]

There's a lot more "gray" area than just that...

You can have two application recipes that access the same repository but 
produce different things. By including one or the other in your image recipe, 
you can control which configuration gets built.

Instead of a distro or machine for each combination, it's also possible to use 
a configuration file in the build directory. Your application recipe just says 
"require my-app-config.inc" and then you can provide the contents of that file 
when preparing the build. Makes things easy to produce but harder to reproduce...

The penalty for building things with unneeded features usually isn't that big, 
so it might be worth it if you have systems with and without screen to build 
them all with "x11" or "wayland" in their distro features. Trick is to adjust 
the image recipe so that the headless machine doesn't install all the graphic 
stuff (usually you'll have to take a look at how some packagegroups get 
included and change that).

Having said that, projects like OpenPLi build for more than 40 different 
MACHINEs. There's no real problem with that, just that at some point you'll 
want a beafy build machine...

Wifi and bluetooth can be compiled as kernel modules and loaded on demand. So 
those aren't a reason to have different MACHINEs. It's also quite feasible to 
load devicetrees and overlays based on things detected at runtime (e.g. have 
u-boot probe the I2C bus for a touch controller and load a different 
devicetree depending on the type of display panel, or have the kernel read 
some config data and load devicetree overlays based on that)




Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijmans@topic.nl
W: www.topic.nl

Please consider the environment before printing this e-mail
On 25-01-2024 15:54, Alexander Kanavin via lists.yoctoproject.org wrote:
> On the image level you can only control which packages go into it.
> Kernel modules are all split into their own packages, so you can
> install (or not) what you want.
> 
> Building the same userspace recipe slightly differently for different
> images isn't possible. You need to accept that and find some other way
> to differentiate between images (e.g. by installing (or not) the
> package that the recipe produces to begin with, or by supplying
> different runtime configurations).
> 
> The alternative is indeed making two (or more) slightly different
> distros or slightly different machines, and running separate builds
> for them, which greatly increases complexity.
> 
> Alex
> 
> On Thu, 25 Jan 2024 at 13:18, Aleksandar Nikolic <an010@live.com> wrote:
>>
>> I thought that could be an issue with multiple machines, thanks for the confirmation.
>>
>> How am I supposed to decide which kernel config fragments are included in which image though, since they go into SRC_URI? The same problem arises if the same userspace package needs to be built slightly differently for different products
>> That's not something I can easily control in the image recipe.
>>
>> Multiple recipes for multiple products is one way to solve this, but I don't find that as an elegant way of solving this problem.
>>
>> Aleksandar
>>
>>
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#62292): https://lists.yoctoproject.org/g/yocto/message/62292
>> Mute This Topic: https://lists.yoctoproject.org/mt/103951016/3618446
>> Group Owner: yocto+owner@lists.yoctoproject.org
>> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [mike.looijmans@topic.nl]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>

Re: [yocto] Multiple images on the same hardware

[Aleksandar Nikolic]

[-- Attachment #1: Type: text/plain, Size: 1384 bytes --]

> 
> Instead of a distro or machine for each combination, it's also possible to
> use a configuration file in the build directory. Your application recipe
> just says "require my-app-config.inc" and then you can provide the
> contents of that file when preparing the build. Makes things easy to
> produce but harder to reproduce...
> 

An interesting approach, didn't occur to me, thanks for the hint.

> 
> The penalty for building things with unneeded features usually isn't that
> big, so it might be worth it if you have systems with and without screen
> to build them all with "x11" or "wayland" in their distro features. Trick
> is to adjust the image recipe so that the headless machine doesn't install
> all the graphic stuff (usually you'll have to take a look at how some
> packagegroups get included and change that).
> 

In this particular case I can use DISTRO_FEATURES in my recipe and nicely use one image recipe to produce multiple images, this is however not possible with e.g. kernel modues, as such variable does not exist.

> 
> Wifi and bluetooth can be compiled as kernel modules and loaded on demand.

Yes, this would work as well. Though some would argue that out of security reasons it's better to reduce the attack surface and not even have kernel modules to be loaded in the first place if they are not needed at all.

Aleksandar

[-- Attachment #2: Type: text/html, Size: 1652 bytes --]

Re: [yocto] Multiple images on the same hardware

[Mike Looijmans]

Met vriendelijke groet / kind regards,

Mike Looijmans
System Expert


TOPIC Embedded Products B.V.
Materiaalweg 4, 5681 RJ Best
The Netherlands

T: +31 (0) 499 33 69 69
E: mike.looijmans@topic.nl
W: www.topic.nl

Please consider the environment before printing this e-mail
On 31-01-2024 23:07, Aleksandar Nikolic via Lists.Yoctoproject.Org wrote:
>     Instead of a distro or machine for each combination, it's also possible to
>     use a configuration file in the build directory. Your application recipe
>     just says "require my-app-config.inc" and then you can provide the
>     contents of that file when preparing the build. Makes things easy to
>     produce but harder to reproduce...
> 
> An interesting approach, didn't occur to me, thanks for the hint.
> 
>     The penalty for building things with unneeded features usually isn't that
>     big, so it might be worth it if you have systems with and without screen
>     to build them all with "x11" or "wayland" in their distro features. Trick
>     is to adjust the image recipe so that the headless machine doesn't install
>     all the graphic stuff (usually you'll have to take a look at how some
>     packagegroups get included and change that).
> 
> In this particular case I can use DISTRO_FEATURES in my recipe and nicely use 
> one image recipe to produce multiple images, this is however not possible with 
> e.g. kernel modues, as such variable does not exist.

DISTRO_FEATURES is rather inconvenient as it requires separate builds.

Why don't you just have multiple image recipes? I usually have something like

project-name-image (for released product)
project-name-image-dev (same, but with development options like a writable 
root and gdbserver)
project-name-production-test (...)

> 
>     Wifi and bluetooth can be compiled as kernel modules and loaded on demand.
> 
> Yes, this would work as well. Though some would argue that out of security 
> reasons it's better to reduce the attack surface and not even have kernel 
> modules to be loaded in the first place if they are not needed at all.

Safer even is to not have the modules on the filesystem at all, then the 
kernel cannot load them...

Normally the kernel only loads modules that it needs (systemd, udev, and even 
busybox' mdev feature on-demand module loading so you never have to forcibly 
"modprobe" anything)

Many images bluntly install "kernel-modules" which throws in everything that 
got built by the kernel recipe. Omit that and you can install kernel modules 
based on exact system requirements.

Re: [yocto] Multiple images on the same hardware

[Aleksandar Nikolic]

[-- Attachment #1: Type: text/plain, Size: 690 bytes --]

> 
> Why don't you just have multiple image recipes?
> 

The initial question was not related to image recipes, but rather to the one linux-kernel recipe. Within the recipe there's no variable I could use to specify which modules get compiled.

> 
> Many images bluntly install "kernel-modules" which throws in everything
> that got built by the kernel recipe. Omit that and you can install kernel
> modules based on exact system requirements.
> 

This is something I will take a look at, thanks for the idea. One "issue" I see with it is that this won't work for built-in modules, however, those "special" scenarios can be covered with modules loaded on demand.

Aleksandar

[-- Attachment #2: Type: text/html, Size: 846 bytes --]