Yocto Binary Distro

Yocto Binary Distro

[Aleksandar Nikolic]

[-- Attachment #1: Type: text/plain, Size: 4192 bytes --]

Greetings!

After watching a talk from Michael Opdenacker from this year's OSS Europe about Yocto Binary Distros (see [1]), I've been experimenting with installing pre-built packages directly onto the target and I have a few questions regarding this topic, maybe someone more experienced could chime in:

1) When I build a recipe previously not installed in my image and want to install it during runtime, the dnf install command fails, saying that the package's dependencies are not met (see [2] for more info). This is fine, as I can see that the tmp/deploy/rpm directory does not contain all of the package's dependencies nor the dependencies' WORKDIRs contain the deploy-rpms directory. Is this behavior correct? Shouldn't building a recipe also lead that all of its dependencies are not just built but also packaged and put into the tpm/deploy/rpm directory (the same as with building the image containing the package in question)?

2) Is there an official, stable, and not some hacky way to install already pre-built packages during the build and not later during runtime? So instead of building from source or from the sstate-cache, to provide a URL to a webserver holding packages, which would then be pulled and installed (if I am not wrong Isar does this)?

3) Is enabling and using the package runtime management recommended and encouraged when it comes to Yocto running on actual devices and not during a development phase? IMHO we are talking about embedded devices here, with little or no user interaction, with secure boot enabled, atomic updates, etc. and having a package management system such as dnf or apt is cool, but is also kind of brittle (a simple power off during apt update can break the system).

Thanks.
Aleksandar

[1] Building and Maintaining Binary Distributions with Yocto - Michael Opdenacker, Root Commit - YouTube ( https://www.youtube.com/watch?v=UdJZ0xz-QLk&list=PLbzoR-pLrL6rC7SpO7MJCZm22Qp5ns3p-&index=66 )

[2]
# building the image
bitbake core-image-minimal
bitbake package-index

# building the recipe which is not in the image
bitbake vim
bitbake package-index

# trying to install the recipe in core-image-minimal fails
root@qemux86-64:~# dnf install vim
OE Remote Repo: rpm core2_64                                                                                                                                                                                                                 970 kB/s | 476 kB     00:00
OE Remote Repo: rpm qemux86_64                                                                                                                                                                                                               238 kB/s |  62 kB     00:00
Error:
Problem: conflicting requests
- nothing provides libXt.so.6()(64bit) needed by vim-9.1.0698-r0.core2_64 from oe-remote-repo-rpm-core2_64
- nothing provides libgdk-3.so.0()(64bit) needed by vim-9.1.0698-r0.core2_64 from oe-remote-repo-rpm-core2_64
- nothing provides libgdk_pixbuf-2.0.so.0()(64bit) needed by vim-9.1.0698-r0.core2_64 from oe-remote-repo-rpm-core2_64
- nothing provides libgtk-3.so.0()(64bit) needed by vim-9.1.0698-r0.core2_64 from oe-remote-repo-rpm-core2_64
- nothing provides libpango-1.0.so.0()(64bit) needed by vim-9.1.0698-r0.core2_64 from oe-remote-repo-rpm-core2_64
- nothing provides libpangocairo-1.0.so.0()(64bit) needed by vim-9.1.0698-r0.core2_64 from oe-remote-repo-rpm-core2_64
- nothing provides libgdk-pixbuf-2.0-0 >= 2.42.12 needed by vim-9.1.0698-r0.core2_64 from oe-remote-repo-rpm-core2_64
- nothing provides libgtk-3.0 >= 3.24.43 needed by vim-9.1.0698-r0.core2_64 from oe-remote-repo-rpm-core2_64
- nothing provides libxt6 >= 1.3.0 needed by vim-9.1.0698-r0.core2_64 from oe-remote-repo-rpm-core2_64
- nothing provides pango >= 1.54.0 needed by vim-9.1.0698-r0.core2_64 from oe-remote-repo-rpm-core2_64
(try to add '--skip-broken' to skip uninstallable packages)

[-- Attachment #2: Type: text/html, Size: 5564 bytes --]

Re: [yocto] Yocto Binary Distro

[Alexander Kanavin]

On Mon, 7 Oct 2024 at 07:06, Aleksandar Nikolic via
lists.yoctoproject.org
<aleksandar.nikolic010=gmail.com@lists.yoctoproject.org> wrote:
> 1) When I build a recipe previously not installed in my image and want to install it during runtime, the dnf install command fails, saying that the package's dependencies are not met (see [2] for more info). This is fine, as I can see that the tmp/deploy/rpm directory does not contain all of the package's dependencies nor the dependencies' WORKDIRs contain the deploy-rpms directory. Is this behavior correct? Shouldn't building a recipe also lead that all of its dependencies are not just built but also packaged and put into the tpm/deploy/rpm directory (the same as with building the image containing the package in question)?

The default behavior is that bitbake runs the absolute minimum of
tasks needed to complete the task you specify (in this case, producing
the rpm). Making a vim rpm requires only do_package of its
dependencies, but not their do_package_write_rpm.

If you want all dependent rpms too, this is easily done with:
bitbake --runall package_write_rpm vim

> 2) Is there an official, stable, and not some hacky way to install already pre-built packages during the build and not later during runtime? So instead of building from source or from the sstate-cache, to provide a URL to a webserver holding packages, which would then be pulled and installed (if I am not wrong Isar does this)?

If you ask me, pulling pre-built packages into a yocto image is in
itself a horrible hack that goes against yocto philosophy, and I would
not want yocto to help that or support it in any way.

We build from source. Isar can do what it wants, but I don't have to
like or support that.

There's a separate question of pulling pre-built *binaries* that
sometimes come in .rpm archives and repackaging them with recipes,
which is sort of supported if you list the rpm in SRC_URI (bitbake
will then fetch/unpack/repackage), but I don't like that either for
different reasons (vendor blobs are evil).

> 3) Is enabling and using the package runtime management recommended and encouraged when it comes to Yocto running on actual devices and not during a development phase? IMHO we are talking about embedded devices here, with little or no user interaction, with secure boot enabled, atomic updates, etc. and having a package management system such as dnf or apt is cool, but is also kind of brittle (a simple power off during apt update can break the system).

That's exactly correct. Classic package management (rpm/dpkg) was
created for the 'Unix workstation' use case where you want to provide
as much available disk space as possible to users (so A/B partitioning
is a non-starter), and users can easily add things they want to their
system. Nobody ever claimed that such mass-upgrade transactions can be
safely interrupted by a power outage, and in all likelihood you'll
have a half-updated system with an undefined behaviour on the next
boot.

Alex

Re: [yocto] Yocto Binary Distro

[Aleksandar Nikolic]

[-- Attachment #1: Type: text/plain, Size: 3542 bytes --]

On Mon, Oct 7, 2024 at 01:31 PM, Alexander Kanavin wrote:

> 
> On Mon, 7 Oct 2024 at 07:06, Aleksandar Nikolic via
> lists.yoctoproject.org
> <aleksandar.nikolic010=gmail.com@lists.yoctoproject.org> wrote:
> 
>> 1) When I build a recipe previously not installed in my image and want to
>> install it during runtime, the dnf install command fails, saying that the
>> package's dependencies are not met (see [2] for more info). This is fine,
>> as I can see that the tmp/deploy/rpm directory does not contain all of the
>> package's dependencies nor the dependencies' WORKDIRs contain the
>> deploy-rpms directory. Is this behavior correct? Shouldn't building a
>> recipe also lead that all of its dependencies are not just built but also
>> packaged and put into the tpm/deploy/rpm directory (the same as with
>> building the image containing the package in question)?
> 
> The default behavior is that bitbake runs the absolute minimum of
> tasks needed to complete the task you specify (in this case, producing
> the rpm). Making a vim rpm requires only do_package of its
> dependencies, but not their do_package_write_rpm.
> 
> If you want all dependent rpms too, this is easily done with:
> bitbake --runall package_write_rpm vim

Thanks, will try this later!

> 
> 
>> 2) Is there an official, stable, and not some hacky way to install already
>> pre-built packages during the build and not later during runtime? So
>> instead of building from source or from the sstate-cache, to provide a URL
>> to a webserver holding packages, which would then be pulled and installed
>> (if I am not wrong Isar does this)?
> 
> If you ask me, pulling pre-built packages into a yocto image is in
> itself a horrible hack that goes against yocto philosophy, and I would
> not want yocto to help that or support it in any way.
> 
> We build from source. Isar can do what it wants, but I don't have to
> like or support that.
> 
> There's a separate question of pulling pre-built *binaries* that
> sometimes come in .rpm archives and repackaging them with recipes,
> which is sort of supported if you list the rpm in SRC_URI (bitbake
> will then fetch/unpack/repackage), but I don't like that either for
> different reasons (vendor blobs are evil).

I am of the same opinion and even though pre-built binaries do come in handy in some occasions, IMHO they bring a lot of other issues along and it's not worth it. Just wanted to hear other opinions on the mailing list.

Aleksandar

> 
> 
>> 3) Is enabling and using the package runtime management recommended and
>> encouraged when it comes to Yocto running on actual devices and not during
>> a development phase? IMHO we are talking about embedded devices here, with
>> little or no user interaction, with secure boot enabled, atomic updates,
>> etc. and having a package management system such as dnf or apt is cool,
>> but is also kind of brittle (a simple power off during apt update can
>> break the system).
> 
> That's exactly correct. Classic package management (rpm/dpkg) was
> created for the 'Unix workstation' use case where you want to provide
> as much available disk space as possible to users (so A/B partitioning
> is a non-starter), and users can easily add things they want to their
> system. Nobody ever claimed that such mass-upgrade transactions can be
> safely interrupted by a power outage, and in all likelihood you'll
> have a half-updated system with an undefined behaviour on the next
> boot.
> 
> Alex

[-- Attachment #2: Type: text/html, Size: 3734 bytes --]

Re: [yocto] Yocto Binary Distro

[Dmitry Baryshkov]

On Mon, 7 Oct 2024 at 13:32, Alexander Kanavin via
lists.yoctoproject.org <alex.kanavin=gmail.com@lists.yoctoproject.org>
wrote:
>
> On Mon, 7 Oct 2024 at 07:06, Aleksandar Nikolic via
> lists.yoctoproject.org
> <aleksandar.nikolic010=gmail.com@lists.yoctoproject.org> wrote:
> > 2) Is there an official, stable, and not some hacky way to install already pre-built packages during the build and not later during runtime? So instead of building from source or from the sstate-cache, to provide a URL to a webserver holding packages, which would then be pulled and installed (if I am not wrong Isar does this)?
>
> If you ask me, pulling pre-built packages into a yocto image is in
> itself a horrible hack that goes against yocto philosophy, and I would
> not want yocto to help that or support it in any way.

I think this needs a bit of explanation.

Ångröm was a binary distro (thanks Koen!). But it existed in a
different time and with different development practices. It was great
when we had a single monolithic openembedded (OE-Classic) repo. Each
recipe had a PR (package revision, the part of the version after the
dash), PRs were increased manually when a recipe was changed in a way
which changed the packages. Then opkg upgrade would pick up updated
packages. Manual work, lots of mistakes, lots of patch conflicts.

Then came OE-Core + layers. Basic recipe comes one layer, other layers
provide .bbappends. Let each bbappend add a local part to the PR so
that each update generates new package to be picked up by the machine.
Even more mess, lots of confusion.

Let's drop PRs and generate them automatically. Great idea, but it
doesn't always work as expected. It's too easy to get the version to
go backwards (because of the SRCREV rollback, or because of other
recipe changes; or just because you have added a layer with bbappend
and then removed it). So the feed has a new package with old PV-PR,
the board has an old package with bigger PV-PR, no upgrades, tons of
confusion.

So, it is possible to create a binary distro on top of Yocto. However
one should be very careful with packages and versions, with using a
stable set of layers, etc.

While the Yocto project probably could provide a feed of build
packages, its usefulness would be pretty limited to just oe-core or
oe-core + meta-poky (or some other fixed layer configuration). There
has been ongoing work to make such configurations work better, but I
have doubts that this is going to work as expected on a general scale.

So, as much as I like OE, if you are thinking about a binary distro, I
think you'd better use some existing purposely binary distro (like
Debian, Armbian or something from the RPM world). If you can afford
having package management on the device, it's not tightly resource
constrained. And thus there is no need to go Yocto way.

> We build from source. Isar can do what it wants, but I don't have to
> like or support that.
>
> There's a separate question of pulling pre-built *binaries* that
> sometimes come in .rpm archives and repackaging them with recipes,
> which is sort of supported if you list the rpm in SRC_URI (bitbake
> will then fetch/unpack/repackage), but I don't like that either for
> different reasons (vendor blobs are evil).
>
> > 3) Is enabling and using the package runtime management recommended and encouraged when it comes to Yocto running on actual devices and not during a development phase? IMHO we are talking about embedded devices here, with little or no user interaction, with secure boot enabled, atomic updates, etc. and having a package management system such as dnf or apt is cool, but is also kind of brittle (a simple power off during apt update can break the system).
>
> That's exactly correct. Classic package management (rpm/dpkg) was
> created for the 'Unix workstation' use case where you want to provide
> as much available disk space as possible to users (so A/B partitioning
> is a non-starter), and users can easily add things they want to their
> system. Nobody ever claimed that such mass-upgrade transactions can be
> safely interrupted by a power outage, and in all likelihood you'll
> have a half-updated system with an undefined behaviour on the next
> boot.

That's another good point. One thing is fixing a workstation or a
laptop with the KBD and display, where in the worst case one cane boot
into single mode or just a shell and play dirty tricks if package
management and FS got really broken. On an embedded device there can
be no UART, no way to change boot args, etc. So, if the `opkg upgrade`
was power-interrupted YMMV.

-- 
With best wishes
Dmitry

Re: [yocto] Yocto Binary Distro

[Alexander Kanavin]

On Mon, 7 Oct 2024 at 21:07, Dmitry Baryshkov <dbaryshkov@gmail.com> wrote:
> > If you ask me, pulling pre-built packages into a yocto image is in
> > itself a horrible hack that goes against yocto philosophy, and I would
> > not want yocto to help that or support it in any way.

...

> So, as much as I like OE, if you are thinking about a binary distro, I
> think you'd better use some existing purposely binary distro (like
> Debian, Armbian or something from the RPM world). If you can afford
> having package management on the device, it's not tightly resource
> constrained. And thus there is no need to go Yocto way.

I think we're talking about two different things here, so just in case
I'd clarify the difference. Point two is about using bitbake to
somehow pull a big pile of prebuilt binary packages from the net and
assemble a target image out of them. I don't like that at all.

On the other hand having a yocto-based binary distro, e.g. a target
image with package management which feels like the traditional linux
distributions is fine with me. I'm not particularly interested in it,
and I'm also not sure it can work well, but people are welcome to work
on it if they find it interesting or useful.

Alex

Re: [yocto] Yocto Binary Distro

[Dmitry Baryshkov]

On Mon, 7 Oct 2024 at 21:27, Alexander Kanavin <alex.kanavin@gmail.com> wrote:
>
> On Mon, 7 Oct 2024 at 21:07, Dmitry Baryshkov <dbaryshkov@gmail.com> wrote:
> > > If you ask me, pulling pre-built packages into a yocto image is in
> > > itself a horrible hack that goes against yocto philosophy, and I would
> > > not want yocto to help that or support it in any way.
>
> ...
>
> > So, as much as I like OE, if you are thinking about a binary distro, I
> > think you'd better use some existing purposely binary distro (like
> > Debian, Armbian or something from the RPM world). If you can afford
> > having package management on the device, it's not tightly resource
> > constrained. And thus there is no need to go Yocto way.
>
> I think we're talking about two different things here, so just in case
> I'd clarify the difference. Point two is about using bitbake to
> somehow pull a big pile of prebuilt binary packages from the net and
> assemble a target image out of them. I don't like that at all.

Ugh, no, thank you. I also hate that idea.

>
> On the other hand having a yocto-based binary distro, e.g. a target
> image with package management which feels like the traditional linux
> distributions is fine with me. I'm not particularly interested in it,
> and I'm also not sure it can work well, but people are welcome to work
> on it if they find it interesting or useful.

Yep, I was thinking about the yocto-with-feeds.

-- 
With best wishes
Dmitry

Re: [yocto] Yocto Binary Distro

[Aleksandar Nikolic]

[-- Attachment #1: Type: text/plain, Size: 2695 bytes --]

Hi,

I'll reply on the last message from Alex, but thanks Dmitry for your initial message and the story how we came from Ångröm to what we have now. Also thanks to both of you for the extensive feedback!

On Mon, Oct 7, 2024 at 09:29 PM, Dmitry Baryshkov wrote:

> 
> On Mon, 7 Oct 2024 at 21:27, Alexander Kanavin <alex.kanavin@gmail.com>
> wrote:
> 
>> On Mon, 7 Oct 2024 at 21:07, Dmitry Baryshkov <dbaryshkov@gmail.com>
>> wrote:
>> 
>>> 
>>>> If you ask me, pulling pre-built packages into a yocto image is in
>>>> itself a horrible hack that goes against yocto philosophy, and I would
>>>> not want yocto to help that or support it in any way.
>>> 
>>> 
>> 
>> ...
>> 
>> 
>>> So, as much as I like OE, if you are thinking about a binary distro, I
>>> think you'd better use some existing purposely binary distro (like
>>> Debian, Armbian or something from the RPM world). If you can afford
>>> having package management on the device, it's not tightly resource
>>> constrained. And thus there is no need to go Yocto way.
>> 
>> I think we're talking about two different things here, so just in case
>> I'd clarify the difference. Point two is about using bitbake to
>> somehow pull a big pile of prebuilt binary packages from the net and
>> assemble a target image out of them. I don't like that at all.
> 
> Ugh, no, thank you. I also hate that idea.

Yep, thanks for separating these two use cases. Regarding the former one (pulling a bunch of prebuilt binary packages from a server and assembling a target image out of them ), some coworkers want exactly this and I am trying to persuade them not to go down that path and just build from source. IMHO, even though installing previously tested binaries might bring some assurance for the QM team, it brings a hell lot of issues with it which are easily avoided by building from source.

> 
> 
>> On the other hand having a yocto-based binary distro, e.g. a target
>> image with package management which feels like the traditional linux
>> distributions is fine with me. I'm not particularly interested in it,
>> and I'm also not sure it can work well, but people are welcome to work
>> on it if they find it interesting or useful.
> 
> Yep, I was thinking about the yocto-with-feeds.

Yes, I also think this use case makes more sense than the first one, but I also see it to be used during development only, so devs can install and use packages faster (than rebuilding the whole image). Updating a device in the field with a package management system is however in my opinion a no-go (from the reasons previously stated by all).

Aleksandar

> 
> --
> With best wishes
> Dmitry

[-- Attachment #2: Type: text/html, Size: 3690 bytes --]

Re: [yocto] Yocto Binary Distro

[Dmitry Baryshkov]

On Mon, 7 Oct 2024 at 22:34, Aleksandar Nikolic via
Lists.Yoctoproject.Org
<aleksandar.nikolic010=gmail.com@lists.yoctoproject.org> wrote:
>
> Hi,
>
> I'll reply on the last message from Alex, but thanks Dmitry for your initial message and the story how we came from Ångröm to what we have now. Also thanks to both of you for the extensive feedback!
>
> On Mon, Oct 7, 2024 at 09:29 PM, Dmitry Baryshkov wrote:
>
> On Mon, 7 Oct 2024 at 21:27, Alexander Kanavin <alex.kanavin@gmail.com> wrote:
>
> On Mon, 7 Oct 2024 at 21:07, Dmitry Baryshkov <dbaryshkov@gmail.com> wrote:
>
> If you ask me, pulling pre-built packages into a yocto image is in
> itself a horrible hack that goes against yocto philosophy, and I would
> not want yocto to help that or support it in any way.
>
> ...
>
> So, as much as I like OE, if you are thinking about a binary distro, I
> think you'd better use some existing purposely binary distro (like
> Debian, Armbian or something from the RPM world). If you can afford
> having package management on the device, it's not tightly resource
> constrained. And thus there is no need to go Yocto way.
>
> I think we're talking about two different things here, so just in case
> I'd clarify the difference. Point two is about using bitbake to
> somehow pull a big pile of prebuilt binary packages from the net and
> assemble a target image out of them. I don't like that at all.
>
> Ugh, no, thank you. I also hate that idea.
>
> Yep, thanks for separating these two use cases. Regarding the former one (pulling a bunch of prebuilt binary packages from a server and assembling a target image out of them ), some coworkers want exactly this and I am trying to persuade them not to go down that path and just build from source. IMHO, even though installing previously tested binaries might bring some assurance for the QM team, it brings a hell lot of issues with it which are easily avoided by building from source.
>
>
> On the other hand having a yocto-based binary distro, e.g. a target
> image with package management which feels like the traditional linux
> distributions is fine with me. I'm not particularly interested in it,
> and I'm also not sure it can work well, but people are welcome to work
> on it if they find it interesting or useful.
>
> Yep, I was thinking about the yocto-with-feeds.
>
> Yes, I also think this use case makes more sense than the first one, but I also see it to be used during development only, so devs can install and use packages faster (than rebuilding the whole image). Updating a device in the field with a package management system is however in my opinion a no-go (from the reasons previously stated by all).

Consider using shared SSTATE, it saves you from all the rebuilds,
while keeping Yocto internals intact and sane.


-- 
With best wishes
Dmitry

Re: [yocto] Yocto Binary Distro

[Aleksandar Nikolic]

[-- Attachment #1: Type: text/plain, Size: 826 bytes --]

On Tue, Oct 8, 2024 at 01:25 AM, Dmitry Baryshkov wrote:

> 
> Consider using shared SSTATE, it saves you from all the rebuilds,
> while keeping Yocto internals intact and sane.

We are using sstate cache, but if I am not mistaken it can happen that even though a package is in the sstate cache, there's a cache miss and the package gets rebuilt - and this is exactly what some coworkers would like to avoid, cause it means a different binary is on the target than the one previously built and tested. That's why I posted another question here (maybe 2 weeks ago) if there's a way to force Yocto to only use sstate-cache and fail if a package needs to be rebuilt or to print a list of packages which were built from source. But there's no an "easy" way to do it.

Aleksandar

> 
> --
> With best wishes
> Dmitry

[-- Attachment #2: Type: text/html, Size: 938 bytes --]

Re: [yocto] Yocto Binary Distro

[Mikko Rapeli]

Hi,

On Tue, Oct 08, 2024 at 01:30:47AM -0700, Aleksandar Nikolic via lists.yoctoproject.org wrote:
> On Tue, Oct 8, 2024 at 01:25 AM, Dmitry Baryshkov wrote:
> 
> > 
> > Consider using shared SSTATE, it saves you from all the rebuilds,
> > while keeping Yocto internals intact and sane.
> 
> We are using sstate cache, but if I am not mistaken it can happen that even though a package is in the sstate cache, there's a cache miss and the package gets rebuilt - and this is exactly what some coworkers would like to avoid, cause it means a different binary is on the target than the one previously built and tested. That's why I posted another question here (maybe 2 weeks ago) if there's a way to force Yocto to only use sstate-cache and fail if a package needs to be rebuilt or to print a list of packages which were built from source. But there's no an "easy" way to do it.

Why is one compilation of the released recipe different from another? Who
maintains the toolchains and dependencies?

Yocto is for building from sources. It doesn't really have ABI checks and layers
which would easily enable to freeze an ABI and integrate binaries on top
of that. Every rebuild can change the ABIs in significant ways.
The approach in bitbake is to rebuild full dependency tree if something changes.
Compilers, glibc etc get updated all the time and it's important to know if something
fails to compile, also to make sure that needed features are enabled, correct
compile flags applied etc.

It is possible to integrate binaries into yocto builds but then it must
be understood that there are no ABI compatibility checks. The binary
may start failing at any time. The recompilations may be costly but
that's how yocto is. sstate cache helps but doesn't take away the
full tree rebuilds when something e.g. native recipe changes.

IMO it is tricky to fight against this and it's better to accept it.
If developers want stable ABIs and integrate binaries then other distros
could be better but they come with different problems. Debian is hard to
optimize for the full stack, e.g. distro and machine features. OBS
may help building only subset of packages from scratch, but even there
ABI checks may not exist and user need to rely on test automation to detect
breakage.

Yocto can build an SDK which can be used to generate 'golden' binaries but
again compatibility of different SDK builds needs to be checked
to know when binaries may start breaking. At various points ABIs change so
much that all binarees need to be recompiled. Think of a security update
which brings a new openssl version with incompatible ABI.

Cheers,

-Mikko

Re: [yocto] Yocto Binary Distro

[Aleksandar Nikolic]

[-- Attachment #1: Type: text/plain, Size: 3024 bytes --]

On Tue, Oct 8, 2024 at 10:53 AM, Mikko Rapeli wrote:

> 
> Hi,
> 
> Why is one compilation of the released recipe different from another? Who
> maintains the toolchains and dependencies?

It's not, the layers would remain the same, meaning the toolchains and dependencies don't change. However, due to custom layers we use (but not maintain) we can't ensure reproducible builds, meaning if we rebuild packages, we can't be sure the built binaries are always the same (bit per bit)... and some coworkers don't like that (even though IMO it makes little to no difference at all).

> 
> Yocto is for building from sources. It doesn't really have ABI checks and
> layers
> which would easily enable to freeze an ABI and integrate binaries on top
> of that. Every rebuild can change the ABIs in significant ways.
> The approach in bitbake is to rebuild full dependency tree if something
> changes.
> Compilers, glibc etc get updated all the time and it's important to know
> if something
> fails to compile, also to make sure that needed features are enabled,
> correct
> compile flags applied etc.
> 
> It is possible to integrate binaries into yocto builds but then it must
> be understood that there are no ABI compatibility checks. The binary
> may start failing at any time. The recompilations may be costly but
> that's how yocto is. sstate cache helps but doesn't take away the
> full tree rebuilds when something e.g. native recipe changes.

Thanks for these points, I didn't know Yocto doesn't have ABI checks and this is a very valid point why building from sources is the way to go and why updating the whole OS and not single packages is important.

> 
> IMO it is tricky to fight against this and it's better to accept it.
> If developers want stable ABIs and integrate binaries then other distros
> could be better but they come with different problems. Debian is hard to
> optimize for the full stack, e.g. distro and machine features. OBS
> may help building only subset of packages from scratch, but even there
> ABI checks may not exist and user need to rely on test automation to
> detect
> breakage.
> 
> Yocto can build an SDK which can be used to generate 'golden' binaries but
> 
> again compatibility of different SDK builds needs to be checked
> to know when binaries may start breaking. At various points ABIs change so
> 
> much that all binarees need to be recompiled. Think of a security update
> which brings a new openssl version with incompatible ABI.

To build from the SDK occurred to us as well, but is there a offical/stable Yocto way to integrate those binaries back into the image? I don't like the idea of building a base image in Yocto, then building binaries separately in SDK and installing them back into the image with a custom script, as this for sure will be unstable and error-prone. Add atomic updates to that and it might become a nightmare to maintain for multiple products.

Cheers,
Aleksandar

> 
> Cheers,
> 
> -Mikko

[-- Attachment #2: Type: text/html, Size: 3267 bytes --]

Re: [yocto] Yocto Binary Distro

[Mikko Rapeli]

Hi,

On Tue, Oct 08, 2024 at 05:09:33AM -0700, Aleksandar Nikolic via Lists.Yoctoproject.Org wrote:
> To build from the SDK occurred to us as well, but is there a offical/stable Yocto way to integrate those binaries back into the image? I don't like the idea of building a base image in Yocto, then building binaries separately in SDK and installing them back into the image with a custom script, as this for sure will be unstable and error-prone. Add atomic updates to that and it might become a nightmare to maintain for multiple products.

bin_package.bbclass helps when SRC_UKI points to a binary RPM package:

https://docs.yoctoproject.org/singleindex.html#bin-package

but it does not solve the ABI issues. Those need to be managed separately.

Cheers,

-Mikko