Hi,

Why is one compilation of the released recipe different from another? Who
maintains the toolchains and dependencies?

It's not, the layers would remain the same, meaning the toolchains and dependencies don't change. However, due to custom layers we use (but not maintain) we can't ensure reproducible builds, meaning if we rebuild packages, we can't be sure the built binaries are always the same (bit per bit)... and some coworkers don't like that (even though IMO it makes little to no difference at all).

Yocto is for building from sources. It doesn't really have ABI checks and layers
which would easily enable to freeze an ABI and integrate binaries on top
of that. Every rebuild can change the ABIs in significant ways.
The approach in bitbake is to rebuild full dependency tree if something changes.
Compilers, glibc etc get updated all the time and it's important to know if something
fails to compile, also to make sure that needed features are enabled, correct
compile flags applied etc.

It is possible to integrate binaries into yocto builds but then it must
be understood that there are no ABI compatibility checks. The binary
may start failing at any time. The recompilations may be costly but
that's how yocto is. sstate cache helps but doesn't take away the
full tree rebuilds when something e.g. native recipe changes.

Thanks for these points, I didn't know Yocto doesn't have ABI checks and this is a very valid point why building from sources is the way to go and why updating the whole OS and not single packages is important.

IMO it is tricky to fight against this and it's better to accept it.
If developers want stable ABIs and integrate binaries then other distros
could be better but they come with different problems. Debian is hard to
optimize for the full stack, e.g. distro and machine features. OBS
may help building only subset of packages from scratch, but even there
ABI checks may not exist and user need to rely on test automation to detect
breakage.

Yocto can build an SDK which can be used to generate 'golden' binaries but
again compatibility of different SDK builds needs to be checked
to know when binaries may start breaking. At various points ABIs change so
much that all binarees need to be recompiled. Think of a security update
which brings a new openssl version with incompatible ABI.

To build from the SDK occurred to us as well, but is there a offical/stable Yocto way to integrate those binaries back into the image? I don't like the idea of building a base image in Yocto, then building binaries separately in SDK and installing them back into the image with a custom script, as this for sure will be unstable and error-prone. Add atomic updates to that and it might become a nightmare to maintain for multiple products.