Re: [Openembedded-architecture] [OE-core] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

[adrian.freihofer@gmail.com]

On Wed, 2023-11-01 at 16:19 +0100, Alexander Kanavin via
lists.openembedded.org wrote:
> On Wed, 1 Nov 2023 at 15:18, <adrian.freihofer@gmail.com> wrote:
> > We are currently experimenting with replacing the eSDK installer
> > with
> > the bitbake build environment for our users. Part of this
> > transformation is, of course, the shared sstate-cache, for which
> > this
> > discussion seems quite relevant. The workflow we are aiming for is
> > as
> > follows:
> > 
> >    1. Setup the layers and build config (out of scope here)
> >    2. Download the sstate for a particular recipe (usually an image
> >       recipe).
> 
> I'm not sure I understand the case for downloading the complete set
> of
> cache objects up front. What's wrong with 'lazy' downloading, e.g.
> only when the objects are actually needed in a build?
> 
> Even then, does --setscene-only do just that, or am I completely
> confused?

If I remember correctly, when configuring a SSTATE_MIRROR, the server
is queried when a Setcene task is executed. This is not ideal for
several reasons:
 * Bitbake Setcene tasks run in the context of a recipe, which means
   that thousands of connections are opened against the mirror server.
   Many servers treat this behavior as a denial of service attack. With
   a separate download tool, this can be handled with a reasonable
   number of parallel connections. With bitbake's architecture, this is
   a barely solvable issue.
 * Especially with an SDK, setcene tasks can run much more frequently
   than do_fetch tasks. So the same artifact is downloaded multiple
   times, which is not efficient.
 * SDK users may be located in different places in the world. This
   results in poor performance depending on the location.
 * With a separate download, you can download on a fast network and
   then switch to a slower network or even work offline.
 * Of course, supporting downloading in advance does not mean that a
   SSTATE_MIRROR should not be supported.

Adrian

> 
> Alex
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#1821):
> https://lists.openembedded.org/g/openembedded-architecture/message/1821
> Mute This Topic: https://lists.openembedded.org/mt/102320110/3616858
> Group Owner: openembedded-architecture+owner@lists.openembedded.org
> Unsubscribe:
> https://lists.openembedded.org/g/openembedded-architecture/unsub [
> adrian.freihofer@siemens.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>