RE: [OE-core] [PATCH 1/7] scripts/oe-setup-builddir: add a check that TEMPLATECONF is valid

[Peter Kjellerstedt <peter.kjellerstedt@axis.com>]

> -----Original Message-----
> From: Alexander Kanavin <alex.kanavin@gmail.com>
> Sent: den 15 september 2022 10:20
> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> Cc: ChenQi <Qi.Chen@windriver.com>; OE-core <openembedded-
> core@lists.openembedded.org>; Alexander Kanavin <alex@linutronix.de>
> Subject: Re: [OE-core] [PATCH 1/7] scripts/oe-setup-builddir: add a check
> that TEMPLATECONF is valid
> 
> On Thu, 15 Sept 2022 at 03:07, Peter Kjellerstedt
> <peter.kjellerstedt@axis.com> wrote:
> > I know you have suggested this to me too as we use a similar setup to
> > generate the bblayers.conf.sample file based on the layers that repo
> > has checked out in our environment, and now I realized why this is not
> > as simple as it seems. The problem (at least in our case) is that we
> > run as a wrapper around oe-init-build-env. This means that before the
> > wrapper sources oe-init-build-env, the build directory and thus the
> > conf directory inside it (where the bblayers.conf file is supposed to
> > go) may not exist. And after it has sourced oe-init-build-env it is
> > too late as the bblayers.conf file needs to exists before. So to create
> > the bblayers.conf file, the wrapper would have to duplicate all the
> > non-trivial code in oe-setup-builddir that determines what the build
> > directory should be called. And while this of course is possible to
> > do, it means duplicating code that is internal to bitbake, and risking
> > missing important changes to the upstream code in the future. Thus it
> > is much easier and safer to generate the template files and let
> > oe-init-build-env do its job.
> 
> Wait, I have to ask for details here. What is the non-trivial code
> specifically? Isn't it simply
> 
> mkdir -p builddir/conf
> write_out bblayers.conf
> 
> where builddir is the same argument that is passed to
> oe-init-build-env? Something doesn't compute here for me.

Since our code is a wrapper around oe-init-build-env, it would have to do 
the exact same calculations as are done in oe-setup-builddir, and if you 
look at that script, you will see that there are actually quite a lot of 
options to determine the name of the build directory.

> 
> > Where I (and apparently WindRiver) are coming from, the decision on what
> > layers are part of the configuration is not something a specific layer
> > is responsible for. I.e., in our setup we fetch the layers we want for
> > a build, write the list of layers to bblayers.conf.sample and source
> > oe-init-build-env. With your setup, if I want a project with the two
> > layers meta-a and meta-b, I would also have to create a layer meta-x
> > where I create a static bblayers.conf.sample file that lists these
> > three layers. Then, if another project wants to use meta-a and meta-c,
> > they would have to add a second static bblayers.conf.sample file in
> > meta-x that lists those three layers. Basically every time there is a
> > new combination of layers fetched, there would also have to be a
> > duplicate of the information in a static bblayers.conf.sample file
> > somewhere. I am probably missing something here, but to me this seems
> > suboptimal. If the list of layers in the static file had been used to
> > fetch the layers, I would better understand it, but then the problem
> > becomes that the list is now in one of the arbitrary layers that are
> > supposed to be fetched.
> 
> The idea is that yes, every possible combination of layers needs its
> own template, and it goes under version control into a layer.

This seems incredibly inflexible if you have many optional layers. 
We use repo's --groups option to handle optional layers. This means 
we can, e.g., use one manifest to build the main firmware, but also 
to build special firmware for production testing by adding an option 
to repo, telling it to fetch an extra layer. This also means that all 
the project manifests don't need to exist in two versions (one for 
the main firmware and one for the production testing) as this is 
handled by the base manifest. 

With your solution, every project would have to create two templates 
to cover this. And if we add another optional layer (we currently don't 
have one, but we used to do), then we would have to add one or two 
more templates and so on.

Also, if we decide to add a layer to the base platform, then all the 
project templates are broken until someone updates them all. But since 
they would be in the project layers, that is not something that I as 
the maintainer of the base platform can do.

> Typically this would be a layer that is common to all of your build
> configurations (which is meta-a in the above example - *not* meta-x)
> and that is where the templates go. Do you have such a layer?

Well, we do for the base platform, but then the project templates would 
have to be in the project layers.

I know you are currently not solving the problem of actually fetching 
the layers, but I do not understand how you are planning to that when 
the definition of what layers to use is inside some random layer. And 
a solution that would fetch all layers would definitely not be usable 
by us at least. Which means that to setup a random product I would 
actually need to have inside knowledge of which layer actually happens 
to have the list of what layers are needed for that product.

Or are you planning on having some definition of what layers to fetch 
outside the layers (where I think it belongs)? Then why would one want 
to have duplicate information in one of the fetched layers to setup 
the BBLAYERS variable?

> 
> I also have to ask, what prevents you from simply listing all of the
> layers in a single unified bblayers.conf.sample. Do they step on each
> other? How?

The project layers can definitely step on each other. We also have some 
layers that modify the build completely by being included (they are 
obviously not Yocto Compatible as there is no reason for them to be, 
given that we only fetch and include the layers in our builds that are 
actually used).

> 
> I also need to remind you that bblayers.conf(.sample) does support
> includes, so you can implement a structured hierarchy of layer config
> fragments, where the common bits are only written once.

AFAIK, oe-init-build-env does not support that. It will only replace 
##OEROOT## in the bblayers.conf.sample file.

> 
> Basically, there are alternatives, and I think all of them are better
> than writing out the list of layers just before setting up a build
> with a custom script.

I cannot for the life of me see how your solution is better in any way 
than the one we have. In our solution we have simple, hierarchical 
manifests that specify what layers are to be used. Then the init scripts 
automatically set up the bitbake environment to use those layers. All 
the manifests are in one repository. Adding a layer to the platform means 
adding one line to one manifest. Adding a new project means adding one 
manifest (with basically one line specifying the project layer).

> 
> > in our manifest repository. This simple manifest is all the project
> > team needs to create to build with their project specific layer. It
> > also means that the only input to our Jenkins jobs to build for this
> > project (instead of the main platform) is the name of the project's
> > repo manifest. It also means that if we make any changes to the main
> > platform, e.g., add a new layer (which is a one line change to the
> > cfp.xml manifest), all projects will automatically get and use that
> > new layer without having to make any changes to all the project
> > manifests.
> 
> This is somewhat tangential, as it is about extending the default
> layer setup config, not the combinations of layers in the build
> configuration. So far, the json is not extensible, but we can think
> about how to extend and overlay it too. As far as development in
> meta-foo goes, the templates (including layer lists) for it can be
> hosted there, until the layer 'graduates' into the common structure.
> 
> Alex

//Peter