defining standard build targets in layer metadata

defining standard build targets in layer metadata

[Alexander Kanavin]

Hello all,

working on bitbake-setup made me consider that a true end-to-end
configuration tool should also know what to build, not only what to
set up.

Earlier bitbake-setup prototypes allowed defining "oe-targets":[...]
in its configuration files, but I later on realized this doesn't scale
and isn't flexible enough, and isn't in the right place to begin with.
So I took it out in the latest patchset.

So here's a quick, not-well-thought-out idea:

BB_TARGETS = "build:'core-image-minimal' test:'-c testmage core-image-minimal' "

defined somewhere in a layer (e.g. conf/distro/poky.conf or in a fragment).

And then one would be able to run

bitbake build
bitbake test
bitbake flash
bitbake deploy
etc.

which would standardize not only setting up a build but also running
it. The particular set of generic target names and their values could
be defined differently, but the basic idea would be similar.

Thoughts?

Alex

Re: [yocto] defining standard build targets in layer metadata

[Mark Hatle]

On 8/20/25 4:42 AM, Alexander Kanavin via lists.yoctoproject.org wrote:
> Hello all,
> 
> working on bitbake-setup made me consider that a true end-to-end
> configuration tool should also know what to build, not only what to
> set up.
> 
> Earlier bitbake-setup prototypes allowed defining "oe-targets":[...]
> in its configuration files, but I later on realized this doesn't scale
> and isn't flexible enough, and isn't in the right place to begin with.
> So I took it out in the latest patchset.
> 
> So here's a quick, not-well-thought-out idea:
> 
> BB_TARGETS = "build:'core-image-minimal' test:'-c testmage core-image-minimal' "
> 
> defined somewhere in a layer (e.g. conf/distro/poky.conf or in a fragment).
> 
> And then one would be able to run
> 
> bitbake build
> bitbake test
> bitbake flash
> bitbake deploy
> etc.
> 
> which would standardize not only setting up a build but also running
> it. The particular set of generic target names and their values could
> be defined differently, but the basic idea would be similar.
> 
> Thoughts?

In my experience this teaches an in-experienced user that there is only one 
target (build), and if they want to customize, they edit their local.conf with 
IMAGE_INSTALL:append, or add bbappends to the one target.

This is unfortunately really dangerous and easy for people to step into.  It's a 
big reason why we recently deprecated our wrappers and we're pushing people 
(internally and externally) to define their own image recipes for what they're 
trying to do, instead of continuously hacking on whatever the default for the 
wrappers (configuration) was.

I do understand the utility of this, but in practice it's just been bad overall. 
  It would be better, IMHO, to allow a layer or configuration or whatever to say 
"we suggest you use one of the following pre-written targets" and then provide 
the user a list of suggested targets, but force the user to issue

bitbake <target>


I've always wondered if there was a way to do a more 'dynamic' conf-notes like 
file.  So when someone runs the equivalent of oe-init-build-env, it can give 
them notes that were provided by the layers that are part of their project. 
Everything from machines to images to configurations they might care about. 
(Obviously we don't want to write a book of configuration here, but especially 
in distribution layers, I've definitely wanted to give the user some guidance.)

The Wind River configuration mechanism had a way to inject info and warnings as 
part of the generated configuration for this reason.

--Mark

> Alex
> 
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#65774): https://lists.yoctoproject.org/g/yocto/message/65774
> Mute This Topic: https://lists.yoctoproject.org/mt/114796592/3616948
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [mark.hatle@kernel.crashing.org]
> -=-=-=-=-=-=-=-=-=-=-=-
> 

RE: [yocto] defining standard build targets in layer metadata

[Peter Kjellerstedt]

> -----Original Message-----
> From: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> On Behalf Of Mark Hatle
> Sent: den 20 augusti 2025 15:19
> To: yocto@lists.yoctoproject.org; alex.kanavin@gmail.com; openembedded-architecture <openembedded-architecture@lists.openembedded.org>
> Subject: Re: [yocto] defining standard build targets in layer metadata
> 
> On 8/20/25 4:42 AM, Alexander Kanavin via lists.yoctoproject.org wrote:
> > Hello all,
> >
> > working on bitbake-setup made me consider that a true end-to-end
> > configuration tool should also know what to build, not only what to
> > set up.
> >
> > Earlier bitbake-setup prototypes allowed defining "oe-targets":[...]
> > in its configuration files, but I later on realized this doesn't scale
> > and isn't flexible enough, and isn't in the right place to begin with.
> > So I took it out in the latest patchset.
> >
> > So here's a quick, not-well-thought-out idea:
> >
> > BB_TARGETS = "build:'core-image-minimal' test:'-c testmage core-image-minimal' "
> >
> > defined somewhere in a layer (e.g. conf/distro/poky.conf or in a fragment).
> >
> > And then one would be able to run
> >
> > bitbake build
> > bitbake test
> > bitbake flash
> > bitbake deploy
> > etc.
> >
> > which would standardize not only setting up a build but also running
> > it. The particular set of generic target names and their values could
> > be defined differently, but the basic idea would be similar.
> >
> > Thoughts?
> 
> In my experience this teaches an in-experienced user that there is only one
> target (build), and if they want to customize, they edit their local.conf with
> IMAGE_INSTALL:append, or add bbappends to the one target.
> 
> This is unfortunately really dangerous and easy for people to step into. It's a
> big reason why we recently deprecated our wrappers and we're pushing people
> (internally and externally) to define their own image recipes for what they're
> trying to do, instead of continuously hacking on whatever the default for the
> wrappers (configuration) was.
> 
> I do understand the utility of this, but in practice it's just been bad overall.
>   It would be better, IMHO, to allow a layer or configuration or whatever to say
> "we suggest you use one of the following pre-written targets" and then provide
> the user a list of suggested targets, but force the user to issue
> 
> bitbake <target>
> 
> 
> I've always wondered if there was a way to do a more 'dynamic' conf-notes like
> file.  So when someone runs the equivalent of oe-init-build-env, it can give
> them notes that were provided by the layers that are part of their project.
> Everything from machines to images to configurations they might care about.
> (Obviously we don't want to write a book of configuration here, but especially
> in distribution layers, I've definitely wanted to give the user some guidance.)

The way we've solved this in our environment is that we have a small wrapper 
around oe-init-build-env which creates a templateconf directory dynamically 
based on the layers that are part of the build. For the local.conf.sample and 
conf-notes.txt files it goes though the layers and looks for corresponding 
files suffixed with .XX where XX is a number between 00 and 99 (it assumes 50 
for any file that has no suffix, i.e., the files from poky). Then it 
concatenates them ordered by the suffix. 

> 
> The Wind River configuration mechanism had a way to inject info and warnings as
> part of the generated configuration for this reason.
> 
> --Mark
> 
> > Alex

//Peter

RE: [yocto] defining standard build targets in layer metadata

[Bird, Tim]

> -----Original Message-----
> From: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> On Behalf Of Alexander Kanavin via lists.yoctoproject.org
> Hello all,
> 
> working on bitbake-setup made me consider that a true end-to-end
> configuration tool should also know what to build, not only what to
> set up.
> 
> Earlier bitbake-setup prototypes allowed defining "oe-targets":[...]
> in its configuration files, but I later on realized this doesn't scale
> and isn't flexible enough, and isn't in the right place to begin with.
> So I took it out in the latest patchset.
> 
> So here's a quick, not-well-thought-out idea:
> 
> BB_TARGETS = "build:'core-image-minimal' test:'-c testmage core-image-minimal' "
> 
> defined somewhere in a layer (e.g. conf/distro/poky.conf or in a fragment).
> 
> And then one would be able to run
> 
> bitbake build
> bitbake test
> bitbake flash
> bitbake deploy
> etc.
> 
> which would standardize not only setting up a build but also running
> it. The particular set of generic target names and their values could
> be defined differently, but the basic idea would be similar.
> 
> Thoughts?

I don't have deep thoughts on implementation details, but I have
experience with optimizing command lines for tools.

I think it's important to have discoverability, and the ability to shorten
argument lists when working with a command line tool.

With my own 'ttc' (target control) tool (similar to labgrid), I support
a 'ttc list' command to show which targets are currently configured in
my lab.  I also have the capability to specify a default target (ie board)
using one of: a setting in a config file, an environment variable (TTC_TARGET),
or a command line option.  So can do stuff like:
export TTC_TARGET=rpi4
ttc kbuild
ttc kinstall
ttc cp some_test target:/usr/local/bin
ttc run /usr/local/bin/some_test

Note that I don't have to specify the target on every command.
ttc also supports a 'ttc setenv <target>' command, that is similar to
oe-init-build-env.  It just creates a new shell with appropriate
environment variables for operating on that target.

I have found these very handy in my own lab.

Separately, I think it would be great if people could use bitbake for deploy
operations.  I'm sure it's possible now with appropriate recipe files, but
I suspect it's not standardized for the Yocto ecosystem.  Maybe it's already
there and I just don't know about it.

Just my 2 cents.
 -- Tim