* [PATCH RFC] parse/ast: add support for additive built-in fragments
@ 2026-09-01 7:52 Antonin Godard
2026-09-01 12:15 ` [bitbake-devel] " Richard Purdie
0 siblings, 1 reply; 5+ messages in thread
From: Antonin Godard @ 2026-09-01 7:52 UTC (permalink / raw)
To: bitbake-devel; +Cc: Thomas Petazzoni, Antonin Godard
Add support for the following syntax specified in a fragment list
definition:
fragmentname:VARIABLE:add
Setting this fragment appends to VARIABLE instead of setting its value.
This does not break pre-existing fragments which still set a variable's
value entirely.
For example, in OE-Core consider the following definition:
OE_FRAGMENTS_BUILTIN = "class:INHERIT:add"
Then one could specify the following:
OE_FRAGMENTS = "class/buildstats class/rm_work"
Which would result in appending " buildstats rm_work" to the INHERIT
variable.
This would also allow bitbake-setup configurations to come with a list
of pre-enabled classes, for example.
Note: The "add" suffix was chosen in the definition to avoid confusion
with the existing "append" usage in Bitbake.
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
Note: The bitbake-config-build OE-Core utility would require adaptations
as it currently removes any pre-existing built-in fragment with the
same suffix. I can also send the associated OE-Core patches.
Note 2: We could also have these fragments specified as:
OE_FRAGMENTS = "class/add/buildstats class/add/retain"
To avoid confusing them with original built-in fragments. While I
agree it disambiguates them from original built-ins, I also think from a
user point of view, being able to set:
OE_FRAGMENTS = "machine/qemuarm distro/poky class/buildstats class/retain"
feels a bit more natural. Discussion is open :)
---
lib/bb/parse/ast.py | 25 +++++++++++++++++++------
1 file changed, 19 insertions(+), 6 deletions(-)
diff --git a/lib/bb/parse/ast.py b/lib/bb/parse/ast.py
index a372b3534e4..a828c49c937 100644
--- a/lib/bb/parse/ast.py
+++ b/lib/bb/parse/ast.py
@@ -367,12 +367,13 @@ class AddFragmentsNode(AstNode):
def check_and_set_builtin_fragment(fragment, data, builtin_fragments):
prefix, value = fragment.split('/', 1)
if prefix in builtin_fragments.keys():
- if data.getVar(builtin_fragments[prefix], noweakdefault=True) != None:
+ fragment_var, fragment_type = builtin_fragments[prefix][0], builtin_fragments[prefix][1]
+ if fragment_type == "set" and data.getVar(fragment_var, noweakdefault=True) is not None:
bb.fatal(
("A builtin fragment '%s' is used while %s has already got an assignment.\n"
"Please either disable the fragment or remove the value assignment.\n"
"To disable the fragment, use 'bitbake-config-build disable-fragment %s'."
- ) % (fragment, builtin_fragments[prefix], fragment))
+ ) % (fragment, fragment_var, fragment))
fragment_history = data.varhistory.variable(self.fragments_variable)
loginfo={}
for fh in fragment_history[::-1]:
@@ -381,15 +382,27 @@ class AddFragmentsNode(AstNode):
loginfo["line"] = fh["line"]
loginfo["detail"] = f"{value} ({self.fragments_variable} contains \"{fragment}\")"
break
- # parsing=True since we want to emulate X=Y and allow X:override=Z to continue to exist
- data.setVar(builtin_fragments[prefix], value, parsing=True, **loginfo)
+ if fragment_type == "set":
+ # parsing=True since we want to emulate X=Y and allow X:override=Z to continue to exist
+ data.setVar(fragment_var, value, parsing=True, **loginfo)
+ elif fragment_type == "add":
+ data.appendVar(fragment_var, f" {value.strip()}", **loginfo)
+ else:
+ bb.fatal(f"Unknown fragment type '{fragment_type}' "
+ f"(from '{prefix}:{fragment_var}:{fragment_type}' in {self.builtin_fragments_variable})")
return True
return False
fragments = data.getVar(self.fragments_variable)
layers = data.getVar('BBLAYERS')
flagged_variables = data.getVar(self.flagged_variables_list_variable).split()
- builtin_fragments = {f[0]:f[1] for f in [f.split(':') for f in data.getVar(self.builtin_fragments_variable).split()] }
+ builtin_fragments = {}
+ for fragment in data.getVar(self.builtin_fragments_variable).split():
+ fragment = fragment.split(':')
+ builtin_fragments[fragment[0]] = (
+ fragment[1],
+ "set" if len(fragment) < 3 else fragment[2],
+ )
if not fragments:
return
@@ -402,7 +415,7 @@ class AddFragmentsNode(AstNode):
fragments.split(),
)
)
- if len(builtin_fragments_list) > 1:
+ if len(builtin_fragments_list) > 1 and not builtin_fragments[builtin_fragment_key][1] == "add":
bb.warn(
("Multiple builtin fragments are enabled for %s via variable %s: %s. "
"This likely points to a mis-configuration in the metadata, as only "
---
base-commit: 18cca50ba3da5ce27bc674478957bb08e7536b9a
change-id: 20260827-appending-fragments-8a308ebdb633
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [bitbake-devel] [PATCH RFC] parse/ast: add support for additive built-in fragments
2026-09-01 7:52 [PATCH RFC] parse/ast: add support for additive built-in fragments Antonin Godard
@ 2026-09-01 12:15 ` Richard Purdie
2026-09-01 18:28 ` Alexander Kanavin
0 siblings, 1 reply; 5+ messages in thread
From: Richard Purdie @ 2026-09-01 12:15 UTC (permalink / raw)
To: antonin.godard, bitbake-devel; +Cc: Thomas Petazzoni
On Tue, 2026-09-01 at 09:52 +0200, Antonin Godard via lists.openembedded.org wrote:
> Add support for the following syntax specified in a fragment list
> definition:
>
> fragmentname:VARIABLE:add
>
> Setting this fragment appends to VARIABLE instead of setting its value.
> This does not break pre-existing fragments which still set a variable's
> value entirely.
>
> For example, in OE-Core consider the following definition:
>
> OE_FRAGMENTS_BUILTIN = "class:INHERIT:add"
>
> Then one could specify the following:
>
> OE_FRAGMENTS = "class/buildstats class/rm_work"
>
> Which would result in appending " buildstats rm_work" to the INHERIT
> variable.
>
> This would also allow bitbake-setup configurations to come with a list
> of pre-enabled classes, for example.
>
> Note: The "add" suffix was chosen in the definition to avoid confusion
> with the existing "append" usage in Bitbake.
>
> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
> ---
> Note: The bitbake-config-build OE-Core utility would require adaptations
> as it currently removes any pre-existing built-in fragment with the
> same suffix. I can also send the associated OE-Core patches.
>
> Note 2: We could also have these fragments specified as:
>
> OE_FRAGMENTS = "class/add/buildstats class/add/retain"
>
> To avoid confusing them with original built-in fragments. While I
> agree it disambiguates them from original built-ins, I also think from a
> user point of view, being able to set:
>
> OE_FRAGMENTS = "machine/qemuarm distro/poky class/buildstats class/retain"
>
> feels a bit more natural. Discussion is open :)
It is definitely an interesting one and the syntax isn't too bad.. Are
there uses beyond class inherit though?
machine/distro were added as they are clear variables in common use
which avoided tons of boilerplate in layer definitions. Do we have
similar needs here or are these inherits better captured in config
fragments?
Cheers,
Richard
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [bitbake-devel] [PATCH RFC] parse/ast: add support for additive built-in fragments
2026-09-01 12:15 ` [bitbake-devel] " Richard Purdie
@ 2026-09-01 18:28 ` Alexander Kanavin
2026-09-01 19:22 ` Richard Purdie
0 siblings, 1 reply; 5+ messages in thread
From: Alexander Kanavin @ 2026-09-01 18:28 UTC (permalink / raw)
To: richard.purdie; +Cc: antonin.godard, bitbake-devel, Thomas Petazzoni
On Tue, 1 Sept 2026 at 14:15, Richard Purdie via
lists.openembedded.org
<richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote:
> It is definitely an interesting one and the syntax isn't too bad.. Are
> there uses beyond class inherit though?
>
> machine/distro were added as they are clear variables in common use
> which avoided tons of boilerplate in layer definitions. Do we have
> similar needs here or are these inherits better captured in config
> fragments?
Antonin first wanted to just add 'real' fragments (that are defined in
files) for common class inherits, and it was my idea to implement
additive built-ins that make it generic. It's something we can
consider and discuss, I don't necessarily insist on a generic
implementation.
I guess we can all look at our various local.conf lying around, and
check what's piled up in them over the years. In mine, two variables
immediately jump out as candidates for additions: IMAGE_INSTALL,
IMAGE_FEATURES. Those certainly will not work with regular fragments.
Alex
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [bitbake-devel] [PATCH RFC] parse/ast: add support for additive built-in fragments
2026-09-01 18:28 ` Alexander Kanavin
@ 2026-09-01 19:22 ` Richard Purdie
2026-09-02 7:56 ` Antonin Godard
0 siblings, 1 reply; 5+ messages in thread
From: Richard Purdie @ 2026-09-01 19:22 UTC (permalink / raw)
To: Alexander Kanavin; +Cc: antonin.godard, bitbake-devel, Thomas Petazzoni
On Tue, 2026-09-01 at 20:28 +0200, Alexander Kanavin wrote:
> On Tue, 1 Sept 2026 at 14:15, Richard Purdie via
> lists.openembedded.org
> <richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote:
>
> > It is definitely an interesting one and the syntax isn't too bad..
> > Are
> > there uses beyond class inherit though?
> >
> > machine/distro were added as they are clear variables in common use
> > which avoided tons of boilerplate in layer definitions. Do we have
> > similar needs here or are these inherits better captured in config
> > fragments?
>
> Antonin first wanted to just add 'real' fragments (that are defined
> in
> files) for common class inherits, and it was my idea to implement
> additive built-ins that make it generic. It's something we can
> consider and discuss, I don't necessarily insist on a generic
> implementation.
>
> I guess we can all look at our various local.conf lying around, and
> check what's piled up in them over the years. In mine, two variables
> immediately jump out as candidates for additions: IMAGE_INSTALL,
> IMAGE_FEATURES. Those certainly will not work with regular fragments.
I guess there are a few things to keep in mind.
Firstly, we're never going to replace local.conf, there are always
going to be some things which users need to do locally for their work
and these do belong there.
The alternative is a fragment language which is an abbreviated short
form of the main bitbake syntax and I'm not sure that is feasible or
desirable.
The things which should become fragments are commonly used groups of
settings, things which set set as a group.
The machine/distro shortcuts are nice as they cover the biggest control
everyone sets immediately but beyond that, is there a pressing case?
I guess some of the "classes" are in fact policy "selections" and kind
of like fragments in their own right but are there enough of them to
justify specific syntax.
Once you get above a handful of image features or image install
packages, that is a packagegroup or a config fragment in their own
right and I'm worried about the abuse such things might get...
I'm not saying "no" but I do have worries.
Cheers,
Richard
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [bitbake-devel] [PATCH RFC] parse/ast: add support for additive built-in fragments
2026-09-01 19:22 ` Richard Purdie
@ 2026-09-02 7:56 ` Antonin Godard
0 siblings, 0 replies; 5+ messages in thread
From: Antonin Godard @ 2026-09-02 7:56 UTC (permalink / raw)
To: Richard Purdie, Alexander Kanavin; +Cc: bitbake-devel, Thomas Petazzoni
Hi,
On Tue Sep 1, 2026 at 9:22 PM CEST, Richard Purdie wrote:
> On Tue, 2026-09-01 at 20:28 +0200, Alexander Kanavin wrote:
>> On Tue, 1 Sept 2026 at 14:15, Richard Purdie via
>> lists.openembedded.org
>> <richard.purdie=linuxfoundation.org@lists.openembedded.org> wrote:
>>
>> > It is definitely an interesting one and the syntax isn't too bad..
>> > Are
>> > there uses beyond class inherit though?
>> >
>> > machine/distro were added as they are clear variables in common use
>> > which avoided tons of boilerplate in layer definitions. Do we have
>> > similar needs here or are these inherits better captured in config
>> > fragments?
>>
>> Antonin first wanted to just add 'real' fragments (that are defined
>> in
>> files) for common class inherits, and it was my idea to implement
>> additive built-ins that make it generic. It's something we can
>> consider and discuss, I don't necessarily insist on a generic
>> implementation.
>>
>> I guess we can all look at our various local.conf lying around, and
>> check what's piled up in them over the years. In mine, two variables
>> immediately jump out as candidates for additions: IMAGE_INSTALL,
>> IMAGE_FEATURES. Those certainly will not work with regular fragments.
>
> I guess there are a few things to keep in mind.
>
> Firstly, we're never going to replace local.conf, there are always
> going to be some things which users need to do locally for their work
> and these do belong there.
>
> The alternative is a fragment language which is an abbreviated short
> form of the main bitbake syntax and I'm not sure that is feasible or
> desirable.
>
> The things which should become fragments are commonly used groups of
> settings, things which set set as a group.
>
> The machine/distro shortcuts are nice as they cover the biggest control
> everyone sets immediately but beyond that, is there a pressing case?
>
> I guess some of the "classes" are in fact policy "selections" and kind
> of like fragments in their own right but are there enough of them to
> justify specific syntax.
I think the main purpose of such a class fragment would be for classes that
provide build settings, such as the retain or buildstats classes, or rm_work,
etc. Those classes do not mean anything for the target image, they only tweak
what happens during the build. I think it makes sense to set those in
bitbake-setup configuration files.
However, I think classes that affect the build output, such as useradd, should
probably be part of a distro configuration file rather than bitbake-setup files.
With that in mind, I agree that there aren't that many use-cases for a class
fragment, and maybe we should provide everything from our distro configuration
files. But aren't distro conf files supposed to represent policy for the
target image, and only that? Would it be relevant to delegate all build system
settings to bitbake-setup? I guess this patch would somewhat aim to achieve this
separation.
> Once you get above a handful of image features or image install
> packages, that is a packagegroup or a config fragment in their own
> right and I'm worried about the abuse such things might get...
>
> I'm not saying "no" but I do have worries.
I agree that this mechanism could be abused. If this exists, maybe we should
limit this _only_ to adding/removing global classes?
Antonin
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-09-02 7:56 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-01 7:52 [PATCH RFC] parse/ast: add support for additive built-in fragments Antonin Godard
2026-09-01 12:15 ` [bitbake-devel] " Richard Purdie
2026-09-01 18:28 ` Alexander Kanavin
2026-09-01 19:22 ` Richard Purdie
2026-09-02 7:56 ` Antonin Godard
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.