pedantry: what is the proper descriptor for :append, :prepend and :remove

pedantry: what is the proper descriptor for :append, :prepend and :remove

[Robert P. J. Day]

  I ask as I have seen documentation that describes those three things
as "overrides", apparently because they appear in the same place as
variable overrides, but that seems wildly misleading. It seems that
they should more properly be classified as "operators", and if that
was the case, I think some documentation could be made clearer by
distinguishing between "operators" and "overrides" when you see stuff
like:

  VAR:append:arm

that sort of thing. So what's the proper descriptor for those three
things?

rday

Re: [docs] pedantry: what is the proper descriptor for :append, :prepend and :remove

[Khem Raj]

On Mon, Jul 14, 2025 at 9:59 AM Robert P. J. Day via
lists.yoctoproject.org <rpjday=crashcourse.ca@lists.yoctoproject.org>
wrote:
>
>
>   I ask as I have seen documentation that describes those three things
> as "overrides", apparently because they appear in the same place as
> variable overrides, but that seems wildly misleading. It seems that
> they should more properly be classified as "operators", and if that
> was the case, I think some documentation could be made clearer by
> distinguishing between "operators" and "overrides" when you see stuff
> like:
>
>   VAR:append:arm
>
> that sort of thing. So what's the proper descriptor for those three
> things?

variable:operator:override

>
> rday
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#7327): https://lists.yoctoproject.org/g/docs/message/7327
> Mute This Topic: https://lists.yoctoproject.org/mt/114151467/1997914
> Group Owner: docs+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [docs] pedantry: what is the proper descriptor for :append, :prepend and :remove

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 1579 bytes --]

On Mon, 14 Jul 2025, Khem Raj wrote:

> On Mon, Jul 14, 2025 at 9:59 AM Robert P. J. Day via
> lists.yoctoproject.org <rpjday=crashcourse.ca@lists.yoctoproject.org>
> wrote:
> >
> >
> >   I ask as I have seen documentation that describes those three things
> > as "overrides", apparently because they appear in the same place as
> > variable overrides, but that seems wildly misleading. It seems that
> > they should more properly be classified as "operators", and if that
> > was the case, I think some documentation could be made clearer by
> > distinguishing between "operators" and "overrides" when you see stuff
> > like:
> >
> >   VAR:append:arm
> >
> > that sort of thing. So what's the proper descriptor for those three
> > things?
>
> variable:operator:override

  actually, i was asking about the proper descriptor for the three
things "append", "prepend" and "remove", but this is making my point
-- those three things are properly described as "operators", as
opposed to "overrides", so one might say that the correct syntax
would be:

  VAR:operator:override(s) = ...

in that order since, as quentin suggests:

  VAR:override:append

is almost always wrong. i remember asking about this distinction a
long time ago, and that distinction is covered in the BB manual:

https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#examples

but maybe that needs to be extended with this caveat -- that an
"operator" should precede any overrides in almost all cases, as i see
no actual valid use cases for that second, weird form.

rday

Re: [docs] pedantry: what is the proper descriptor for :append, :prepend and :remove

[Quentin Schulz]

Hi Robert,

On 7/15/25 11:27 AM, Robert P. J. Day via lists.yoctoproject.org wrote:
> On Mon, 14 Jul 2025, Khem Raj wrote:
> 
>> On Mon, Jul 14, 2025 at 9:59 AM Robert P. J. Day via
>> lists.yoctoproject.org <rpjday=crashcourse.ca@lists.yoctoproject.org>
>> wrote:
>>>
>>>
>>>    I ask as I have seen documentation that describes those three things
>>> as "overrides", apparently because they appear in the same place as
>>> variable overrides, but that seems wildly misleading. It seems that
>>> they should more properly be classified as "operators", and if that
>>> was the case, I think some documentation could be made clearer by
>>> distinguishing between "operators" and "overrides" when you see stuff
>>> like:
>>>
>>>    VAR:append:arm
>>>
>>> that sort of thing. So what's the proper descriptor for those three
>>> things?
>>
>> variable:operator:override
> 
>    actually, i was asking about the proper descriptor for the three
> things "append", "prepend" and "remove", but this is making my point
> -- those three things are properly described as "operators", as
> opposed to "overrides", so one might say that the correct syntax

Both are correct.

> would be:
> 
>    VAR:operator:override(s) = ...
> 
> in that order since, as quentin suggests:
> 
>    VAR:override:append
> 
> is almost always wrong. i remember asking about this distinction a

Almost always wrong but still might be something you want, so we need to 
explain it. By explaining the difference between both, people would get 
a better understanding as to why it is most likely wrong for their 
usecase and maybe even have them go through their codebase to check if 
what they think works actually does work.

A typical use-case where this makes sense is the following:

do_install() {
}

do_install:class-target() {
}

If you want to add to the class-target version of the task only,

do_install:append:class-target() {
}

will NOT do what you want, but

do_install:class-target:append() {
}

will.

The former will add its content to "vanilla"/"default" do_install 
whenever building for the target, but the actual do_install will 
actually be of the content of do_install:class-target, so you need to 
append to that.

> long time ago, and that distinction is covered in the BB manual:
> 
> https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#examples
> 
> but maybe that needs to be extended with this caveat -- that an
> "operator" should precede any overrides in almost all cases, as i see
> no actual valid use cases for that second, weird form.
> 

I couldn't really find any actual user in poky, c.f.:

$ git grep -w -E '.*:[-a-zA-Z0-9_]*:(append|prepend|remove)'
bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst: 
A:foo:append = "X"
bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst: 
A:foo:append = "Z"
bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst: 
A:foo:append = "X"
bitbake/lib/bb/tests/data.py: 
self.d.setVar("TEST:some_val:remove", "testvalue3")
bitbake/lib/bb/tests/data.py:        self.d.setVar("TEST:bar:append", 
"testvalue2")
documentation/migration-guides/release-notes-4.3.rst: 
ERROR_QA:layer-core:append = " patch-status"
meta/conf/distro/include/no-gplv3.inc:RDEPENDS:packagegroup-core-full-cmdline-utils:remove 
= "bash bc coreutils cpio ed findutils gawk grep mc mc-shell mc-helpers 
mc-helpers-perl sed tar time"
meta/conf/distro/include/no-gplv3.inc:RDEPENDS:packagegroup-core-full-cmdline-dev-utils:remove 
= "diffutils m4 make patch"
meta/conf/distro/include/no-gplv3.inc:RDEPENDS:packagegroup-core-full-cmdline-multiuser:remove 
= "gzip"
meta/conf/distro/include/no-gplv3.inc:RRECOMMENDS:packagegroup-base-vfat:remove 
= "dosfstools"
meta/lib/oeqa/selftest/cases/bbtests.py: 
self.write_config('PV:pn-gitunpackoffline:append = "+${UNDEFVAL}"')
meta/lib/oeqa/selftest/cases/uboot.py:IMAGE_FSTYPES:pn-core-image-minimal:append 
= " wic"
meta/lib/oeqa/selftest/cases/uki.py:IMAGE_FSTYPES:pn-core-image-base:append 
= " wic"
meta/lib/oeqa/selftest/cases/wic.py:IMAGE_FSTYPES:pn-core-image-base:append 
= " wic"
meta/lib/oeqa/selftest/cases/wic.py:IMAGE_FSTYPES:pn-core-image-base:append 
= " wic"
meta/lib/oeqa/selftest/cases/wic.py:IMAGE_FSTYPES:pn-core-image-base:append 
= " wic"
meta/recipes-bsp/gnu-efi/gnu-efi_4.0.1.bb:do_configure:linux-gnux32:prepend() 
{
meta/recipes-connectivity/avahi/avahi_0.8.bb:RRECOMMENDS:avahi-daemon:append:libc-glibc 
= " avahi-libnss-mdns"
meta/recipes-core/images/build-appliance-image_15.0.0.bb:IMAGE_CMD:ext4:append 
() {
meta/recipes-core/packagegroups/packagegroup-base.bb:RDEPENDS:packagegroup-base-zeroconf:append:libc-glibc 
= "\
meta/recipes-core/packagegroups/packagegroup-self-hosted.bb:RDEPENDS:packagegroup-self-hosted-sdk:append:mingw32 
= "\
meta/recipes-core/packagegroups/packagegroup-self-hosted.bb:RDEPENDS:packagegroup-self-hosted-sdk:append:libc-glibc 
= "\
meta/recipes-devtools/clang/clang/0009-clang-Look-inside-the-target-sysroot-for-compiler-ru.patch:+ 
  llvm::sys::path::append(Path, "/usr/", ClangLibdirBasename, "clang",
meta/recipes-devtools/clang/clang/0009-clang-Look-inside-the-target-sysroot-for-compiler-ru.patch: 
     llvm::sys::path::append(Path, "lib", getOSLibName());
meta/recipes-devtools/clang/clang/0010-clang-Define-releative-gcc-installation-dir.patch: 
   llvm::sys::path::append(DriverIncludeDir, "..", "include");
meta/recipes-devtools/clang/clang/0015-clang-Fix-resource-dir-location-for-cross-toolchains.patch: 
     llvm::sys::path::append(P, CLANG_INSTALL_LIBDIR_BASENAME, "clang",
meta/recipes-devtools/gdb/gdb-common.inc:RRECOMMENDS:gdb:append:linux = 
" glibc-thread-db "
meta/recipes-devtools/gdb/gdb-common.inc:RRECOMMENDS:gdb:append:linux-gnueabi 
= " glibc-thread-db "
meta/recipes-devtools/gdb/gdb-common.inc:RRECOMMENDS:gdbserver:append:linux 
= " glibc-thread-db "
meta/recipes-devtools/gdb/gdb-common.inc:RRECOMMENDS:gdbserver:append:linux-gnueabi 
= " glibc-thread-db "
meta/recipes-devtools/python/python3_3.13.5.bb:RDEPENDS:libpython3:append:libc-glibc 
= " libgcc"
meta/recipes-extended/baremetal-example/baremetal-helloworld_git.bb:DEPENDS:qemux86:append 
= " nasm-native"


IMAGE_CMD:ext4 isn't a "real" override per-se, same as for the 
RRECOMMENDS, RDEPENDS.

The latter may be incorrect though as DEPENDS:qemux86:append would 
override anything in "vanilla"/"default" DEPENDS by using 
DEPENDS:qemux86 instead (which may not exist but will be created by 
DEPENDS:qemux86:append eventually) and then append nasm-native to it.

I assume it's likely we need

DEPENDS:qemux86 = "nasm-native"

if that's the only required dependency for qemux86

or

DEPENDS:append:qemux86 = " nasm-native"

if we want to add the nasm-native dependency only for qemux86.

IMAGE_FSTYPES:pn-core-image-minimal:append = " wic"

might be wrong as well as I would expect 
IMAGE_FSTYPES:pn-core-image-minimal to never be set, so this essentially 
just replaces whatever IMAGE_FSTYPES there exists when building 
core-image-minimal. I believe :append:pn-core-image-minimal may be more 
appropriate there.

Similarly, I'm assuming do_configure:linux-gnux32:prepend() in 
meta/recipes-bsp/gnu-efi/gnu-efi_4.0.1.bb should either be

do_configure:linux-gnux32()

to entirely replace do_configure (as it is what it's doing today), or

do_configure:prepend:linux-gnux32()

to add this content before do_configure when building for linux-gnux32

Cheers,
Quentin

Re: [docs] pedantry: what is the proper descriptor for :append, :prepend and :remove

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 7938 bytes --]


  I'll go over all this when I get a moment, but I want to thank
Quentin for the time he invests in educating (and generally
correcting) me.

On Tue, 15 Jul 2025, Quentin Schulz wrote:

> Hi Robert,
>
> On 7/15/25 11:27 AM, Robert P. J. Day via lists.yoctoproject.org wrote:
> > On Mon, 14 Jul 2025, Khem Raj wrote:
> >
> > > On Mon, Jul 14, 2025 at 9:59 AM Robert P. J. Day via
> > > lists.yoctoproject.org <rpjday=crashcourse.ca@lists.yoctoproject.org>
> > > wrote:
> > > >
> > > >
> > > >    I ask as I have seen documentation that describes those three things
> > > > as "overrides", apparently because they appear in the same place as
> > > > variable overrides, but that seems wildly misleading. It seems that
> > > > they should more properly be classified as "operators", and if that
> > > > was the case, I think some documentation could be made clearer by
> > > > distinguishing between "operators" and "overrides" when you see stuff
> > > > like:
> > > >
> > > >    VAR:append:arm
> > > >
> > > > that sort of thing. So what's the proper descriptor for those three
> > > > things?
> > >
> > > variable:operator:override
> >
> >    actually, i was asking about the proper descriptor for the three
> > things "append", "prepend" and "remove", but this is making my point
> > -- those three things are properly described as "operators", as
> > opposed to "overrides", so one might say that the correct syntax
>
> Both are correct.
>
> > would be:
> >
> >    VAR:operator:override(s) = ...
> >
> > in that order since, as quentin suggests:
> >
> >    VAR:override:append
> >
> > is almost always wrong. i remember asking about this distinction a
>
> Almost always wrong but still might be something you want, so we need to
> explain it. By explaining the difference between both, people would get a
> better understanding as to why it is most likely wrong for their usecase and
> maybe even have them go through their codebase to check if what they think
> works actually does work.
>
> A typical use-case where this makes sense is the following:
>
> do_install() {
> }
>
> do_install:class-target() {
> }
>
> If you want to add to the class-target version of the task only,
>
> do_install:append:class-target() {
> }
>
> will NOT do what you want, but
>
> do_install:class-target:append() {
> }
>
> will.
>
> The former will add its content to "vanilla"/"default" do_install whenever
> building for the target, but the actual do_install will actually be of the
> content of do_install:class-target, so you need to append to that.
>
> > long time ago, and that distinction is covered in the BB manual:
> >
> > https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#examples
> >
> > but maybe that needs to be extended with this caveat -- that an
> > "operator" should precede any overrides in almost all cases, as i see
> > no actual valid use cases for that second, weird form.
> >
>
> I couldn't really find any actual user in poky, c.f.:
>
> $ git grep -w -E '.*:[-a-zA-Z0-9_]*:(append|prepend|remove)'
> bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst: A:foo:append
> = "X"
> bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst: A:foo:append
> = "Z"
> bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst: A:foo:append
> = "X"
> bitbake/lib/bb/tests/data.py: self.d.setVar("TEST:some_val:remove",
> "testvalue3")
> bitbake/lib/bb/tests/data.py:        self.d.setVar("TEST:bar:append",
> "testvalue2")
> documentation/migration-guides/release-notes-4.3.rst:
> ERROR_QA:layer-core:append = " patch-status"
> meta/conf/distro/include/no-gplv3.inc:RDEPENDS:packagegroup-core-full-cmdline-utils:remove
> = "bash bc coreutils cpio ed findutils gawk grep mc mc-shell mc-helpers
> mc-helpers-perl sed tar time"
> meta/conf/distro/include/no-gplv3.inc:RDEPENDS:packagegroup-core-full-cmdline-dev-utils:remove
> = "diffutils m4 make patch"
> meta/conf/distro/include/no-gplv3.inc:RDEPENDS:packagegroup-core-full-cmdline-multiuser:remove
> = "gzip"
> meta/conf/distro/include/no-gplv3.inc:RRECOMMENDS:packagegroup-base-vfat:remove
> = "dosfstools"
> meta/lib/oeqa/selftest/cases/bbtests.py:
> self.write_config('PV:pn-gitunpackoffline:append = "+${UNDEFVAL}"')
> meta/lib/oeqa/selftest/cases/uboot.py:IMAGE_FSTYPES:pn-core-image-minimal:append
> = " wic"
> meta/lib/oeqa/selftest/cases/uki.py:IMAGE_FSTYPES:pn-core-image-base:append =
> " wic"
> meta/lib/oeqa/selftest/cases/wic.py:IMAGE_FSTYPES:pn-core-image-base:append =
> " wic"
> meta/lib/oeqa/selftest/cases/wic.py:IMAGE_FSTYPES:pn-core-image-base:append =
> " wic"
> meta/lib/oeqa/selftest/cases/wic.py:IMAGE_FSTYPES:pn-core-image-base:append =
> " wic"
> meta/recipes-bsp/gnu-efi/gnu-efi_4.0.1.bb:do_configure:linux-gnux32:prepend()
> {
> meta/recipes-connectivity/avahi/avahi_0.8.bb:RRECOMMENDS:avahi-daemon:append:libc-glibc
> = " avahi-libnss-mdns"
> meta/recipes-core/images/build-appliance-image_15.0.0.bb:IMAGE_CMD:ext4:append
> () {
> meta/recipes-core/packagegroups/packagegroup-base.bb:RDEPENDS:packagegroup-base-zeroconf:append:libc-glibc
> = "\
> meta/recipes-core/packagegroups/packagegroup-self-hosted.bb:RDEPENDS:packagegroup-self-hosted-sdk:append:mingw32
> = "\
> meta/recipes-core/packagegroups/packagegroup-self-hosted.bb:RDEPENDS:packagegroup-self-hosted-sdk:append:libc-glibc
> = "\
> meta/recipes-devtools/clang/clang/0009-clang-Look-inside-the-target-sysroot-for-compiler-ru.patch:+
> llvm::sys::path::append(Path, "/usr/", ClangLibdirBasename, "clang",
> meta/recipes-devtools/clang/clang/0009-clang-Look-inside-the-target-sysroot-for-compiler-ru.patch:
> llvm::sys::path::append(Path, "lib", getOSLibName());
> meta/recipes-devtools/clang/clang/0010-clang-Define-releative-gcc-installation-dir.patch:
> llvm::sys::path::append(DriverIncludeDir, "..", "include");
> meta/recipes-devtools/clang/clang/0015-clang-Fix-resource-dir-location-for-cross-toolchains.patch:
> llvm::sys::path::append(P, CLANG_INSTALL_LIBDIR_BASENAME, "clang",
> meta/recipes-devtools/gdb/gdb-common.inc:RRECOMMENDS:gdb:append:linux = "
> glibc-thread-db "
> meta/recipes-devtools/gdb/gdb-common.inc:RRECOMMENDS:gdb:append:linux-gnueabi
> = " glibc-thread-db "
> meta/recipes-devtools/gdb/gdb-common.inc:RRECOMMENDS:gdbserver:append:linux =
> " glibc-thread-db "
> meta/recipes-devtools/gdb/gdb-common.inc:RRECOMMENDS:gdbserver:append:linux-gnueabi
> = " glibc-thread-db "
> meta/recipes-devtools/python/python3_3.13.5.bb:RDEPENDS:libpython3:append:libc-glibc
> = " libgcc"
> meta/recipes-extended/baremetal-example/baremetal-helloworld_git.bb:DEPENDS:qemux86:append
> = " nasm-native"
>
>
> IMAGE_CMD:ext4 isn't a "real" override per-se, same as for the RRECOMMENDS,
> RDEPENDS.
>
> The latter may be incorrect though as DEPENDS:qemux86:append would override
> anything in "vanilla"/"default" DEPENDS by using DEPENDS:qemux86 instead
> (which may not exist but will be created by DEPENDS:qemux86:append eventually)
> and then append nasm-native to it.
>
> I assume it's likely we need
>
> DEPENDS:qemux86 = "nasm-native"
>
> if that's the only required dependency for qemux86
>
> or
>
> DEPENDS:append:qemux86 = " nasm-native"
>
> if we want to add the nasm-native dependency only for qemux86.
>
> IMAGE_FSTYPES:pn-core-image-minimal:append = " wic"
>
> might be wrong as well as I would expect IMAGE_FSTYPES:pn-core-image-minimal
> to never be set, so this essentially just replaces whatever IMAGE_FSTYPES
> there exists when building core-image-minimal. I believe
> :append:pn-core-image-minimal may be more appropriate there.
>
> Similarly, I'm assuming do_configure:linux-gnux32:prepend() in
> meta/recipes-bsp/gnu-efi/gnu-efi_4.0.1.bb should either be
>
> do_configure:linux-gnux32()
>
> to entirely replace do_configure (as it is what it's doing today), or
>
> do_configure:prepend:linux-gnux32()
>
> to add this content before do_configure when building for linux-gnux32
>
> Cheers,
> Quentin
>