[PATCH 0/2] Add documentation on fragments

[PATCH 0/2] Add documentation on fragments

[Antonin Godard]

First patch introduces doc for auto.conf, and the second one adds docs
on fragments.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
Antonin Godard (2):
      ref-manual/structure: document the auto.conf file
      Add documentation on fragments

 documentation/dev-manual/index.rst           |   1 +
 documentation/dev-manual/using-fragments.rst | 101 +++++++++++
 documentation/ref-manual/fragments.rst       | 249 +++++++++++++++++++++++++++
 documentation/ref-manual/index.rst           |   1 +
 documentation/ref-manual/structure.rst       |   9 +
 documentation/ref-manual/terms.rst           |  73 ++++++++
 documentation/ref-manual/variables.rst       |  32 ++++
 7 files changed, 466 insertions(+)
---
base-commit: e77139b5c616e6e5ad436eb91416fd804389425f
change-id: 20250924-fragments-bb3da11e23aa

Best regards,
--  
Antonin Godard <antonin.godard@bootlin.com>

[PATCH 1/2] ref-manual/structure: document the auto.conf file

[Antonin Godard]

Add documentation for auto.conf, which is used by external tools for
automatically setting variables.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/ref-manual/structure.rst | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/documentation/ref-manual/structure.rst b/documentation/ref-manual/structure.rst
index c9c44bc63..ef6523ed8 100644
--- a/documentation/ref-manual/structure.rst
+++ b/documentation/ref-manual/structure.rst
@@ -345,6 +345,15 @@ This configuration file is generated by :doc:`bblock </dev-manual/bblock>` and
 contains the signatures locked by ``bblock``. By default, it does not exist
 and will be created upon the first invocation of ``bblock``.
 
+.. _structure-build-conf-auto.conf:
+
+``build/conf/auto.conf``
+------------------------
+
+This file contains configuration variables that are automatically modified by
+tools such as :oe_git:`bitbake-config-build </bitbake/tree/bin/bitbake-config-build>`.
+This file should not be modified manually.
+
 .. _structure-build-downloads:
 
 ``build/downloads/``

-- 
2.51.0

[PATCH 2/2] Add documentation on fragments

[Antonin Godard]

- terms.rst: Provide the definitions of a Configuration Fragment and a
  Built-in Fragment.
- ref-manual: Add a quick reference guide on bitbake-config-build, and
  list the available fragments in OE-Core.
  Document the underlying variables related to fragments in the
  glossary.
- dev-manual: give instructions on how to create new custom fragments.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/dev-manual/index.rst           |   1 +
 documentation/dev-manual/using-fragments.rst | 101 +++++++++++
 documentation/ref-manual/fragments.rst       | 249 +++++++++++++++++++++++++++
 documentation/ref-manual/index.rst           |   1 +
 documentation/ref-manual/terms.rst           |  73 ++++++++
 documentation/ref-manual/variables.rst       |  32 ++++
 6 files changed, 457 insertions(+)

diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst
index ea528855a..7733d4554 100644
--- a/documentation/dev-manual/index.rst
+++ b/documentation/dev-manual/index.rst
@@ -11,6 +11,7 @@ Yocto Project Development Tasks Manual
    intro
    start
    layers
+   using-fragments
    customizing-images
    new-recipe
    new-machine
diff --git a/documentation/dev-manual/using-fragments.rst b/documentation/dev-manual/using-fragments.rst
new file mode 100644
index 000000000..d2de0edd0
--- /dev/null
+++ b/documentation/dev-manual/using-fragments.rst
@@ -0,0 +1,101 @@
+.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
+
+Using Configuration Fragments In Your Build
+*******************************************
+
+The Yocto Project supports enabling and disable some configuration variables
+through :term:`Configuration Fragments <Configuration Fragment>`. This mechanism
+allows modifying the top-level configuration of a build from the command-line
+using the :oe_git:`bitbake-config-build </bitbake/tree/bin/bitbake-config-build>`
+tool. This section will describe how to create new fragments for your builds.
+
+There are two kinds of configuration fragments:
+
+-  :term:`Configuration Fragments <Configuration Fragment>` which a stored
+   in a file. These fragment include a summary and a description.
+
+-  :term:`Built-in Fragments <Built-in Fragment>` which are used to set a
+   variable value.
+
+Creating new :term:`Built-in Fragments <Built-in Fragment>` is not supported.
+You should instead create :term:`Configuration Fragment` files as documented
+below to customize a build to your need.
+
+Creating A Configuration Fragment File
+======================================
+
+By default, all configuration fragments are located within the
+``conf/fragments`` directory of a :term:`layer`. This location is defined by the
+:term:`OE_FRAGMENTS_PREFIX` variable.
+
+You can create one or more :term:`configuration fragment` files in your
+:term:`layer` in this directory. Let's take the following example, where
+``custom-fragment.conf`` is our custom fragment file::
+
+   meta-custom
+   ├── conf
+   │   ├── fragments
+   │   │   └── custom-fragment.conf
+   │   └── layer.conf
+   ...
+
+For our ``custom-fragment.conf`` file, the following variables **must** be set
+for our fragment to be considered a valid fragment by the :term:`OpenEmbedded
+Build System`:
+
+-  :term:`BB_CONF_FRAGMENT_SUMMARY`: a one-line summary of this fragment.
+
+-  :term:`BB_CONF_FRAGMENT_DESCRIPTION`: a description of this fragment.
+
+After creating these variables, our custom fragment should look like the
+following:
+
+.. code-block::
+   :caption: custom-fragment.conf
+
+   BB_CONF_FRAGMENT_SUMMARY = "My custom fragment summary"
+   BB_CONF_FRAGMENT_DESCRIPTION = "My custom fragment description, \
+   which can be multiple lines."
+
+For now, our fragment does not set any additional variable. Let's add the
+following assignment to our fragment:
+
+.. code-block::
+   :caption: custom-fragment.conf (continued)
+
+   BB_NUMBER_THREADS = "4"
+   BB_NUMBER_PARSE_THREADS = "4"
+
+This means that our fragment can be enabled to set a limit on the number of
+threads :term:`BitBake` will use with the :term:`BB_NUMBER_THREADS` and
+:term:`BB_NUMBER_PARSE_THREADS` variables.
+
+For now, our fragment exists and is listed by the
+:ref:`ref-bitbake-config-build-list-fragments` command, but is not enabled. To
+make this fragment part of the build, use the
+:ref:`ref-bitbake-config-build-enable-fragment` command::
+
+   bitbake-config-build enable-fragment meta-custom/custom-fragment
+
+.. note::
+
+   The ``meta-custom`` prefix in the above command depends on the name of your
+   layer. This name is defined by the :term:`BBFILE_COLLECTIONS` variable in
+   the ``conf/layer.conf`` file of your layer.
+
+Configuration fragment files can be organizes in a more complex way. For
+example, it's possible to create sub-directories to organize your fragments::
+
+   meta-custom
+   ├── conf
+   │   ├── fragments
+   │   │   ├── networking
+   │   │   │   └── mirrors.conf
+   │   │   └── resources
+   │   │       └── numberthreads.conf
+   │   └── layer.conf
+   ...
+
+In the above example, the ``meta-custom/networking/mirrors`` and
+``meta-custom/resources/numberthreads`` fragments will be available in your
+build.
diff --git a/documentation/ref-manual/fragments.rst b/documentation/ref-manual/fragments.rst
new file mode 100644
index 000000000..567b02574
--- /dev/null
+++ b/documentation/ref-manual/fragments.rst
@@ -0,0 +1,249 @@
+.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
+
+***********************
+Configuration Fragments
+***********************
+
+:term:`Configuration Fragments <Configuration Fragment>` are used to provide a
+snippet of global configuration that can be enabled or disabled for a build.
+This document provides a quick reference of the :oe_git:`bitbake-config-build
+</bitbake/tree/bin/bitbake-config-build>` tool and lists the
+:term:`Configuration Fragments <Configuration Fragment>` and :term:`Built-in
+Fragments <Built-in Fragment>` available in the :term:`OpenEmbedded Build
+System` core repositories.
+
+.. note::
+
+   For details on how to define new fragments in your build, see the
+   :doc:`/dev-manual/using-fragments` section of the Yocto Project Development
+   Tasks Manual.
+
+``bitbake-config-build`` Quick Reference
+========================================
+
+:term:`Configuration Fragments <Configuration Fragment>` are managed with the
+:oe_git:`bitbake-config-build </bitbake/tree/bin/bitbake-config-build>`
+command-line tool, which is available after :ref:`dev-manual/start:Initializing
+the Build Environment`.
+
+The ``bitbake-config-build`` command-line tool uses sub-commands to manage
+fragments, which are detailed in the sections below. For each sub-command, the
+``--help`` flag can be passed to get more information on the sub-command.
+
+.. _ref-bitbake-config-build-list-fragments:
+
+``bitbake-config-build list-fragments``
+---------------------------------------
+
+The :ref:`ref-bitbake-config-build-list-fragments` will list the :term:`Built-in
+Fragments <Built-in Fragment>` and :term:`Configuration Fragments <Configuration
+Fragment>` that are currently available, and will also print which fragments are
+enabled or disabled.
+
+.. _ref-bitbake-config-build-show-fragment:
+
+``bitbake-config-build show-fragment``
+--------------------------------------
+
+The :ref:`ref-bitbake-config-build-show-fragment` is used to show the
+location and value of a fragment. For example, running ``bitbake-config-build
+show-fragment core/yocto/sstate-mirror-cdn`` will show the content of the
+:ref:`ref-fragments-core-yocto-sstate-mirror-cdn` fragment.
+
+.. _ref-bitbake-config-build-enable-fragment:
+
+``bitbake-config-build enable-fragment``
+----------------------------------------
+
+The :ref:`ref-bitbake-config-build-enable-fragment` is used to enable a
+fragment. When a fragment is enabled, the configuration variables of this
+fragment are parsed by :term:`BitBake` and their values are available globally
+in your build.
+
+From the list obtained with the :ref:`ref-bitbake-config-build-list-fragments`
+sub-command, you can determine which fragments can be enabled for your build
+environment.
+
+For example, the following command would enable the
+``core/yocto/sstate-mirror-cdn`` fragment::
+
+   bitbake-config-build enable-fragment core/yocto/sstate-mirror-cdn
+
+.. note::
+
+   Multiple fragments can be enabled at the same time in the same command::
+
+      bitbake-config-build enable-fragment <fragment1> <fragment2> ...
+
+:term:`Built-in fragments <Built-in Fragment>` are enabled the same way, and
+their values are defined from the command-line directly. For example, the
+following command sets the ``qemuarm64`` :term:`MACHINE` through the
+:ref:`ref-fragment-builtin-core-machine` fragment::
+
+   bitbake-config-build enable-fragment machine/qemuarm64
+
+This fragment can be overridden from the command-line be setting it to another
+value, for example::
+
+   bitbake-config-build enable-fragment machine/qemux86-64
+
+Note that in this case, the fragment will be defined twice in
+:term:`OE_FRAGMENTS`, and the last value is taken into account:
+
+.. code-block::
+   :caption: auto.conf
+
+   OE_FRAGMENTS = " ... machine/qemuarm64 machine/qemux86-64"
+
+In the above example, the value of :term:`MACHINE` is thus equal to
+``qemux86-64``.
+
+When a fragment is enabled with :ref:`ref-bitbake-config-build-enable-fragment`,
+its name is automatically appended to the :term:`OE_FRAGMENTS` variable in
+:ref:`structure-build-conf-auto.conf`.
+
+.. _ref-bitbake-config-build-disable-fragment:
+
+``bitbake-config-build disable-fragment``
+-----------------------------------------
+
+Any fragment enabled with the :ref:`ref-bitbake-config-build-enable-fragment`
+command can be disabled with the :ref:`ref-bitbake-config-build-disable-fragment`
+command. The list of enabled fragments can be obtained with
+:ref:`ref-bitbake-config-build-list-fragments`.
+
+For example, the following command disables the
+:ref:`ref-fragments-core-yocto-sstate-mirror-cdn` fragment::
+
+   bitbake-config-build disable-fragment core/yocto/sstate-mirror-cdn
+
+Likewise, :term:`built-in fragments <Built-in Fragment>` are disabled the
+same way. For example, this would remove the ``machine/qemuarm64`` fragment::
+
+   bitbake-config-build disable-fragment machine/qemuarm64
+
+.. note::
+
+   Multiple fragment can be disabled in the same command::
+
+      bitbake-config-build disable-fragment <fragment1> <fragment2>
+
+.. _ref-bitbake-config-build-disable-all-fragments:
+
+``bitbake-config-build disable-all-fragments``
+----------------------------------------------
+
+The :ref:`ref-bitbake-config-build-disable-all-fragments` disables all of the
+currently enabled fragments.  The list of enabled fragments can be obtained with
+:ref:`ref-bitbake-config-build-list-fragments`.
+
+This command is run without arguments::
+
+   bitbake-config-build disable-all-fragments
+
+Core Fragments
+==============
+
+Core Built-in Fragments
+-----------------------
+
+:term:`Built-in Fragments <Built-in Fragment>` are used to assign a single
+variable globally. The :term:`OpenEmbedded Build System` defines multiple
+built-in fragments that are detailed in this section.
+
+.. _ref-fragment-builtin-core-machine:
+
+``machine/``
+~~~~~~~~~~~~
+
+The ``machine/`` :term:`built-in fragment` can be used to assign the value of
+the :term:`MACHINE` variable globally.
+
+.. _ref-fragment-builtin-core-distro:
+
+``distro/``
+~~~~~~~~~~~
+
+The ``distro/`` :term:`built-in fragment` can be used to assign the value of
+the :term:`DISTRO` variable globally.
+
+Core Configuration Fragments
+----------------------------
+
+Yocto Project Fragments
+~~~~~~~~~~~~~~~~~~~~~~~
+
+This group defines fragment related to the Yocto Project infrastructure in
+general.
+
+.. _ref-fragments-core-yocto-sstate-mirror-cdn:
+
+``core/yocto/sstate-mirror-cdn``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``core/yocto/sstate-mirror-cdn`` :term:`configuration fragment` can be used
+to set up :term:`BB_HASHSERVE_UPSTREAM` and :term:`SSTATE_MIRRORS` to use
+pre-built :ref:`shared state cache <overview-manual/concepts:shared state
+cache>` artifacts for standard Yocto build configurations.
+
+This will mean the build will query the Yocto Project mirrors to check for
+artifacts at the start of builds, which does slow it down initially but it will
+then speed up the builds by not having to build things if they are present in
+the cache. It assumes you can download something faster than you can build it
+which will depend on your network configuration.
+
+Yocto Project Autobuiler Fragments
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This group defines fragment used for the Yocto Project Autobuilder. For details,
+see the :ref:`test-manual/intro:Yocto Project Autobuilder Overview` section of
+the Yocto Project Test Environment Manual.
+
+.. _ref-fragment-core-yocto-autobuilder-autobuilder:
+
+``core/yocto-autobuilder/autobuilder``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``core/yocto-autobuilder/autobuilder`` fragment defines common variables
+used in builds started by the Yocto Project Autobuilder.
+
+.. _ref-fragment-core-yocto-autobuilder-autobuilder-resource-constraints:
+
+``core/yocto-autobuilder/autobuilder-resource-constraints``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``core/yocto-autobuilder/autobuilder`` fragment defines variables for
+limiting the resources used by the Yocto Project Autobuilder during builds. For
+more details on how to limit resources, see the :doc:`/dev-manual/limiting-resources`
+section of the Yocto Project Development Tasks Manual.
+
+.. _ref-fragment-core-yocto-autobuilder-multilib-mips64-n32:
+
+``core/yocto-autobuilder/multilib-mips64-n32``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``core/yocto-autobuilder/multilib-mips64-n32`` fragment enables
+tri-architecture :ref:`multilib <dev-manual/libraries:Combining Multiple
+Versions of Library Files into One Image>` configurations for :wikipedia:`MIPS64
+<MIPS_architecture>` machines, which includes ``mips64-n32``, ``mips64``, and
+``mips32r2``.
+
+.. _ref-fragment-core-yocto-autobuilder-multilib-x86-lib32:
+
+``core/yocto-autobuilder/multilib-x86-lib32``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``core/yocto-autobuilder/multilib-x86-lib32`` fragment enables
+:ref:`multilib <dev-manual/libraries:Combining Multiple Versions of Library
+Files into One Image>` configurations for supporting 32-bit libraries on 64-bit
+:wikipedia:`X86 <X86>` builds.
+
+.. _ref-fragment-core-yocto-autobuilder-multilib-x86-lib64:
+
+``core/yocto-autobuilder/multilib-x86-lib64``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``core/yocto-autobuilder/multilib-x86-lib64`` fragment enables
+:ref:`multilib <dev-manual/libraries:Combining Multiple Versions of Library
+Files into One Image>` configurations for supporting 64-bit libraries on 32-bit
+:wikipedia:`X86 <X86>` builds.
diff --git a/documentation/ref-manual/index.rst b/documentation/ref-manual/index.rst
index 53fa98cc9..aa1a63e05 100644
--- a/documentation/ref-manual/index.rst
+++ b/documentation/ref-manual/index.rst
@@ -17,6 +17,7 @@ Yocto Project Reference Manual
    structure
    classes
    tasks
+   fragments
    devtool-reference
    kickstart
    qa-checks
diff --git a/documentation/ref-manual/terms.rst b/documentation/ref-manual/terms.rst
index f3d3d059d..641b1ad7c 100644
--- a/documentation/ref-manual/terms.rst
+++ b/documentation/ref-manual/terms.rst
@@ -131,6 +131,48 @@ universal, the list includes them just in case:
       A variant of :term:`buildtools`, just providing the required
       version of ``make`` to run the OpenEmbedded build system.
 
+   :term:`Built-in Fragment`
+      A built-in fragment is a specific kind of :term:`Configuration Fragment`
+      that affects the value of a single variable globally. Like
+      a normal :term:`Configuration Fragment`, Built-in Fragments can be enabled
+      or disabled using the :oe_git:`bitbake-config-build </bitbake/tree/bin/bitbake-config-build>`
+      command-line utility.
+
+      When declared, a built-in fragment follows the following naming
+      convention::
+
+         <fragment>:<variable name>
+
+      Where:
+
+      -  ``<fragment>`` is the name of the built-in fragment.
+      -  ``<variable name>`` is the name of the variable to be modified by this
+         fragment.
+
+      For example::
+
+         machine:MACHINE
+
+      Will setup the ``machine`` Built-in Fragment for modifying the value of
+      the :term:`MACHINE` variable.
+
+      Setting the :term:`MACHINE` variable through this fragment must follow
+      this syntax::
+
+         machine/qemux86-64
+
+      This sets the value of :term:`MACHINE` to ``qemux86-64``.
+
+      For more details on fragments, see:
+
+      -  The :doc:`/ref-manual/fragments` section of the Yocto Project Reference
+         Manual for a list of fragments the :term:`OpenEmbedded Build System`
+         supports, and a quick reference guide on how to manage fragments.
+
+      -  The :doc:`/dev-manual/using-fragments` section of the Yocto Project
+         Development Tasks Manual for details on how to create new fragments
+         in your build.
+
    :term:`Classes`
       Files that provide for logic encapsulation and inheritance so that
       commonly used patterns can be defined once and then easily used in
@@ -154,6 +196,37 @@ universal, the list includes them just in case:
       only used when building for that target (e.g. the
       :file:`machine/beaglebone.conf` configuration file defines variables for
       the Texas Instruments ARM Cortex-A8 development board).
+      :term:`Configuration Fragments <Configuration Fragment>` such as
+      :ref:`ref-fragments-core-yocto-sstate-mirror-cdn` define snippets of
+      configuration that can be enabled from the command-line.
+
+   :term:`Configuration Fragment`
+      A configuration fragment is a :term:`Configuration File` that contains
+      variable assignments affecting the build at a global-level when the
+      fragment is enabled. By default, configuration fragments are located in
+      the :file:`conf/fragments/` directory of a :term:`Layer`.
+
+      A fragment :term:`configuration file` must contain a summary
+      (:term:`BB_CONF_FRAGMENT_SUMMARY`) and a description
+      (:term:`BB_CONF_FRAGMENT_DESCRIPTION`) explaining the purpose of the
+      fragment.
+
+      Another form of fragment are :term:`Built-in Fragments <Built-in Fragment>`
+      which are used to assign a single variable value globally.
+
+      Fragments can be listed, enabled and disabled with the
+      :oe_git:`bitbake-config-build </bitbake/tree/bin/bitbake-config-build>`
+      command-line utility.
+
+      For more details on fragments, see:
+
+      -  The :doc:`/ref-manual/fragments` section of the Yocto Project Reference
+         Manual for a list of fragments the :term:`OpenEmbedded Build System`
+         supports, and a quick reference guide on how to manage fragments.
+
+      -  The :doc:`/dev-manual/using-fragments` section of the Yocto Project
+         Development Tasks Manual for details on how to create new fragments
+         in your build.
 
    :term:`Container Layer`
       A flexible definition that typically refers to a single Git checkout
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 18ead7d04..0c62fdffe 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -397,6 +397,18 @@ system and gives an overview of their function and contents.
    :term:`BB_CHECK_SSL_CERTS`
       See :term:`bitbake:BB_CHECK_SSL_CERTS` in the BitBake manual.
 
+   :term:`BB_CONF_FRAGMENT_DESCRIPTION`
+      The :term:`BB_CONF_FRAGMENT_DESCRIPTION` variable defines the textual
+      description of a :term:`Configuration Fragment`. For details on how to use
+      fragments, see the :doc:`/ref-manual/fragments` section of the Yocto
+      Project Reference Manual.
+
+   :term:`BB_CONF_FRAGMENT_SUMMARY`
+      The :term:`BB_CONF_FRAGMENT_SUMMARY` variable defines the one-line textual
+      summary of a :term:`Configuration Fragment`. For details on how to use
+      fragments, see the :doc:`/ref-manual/fragments` section of the Yocto
+      Project Reference Manual.
+
    :term:`BB_CONSOLELOG`
       See :term:`bitbake:BB_CONSOLELOG` in the BitBake manual.
 
@@ -6231,6 +6243,26 @@ system and gives an overview of their function and contents.
       :term:`Source Directory` for details on how this class
       applies these additional sed command arguments.
 
+   :term:`OE_FRAGMENTS`
+      The :term:`OE_FRAGMENTS` variable holds the list of :term:`Configuration
+      Fragments <Configuration Fragment>` currently enabled for the build. For
+      details on how to use fragments, see the :doc:`/ref-manual/fragments`
+      section of the Yocto Project Reference Manual.
+
+   :term:`OE_FRAGMENTS_BUILTIN`
+      The :term:`OE_FRAGMENTS_BUILTIN` variable holds the list of
+      :term:`Built-in Fragments <Built-in Fragment>` available for being set with
+      :oe_git:`bitbake-config-build </bitbake/tree/bin/bitbake-config-build>`.
+      For details on how to use fragments, see the :doc:`/ref-manual/fragments`
+      section of the Yocto Project Reference Manual.
+
+   :term:`OE_FRAGMENTS_PREFIX`
+      The :term:`OE_FRAGMENTS_PREFIX` variable defines the prefix where
+      :term:`BitBake` tries to locate :term:`Configuration Fragments
+      <Configuration Fragment>` in :term:`layers <Layer>`. For details on how to
+      use fragments, see the :doc:`/ref-manual/fragments` section of the Yocto
+      Project Reference Manual.
+
    :term:`OE_INIT_ENV_SCRIPT`
       The name of the build environment setup script for the purposes of
       setting up the environment within the extensible SDK. The default

-- 
2.51.0

Re: [PATCH 2/2] Add documentation on fragments

[Alexander Kanavin]

On Thu, 25 Sept 2025 at 09:29, Antonin Godard
<antonin.godard@bootlin.com> wrote:
>
> - terms.rst: Provide the definitions of a Configuration Fragment and a
>   Built-in Fragment.
> - ref-manual: Add a quick reference guide on bitbake-config-build, and
>   list the available fragments in OE-Core.
>   Document the underlying variables related to fragments in the
>   glossary.
> - dev-manual: give instructions on how to create new custom fragments.

Thanks for writing these up. It's mostly fine, but I do have a few comments.

> +Using Configuration Fragments In Your Build
> +*******************************************

What follows is not about using fragments (e.g. enabling/disabling
them), it's about making new fragments. I think the section should be
titled accordingly.

> +The Yocto Project supports enabling and disable some configuration variables
> +through :term:`Configuration Fragments <Configuration Fragment>`. This mechanism
> +allows modifying the top-level configuration of a build from the command-line
> +using the :oe_git:`bitbake-config-build </bitbake/tree/bin/bitbake-config-build>`
> +tool. This section will describe how to create new fragments for your builds.

I think here and elsewhere (where you need to explain what fragments
are) it's good to introduce fragments like this:
'Fragments define top level build configuration features that can be
independently enabled and disabled using standard tooling. Such
features are made of one or several build configuration statements
that are either contained in a fragment file, or are set indirectly
using the built-in fragment mechanism. '.

> +There are two kinds of configuration fragments:
> +
> +-  :term:`Configuration Fragments <Configuration Fragment>` which a stored
> +   in a file. These fragment include a summary and a description.

Spelling issues. They are also in some other places, I won't nit-pick
on those. Let's get the content correct first.

'Standard :term:`Configuration Fragments <Configuration Fragment>`
which are defined in a file. These fragments include a summary and a
description, followed by configuration statements.

> +-  :term:`Built-in Fragments <Built-in Fragment>` which are used to set a
> +   variable value.

... which can be used to assign a value to a single variable and do
not require a separate definition file. They are especially useful
when a list of possible values is very long (or infinite).

> +Creating new :term:`Built-in Fragments <Built-in Fragment>` is not supported.

I think it is, but that requires putting an additional addfragment
statement somewhere, or some way to append to OE_FRAGMENTS_BUILTIN
*before* the oe-core's addfragments is processed.

> +You should instead create :term:`Configuration Fragment` files as documented
> +below to customize a build to your need.

I would try to describe adding built-in fragments as well.

> +Creating A Configuration Fragment File
> +======================================
> +
> +By default, all configuration fragments are located within the
> +``conf/fragments`` directory of a :term:`layer`. This location is defined by the
> +:term:`OE_FRAGMENTS_PREFIX` variable.

... which, in turn, is used as a parameter in a addfragments statement
in meta/conf/bitbake.conf

> +You can create one or more :term:`configuration fragment` files in your
> +:term:`layer` in this directory. Let's take the following example, where
> +``custom-fragment.conf`` is our custom fragment file::
> +
> +   meta-custom
> +   ├── conf
> +   │   ├── fragments
> +   │   │   └── custom-fragment.conf
> +   │   └── layer.conf
> +   ...
> +
> +For our ``custom-fragment.conf`` file, the following variables **must** be set
> +for our fragment to be considered a valid fragment by the :term:`OpenEmbedded
> +Build System`:
> +
> +-  :term:`BB_CONF_FRAGMENT_SUMMARY`: a one-line summary of this fragment.
> +
> +-  :term:`BB_CONF_FRAGMENT_DESCRIPTION`: a description of this fragment.

These are again defined through a parameter to addfragments, so it
would be good to reference that.

> +After creating these variables, our custom fragment should look like the
> +following:
> +
> +.. code-block::
> +   :caption: custom-fragment.conf
> +
> +   BB_CONF_FRAGMENT_SUMMARY = "My custom fragment summary"
> +   BB_CONF_FRAGMENT_DESCRIPTION = "My custom fragment description, \
> +   which can be multiple lines."

"This fragment sets a limit of 4 bitbake threads and 4 parsing threads."
"This fragment is useful to constrain resource consumption when the
Yocto default \
 is causing an overload of host machine's memory and cpu resources."

> +***********************
> +Configuration Fragments

Using configuration fragments.


> +***********************
> +
> +:term:`Configuration Fragments <Configuration Fragment>` are used to provide a
> +snippet of global configuration that can be enabled or disabled for a build.
> +This document provides a quick reference of the :oe_git:`bitbake-config-build
> +</bitbake/tree/bin/bitbake-config-build>` tool and lists the
> +:term:`Configuration Fragments <Configuration Fragment>` and :term:`Built-in
> +Fragments <Built-in Fragment>` available in the :term:`OpenEmbedded Build
> +System` core repositories.

See above how the fragments could be introduced.

> +When a fragment is enabled with :ref:`ref-bitbake-config-build-enable-fragment`,
> +its name is automatically appended to the :term:`OE_FRAGMENTS` variable in
> +:ref:`structure-build-conf-auto.conf`.

It is also possible (but not recommended) to manually add to this
variable from local.conf.

> +   :term:`Built-in Fragment`
> +      A built-in fragment is a specific kind of :term:`Configuration Fragment`
> +      that affects the value of a single variable globally. Like
> +      a normal :term:`Configuration Fragment`, Built-in Fragments can be enabled
> +      or disabled using the :oe_git:`bitbake-config-build </bitbake/tree/bin/bitbake-config-build>`
> +      command-line utility.
> +
> +      When declared, a built-in fragment follows the following naming
> +      convention::
> +
> +         <fragment>:<variable name>
> +
> +      Where:
> +
> +      -  ``<fragment>`` is the name of the built-in fragment.
> +      -  ``<variable name>`` is the name of the variable to be modified by this
> +         fragment.
> +
> +      For example::
> +
> +         machine:MACHINE
> +
> +      Will setup the ``machine`` Built-in Fragment for modifying the value of
> +      the :term:`MACHINE` variable.
> +
> +      Setting the :term:`MACHINE` variable through this fragment must follow
> +      this syntax::
> +
> +         machine/qemux86-64
> +
> +      This sets the value of :term:`MACHINE` to ``qemux86-64``.
> +
> +      For more details on fragments, see:
> +
> +      -  The :doc:`/ref-manual/fragments` section of the Yocto Project Reference
> +         Manual for a list of fragments the :term:`OpenEmbedded Build System`
> +         supports, and a quick reference guide on how to manage fragments.
> +
> +      -  The :doc:`/dev-manual/using-fragments` section of the Yocto Project
> +         Development Tasks Manual for details on how to create new fragments
> +         in your build.

Two more points need to be made:

- built-in fragments can be used to assign a value to a single
variable and do not require a separate definition file. They are
especially useful when a list of possible values is very long (or
infinite), as that avoids making large amounts of fragment files, all
very similar to each other.

- the definition of available built-in fragments is in meta/conf/bitbake.conf:
OE_FRAGMENTS_BUILTIN ?= "machine:MACHINE distro:DISTRO"

In turn, what variable contains the definition is specified in the
addfragments directive in the same file:
addfragments ${OE_FRAGMENTS_PREFIX} OE_FRAGMENTS
OE_FRAGMENTS_METADATA_VARS OE_FRAGMENTS_BUILTIN

>     :term:`Classes`
>        Files that provide for logic encapsulation and inheritance so that
>        commonly used patterns can be defined once and then easily used in
> @@ -154,6 +196,37 @@ universal, the list includes them just in case:
>        only used when building for that target (e.g. the
>        :file:`machine/beaglebone.conf` configuration file defines variables for
>        the Texas Instruments ARM Cortex-A8 development board).
> +      :term:`Configuration Fragments <Configuration Fragment>` such as
> +      :ref:`ref-fragments-core-yocto-sstate-mirror-cdn` define snippets of
> +      configuration that can be enabled from the command-line.
> +
> +   :term:`Configuration Fragment`
> +      A configuration fragment is a :term:`Configuration File` that contains
> +      variable assignments affecting the build at a global-level when the

And directives, such as 'require ...'. I'd instead say 'configuration
statements, such as variable assignments'.

> +      fragment is enabled. By default, configuration fragments are located in
> +      the :file:`conf/fragments/` directory of a :term:`Layer`.
> +
> +      A fragment :term:`configuration file` must contain a summary
> +      (:term:`BB_CONF_FRAGMENT_SUMMARY`) and a description
> +      (:term:`BB_CONF_FRAGMENT_DESCRIPTION`) explaining the purpose of the
> +      fragment.

In oe-core, the location of fragments and what variables are required
in a fragment is specified via meta/conf/bitbake.conf:

OE_FRAGMENTS_PREFIX ?= "conf/fragments"
OE_FRAGMENTS_METADATA_VARS ?= "BB_CONF_FRAGMENT_SUMMARY
BB_CONF_FRAGMENT_DESCRIPTION"
addfragments ${OE_FRAGMENTS_PREFIX} OE_FRAGMENTS
OE_FRAGMENTS_METADATA_VARS OE_FRAGMENTS_BUILTIN


Thanks!

Alex

Re: [PATCH 2/2] Add documentation on fragments

[Antonin Godard]

On Thu Sep 25, 2025 at 1:07 PM CEST, Alexander Kanavin wrote:
> On Thu, 25 Sept 2025 at 09:29, Antonin Godard
> <antonin.godard@bootlin.com> wrote:
>>
>> - terms.rst: Provide the definitions of a Configuration Fragment and a
>>   Built-in Fragment.
>> - ref-manual: Add a quick reference guide on bitbake-config-build, and
>>   list the available fragments in OE-Core.
>>   Document the underlying variables related to fragments in the
>>   glossary.
>> - dev-manual: give instructions on how to create new custom fragments.
>
> Thanks for writing these up. It's mostly fine, but I do have a few comments.
>
>> +Using Configuration Fragments In Your Build
>> +*******************************************
>
> What follows is not about using fragments (e.g. enabling/disabling
> them), it's about making new fragments. I think the section should be
> titled accordingly.

Right, this turned out to be a document different from my initial draft and
forgot to update the title. Thanks for spotting this.

>> +The Yocto Project supports enabling and disable some configuration variables
>> +through :term:`Configuration Fragments <Configuration Fragment>`. This mechanism
>> +allows modifying the top-level configuration of a build from the command-line
>> +using the :oe_git:`bitbake-config-build </bitbake/tree/bin/bitbake-config-build>`
>> +tool. This section will describe how to create new fragments for your builds.
>
> I think here and elsewhere (where you need to explain what fragments
> are) it's good to introduce fragments like this:
> 'Fragments define top level build configuration features that can be
> independently enabled and disabled using standard tooling. Such
> features are made of one or several build configuration statements
> that are either contained in a fragment file, or are set indirectly
> using the built-in fragment mechanism. '.

Ok, I felt that I needed a good catch phrase on fragments to put in different
places - I'll use that one.

>> +There are two kinds of configuration fragments:
>> +
>> +-  :term:`Configuration Fragments <Configuration Fragment>` which a stored
>> +   in a file. These fragment include a summary and a description.
>
> Spelling issues. They are also in some other places, I won't nit-pick
> on those. Let's get the content correct first.
>
> 'Standard :term:`Configuration Fragments <Configuration Fragment>`
> which are defined in a file. These fragments include a summary and a
> description, followed by configuration statements.

Thanks for your feedback on this. I had a struggle whether we should name
fragments in file "standard" or "regular", or just "fragments". I think standard
is good.

>> +-  :term:`Built-in Fragments <Built-in Fragment>` which are used to set a
>> +   variable value.
>
> ... which can be used to assign a value to a single variable and do
> not require a separate definition file. They are especially useful
> when a list of possible values is very long (or infinite).
>
>> +Creating new :term:`Built-in Fragments <Built-in Fragment>` is not supported.
>
> I think it is, but that requires putting an additional addfragment
> statement somewhere, or some way to append to OE_FRAGMENTS_BUILTIN
> *before* the oe-core's addfragments is processed.

Yes exactly, I did have some struggle yesterday so in the meantime wrote this.
But I know it can be supported.

>> +You should instead create :term:`Configuration Fragment` files as documented
>> +below to customize a build to your need.
>
> I would try to describe adding built-in fragments as well.

I tried but failed to have something consistent yesterday.

Setting it from a distro configuration file, or any configuration file that
comes after the addfragments call in bitbake.conf does not work in my
experience. Moving the addfragments call after the distro file include is not
possible as the the distro/ builtin fragment sets the DISTRO (chicken and egg
problem :)).

There are multiple solutions:

- Telling users to define their custom builtin fragments in local.conf,
  auto.conf, site.conf, or any file that comes before the addfragments call.
  This is not desirable because this means we don't encourage to share
  configurations.

- Telling users to defines their builtin fragments in layer.conf. This works,
  but this means layers can "pollute" the existing builtin fragments by
  overriding them when added to bblayers.conf. Other built-in fragments that have
  different names (not machine/ or distro/) is fine I guess, because
  just declaring a fragment as available does not affect the build.

  Maybe this could be reasonable, but we enforce layer maintainers to _not_
  override core built-in fragments (yocto-check-layer test, maybe?)

- One idea I had was to introduce an `addbuiltinfragments FRAGMENTS_BUILTIN`
  directive. This way we handle standard and built-in fragments separately.
  This turns out to be difficult as well, as standard and built-in fragments
  are mixed in together in OE_FRAGMENTS.

I feel like our best option is option 2 here. What do you think? Other ideas
maybe?

>> +Creating A Configuration Fragment File
>> +======================================
>> +
>> +By default, all configuration fragments are located within the
>> +``conf/fragments`` directory of a :term:`layer`. This location is defined by the
>> +:term:`OE_FRAGMENTS_PREFIX` variable.
>
> ... which, in turn, is used as a parameter in a addfragments statement
> in meta/conf/bitbake.conf
>
>> +You can create one or more :term:`configuration fragment` files in your
>> +:term:`layer` in this directory. Let's take the following example, where
>> +``custom-fragment.conf`` is our custom fragment file::
>> +
>> +   meta-custom
>> +   ├── conf
>> +   │   ├── fragments
>> +   │   │   └── custom-fragment.conf
>> +   │   └── layer.conf
>> +   ...
>> +
>> +For our ``custom-fragment.conf`` file, the following variables **must** be set
>> +for our fragment to be considered a valid fragment by the :term:`OpenEmbedded
>> +Build System`:
>> +
>> +-  :term:`BB_CONF_FRAGMENT_SUMMARY`: a one-line summary of this fragment.
>> +
>> +-  :term:`BB_CONF_FRAGMENT_DESCRIPTION`: a description of this fragment.
>
> These are again defined through a parameter to addfragments, so it
> would be good to reference that.

My initial approach was to document this as if a user did not need to have
custom addfragments calls and all that was required was to place fragment files
in the good spot in their layer, and modify OE_FRAGMENTS_BUILTIN as needed
(turns the second use case is not that easy in the end).

For example, we don't make the location of machine configurations configurable
AFAIK - why would changing the location of fragments be needed for users of OE?

But I can add references to this command, I just don't think we should emphasize
too much on it.

>> +After creating these variables, our custom fragment should look like the
>> +following:
>> +
>> +.. code-block::
>> +   :caption: custom-fragment.conf
>> +
>> +   BB_CONF_FRAGMENT_SUMMARY = "My custom fragment summary"
>> +   BB_CONF_FRAGMENT_DESCRIPTION = "My custom fragment description, \
>> +   which can be multiple lines."
>
> "This fragment sets a limit of 4 bitbake threads and 4 parsing threads."
> "This fragment is useful to constrain resource consumption when the
> Yocto default \
>  is causing an overload of host machine's memory and cpu resources."

Thanks :) I could have made the extra effort here!

>> +***********************
>> +Configuration Fragments
>
> Using configuration fragments.
>
>
>> +***********************
>> +
>> +:term:`Configuration Fragments <Configuration Fragment>` are used to provide a
>> +snippet of global configuration that can be enabled or disabled for a build.
>> +This document provides a quick reference of the :oe_git:`bitbake-config-build
>> +</bitbake/tree/bin/bitbake-config-build>` tool and lists the
>> +:term:`Configuration Fragments <Configuration Fragment>` and :term:`Built-in
>> +Fragments <Built-in Fragment>` available in the :term:`OpenEmbedded Build
>> +System` core repositories.
>
> See above how the fragments could be introduced.
>
>> +When a fragment is enabled with :ref:`ref-bitbake-config-build-enable-fragment`,
>> +its name is automatically appended to the :term:`OE_FRAGMENTS` variable in
>> +:ref:`structure-build-conf-auto.conf`.
>
> It is also possible (but not recommended) to manually add to this
> variable from local.conf.

Will mention this

>> +   :term:`Built-in Fragment`
>> +      A built-in fragment is a specific kind of :term:`Configuration Fragment`
>> +      that affects the value of a single variable globally. Like
>> +      a normal :term:`Configuration Fragment`, Built-in Fragments can be enabled
>> +      or disabled using the :oe_git:`bitbake-config-build </bitbake/tree/bin/bitbake-config-build>`
>> +      command-line utility.
>> +
>> +      When declared, a built-in fragment follows the following naming
>> +      convention::
>> +
>> +         <fragment>:<variable name>
>> +
>> +      Where:
>> +
>> +      -  ``<fragment>`` is the name of the built-in fragment.
>> +      -  ``<variable name>`` is the name of the variable to be modified by this
>> +         fragment.
>> +
>> +      For example::
>> +
>> +         machine:MACHINE
>> +
>> +      Will setup the ``machine`` Built-in Fragment for modifying the value of
>> +      the :term:`MACHINE` variable.
>> +
>> +      Setting the :term:`MACHINE` variable through this fragment must follow
>> +      this syntax::
>> +
>> +         machine/qemux86-64
>> +
>> +      This sets the value of :term:`MACHINE` to ``qemux86-64``.
>> +
>> +      For more details on fragments, see:
>> +
>> +      -  The :doc:`/ref-manual/fragments` section of the Yocto Project Reference
>> +         Manual for a list of fragments the :term:`OpenEmbedded Build System`
>> +         supports, and a quick reference guide on how to manage fragments.
>> +
>> +      -  The :doc:`/dev-manual/using-fragments` section of the Yocto Project
>> +         Development Tasks Manual for details on how to create new fragments
>> +         in your build.
>
> Two more points need to be made:
>
> - built-in fragments can be used to assign a value to a single
> variable and do not require a separate definition file. They are
> especially useful when a list of possible values is very long (or
> infinite), as that avoids making large amounts of fragment files, all
> very similar to each other.
>
> - the definition of available built-in fragments is in meta/conf/bitbake.conf:
> OE_FRAGMENTS_BUILTIN ?= "machine:MACHINE distro:DISTRO"
>
> In turn, what variable contains the definition is specified in the
> addfragments directive in the same file:
> addfragments ${OE_FRAGMENTS_PREFIX} OE_FRAGMENTS
> OE_FRAGMENTS_METADATA_VARS OE_FRAGMENTS_BUILTIN

Will add that

>>     :term:`Classes`
>>        Files that provide for logic encapsulation and inheritance so that
>>        commonly used patterns can be defined once and then easily used in
>> @@ -154,6 +196,37 @@ universal, the list includes them just in case:
>>        only used when building for that target (e.g. the
>>        :file:`machine/beaglebone.conf` configuration file defines variables for
>>        the Texas Instruments ARM Cortex-A8 development board).
>> +      :term:`Configuration Fragments <Configuration Fragment>` such as
>> +      :ref:`ref-fragments-core-yocto-sstate-mirror-cdn` define snippets of
>> +      configuration that can be enabled from the command-line.
>> +
>> +   :term:`Configuration Fragment`
>> +      A configuration fragment is a :term:`Configuration File` that contains
>> +      variable assignments affecting the build at a global-level when the
>
> And directives, such as 'require ...'. I'd instead say 'configuration
> statements, such as variable assignments'.

Indeed, thanks.

>> +      fragment is enabled. By default, configuration fragments are located in
>> +      the :file:`conf/fragments/` directory of a :term:`Layer`.
>> +
>> +      A fragment :term:`configuration file` must contain a summary
>> +      (:term:`BB_CONF_FRAGMENT_SUMMARY`) and a description
>> +      (:term:`BB_CONF_FRAGMENT_DESCRIPTION`) explaining the purpose of the
>> +      fragment.
>
> In oe-core, the location of fragments and what variables are required
> in a fragment is specified via meta/conf/bitbake.conf:
>
> OE_FRAGMENTS_PREFIX ?= "conf/fragments"
> OE_FRAGMENTS_METADATA_VARS ?= "BB_CONF_FRAGMENT_SUMMARY
> BB_CONF_FRAGMENT_DESCRIPTION"
> addfragments ${OE_FRAGMENTS_PREFIX} OE_FRAGMENTS
> OE_FRAGMENTS_METADATA_VARS OE_FRAGMENTS_BUILTIN
>

I usually don't include code snippets from OE-Core as it is harder to maintain,
but I can reference the variable and location, this should be similar without
the maintenance burden.

> Thanks!
>
> Alex

Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [PATCH 2/2] Add documentation on fragments

[Alexander Kanavin]

On Thu, 25 Sept 2025 at 15:33, Antonin Godard
<antonin.godard@bootlin.com> wrote:
> > I would try to describe adding built-in fragments as well.
>
> I tried but failed to have something consistent yesterday.
>
> Setting it from a distro configuration file, or any configuration file that
> comes after the addfragments call in bitbake.conf does not work in my
> experience. Moving the addfragments call after the distro file include is not
> possible as the the distro/ builtin fragment sets the DISTRO (chicken and egg
> problem :)).
>
> There are multiple solutions:
>
> - Telling users to define their custom builtin fragments in local.conf,
>   auto.conf, site.conf, or any file that comes before the addfragments call.
>   This is not desirable because this means we don't encourage to share
>   configurations.
>
> - Telling users to defines their builtin fragments in layer.conf. This works,
>   but this means layers can "pollute" the existing builtin fragments by
>   overriding them when added to bblayers.conf. Other built-in fragments that have
>   different names (not machine/ or distro/) is fine I guess, because
>   just declaring a fragment as available does not affect the build.
>
>   Maybe this could be reasonable, but we enforce layer maintainers to _not_
>   override core built-in fragments (yocto-check-layer test, maybe?)
>
> - One idea I had was to introduce an `addbuiltinfragments FRAGMENTS_BUILTIN`
>   directive. This way we handle standard and built-in fragments separately.
>   This turns out to be difficult as well, as standard and built-in fragments
>   are mixed in together in OE_FRAGMENTS.
>
> I feel like our best option is option 2 here. What do you think? Other ideas
> maybe?

Right, yes, thanks for thinking it through and writing it down, and I
agree with this. There indeed doesn't seem to be an easy solution. On
the other hand, built-in fragments are intended for setting something
really common and well known, there's currently DISTRO/MACHINE, but I
can also imagine things like TCLIBC / TOOLCHAIN / INIT_MANAGER and so
on. Those are not particularly controversial to just add to the
existing builtin list in oecore's meta/conf/bitbake.conf. Perhaps you
could somehow express this, without overwhelming the reader with the
above technicalities.

> > These are again defined through a parameter to addfragments, so it
> > would be good to reference that.
>
> My initial approach was to document this as if a user did not need to have
> custom addfragments calls and all that was required was to place fragment files
> in the good spot in their layer, and modify OE_FRAGMENTS_BUILTIN as needed
> (turns the second use case is not that easy in the end).
>
> For example, we don't make the location of machine configurations configurable
> AFAIK - why would changing the location of fragments be needed for users of OE?
>
> But I can add references to this command, I just don't think we should emphasize
> too much on it.

Yes, I don't have a strong opinion here. If you feel that this would
overwhelm the reader with unnecessary implementation details, that's
fine. There's a balance to be struck here, so that people that want to
know how things really work get a hint on where to look for it.

Alex