Re: [docs] [PATCH 6/6] dev-manual/multiconfig: add suggested best practices and baremetal sections

["Antonin Godard" <antonin.godard@bootlin.com>]

Hi Quentin,

On Tue Feb 11, 2025 at 1:08 PM CET, Quentin Schulz wrote:
> Hi Antonin,
>
> On 2/7/25 5:28 PM, Antonin Godard via lists.yoctoproject.org wrote:
>> After the suggestions from Mark Hatle on the list
>> (https://lists.yoctoproject.org/g/docs/topic/110487932), add two
>> sections to the multiconfig doc:
>> 
>> - Suggested best practices: suggestion for better design of multiconfig
>>    builds.
>> 
>> - Common use case: baremetal build.
>> 
>>    This section applies the guidelines from the first sections and apply
>>    it to a real-life example of how to use multiconfig. This one to build
>>    some baremetal firmware alongside a regular Linux build.
>> 
>> Suggested-by: Mark Hatle <mark.hatle@kernel.crashing.org>
>> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
>> ---
>>   documentation/dev-manual/multiconfig.rst | 106 +++++++++++++++++++++++++++++++
>>   1 file changed, 106 insertions(+)
>> 
>> diff --git a/documentation/dev-manual/multiconfig.rst b/documentation/dev-manual/multiconfig.rst
>> index 27442a042..9560ce5a0 100644
>> --- a/documentation/dev-manual/multiconfig.rst
>> +++ b/documentation/dev-manual/multiconfig.rst
>> @@ -171,3 +171,109 @@ and have separate configuration files, BitBake places the artifacts for
>>   each build in the respective temporary build directories (i.e.
>>   :term:`TMPDIR`).
>>   
>> +Suggested best practices
>> +========================
>> +
>> +-  :term:`TMPDIR` (other then the default set in bitbake.conf) is only set in
>
> s/then/than/
>
> and tick quote bitbake.conf
>
>> +   ``local.conf`` by the user. This means that we should **not** manipulate
>> +   :term:`TMPDIR` in any way within the Machine or Distro :term:`configuration
>> +   file`.
>> +
>> +-  A multiconfig should specify a :term:`TMPDIR`, and should specify it by
>> +   appending the multiconfig name via :term:`BB_CURRENT_MC`.
>> +
>
> Provide the example you gave in the previous patch.
>
>> +-  Recipes that are used to transfer the output from a multiconfig build should
>
> +to another
>
> i.e.
>
> from a multiconfig build to another
>
> ?
>
>> +   use ``task[mcdepends]`` to trigger the build of the component, and then
>
> maybe use do_task[mcdepends] here?
>
>> +   transfer the item to the current configuration in :ref:`ref-tasks-install`
>> +   :ref:`ref-tasks-deploy`, assuming the value of the deployed item based on
>> +   :term:`TMPDIR`.
>> +
>
> Please provide an example with fictitious recipes and tasks?
>
> There's also a missing word between the install and deploy task refs.

Will be part of v2.

>> +-  Firmware recipes can set the :term:`INHIBIT_DEFAULT_DEPS` variable to ``1``
>> +   if they don't rely on default dependencies such as the standard C library.
>> +
>> +Common use case: building baremetal firmware alongside a Linux build
>> +====================================================================
>> +
>> +A common use for multiconfig is to use the default configuration as the regular
>> +Linux build, while one or more multiconfigs can be used to build special
>> +components, such as baremetal firmware. This section details how one can achieve
>> +this scenario.
>> +
>> +Adding a multiconfig configuration file and recipe for a baremetal firmware
>> +---------------------------------------------------------------------------
>> +
>> +As described in :ref:`dev-manual/multiconfig:Setting Up and Running a Multiple
>> +Configuration Build`, each multiconfig will require a separate
>> +:term:`Configuration File`. In our case, we will make a separate temporary
>> +directory for our baremetal firmware build configuration.
>> +
>> +For example, we will define a new ``conf/multiconfig/baremetal-firmware.conf``
>> +as follows::
>> +
>> +   TMPDIR .= "-${BB_CURRENT_MC}"
>> +   TCLIBC = "newlib"
>> +
>> +The ``baremetal-firmware.conf`` configure a separate :term:`TMPDIR` for holding
>> +binaries compiled with the `newlib <https://sourceware.org/newlib/>`__ toolchain
>> +(see :term:`TCLIBC`).
>> +
>> +We then create a recipe ``my-firmware.bb`` that defines how the baremetal
>> +firmware is built. The recipe should contain enough information for the
>> +:term:`OpenEmbedded build system` to properly compile the firmware with our
>> +toolchain. The building tasks may vary depending on the nature of the firmware.
>> +However, the recipe should define a :ref:`ref-classes-deploy` task that deploys
>> +the output into the :term:`DEPLOYDIR` directory. We will consider in the
>> +following that the file is named ``my-firmware.elf``.
>> +
>> +Building the firmware
>> +---------------------
>> +
>> +The firmware can be built with BitBake with the following command::
>> +
>> +   $ bitbake mc:baremetal-firmware:my-firmware
>> +
>> +However, we would prefer for ``my-firmware`` to be automatically built when
>> +triggering a normal Linux build.
>> +
>> +Using an ``mcdepend``, a recipe belonging to the Linux build can trigger the
>> +build of ``my-firmware``. For example, let's consider that our Linux build needs
>> +to assemble a "special" firmware that uses the output of our ``my-firmware``
>> +recipe - let's call it ``my-parent-firmware.bb``. Then, we should specify this
>> +dependency in ``my-parent-firmware.bb`` with::
>> +
>> +   do_compile[mcdepends] = "mc::baremetal-firmware:my-firmware:do_deploy"
>> +
>> +The above will ensure that when the :ref:`ref-tasks-compile` task of
>> +``my-parent-firmware`` is triggered, the :ref:`ref-tasks-deploy` task of
>> +``my-firmware`` will already have run successfully.
>> +
>> +Using the output of ``my-firmware``
>> +-----------------------------------
>> +
>> +After we have deployed ``my-firmware`` by using ``mcdepends``, we need to use
>> +the output in some way. We can make a series of assumptions, based on the
>> +default Yocto Project variables in order to get the binary for packaging.
>> +
>> +First, we can set the following in ``my-parent-firmware.bb``::
>> +
>> +   FIRMWARE_FILE ??= "${TMPDIR}-baremetal-firmware/deploy/images/<machine>/my-firmware.elf"
>> +   FIRMWARE_FILE[vardepsexclude] += "TMPDIR"
>> +
>> +The first assignment stores the value of the path to the firmware built and
>> +deployed by the ``my-firmware.bb`` recipe. The second assignment excludes the
>> +:term:`TMPDIR` variable from being part of ``FIRMWARE_FILE``'s dependencies -
>> +meaning that changing the value of :term:`TMPDIR` (for example, changing the
>> +host on which the firmware is built) will not invalidate the :ref:`shared state
>> +cache <overview-manual/concepts:shared state cache>`.
>> +
>> +Additionally, ``<machine>`` should be replaced by the machine for which we are
>
> s/machine/:term:`MACHINE`/ ?

Here ``<machine>`` refers to the example, where it is written like so. However I
think I'll replace "by the machine" by "by the :term:`MACHINE`".

>> +building in the Linux context.
>> +
>
> In the baremetal-firmware context rather?

Yes, thanks for pointing that out!

>> +We can add an :ref:`ref-tasks-install` task to the ``my-parent-firmware``::
>
> We can +then ?
>
> s/an/a/ ?
>
> Coming from the embedded industry, I find an example using "main SoC" vs 
> "companion microcontroller" on the same board drawing a better picture 
> than "baremetal firmware" vs "Linux build" especially since we don't 
> specify here they could be for different architectures (so implied they 
> are for the same).

I will try mentioning that this section is not limited to the baremetal firmware
running under the same architecture. Specifically, that the MACHINE variable may
be overridden in the multiconfig conf, if one wants to do so.

> Other use cases I've seen for multiconfig is rootfs within another 
> rootfs, or building a recovery image for a system and putting in on the 
> same disk image as the normal rootfs, or building an initramfs with 
> another init system than the one used for the main rootfs. We don't have 
> to list them all, just sharing here for reference :)

Yes, those are really good examples, hopefully this document can be used in the
future to add and document these! I think this was initially the intent of
Mark's original message.

Thanks!
Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com