Re: [docs] [PATCH] Add a document on limiting host resources

[Mikko Rapeli <mikko.rapeli@linaro.org>]

Hi,

On Tue, Jun 24, 2025 at 04:42:15PM +0200, Antonin Godard via lists.yoctoproject.org wrote:
> Add a "Limiting the Host Resources Usage" document to share the
> different techniques that can be used to limit the host resources usage.
> We do have a document to document how to speed up a build, so this
> document comes right after.
> 
> [YOCTO #15111]
> 
> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
> ---
>  documentation/dev-manual/index.rst              |  1 +
>  documentation/dev-manual/limiting-resources.rst | 85 +++++++++++++++++++++++++
>  2 files changed, 86 insertions(+)
> 
> diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst
> index 63736e0ab..4abfa59e9 100644
> --- a/documentation/dev-manual/index.rst
> +++ b/documentation/dev-manual/index.rst
> @@ -22,6 +22,7 @@ Yocto Project Development Tasks Manual
>     building
>     multiconfig
>     speeding-up-build
> +   limiting-resources
>     libraries
>     prebuilt-libraries
>     devtool
> diff --git a/documentation/dev-manual/limiting-resources.rst b/documentation/dev-manual/limiting-resources.rst
> new file mode 100644
> index 000000000..df8208bdc
> --- /dev/null
> +++ b/documentation/dev-manual/limiting-resources.rst
> @@ -0,0 +1,85 @@
> +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
> +
> +Limiting the Host Resources Usage
> +*********************************
> +
> +While you sometimes need to :doc:`speed up a build
> +</dev-manual/speeding-up-build>`, you may also need to limit the resources used
> +by the :term:`OpenEmbedded Build System`, especially on shared infrastructures
> +where multiple users start heavy-load builds.
> +
> +This document aims at giving the different configuration variables available to
> +limit the resources used by the build system. These variables should be set from
> +a :term:`configuration file` and thus take effect over the entire build environment.
> +For each variable, also see the variable description in the glossary for more
> +details.
> +
> +-  :term:`BB_NUMBER_THREADS`:
> +
> +   This sets a hard limit on the number of threads :term:`BitBake` can run at the
> +   same time. Lowering this value will set a limit to the number of
> +   :term:`BitBake` threads, but will not prevent a single task from starting more
> +   compilation threads (see :term:`PARALLEL_MAKE`).
> +

I would add here or to intro that confusingly, CPU load caused by too much parallellism
may not be the problem but running out of physical RAM which trigger kernel
out-of-memory killer to kill processes related to builds causing very
odd looking failures in build logs. The oom killer actions are visible in build host
kernel dmesg logs, but only there. Thus if machines has low physical RAM
to CPU thread ratio, say less than 2 Gb, then limiting both BB_NUMBER_THREADS
and PARALLEL_MAKE can help.

Cheers,

-Mikko

> +-  :term:`BB_NUMBER_PARSE_THREADS`:
> +
> +   Like :term:`BB_NUMBER_THREADS`, but this variable sets a limit to the number
> +   of threads using during parsing of the environment (before executing tasks).
> +
> +-  :term:`PARALLEL_MAKE`:
> +
> +   This variable should be set in the form of ``-jN``, where ``N`` is a positive
> +   integer. This integer controls the number of threads used when starting
> +   ``make``. Note that this variable is not limited to the usage of ``make``,
> +   but extends to the compilation commands defined by the
> +   :ref:`ref-classes-meson`, :ref:`ref-classes-cmake` and such classes.
> +
> +   If you have identified a recipe that you want to limit separately from the
> +   rest, it is also possible to achieve with the following syntax::
> +
> +      PARALLEL_MAKE:pn-linux-yocto = "-j4"
> +
> +   The above example will limit the number of threads using by ``make`` for the
> +   ``linux-yocto`` recipe to 4.
> +
> +-  :term:`PARALLEL_MAKEINST`:
> +
> +   Like :term:`PARALLEL_MAKE`, but this variable controls the number of threads
> +   used during the :ref:`ref-tasks-install` task.
> +
> +-  :term:`BB_PRESSURE_MAX_CPU`, :term:`BB_PRESSURE_MAX_IO` and
> +   :term:`BB_PRESSURE_MAX_MEMORY`:
> +
> +   This variable controls the limit of pressure (as defined by
> +   https://docs.kernel.org/accounting/psi.html) on the system, and will
> +   limit the number of :term:`BitBake` threads dynamically depending on the
> +   current pressure of the system.
> +
> +   These variable take a positive integer between 1 (extremely low limit) and
> +   1000000 (value unlikely ever reached). Setting an extremely low value, such
> +   as 2, is not desirable as it will result in :term:`BitBake` limiting the
> +   number of threads to 1 most of the time.
> +
> +   However, having a low value at the beginning makes sense to get a sense of a
> +   reasonable value to set for your system. Indeed, :term:`BitBake` will print
> +   message on the console in the following format each time the current pressure
> +   exceeds of the limit set by the above variables::
> +
> +      Pressure status changed to CPU: True, IO: False, Mem: False (CPU: 1105.9/2.0, IO: 0.0/2.0, Mem: 0.0/2.0) - using 1/64 bitbake threads
> +
> +   Take a look at the value between parenthesis: ``CPU: 1105.9/2.0, IO: 0.0/2.0,
> +   Mem: 0.0/2.0``. They correspond to the current pressure value for the CPU, IO
> +   and memory respectively. Monitoring these values during a build should tell
> +   you what a reasonable value for your host system is.
> +
> +-  :term:`BB_LOADFACTOR_MAX`:
> +
> +   This variable will limit the number of threads :term:`BitBake` will start
> +   by monitoring the current load of the host system. :term:`BitBake` will print
> +   the following when the limit set by :term:`BB_LOADFACTOR_MAX` is reached::
> +
> +      Load average limiting set to True as load average: 0.7188262939453125 - using 37/64 bitbake threads
> +
> +   This variable has no effect when any of :term:`BB_PRESSURE_MAX_CPU`,
> +   :term:`BB_PRESSURE_MAX_IO` or :term:`BB_PRESSURE_MAX_MEMORY` is set, as it
> +   was designed for system that do not have pressure information available.
> 
> ---
> base-commit: 85e58c4fc5d1be9a2ea9ed6b813d0168e3162dab
> change-id: 20250417-resource-limiting-12fe772af904
> 
> Best regards,
> --  
> Antonin Godard <antonin.godard@bootlin.com>
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#7215): https://lists.yoctoproject.org/g/docs/message/7215
> Mute This Topic: https://lists.yoctoproject.org/mt/113808179/7159507
> Group Owner: docs+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [mikko.rapeli@linaro.org]
> -=-=-=-=-=-=-=-=-=-=-=-
>