* [PATCH v4 2/3] ref-manual: variables: add SIGGEN_LOCKEDSIGS* variables
2024-11-06 9:02 [PATCH v4 0/3] Add bblock documentation Julien Stephan
2024-11-06 9:02 ` [PATCH v4 1/3] styles: vocabularies: Yocto: add sstate Julien Stephan
@ 2024-11-06 9:02 ` Julien Stephan
2024-11-06 10:30 ` [docs] " Antonin Godard
2024-11-06 9:02 ` [PATCH v4 3/3] dev-manual: add bblock documentation Julien Stephan
2 siblings, 1 reply; 7+ messages in thread
From: Julien Stephan @ 2024-11-06 9:02 UTC (permalink / raw)
To: docs, Julien Stephan
Variables SIGGEN_LOCKEDSIGS, SIGGEN_LOCKEDSIGS_TASKSIG_CHECK and
SIGGEN_LOCKEDSIGS_TYPES are used to lock specific tasks to specific
signatures. They are used by bitbake -S <lockedsigs> and bblock, so add
documentation for them.
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
---
documentation/ref-manual/variables.rst | 44 ++++++++++++++++++++++++++++++++++
1 file changed, 44 insertions(+)
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 07ed143ac5606617c720301ac6621355a341f90a..76fc50ecbfb841125860e7dac42d4d4c102942cf 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -7880,6 +7880,50 @@ system and gives an overview of their function and contents.
might break at runtime if the interface of the recipe was changed
after the other had been built.
+ :term:`SIGGEN_LOCKEDSIGS`
+ The list of locked tasks, with the form::
+
+ SIGGEN_LOCKEDSIGS += "<package>:<task>:<signature>"
+
+ If ``<signature>`` exists for the specified ``<task>`` and ``<package>``
+ in the sstate cache, BitBake will use the cached output instead of
+ rebuilding the ``<task>``. If it does not exist, BitBake will build the
+ ``<task>`` and the sstate cache will be used next time.
+
+ Example::
+
+ SIGGEN_LOCKEDSIGS += "bc:do_compile:09772aa4532512baf96d433484f27234d4b7c11dd9cda0d6f56fa1b7ce6f25f0"
+
+ You can obtain the signature of all the tasks for the recipe ``bc`` using::
+
+ bitbake -S none bc
+
+ Then you can look at files in ``build/stamps/<arch>/bc`` and search for the
+ file: ``<PV>.do_compile.sigdata.09772aa4532512baf96d433484f27234d4b7c11dd9cda0d6f56fa1b7ce6f25f0``.
+
+ :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`
+ Specifies the debug level of task signature check. 3 levels are supported:
+
+ * ``info``: displays a "Note" message to remind the user that a task is locked
+ and the current signature matches the locked one.
+ * ``warn``: displays a "Warning" message if a task is locked and the current
+ signature does not match the locked one.
+ * ``error``: same as warn but displays an "Error" message and aborts.
+
+ :term:`SIGGEN_LOCKEDSIGS_TYPES`
+ Allowed overrides for :term:`SIGGEN_LOCKEDSIGS`. This is mainly used
+ for architecture specific locks. A common value for
+ :term:`SIGGEN_LOCKEDSIGS_TYPES` is ``${PACKAGE_ARCHS}``::
+
+ SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
+
+ SIGGEN_LOCKEDSIGS_core2-64 += "bc:do_compile:09772aa4532512baf96d433484f27234d4b7c11dd9cda0d6f56fa1b7ce6f25f0"
+ SIGGEN_LOCKEDSIGS_cortexa57 += "bc:do_compile:12178eb6d55ef602a8fe638e49862fd247e07b228f0f08967697b655bfe4bb61"
+
+ Here, the ``do_compile`` task from ``bc`` will be locked only for
+ ``core2-64`` and ``cortexa57`` but not for other architectures such as
+ ``mips32r2``.
+
:term:`SITEINFO_BITS`
Specifies the number of bits for the target system CPU. The value
should be either "32" or "64".
--
2.47.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH v4 3/3] dev-manual: add bblock documentation
2024-11-06 9:02 [PATCH v4 0/3] Add bblock documentation Julien Stephan
2024-11-06 9:02 ` [PATCH v4 1/3] styles: vocabularies: Yocto: add sstate Julien Stephan
2024-11-06 9:02 ` [PATCH v4 2/3] ref-manual: variables: add SIGGEN_LOCKEDSIGS* variables Julien Stephan
@ 2024-11-06 9:02 ` Julien Stephan
2 siblings, 0 replies; 7+ messages in thread
From: Julien Stephan @ 2024-11-06 9:02 UTC (permalink / raw)
To: docs, Julien Stephan
bblock is a helper tool to lock/unlock tasks and recipes to specific
signatures. Add a documentation page for it.
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
---
documentation/dev-manual/bblock.rst | 129 +++++++++++++++++++++++++++++++++
documentation/dev-manual/index.rst | 1 +
documentation/ref-manual/structure.rst | 9 +++
documentation/ref-manual/variables.rst | 3 +
4 files changed, 142 insertions(+)
diff --git a/documentation/dev-manual/bblock.rst b/documentation/dev-manual/bblock.rst
new file mode 100644
index 0000000000000000000000000000000000000000..68292903bab039ad66936a5061887a0ed569dfb4
--- /dev/null
+++ b/documentation/dev-manual/bblock.rst
@@ -0,0 +1,129 @@
+.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
+
+Locking and Unlocking Recipes Using ``bblock``
+**********************************************
+
+By design, the OpenEmbedded build system builds everything from scratch
+unless BitBake determines that specific tasks do not require rebuilding.
+At startup, it computes a signature for all tasks, based on the task's input.
+Then, it compares these signatures with the ones from the sstate cache (if they
+exist). Any changes cause the task to rerun.
+
+During development, changes might trigger BitBake to rebuild certain
+recipes, even when we know they do not require rebuilding at that stage.
+For example, modifying a recipe can lead to rebuilding its native
+counterpart, which might prove unnecessary. Editing the ``python3`` recipe,
+for instance, can prompt BitBake to rebuild ``python3-native`` along with any
+recipes that depend on it.
+
+To prevent this, use ``bblock`` to lock specific tasks or recipes to
+specific signatures, forcing BitBake to use the sstate cache for them.
+
+.. attention::
+
+ Use ``bblock`` only during the development phase.
+
+ Forcing BitBake to use the sstate cache, regardless of input changes, means
+ the recipe metadata no longer directly reflect the output. Use this feature
+ with caution. If you do not understand why signatures change, see the section
+ on :yocto_wiki:`understanding what changed </TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`.
+
+
+Locking tasks and recipes
+-------------------------
+
+To lock a recipe, use::
+
+ $ bblock recipe
+
+You can also use a space-separated list of recipes to lock multiple recipes::
+
+ $ bblock recipe1 recipe2
+
+Locking a recipe means locking all tasks of the recipe. If you need to
+lock only particular tasks, use the `-t` option with a comma-separated
+list of tasks::
+
+ $ bblock -t task1,task2 recipe
+
+
+Unlocking tasks and recipes
+---------------------------
+
+To unlock a recipe, use the ``-r`` option::
+
+ $ bblock -r recipe
+
+You can also use a space-separated list of recipes to unlock multiple recipes::
+
+ $ bblock -r recipe1 recipe2
+
+Unlocking a recipe means unlocking all tasks of the recipe. If you need to
+unlock only particular tasks use the ``-t`` option with a comma-separated
+list of tasks::
+
+ $ bblock -r -t task1,task2 recipe
+
+To unlock all recipes, do not specify any recipe::
+
+ $ bblock -r
+
+
+Configuration file
+------------------
+
+``bblock`` will dump the signatures in the ``build/conf/bblock.conf`` file,
+included by default in :oe_git:`meta/conf/bitbake.conf </openembedded-core/tree/meta/conf/bitbake.conf>`.
+
+To dump the file, use the ``-d`` option::
+
+ $ bblock -d
+
+
+Locking mechanism
+-----------------
+
+``bblock`` computes the signature(s) of the task(s) and sets the 3 following
+variables: :term:`SIGGEN_LOCKEDSIGS`, :term:`SIGGEN_LOCKEDSIGS_TYPES`
+and :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`.
+
+In particular, ``bblock`` sets::
+
+ SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
+ SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
+
+ SIGGEN_LOCKEDSIGS_<package_arch> += "<recipe>:<task>:<signature>"
+
+This produces architecture specific locks and reminds user that some tasks
+have locked signatures.
+
+Example
+-------
+
+When working on the ``python3`` recipe, we can lock ``python3-native`` with
+the following::
+
+ $ bblock python3-native
+ $ bblock -d
+ # Generated by bblock
+ SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
+ SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
+
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_patch:865859c27e603ba42025b7bb766c3cd4c0f477e4962cfd39128c0619d695fce7"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_sysroot:f8fa5d3194cef638416000252b959e86d0a19f6b7898e1f56b643c588cdd8605"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_prepare_recipe_sysroot:fe295ac505d9d1143313424b201c6f3f2a0a90da40a13a905b86b874705f226a"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_fetch:1b6e4728fee631bc7a8a7006855c5b8182a8224579e32e3d0a2db77c26459f25"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_unpack:2ad74d6f865ef75c35c0e6bbe3f9a90923a6b2c62c18a3ddef514ea31fbc588f"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_deploy_source_date_epoch:15f89b8483c1ad7507480f337619bb98c26e231227785eb3543db163593e7b42"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_configure:7960c13d23270fdb12b3a7c426ce1da0d2f5c7cf5e5d3f5bdce5fa330eb7d482"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_compile:012e1d4a63f1a78fc2143bd90d704dbcf5865c5257d6272aa7540ec1cd3063d9"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_install:d3401cc2afa4c996beb154beaad3e45fa0272b9c56fb86e9db14ec3544c68f9d"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_build:fa88bb7afb9046c0417c24a3fa98a058653805a8b00eda2c2d7fea68fc42f882"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_collect_spdx_deps:cc9c53ba7c495567e9a38ec4801830c425c0d1f895aa2fc66930a2edd510d9b4"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_spdx:766a1d09368438b7b5a1a8e2a8f823b2b731db44b57e67d8b3196de91966f9c5"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_package_spdx:46f80faeab25575e9977ba3bf14c819489c3d489432ae5145255635108c21020"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_recipe_qa:cb960cdb074e7944e894958db58f3dc2a0436ecf87c247feb3e095e214fec0e4"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_lic:15657441621ee83f15c2e650e7edbb036870b56f55e72e046c6142da3c5783fd"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_manifest:24f0abbec221d27bbb2909b6e846288b12cab419f1faf9f5006ed80423d37e28"
+ SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_addto_recipe_sysroot:bcb6a1905f113128de3f88d702b706befd6a786267c045ee82532759a7c214d7"
+
diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst
index 9ccf60f7012ea344612298e77ebabed84ad6de3e..7afd0d820e94d56cb2145962b9b1a976c7951936 100644
--- a/documentation/dev-manual/index.rst
+++ b/documentation/dev-manual/index.rst
@@ -48,5 +48,6 @@ Yocto Project Development Tasks Manual
error-reporting-tool
wayland
qemu
+ bblock
.. include:: /boilerplate.rst
diff --git a/documentation/ref-manual/structure.rst b/documentation/ref-manual/structure.rst
index bbeda78c791d430faa55356a3209f5eeb1402837..616c3c4c9bf375bb60ab34587b2c70bdb42d280b 100644
--- a/documentation/ref-manual/structure.rst
+++ b/documentation/ref-manual/structure.rst
@@ -335,6 +335,15 @@ Once the build process gets the sample file, it uses ``sed`` to substitute final
version of the ``bblayers.conf.sample`` file in the ``meta-poky/conf/templates/default``
directory.
+.. _structure-build-conf-bblock.conf:
+
+``build/conf/bblock.conf``
+--------------------------
+
+This configuration file is generated by :doc:`bblock </dev-manual/bblock>` and
+contains the signatures locked by ``bblock``. By default, it does not exist
+and will be created upon the first invocation of ``bblock``.
+
.. _structure-build-downloads:
``build/downloads/``
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 76fc50ecbfb841125860e7dac42d4d4c102942cf..831b7c54c89bdeb64b8ce3bbf512a510b94dfc02 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -7901,6 +7901,9 @@ system and gives an overview of their function and contents.
Then you can look at files in ``build/stamps/<arch>/bc`` and search for the
file: ``<PV>.do_compile.sigdata.09772aa4532512baf96d433484f27234d4b7c11dd9cda0d6f56fa1b7ce6f25f0``.
+ Alternatively, you can also use :doc:`bblock </dev-manual/bblock>` to
+ generate this line for you.
+
:term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`
Specifies the debug level of task signature check. 3 levels are supported:
--
2.47.0
^ permalink raw reply related [flat|nested] 7+ messages in thread