[PATCH 0/5] Add bblock documentation

[PATCH 0/5] Add bblock documentation

[Julien Stephan]

Hello,

The main purpose of this series is to add documentation for bblock.

bblock uses internally the 3 following variables: SIGGEN_LOCKEDSIGS,
SIGGEN_LOCKEDSIGS_TASKSIG_CHECK and SIGGEN_LOCKEDSIGS_TYPE which are not
documentated, so adding them to the glossary.

This series also adds SPHINXLINTDOCS variable in documentation/Makefile to
help cheking subset of the documentation.

documentation/README is also updated to show how to use SHPHINXLINTDOCS and
VALEDOCS to help user sylecheck and lintcheck their own documentation file.

I also added "sstate" into the Yocto vocabulary to avoir error when running
Vale.

This series was built and checked with:

  $ make html
  $ make stylecheck VALEDOCS=dev-manual/bblock.rst
  $ make sphinx-lint SPHINXLINTDOCS=dev-manual/bblock.rst

I tried my best to make Vale happy, but I still have the following
warnings:

 dev-manual/bblock.rst
 1:1  warning     Try to keep the Coleman–Liau Index grade (10.29) below 9.                           Readability.ColemanLiau
 1:1  warning     Try to keep the Flesch reading ease score (57.51) above 70.                         Readability.FleschReadingEase
 1:1  warning     Try to keep the Flesch–Kincaid grade level (8.28) below 8.                          Readability.FleschKincaid
 1:1  warning     Try to keep the SMOG grade (11.42) below 10.                                        Readability.SMOG
 1:1  warning     Try to keep the LIX score (38.20) below 35.                                         Readability.LIX
 3:1  suggestion  Use sentence-style capitalization in 'Locking and Unlocking Recipes Using ******'.  RedHat.Headings

About the Readability, I am not sure I can do better ^^ But anyone with
better english skills than me can help here!

About the capitalization: I decided to ignore it to keep consistency with
all other headings of the reference manual.

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
---
Julien Stephan (5):
      README: add instruction to run Vale on a subset
      documentation: Makefile: add SPHINXLINTDOCS to specify subset to sphinx-lint
      styles: vocabularies: Yocto: add sstate
      ref-manual: variables: add SIGGEN_LOCKEDSIGS* variables
      dev-manual: add bblock documentation

 documentation/Makefile                             |  23 ++--
 documentation/README                               |  16 ++-
 documentation/dev-manual/bblock.rst                | 129 +++++++++++++++++++++
 documentation/dev-manual/index.rst                 |   1 +
 documentation/ref-manual/variables.rst             |  33 ++++++
 .../styles/config/vocabularies/Yocto/accept.txt    |   1 +
 6 files changed, 191 insertions(+), 12 deletions(-)
---
base-commit: 9563855ccd92e21fb6f8320c96a3a83e115c947e
change-id: 20241031-add-bblock-documentation-4a5f8cf9c6d2

Best regards,
-- 
Julien Stephan <jstephan@baylibre.com>

[PATCH 1/5] README: add instruction to run Vale on a subset

[Julien Stephan]

make stylecheck runs Vale on the whole documentation which can be long
and reports a lot of errors/warnings. We can run Vale on a subset using
the VALEDOCS variable, so update documentation to highlight it.

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
---
 documentation/README | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/documentation/README b/documentation/README
index c888b666a8632a1d6f576c68e4f9ca966d79f134..6a4db97d65b9c23828163d8b548485b2e302bc97 100644
--- a/documentation/README
+++ b/documentation/README
@@ -142,6 +142,13 @@ To run Vale:
 
  $ make stylecheck
 
+Style checking the whole documentation might take some time and generate a
+lot of warnings/errors, thus one can run Vale on a subset:
+
+  $ make stylecheck VALEDOCS=<file>
+  $ make stylecheck VALEDOCS="<file1> <file2>"
+  $ make stylecheck VALEDOCS=<dir>
+
 Link checking the Yocto Project documentation
 =============================================
 

-- 
2.47.0

[PATCH 2/5] documentation: Makefile: add SPHINXLINTDOCS to specify subset to sphinx-lint

[Julien Stephan]

make sphinx-lint runs sphinx-lint on the whole documentation which can be
long and reports a lot or errors/warnings. Let's add a new
SHPINXLINTDOCS variable to allow specifying a subset, just as VALEDOCS
does. Keep variable assignment aligned and also use $(SOURCEDIR) by
default for SPHINXLINTDOCS and VALEDOCS variables.

Also update the README file and fix a typo in Link checking section
title.

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
---
 documentation/Makefile | 23 ++++++++++++-----------
 documentation/README   |  9 ++++++++-
 2 files changed, 20 insertions(+), 12 deletions(-)

diff --git a/documentation/Makefile b/documentation/Makefile
index 189bd1dfac69d3b3185aa3a2ff7c212084ae280b..0bd9db5eaeb9122f9f396cda61631c403a1325a0 100644
--- a/documentation/Makefile
+++ b/documentation/Makefile
@@ -3,17 +3,18 @@
 
 # You can set these variables from the command line, and also
 # from the environment for the first two.
-SPHINXOPTS    ?= -W --keep-going -j auto
-SPHINXBUILD   ?= sphinx-build
+SPHINXOPTS     ?= -W --keep-going -j auto
+SPHINXBUILD    ?= sphinx-build
 # Release notes are excluded because they contain contributor names and commit messages which can't be modified
-VALEOPTS      ?= --no-wrap --glob '!migration-guides/release-notes-*.rst'
-VALEDOCS      ?= .
-SOURCEDIR     = .
-IMAGEDIRS     = */svg
-BUILDDIR      = _build
-DESTDIR       = final
-SVG2PNG       = inkscape
-SVG2PDF       = inkscape
+VALEOPTS       ?= --no-wrap --glob '!migration-guides/release-notes-*.rst'
+SOURCEDIR      = .
+VALEDOCS       ?= $(SOURCEDIR)
+SPHINXLINTDOCS ?= $(SOURCEDIR)
+IMAGEDIRS      = */svg
+BUILDDIR       = _build
+DESTDIR        = final
+SVG2PNG        = inkscape
+SVG2PDF        = inkscape
 
 ifeq ($(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi),0)
 $(error "The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed")
@@ -54,7 +55,7 @@ stylecheck:
 	vale $(VALEOPTS) $(VALEDOCS)
 
 sphinx-lint:
-	sphinx-lint $(SOURCEDIR)
+	sphinx-lint $(SPHINXLINTDOCS)
 
 epub: $(PNGs)
 	$(SOURCEDIR)/set_versions.py
diff --git a/documentation/README b/documentation/README
index 6a4db97d65b9c23828163d8b548485b2e302bc97..d2676ae2de9cb9c2dbe4191c0e38f8847cd8603b 100644
--- a/documentation/README
+++ b/documentation/README
@@ -149,7 +149,7 @@ lot of warnings/errors, thus one can run Vale on a subset:
   $ make stylecheck VALEDOCS="<file1> <file2>"
   $ make stylecheck VALEDOCS=<dir>
 
-Link checking the Yocto Project documentation
+Lint checking the Yocto Project documentation
 =============================================
 
 To fix errors which are not reported by Sphinx itself,
@@ -163,6 +163,13 @@ To run sphinx-lint:
 
  $ make sphinx-lint
 
+Lint checking the whole documentation might take some time and generate a
+lot of warnings/errors, thus one can run sphinx-lint on a specific file:
+
+  $ make sphinx-lint SPHINXLINTDOCS=<file>
+  $ make sphinx-lint SPHINXLINTDOCS="<file1> <file2>"
+  $ make sphinx-lint SPHINXLINTDOCS=<dir>
+
 Sphinx theme and CSS customization
 ==================================
 

-- 
2.47.0

[PATCH 3/5] styles: vocabularies: Yocto: add sstate

[Julien Stephan]

Add sstate as an accepted word to avoid errors when runnign make
stylecheck.

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
---
 documentation/styles/config/vocabularies/Yocto/accept.txt | 1 +
 1 file changed, 1 insertion(+)

diff --git a/documentation/styles/config/vocabularies/Yocto/accept.txt b/documentation/styles/config/vocabularies/Yocto/accept.txt
index ca622ba41274c892f1fc1e16d8332a02f07a3531..7fedda8ae23c22f3660292faa0a1c3df23da353e 100644
--- a/documentation/styles/config/vocabularies/Yocto/accept.txt
+++ b/documentation/styles/config/vocabularies/Yocto/accept.txt
@@ -2,4 +2,5 @@ BitBake
 BSP
 crosstap
 OpenEmbedded
+sstate
 Yocto

-- 
2.47.0

[PATCH 4/5] ref-manual: variables: add SIGGEN_LOCKEDSIGS* variables

[Julien Stephan]

Variables SIGGEN_LOCKEDSIGS, SIGGEN_LOCKEDSIGS_TASKSIG_CHECK and
SIGGEN_LOCKEDSIGS_TYPES are used to lock specific tasks to specific
signatures. They are used by bitbake -S <lockedsigs> and bblock, so add
documentation for them.

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
---
 documentation/ref-manual/variables.rst | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 07ed143ac5606617c720301ac6621355a341f90a..8cfb978e795e43e3ef25f3e18e11baf7c2a28d54 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -7880,6 +7880,39 @@ system and gives an overview of their function and contents.
          might break at runtime if the interface of the recipe was changed
          after the other had been built.
 
+   :term:`SIGGEN_LOCKEDSIGS`
+     The list of locked taks, with the form::
+
+       SIGGEN_LOCKEDSIGS += "<package>:<task>:<signature>"
+
+     If ``<signature>`` exists for the specified ``<task>`` and ``<package>``
+     in sstate, BitBake will use the cached output instead of rebuilding the
+     ``<task>``. If it does not exist, BitBake will build the ``<task>`` and
+     the sstate will be used next time.
+
+   :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`
+     Specifies the debug level of task signature check. 3 levels are supported:
+
+     * info: displays a "Note" message to remind user that a task is locked
+       and current signature matches the locked one.
+     * warn: displays a "Warning" message if a task is locked and current
+       signature does not match the locked one.
+     * error: same as warn but displays an “Error” message and abort.
+
+   :term:`SIGGEN_LOCKEDSIGS_TYPES`
+     Allowed overrides for :term:`SIGGEN_LOCKEDSIGS`. This is mainly used
+     for achitecture specific lock. A common value for :term:`SIGGEN_LOCKEDSIGS_TYPES`
+     is ``${PACKAGE_ARCHS}``::
+
+       SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
+
+       SIGGEN_LOCKEDSIGS_core2-64 += "bc:do_compile:abcd"
+       SIGGEN_LOCKEDSIGS_cortexa57 += "bc:do_compile:efgh"
+
+     Here, the ``do_compile`` task from ``bc`` will be locked only for
+     ``x86-64`` and ``arm64`` but not for other architectures such as
+     ``qemumips``.
+
    :term:`SITEINFO_BITS`
       Specifies the number of bits for the target system CPU. The value
       should be either "32" or "64".

-- 
2.47.0

[PATCH 5/5] dev-manual: add bblock documentation

[Julien Stephan]

bblock is a helper tool to lock/unlock tasks and recipes to specific
signatures. Add a documentation page for it.

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
---
 documentation/dev-manual/bblock.rst | 129 ++++++++++++++++++++++++++++++++++++
 documentation/dev-manual/index.rst  |   1 +
 2 files changed, 130 insertions(+)

diff --git a/documentation/dev-manual/bblock.rst b/documentation/dev-manual/bblock.rst
new file mode 100644
index 0000000000000000000000000000000000000000..3df75799aa638d173764607df191bd99612034cf
--- /dev/null
+++ b/documentation/dev-manual/bblock.rst
@@ -0,0 +1,129 @@
+.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
+
+Locking and Unlocking Recipes Using ``bblock``
+**********************************************
+
+By design, the OpenEmbedded build system builds everything from scratch
+unless BitBake determines that specific parts do not require rebuilding.
+At startup, it computes a signature for each tasks, based on the task's input.
+Then, it compares these signatures with the ones from sstate (if exist). Any
+changes cause the task to rerun.
+
+During development, changes might trigger BitBake to rebuild certain
+recipes, even when we know they do not require rebuilding at that stage.
+For example, modifying a recipe can lead to the rebuilding of its native
+counterpart, which might proves unnecessary. Editing the ``python3`` recipe,
+for instance, can prompt BitBake to rebuild ``python3-native`` along with any
+recipes that depend on it.
+
+To prevent this, use ``bblock`` to lock specific tasks or recipes to
+specific signatures, forcing BitBake to use the sstate cache for them.
+
+.. attention::
+
+   Use ``bblock`` only during the development phase.
+
+   Forcing BitBake to use sstate cache, regardless of input changes, means
+   the recipe metadata no longer directly reflect the output. Use this feature
+   with caution. If you do not understand why signatures change, see the section
+   on `understanding what changed <https://wiki.yoctoproject.org/wiki/TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`__.
+
+
+Locking tasks and recipes
+-------------------------
+
+To lock a recipe, use::
+
+   $ bblock recipe
+
+You can also use a space-separated list of recipes to lock many recipes::
+
+   $ bblock recipe1 recipe2
+
+Locking a recipe, means locking all tasks of the recipe. If you need to
+lock only particular tasks use the `-t` switch with a comma-separated
+list of tasks::
+
+  $ bblock -t task1,task2 recipe
+
+
+Unlocking tasks and recipes
+---------------------------
+
+To unlock a recipe, use::
+
+   $ bblock -r recipe
+
+You can also use a space-separated list of recipes to unlock many recipes::
+
+   $ bblock -r recipe1 recipe2
+
+Unlocking a recipe, means unlocking all tasks of the recipe. If you need to
+unlock only particular tasks use the `-t` switch with a comma-separated
+list of tasks::
+
+  $ bblock -r -t task1,task2 recipe
+
+To unlock all recipes, do not specify any recipe::
+
+  $ bblock -r
+
+
+Configuration file
+------------------
+
+``bblock`` will dump the signatures in the ``build/conf/bblock.conf`` file,
+included by default in ``poky/meta/conf/bitbake.conf``.
+
+To dump it, use::
+
+  $ bblock -d
+
+
+Locking mechanism
+-----------------
+
+``bblock`` computes the signature(s) of the task(s) then uses the 3 following
+variables: :term:`SIGGEN_LOCKEDSIGS`, :term:`SIGGEN_LOCKEDSIGS_TYPES`
+and :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`.
+
+In particular, ``bblock`` sets::
+
+  SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
+  SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
+
+  SIGGEN_LOCKEDSIGS_<package_arch> += "<recipe>:<task>:<signature>"
+
+This produces architecture specific locks and reminds user that some tasks
+have locked signatures.
+
+Example
+-------
+
+When working on ``python3`` recipe, we can lock ``pyton3-native`` with the
+following::
+
+  $ bblock python3-native
+  $ bblock -d
+  # Generated by bblock
+  SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
+  SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
+
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_patch:865859c27e603ba42025b7bb766c3cd4c0f477e4962cfd39128c0619d695fce7"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_sysroot:f8fa5d3194cef638416000252b959e86d0a19f6b7898e1f56b643c588cdd8605"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_prepare_recipe_sysroot:fe295ac505d9d1143313424b201c6f3f2a0a90da40a13a905b86b874705f226a"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_fetch:1b6e4728fee631bc7a8a7006855c5b8182a8224579e32e3d0a2db77c26459f25"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_unpack:2ad74d6f865ef75c35c0e6bbe3f9a90923a6b2c62c18a3ddef514ea31fbc588f"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_deploy_source_date_epoch:15f89b8483c1ad7507480f337619bb98c26e231227785eb3543db163593e7b42"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_configure:7960c13d23270fdb12b3a7c426ce1da0d2f5c7cf5e5d3f5bdce5fa330eb7d482"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_compile:012e1d4a63f1a78fc2143bd90d704dbcf5865c5257d6272aa7540ec1cd3063d9"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_install:d3401cc2afa4c996beb154beaad3e45fa0272b9c56fb86e9db14ec3544c68f9d"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_build:fa88bb7afb9046c0417c24a3fa98a058653805a8b00eda2c2d7fea68fc42f882"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_collect_spdx_deps:cc9c53ba7c495567e9a38ec4801830c425c0d1f895aa2fc66930a2edd510d9b4"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_spdx:766a1d09368438b7b5a1a8e2a8f823b2b731db44b57e67d8b3196de91966f9c5"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_package_spdx:46f80faeab25575e9977ba3bf14c819489c3d489432ae5145255635108c21020"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_recipe_qa:cb960cdb074e7944e894958db58f3dc2a0436ecf87c247feb3e095e214fec0e4"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_lic:15657441621ee83f15c2e650e7edbb036870b56f55e72e046c6142da3c5783fd"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_manifest:24f0abbec221d27bbb2909b6e846288b12cab419f1faf9f5006ed80423d37e28"
+  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_addto_recipe_sysroot:bcb6a1905f113128de3f88d702b706befd6a786267c045ee82532759a7c214d7"
+
diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst
index 9ccf60f7012ea344612298e77ebabed84ad6de3e..7afd0d820e94d56cb2145962b9b1a976c7951936 100644
--- a/documentation/dev-manual/index.rst
+++ b/documentation/dev-manual/index.rst
@@ -48,5 +48,6 @@ Yocto Project Development Tasks Manual
    error-reporting-tool
    wayland
    qemu
+   bblock
 
 .. include:: /boilerplate.rst

-- 
2.47.0

Re: [docs] [PATCH 5/5] dev-manual: add bblock documentation

[Ulrich Ölmann]

Hi Julien,

great to have that documented!
Just some typos I came across.

On Thu, Oct 31 2024 at 17:56 +0100, "Julien Stephan" <jstephan@baylibre.com> wrote:
> bblock is a helper tool to lock/unlock tasks and recipes to specific
> signatures. Add a documentation page for it.
>
> Signed-off-by: Julien Stephan <jstephan@baylibre.com>
> ---
>  documentation/dev-manual/bblock.rst | 129 ++++++++++++++++++++++++++++++++++++
>  documentation/dev-manual/index.rst  |   1 +
>  2 files changed, 130 insertions(+)
>
> diff --git a/documentation/dev-manual/bblock.rst b/documentation/dev-manual/bblock.rst
> new file mode 100644
> index 0000000000000000000000000000000000000000..3df75799aa638d173764607df191bd99612034cf
> --- /dev/null
> +++ b/documentation/dev-manual/bblock.rst
> @@ -0,0 +1,129 @@
> +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
> +
> +Locking and Unlocking Recipes Using ``bblock``
> +**********************************************
> +
> +By design, the OpenEmbedded build system builds everything from scratch
> +unless BitBake determines that specific parts do not require rebuilding.
> +At startup, it computes a signature for each tasks, based on the task's input.

s/for each tasks/for each task/   or perhaps    s/for each tasks/for all tasks/

> +Then, it compares these signatures with the ones from sstate (if exist). Any

s/if exist/if they exist/   or   s/if exist/if it exists/   depending on
if you meant the signatures or the sstate to exist.

> +changes cause the task to rerun.
> +
> +During development, changes might trigger BitBake to rebuild certain
> +recipes, even when we know they do not require rebuilding at that stage.
> +For example, modifying a recipe can lead to the rebuilding of its native

s/to the rebuilding/to rebuilding/

> +counterpart, which might proves unnecessary. Editing the ``python3`` recipe,

s/which might proves/which might prove/

> +for instance, can prompt BitBake to rebuild ``python3-native`` along with any
> +recipes that depend on it.
> +
> +To prevent this, use ``bblock`` to lock specific tasks or recipes to
> +specific signatures, forcing BitBake to use the sstate cache for them.
> +
> +.. attention::
> +
> +   Use ``bblock`` only during the development phase.
> +
> +   Forcing BitBake to use sstate cache, regardless of input changes, means
> +   the recipe metadata no longer directly reflect the output. Use this feature
> +   with caution. If you do not understand why signatures change, see the section
> +   on `understanding what changed <https://wiki.yoctoproject.org/wiki/TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`__.
> +
> +
> +Locking tasks and recipes
> +-------------------------
> +
> +To lock a recipe, use::
> +
> +   $ bblock recipe
> +
> +You can also use a space-separated list of recipes to lock many recipes::
> +
> +   $ bblock recipe1 recipe2
> +
> +Locking a recipe, means locking all tasks of the recipe. If you need to

s/Locking a recipe, means locking/Locking a recipe means locking/

> +lock only particular tasks use the `-t` switch with a comma-separated
> +list of tasks::
> +
> +  $ bblock -t task1,task2 recipe
> +
> +
> +Unlocking tasks and recipes
> +---------------------------
> +
> +To unlock a recipe, use::
> +
> +   $ bblock -r recipe
> +
> +You can also use a space-separated list of recipes to unlock many recipes::
> +
> +   $ bblock -r recipe1 recipe2
> +
> +Unlocking a recipe, means unlocking all tasks of the recipe. If you need to

s/Unlocking a recipe, means unlocking/Unlocking a recipe means unlocking/

> +unlock only particular tasks use the `-t` switch with a comma-separated
> +list of tasks::
> +
> +  $ bblock -r -t task1,task2 recipe
> +
> +To unlock all recipes, do not specify any recipe::
> +
> +  $ bblock -r
> +
> +
> +Configuration file
> +------------------
> +
> +``bblock`` will dump the signatures in the ``build/conf/bblock.conf`` file,
> +included by default in ``poky/meta/conf/bitbake.conf``.
> +
> +To dump it, use::
> +
> +  $ bblock -d
> +
> +
> +Locking mechanism
> +-----------------
> +
> +``bblock`` computes the signature(s) of the task(s) then uses the 3 following
> +variables: :term:`SIGGEN_LOCKEDSIGS`, :term:`SIGGEN_LOCKEDSIGS_TYPES`
> +and :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`.
> +
> +In particular, ``bblock`` sets::
> +
> +  SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
> +  SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
> +
> +  SIGGEN_LOCKEDSIGS_<package_arch> += "<recipe>:<task>:<signature>"
> +
> +This produces architecture specific locks and reminds user that some tasks

s/reminds user/reminds the user/

Best regards
Ulrich


> +have locked signatures.
> +
> +Example
> +-------
> +
> +When working on ``python3`` recipe, we can lock ``pyton3-native`` with the
> +following::
> +
> +  $ bblock python3-native
> +  $ bblock -d
> +  # Generated by bblock
> +  SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
> +  SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
> +
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_patch:865859c27e603ba42025b7bb766c3cd4c0f477e4962cfd39128c0619d695fce7"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_sysroot:f8fa5d3194cef638416000252b959e86d0a19f6b7898e1f56b643c588cdd8605"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_prepare_recipe_sysroot:fe295ac505d9d1143313424b201c6f3f2a0a90da40a13a905b86b874705f226a"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_fetch:1b6e4728fee631bc7a8a7006855c5b8182a8224579e32e3d0a2db77c26459f25"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_unpack:2ad74d6f865ef75c35c0e6bbe3f9a90923a6b2c62c18a3ddef514ea31fbc588f"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_deploy_source_date_epoch:15f89b8483c1ad7507480f337619bb98c26e231227785eb3543db163593e7b42"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_configure:7960c13d23270fdb12b3a7c426ce1da0d2f5c7cf5e5d3f5bdce5fa330eb7d482"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_compile:012e1d4a63f1a78fc2143bd90d704dbcf5865c5257d6272aa7540ec1cd3063d9"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_install:d3401cc2afa4c996beb154beaad3e45fa0272b9c56fb86e9db14ec3544c68f9d"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_build:fa88bb7afb9046c0417c24a3fa98a058653805a8b00eda2c2d7fea68fc42f882"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_collect_spdx_deps:cc9c53ba7c495567e9a38ec4801830c425c0d1f895aa2fc66930a2edd510d9b4"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_spdx:766a1d09368438b7b5a1a8e2a8f823b2b731db44b57e67d8b3196de91966f9c5"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_package_spdx:46f80faeab25575e9977ba3bf14c819489c3d489432ae5145255635108c21020"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_recipe_qa:cb960cdb074e7944e894958db58f3dc2a0436ecf87c247feb3e095e214fec0e4"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_lic:15657441621ee83f15c2e650e7edbb036870b56f55e72e046c6142da3c5783fd"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_manifest:24f0abbec221d27bbb2909b6e846288b12cab419f1faf9f5006ed80423d37e28"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_addto_recipe_sysroot:bcb6a1905f113128de3f88d702b706befd6a786267c045ee82532759a7c214d7"
> +
> diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst
> index 9ccf60f7012ea344612298e77ebabed84ad6de3e..7afd0d820e94d56cb2145962b9b1a976c7951936 100644
> --- a/documentation/dev-manual/index.rst
> +++ b/documentation/dev-manual/index.rst
> @@ -48,5 +48,6 @@ Yocto Project Development Tasks Manual
>     error-reporting-tool
>     wayland
>     qemu
> +   bblock
>  
>  .. include:: /boilerplate.rst
-- 
Pengutronix e.K.                           | Ulrich Ölmann               |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [docs] [PATCH 4/5] ref-manual: variables: add SIGGEN_LOCKEDSIGS* variables

[Ulrich Ölmann]

Hi Julien,

thanks for your documentation work!
Please find some typo fixes below.

On Thu, Oct 31 2024 at 17:56 +0100, "Julien Stephan" <jstephan@baylibre.com> wrote:
> Variables SIGGEN_LOCKEDSIGS, SIGGEN_LOCKEDSIGS_TASKSIG_CHECK and
> SIGGEN_LOCKEDSIGS_TYPES are used to lock specific tasks to specific
> signatures. They are used by bitbake -S <lockedsigs> and bblock, so add
> documentation for them.
>
> Signed-off-by: Julien Stephan <jstephan@baylibre.com>
> ---
>  documentation/ref-manual/variables.rst | 33 +++++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
>
> diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
> index 07ed143ac5606617c720301ac6621355a341f90a..8cfb978e795e43e3ef25f3e18e11baf7c2a28d54 100644
> --- a/documentation/ref-manual/variables.rst
> +++ b/documentation/ref-manual/variables.rst
> @@ -7880,6 +7880,39 @@ system and gives an overview of their function and contents.
>           might break at runtime if the interface of the recipe was changed
>           after the other had been built.
>
> +   :term:`SIGGEN_LOCKEDSIGS`
> +     The list of locked taks, with the form::
> +
> +       SIGGEN_LOCKEDSIGS += "<package>:<task>:<signature>"
> +
> +     If ``<signature>`` exists for the specified ``<task>`` and ``<package>``
> +     in sstate, BitBake will use the cached output instead of rebuilding the
> +     ``<task>``. If it does not exist, BitBake will build the ``<task>`` and
> +     the sstate will be used next time.
> +
> +   :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`
> +     Specifies the debug level of task signature check. 3 levels are supported:
> +
> +     * info: displays a "Note" message to remind user that a task is locked

s/to remind user/to remind the user/

> +       and current signature matches the locked one.

s/and current signature/and the current signature/

> +     * warn: displays a "Warning" message if a task is locked and current

s/and current/and the current/

> +       signature does not match the locked one.
> +     * error: same as warn but displays an “Error” message and abort.

s/and abort./and aborts./

> +
> +   :term:`SIGGEN_LOCKEDSIGS_TYPES`
> +     Allowed overrides for :term:`SIGGEN_LOCKEDSIGS`. This is mainly used
> +     for achitecture specific lock. A common value for :term:`SIGGEN_LOCKEDSIGS_TYPES`

s/for achitecture specific lock./for achitecture specific locks./

Best regards
Ulrich

> +     is ``${PACKAGE_ARCHS}``::
> +
> +       SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
> +
> +       SIGGEN_LOCKEDSIGS_core2-64 += "bc:do_compile:abcd"
> +       SIGGEN_LOCKEDSIGS_cortexa57 += "bc:do_compile:efgh"
> +
> +     Here, the ``do_compile`` task from ``bc`` will be locked only for
> +     ``x86-64`` and ``arm64`` but not for other architectures such as
> +     ``qemumips``.
> +
>     :term:`SITEINFO_BITS`
>        Specifies the number of bits for the target system CPU. The value
>        should be either "32" or "64".
-- 
Pengutronix e.K.                           | Ulrich Ölmann               |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [docs] [PATCH 1/5] README: add instruction to run Vale on a subset

[Antonin Godard]

Hi Julien,

Thanks for your patches.

On Thu Oct 31, 2024 at 5:56 PM CET, Julien Stephan wrote:
> make stylecheck runs Vale on the whole documentation which can be long
> and reports a lot of errors/warnings. We can run Vale on a subset using
> the VALEDOCS variable, so update documentation to highlight it.
>
> Signed-off-by: Julien Stephan <jstephan@baylibre.com>
> ---
>  documentation/README | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/documentation/README b/documentation/README
> index c888b666a8632a1d6f576c68e4f9ca966d79f134..6a4db97d65b9c23828163d8b548485b2e302bc97 100644
> --- a/documentation/README
> +++ b/documentation/README
> @@ -142,6 +142,13 @@ To run Vale:
>  
>   $ make stylecheck
>  
> +Style checking the whole documentation might take some time and generate a
> +lot of warnings/errors, thus one can run Vale on a subset:

s/on a subset:/on a subset of files or directories:/ I would say

> +  $ make stylecheck VALEDOCS=<file>
> +  $ make stylecheck VALEDOCS="<file1> <file2>"
> +  $ make stylecheck VALEDOCS=<dir>
> +
>  Link checking the Yocto Project documentation
>  =============================================
>  

Cheers,
Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH 2/5] documentation: Makefile: add SPHINXLINTDOCS to specify subset to sphinx-lint

[Antonin Godard]

Hi Julien,

On Thu Oct 31, 2024 at 5:56 PM CET, Julien Stephan wrote:
> make sphinx-lint runs sphinx-lint on the whole documentation which can be
> long and reports a lot or errors/warnings. Let's add a new
> SHPINXLINTDOCS variable to allow specifying a subset, just as VALEDOCS
> does. Keep variable assignment aligned and also use $(SOURCEDIR) by
> default for SPHINXLINTDOCS and VALEDOCS variables.
>
> Also update the README file and fix a typo in Link checking section
> title.
>
> Signed-off-by: Julien Stephan <jstephan@baylibre.com>
> ---
>  documentation/Makefile | 23 ++++++++++++-----------
>  documentation/README   |  9 ++++++++-
>  2 files changed, 20 insertions(+), 12 deletions(-)
>
> diff --git a/documentation/Makefile b/documentation/Makefile
> index 189bd1dfac69d3b3185aa3a2ff7c212084ae280b..0bd9db5eaeb9122f9f396cda61631c403a1325a0 100644
> --- a/documentation/Makefile
> +++ b/documentation/Makefile
> @@ -3,17 +3,18 @@
>  
>  # You can set these variables from the command line, and also
>  # from the environment for the first two.
> -SPHINXOPTS    ?= -W --keep-going -j auto
> -SPHINXBUILD   ?= sphinx-build
> +SPHINXOPTS     ?= -W --keep-going -j auto
> +SPHINXBUILD    ?= sphinx-build
>  # Release notes are excluded because they contain contributor names and commit messages which can't be modified
> -VALEOPTS      ?= --no-wrap --glob '!migration-guides/release-notes-*.rst'
> -VALEDOCS      ?= .
> -SOURCEDIR     = .
> -IMAGEDIRS     = */svg
> -BUILDDIR      = _build
> -DESTDIR       = final
> -SVG2PNG       = inkscape
> -SVG2PDF       = inkscape
> +VALEOPTS       ?= --no-wrap --glob '!migration-guides/release-notes-*.rst'
> +SOURCEDIR      = .
> +VALEDOCS       ?= $(SOURCEDIR)
> +SPHINXLINTDOCS ?= $(SOURCEDIR)
> +IMAGEDIRS      = */svg
> +BUILDDIR       = _build
> +DESTDIR        = final
> +SVG2PNG        = inkscape
> +SVG2PDF        = inkscape
>  
>  ifeq ($(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi),0)
>  $(error "The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed")
> @@ -54,7 +55,7 @@ stylecheck:
>  	vale $(VALEOPTS) $(VALEDOCS)
>  
>  sphinx-lint:
> -	sphinx-lint $(SOURCEDIR)
> +	sphinx-lint $(SPHINXLINTDOCS)
>  
>  epub: $(PNGs)
>  	$(SOURCEDIR)/set_versions.py
> diff --git a/documentation/README b/documentation/README
> index 6a4db97d65b9c23828163d8b548485b2e302bc97..d2676ae2de9cb9c2dbe4191c0e38f8847cd8603b 100644
> --- a/documentation/README
> +++ b/documentation/README
> @@ -149,7 +149,7 @@ lot of warnings/errors, thus one can run Vale on a subset:
>    $ make stylecheck VALEDOCS="<file1> <file2>"
>    $ make stylecheck VALEDOCS=<dir>
>  
> -Link checking the Yocto Project documentation
> +Lint checking the Yocto Project documentation
>  =============================================
>  
>  To fix errors which are not reported by Sphinx itself,
> @@ -163,6 +163,13 @@ To run sphinx-lint:
>  
>   $ make sphinx-lint
>  
> +Lint checking the whole documentation might take some time and generate a
> +lot of warnings/errors, thus one can run sphinx-lint on a specific file:

s/on a specific file:/on a subset of files or directories:/

> +
> +  $ make sphinx-lint SPHINXLINTDOCS=<file>
> +  $ make sphinx-lint SPHINXLINTDOCS="<file1> <file2>"
> +  $ make sphinx-lint SPHINXLINTDOCS=<dir>
> +
>  Sphinx theme and CSS customization
>  ==================================
>  

Cheers,
Antonin

--
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH 4/5] ref-manual: variables: add SIGGEN_LOCKEDSIGS* variables

[Antonin Godard]

Hi Julien,

On top of Ulrich's comments, here are my suggestions/corrections:

On Thu Oct 31, 2024 at 5:56 PM CET, Julien Stephan wrote:
> Variables SIGGEN_LOCKEDSIGS, SIGGEN_LOCKEDSIGS_TASKSIG_CHECK and
> SIGGEN_LOCKEDSIGS_TYPES are used to lock specific tasks to specific
> signatures. They are used by bitbake -S <lockedsigs> and bblock, so add
> documentation for them.
>
> Signed-off-by: Julien Stephan <jstephan@baylibre.com>
> ---
>  documentation/ref-manual/variables.rst | 33 +++++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
>
> diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
> index 07ed143ac5606617c720301ac6621355a341f90a..8cfb978e795e43e3ef25f3e18e11baf7c2a28d54 100644
> --- a/documentation/ref-manual/variables.rst
> +++ b/documentation/ref-manual/variables.rst
> @@ -7880,6 +7880,39 @@ system and gives an overview of their function and contents.
>           might break at runtime if the interface of the recipe was changed
>           after the other had been built.
>  
> +   :term:`SIGGEN_LOCKEDSIGS`
> +     The list of locked taks, with the form::
> +
> +       SIGGEN_LOCKEDSIGS += "<package>:<task>:<signature>"
> +
> +     If ``<signature>`` exists for the specified ``<task>`` and ``<package>``

How would I find such a signature by hand? Maybe worth explaining it here?

> +     in sstate, BitBake will use the cached output instead of rebuilding the
> +     ``<task>``. If it does not exist, BitBake will build the ``<task>`` and
> +     the sstate will be used next time.

Could you give a real-life example of that at the end here?

> +   :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`
> +     Specifies the debug level of task signature check. 3 levels are supported:
> +
> +     * info: displays a "Note" message to remind user that a task is locked
> +       and current signature matches the locked one.
> +     * warn: displays a "Warning" message if a task is locked and current
> +       signature does not match the locked one.
> +     * error: same as warn but displays an “Error” message and abort.

For "info", "warn" and "error" could you enclose that in double ticks? ``info``
and so on.

> +   :term:`SIGGEN_LOCKEDSIGS_TYPES`
> +     Allowed overrides for :term:`SIGGEN_LOCKEDSIGS`. This is mainly used
> +     for achitecture specific lock. A common value for :term:`SIGGEN_LOCKEDSIGS_TYPES`

s/achitecture/architecture/

By "mainly", do you mean "only" or could there by anything other than
architectures covered by the variable?

> +     is ``${PACKAGE_ARCHS}``::
> +
> +       SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
> +
> +       SIGGEN_LOCKEDSIGS_core2-64 += "bc:do_compile:abcd"
> +       SIGGEN_LOCKEDSIGS_cortexa57 += "bc:do_compile:efgh"

Maybe real hashes would be better here

> +
> +     Here, the ``do_compile`` task from ``bc`` will be locked only for
> +     ``x86-64`` and ``arm64`` but not for other architectures such as

Wouldn't it be locked for ``core2-64`` and ``cortexa57`` more specifically? I
might misunderstand something here. :)

> +     ``qemumips``.
> +
>     :term:`SITEINFO_BITS`
>        Specifies the number of bits for the target system CPU. The value
>        should be either "32" or "64".

Also I could suggest splitting the series in patches for sphinx-lint/vale and
patches for documentation of variables/bblock.

Thanks!
Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH 5/5] dev-manual: add bblock documentation

[Antonin Godard]

Hi Julien,

Thanks for taking the time to document your tool!
On top of Ulrich's comments, here are my suggestions/corrections:

On Thu Oct 31, 2024 at 5:56 PM CET, Julien Stephan wrote:
> bblock is a helper tool to lock/unlock tasks and recipes to specific
> signatures. Add a documentation page for it.
>
> Signed-off-by: Julien Stephan <jstephan@baylibre.com>
> ---
>  documentation/dev-manual/bblock.rst | 129 ++++++++++++++++++++++++++++++++++++
>  documentation/dev-manual/index.rst  |   1 +
>  2 files changed, 130 insertions(+)
>
> diff --git a/documentation/dev-manual/bblock.rst b/documentation/dev-manual/bblock.rst
> new file mode 100644
> index 0000000000000000000000000000000000000000..3df75799aa638d173764607df191bd99612034cf
> --- /dev/null
> +++ b/documentation/dev-manual/bblock.rst
> @@ -0,0 +1,129 @@
> +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
> +
> +Locking and Unlocking Recipes Using ``bblock``
> +**********************************************
> +
> +By design, the OpenEmbedded build system builds everything from scratch
> +unless BitBake determines that specific parts do not require rebuilding.

s/parts/tasks/?

> +At startup, it computes a signature for each tasks, based on the task's input.
> +Then, it compares these signatures with the ones from sstate (if exist). Any

s/from sstate/from the sstate cache/?
s/if exist/if they exist/

> +changes cause the task to rerun.
> +
> +During development, changes might trigger BitBake to rebuild certain
> +recipes, even when we know they do not require rebuilding at that stage.
> +For example, modifying a recipe can lead to the rebuilding of its native
> +counterpart, which might proves unnecessary. Editing the ``python3`` recipe,
> +for instance, can prompt BitBake to rebuild ``python3-native`` along with any
> +recipes that depend on it.
> +
> +To prevent this, use ``bblock`` to lock specific tasks or recipes to
> +specific signatures, forcing BitBake to use the sstate cache for them.
> +
> +.. attention::
> +
> +   Use ``bblock`` only during the development phase.
> +
> +   Forcing BitBake to use sstate cache, regardless of input changes, means
> +   the recipe metadata no longer directly reflect the output. Use this feature

s/reflect/reflects/ unless metadata is a plural by default?

> +   with caution. If you do not understand why signatures change, see the section
> +   on `understanding what changed <https://wiki.yoctoproject.org/wiki/TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`__.

You can use :yocto_wiki: here:

:yocto_wiki:`understanding what changed </TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`.

(syntax needs double-checking ;))

> +
> +Locking tasks and recipes
> +-------------------------
> +
> +To lock a recipe, use::
> +
> +   $ bblock recipe
> +
> +You can also use a space-separated list of recipes to lock many recipes::
> +
> +   $ bblock recipe1 recipe2
> +
> +Locking a recipe, means locking all tasks of the recipe. If you need to

No comma "," between recipe and means, I think.

> +lock only particular tasks use the `-t` switch with a comma-separated

Comma after "tasks" I think would make more sense here (to me at least).

> +list of tasks::
> +
> +  $ bblock -t task1,task2 recipe
> +

Maybe a real-life example would be great here!

> +Unlocking tasks and recipes
> +---------------------------
> +
> +To unlock a recipe, use::

...use the ``-r`` option::

> +
> +   $ bblock -r recipe
> +
> +You can also use a space-separated list of recipes to unlock many recipes::

s/many/multiple/ (many means "a lot of")

> +
> +   $ bblock -r recipe1 recipe2
> +
> +Unlocking a recipe, means unlocking all tasks of the recipe. If you need to
> +unlock only particular tasks use the `-t` switch with a comma-separated
> +list of tasks::
> +
> +  $ bblock -r -t task1,task2 recipe
> +
> +To unlock all recipes, do not specify any recipe::
> +
> +  $ bblock -r
> +
> +
> +Configuration file
> +------------------
> +
> +``bblock`` will dump the signatures in the ``build/conf/bblock.conf`` file,
> +included by default in ``poky/meta/conf/bitbake.conf``.

Here you can use:

:oe_git:`meta/conf/bitbake.conf </openembedded-core/tree/meta/conf/bitbake.conf>`

Also, I believe that bblock.conf is a candidate for being in
`documentation/ref-manual/structure.rst`, alongside local.conf and
bblayers.conf in the section "The Build Directory --- ``build/``". Could you add
a section there that explains this file briefly and that redirects the user to
this document?

> +
> +To dump it, use::

s/To dump it/To dump the file/

> +
> +  $ bblock -d
> +
> +
> +Locking mechanism
> +-----------------
> +
> +``bblock`` computes the signature(s) of the task(s) then uses the 3 following

s/then uses/and sets/? I.e. explain what it really does with them.

> +variables: :term:`SIGGEN_LOCKEDSIGS`, :term:`SIGGEN_LOCKEDSIGS_TYPES`
> +and :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`.
> +
> +In particular, ``bblock`` sets::
> +
> +  SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
> +  SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
> +
> +  SIGGEN_LOCKEDSIGS_<package_arch> += "<recipe>:<task>:<signature>"
> +
> +This produces architecture specific locks and reminds user that some tasks
> +have locked signatures.
> +
> +Example
> +-------
> +
> +When working on ``python3`` recipe, we can lock ``pyton3-native`` with the

s/on ``python3`` recipe/on the ``python3`` recipe/
s/pyton3-native/python3-native/

> +following::
> +
> +  $ bblock python3-native
> +  $ bblock -d
> +  # Generated by bblock
> +  SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
> +  SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
> +
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_patch:865859c27e603ba42025b7bb766c3cd4c0f477e4962cfd39128c0619d695fce7"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_sysroot:f8fa5d3194cef638416000252b959e86d0a19f6b7898e1f56b643c588cdd8605"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_prepare_recipe_sysroot:fe295ac505d9d1143313424b201c6f3f2a0a90da40a13a905b86b874705f226a"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_fetch:1b6e4728fee631bc7a8a7006855c5b8182a8224579e32e3d0a2db77c26459f25"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_unpack:2ad74d6f865ef75c35c0e6bbe3f9a90923a6b2c62c18a3ddef514ea31fbc588f"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_deploy_source_date_epoch:15f89b8483c1ad7507480f337619bb98c26e231227785eb3543db163593e7b42"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_configure:7960c13d23270fdb12b3a7c426ce1da0d2f5c7cf5e5d3f5bdce5fa330eb7d482"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_compile:012e1d4a63f1a78fc2143bd90d704dbcf5865c5257d6272aa7540ec1cd3063d9"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_install:d3401cc2afa4c996beb154beaad3e45fa0272b9c56fb86e9db14ec3544c68f9d"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_build:fa88bb7afb9046c0417c24a3fa98a058653805a8b00eda2c2d7fea68fc42f882"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_collect_spdx_deps:cc9c53ba7c495567e9a38ec4801830c425c0d1f895aa2fc66930a2edd510d9b4"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_spdx:766a1d09368438b7b5a1a8e2a8f823b2b731db44b57e67d8b3196de91966f9c5"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_package_spdx:46f80faeab25575e9977ba3bf14c819489c3d489432ae5145255635108c21020"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_recipe_qa:cb960cdb074e7944e894958db58f3dc2a0436ecf87c247feb3e095e214fec0e4"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_lic:15657441621ee83f15c2e650e7edbb036870b56f55e72e046c6142da3c5783fd"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_manifest:24f0abbec221d27bbb2909b6e846288b12cab419f1faf9f5006ed80423d37e28"
> +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_addto_recipe_sysroot:bcb6a1905f113128de3f88d702b706befd6a786267c045ee82532759a7c214d7"
> +
> diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst
> index 9ccf60f7012ea344612298e77ebabed84ad6de3e..7afd0d820e94d56cb2145962b9b1a976c7951936 100644
> --- a/documentation/dev-manual/index.rst
> +++ b/documentation/dev-manual/index.rst
> @@ -48,5 +48,6 @@ Yocto Project Development Tasks Manual
>     error-reporting-tool
>     wayland
>     qemu
> +   bblock
>  
>  .. include:: /boilerplate.rst

Thanks!
Antonin

--
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH 4/5] ref-manual: variables: add SIGGEN_LOCKEDSIGS* variables

[Julien Stephan]

Le lun. 4 nov. 2024 à 14:32, Antonin Godard
<antonin.godard@bootlin.com> a écrit :
>
> Hi Julien,
>
> On top of Ulrich's comments, here are my suggestions/corrections:
>
> On Thu Oct 31, 2024 at 5:56 PM CET, Julien Stephan wrote:
> > Variables SIGGEN_LOCKEDSIGS, SIGGEN_LOCKEDSIGS_TASKSIG_CHECK and
> > SIGGEN_LOCKEDSIGS_TYPES are used to lock specific tasks to specific
> > signatures. They are used by bitbake -S <lockedsigs> and bblock, so add
> > documentation for them.
> >
> > Signed-off-by: Julien Stephan <jstephan@baylibre.com>
> > ---
> >  documentation/ref-manual/variables.rst | 33 +++++++++++++++++++++++++++++++++
> >  1 file changed, 33 insertions(+)
> >
> > diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
> > index 07ed143ac5606617c720301ac6621355a341f90a..8cfb978e795e43e3ef25f3e18e11baf7c2a28d54 100644
> > --- a/documentation/ref-manual/variables.rst
> > +++ b/documentation/ref-manual/variables.rst
> > @@ -7880,6 +7880,39 @@ system and gives an overview of their function and contents.
> >           might break at runtime if the interface of the recipe was changed
> >           after the other had been built.
> >
> > +   :term:`SIGGEN_LOCKEDSIGS`
> > +     The list of locked taks, with the form::
> > +
> > +       SIGGEN_LOCKEDSIGS += "<package>:<task>:<signature>"
> > +
> > +     If ``<signature>`` exists for the specified ``<task>`` and ``<package>``
>
> How would I find such a signature by hand? Maybe worth explaining it here?
>
> > +     in sstate, BitBake will use the cached output instead of rebuilding the
> > +     ``<task>``. If it does not exist, BitBake will build the ``<task>`` and
> > +     the sstate will be used next time.
>
> Could you give a real-life example of that at the end here?
>
> > +   :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`
> > +     Specifies the debug level of task signature check. 3 levels are supported:
> > +
> > +     * info: displays a "Note" message to remind user that a task is locked
> > +       and current signature matches the locked one.
> > +     * warn: displays a "Warning" message if a task is locked and current
> > +       signature does not match the locked one.
> > +     * error: same as warn but displays an “Error” message and abort.
>
> For "info", "warn" and "error" could you enclose that in double ticks? ``info``
> and so on.
>
> > +   :term:`SIGGEN_LOCKEDSIGS_TYPES`
> > +     Allowed overrides for :term:`SIGGEN_LOCKEDSIGS`. This is mainly used
> > +     for achitecture specific lock. A common value for :term:`SIGGEN_LOCKEDSIGS_TYPES`
>
> s/achitecture/architecture/
>
> By "mainly", do you mean "only" or could there by anything other than
> architectures covered by the variable?

You can literally put anything you want in this variable..

> > +     is ``${PACKAGE_ARCHS}``::
> > +
> > +       SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
> > +
> > +       SIGGEN_LOCKEDSIGS_core2-64 += "bc:do_compile:abcd"
> > +       SIGGEN_LOCKEDSIGS_cortexa57 += "bc:do_compile:efgh"
>
> Maybe real hashes would be better here
>
> > +
> > +     Here, the ``do_compile`` task from ``bc`` will be locked only for
> > +     ``x86-64`` and ``arm64`` but not for other architectures such as
>
> Wouldn't it be locked for ``core2-64`` and ``cortexa57`` more specifically? I
> might misunderstand something here. :)

You're right. I wanted to give an example for machines first, then
decided to keep architecture and forgot to update.
Thank you  for pointing it out.


>
> > +     ``qemumips``.
> > +
> >     :term:`SITEINFO_BITS`
> >        Specifies the number of bits for the target system CPU. The value
> >        should be either "32" or "64".
>
> Also I could suggest splitting the series in patches for sphinx-lint/vale and
> patches for documentation of variables/bblock.

Sure, I'll do that

Cheers
Julien
>
> Thanks!
> Antonin
>
> --
> Antonin Godard, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>

Re: [docs] [PATCH 5/5] dev-manual: add bblock documentation

[Julien Stephan]

Le lun. 4 nov. 2024 à 14:32, Antonin Godard
<antonin.godard@bootlin.com> a écrit :
>
> Hi Julien,
>
> Thanks for taking the time to document your tool!
> On top of Ulrich's comments, here are my suggestions/corrections:
>
> On Thu Oct 31, 2024 at 5:56 PM CET, Julien Stephan wrote:
> > bblock is a helper tool to lock/unlock tasks and recipes to specific
> > signatures. Add a documentation page for it.
> >
> > Signed-off-by: Julien Stephan <jstephan@baylibre.com>
> > ---
> >  documentation/dev-manual/bblock.rst | 129 ++++++++++++++++++++++++++++++++++++
> >  documentation/dev-manual/index.rst  |   1 +
> >  2 files changed, 130 insertions(+)
> >
> > diff --git a/documentation/dev-manual/bblock.rst b/documentation/dev-manual/bblock.rst
> > new file mode 100644
> > index 0000000000000000000000000000000000000000..3df75799aa638d173764607df191bd99612034cf
> > --- /dev/null
> > +++ b/documentation/dev-manual/bblock.rst
> > @@ -0,0 +1,129 @@
> > +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
> > +
> > +Locking and Unlocking Recipes Using ``bblock``
> > +**********************************************
> > +
> > +By design, the OpenEmbedded build system builds everything from scratch
> > +unless BitBake determines that specific parts do not require rebuilding.
>
> s/parts/tasks/?
>
> > +At startup, it computes a signature for each tasks, based on the task's input.
> > +Then, it compares these signatures with the ones from sstate (if exist). Any
>
> s/from sstate/from the sstate cache/?
> s/if exist/if they exist/
>
> > +changes cause the task to rerun.
> > +
> > +During development, changes might trigger BitBake to rebuild certain
> > +recipes, even when we know they do not require rebuilding at that stage.
> > +For example, modifying a recipe can lead to the rebuilding of its native
> > +counterpart, which might proves unnecessary. Editing the ``python3`` recipe,
> > +for instance, can prompt BitBake to rebuild ``python3-native`` along with any
> > +recipes that depend on it.
> > +
> > +To prevent this, use ``bblock`` to lock specific tasks or recipes to
> > +specific signatures, forcing BitBake to use the sstate cache for them.
> > +
> > +.. attention::
> > +
> > +   Use ``bblock`` only during the development phase.
> > +
> > +   Forcing BitBake to use sstate cache, regardless of input changes, means
> > +   the recipe metadata no longer directly reflect the output. Use this feature
>
> s/reflect/reflects/ unless metadata is a plural by default?

I tend to consider metadata plural by default. Maybe I'm wrong?

>
> > +   with caution. If you do not understand why signatures change, see the section
> > +   on `understanding what changed <https://wiki.yoctoproject.org/wiki/TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`__.
>
> You can use :yocto_wiki: here:
>
> :yocto_wiki:`understanding what changed </TipsAndTricks/Understanding_what_changed_(diffsigs_etc)>`.
>
> (syntax needs double-checking ;))
>
> > +
> > +Locking tasks and recipes
> > +-------------------------
> > +
> > +To lock a recipe, use::
> > +
> > +   $ bblock recipe
> > +
> > +You can also use a space-separated list of recipes to lock many recipes::
> > +
> > +   $ bblock recipe1 recipe2
> > +
> > +Locking a recipe, means locking all tasks of the recipe. If you need to
>
> No comma "," between recipe and means, I think.
>
> > +lock only particular tasks use the `-t` switch with a comma-separated
>
> Comma after "tasks" I think would make more sense here (to me at least).
>
> > +list of tasks::
> > +
> > +  $ bblock -t task1,task2 recipe
> > +
>
> Maybe a real-life example would be great here!

I already put a real life example at the end. The idea was to avoid
having an example in each section. Is it sufficient?

>
> > +Unlocking tasks and recipes
> > +---------------------------
> > +
> > +To unlock a recipe, use::
>
> ...use the ``-r`` option::
>
> > +
> > +   $ bblock -r recipe
> > +
> > +You can also use a space-separated list of recipes to unlock many recipes::
>
> s/many/multiple/ (many means "a lot of")

Yeah.. I will put "multiple" back.. But according to Vale:

warning     'multiple' is too wordy.
                         write-good.TooWordy
warning     'several' is a weasel word!
                         write-good.Weasel

That's the reason why I put "many" ^^(but I agree it does not fit well here)
>
> > +
> > +   $ bblock -r recipe1 recipe2
> > +
> > +Unlocking a recipe, means unlocking all tasks of the recipe. If you need to
> > +unlock only particular tasks use the `-t` switch with a comma-separated
> > +list of tasks::
> > +
> > +  $ bblock -r -t task1,task2 recipe
> > +
> > +To unlock all recipes, do not specify any recipe::
> > +
> > +  $ bblock -r
> > +
> > +
> > +Configuration file
> > +------------------
> > +
> > +``bblock`` will dump the signatures in the ``build/conf/bblock.conf`` file,
> > +included by default in ``poky/meta/conf/bitbake.conf``.
>
> Here you can use:
>
> :oe_git:`meta/conf/bitbake.conf </openembedded-core/tree/meta/conf/bitbake.conf>`
>
> Also, I believe that bblock.conf is a candidate for being in
> `documentation/ref-manual/structure.rst`, alongside local.conf and
> bblayers.conf in the section "The Build Directory --- ``build/``". Could you add
> a section there that explains this file briefly and that redirects the user to
> this document?
>

Sure!
> > +
> > +To dump it, use::
>
> s/To dump it/To dump the file/
>
> > +
> > +  $ bblock -d
> > +
> > +
> > +Locking mechanism
> > +-----------------
> > +
> > +``bblock`` computes the signature(s) of the task(s) then uses the 3 following
>
> s/then uses/and sets/? I.e. explain what it really does with them.
>
> > +variables: :term:`SIGGEN_LOCKEDSIGS`, :term:`SIGGEN_LOCKEDSIGS_TYPES`
> > +and :term:`SIGGEN_LOCKEDSIGS_TASKSIG_CHECK`.
> > +
> > +In particular, ``bblock`` sets::
> > +
> > +  SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
> > +  SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
> > +
> > +  SIGGEN_LOCKEDSIGS_<package_arch> += "<recipe>:<task>:<signature>"
> > +
> > +This produces architecture specific locks and reminds user that some tasks
> > +have locked signatures.
> > +
> > +Example
> > +-------
> > +
> > +When working on ``python3`` recipe, we can lock ``pyton3-native`` with the
>
> s/on ``python3`` recipe/on the ``python3`` recipe/
> s/pyton3-native/python3-native/
>
> > +following::
> > +
> > +  $ bblock python3-native
> > +  $ bblock -d
> > +  # Generated by bblock
> > +  SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"
> > +  SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"
> > +
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_patch:865859c27e603ba42025b7bb766c3cd4c0f477e4962cfd39128c0619d695fce7"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_sysroot:f8fa5d3194cef638416000252b959e86d0a19f6b7898e1f56b643c588cdd8605"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_prepare_recipe_sysroot:fe295ac505d9d1143313424b201c6f3f2a0a90da40a13a905b86b874705f226a"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_fetch:1b6e4728fee631bc7a8a7006855c5b8182a8224579e32e3d0a2db77c26459f25"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_unpack:2ad74d6f865ef75c35c0e6bbe3f9a90923a6b2c62c18a3ddef514ea31fbc588f"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_deploy_source_date_epoch:15f89b8483c1ad7507480f337619bb98c26e231227785eb3543db163593e7b42"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_configure:7960c13d23270fdb12b3a7c426ce1da0d2f5c7cf5e5d3f5bdce5fa330eb7d482"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_compile:012e1d4a63f1a78fc2143bd90d704dbcf5865c5257d6272aa7540ec1cd3063d9"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_install:d3401cc2afa4c996beb154beaad3e45fa0272b9c56fb86e9db14ec3544c68f9d"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_build:fa88bb7afb9046c0417c24a3fa98a058653805a8b00eda2c2d7fea68fc42f882"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_collect_spdx_deps:cc9c53ba7c495567e9a38ec4801830c425c0d1f895aa2fc66930a2edd510d9b4"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_spdx:766a1d09368438b7b5a1a8e2a8f823b2b731db44b57e67d8b3196de91966f9c5"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_package_spdx:46f80faeab25575e9977ba3bf14c819489c3d489432ae5145255635108c21020"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_recipe_qa:cb960cdb074e7944e894958db58f3dc2a0436ecf87c247feb3e095e214fec0e4"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_populate_lic:15657441621ee83f15c2e650e7edbb036870b56f55e72e046c6142da3c5783fd"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_create_manifest:24f0abbec221d27bbb2909b6e846288b12cab419f1faf9f5006ed80423d37e28"
> > +  SIGGEN_LOCKEDSIGS_x86_64 += "python3-native:do_addto_recipe_sysroot:bcb6a1905f113128de3f88d702b706befd6a786267c045ee82532759a7c214d7"
> > +
> > diff --git a/documentation/dev-manual/index.rst b/documentation/dev-manual/index.rst
> > index 9ccf60f7012ea344612298e77ebabed84ad6de3e..7afd0d820e94d56cb2145962b9b1a976c7951936 100644
> > --- a/documentation/dev-manual/index.rst
> > +++ b/documentation/dev-manual/index.rst
> > @@ -48,5 +48,6 @@ Yocto Project Development Tasks Manual
> >     error-reporting-tool
> >     wayland
> >     qemu
> > +   bblock
> >
> >  .. include:: /boilerplate.rst
>
> Thanks!
> Antonin
>
> --
> Antonin Godard, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>