[nanbield][PATCH 0/7] documentation backports

[nanbield][PATCH 0/7] documentation backports

[michael.opdenacker]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

Apply relevant patches from the "master" branch.

Geoff Parker (1):
  ref-manual: variables: adding multiple groups in GROUPADD_PARAM

Johan Bezem (1):
  ref-manual: variables: correct sdk installation default path

Luca Ceresoli (2):
  ref-manual: tasks: do_cleanall: recommend using '-f' instead
  ref-manual: tasks: do_cleansstate: recommend using '-f' instead for a
    shared sstate

Michael Opdenacker (2):
  dev-manual: packages: fix capitalization
  manuals: document VIRTUAL-RUNTIME variables

Simone Weiß (1):
  contributor-guide: add notes for tests

 .../contributor-guide/submit-changes.rst      | 37 +++++++++++++-
 documentation/dev-manual/device-manager.rst   |  8 +--
 documentation/dev-manual/packages.rst         |  2 +-
 documentation/overview-manual/concepts.rst    |  2 +-
 documentation/ref-manual/tasks.rst            | 38 ++++++++++++--
 documentation/ref-manual/variables.rst        | 51 ++++++++++++++++---
 6 files changed, 121 insertions(+), 17 deletions(-)

-- 
2.34.1

[nanbield][PATCH 1/7] ref-manual: tasks: do_cleanall: recommend using '-f' instead

[michael.opdenacker]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

From: Luca Ceresoli <luca.ceresoli@bootlin.com>

do_cleanall can produce failures when used in legitimate cases, such as
with recipe variants (foo and foo-native) or a shared DL_DIR. This is why
it is forbidden when writing tests that will run on the autobuilders
(https://docs.yoctoproject.org/test-manual/intro.html?highlight=cleanall#considerations-when-writing-tests).

Reword the documentation to clearly discourage, provide a safe alternative
(bitbake -f -c fetch), and the rationale with an example.

Reported-by: Sam Liddicott
Link: https://bootlin.com/blog/yocto-sharing-the-sstate-cache-and-download-directories/#comment-2650335
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Reviewed-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
---
 documentation/ref-manual/tasks.rst | 26 +++++++++++++++++++++++---
 1 file changed, 23 insertions(+), 3 deletions(-)

diff --git a/documentation/ref-manual/tasks.rst b/documentation/ref-manual/tasks.rst
index 0db960b22f..16b48ca0bc 100644
--- a/documentation/ref-manual/tasks.rst
+++ b/documentation/ref-manual/tasks.rst
@@ -470,9 +470,29 @@ You can run this task using BitBake as follows::
 
    $ bitbake -c cleanall recipe
 
-Typically, you would not normally use the :ref:`ref-tasks-cleanall` task. Do so only
-if you want to start fresh with the :ref:`ref-tasks-fetch`
-task.
+You should never use the :ref:`ref-tasks-cleanall` task in a normal
+scenario. If you want to start fresh with the :ref:`ref-tasks-fetch` task,
+use instead::
+
+  $ bitbake -f -c fetch recipe
+
+.. note::
+
+   The reason to prefer ``bitbake -f -c fetch`` is that the
+   :ref:`ref-tasks-cleanall` task would break in some cases, such as::
+
+      $ bitbake -c fetch    recipe
+      $ bitbake -c cleanall recipe-native
+      $ bitbake -c unpack   recipe
+
+   because after step 1 there is a stamp file for the
+   :ref:`ref-tasks-fetch` task of ``recipe``, and it won't be removed at
+   step 2 because step 2 uses a different work directory. So the unpack task
+   at step 3 will try to extract the downloaded archive and fail as it has
+   been deleted in step 2.
+
+   Note that this also applies to BitBake from concurrent processes when a
+   shared download directory (:term:`DL_DIR`) is setup.
 
 .. _ref-tasks-cleansstate:
 
-- 
2.34.1

[nanbield][PATCH 2/7] ref-manual: tasks: do_cleansstate: recommend using '-f' instead for a shared sstate

[michael.opdenacker]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

From: Luca Ceresoli <luca.ceresoli@bootlin.com>

do_cleansstat can produce build errors when using a shared sstate cache.

Add a note to clearly discourage, provide a safe alternative (bitbake -f),
and the rationale.

Suggested-by: Quentin Schulz  <quentin.schulz@theobroma-systems.com>
Link: https://lore.kernel.org/yocto-docs/20240219155513.76738-1-luca.ceresoli@bootlin.com/T/#m5529687ecb0f9ec2dacddcb6ff58e2df73af9cde
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Reviewed-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
---
 documentation/ref-manual/tasks.rst | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/documentation/ref-manual/tasks.rst b/documentation/ref-manual/tasks.rst
index 16b48ca0bc..c28cd7a94a 100644
--- a/documentation/ref-manual/tasks.rst
+++ b/documentation/ref-manual/tasks.rst
@@ -514,6 +514,18 @@ When you run the :ref:`ref-tasks-cleansstate` task, the OpenEmbedded build syste
 no longer uses any sstate. Consequently, building the recipe from
 scratch is guaranteed.
 
+.. note::
+
+   Using :ref:`ref-tasks-cleansstate` with a shared :term:`SSTATE_DIR` is
+   not recommended because it could trigger an error during the build of a
+   separate BitBake instance. This is because the builds check sstate "up
+   front" but download the files later, so it if is deleted in the
+   meantime, it will cause an error but not a total failure as it will
+   rebuild it.
+
+   The reliable and preferred way to force a new build is to use ``bitbake
+   -f`` instead.
+
 .. note::
 
    The :ref:`ref-tasks-cleansstate` task cannot remove sstate from a remote sstate
-- 
2.34.1

[nanbield][PATCH 3/7] ref-manual: variables: correct sdk installation default path

[michael.opdenacker]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

From: Johan Bezem <jbezem.extern@arri.de>

The SDKPATH variable seems mistakenly identified as the
default path where the SDK will be installed by the generated
installation script, unless option '-d' or a manual input
overrides this default.

The intended variable is SDKPATHINSTALL. SDKPATH indicates
where the SDK is being composed and built.

The definitions have been added/updated.

Signed-off-by: Johan Bezem <jbezem.extern@arri.de>
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/overview-manual/concepts.rst | 2 +-
 documentation/ref-manual/variables.rst     | 6 +++++-
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/documentation/overview-manual/concepts.rst b/documentation/overview-manual/concepts.rst
index d177ca33b7..62f2327a7e 100644
--- a/documentation/overview-manual/concepts.rst
+++ b/documentation/overview-manual/concepts.rst
@@ -1375,7 +1375,7 @@ This next list, shows the variables associated with a standard SDK:
    Lists packages that make up the target part of the SDK (i.e. the part
    built for the target hardware).
 
--  :term:`SDKPATH`: Defines the
+-  :term:`SDKPATHINSTALL`: Defines the
    default SDK installation path offered by the installation script.
 
 -  :term:`SDK_HOST_MANIFEST`:
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index f4beecf0d2..448bea9938 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -7612,6 +7612,10 @@ system and gives an overview of their function and contents.
          configuration will not take effect.
 
    :term:`SDKPATH`
+      Defines the path used to collect the SDK components and build the
+      installer.
+
+   :term:`SDKPATHINSTALL`
       Defines the path offered to the user for installation of the SDK that
       is generated by the OpenEmbedded build system. The path appears as
       the default location for installing the SDK when you run the SDK's
@@ -7621,7 +7625,7 @@ system and gives an overview of their function and contents.
    :term:`SDKTARGETSYSROOT`
       The full path to the sysroot used for cross-compilation within an SDK
       as it will be when installed into the default
-      :term:`SDKPATH`.
+      :term:`SDKPATHINSTALL`.
 
    :term:`SECTION`
       The section in which packages should be categorized. Package
-- 
2.34.1

[nanbield][PATCH 4/7] ref-manual: variables: adding multiple groups in GROUPADD_PARAM

[michael.opdenacker]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

From: Geoff Parker <geoffrey.parker@arthrex.com>

Add missing documentation on how to add multiple groups with a single
GROUPADD_PARAM:${PN}

Signed-off-by: Geoff Parker <geoffrey.parker@arthrex.com>
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/ref-manual/variables.rst | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 448bea9938..ec879f9f86 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -3234,6 +3234,14 @@ system and gives an overview of their function and contents.
 
          GROUPADD_PARAM:${PN} = "-r netdev"
 
+      More than one group can be added by separating each set of different
+      groups' parameters with a semicolon.
+
+      Here is an example adding multiple groups from the ``useradd-example.bb``
+      file in the ``meta-skeleton`` layer::
+
+         GROUPADD_PARAM:${PN} = "-g 880 group1; -g 890 group2"
+
       For information on the standard Linux shell command
       ``groupadd``, see https://linux.die.net/man/8/groupadd.
 
-- 
2.34.1

[nanbield][PATCH 5/7] dev-manual: packages: fix capitalization

[michael.opdenacker]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

Using "PR service" instead of "PR Service", like in the other
two instances in this document.

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Reported-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Reviewed-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
---
 documentation/dev-manual/packages.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/documentation/dev-manual/packages.rst b/documentation/dev-manual/packages.rst
index a6e6af0de7..e5028fffdc 100644
--- a/documentation/dev-manual/packages.rst
+++ b/documentation/dev-manual/packages.rst
@@ -206,7 +206,7 @@ history, see the
    part of the shared state (sstate) packages. If you maintain an sstate
    feed, it's expected that either all your building systems that
    contribute to the sstate feed use a shared PR service, or you do not
-   run a PR Service on any of your building systems.
+   run a PR service on any of your building systems.
 
    That's because if you had multiple machines sharing a PR service but
    not their sstate feed, you could end up with "diverging" hashes for
-- 
2.34.1

[nanbield][PATCH 6/7] contributor-guide: add notes for tests

[michael.opdenacker]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

From: Simone Weiß <simone.p.weiss@posteo.com>

This adds some hints that and how changes should be tested when contributing.

Fixes [YOCTO #15412]

Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 .../contributor-guide/submit-changes.rst      | 37 ++++++++++++++++++-
 1 file changed, 35 insertions(+), 2 deletions(-)

diff --git a/documentation/contributor-guide/submit-changes.rst b/documentation/contributor-guide/submit-changes.rst
index 61f3157d60..59f3c1e406 100644
--- a/documentation/contributor-guide/submit-changes.rst
+++ b/documentation/contributor-guide/submit-changes.rst
@@ -221,6 +221,38 @@ to add the upgraded version.
    <https://www.kernel.org/doc/html/latest/process/submitting-patches.html#using-reported-by-tested-by-reviewed-by-suggested-by-and-fixes>`__
    in the Linux kernel documentation.
 
+Test your changes
+-----------------
+
+For each contributions you make, you should test your changes as well.
+For this the Yocto Project offers several types of tests. Those tests cover
+different areas and it depends on your changes which are feasible. For example run:
+
+   -  For changes that affect the build environment:
+
+      -  ``bitbake-selftest``: for changes within BitBake
+
+      -  ``oe-selftest``: to test combinations of BitBake runs
+
+      -  ``oe-build-perf-test``: to test the performance of common build scenarios
+
+   -  For changes in a recipe:
+
+      - ``ptest``: run package specific tests, if they exist
+
+      - ``testimage``: build an image, boot it and run testcases on it
+
+      - If applicable, ensure also the ``native`` and ``nativesdk`` variants builds
+
+   -  For changes relating to the SDK:
+
+      - ``testsdk``: to build, install and run tests against a SDK
+
+      - ``testsdk_ext``: to build, install and run tests against an extended SDK
+
+Note that this list just gives suggestions and is not exhaustive. More details can
+be found here: :ref:`test-manual/intro:Yocto Project Tests --- Types of Testing Overview`.
+
 Creating Patches
 ================
 
@@ -285,8 +317,9 @@ Validating Patches with Patchtest
 ``patchtest`` is available in ``openembedded-core`` as a tool for making
 sure that your patches are well-formatted and contain important info for
 maintenance purposes, such as ``Signed-off-by`` and ``Upstream-Status``
-tags. Currently, it only supports testing patches for
-``openembedded-core`` branches. To setup, perform the following::
+tags. Note that no functional testing of the changes will be performed by ``patchtest``.
+Currently, it only supports testing patches for ``openembedded-core`` branches.
+To setup, perform the following::
 
     pip install -r meta/lib/patchtest/requirements.txt
     source oe-init-build-env
-- 
2.34.1

[nanbield][PATCH 7/7] manuals: document VIRTUAL-RUNTIME variables

[michael.opdenacker]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

Document the convention to use variables prefixed by VIRTUAL_RUNTIME.
Add references to the new term where possible.

Another reason is that such variables are recommended
in a warning issued by meta/classes-global/insane.bbclass

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Reported-by: Tim Orling <ticotimo@gmail.com>
---
 documentation/dev-manual/device-manager.rst |  8 ++---
 documentation/ref-manual/variables.rst      | 37 ++++++++++++++++++---
 2 files changed, 36 insertions(+), 9 deletions(-)

diff --git a/documentation/dev-manual/device-manager.rst b/documentation/dev-manual/device-manager.rst
index 0343d19b9c..49fc785fec 100644
--- a/documentation/dev-manual/device-manager.rst
+++ b/documentation/dev-manual/device-manager.rst
@@ -60,10 +60,10 @@ kernel.
 All devices created by ``devtmpfs`` will be owned by ``root`` and have
 permissions ``0600``.
 
-To have more control over the device nodes, you can use a device manager
-like ``udev`` or ``busybox-mdev``. You choose the device manager by
-defining the ``VIRTUAL-RUNTIME_dev_manager`` variable in your machine or
-distro configuration file. Alternatively, you can set this variable in
+To have more control over the device nodes, you can use a device manager like
+``udev`` or ``busybox-mdev``. You choose the device manager by defining the
+:term:`VIRTUAL-RUNTIME_dev_manager <VIRTUAL-RUNTIME>` variable in your machine
+or distro configuration file. Alternatively, you can set this variable in
 your ``local.conf`` configuration file::
 
    VIRTUAL-RUNTIME_dev_manager = "udev"
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index ec879f9f86..52fd7d9002 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -6750,11 +6750,11 @@ system and gives an overview of their function and contents.
 
       .. note::
 
-         A corresponding mechanism for virtual runtime dependencies
-         (packages) exists. However, the mechanism does not depend on any
-         special functionality beyond ordinary variable assignments. For
-         example, ``VIRTUAL-RUNTIME_dev_manager`` refers to the package of
-         the component that manages the ``/dev`` directory.
+         A corresponding mechanism for virtual runtime dependencies (packages)
+         exists. However, the mechanism does not depend on any special
+         functionality beyond ordinary variable assignments. For example,
+         :term:`VIRTUAL-RUNTIME_dev_manager <VIRTUAL-RUNTIME>` refers to the
+         package of the component that manages the ``/dev`` directory.
 
          Setting the "preferred provider" for runtime dependencies is as
          simple as using the following assignment in a configuration file::
@@ -9880,6 +9880,33 @@ system and gives an overview of their function and contents.
       Additionally, you should also set the
       :term:`USERADD_ERROR_DYNAMIC` variable.
 
+   :term:`VIRTUAL-RUNTIME`
+      :term:`VIRTUAL-RUNTIME` is a commonly used prefix for defining virtual
+      packages for runtime usage, typically for use in :term:`RDEPENDS`
+      or in image definitions.
+
+      An example is ``VIRTUAL-RUNTIME_base-utils`` that makes it possible
+      to either use BusyBox based utilities::
+
+         VIRTUAL-RUNTIME_base-utils = "busybox"
+
+      or their full featured implementations from GNU Coreutils
+      and other projects::
+
+         VIRTUAL-RUNTIME_base-utils = "packagegroup-core-base-utils"
+
+      Here are two examples using this virtual runtime package. The
+      first one is in :yocto_git:`initramfs-framework_1.0.bb
+      </poky/tree/meta/recipes-core/initrdscripts/initramfs-framework_1.0.bb?h=scarthgap>`::
+
+         RDEPENDS:${PN} += "${VIRTUAL-RUNTIME_base-utils}"
+
+      The second example is in the :yocto_git:`core-image-initramfs-boot
+      </poky/tree/meta/recipes-core/images/core-image-initramfs-boot.bb?h=scarthgap>`
+      image definition::
+
+         PACKAGE_INSTALL = "${INITRAMFS_SCRIPTS} ${VIRTUAL-RUNTIME_base-utils} base-passwd"
+
    :term:`VOLATILE_LOG_DIR`
       Specifies the persistence of the target's ``/var/log`` directory,
       which is used to house postinstall target log files.
-- 
2.34.1