[PATCH v3] bitbake: doc: Add section for variable context

[PATCH v3] bitbake: doc: Add section for variable context

[simone.p.weiss]

From: Simone Weiß <simone.p.weiss@posteo.com>

This is inspired by the same section in the yocto-docs. It aims to provide
information in what contexts(recipes, .conf, bbclass,...) a variable is usually
used. For that I tried to group similar variables, so that a short overview is
given. This was inspired by [YOCTO #14072], but of course does not implement a
warning if a variable is used in an unintended context.

Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---

Notes to v3: 
- Applied the fixes suggested in review (thanks for the patch! I hope adding the
  Signed-off like this during squashing is fine)
- For the ref label, I plan to come up with an svg that illustrates tasks,
  recipes, classes, conf (thanks Quentin for the suggestion), but in a generic
  way and add it to the Concepts section, I want to add a ref from there.
 
 ...bake-user-manual-ref-variables-context.rst | 91 +++++++++++++++++++
 doc/index.rst                                 |  1 +
 2 files changed, 92 insertions(+)
 create mode 100644 doc/bitbake-user-manual/bitbake-user-manual-ref-variables-context.rst

diff --git a/doc/bitbake-user-manual/bitbake-user-manual-ref-variables-context.rst b/doc/bitbake-user-manual/bitbake-user-manual-ref-variables-context.rst
new file mode 100644
index 00000000..e9c454ba
--- /dev/null
+++ b/doc/bitbake-user-manual/bitbake-user-manual-ref-variables-context.rst
@@ -0,0 +1,91 @@
+.. SPDX-License-Identifier: CC-BY-2.5
+
+================
+Variable Context
+================
+
+|
+
+Variables might only have an impact or can be used in certain contexts. Some
+should only be used in global files like ``.conf``, while others are intended only
+for local files like ``.bb``. This chapter aims to describe some important variable
+contexts.
+
+.. _ref-varcontext-configuration:
+
+BitBake's own configuration
+===========================
+
+Variables starting with ``BB_`` usually configure the behaviour of BitBake itself.
+For example, one could configure:
+
+- System resources, like disk space to be used (:term:`BB_DISKMON_DIRS`),
+  or the number of tasks to be run in parallel by BitBake (:term:`BB_NUMBER_THREADS`).
+
+- How the fetchers shall behave, e.g., :term:`BB_FETCH_PREMIRRORONLY` is used
+  by BitBake to determine if BitBake's fetcher shall search only
+  :term:`PREMIRRORS` for files.
+
+Those variables are usually configured globally.
+
+BitBake configuration
+=====================
+
+There are variables:
+
+- Like :term:`B` or :term:`T`, that are used to specify directories used by
+  BitBake during the build of a particular recipe. Those variables are
+  specified in ``bitbake.conf``. Some, like :term:`B`, are quite often
+  overwritten in recipes.
+
+- Starting with ``FAKEROOT``, to configure how the ``fakeroot`` command is
+  handled. Those are usually set by ``bitbake.conf`` and might get adapted in a
+  ``bbclass``.
+
+- Detailing where BitBake will store and fetch information from, for
+  data reuse between build runs like :term:`CACHE`, :term:`DL_DIR` or
+  :term:`PERSISTENT_DIR`. Those are usually global.
+
+
+Layers and files
+================
+
+Variables starting with ``LAYER`` configure how BitBake handles layers.
+Additionally, variables starting with ``BB`` configure how layers and files are
+handled. For example:
+
+- :term:`LAYERDEPENDS` is used to configure on which layers a given layer
+  depends.
+
+- The configured layers are contained in :term:`BBLAYERS` and files in
+  :term:`BBFILES`.
+
+Those variables are often used in the files ``layer.conf`` and ``bblayers.conf``.
+
+Recipes and packages
+====================
+
+Variables handling recipes and packages can be split into:
+
+- :term:`PN`, :term:`PV` or :term:`PF` for example, contain information about
+  the name or revision of a recipe or package. Usually, the default set in
+  ``bitbake.conf`` is used, but those are from time to time overwritten in
+  recipes.
+
+- :term:`SUMMARY`, :term:`DESCRIPTION`, :term:`LICENSE` or :term:`HOMEPAGE`
+  contain the expected information and should be set specifically for every
+  recipe.
+
+- In recipes, variables are also used to control build and runtime
+  dependencies between recipes/packages with other recipes/packages. The
+  most common should be: :term:`PROVIDES`, :term:`RPROVIDES`, :term:`DEPENDS`,
+  and :term:`RDEPENDS`.
+
+- There are further variables starting with ``SRC`` that specify the sources in
+  a recipe like :term:`SRC_URI` or :term:`SRCDATE`. Those are also usually set
+  in recipes.
+
+- Which version or provider of a recipe should be given preference when
+  multiple recipes would provide the same item, is controlled by variables
+  starting with ``PREFERRED_``. Those are normally set in the configuration
+  files of a ``MACHINE`` or ``DISTRO``.
diff --git a/doc/index.rst b/doc/index.rst
index 3ff8b158..ee1660ac 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -13,6 +13,7 @@ BitBake User Manual
    bitbake-user-manual/bitbake-user-manual-intro
    bitbake-user-manual/bitbake-user-manual-execution
    bitbake-user-manual/bitbake-user-manual-metadata
+   bitbake-user-manual/bitbake-user-manual-ref-variables-context
    bitbake-user-manual/bitbake-user-manual-fetching
    bitbake-user-manual/bitbake-user-manual-ref-variables
    bitbake-user-manual/bitbake-user-manual-hello
-- 
2.39.2

Re: [bitbake-devel] [PATCH v3] bitbake: doc: Add section for variable context

[Michael Opdenacker]

Hi Simone,

On 2/23/24 at 20:20, Simone Weiß wrote:
> From: Simone Weiß <simone.p.weiss@posteo.com>
>
> This is inspired by the same section in the yocto-docs. It aims to provide
> information in what contexts(recipes, .conf, bbclass,...) a variable is usually
> used. For that I tried to group similar variables, so that a short overview is
> given. This was inspired by [YOCTO #14072], but of course does not implement a
> warning if a variable is used in an unintended context.
>
> Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
> ---
>
> Notes to v3:
> - Applied the fixes suggested in review (thanks for the patch! I hope adding the
>    Signed-off like this during squashing is fine)
> - For the ref label, I plan to come up with an svg that illustrates tasks,
>    recipes, classes, conf (thanks Quentin for the suggestion), but in a generic
>    way and add it to the Concepts section, I want to add a ref from there.
>   
>   ...bake-user-manual-ref-variables-context.rst | 91 +++++++++++++++++++
>   doc/index.rst                                 |  1 +
>   2 files changed, 92 insertions(+)
>   create mode 100644 doc/bitbake-user-manual/bitbake-user-manual-ref-variables-context.rst


Thanks for this new update of this worthy addition to the BitBake 
documentation! Everything looks good now.
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Cheers
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [bitbake-devel] [PATCH v3] bitbake: doc: Add section for variable context

[Quentin Schulz]

Hi all,

On 2/26/24 15:16, Michael Opdenacker via lists.yoctoproject.org wrote:
> Hi Simone,
> 
> On 2/23/24 at 20:20, Simone Weiß wrote:
>> From: Simone Weiß <simone.p.weiss@posteo.com>
>>
>> This is inspired by the same section in the yocto-docs. It aims to 
>> provide
>> information in what contexts(recipes, .conf, bbclass,...) a variable 
>> is usually
>> used. For that I tried to group similar variables, so that a short 
>> overview is
>> given. This was inspired by [YOCTO #14072], but of course does not 
>> implement a
>> warning if a variable is used in an unintended context.
>>
>> Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
>> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
>> ---
>>
>> Notes to v3:
>> - Applied the fixes suggested in review (thanks for the patch! I hope 
>> adding the
>>    Signed-off like this during squashing is fine)
>> - For the ref label, I plan to come up with an svg that illustrates 
>> tasks,
>>    recipes, classes, conf (thanks Quentin for the suggestion), but in 
>> a generic
>>    way and add it to the Concepts section, I want to add a ref from 
>> there.
>>   ...bake-user-manual-ref-variables-context.rst | 91 +++++++++++++++++++
>>   doc/index.rst                                 |  1 +
>>   2 files changed, 92 insertions(+)
>>   create mode 100644 
>> doc/bitbake-user-manual/bitbake-user-manual-ref-variables-context.rst
> 
> 
> Thanks for this new update of this worthy addition to the BitBake 
> documentation! Everything looks good now.
> Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>

Reviewed-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>

A section on task context would be nice but it can come up in a later 
patch I guess.

Thanks!

By the way, I had an old doc build lying around and just git pull my 
local branch + applying this patch and somehow the navbar was not 
updated to contain this new section, I had to remove the build dir and 
compile it from scratch. I'm wondering if we aren't missing something 
somewhere so that this navbar is updated whenever it needs to?

Cheers,
Quentin

Re: [docs] [bitbake-devel] [PATCH v3] bitbake: doc: Add section for variable context

[Michael Opdenacker]

Hi Richard

On 2/27/24 at 17:38, Quentin Schulz via lists.openembedded.org wrote:
> Hi all,
>
> On 2/26/24 15:16, Michael Opdenacker via lists.yoctoproject.org wrote:
>> Hi Simone,
>>
>> On 2/23/24 at 20:20, Simone Weiß wrote:
>>> From: Simone Weiß <simone.p.weiss@posteo.com>
>>>
>>> This is inspired by the same section in the yocto-docs. It aims to 
>>> provide
>>> information in what contexts(recipes, .conf, bbclass,...) a variable 
>>> is usually
>>> used. For that I tried to group similar variables, so that a short 
>>> overview is
>>> given. This was inspired by [YOCTO #14072], but of course does not 
>>> implement a
>>> warning if a variable is used in an unintended context.
>>>
>>> Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
>>> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
>>> ---
>>>
>>> Notes to v3:
>>> - Applied the fixes suggested in review (thanks for the patch! I 
>>> hope adding the
>>>    Signed-off like this during squashing is fine)
>>> - For the ref label, I plan to come up with an svg that illustrates 
>>> tasks,
>>>    recipes, classes, conf (thanks Quentin for the suggestion), but 
>>> in a generic
>>>    way and add it to the Concepts section, I want to add a ref from 
>>> there.
>>>   ...bake-user-manual-ref-variables-context.rst | 91 
>>> +++++++++++++++++++
>>>   doc/index.rst                                 |  1 +
>>>   2 files changed, 92 insertions(+)
>>>   create mode 100644 
>>> doc/bitbake-user-manual/bitbake-user-manual-ref-variables-context.rst
>>
>>
>> Thanks for this new update of this worthy addition to the BitBake 
>> documentation! Everything looks good now.
>> Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
>
> Reviewed-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
>
> A section on task context would be nice but it can come up in a later 
> patch I guess.
>
> Thanks!
>
> By the way, I had an old doc build lying around and just git pull my 
> local branch + applying this patch and somehow the navbar was not 
> updated to contain this new section, I had to remove the build dir and 
> compile it from scratch. I'm wondering if we aren't missing something 
> somewhere so that this navbar is updated whenever it needs to?
>
> Cheers,
> Quentin


Ping for this nice patch from Simone.
Both of us have given it our "Reviewed-by".

Thanks in advance
Cheers
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [bitbake-devel] [PATCH v3] bitbake: doc: Add section for variable context

[Richard Purdie]

On Thu, 2024-04-11 at 15:34 +0200, Michael Opdenacker wrote:
> Hi Richard
> > Reviewed-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
> > 
> > A section on task context would be nice but it can come up in a
> > later 
> > patch I guess.
> > 
> > Thanks!
> > 
> > By the way, I had an old doc build lying around and just git pull
> > my 
> > local branch + applying this patch and somehow the navbar was not 
> > updated to contain this new section, I had to remove the build dir
> > and 
> > compile it from scratch. I'm wondering if we aren't missing
> > something 
> > somewhere so that this navbar is updated whenever it needs to?
> > 
> > Cheers,
> > Quentin
> 
> 
> Ping for this nice patch from Simone.
> Both of us have given it our "Reviewed-by".

I think I misread Quentin's comment, sorry. I've merged it now.

Cheers,

Richard