[PATCH 0/4] Improvements around PERSISTENT_DIR

[PATCH 0/4] Improvements around PERSISTENT_DIR

[Antonin Godard]

The PERSISTENT_DIR directory should be shared along the sstate cache to
ensure that the hash equivalence database is also shared. Otherwise this
can result in bugs described in YOCTO #15921 [1].

[1]: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15921

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
Antonin Godard (4):
      test-manual/understand-autobuilder.rst: mention hashequiv server
      ref-manual/structure.rst: update with info on PERSISTENT_DIR
      dev-manual/start.rst: mention that PERSISTENT_DIR should be shared too
      overview-manual/concepts.rst: mention PERSISTENT_DIR for user configuration

 documentation/dev-manual/start.rst                   |  5 +++++
 documentation/overview-manual/concepts.rst           |  3 +++
 documentation/ref-manual/structure.rst               | 14 +++++++++++++-
 documentation/test-manual/understand-autobuilder.rst |  6 ++++++
 4 files changed, 27 insertions(+), 1 deletion(-)
---
base-commit: 80084a4cabdf7f61c7e93eda8ddbd5bc7d54e041
change-id: 20250708-update-sstate-dir-docs-8763b1387ff5

Best regards,
--  
Antonin Godard <antonin.godard@bootlin.com>

[PATCH 1/4] test-manual/understand-autobuilder.rst: mention hashequiv server

[Antonin Godard]

The autobuilder also uses a shared Hash Equivalence server, so mention
it here too.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/test-manual/understand-autobuilder.rst | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/documentation/test-manual/understand-autobuilder.rst b/documentation/test-manual/understand-autobuilder.rst
index 7f4d1be3c..23b3ef7d7 100644
--- a/documentation/test-manual/understand-autobuilder.rst
+++ b/documentation/test-manual/understand-autobuilder.rst
@@ -184,6 +184,12 @@ directory to be shared between them. This means once a Worker has built
 an artifact, all the others can benefit from it. The usage of the directory
 within the build system is designed for sharing over NFS.
 
+Shared Hash Equivalence Server
+------------------------------
+
+The Workers all use the same Hash Equivalence server, through a common
+definition for :term:`BB_HASHSERVE`.
+
 Resulttool
 ----------
 

-- 
2.50.0

[PATCH 2/4] ref-manual/structure.rst: update with info on PERSISTENT_DIR

[Antonin Godard]

Since we mention that SSTATE_DIR can be shared for multiple builds, also
mention that PERSISTENT_DIR should be shared alongside SSTATE_DIR.

[YOCTO #15921]

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/ref-manual/structure.rst | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/documentation/ref-manual/structure.rst b/documentation/ref-manual/structure.rst
index d6dbb2940..e4e800def 100644
--- a/documentation/ref-manual/structure.rst
+++ b/documentation/ref-manual/structure.rst
@@ -246,7 +246,8 @@ section in the Yocto Project Development Tasks Manual.
 ----------------
 
 This directory contains several internal files used by the OpenEmbedded
-build system.
+build system. The path to this directory is defined by the
+:term:`PERSISTENT_DIR` variable.
 
 It also contains ``sanity_info``, a text file keeping track of important
 build information such as the values of :term:`TMPDIR`, :term:`SSTATE_DIR`,
@@ -364,6 +365,13 @@ directory for multiple builds or move the directory to another location.
 You can control the location of this directory through the
 :term:`SSTATE_DIR` variable.
 
+.. note::
+
+   If you are sharing the :term:`SSTATE_DIR` directory for multiple builds and
+   you are using a :doc:`/overview-manual/concepts:Hash Equivalence` server with
+   :term:`BB_HASHSERVE` you should also share the :term:`PERSISTENT_DIR`
+   directory. See :ref:`structure-build-cache` below.
+
 .. _structure-build-tmp:
 
 ``build/tmp/``
@@ -403,6 +411,10 @@ modification time (mtime) changes and hashing of file contents. If no
 changes to the file are detected, then the parsed result stored in the
 cache is reused. If the file has changed, it is reparsed.
 
+This directory and the :term:`SSTATE_DIR` directory can be shared for multiple
+builds when using a :doc:`/overview-manual/concepts:Hash Equivalence` server
+with :term:`BB_HASHSERVE`.
+
 .. _structure-build-tmp-deploy:
 
 ``build/tmp/deploy/``

-- 
2.50.0

Re: [docs] [PATCH 2/4] ref-manual/structure.rst: update with info on PERSISTENT_DIR

[Alexander Kanavin]

On Wed, 9 Jul 2025 at 09:57, Antonin Godard via lists.yoctoproject.org
<antonin.godard=bootlin.com@lists.yoctoproject.org> wrote:
>
> Since we mention that SSTATE_DIR can be shared for multiple builds, also
> mention that PERSISTENT_DIR should be shared alongside SSTATE_DIR.

NO. These changes are incorrect, please do not merge or revert if
already merged.

PERSISTENT_DIR can be copied from one build directory to another when
bitbake is not running, but it cannot be shared between two active
builds.

If one wants to share sstate, they either have to setup a hash
equivalence server (and then sharing PERSISTENT_DIR is not needed), or
disable hash equivalency (in which case it is also not needed).

Alex

Re: [docs] [PATCH 2/4] ref-manual/structure.rst: update with info on PERSISTENT_DIR

[Richard Purdie]

On Fri, 2025-07-11 at 19:09 +0200, Alexander Kanavin via lists.yoctoproject.org wrote:
> On Wed, 9 Jul 2025 at 09:57, Antonin Godard via lists.yoctoproject.org
> <antonin.godard=bootlin.com@lists.yoctoproject.org> wrote:
> > 
> > Since we mention that SSTATE_DIR can be shared for multiple builds, also
> > mention that PERSISTENT_DIR should be shared alongside SSTATE_DIR.
> 
> NO. These changes are incorrect, please do not merge or revert if
> already merged.
> 
> PERSISTENT_DIR can be copied from one build directory to another when
> bitbake is not running, but it cannot be shared between two active
> builds.
> 
> If one wants to share sstate, they either have to setup a hash
> equivalence server (and then sharing PERSISTENT_DIR is not needed), or
> disable hash equivalency (in which case it is also not needed).

The changes did merge and definitely are unclear.

The point is that if you share sstate, you also need to have common
hash equivalence data.

That may be via having common hashequiv server, or, of you have a local
sstate directory and delete the build directory, you need to preserve
the hashequiv database file to make the sstate useful.

You cannot have a common PERSISTENT_DIR shared by two builds, that is
not supported.

Hopefully Antonin can tweak things to make this all clearer. I should
have caught this, sorry :/

Cheers,

Richard

Re: [docs] [PATCH 2/4] ref-manual/structure.rst: update with info on PERSISTENT_DIR

[Antonin Godard]

On Fri Jul 11, 2025 at 11:45 PM CEST, Richard Purdie wrote:
> On Fri, 2025-07-11 at 19:09 +0200, Alexander Kanavin via lists.yoctoproject.org wrote:
>> On Wed, 9 Jul 2025 at 09:57, Antonin Godard via lists.yoctoproject.org
>> <antonin.godard=bootlin.com@lists.yoctoproject.org> wrote:
>> > 
>> > Since we mention that SSTATE_DIR can be shared for multiple builds, also
>> > mention that PERSISTENT_DIR should be shared alongside SSTATE_DIR.
>> 
>> NO. These changes are incorrect, please do not merge or revert if
>> already merged.
>> 
>> PERSISTENT_DIR can be copied from one build directory to another when
>> bitbake is not running, but it cannot be shared between two active
>> builds.
>> 
>> If one wants to share sstate, they either have to setup a hash
>> equivalence server (and then sharing PERSISTENT_DIR is not needed), or
>> disable hash equivalency (in which case it is also not needed).
>
> The changes did merge and definitely are unclear.
>
> The point is that if you share sstate, you also need to have common
> hash equivalence data.
>
> That may be via having common hashequiv server, or, of you have a local
> sstate directory and delete the build directory, you need to preserve
> the hashequiv database file to make the sstate useful.
>
> You cannot have a common PERSISTENT_DIR shared by two builds, that is
> not supported.
>
> Hopefully Antonin can tweak things to make this all clearer. I should
> have caught this, sorry :/

Hi all,

Sorry! I drew a wrong conclusion from bug #15921, I will correct this soon
this week.

Alex, thanks for catching this.

Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH 3/4] dev-manual/start.rst: mention that PERSISTENT_DIR should be shared too

[Antonin Godard]

In the "Development environment summary", we mention that the shared
state dir can be shared for developers, but PERSISTENT_DIR should be
shared as well, otherwise bugs described in [1] can happen.

[1]: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15921

[YOCTO #15921]

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/dev-manual/start.rst | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/documentation/dev-manual/start.rst b/documentation/dev-manual/start.rst
index 44bd2de13..1c3f87ea9 100644
--- a/documentation/dev-manual/start.rst
+++ b/documentation/dev-manual/start.rst
@@ -228,6 +228,11 @@ particular working environment and set of practices.
        used by developers in the same organization and share the same
        source directories on their machines.
 
+       If the :term:`SSTATE_DIR` directory is shared for multiple developers,
+       and you are using a :doc:`/overview-manual/concepts:Hash Equivalence`
+       server with :term:`BB_HASHSERVE`, the :term:`PERSISTENT_DIR` directory
+       (``${TOPDIR}/cache`` by default) should be shared too.
+
     -  Set up an Autobuilder and have it populate the sstate cache and
        source directories.
 

-- 
2.50.0

Re: [docs] [PATCH 3/4] dev-manual/start.rst: mention that PERSISTENT_DIR should be shared too

[Alexander Kanavin]

On Wed, 9 Jul 2025 at 09:57, Antonin Godard via lists.yoctoproject.org
<antonin.godard=bootlin.com@lists.yoctoproject.org> wrote:
> +       If the :term:`SSTATE_DIR` directory is shared for multiple developers,
> +       and you are using a :doc:`/overview-manual/concepts:Hash Equivalence`
> +       server with :term:`BB_HASHSERVE`, the :term:`PERSISTENT_DIR` directory
> +       (``${TOPDIR}/cache`` by default) should be shared too.

Same as patch 2: incorrect, do not merge.

Alex

[PATCH 4/4] overview-manual/concepts.rst: mention PERSISTENT_DIR for user configuration

[Antonin Godard]

The PERSISTENT_DIR directory can also be customized to be shared, so
mention it in the list of configuration variables in local.conf.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/overview-manual/concepts.rst | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/documentation/overview-manual/concepts.rst b/documentation/overview-manual/concepts.rst
index b34de4d36..99a8f02a0 100644
--- a/documentation/overview-manual/concepts.rst
+++ b/documentation/overview-manual/concepts.rst
@@ -277,6 +277,9 @@ in the ``meta-poky`` layer:
 -  *Shared State Directory:* Controlled by the
    :term:`SSTATE_DIR` variable.
 
+-  *Persistent Data Directory:* Controlled by the
+   :term:`PERSISTENT_DIR` variable.
+
 -  *Build Output:* Controlled by the
    :term:`TMPDIR` variable.
 

-- 
2.50.0

Re: [docs] [PATCH 4/4] overview-manual/concepts.rst: mention PERSISTENT_DIR for user configuration

[Alexander Kanavin]

On Wed, 9 Jul 2025 at 09:57, Antonin Godard via lists.yoctoproject.org
<antonin.godard=bootlin.com@lists.yoctoproject.org> wrote:
> The PERSISTENT_DIR directory can also be customized to be shared, so
> mention it in the list of configuration variables in local.conf.

No. PESISTENT_DIR should never ever be shared.

Alex