[PATCH 0/2] Remove shared PERSISTENT_DIR mentions

[PATCH 0/2] Remove shared PERSISTENT_DIR mentions

[Antonin Godard]

I wrongfully thought that PERSISTENT_DIR could be shared after reading
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15921. Remove the
mentions of this I previously added. In dev-manual/start, mention the
hash equivalence server setup instead.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
Antonin Godard (2):
      ref-manual/structure.rst: remove shared PERSISTENT_DIR mentions
      dev-manual/start.rst: remove shared PERSISTENT_DIR mentions

 documentation/dev-manual/start.rst     | 14 ++++++++++----
 documentation/ref-manual/structure.rst | 11 -----------
 2 files changed, 10 insertions(+), 15 deletions(-)
---
base-commit: 576677eae6960dbc2d2ececeba0fde5bba7bb69f
change-id: 20250715-fix-persistent-dir-e7ec4cf42a1a

Best regards,
--  
Antonin Godard <antonin.godard@bootlin.com>

[PATCH 1/2] ref-manual/structure.rst: remove shared PERSISTENT_DIR mentions

[Antonin Godard]

PERSISTENT_DIR should _not_ be shared, I got that wrong in my previous
commit 741aa29898dc ("ref-manual/structure.rst: update with info on
PERSISTENT_DIR"). Remove these mentions.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/ref-manual/structure.rst | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/documentation/ref-manual/structure.rst b/documentation/ref-manual/structure.rst
index bf23e2cfe..c9c44bc63 100644
--- a/documentation/ref-manual/structure.rst
+++ b/documentation/ref-manual/structure.rst
@@ -365,13 +365,6 @@ directory for multiple builds or move the directory to another location.
 You can control the location of this directory through the
 :term:`SSTATE_DIR` variable.
 
-.. note::
-
-   If you are sharing the :term:`SSTATE_DIR` directory for multiple builds and
-   you are using a :ref:`overview-manual/concepts:Hash Equivalence` server with
-   :term:`BB_HASHSERVE` you should also share the :term:`PERSISTENT_DIR`
-   directory. See :ref:`structure-build-cache` below.
-
 .. _structure-build-tmp:
 
 ``build/tmp/``
@@ -411,10 +404,6 @@ modification time (mtime) changes and hashing of file contents. If no
 changes to the file are detected, then the parsed result stored in the
 cache is reused. If the file has changed, it is reparsed.
 
-This directory and the :term:`SSTATE_DIR` directory can be shared for multiple
-builds when using a :ref:`overview-manual/concepts:Hash Equivalence` server
-with :term:`BB_HASHSERVE`.
-
 .. _structure-build-tmp-deploy:
 
 ``build/tmp/deploy/``

-- 
2.50.0

[PATCH 2/2] dev-manual/start.rst: remove shared PERSISTENT_DIR mentions

[Antonin Godard]

Remove the shared PERSISTENT_DIR mention introduced by 3954eda78f22
("dev-manual/start.rst: mention that PERSISTENT_DIR should be shared
too"), as it should _not_ be shared.

Instead recommend setting up a hash equivalence server.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/dev-manual/start.rst | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/documentation/dev-manual/start.rst b/documentation/dev-manual/start.rst
index 7aea3974a..e10367752 100644
--- a/documentation/dev-manual/start.rst
+++ b/documentation/dev-manual/start.rst
@@ -228,10 +228,16 @@ particular working environment and set of practices.
        used by developers in the same organization and share the same
        source directories on their machines.
 
-       If the :term:`SSTATE_DIR` directory is shared for multiple developers,
-       and you are using a :ref:`overview-manual/concepts:Hash Equivalence`
-       server with :term:`BB_HASHSERVE`, the :term:`PERSISTENT_DIR` directory
-       (``${TOPDIR}/cache`` by default) should be shared too.
+    -  Set up a local :ref:`overview-manual/concepts:Hash Equivalence` server
+       with :term:`BB_HASHSERVE` and :term:`BB_SIGNATURE_HANDLER`. A basic setup
+       would be::
+
+          BB_HASHSERVE = "auto"
+          BB_SIGNATURE_HANDLER = "OEEquivHash"
+
+       See the :ref:`overview-manual/concepts:Hash Equivalence` section of the
+       Yocto Project Overview and Concepts Manual for more details on the hash
+       equivalence feature.
 
     -  Set up an Autobuilder and have it populate the sstate cache and
        source directories.

-- 
2.50.0

Re: [docs] [PATCH 2/2] dev-manual/start.rst: remove shared PERSISTENT_DIR mentions

[Alexander Kanavin]

On Tue, 15 Jul 2025 at 11:56, Antonin Godard via
lists.yoctoproject.org
<antonin.godard=bootlin.com@lists.yoctoproject.org> wrote:

> +    -  Set up a local :ref:`overview-manual/concepts:Hash Equivalence` server
> +       with :term:`BB_HASHSERVE` and :term:`BB_SIGNATURE_HANDLER`. A basic setup
> +       would be::
> +
> +          BB_HASHSERVE = "auto"
> +          BB_SIGNATURE_HANDLER = "OEEquivHash"

This is still problematic and should be clarified further:

1. BB_HASHSERVE and BB_SIGNATURE_HANDLER do not set up or start a
server, they set up the hash equivalence client built into bitbake.

2. The above client configuration (used by poky) also starts a private
server that is specific to a particular build directory. So hash equiv
information would not be shared between multiple build directories,
and this will cause sstate mismatches (as reported in the bug), if
sstate is shared.

Maybe it's easier to just remove the above, and only refer to the hash
equiv section in the other manual:

+    -  Set up a local :ref:`overview-manual/concepts :Hash Equivalence` server
+
+       See the :ref:`overview-manual/concepts:Hash Equivalence` section of the
+       Yocto Project Overview and Concepts Manual for more details on the hash
+       equivalence feature.

Then we can do a correct, comprehensive description there.

Alex

Re: [docs] [PATCH 2/2] dev-manual/start.rst: remove shared PERSISTENT_DIR mentions

[Antonin Godard]

On Tue Jul 15, 2025 at 5:55 PM CEST, Alexander Kanavin wrote:
> On Tue, 15 Jul 2025 at 11:56, Antonin Godard via
> lists.yoctoproject.org
> <antonin.godard=bootlin.com@lists.yoctoproject.org> wrote:
>
>> +    -  Set up a local :ref:`overview-manual/concepts:Hash Equivalence` server
>> +       with :term:`BB_HASHSERVE` and :term:`BB_SIGNATURE_HANDLER`. A basic setup
>> +       would be::
>> +
>> +          BB_HASHSERVE = "auto"
>> +          BB_SIGNATURE_HANDLER = "OEEquivHash"
>
> This is still problematic and should be clarified further:
>
> 1. BB_HASHSERVE and BB_SIGNATURE_HANDLER do not set up or start a
> server, they set up the hash equivalence client built into bitbake.
>
> 2. The above client configuration (used by poky) also starts a private
> server that is specific to a particular build directory. So hash equiv
> information would not be shared between multiple build directories,
> and this will cause sstate mismatches (as reported in the bug), if
> sstate is shared.
>
> Maybe it's easier to just remove the above, and only refer to the hash
> equiv section in the other manual:
>
> +    -  Set up a local :ref:`overview-manual/concepts :Hash Equivalence` server
> +
> +       See the :ref:`overview-manual/concepts:Hash Equivalence` section of the
> +       Yocto Project Overview and Concepts Manual for more details on the hash
> +       equivalence feature.
>
> Then we can do a correct, comprehensive description there.

Yes, I also feel that hash equivalence could deserve its own document, and
should go further than a server/client local to the build directory only. Thanks
for the clarification.

I will try to work on this soon, at least for now the shared PERSISTENT_DIR
mention is reverted, which is good.

Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH 2/2] dev-manual/start.rst: remove shared PERSISTENT_DIR mentions

[Alexander Kanavin]

On Wed, 16 Jul 2025 at 09:28, Antonin Godard <antonin.godard@bootlin.com> wrote:

> Yes, I also feel that hash equivalence could deserve its own document, and
> should go further than a server/client local to the build directory only. Thanks
> for the clarification.
>
> I will try to work on this soon, at least for now the shared PERSISTENT_DIR
> mention is reverted, which is good.

That would be good, I also feel that the whole hash equivalency
feature is very under-documented, and poorly understood as a result.
We could probably also issue a bitbake warning when sstate is shared
(e.g. placed outside of ${TOPDIR}) but hash equivalency information is
not.

But please remove the suggestions first, they're just repeating poky
defaults and will cause sstate mismatches with shared sstate.

Alex