[PATCH v3] sanity.bbclass: warn when sstate is outside of build dir, but hash equiv database is inside it

[PATCH v3] sanity.bbclass: warn when sstate is outside of build dir, but hash equiv database is inside it

[Alexander Kanavin]

From: Alexander Kanavin <alex@linutronix.de>

This should help with the long-standing usability problem: when
someone tweaks the configuration to put sstate somewhere else than
the default (so that it can be shared between local builds, or over NFS),
they should also share the hash equivalency database, but no indication
would be given to the user to do so.

This will issue a warning and recommend to start a dedicated hash equivalency
server (if sstate is on NFS), or set BB_HASHSERVE_DB_DIR (if it isn't).

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
v3: moved the check to oe-core's sanity class (v1/2 had it in bitbake cooker),
due to use of SSTATE_DIR variable which is specific to oe-core.
---
 meta/classes-global/sanity.bbclass | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/meta/classes-global/sanity.bbclass b/meta/classes-global/sanity.bbclass
index 7e81d65939..a170c3f5f5 100644
--- a/meta/classes-global/sanity.bbclass
+++ b/meta/classes-global/sanity.bbclass
@@ -1011,6 +1011,29 @@ def check_sanity_everybuild(status, d):
     if d.getVar("SSTATE_MIRRORS") and hashserv and hashserv.startswith("unix://") and not d.getVar("BB_HASHSERVE_UPSTREAM"):
         bb.warn("You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.")
 
+    # Check that when SSTATE_DIR is shared between builds, hashserve database is not private to a build
+    hashserv_proto,_,hashserv_path,_,_,_ = bb.fetch2.decodeurl(hashserv)
+    if hashserv_proto == "unix":
+        dbdir = d.getVar("BB_HASHSERVE_DB_DIR") or d.getVar("PERSISTENT_DIR") or d.getVar("CACHE")
+        topdir = d.getVar("TOPDIR")
+        sstatedir = d.getVar("SSTATE_DIR")
+
+        if (hashserv_path.startswith(topdir) and dbdir.startswith(topdir) and not sstatedir.startswith(topdir)):
+            if bb.utils.is_path_on_nfs(sstatedir):
+                bb.warn("""Sstate directory is on a shared NFS (it is set via SSTATE_DIR to {}),
+    but hash equivalency database is inside this particular build directory {}.
+
+    This will prevent sstate reuse, and it is recommended to set up a permanently running hash equivalency server
+    according to https://docs.yoctoproject.org/dev-manual/hashequivserver.html""".format(sstatedir, topdir))
+            else:
+                bb.warn("""Sstate directory is shared between several builds (it is set via SSTATE_DIR to {}),
+    but hash equivalency database is inside this particular build directory {}.
+
+    This will prevent sstate reuse, and it is recommended to set the location for the database to a common path
+    via BB_HASHSERVE_DB_DIR, for example:
+
+    BB_HASHSERVE_DB_DIR = \"${{SSTATE_DIR}}\"""".format(sstatedir, topdir))
+
     # Check that TMPDIR hasn't changed location since the last time we were run
     tmpdir = d.getVar('TMPDIR')
     checkfile = os.path.join(tmpdir, "saved_tmpdir")
-- 
2.47.3

Patchtest results for [PATCH v3] sanity.bbclass: warn when sstate is outside of build dir, but hash equiv database is inside it

[patchtest]

[-- Attachment #1: Type: text/plain, Size: 2255 bytes --]

Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch /home/patchtest/share/mboxes/v3-sanity.bbclass-warn-when-sstate-is-outside-of-build-dir-but-hash-equiv-database-is-inside-it.patch

FAIL: test shortlog length: Edit shortlog so that it is 90 characters or less (currently 94 characters) (test_mbox.TestMbox.test_shortlog_length)

PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest pylint: No python related patches, skipping test (test_python_pylint.PyLint.pretest_pylint)
SKIP: test CVE tag format: No new CVE patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new CVE patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new CVE patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test pylint: No python related patches, skipping test (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!