[PATCH 1/3] selftest/sstatetests: add tests for 'bitbake -S printdiff'

[PATCH 1/3] selftest/sstatetests: add tests for 'bitbake -S printdiff'

[Alexander Kanavin]

'bitbake -S printdiff' is a useful diagnostic facility for finding out
why sstate is not being reused, but until now it had no tests that would
ensure it works. This commit adds three basic scenarios:

1. make a change in a really basic, common recipe that is at the very root
of dependency trees (quilt-native), and ensure that change is correctly discovered when
building an image.

2. make a change in gcc-source recipe, which is somewhat special
(operates in work-shared), and ensure that gcc-runtime builds track
that down as well.

3. make a change in base_do_configure() definition from base.bbclass,
which is not recipe-specific, but affects many basic recipes, and ensure that
is correctly reported as well.

The test itself actually runs twice:
- first against a fully populated build directory, where
the printdiff code is guaranteed to find the correct previous
stamp that can be compared with in a predictable manner.

- then in an empty build directory where the printdiff code
goes to look in the sstate cache, and so the existence of the
previous signature can be tested, but not the difference with it
(what the exact difference would be is unpredictable as the
sstate cache is indeed shared between many builds).

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 .../base-do-configure-modified.bbclass        |   3 +
 .../gcc-source/gcc-source_%.bbappend          |   2 +
 .../quilt-native/quilt-native_%.bbappend      |   2 +
 meta/lib/oeqa/selftest/cases/sstatetests.py   | 113 ++++++++++++++++++
 4 files changed, 120 insertions(+)
 create mode 100644 meta-selftest/classes/base-do-configure-modified.bbclass
 create mode 100644 meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
 create mode 100644 meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend

diff --git a/meta-selftest/classes/base-do-configure-modified.bbclass b/meta-selftest/classes/base-do-configure-modified.bbclass
new file mode 100644
index 00000000000..3f96827a428
--- /dev/null
+++ b/meta-selftest/classes/base-do-configure-modified.bbclass
@@ -0,0 +1,3 @@
+base_do_configure:append () {
+	echo "this changes base_do_configure() definiton"
+}
diff --git a/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
new file mode 100644
index 00000000000..205720982cb
--- /dev/null
+++ b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
@@ -0,0 +1,2 @@
+# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests.
+include test_recipe.inc
diff --git a/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
new file mode 100644
index 00000000000..205720982cb
--- /dev/null
+++ b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
@@ -0,0 +1,2 @@
+# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests.
+include test_recipe.inc
diff --git a/meta/lib/oeqa/selftest/cases/sstatetests.py b/meta/lib/oeqa/selftest/cases/sstatetests.py
index bdad9088d37..b96eacc9ad8 100644
--- a/meta/lib/oeqa/selftest/cases/sstatetests.py
+++ b/meta/lib/oeqa/selftest/cases/sstatetests.py
@@ -773,3 +773,116 @@ addtask tmptask2 before do_tmptask1
                 latestfiles = sorted(filedates.keys(), key=lambda f: filedates[f])[-2:]
                 bb.siggen.compare_sigfiles(latestfiles[-2], latestfiles[-1], recursecb)
                 self.assertEqual(recursecb_count,1)
+
+class SStatePrintdiff(SStateBase):
+    # FIXME: OEBasicHash setting is necessary for now as otherwise the following error can occur:
+    # ERROR: Can't find a task we're supposed to have written out? (hash: e79d70b9c2cc72030c1ce822525510699a1eeb1ddf5986271d3217422244366a)?
+    # The underlying issue should be investigated and addressed.
+    def run_test_printdiff_changerecipe(self, target, change_recipe, change_bbtask, change_content, expected_sametmp_output, expected_difftmp_output):
+        self.write_config("""
+TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
+BB_SIGNATURE_HANDLER = "OEBasicHash"
+""")
+        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
+        bitbake(target)
+        bitbake("-S none {}".format(target))
+        bitbake(change_bbtask)
+        self.write_recipeinc(change_recipe, change_content)
+        result_sametmp = bitbake("-S printdiff {}".format(target))
+
+        self.write_config("""
+TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
+BB_SIGNATURE_HANDLER = "OEBasicHash"
+""")
+        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff-2")
+        result_difftmp = bitbake("-S printdiff {}".format(target))
+
+        self.delete_recipeinc(change_recipe)
+        for item in expected_sametmp_output:
+            self.assertIn(item, result_sametmp.output)
+        for item in expected_difftmp_output:
+            self.assertIn(item, result_difftmp.output)
+
+    def run_test_printdiff_changeconfig(self, target, change_content, expected_sametmp_output, expected_difftmp_output):
+        self.write_config("""
+TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
+BB_SIGNATURE_HANDLER = "OEBasicHash"
+""")
+        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
+        bitbake(target)
+        bitbake("-S none {}".format(target))
+        self.append_config(change_content)
+        result_sametmp = bitbake("-S printdiff {}".format(target))
+
+        self.write_config("""
+TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
+BB_SIGNATURE_HANDLER = "OEBasicHash"
+""")
+        self.append_config(change_content)
+        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff-2")
+        result_difftmp = bitbake("-S printdiff {}".format(target))
+
+        for item in expected_sametmp_output:
+            self.assertIn(item, result_sametmp.output)
+        for item in expected_difftmp_output:
+            self.assertIn(item, result_difftmp.output)
+
+
+    # Check if printdiff walks the full dependency chain from the image target to where the change is in a specific recipe
+    def test_image_minimal_vs_quilt(self):
+        expected_output = ("Task quilt-native:do_install couldn't be used from the cache because:",
+"We need hash",
+"most recent matching task was")
+        expected_sametmp_output = expected_output + ("Variable do_install value changed",'+    echo "this changes the task signature"')
+        expected_difftmp_output = expected_output
+
+        self.run_test_printdiff_changerecipe("core-image-minimal", "quilt-native", "-c do_install quilt-native",
+"""
+do_install:append() {
+    echo "this changes the task signature"
+}
+""",
+expected_sametmp_output, expected_difftmp_output)
+
+    # Check if changes to gcc-source (which uses tmp/work-shared) are correctly discovered
+    def test_gcc_runtime_vs_gcc_source(self):
+        gcc_source_pn = 'gcc-source-%s' % get_bb_vars(['PV'], 'gcc')['PV']
+
+        expected_output = ("Task {}:do_preconfigure couldn't be used from the cache because:".format(gcc_source_pn),
+"We need hash",
+"most recent matching task was")
+        expected_sametmp_output = expected_output + ("Variable do_preconfigure value changed",'+    print("this changes the task signature")')
+        #FIXME: printdiff is supposed to find at least one preconfigure task signature in the sstate cache, but isn't able to
+        #expected_difftmp_output = expected_output
+        expected_difftmp_output = ()
+
+        self.run_test_printdiff_changerecipe("gcc-runtime", "gcc-source", "-c do_preconfigure {}".format(gcc_source_pn),
+"""
+python do_preconfigure:append() {
+    print("this changes the task signature")
+}
+""",
+expected_sametmp_output, expected_difftmp_output)
+
+    # Check if changing a really base task definiton is reported against multiple core recipes using it
+    def test_image_minimal_vs_base_do_configure(self):
+        expected_output = ("Task zstd-native:do_configure couldn't be used from the cache because:",
+"Task texinfo-dummy-native:do_configure couldn't be used from the cache because:",
+"Task ldconfig-native:do_configure couldn't be used from the cache because:",
+"Task gettext-minimal-native:do_configure couldn't be used from the cache because:",
+"Task tzcode-native:do_configure couldn't be used from the cache because:",
+"Task makedevs-native:do_configure couldn't be used from the cache because:",
+"Task pigz-native:do_configure couldn't be used from the cache because:",
+"Task update-rc.d-native:do_configure couldn't be used from the cache because:",
+"Task unzip-native:do_configure couldn't be used from the cache because:",
+"Task gnu-config-native:do_configure couldn't be used from the cache because:",
+"We need hash",
+"most recent matching task was")
+        expected_sametmp_output = expected_output + ("Variable base_do_configure value changed",'+	echo "this changes base_do_configure() definiton"')
+        expected_difftmp_output = expected_output
+
+        self.run_test_printdiff_changeconfig("core-image-minimal",
+"""
+INHERIT += "base-do-configure-modified"
+""",
+expected_sametmp_output, expected_difftmp_output)
-- 
2.39.2

[PATCH 2/3] lib/oe/sstatesig.py: dump locked.sigs.inc only when explicitly asked via -S lockedsigs

[Alexander Kanavin]

This was writing out locked-sigs.inc into cwd with every
'bitbake -S' invocation. When the intent is only to to get task
stamps (-S none), or print the difference between them (-S printdiff),
the file is unnecessary clutter.

A couple of selftests/scripts were however relying on this, so they're
adjusted to explicitly request the file.

eSDK code calls dump_lockedsigs() separately via
oe.copy_buildsystem.generate_locked_sigs() and so isn't affected.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 meta/lib/oe/sstatesig.py                 | 7 ++++---
 meta/lib/oeqa/selftest/cases/archiver.py | 2 +-
 meta/lib/oeqa/selftest/cases/signing.py  | 2 +-
 scripts/lib/checklayer/__init__.py       | 2 +-
 4 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index 42e13a8c800..e250f51c124 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -142,9 +142,10 @@ class SignatureGeneratorOEBasicHashMixIn(object):
         super().set_taskdata(data[3:])
 
     def dump_sigs(self, dataCache, options):
-        sigfile = os.getcwd() + "/locked-sigs.inc"
-        bb.plain("Writing locked sigs to %s" % sigfile)
-        self.dump_lockedsigs(sigfile)
+        if 'lockedsigs' in options:
+            sigfile = os.getcwd() + "/locked-sigs.inc"
+            bb.plain("Writing locked sigs to %s" % sigfile)
+            self.dump_lockedsigs(sigfile)
         return super(bb.siggen.SignatureGeneratorBasicHash, self).dump_sigs(dataCache, options)
 
 
diff --git a/meta/lib/oeqa/selftest/cases/archiver.py b/meta/lib/oeqa/selftest/cases/archiver.py
index 3fa59fff510..3cb888c5067 100644
--- a/meta/lib/oeqa/selftest/cases/archiver.py
+++ b/meta/lib/oeqa/selftest/cases/archiver.py
@@ -141,7 +141,7 @@ class Archiver(OESelftestTestCase):
         pn = 'gcc-source-%s' % get_bb_vars(['PV'], 'gcc')['PV']
 
         # Generate the tasks signatures
-        bitbake('mc:mc1:%s mc:mc2:%s -c %s -S none' % (pn, pn, task))
+        bitbake('mc:mc1:%s mc:mc2:%s -c %s -S lockedsigs' % (pn, pn, task))
 
         # Check the tasks signatures
         # To be machine agnostic the tasks needs to generate the same signature for each machine
diff --git a/meta/lib/oeqa/selftest/cases/signing.py b/meta/lib/oeqa/selftest/cases/signing.py
index 322e753ed3b..18cce0ba258 100644
--- a/meta/lib/oeqa/selftest/cases/signing.py
+++ b/meta/lib/oeqa/selftest/cases/signing.py
@@ -191,7 +191,7 @@ class LockedSignatures(OESelftestTestCase):
 
         bitbake(test_recipe)
         # Generate locked sigs include file
-        bitbake('-S none %s' % test_recipe)
+        bitbake('-S lockedsigs %s' % test_recipe)
 
         feature = 'require %s\n' % locked_sigs_file
         feature += 'SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "warn"\n'
diff --git a/scripts/lib/checklayer/__init__.py b/scripts/lib/checklayer/__init__.py
index 0a0db2f02ac..8271ed7fe3b 100644
--- a/scripts/lib/checklayer/__init__.py
+++ b/scripts/lib/checklayer/__init__.py
@@ -307,7 +307,7 @@ def get_signatures(builddir, failsafe=False, machine=None, extravars=None):
     cmd += 'bitbake '
     if failsafe:
         cmd += '-k '
-    cmd += '-S none world'
+    cmd += '-S lockedsigs world'
     sigs_file = os.path.join(builddir, 'locked-sigs.inc')
     if os.path.exists(sigs_file):
         os.unlink(sigs_file)
-- 
2.39.2

[PATCH 3/3] selftest/sstatetests: add a test for CDN sstate cache

[Alexander Kanavin]

Specifically, the test checks that everything needed for building
standard oe-core images for x86_64 and arm64 is available from
the cache (with minor exceptions). Going forward, a complete
world check could be enabled and additional configurations,
but that requires improvements to performance of hash equivalence
server in particular.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 meta/lib/oeqa/selftest/cases/sstatetests.py | 47 +++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/sstatetests.py b/meta/lib/oeqa/selftest/cases/sstatetests.py
index b96eacc9ad8..e895f66360c 100644
--- a/meta/lib/oeqa/selftest/cases/sstatetests.py
+++ b/meta/lib/oeqa/selftest/cases/sstatetests.py
@@ -14,6 +14,7 @@ import re
 
 from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer, get_bb_vars
 from oeqa.selftest.case import OESelftestTestCase
+from oeqa.core.decorator import OETestTag
 
 import oe
 import bb.siggen
@@ -886,3 +887,49 @@ expected_sametmp_output, expected_difftmp_output)
 INHERIT += "base-do-configure-modified"
 """,
 expected_sametmp_output, expected_difftmp_output)
+
+@OETestTag("yocto-mirrors")
+class SStateMirrors(SStateBase):
+    def check_bb_output(self, output, exceptions):
+        in_tasks = False
+        missing_objects = []
+        for l in output.splitlines():
+            if "The differences between the current build and any cached tasks start at the following tasks" in l:
+                in_tasks = True
+                continue
+            if "Writing task signature files" in l:
+                in_tasks = False
+                continue
+            if in_tasks:
+                recipe_task = l.split("/")[-1]
+                recipe, task = recipe_task.split(":")
+                for e in exceptions:
+                    if e[0] in recipe and task == e[1]:
+                        break
+                else:
+                    missing_objects.append(recipe_task)
+        self.assertTrue(len(missing_objects) == 0, "Missing objects in the cache:\n{}".format("\n".join(missing_objects)))
+
+
+    def run_test_cdn_mirror(self, machine, targets, exceptions):
+        exceptions = exceptions + [[t, "do_deploy_source_date_epoch"] for t in targets.split()]
+        exceptions = exceptions + [[t, "do_image_qa"] for t in targets.split()]
+        self.config_sstate(True)
+        self.append_config("""
+MACHINE = "{}"
+BB_HASHSERVE_UPSTREAM = "hashserv.yocto.io:8687"
+SSTATE_MIRRORS ?= "file://.* http://cdn.jsdelivr.net/yocto/sstate/all/PATH;downloadfilename=PATH"
+""".format(machine))
+        result = bitbake("-S printdiff {}".format(targets))
+        self.check_bb_output(result.output, exceptions)
+
+    def test_cdn_mirror_qemux86_64(self):
+        # Example:
+        # exceptions = [ ["packagegroup-core-sdk","do_package"] ]
+        exceptions = []
+        self.run_test_cdn_mirror("qemux86-64", "core-image-minimal core-image-full-cmdline core-image-weston core-image-sato-sdk", exceptions)
+
+    def test_cdn_mirror_qemuarm64(self):
+        exceptions = []
+        # core-image-weston isn't produced for arm64 currently
+        self.run_test_cdn_mirror("qemuarm64", "core-image-minimal core-image-full-cmdline core-image-sato-sdk", exceptions)
-- 
2.39.2

Re: [OE-core] [PATCH 3/3] selftest/sstatetests: add a test for CDN sstate cache

[Luca Ceresoli]

Hello Alex,

On Mon, 23 Oct 2023 13:06:18 +0200
"Alexander Kanavin" <alex.kanavin@gmail.com> wrote:

> Specifically, the test checks that everything needed for building
> standard oe-core images for x86_64 and arm64 is available from
> the cache (with minor exceptions). Going forward, a complete
> world check could be enabled and additional configurations,
> but that requires improvements to performance of hash equivalence
> server in particular.
> 
> Signed-off-by: Alexander Kanavin <alex@linutronix.de>

This is triggering:

AssertionError: False is not true : Missing objects in the cache:
weston_12.0.2.bb:do_package_write_ipk
weston_12.0.2.bb:do_package_write_deb

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6091/steps/23/logs/stdio

Luca

-- 
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH 3/3] selftest/sstatetests: add a test for CDN sstate cache

[Alexander Kanavin]

On Tue, 24 Oct 2023 at 10:33, Luca Ceresoli <luca.ceresoli@bootlin.com> wrote:
> This is triggering:
>
> AssertionError: False is not true : Missing objects in the cache:
> weston_12.0.2.bb:do_package_write_ipk
> weston_12.0.2.bb:do_package_write_deb
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6091/steps/23/logs/stdio

This is for qemux86_64 and seems to indicate that world builds (with
multilib enabled) don't match plain builds (which don't build
core-image-weston). I'll resubmit with that removed.

Alex

Re: [OE-core] [PATCH 3/3] selftest/sstatetests: add a test for CDN sstate cache

[Richard Purdie]

On Thu, 2023-10-26 at 12:26 +0200, Alexander Kanavin wrote:
> On Tue, 24 Oct 2023 at 10:33, Luca Ceresoli <luca.ceresoli@bootlin.com> wrote:
> > This is triggering:
> > 
> > AssertionError: False is not true : Missing objects in the cache:
> > weston_12.0.2.bb:do_package_write_ipk
> > weston_12.0.2.bb:do_package_write_deb
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6091/steps/23/logs/stdio
> 
> This is for qemux86_64 and seems to indicate that world builds (with
> multilib enabled) don't match plain builds (which don't build
> core-image-weston). I'll resubmit with that removed.

Can we file a bug to look into that?

I'm also a little puzzled as this does seem to work some of the time as
I managed green builds of this.

Thanks,

Richard

Re: [OE-core] [PATCH 3/3] selftest/sstatetests: add a test for CDN sstate cache

[Alexander Kanavin]

On Thu, 26 Oct 2023 at 12:53, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> Can we file a bug to look into that?
>
> I'm also a little puzzled as this does seem to work some of the time as
> I managed green builds of this.

Filed:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15253

We need to think of ways to test/debug this that don't require running a-full.

Alex

Re: [OE-core] [PATCH 3/3] selftest/sstatetests: add a test for CDN sstate cache

[Richard Purdie]

On Thu, 2023-10-26 at 13:16 +0200, Alexander Kanavin wrote:
> On Thu, 26 Oct 2023 at 12:53, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > Can we file a bug to look into that?
> > 
> > I'm also a little puzzled as this does seem to work some of the time as
> > I managed green builds of this.
> 
> Filed:
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15253
> 
> We need to think of ways to test/debug this that don't require running a-full.

We can create a separate target (builder) on the AB to just run the
test?

Cheers,

Richard

Re: [OE-core] [PATCH 3/3] selftest/sstatetests: add a test for CDN sstate cache

[Alexander Kanavin]

On Thu, 26 Oct 2023 at 13:26, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> > We need to think of ways to test/debug this that don't require running a-full.
>
> We can create a separate target (builder) on the AB to just run the
> test?

But it needs to run just after the task (e.g. world or some other
which should be dynamically specified) that is supposed to create the
required objects. I guess it needs to be done with a custom branch on
yocto-autobuilder-helper which would define that, but then how to
trigger it through the UI?

Alex

Re: [OE-core] [PATCH 3/3] selftest/sstatetests: add a test for CDN sstate cache

[Richard Purdie]

On Thu, 2023-10-26 at 13:32 +0200, Alexander Kanavin wrote:
> On Thu, 26 Oct 2023 at 13:26, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > > We need to think of ways to test/debug this that don't require running a-full.
> > 
> > We can create a separate target (builder) on the AB to just run the
> > test?
> 
> But it needs to run just after the task (e.g. world or some other
> which should be dynamically specified) that is supposed to create the
> required objects. I guess it needs to be done with a custom branch on
> yocto-autobuilder-helper which would define that, but then how to
> trigger it through the UI?

We could run the appropriate build and then the test in the next step?
It could be a standalone builder target which just isn't triggered by
default? Not all targets have to be triggered by something day to day.

Cheers,

Richard

Re: [OE-core] [PATCH 3/3] selftest/sstatetests: add a test for CDN sstate cache

[Richard Purdie]

On Thu, 2023-10-26 at 13:32 +0200, Alexander Kanavin wrote:
> On Thu, 26 Oct 2023 at 13:26, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > > We need to think of ways to test/debug this that don't require running a-full.
> > 
> > We can create a separate target (builder) on the AB to just run the
> > test?
> 
> But it needs to run just after the task (e.g. world or some other
> which should be dynamically specified) that is supposed to create the
> required objects. I guess it needs to be done with a custom branch on
> yocto-autobuilder-helper which would define that, but then how to
> trigger it through the UI?

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6105/steps/25/logs/stdio

AssertionError: False is not true : Missing objects in the cache:
connman-gnome_0.7.bb:do_package_write_rpm
font-util_1.4.1.bb:do_package_write_rpm
gdk-pixbuf_2.42.10.bb:do_package_write_rpm
librsvg_2.56.3.bb:do_package_write_rpm
gstreamer1.0-plugins-bad_1.22.6.bb:do_package_write_rpm
gst-examples_1.18.6.bb:do_package_write_rpm

which is a little worrying :/

Cheers,

Richard

Re: [OE-core] [PATCH 3/3] selftest/sstatetests: add a test for CDN sstate cache

[Alexander Kanavin]

On Thu, 26 Oct 2023 at 14:53, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6105/steps/25/logs/stdio
>
> AssertionError: False is not true : Missing objects in the cache:
> connman-gnome_0.7.bb:do_package_write_rpm
> font-util_1.4.1.bb:do_package_write_rpm
> gdk-pixbuf_2.42.10.bb:do_package_write_rpm
> librsvg_2.56.3.bb:do_package_write_rpm
> gstreamer1.0-plugins-bad_1.22.6.bb:do_package_write_rpm
> gst-examples_1.18.6.bb:do_package_write_rpm
>
> which is a little worrying :/

I've seen this in local testing some days back too, with the same set
of missing objects, but I had ascribed it to master not being
populated yet. Seems like it's not that simple, but I don't have a
clear plan forward right now.

Alex

Re: [OE-core] [PATCH 3/3] selftest/sstatetests: add a test for CDN sstate cache

[Richard Purdie]

On Thu, 2023-10-26 at 16:36 +0200, Alexander Kanavin wrote:
> On Thu, 26 Oct 2023 at 14:53, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6105/steps/25/logs/stdio
> > 
> > AssertionError: False is not true : Missing objects in the cache:
> > connman-gnome_0.7.bb:do_package_write_rpm
> > font-util_1.4.1.bb:do_package_write_rpm
> > gdk-pixbuf_2.42.10.bb:do_package_write_rpm
> > librsvg_2.56.3.bb:do_package_write_rpm
> > gstreamer1.0-plugins-bad_1.22.6.bb:do_package_write_rpm
> > gst-examples_1.18.6.bb:do_package_write_rpm
> > 
> > which is a little worrying :/
> 
> I've seen this in local testing some days back too, with the same set
> of missing objects, but I had ascribed it to master not being
> populated yet. Seems like it's not that simple, but I don't have a
> clear plan forward right now.

Lets make the test print the hashes it can't find?

It would be useful to know if they're perhaps delayed somehow on the
CDN?

Seems odd it is the same list of things though.

We need to find the hashes we build, and the ones we can't find and see
if they're the same. If we get the siginfo files for them, we can
compare and see why they're changing.

Cheers,

Richard

Re: [OE-core] [PATCH 3/3] selftest/sstatetests: add a test for CDN sstate cache

[Alexander Kanavin]

On Thu, 26 Oct 2023 at 18:13, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> Lets make the test print the hashes it can't find?
>
> It would be useful to know if they're perhaps delayed somehow on the
> CDN?
>
> Seems odd it is the same list of things though.
>
> We need to find the hashes we build, and the ones we can't find and see
> if they're the same. If we get the siginfo files for them, we can
> compare and see why they're changing.

I've just submitted a modified patch for the test that prints hashes
that were requested if the test fails.

I've also done two run-throughs of the standalone builder for it:
https://autobuilder.yoctoproject.org/typhoon/#/builders/158

The first one failed (which was expected), then I ran two standalone
qemuarm64/qemux86-64 jobs (which did actually rebuild the missing
stuff like perf etc.), and the second run succeeded.

So let's see it fail mysteriously again in a-full or a-quick; there
should now be more info to try and dig into that.

Alex