Re: [OE-core] [PATCH 2/2] selftest/sstatetests: add a test for CDN sstate cache

[Alexandre Belloni <alexandre.belloni@bootlin.com>]

Hello Alex,

This doesn't apply cleanly, it seems the context is from another series.

On 20/10/2023 19:12:32+0200, Alexander Kanavin wrote:
> Specifically, the test checks that everything needed for building
> standard oe-core images for x86_64 and arm64 is available from
> the cache (with minor exceptions). Going forward, a complete
> world check could be enabled and additional configurations,
> but that requires improvements to performance of hash equivalence
> server in particular.
> 
> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> ---
>  meta/lib/oeqa/selftest/cases/sstatetests.py | 47 +++++++++++++++++++++
>  1 file changed, 47 insertions(+)
> 
> diff --git a/meta/lib/oeqa/selftest/cases/sstatetests.py b/meta/lib/oeqa/selftest/cases/sstatetests.py
> index 6ef339897bf..e392d6a1a63 100644
> --- a/meta/lib/oeqa/selftest/cases/sstatetests.py
> +++ b/meta/lib/oeqa/selftest/cases/sstatetests.py
> @@ -14,6 +14,7 @@ import re
>  
>  from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer, get_bb_vars
>  from oeqa.selftest.case import OESelftestTestCase
> +from oeqa.core.decorator import OETestTag
>  
>  import oe
>  import bb.siggen
> @@ -879,3 +880,49 @@ expected_sametmp_output, expected_difftmp_output)
>  INHERIT += "base-do-configure-modified"
>  """,
>  expected_sametmp_output, expected_difftmp_output)
> +
> +@OETestTag("yocto-mirrors")
> +class SStateMirrors(SStateBase):
> +    def check_bb_output(self, output, exceptions):
> +        in_tasks = False
> +        missing_objects = []
> +        for l in output.splitlines():
> +            if "The differences between the current build and any cached tasks start at the following tasks" in l:
> +                in_tasks = True
> +                continue
> +            if "Writing task signature files" in l:
> +                in_tasks = False
> +                continue
> +            if in_tasks:
> +                recipe_task = l.split("/")[-1]
> +                recipe, task = recipe_task.split(":")
> +                for e in exceptions:
> +                    if e[0] in recipe and task == e[1]:
> +                        break
> +                else:
> +                    missing_objects.append(recipe_task)
> +        self.assertTrue(len(missing_objects) == 0, "Missing objects in the cache:\n{}".format("\n".join(missing_objects)))
> +
> +
> +    def run_test_cdn_mirror(self, machine, targets, exceptions):
> +        exceptions = exceptions + [[t, "do_deploy_source_date_epoch"] for t in targets.split()]
> +        exceptions = exceptions + [[t, "do_image_qa"] for t in targets.split()]
> +        self.config_sstate(True)
> +        self.append_config("""
> +MACHINE = "{}"
> +BB_HASHSERVE_UPSTREAM = "hashserv.yocto.io:8687"
> +SSTATE_MIRRORS ?= "file://.* http://cdn.jsdelivr.net/yocto/sstate/all/PATH;downloadfilename=PATH"
> +""".format(machine))
> +        result = bitbake("-S printdiff {}".format(targets))
> +        self.check_bb_output(result.output, exceptions)
> +
> +    def test_cdn_mirror_qemux86_64(self):
> +        # Example:
> +        # exceptions = [ ["packagegroup-core-sdk","do_package"] ]
> +        exceptions = []
> +        self.run_test_cdn_mirror("qemux86-64", "core-image-minimal core-image-full-cmdline core-image-weston core-image-sato-sdk", exceptions)
> +
> +    def test_cdn_mirror_qemuarm64(self):
> +        exceptions = []
> +        # core-image-weston isn't produced for arm64 currently
> +        self.run_test_cdn_mirror("qemuarm64", "core-image-minimal core-image-full-cmdline core-image-sato-sdk", exceptions)
> -- 
> 2.39.2
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#189575): https://lists.openembedded.org/g/openembedded-core/message/189575
> Mute This Topic: https://lists.openembedded.org/mt/102085736/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com