[PATCH] rootfs-postcommands: Set vardeps for write_image_test_data

[PATCH] rootfs-postcommands: Set vardeps for write_image_test_data

[Peter Hoyes]

From: Peter Hoyes <Peter.Hoyes@arm.com>

The testdata.json file generated as part of the rootfs postprocess
commands contains almost all Bitbake variables and is used by OEQA test
cases to inspect the build environment. However it is only regenerated
when the rootfs task is otherwise retriggered, complicating the process
of developing OEQA test cases.

Use the vardeps mechanism to add a dependency on all the generated
datastore keys. Split out exportkeys from export2json to support this.

Add a demonstrative OE selftest to rootfspostcommandstests.

Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
---
 .../rootfs-postcommands.bbclass               |  3 ++-
 meta/lib/oe/data.py                           |  9 +++++++--
 .../selftest/cases/rootfspostcommandstests.py | 19 +++++++++++++++++++
 3 files changed, 28 insertions(+), 3 deletions(-)

diff --git a/meta/classes-recipe/rootfs-postcommands.bbclass b/meta/classes-recipe/rootfs-postcommands.bbclass
index 690fa976aa..b91498c32e 100644
--- a/meta/classes-recipe/rootfs-postcommands.bbclass
+++ b/meta/classes-recipe/rootfs-postcommands.bbclass
@@ -386,7 +386,8 @@ python write_image_test_data() {
                 os.remove(testdata_link)
             os.symlink(os.path.basename(testdata_name), testdata_link)
 }
-write_image_test_data[vardepsexclude] += "TOPDIR"
+write_image_test_data[vardeps] += "${@' '.join(oe.data.exportkeys(d))}"
+write_image_test_data[vardepsexclude] += "TOPDIR DATETIME BUILDNAME ${BB_HASHCONFIG_IGNORE_VARS}"
 
 # Check for unsatisfied recommendations (RRECOMMENDS)
 python rootfs_log_check_recommends() {
diff --git a/meta/lib/oe/data.py b/meta/lib/oe/data.py
index 37121cfad2..9989230638 100644
--- a/meta/lib/oe/data.py
+++ b/meta/lib/oe/data.py
@@ -23,8 +23,7 @@ def typed_value(key, d):
     except (TypeError, ValueError) as exc:
         bb.msg.fatal("Data", "%s: %s" % (key, str(exc)))
 
-def export2json(d, json_file, expand=True, searchString="",replaceString=""):
-    data2export = {}
+def exportkeys(d):
     keys2export = []
 
     for key in d.keys():
@@ -41,6 +40,12 @@ def export2json(d, json_file, expand=True, searchString="",replaceString=""):
 
         keys2export.append(key)
 
+    return keys2export
+
+def export2json(d, json_file, expand=True, searchString="",replaceString=""):
+    data2export = {}
+    keys2export = exportkeys(d)
+
     for key in keys2export:
         try:
             data2export[key] = d.getVar(key, expand).replace(searchString,replaceString)
diff --git a/meta/lib/oeqa/selftest/cases/rootfspostcommandstests.py b/meta/lib/oeqa/selftest/cases/rootfspostcommandstests.py
index 44e2c09a6f..5de4ea378a 100644
--- a/meta/lib/oeqa/selftest/cases/rootfspostcommandstests.py
+++ b/meta/lib/oeqa/selftest/cases/rootfspostcommandstests.py
@@ -2,6 +2,7 @@
 #
 # SPDX-License-Identifier: MIT
 
+import json
 import os
 import oe
 import unittest
@@ -95,3 +96,21 @@ class ShadowUtilsTidyFiles(OESelftestTestCase):
                 unsorted.append(file)
         if (unsorted):
             raise Exception("The following files were not sorted by ID as expected: %s" % unsorted)
+
+
+class TestDataTests(OESelftestTestCase):
+    def test_vardeps(self):
+        """
+        Test that variables changes are reflected in testdata.json
+        """
+        test_image = "core-image-minimal"
+        self.write_config('TEST_VARIABLE = "VALUE1"')
+        bitbake(test_image)
+        self.write_config('TEST_VARIABLE = "VALUE2"')
+        bitbake(test_image)
+
+        vars = get_bb_vars(('DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'), test_image)
+        testdata_json = "%s/%s.testdata.json" % (vars['DEPLOY_DIR_IMAGE'], vars['IMAGE_LINK_NAME'])
+        with open(testdata_json, 'r') as tf:
+            testdata_vars = json.load(tf)
+        self.assertEqual(testdata_vars['TEST_VARIABLE'], 'VALUE2')
-- 
2.34.1

Re: [OE-core] [PATCH] rootfs-postcommands: Set vardeps for write_image_test_data

[Alexandre Belloni]

Hello,

This causes the following failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/2223/steps/14/logs/stdio

2023-05-23 09:43:18,091 - oe-selftest - INFO - sstatetests.SStateHashSameSigs.test_sstate_nativelsbstring_same_hash (subunit.RemotedTestCase)
2023-05-23 09:43:18,092 - oe-selftest - INFO -  ... FAIL

AssertionError: Element counts were not equal:
First has 1, Second has 0:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_image_ext4.sigdata.32fce21a23364030330f043131f6e715697e698907cf05253f68eb68747e8651'
First has 1, Second has 0:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_build.sigdata.7540db7c892d20b2702bcf638dc6c18a0d6edf6f60215167fc0742fefe6929f6'
First has 1, Second has 0:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_rootfs.sigdata.7906ddc0f34b4f06edecb890f66feef9150988e3bbda1532967aed1bd5559ef4'
First has 1, Second has 0:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_flush_pseudodb.sigdata.90cb70b6dc16db560cb18bf9836a11d5f149dee98aa7c2b5bfba19dca84847ea'
First has 1, Second has 0:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_image_qa.sigdata.1d332e993af8aba743e5e936b9d3dfebc3063cfb04d3fafc8fdf6dec29784d1e'
First has 1, Second has 0:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_populate_lic_deploy.sigdata.d27ea0bc010781b25fa2f4defaaea023a8592390aa0925cdec7654685b6c3fdc'
First has 1, Second has 0:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_write_qemuboot_conf.sigdata.e6b137de8f3b3129e908911e55b36c275918d80dc07f688b9c5d7c246cdd45f2'
First has 1, Second has 0:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_image.sigdata.be5b089f99d6088772be377211ba36c35c3782d5415b6f1a3ad52bc5aa7d4197'
First has 1, Second has 0:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_image_tar.sigdata.b66fe7a3630ad0bc32c9d9b0f8f549c294271709d863edc3e0f1ac58a5484b23'
First has 1, Second has 0:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_image_complete.sigdata.564504f044a363d2f273fa9a1ce157bc00bd862b962225883eeac04178475545'
First has 0, Second has 1:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_image_complete.sigdata.812cbe9664ec8201f7a46ee5fea4632c83c27053866feee414f35a809fba1b1f'
First has 0, Second has 1:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_rootfs.sigdata.f31109ec7b174060a3565b70a2e00f5739ec5f571f8bf7a0a39815b22a261573'
First has 0, Second has 1:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_build.sigdata.c49c5c0aa3bb6c94c722b2d434161fa4feb41493f81384d48b79e662502b08c8'
First has 0, Second has 1:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_image.sigdata.b220fa76a27967bd326e11d623dff03d19ca4290b50922ad77a2772ec2462e0c'
First has 0, Second has 1:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_populate_lic_deploy.sigdata.9826e7c099507f06923e61a827318e091cc5960cf7d04dc854f460b006fa317e'
First has 0, Second has 1:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_write_qemuboot_conf.sigdata.5c0899d566ed8e0b96483568df4bcb247ebe804fcd0ad22720a068ee502a97ce'
First has 0, Second has 1:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_flush_pseudodb.sigdata.b8d02d9da9c6571f53372393cc2e7eac0292515fc93864e0cba4b93268068ba2'
First has 0, Second has 1:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_image_ext4.sigdata.85541792d84ac6aba4a2637659440fc0c8b3fcd31635f02faabb97abbe3adb4a'
First has 0, Second has 1:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_image_qa.sigdata.093433c41fd25221ee766fd7479f13b66ab652a6ae65d699473279c48eb8a684'
First has 0, Second has 1:  '/home/pokybuild/yocto-worker/oe-selftest/build/build-st-3579934/tmp-sstatesamehash/stamps/qemux86_64-poky-linux/core-image-weston/1.0-r0.do_image_tar.sigdata.bfadfb2a2c3ed4f5d954a4419f88c0b223ae09e8dfdac9dadf7193f2a53581dd'


On 17/05/2023 17:29:06+0100, Peter Hoyes wrote:
> From: Peter Hoyes <Peter.Hoyes@arm.com>
> 
> The testdata.json file generated as part of the rootfs postprocess
> commands contains almost all Bitbake variables and is used by OEQA test
> cases to inspect the build environment. However it is only regenerated
> when the rootfs task is otherwise retriggered, complicating the process
> of developing OEQA test cases.
> 
> Use the vardeps mechanism to add a dependency on all the generated
> datastore keys. Split out exportkeys from export2json to support this.
> 
> Add a demonstrative OE selftest to rootfspostcommandstests.
> 
> Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
> ---
>  .../rootfs-postcommands.bbclass               |  3 ++-
>  meta/lib/oe/data.py                           |  9 +++++++--
>  .../selftest/cases/rootfspostcommandstests.py | 19 +++++++++++++++++++
>  3 files changed, 28 insertions(+), 3 deletions(-)
> 
> diff --git a/meta/classes-recipe/rootfs-postcommands.bbclass b/meta/classes-recipe/rootfs-postcommands.bbclass
> index 690fa976aa..b91498c32e 100644
> --- a/meta/classes-recipe/rootfs-postcommands.bbclass
> +++ b/meta/classes-recipe/rootfs-postcommands.bbclass
> @@ -386,7 +386,8 @@ python write_image_test_data() {
>                  os.remove(testdata_link)
>              os.symlink(os.path.basename(testdata_name), testdata_link)
>  }
> -write_image_test_data[vardepsexclude] += "TOPDIR"
> +write_image_test_data[vardeps] += "${@' '.join(oe.data.exportkeys(d))}"
> +write_image_test_data[vardepsexclude] += "TOPDIR DATETIME BUILDNAME ${BB_HASHCONFIG_IGNORE_VARS}"
>  
>  # Check for unsatisfied recommendations (RRECOMMENDS)
>  python rootfs_log_check_recommends() {
> diff --git a/meta/lib/oe/data.py b/meta/lib/oe/data.py
> index 37121cfad2..9989230638 100644
> --- a/meta/lib/oe/data.py
> +++ b/meta/lib/oe/data.py
> @@ -23,8 +23,7 @@ def typed_value(key, d):
>      except (TypeError, ValueError) as exc:
>          bb.msg.fatal("Data", "%s: %s" % (key, str(exc)))
>  
> -def export2json(d, json_file, expand=True, searchString="",replaceString=""):
> -    data2export = {}
> +def exportkeys(d):
>      keys2export = []
>  
>      for key in d.keys():
> @@ -41,6 +40,12 @@ def export2json(d, json_file, expand=True, searchString="",replaceString=""):
>  
>          keys2export.append(key)
>  
> +    return keys2export
> +
> +def export2json(d, json_file, expand=True, searchString="",replaceString=""):
> +    data2export = {}
> +    keys2export = exportkeys(d)
> +
>      for key in keys2export:
>          try:
>              data2export[key] = d.getVar(key, expand).replace(searchString,replaceString)
> diff --git a/meta/lib/oeqa/selftest/cases/rootfspostcommandstests.py b/meta/lib/oeqa/selftest/cases/rootfspostcommandstests.py
> index 44e2c09a6f..5de4ea378a 100644
> --- a/meta/lib/oeqa/selftest/cases/rootfspostcommandstests.py
> +++ b/meta/lib/oeqa/selftest/cases/rootfspostcommandstests.py
> @@ -2,6 +2,7 @@
>  #
>  # SPDX-License-Identifier: MIT
>  
> +import json
>  import os
>  import oe
>  import unittest
> @@ -95,3 +96,21 @@ class ShadowUtilsTidyFiles(OESelftestTestCase):
>                  unsorted.append(file)
>          if (unsorted):
>              raise Exception("The following files were not sorted by ID as expected: %s" % unsorted)
> +
> +
> +class TestDataTests(OESelftestTestCase):
> +    def test_vardeps(self):
> +        """
> +        Test that variables changes are reflected in testdata.json
> +        """
> +        test_image = "core-image-minimal"
> +        self.write_config('TEST_VARIABLE = "VALUE1"')
> +        bitbake(test_image)
> +        self.write_config('TEST_VARIABLE = "VALUE2"')
> +        bitbake(test_image)
> +
> +        vars = get_bb_vars(('DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'), test_image)
> +        testdata_json = "%s/%s.testdata.json" % (vars['DEPLOY_DIR_IMAGE'], vars['IMAGE_LINK_NAME'])
> +        with open(testdata_json, 'r') as tf:
> +            testdata_vars = json.load(tf)
> +        self.assertEqual(testdata_vars['TEST_VARIABLE'], 'VALUE2')
> -- 
> 2.34.1
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#181496): https://lists.openembedded.org/g/openembedded-core/message/181496
> Mute This Topic: https://lists.openembedded.org/mt/98951993/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH] rootfs-postcommands: Set vardeps for write_image_test_data

[Ross Burton]

On 17 May 2023, at 17:29, Peter Hoyes via lists.openembedded.org <peter.hoyes=arm.com@lists.openembedded.org> wrote:
> The testdata.json file generated as part of the rootfs postprocess
> commands contains almost all Bitbake variables and is used by OEQA test
> cases to inspect the build environment. However it is only regenerated
> when the rootfs task is otherwise retriggered, complicating the process
> of developing OEQA test cases.

I’m about to run for dinner, but would it make more sense for this function to be a separate task inside testimage.bbclass that happens after rootfs?  This will stop the explosion of variable dependencies causing rootfs to re-run.

Ross

Re: [OE-core] [PATCH] rootfs-postcommands: Set vardeps for write_image_test_data

[Peter Hoyes]

On 23/05/2023 18:06, Ross Burton wrote:
> On 17 May 2023, at 17:29, Peter Hoyes via lists.openembedded.org <peter.hoyes=arm.com@lists.openembedded.org> wrote:
>> The testdata.json file generated as part of the rootfs postprocess
>> commands contains almost all Bitbake variables and is used by OEQA test
>> cases to inspect the build environment. However it is only regenerated
>> when the rootfs task is otherwise retriggered, complicating the process
>> of developing OEQA test cases.
> I’m about to run for dinner, but would it make more sense for this function to be a separate task inside testimage.bbclass that happens after rootfs?  This will stop the explosion of variable dependencies causing rootfs to re-run.
>
> Ross

When running testimage under bitbake, would it make sense to simply do:

--- a/meta/classes-recipe/testimage.bbclass
+++ b/meta/classes-recipe/testimage.bbclass
@@ -350,7 +350,7 @@ def testimage_main(d):
          d.getVar("TEST_SERVER_IP"), **target_kwargs)

      # test context
-    tc = OERuntimeTestContext(td, logger, target, host_dumper,
+    tc = OERuntimeTestContext(d, logger, target, host_dumper,
                                image_packages, extract_dir)

      # Load tests before starting the target

(and a few other associated tidy-ups)

Peter

Re: [OE-core] [PATCH] rootfs-postcommands: Set vardeps for write_image_test_data

[Ross Burton]

On 24 May 2023, at 18:01, Peter Hoyes <Peter.Hoyes@arm.com> wrote:
> When running testimage under bitbake, would it make sense to simply do:
> 
> --- a/meta/classes-recipe/testimage.bbclass
> +++ b/meta/classes-recipe/testimage.bbclass
> @@ -350,7 +350,7 @@ def testimage_main(d):
>          d.getVar("TEST_SERVER_IP"), **target_kwargs)
> 
>      # test context
> -    tc = OERuntimeTestContext(td, logger, target, host_dumper,
> +    tc = OERuntimeTestContext(d, logger, target, host_dumper,
>                                image_packages, extract_dir)
> 
>      # Load tests before starting the target
> 
> (and a few other associated tidy-ups)

I’ll let Richard correct me, but I _think_ the intended use-case here is the ability to run testimage on a prebuild image that someone else build, with just a minimal oe-core.  So someone can give you an image that they’ve built, you put it in the right place and can testimage it, without having to have their configuration/layers/etc.

Honestly, no idea if that is actually used or tested.  Would be an interesting experiment!

Ross

Re: [OE-core] [PATCH] rootfs-postcommands: Set vardeps for write_image_test_data

[Peter Hoyes]

On 23/05/2023 18:06, Ross Burton wrote:
> On 17 May 2023, at 17:29, Peter Hoyes via lists.openembedded.org <peter.hoyes=arm.com@lists.openembedded.org> wrote:
>> The testdata.json file generated as part of the rootfs postprocess
>> commands contains almost all Bitbake variables and is used by OEQA test
>> cases to inspect the build environment. However it is only regenerated
>> when the rootfs task is otherwise retriggered, complicating the process
>> of developing OEQA test cases.
> I’m about to run for dinner, but would it make more sense for this function to be a separate task inside testimage.bbclass that happens after rootfs?  This will stop the explosion of variable dependencies causing rootfs to re-run.
>
> Ross

Thanks for the suggestion - I have implemented this for consideration in 
https://lists.openembedded.org/g/openembedded-core/topic/patch_1_2/99501785

Peter

Re: [OE-core] [PATCH] rootfs-postcommands: Set vardeps for write_image_test_data

[Richard Purdie]

On Tue, 2023-06-13 at 10:55 +0100, Peter Hoyes wrote:
> On 23/05/2023 18:06, Ross Burton wrote:
> > On 17 May 2023, at 17:29, Peter Hoyes via lists.openembedded.org
> > <peter.hoyes=arm.com@lists.openembedded.org> wrote:
> > > The testdata.json file generated as part of the rootfs
> > > postprocess
> > > commands contains almost all Bitbake variables and is used by
> > > OEQA test
> > > cases to inspect the build environment. However it is only
> > > regenerated
> > > when the rootfs task is otherwise retriggered, complicating the
> > > process
> > > of developing OEQA test cases.
> > I’m about to run for dinner, but would it make more sense for this
> > function to be a separate task inside testimage.bbclass that
> > happens after rootfs?  This will stop the explosion of variable
> > dependencies causing rootfs to re-run.
> > 
> > Ross
> 
> Thanks for the suggestion - I have implemented this for consideration
> in 
> https://lists.openembedded.org/g/openembedded-core/topic/patch_1_2/99501785

Making it a separate task is fine, the challenge is that it needs to
run at the same time as image processing so that DATE/TIME are
consistent and the image names are correct.

That implies that it has to be before do_image_complete which means any
change in variables will retrigger the image creation :(.

I think we need to try and fix this differently and stop exporting
every variable.

Cheers,

Richard

Re: [OE-core] [PATCH] rootfs-postcommands: Set vardeps for write_image_test_data

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2435 bytes --]

On Tue, Jun 13, 2023 at 1:33 PM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Tue, 2023-06-13 at 10:55 +0100, Peter Hoyes wrote:
> > On 23/05/2023 18:06, Ross Burton wrote:
> > > On 17 May 2023, at 17:29, Peter Hoyes via lists.openembedded.org
> > > <peter.hoyes=arm.com@lists.openembedded.org> wrote:
> > > > The testdata.json file generated as part of the rootfs
> > > > postprocess
> > > > commands contains almost all Bitbake variables and is used by
> > > > OEQA test
> > > > cases to inspect the build environment. However it is only
> > > > regenerated
> > > > when the rootfs task is otherwise retriggered, complicating the
> > > > process
> > > > of developing OEQA test cases.
> > > I’m about to run for dinner, but would it make more sense for this
> > > function to be a separate task inside testimage.bbclass that
> > > happens after rootfs?  This will stop the explosion of variable
> > > dependencies causing rootfs to re-run.
> > >
> > > Ross
> >
> > Thanks for the suggestion - I have implemented this for consideration
> > in
> >
> https://lists.openembedded.org/g/openembedded-core/topic/patch_1_2/99501785
>
> Making it a separate task is fine, the challenge is that it needs to
> run at the same time as image processing so that DATE/TIME are
> consistent and the image names are correct.
>

Maybe the final change from [YOCTO #12937] should help with this as with
that the do_image_complete will produce the images without version suffix
(and only when something in them changed) and separate task will create new
hardlinks with version suffix (which might point to the same image from
previous build or newly built one).

Cheers,


> That implies that it has to be before do_image_complete which means any
> change in variables will retrigger the image creation :(.
>
> I think we need to try and fix this differently and stop exporting
> every variable.
>
> Cheers,
>
> Richard
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#182712):
> https://lists.openembedded.org/g/openembedded-core/message/182712
> Mute This Topic: https://lists.openembedded.org/mt/98951993/3617156
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> Martin.Jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 3861 bytes --]

Re: [OE-core] [PATCH] rootfs-postcommands: Set vardeps for write_image_test_data

[Peter Hoyes]

[-- Attachment #1: Type: text/plain, Size: 2550 bytes --]

On 13/06/2023 12:33, Richard Purdie wrote:
> On Tue, 2023-06-13 at 10:55 +0100, Peter Hoyes wrote:
>> On 23/05/2023 18:06, Ross Burton wrote:
>>> On 17 May 2023, at 17:29, Peter Hoyes via lists.openembedded.org
>>> <peter.hoyes=arm.com@lists.openembedded.org>  wrote:
>>>> The testdata.json file generated as part of the rootfs
>>>> postprocess
>>>> commands contains almost all Bitbake variables and is used by
>>>> OEQA test
>>>> cases to inspect the build environment. However it is only
>>>> regenerated
>>>> when the rootfs task is otherwise retriggered, complicating the
>>>> process
>>>> of developing OEQA test cases.
>>> I’m about to run for dinner, but would it make more sense for this
>>> function to be a separate task inside testimage.bbclass that
>>> happens after rootfs?  This will stop the explosion of variable
>>> dependencies causing rootfs to re-run.
>>>
>>> Ross
>> Thanks for the suggestion - I have implemented this for consideration
>> in
>> https://lists.openembedded.org/g/openembedded-core/topic/patch_1_2/99501785
> Making it a separate task is fine, the challenge is that it needs to
> run at the same time as image processing so that DATE/TIME are
> consistent and the image names are correct.
>
> That implies that it has to be before do_image_complete which means any
> change in variables will retrigger the image creation :(.
>
> I think we need to try and fix this differently and stop exporting
> every variable.
>
> Cheers,
>
> Richard

If I understand correctly...

It sounds like moving testdata creation to a separate task doesn't bring 
much benefit so I'll revert to something based off this patch.

It looks like a relatively small number of variables are actually used 
by OEQA tests in OE-core:

  * IMAGE
  * ARCH
  * MACHINE
  * PACKAGE_FEED_GPG_NAME
  * VIRTUAL-RUNTIME_init_manager
  * WORKDIR
  * DISTRO_FEATURES
  * PTEST_EXPECT_FAILURE
  * TEST_LOG_DIR
  * libdir
  * T
  * MULTILIB_VARIANTS
  * SDKPATH
  * SDK_DEPLOY
  * TOOLCHAINEXT_OUTPUTNAME
  * QEMU_USE_KVM
  * DL_DIR

Notably, DATE/TIME do not seem to be referenced at all (apart from a 
comment in oeqa/runtime/cases/ptest.py recommending *not* using DATETIME 
for test output). Additionally, DL_DIR and WORKDIR are updated on each 
testimage execution using TESTIMAGE_UPDATE_VARS.

How about introducing a new variable, e.g. TESTIMAGE_EXPORT_VARS, 
containing just the list above (to which other variables could 
optionally be appended in other layers)?

Peter

[-- Attachment #2: Type: text/html, Size: 3883 bytes --]

Re: [OE-core] [PATCH] rootfs-postcommands: Set vardeps for write_image_test_data

[Richard Purdie]

On Mon, 2023-06-19 at 12:47 +0100, Peter Hoyes wrote:
>  
> It sounds like moving testdata creation to a separate task doesn't
> bring much benefit so I'll revert to something based off this patch.
> 
> It looks like a relatively small number of variables are actually
> used by OEQA tests in OE-core:
>  
>  * IMAGE
>  * ARCH
>  * MACHINE
>  * PACKAGE_FEED_GPG_NAME
>  * VIRTUAL-RUNTIME_init_manager
>  * WORKDIR
>  * DISTRO_FEATURES
>  * PTEST_EXPECT_FAILURE
>  * TEST_LOG_DIR
>  * libdir
>  * T
>  * MULTILIB_VARIANTS
>  * SDKPATH
>  * SDK_DEPLOY
>  * TOOLCHAINEXT_OUTPUTNAME
>  * QEMU_USE_KVM
>     
>  * DL_DIR
>     
> Notably, DATE/TIME do not seem to be referenced at all (apart from a
> comment in oeqa/runtime/cases/ptest.py recommending *not* using
> DATETIME for test output). Additionally, DL_DIR and WORKDIR are
> updated on each testimage execution using TESTIMAGE_UPDATE_VARS.
>  
>  
> How about introducing a new variable, e.g. TESTIMAGE_EXPORT_VARS,
> containing just the list above (to which other variables could
> optionally be appended in other layers)?
> 

I think that is certainly worth trying.

Cheers,

Richard

Re: [OE-core] [PATCH] rootfs-postcommands: Set vardeps for write_image_test_data

[Peter Hoyes]

On 19/06/2023 13:07, Richard Purdie wrote:
> On Mon, 2023-06-19 at 12:47 +0100, Peter Hoyes wrote:
>>   
>> It sounds like moving testdata creation to a separate task doesn't
>> bring much benefit so I'll revert to something based off this patch.
>>
>> It looks like a relatively small number of variables are actually
>> used by OEQA tests in OE-core:
>>   
>>   * IMAGE
>>   * ARCH
>>   * MACHINE
>>   * PACKAGE_FEED_GPG_NAME
>>   * VIRTUAL-RUNTIME_init_manager
>>   * WORKDIR
>>   * DISTRO_FEATURES
>>   * PTEST_EXPECT_FAILURE
>>   * TEST_LOG_DIR
>>   * libdir
>>   * T
>>   * MULTILIB_VARIANTS
>>   * SDKPATH
>>   * SDK_DEPLOY
>>   * TOOLCHAINEXT_OUTPUTNAME
>>   * QEMU_USE_KVM
>>      
>>   * DL_DIR
>>      
>> Notably, DATE/TIME do not seem to be referenced at all (apart from a
>> comment in oeqa/runtime/cases/ptest.py recommending *not* using
>> DATETIME for test output). Additionally, DL_DIR and WORKDIR are
>> updated on each testimage execution using TESTIMAGE_UPDATE_VARS.
>>   
>>   
>> How about introducing a new variable, e.g. TESTIMAGE_EXPORT_VARS,
>> containing just the list above (to which other variables could
>> optionally be appended in other layers)?
>>
> I think that is certainly worth trying.
>
> Cheers,
>
> Richard

This is implemented in v2.

I have validated the new patch locally on:

  * The new test case
  * sstatetests.SStateHashSameSigs
  * qemuarm64 core-image-sato testimage
  * qemuarm64 core-image-minimal-dev testimage

Thanks,

Peter