[OE-Core][PATCH v2 0/5] testimage: add failed test post actions and fetch more data

[OE-Core][PATCH v2 0/5] testimage: add failed test post actions and fetch more data

[Alexis Lothoré]

Hello,
this small series is related to some disk space issue observed by the SWAT
team (see [1]) which eventually leads to tests failure. In order to help
diagnose those issues, I propose to enrich the retrieved artifacts with
some additional data, like disk usage on target and on host (in case the
target is a virtualized guest)

This v2 is mostly about creating a python module instead of a bbclass.
Link to v1: [2]

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15220
[2] https://lore.kernel.org/openembedded-core/20240220200159.13419-1-alexis.lothore@bootlin.com/

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>

Alexis Lothoré (5):
  lib/oeqa: share get_json_result_dir helper
  testimage: create a list of failed test post actions
  oeqa/utils/postactions: isolate directory creation in dedicated action
  oeqa/utils/postactions: add target disk usage stat as post action
  oeqa/utils/postactions: testimage: add host disk usage stat as post
    action

 meta/classes-recipe/testimage.bbclass | 54 ++-------------
 meta/lib/oeqa/sdk/testsdk.py          | 11 +--
 meta/lib/oeqa/utils/__init__.py       |  7 ++
 meta/lib/oeqa/utils/postactions.py    | 98 +++++++++++++++++++++++++++
 4 files changed, 112 insertions(+), 58 deletions(-)
 create mode 100644 meta/lib/oeqa/utils/postactions.py

-- 
2.43.1

[OE-Core][PATCH v2 1/5] lib/oeqa: share get_json_result_dir helper

[Alexis Lothoré]

From: Alexis Lothoré <alexis.lothore@bootlin.com>

Multiple places in oeqa need to get the log output path, and redefine a
small helper to accomplish this

Define this helper in lib/oeqa/utils/__init__.py and import it wherever
needed to allow using it.

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
---
There is one additional place re-definining (slightly) differently this
helper, which is in selftest/context.py. This one does not check
OEQA_JSON_RESULT_DIR from the datastore but through test data embedded in
the test context. Trying to check the origin of this test data, I
eventually see that it comes from bitbake environment, so I am not sure it
is 100% compatible with the datastore content. Please let me know if I'm
wrong, if so I can also replace selftest redefinition with the shared
helper too.
---
 meta/classes-recipe/testimage.bbclass | 12 +++---------
 meta/lib/oeqa/sdk/testsdk.py          | 11 ++---------
 meta/lib/oeqa/utils/__init__.py       |  7 +++++++
 3 files changed, 12 insertions(+), 18 deletions(-)

diff --git a/meta/classes-recipe/testimage.bbclass b/meta/classes-recipe/testimage.bbclass
index bee19674ef4f..c450566dadea 100644
--- a/meta/classes-recipe/testimage.bbclass
+++ b/meta/classes-recipe/testimage.bbclass
@@ -149,13 +149,6 @@ def get_testimage_configuration(d, test_type, machine):
     return configuration
 get_testimage_configuration[vardepsexclude] = "DATETIME"
 
-def get_testimage_json_result_dir(d):
-    json_result_dir = os.path.join(d.getVar("LOG_DIR"), 'oeqa')
-    custom_json_result_dir = d.getVar("OEQA_JSON_RESULT_DIR")
-    if custom_json_result_dir:
-        json_result_dir = custom_json_result_dir
-    return json_result_dir
-
 def get_testimage_result_id(configuration):
     return '%s_%s_%s_%s' % (configuration['TEST_TYPE'], configuration['IMAGE_BASENAME'], configuration['MACHINE'], configuration['STARTTIME'])
 
@@ -224,6 +217,7 @@ def testimage_main(d):
     from oeqa.core.target.qemu import supported_fstypes
     from oeqa.core.utils.test import getSuiteCases
     from oeqa.utils import make_logger_bitbake_compatible
+    from oeqa.utils import get_json_result_dir
 
     def sigterm_exception(signum, stackframe):
         """
@@ -427,14 +421,14 @@ def testimage_main(d):
     # Show results (if we have them)
     if results:
         configuration = get_testimage_configuration(d, 'runtime', machine)
-        results.logDetails(get_testimage_json_result_dir(d),
+        results.logDetails(get_json_result_dir(d),
                         configuration,
                         get_testimage_result_id(configuration),
                         dump_streams=d.getVar('TESTREPORT_FULLLOGS'))
         results.logSummary(pn)
 
     # Copy additional logs to tmp/log/oeqa so it's easier to find them
-    targetdir = os.path.join(get_testimage_json_result_dir(d), d.getVar("PN"))
+    targetdir = os.path.join(get_json_result_dir(d), d.getVar("PN"))
     os.makedirs(targetdir, exist_ok=True)
     os.symlink(bootlog, os.path.join(targetdir, os.path.basename(bootlog)))
     os.symlink(d.getVar("BB_LOGFILE"), os.path.join(targetdir, os.path.basename(d.getVar("BB_LOGFILE") + "." + d.getVar('DATETIME'))))
diff --git a/meta/lib/oeqa/sdk/testsdk.py b/meta/lib/oeqa/sdk/testsdk.py
index b4719110edbc..518b09febb61 100644
--- a/meta/lib/oeqa/sdk/testsdk.py
+++ b/meta/lib/oeqa/sdk/testsdk.py
@@ -22,14 +22,6 @@ class TestSDKBase(object):
                         'LAYERS': get_layers(d.getVar("BBLAYERS"))}
         return configuration
 
-    @staticmethod
-    def get_sdk_json_result_dir(d):
-        json_result_dir = os.path.join(d.getVar("LOG_DIR"), 'oeqa')
-        custom_json_result_dir = d.getVar("OEQA_JSON_RESULT_DIR")
-        if custom_json_result_dir:
-            json_result_dir = custom_json_result_dir
-        return json_result_dir
-
     @staticmethod
     def get_sdk_result_id(configuration):
         return '%s_%s_%s_%s_%s' % (configuration['TEST_TYPE'], configuration['IMAGE_BASENAME'], configuration['SDKMACHINE'], configuration['MACHINE'], configuration['STARTTIME'])
@@ -72,6 +64,7 @@ class TestSDK(TestSDKBase):
 
         from bb.utils import export_proxies
         from oeqa.utils import make_logger_bitbake_compatible
+        from oeqa.utils import get_json_result_dir
 
         pn = d.getVar("PN")
         logger = make_logger_bitbake_compatible(logging.getLogger("BitBake"))
@@ -134,7 +127,7 @@ class TestSDK(TestSDKBase):
             component = "%s %s" % (pn, self.context_executor_class.name)
             context_msg = "%s:%s" % (os.path.basename(tcname), os.path.basename(sdk_env))
             configuration = self.get_sdk_configuration(d, self.test_type)
-            result.logDetails(self.get_sdk_json_result_dir(d),
+            result.logDetails(get_json_result_dir(d),
                             configuration,
                             self.get_sdk_result_id(configuration))
             result.logSummary(component, context_msg)
diff --git a/meta/lib/oeqa/utils/__init__.py b/meta/lib/oeqa/utils/__init__.py
index fbc7f7d525d8..53bdcbf26618 100644
--- a/meta/lib/oeqa/utils/__init__.py
+++ b/meta/lib/oeqa/utils/__init__.py
@@ -90,3 +90,10 @@ def load_test_components(logger, executor):
                                 "_executor_class defined." % (comp_name, comp_context))
 
     return components
+
+def get_json_result_dir(d):
+    json_result_dir = os.path.join(d.getVar("LOG_DIR"), 'oeqa')
+    custom_json_result_dir = d.getVar("OEQA_JSON_RESULT_DIR")
+    if custom_json_result_dir:
+        json_result_dir = custom_json_result_dir
+    return json_result_dir
\ No newline at end of file
-- 
2.43.1

[OE-Core][PATCH v2 2/5] testimage: create a list of failed test post actions

[Alexis Lothoré]

From: Alexis Lothoré <alexis.lothore@bootlin.com>

testimage is able to detect whenever a test run leads to some tests
failing, and execute some actions in this case. The only action currently
defined in such case is to retrieve artifacts from the target under test,
as listed in TESTIMAGE_FAILED_QA_ARTIFACTS

In order to be able to add multiple actions, define a central function to
gather all "post actions" to run whenever a test has failed
(run_failed_tests_post_actions). This function contains a table listing all
functions to be called whenever a test fails. Any function in this table
will be provided with bitbake internal data dictionary ("d") and the
current runtime testing context ("tc"). Isolate all this feature in a
dedicated postactions.py file inherited by testimage.
This patch does not bring any functional change.

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
---
Changes in v2:
- move functions in a lib in oeqa instead of a bbclass
- use new shared helper get_json_result_dir()
---
 meta/classes-recipe/testimage.bbclass | 42 +----------------
 meta/lib/oeqa/utils/postactions.py    | 68 +++++++++++++++++++++++++++
 2 files changed, 70 insertions(+), 40 deletions(-)
 create mode 100644 meta/lib/oeqa/utils/postactions.py

diff --git a/meta/classes-recipe/testimage.bbclass b/meta/classes-recipe/testimage.bbclass
index c450566dadea..ad040ee8f071 100644
--- a/meta/classes-recipe/testimage.bbclass
+++ b/meta/classes-recipe/testimage.bbclass
@@ -170,40 +170,6 @@ def get_testimage_boot_patterns(d):
                 boot_patterns[flag] = flagval.encode().decode('unicode-escape')
     return boot_patterns
 
-def get_artifacts_list(target, raw_list):
-    result = []
-    # Passed list may contains patterns in paths, expand them directly on target
-    for raw_path in raw_list.split():
-        cmd = f"for p in {raw_path}; do if [ -e $p ]; then echo $p; fi; done"
-        try:
-            status, output = target.run(cmd)
-            if status != 0 or not output:
-                raise Exception()
-            result += output.split()
-        except:
-            bb.note(f"No file/directory matching path {raw_path}")
-
-    return result
-
-def retrieve_test_artifacts(target, artifacts_list, target_dir):
-    import shutil
-
-    local_artifacts_dir = os.path.join(target_dir, "artifacts")
-    if os.path.isdir(local_artifacts_dir):
-        shutil.rmtree(local_artifacts_dir)
-
-    os.makedirs(local_artifacts_dir)
-    for artifact_path in artifacts_list:
-        if not os.path.isabs(artifact_path):
-            bb.warn(f"{artifact_path} is not an absolute path")
-            continue
-        try:
-            dest_dir = os.path.join(local_artifacts_dir, os.path.dirname(artifact_path[1:]))
-            os.makedirs(dest_dir, exist_ok=True)
-            target.copyFrom(artifact_path, dest_dir)
-        except Exception as e:
-            bb.warn(f"Can not retrieve {artifact_path} from test target: {e}")
-
 def testimage_main(d):
     import os
     import json
@@ -218,6 +184,7 @@ def testimage_main(d):
     from oeqa.core.utils.test import getSuiteCases
     from oeqa.utils import make_logger_bitbake_compatible
     from oeqa.utils import get_json_result_dir
+    from oeqa.utils.postactions import run_failed_tests_post_actions
 
     def sigterm_exception(signum, stackframe):
         """
@@ -400,12 +367,7 @@ def testimage_main(d):
         results = tc.runTests()
         complete = True
         if results.hasAnyFailingTest():
-            artifacts_list = get_artifacts_list(tc.target, d.getVar("TESTIMAGE_FAILED_QA_ARTIFACTS"))
-            if not artifacts_list:
-                bb.warn("Could not load artifacts list, skip artifacts retrieval")
-            else:
-                bb.warn(f"Retrieving artifacts: {d.getVar('TESTIMAGE_FAILED_QA_ARTIFACTS')}")
-                retrieve_test_artifacts(tc.target, artifacts_list, get_testimage_json_result_dir(d))
+            run_failed_tests_post_actions(d, tc)
     except (KeyboardInterrupt, BlockingIOError) as err:
         if isinstance(err, KeyboardInterrupt):
             bb.error('testimage interrupted, shutting down...')
diff --git a/meta/lib/oeqa/utils/postactions.py b/meta/lib/oeqa/utils/postactions.py
new file mode 100644
index 000000000000..09c338ef6886
--- /dev/null
+++ b/meta/lib/oeqa/utils/postactions.py
@@ -0,0 +1,68 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+# Run a set of actions after tests. The runner provides internal data
+# dictionary as well as test context to any action to run.
+
+from oeqa.utils import get_json_result_dir
+
+##################################################################
+# Artifacts retrieval
+##################################################################
+
+def get_artifacts_list(target, raw_list):
+    result = []
+    # Passed list may contains patterns in paths, expand them directly on target
+    for raw_path in raw_list.split():
+        cmd = f"for p in {raw_path}; do if [ -e $p ]; then echo $p; fi; done"
+        try:
+            status, output = target.run(cmd)
+            if status != 0 or not output:
+                raise Exception()
+            result += output.split()
+        except:
+            bb.note(f"No file/directory matching path {raw_path}")
+
+    return result
+
+def retrieve_test_artifacts(target, artifacts_list, target_dir):
+    import shutil
+
+    local_artifacts_dir = os.path.join(target_dir, "artifacts")
+    if os.path.isdir(local_artifacts_dir):
+        shutil.rmtree(local_artifacts_dir)
+
+    os.makedirs(local_artifacts_dir)
+    for artifact_path in artifacts_list:
+        if not os.path.isabs(artifact_path):
+            bb.warn(f"{artifact_path} is not an absolute path")
+            continue
+        try:
+            dest_dir = os.path.join(local_artifacts_dir, os.path.dirname(artifact_path[1:]))
+            os.makedirs(dest_dir, exist_ok=True)
+            target.copyFrom(artifact_path, dest_dir)
+        except Exception as e:
+            bb.warn(f"Can not retrieve {artifact_path} from test target: {e}")
+
+def list_and_fetch_failed_tests_artifacts(d, tc):
+    artifacts_list = get_artifacts_list(tc.target, d.getVar("TESTIMAGE_FAILED_QA_ARTIFACTS"))
+    if not artifacts_list:
+        bb.warn("Could not load artifacts list, skip artifacts retrieval")
+    else:
+        retrieve_test_artifacts(tc.target, artifacts_list, get_json_result_dir(d))
+
+
+##################################################################
+# General post actions runner
+##################################################################
+
+def run_failed_tests_post_actions(d, tc):
+    post_actions=[
+        list_and_fetch_failed_tests_artifacts
+    ]
+
+    for action in post_actions:
+        action(d, tc)
-- 
2.43.1

[OE-Core][PATCH v2 3/5] oeqa/utils/postactions: isolate directory creation in dedicated action

[Alexis Lothoré]

From: Alexis Lothoré <alexis.lothore@bootlin.com>

In order to be able to create actions that could store new files during
failed test post actions, we need to split artifacts directory creation
from artifacts retrieval.

Create a new dedicated action to create artifacts main directory so we can
add actions creating files in this new directory, without worrying about
actions order if at least this action is set first.

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
---
Changes in v2:
- use new shared helper get_json_result_dir()
---
 meta/lib/oeqa/utils/postactions.py | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/meta/lib/oeqa/utils/postactions.py b/meta/lib/oeqa/utils/postactions.py
index 09c338ef6886..7014b2830a9c 100644
--- a/meta/lib/oeqa/utils/postactions.py
+++ b/meta/lib/oeqa/utils/postactions.py
@@ -9,6 +9,15 @@
 
 from oeqa.utils import get_json_result_dir
 
+def create_artifacts_directory(d, tc):
+    import shutil
+
+    local_artifacts_dir = os.path.join(get_json_result_dir(d), "artifacts")
+    if os.path.isdir(local_artifacts_dir):
+        shutil.rmtree(local_artifacts_dir)
+
+    os.makedirs(local_artifacts_dir)
+
 ##################################################################
 # Artifacts retrieval
 ##################################################################
@@ -29,13 +38,7 @@ def get_artifacts_list(target, raw_list):
     return result
 
 def retrieve_test_artifacts(target, artifacts_list, target_dir):
-    import shutil
-
     local_artifacts_dir = os.path.join(target_dir, "artifacts")
-    if os.path.isdir(local_artifacts_dir):
-        shutil.rmtree(local_artifacts_dir)
-
-    os.makedirs(local_artifacts_dir)
     for artifact_path in artifacts_list:
         if not os.path.isabs(artifact_path):
             bb.warn(f"{artifact_path} is not an absolute path")
@@ -61,6 +64,7 @@ def list_and_fetch_failed_tests_artifacts(d, tc):
 
 def run_failed_tests_post_actions(d, tc):
     post_actions=[
+        create_artifacts_directory,
         list_and_fetch_failed_tests_artifacts
     ]
 
-- 
2.43.1

[OE-Core][PATCH v2 4/5] oeqa/utils/postactions: add target disk usage stat as post action

[Alexis Lothoré]

From: Alexis Lothoré <alexis.lothore@bootlin.com>

In order to debug issues related to disk space (see [1]),  add a failed
tests post action to retrieve disk usage on the target. Rely on the test
context object to run the corresponding command onto the target

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15220

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
---
Changes in v2:
- - use new shared helper get_json_result_dir()
---
 meta/lib/oeqa/utils/postactions.py | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/utils/postactions.py b/meta/lib/oeqa/utils/postactions.py
index 7014b2830a9c..008968b56a3f 100644
--- a/meta/lib/oeqa/utils/postactions.py
+++ b/meta/lib/oeqa/utils/postactions.py
@@ -18,6 +18,20 @@ def create_artifacts_directory(d, tc):
 
     os.makedirs(local_artifacts_dir)
 
+##################################################################
+# Host/target statistics
+##################################################################
+
+def get_target_disk_usage(d, tc):
+    output_file = os.path.join(get_json_result_dir(d), "artifacts", "target_disk_usage.txt")
+    try:
+        (status, output) = tc.target.run('df -hl')
+        with open(output_file, 'w') as f:
+            f.write(output)
+            f.write("\n")
+    except Exception as e:
+        bb.warn(f"Can not get target disk usage: {e}")
+
 ##################################################################
 # Artifacts retrieval
 ##################################################################
@@ -65,7 +79,8 @@ def list_and_fetch_failed_tests_artifacts(d, tc):
 def run_failed_tests_post_actions(d, tc):
     post_actions=[
         create_artifacts_directory,
-        list_and_fetch_failed_tests_artifacts
+        list_and_fetch_failed_tests_artifacts,
+        get_target_disk_usage
     ]
 
     for action in post_actions:
-- 
2.43.1

[OE-Core][PATCH v2 5/5] oeqa/utils/postactions: testimage: add host disk usage stat as post action

[Alexis Lothoré]

From: Alexis Lothoré <alexis.lothore@bootlin.com>

Since the target under test can be a virtualized guest, when some tests
fail because of disk usage (see [1]), also fetch disk usage statistics from
host to allow checking whether a host disk space saturation could affect
running tests.

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15220

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
---
Changes in v2:
- use new shared helper get_json_result_dir()
---
 meta/lib/oeqa/utils/postactions.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/utils/postactions.py b/meta/lib/oeqa/utils/postactions.py
index 008968b56a3f..03cecdc21578 100644
--- a/meta/lib/oeqa/utils/postactions.py
+++ b/meta/lib/oeqa/utils/postactions.py
@@ -32,6 +32,16 @@ def get_target_disk_usage(d, tc):
     except Exception as e:
         bb.warn(f"Can not get target disk usage: {e}")
 
+def get_host_disk_usage(d, tc):
+    import subprocess
+
+    output_file = os.path.join(get_json_result_dir(d), "artifacts", "host_disk_usage.txt")
+    try:
+        with open(output_file, 'w') as f:
+            output = subprocess.run(['/usr/bin/df', '-hl'], check=True, text=True, stdout=f)
+    except Exception as e:
+        bb.warn(f"Can not get host disk usage: {e}")
+
 ##################################################################
 # Artifacts retrieval
 ##################################################################
@@ -80,7 +90,8 @@ def run_failed_tests_post_actions(d, tc):
     post_actions=[
         create_artifacts_directory,
         list_and_fetch_failed_tests_artifacts,
-        get_target_disk_usage
+        get_target_disk_usage,
+        get_host_disk_usage
     ]
 
     for action in post_actions:
-- 
2.43.1

Re: [OE-Core][PATCH v2 1/5] lib/oeqa: share get_json_result_dir helper

[Richard Purdie]

On Fri, 2024-02-23 at 15:03 +0100, Alexis Lothoré via lists.openembedded.org wrote:
> From: Alexis Lothoré <alexis.lothore@bootlin.com>
> 
> Multiple places in oeqa need to get the log output path, and redefine a
> small helper to accomplish this
> 
> Define this helper in lib/oeqa/utils/__init__.py and import it wherever
> needed to allow using it.
> 
> Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
> ---
> There is one additional place re-definining (slightly) differently this
> helper, which is in selftest/context.py. This one does not check
> OEQA_JSON_RESULT_DIR from the datastore but through test data embedded in
> the test context. Trying to check the origin of this test data, I
> eventually see that it comes from bitbake environment, so I am not sure it
> is 100% compatible with the datastore content. Please let me know if I'm
> wrong, if so I can also replace selftest redefinition with the shared
> helper too.

Some of the test code has the ability to run without a bitbake datastore 
so I suspect that may be the abstraction you're running into. That said,
from memory I think it emulates the getVar calls so I think it could be 
converted too. Perhaps a follow up patch we can test?

Cheers,

Richard

Re: [OE-Core][PATCH v2 2/5] testimage: create a list of failed test post actions

[Richard Purdie]

On Fri, 2024-02-23 at 15:03 +0100, Alexis Lothoré via lists.openembedded.org wrote:
> From: Alexis Lothoré <alexis.lothore@bootlin.com>
> 
> testimage is able to detect whenever a test run leads to some tests
> failing, and execute some actions in this case. The only action currently
> defined in such case is to retrieve artifacts from the target under test,
> as listed in TESTIMAGE_FAILED_QA_ARTIFACTS
> 
> In order to be able to add multiple actions, define a central function to
> gather all "post actions" to run whenever a test has failed
> (run_failed_tests_post_actions). This function contains a table listing all
> functions to be called whenever a test fails. Any function in this table
> will be provided with bitbake internal data dictionary ("d") and the
> current runtime testing context ("tc"). Isolate all this feature in a
> dedicated postactions.py file inherited by testimage.
> This patch does not bring any functional change.
> 
> Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
> ---
> Changes in v2:
> - move functions in a lib in oeqa instead of a bbclass
> - use new shared helper get_json_result_dir()
> ---
>  meta/classes-recipe/testimage.bbclass | 42 +----------------
>  meta/lib/oeqa/utils/postactions.py    | 68 +++++++++++++++++++++++++++
>  2 files changed, 70 insertions(+), 40 deletions(-)
>  create mode 100644 meta/lib/oeqa/utils/postactions.py

This patch doesn't apply in testimage.bbclass (hunk #3). Could you
check the branch you're basing these off please?

Thanks,

Richard

Re: [OE-Core][PATCH v2 2/5] testimage: create a list of failed test post actions

[Alexis Lothoré]

On 2/23/24 15:42, Richard Purdie wrote:
> On Fri, 2024-02-23 at 15:03 +0100, Alexis Lothoré via lists.openembedded.org wrote:
>> From: Alexis Lothoré <alexis.lothore@bootlin.com>
>>
>> testimage is able to detect whenever a test run leads to some tests
>> failing, and execute some actions in this case. The only action currently
>> defined in such case is to retrieve artifacts from the target under test,
>> as listed in TESTIMAGE_FAILED_QA_ARTIFACTS
>>
>> In order to be able to add multiple actions, define a central function to
>> gather all "post actions" to run whenever a test has failed
>> (run_failed_tests_post_actions). This function contains a table listing all
>> functions to be called whenever a test fails. Any function in this table
>> will be provided with bitbake internal data dictionary ("d") and the
>> current runtime testing context ("tc"). Isolate all this feature in a
>> dedicated postactions.py file inherited by testimage.
>> This patch does not bring any functional change.
>>
>> Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
>> ---
>> Changes in v2:
>> - move functions in a lib in oeqa instead of a bbclass
>> - use new shared helper get_json_result_dir()
>> ---
>>  meta/classes-recipe/testimage.bbclass | 42 +----------------
>>  meta/lib/oeqa/utils/postactions.py    | 68 +++++++++++++++++++++++++++
>>  2 files changed, 70 insertions(+), 40 deletions(-)
>>  create mode 100644 meta/lib/oeqa/utils/postactions.py
> 
> This patch doesn't apply in testimage.bbclass (hunk #3). Could you
> check the branch you're basing these off please?

My bad, I was _almost_ on master, but with a small commit adding some logs, and
I forgot about it.
v3 incoming.

Thanks,

Alexis

-- 
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com