From: alexis.lothore@bootlin.com
To: openembedded-core@lists.openembedded.org
Cc: alexandre.belloni@bootlin.com, thomas.petazzoni@bootlin.com
Subject: [PATCH v3 1/6] scripts/oe-selftest: append metadata to tests results
Date: Fri, 24 Feb 2023 17:45:50 +0100 [thread overview]
Message-ID: <20230224164555.67634-2-alexis.lothore@bootlin.com> (raw)
In-Reply-To: <20230224164555.67634-1-alexis.lothore@bootlin.com>
From: Alexis Lothoré <alexis.lothore@bootlin.com>
Many stored results TEST_TYPE are set to "oeselftest", however all those
tests are not run with the same sets of parameters, so those tests results may
not be comparable.
Attach relevant parameters as tests metadata to allow identifying tests
configuration so we can compare tests only when they are run with the same
parameters.
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
---
meta/lib/oeqa/selftest/context.py | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oeqa/selftest/context.py b/meta/lib/oeqa/selftest/context.py
index c7dd03ce378..8cc46283ed0 100644
--- a/meta/lib/oeqa/selftest/context.py
+++ b/meta/lib/oeqa/selftest/context.py
@@ -22,6 +22,17 @@ from oeqa.core.exception import OEQAPreRun, OEQATestNotFound
from oeqa.utils.commands import runCmd, get_bb_vars, get_test_layer
+OESELFTEST_METADATA=["run_all_tests", "run_tests", "skips", "machine", "select_tags", "exclude_tags"]
+
+def get_oeselftest_metadata(args):
+ result = {}
+ raw_args = vars(args)
+ for metadata in OESELFTEST_METADATA:
+ if metadata in raw_args:
+ result[metadata] = raw_args[metadata]
+
+ return result
+
class NonConcurrentTestSuite(unittest.TestSuite):
def __init__(self, suite, processes, setupfunc, removefunc):
super().__init__([suite])
@@ -334,12 +345,14 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
import platform
from oeqa.utils.metadata import metadata_from_bb
metadata = metadata_from_bb()
+ oeselftest_metadata = get_oeselftest_metadata(args)
configuration = {'TEST_TYPE': 'oeselftest',
'STARTTIME': args.test_start_time,
'MACHINE': self.tc.td["MACHINE"],
'HOST_DISTRO': oe.lsb.distro_identifier().replace(' ', '-'),
'HOST_NAME': metadata['hostname'],
- 'LAYERS': metadata['layers']}
+ 'LAYERS': metadata['layers'],
+ 'OESELFTEST_METADATA':oeselftest_metadata}
return configuration
def get_result_id(self, configuration):
--
2.39.1
next prev parent reply other threads:[~2023-02-24 16:46 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-24 16:45 [PATCH v3 0/6] scripts/resulttool/regression: add metadata filtering alexis.lothore
2023-02-24 16:45 ` alexis.lothore [this message]
2023-02-24 16:45 ` [PATCH v3 2/6] scripts/resulttool/regression: remove unused import alexis.lothore
2023-02-24 16:45 ` [PATCH v3 3/6] scripts/resulttool/regression: add metadata filtering for oeselftest alexis.lothore
2023-02-24 16:45 ` [PATCH v3 4/6] oeqa/selftest/resulttool: add test for metadata filtering on regression alexis.lothore
2023-02-26 0:03 ` [OE-core] " Richard Purdie
2023-02-24 16:45 ` [PATCH v3 5/6] scripts: add new helper for regression report generation alexis.lothore
2023-02-24 16:45 ` [PATCH v3 6/6] oeqa/selftest: add test for yocto_testresults_query.py alexis.lothore
2023-02-24 18:06 ` [OE-core] [PATCH v3 0/6] scripts/resulttool/regression: add metadata filtering Richard Purdie
[not found] ` <1746D4E8592324E9.29542@lists.openembedded.org>
2023-02-25 9:15 ` Richard Purdie
[not found] ` <1747067DAE80068A.29542@lists.openembedded.org>
2023-02-25 12:32 ` Richard Purdie
2023-02-25 15:59 ` Alexis Lothoré
2023-02-26 12:15 ` Richard Purdie
2023-02-26 15:42 ` Alexis Lothoré
2023-02-27 13:41 ` Richard Purdie
[not found] ` <1747113C8A4DBAD6.29542@lists.openembedded.org>
2023-02-25 12:44 ` Richard Purdie
2023-02-27 13:14 ` Ross Burton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230224164555.67634-2-alexis.lothore@bootlin.com \
--to=alexis.lothore@bootlin.com \
--cc=alexandre.belloni@bootlin.com \
--cc=openembedded-core@lists.openembedded.org \
--cc=thomas.petazzoni@bootlin.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.