Openembedded Core Discussions
 help / color / mirror / Atom feed
From: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
To: openembedded-core@lists.openembedded.org
Subject: [PATCH 2/5] oeqa/core/runner: write testresult to json files
Date: Thu, 18 Oct 2018 17:11:06 +0800	[thread overview]
Message-ID: <1539853869-53465-2-git-send-email-ee.peng.yeoh@intel.com> (raw)
In-Reply-To: <1539853869-53465-1-git-send-email-ee.peng.yeoh@intel.com>

As part of the solution to replace Testopia to store testresult,
OEQA need to output testresult into json file, where these json
testresult file will be stored in git repository by the future
test-case-management tools.

Both the testresult (eg. PASSED, FAILED, ERROR) and  the test log
(eg. message from unit test assertion) will be created for storing.

Also the library class inside this patch will be reused by the future
test-case-management tools to write json testresult for manual test
case executed.

Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
---
 meta/lib/oeqa/core/runner.py | 30 +++++++++++++++++++++++++++++-
 1 file changed, 29 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py
index f1dd080..e82acdf 100644
--- a/meta/lib/oeqa/core/runner.py
+++ b/meta/lib/oeqa/core/runner.py
@@ -6,6 +6,7 @@ import time
 import unittest
 import logging
 import re
+import json
 
 from unittest import TextTestResult as _TestResult
 from unittest import TextTestRunner as _TestRunner
@@ -119,8 +120,9 @@ class OETestResult(_TestResult):
         self.successes.append((test, None))
         super(OETestResult, self).addSuccess(test)
 
-    def logDetails(self):
+    def logDetails(self, json_file_dir=None, test_environments=None):
         self.tc.logger.info("RESULTS:")
+        results = {}
         for case_name in self.tc._registry['cases']:
             case = self.tc._registry['cases'][case_name]
 
@@ -137,6 +139,11 @@ class OETestResult(_TestResult):
                 t = " (" + "{0:.2f}".format(self.endtime[case.id()] - self.starttime[case.id()]) + "s)"
 
             self.tc.logger.info("RESULTS - %s - Testcase %s: %s%s" % (case.id(), oeid, status, t))
+            results[case.id()] = (status, log)
+
+        if json_file_dir:
+            tresultjsonhelper = OETestResultJSONHelper()
+            tresultjsonhelper.dump_testresult_file(results, test_environments, json_file_dir)
 
 class OEListTestsResult(object):
     def wasSuccessful(self):
@@ -249,3 +256,24 @@ class OETestRunner(_TestRunner):
             self._list_tests_module(suite)
 
         return OEListTestsResult()
+
+class OETestResultJSONHelper(object):
+
+    def _create_json_testresult_string(self, test_results, test_environments):
+        testcase_dict = {}
+        for testcase in sorted(test_results):
+            testcase_dict[testcase] = {"testresult": test_results[testcase][0], "log": test_results[testcase][1]}
+        testresult_object = {'testenvironment': test_environments,
+                             'testcase': testcase_dict}
+        return json.dumps(testresult_object, sort_keys=True, indent=4)
+
+    def _write_file(self, write_dir, file_name, file_content):
+        file_path = os.path.join(write_dir, file_name)
+        with open(file_path, 'w') as the_file:
+            the_file.write(file_content)
+
+    def dump_testresult_file(self, test_results, test_environments, write_dir):
+        if not os.path.exists(write_dir):
+            bb.utils.mkdirhier(write_dir)
+        json_testresult = self._create_json_testresult_string(test_results, test_environments)
+        self._write_file(write_dir, 'testresults.json', json_testresult)
-- 
2.7.4



  reply	other threads:[~2018-10-18  9:26 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-18  9:11 [PATCH 1/5] oeqa/core/runner: refactor for OEQA to write json testresult Yeoh Ee Peng
2018-10-18  9:11 ` Yeoh Ee Peng [this message]
2018-10-18  9:11 ` [PATCH 3/5] oeqa/selftest/context: write testresult to json files Yeoh Ee Peng
2018-10-18  9:11 ` [PATCH 4/5] testimage.bbclass: " Yeoh Ee Peng
2018-10-18  9:11 ` [PATCH 5/5] testsdk.bbclass: " Yeoh Ee Peng
  -- strict thread matches above, loose matches on Subject: below --
2018-10-15  7:24 [PATCH 1/5] oeqa/core/runner: refactor for OEQA to write json testresult Yeoh Ee Peng
2018-10-15  7:24 ` [PATCH 2/5] oeqa/core/runner: write testresult to json files Yeoh Ee Peng
2018-10-15  8:53   ` Richard Purdie
2018-10-15 10:20     ` Yeoh, Ee Peng
2018-10-18  9:47     ` Yeoh, Ee Peng
2018-10-18 21:56       ` richard.purdie
2018-10-20  1:50         ` Yeoh, Ee Peng
2018-10-22  7:23         ` Yeoh, Ee Peng
2018-10-22  8:17         ` Yeoh, Ee Peng
2018-10-22  8:52           ` richard.purdie

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1539853869-53465-2-git-send-email-ee.peng.yeoh@intel.com \
    --to=ee.peng.yeoh@intel.com \
    --cc=openembedded-core@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox