From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mail.openembedded.org (Postfix) with ESMTP id A095579BE1 for ; Mon, 22 Oct 2018 07:09:35 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 Oct 2018 00:09:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,411,1534834800"; d="scan'208";a="243249369" Received: from andromeda02.png.intel.com ([10.221.183.11]) by orsmga004.jf.intel.com with ESMTP; 22 Oct 2018 00:09:35 -0700 From: Yeoh Ee Peng To: openembedded-core@lists.openembedded.org Date: Mon, 22 Oct 2018 14:54:21 +0800 Message-Id: <1540191264-12332-1-git-send-email-ee.peng.yeoh@intel.com> X-Mailer: git-send-email 2.7.4 Subject: [PATCH 1/4] oeqa/core/runner: write testresult to json files X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2018 07:09:35 -0000 As part of the solution to replace Testopia to store testresult, OEQA need to output testresult into single json file, where json testresult file will be stored in git repository by the future test-case-management tools. The json testresult file will store more than one set of results, where each set of results was uniquely identified by the result_id. The result_id would be like "runtime-qemux86-core-image-sato", where it was a runtime test with target machine equal to qemux86 and running on core-image-sato image. The json testresult file will only store the latest testresult for a given result_id. The json testresult file contains the configuration (eg. COMMIT, BRANCH, MACHINE, IMAGE), result (eg. PASSED, FAILED, ERROR), test log, and result_id. Based on the destination json testresult file directory provided, it could have multiple instances of bitbake trying to write json testresult to a single testresult file, using locking a lockfile alongside the results file directory to prevent races. Also the library class inside this patch will be reused by the future test-case-management tools to write json testresult for manual test case executed. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/core/runner.py | 40 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 39 insertions(+), 1 deletion(-) diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py index f1dd080..82463cf 100644 --- a/meta/lib/oeqa/core/runner.py +++ b/meta/lib/oeqa/core/runner.py @@ -6,6 +6,7 @@ import time import unittest import logging import re +import json from unittest import TextTestResult as _TestResult from unittest import TextTestRunner as _TestRunner @@ -119,8 +120,9 @@ class OETestResult(_TestResult): self.successes.append((test, None)) super(OETestResult, self).addSuccess(test) - def logDetails(self): + def logDetails(self, json_file_dir=None, configuration=None, result_id=None): self.tc.logger.info("RESULTS:") + result = {} for case_name in self.tc._registry['cases']: case = self.tc._registry['cases'][case_name] @@ -137,6 +139,11 @@ class OETestResult(_TestResult): t = " (" + "{0:.2f}".format(self.endtime[case.id()] - self.starttime[case.id()]) + "s)" self.tc.logger.info("RESULTS - %s - Testcase %s: %s%s" % (case.id(), oeid, status, t)) + result[case.id()] = {'status': status, 'log': log} + + if json_file_dir: + tresultjsonhelper = OETestResultJSONHelper() + tresultjsonhelper.dump_testresult_file(result_id, result, configuration, json_file_dir) class OEListTestsResult(object): def wasSuccessful(self): @@ -249,3 +256,34 @@ class OETestRunner(_TestRunner): self._list_tests_module(suite) return OEListTestsResult() + +class OETestResultJSONHelper(object): + + testresult_filename = 'testresults.json' + + def _get_testresults(self, write_dir): + testresults = {} + file = os.path.join(write_dir, self.testresult_filename) + if os.path.exists(file): + with open(file, "r") as f: + testresults = json.load(f) + return testresults + + def _create_json_testresults_string(self, result_id, test_result, configuration, write_dir): + testresults = self._get_testresults(write_dir) + testresult = {'configuration': configuration, + 'result': test_result} + testresults[result_id] = testresult + return json.dumps(testresults, sort_keys=True, indent=4) + + def _write_file(self, write_dir, file_name, file_content): + file_path = os.path.join(write_dir, file_name) + with open(file_path, 'w') as the_file: + the_file.write(file_content) + + def dump_testresult_file(self, result_id, test_result, configuration, write_dir): + bb.utils.mkdirhier(write_dir) + lf = bb.utils.lockfile(os.path.join(write_dir, 'jsontestresult.lock')) + json_testresults = self._create_json_testresults_string(result_id, test_result, configuration, write_dir) + self._write_file(write_dir, self.testresult_filename, json_testresults) + bb.utils.unlockfile(lf) -- 2.7.4