* [PATCH 01/17] testsdk.bbclass: Remove unused import of OEStreamLogger
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 02/17] oeqa/sdk/context.py: Import argparse_oe at OESDKTestContext.run method Aníbal Limón
` (15 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/classes/testsdk.bbclass | 1 -
1 file changed, 1 deletion(-)
diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
index 6a201aa..4740233 100644
--- a/meta/classes/testsdk.bbclass
+++ b/meta/classes/testsdk.bbclass
@@ -21,7 +21,6 @@ def testsdk_main(d):
import logging
from bb.utils import export_proxies
- from oeqa.core.runner import OEStreamLogger
from oeqa.sdk.context import OESDKTestContext, OESDKTestContextExecutor
from oeqa.utils import make_logger_bitbake_compatible
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 02/17] oeqa/sdk/context.py: Import argparse_oe at OESDKTestContext.run method
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
2017-05-10 19:47 ` [PATCH 01/17] testsdk.bbclass: Remove unused import of OEStreamLogger Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 03/17] oeqa/sdk/context.py: Add return to OESDKTestContext.run() method Aníbal Limón
` (14 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
This import was at level of OESDKTestContext.register_commands
but OESDKTestContext.run method need it to raise exceptions.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/sdk/context.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/lib/oeqa/sdk/context.py b/meta/lib/oeqa/sdk/context.py
index 0189ed8..509d2b4 100644
--- a/meta/lib/oeqa/sdk/context.py
+++ b/meta/lib/oeqa/sdk/context.py
@@ -44,8 +44,6 @@ class OESDKTestContextExecutor(OETestContextExecutor):
default_test_data = None
def register_commands(self, logger, subparsers):
- import argparse_oe
-
super(OESDKTestContextExecutor, self).register_commands(logger, subparsers)
sdk_group = self.parser.add_argument_group('sdk options')
@@ -109,6 +107,8 @@ class OESDKTestContextExecutor(OETestContextExecutor):
log(env)
def run(self, logger, args):
+ import argparse_oe
+
if not args.sdk_dir:
raise argparse_oe.ArgumentUsageError("No SDK directory "\
"specified please do, --sdk-dir SDK_DIR", self.name)
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 03/17] oeqa/sdk/context.py: Add return to OESDKTestContext.run() method
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
2017-05-10 19:47 ` [PATCH 01/17] testsdk.bbclass: Remove unused import of OEStreamLogger Aníbal Limón
2017-05-10 19:47 ` [PATCH 02/17] oeqa/sdk/context.py: Import argparse_oe at OESDKTestContext.run method Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 04/17] oeqa/core: Don't expose OEStreamLogger in OETestContext Aníbal Limón
` (13 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
The run() methods of a OETestContext's are expected to return the
results.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/sdk/context.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/lib/oeqa/sdk/context.py b/meta/lib/oeqa/sdk/context.py
index 509d2b4..82e4c19 100644
--- a/meta/lib/oeqa/sdk/context.py
+++ b/meta/lib/oeqa/sdk/context.py
@@ -128,6 +128,6 @@ class OESDKTestContextExecutor(OETestContextExecutor):
"environment (%s) specified" % args.sdk_env, self.name)
self.sdk_env = sdk_envs[args.sdk_env]
- super(OESDKTestContextExecutor, self).run(logger, args)
+ return super(OESDKTestContextExecutor, self).run(logger, args)
_executor_class = OESDKTestContextExecutor
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 04/17] oeqa/core: Don't expose OEStreamLogger in OETestContext
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (2 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 03/17] oeqa/sdk/context.py: Add return to OESDKTestContext.run() method Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 05/17] oeqa/core/runner: OETestResult remove unneeded override of startTest Aníbal Limón
` (12 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
The OEStreamLogger class is used for redirect PyUnit output
to a certain logger so there is not need to expose at level
of OETestContext because only OETestRunner needs to know.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/context.py | 6 ++----
meta/lib/oeqa/core/runner.py | 3 +++
2 files changed, 5 insertions(+), 4 deletions(-)
diff --git a/meta/lib/oeqa/core/context.py b/meta/lib/oeqa/core/context.py
index 4476750..2628651 100644
--- a/meta/lib/oeqa/core/context.py
+++ b/meta/lib/oeqa/core/context.py
@@ -10,12 +10,11 @@ import collections
import re
from oeqa.core.loader import OETestLoader
-from oeqa.core.runner import OETestRunner, OEStreamLogger, xmlEnabled
+from oeqa.core.runner import OETestRunner, xmlEnabled
class OETestContext(object):
loaderClass = OETestLoader
runnerClass = OETestRunner
- streamLoggerClass = OEStreamLogger
files_dir = os.path.abspath(os.path.join(os.path.dirname(
os.path.abspath(__file__)), "../files"))
@@ -52,8 +51,7 @@ class OETestContext(object):
self.suites = self.loader.discover()
def runTests(self):
- streamLogger = self.streamLoggerClass(self.logger)
- self.runner = self.runnerClass(self, stream=streamLogger, verbosity=2)
+ self.runner = self.runnerClass(self, verbosity=2)
self._run_start_time = time.time()
result = self.runner.run(self.suites)
diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py
index 44ffecb..40fbf3b 100644
--- a/meta/lib/oeqa/core/runner.py
+++ b/meta/lib/oeqa/core/runner.py
@@ -48,12 +48,15 @@ class OETestResult(_TestResult):
super(OETestResult, self).startTest(test)
class OETestRunner(_TestRunner):
+ streamLoggerClass = OEStreamLogger
+
def __init__(self, tc, *args, **kwargs):
if xmlEnabled:
if not kwargs.get('output'):
kwargs['output'] = os.path.join(os.getcwd(),
'TestResults_%s_%s' % (time.strftime("%Y%m%d%H%M%S"), os.getpid()))
+ kwargs['stream'] = self.streamLoggerClass(tc.logger)
super(OETestRunner, self).__init__(*args, **kwargs)
self.tc = tc
self.resultclass = OETestResult
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 05/17] oeqa/core/runner: OETestResult remove unneeded override of startTest
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (3 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 04/17] oeqa/core: Don't expose OEStreamLogger in OETestContext Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 06/17] oeqa/core: Move OETestContext.log{Summary, Details} into OETestResult Aníbal Limón
` (11 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
I override this method before for keep track of results and forget
to remove it, now isn't need.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/runner.py | 3 ---
1 file changed, 3 deletions(-)
diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py
index 40fbf3b..0b04e8d 100644
--- a/meta/lib/oeqa/core/runner.py
+++ b/meta/lib/oeqa/core/runner.py
@@ -44,9 +44,6 @@ class OETestResult(_TestResult):
self.tc._results['skipped'] = self.skipped
self.tc._results['expectedFailures'] = self.expectedFailures
- def startTest(self, test):
- super(OETestResult, self).startTest(test)
-
class OETestRunner(_TestRunner):
streamLoggerClass = OEStreamLogger
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 06/17] oeqa/core: Move OETestContext.log{Summary, Details} into OETestResult
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (4 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 05/17] oeqa/core/runner: OETestResult remove unneeded override of startTest Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 07/17] oeqa/core/threaded: Add new module with OETestLoaderThreaded Aníbal Limón
` (10 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
Those methods are used to write in the log the results so
it makes sense to have defined at OETestResult because
is a format of the result itself.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/classes/testimage.bbclass | 4 +-
meta/classes/testsdk.bbclass | 8 ++--
meta/lib/oeqa/core/context.py | 93 ++----------------------------------------
meta/lib/oeqa/core/runner.py | 88 +++++++++++++++++++++++++++++++++++++++
4 files changed, 97 insertions(+), 96 deletions(-)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index fb21460..d42907c 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -292,8 +292,8 @@ def testimage_main(d):
# Show results (if we have them)
if not results:
bb.fatal('%s - FAILED - tests were interrupted during execution' % pn)
- tc.logSummary(results, pn)
- tc.logDetails()
+ results.logSummary(pn)
+ results.logDetails()
if not results.wasSuccessful():
bb.fatal('%s - FAILED - check the task log and the ssh log' % pn)
diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
index 4740233..edea89c 100644
--- a/meta/classes/testsdk.bbclass
+++ b/meta/classes/testsdk.bbclass
@@ -70,8 +70,8 @@ def testsdk_main(d):
component = "%s %s" % (pn, OESDKTestContextExecutor.name)
context_msg = "%s:%s" % (os.path.basename(tcname), os.path.basename(sdk_env))
- tc.logSummary(result, component, context_msg)
- tc.logDetails()
+ result.logSummary(component, context_msg)
+ result.logDetails()
if not result.wasSuccessful():
fail = True
@@ -172,8 +172,8 @@ def testsdkext_main(d):
component = "%s %s" % (pn, OESDKExtTestContextExecutor.name)
context_msg = "%s:%s" % (os.path.basename(tcname), os.path.basename(sdk_env))
- tc.logSummary(result, component, context_msg)
- tc.logDetails()
+ result.logSummary(component, context_msg)
+ result.logDetails()
if not result.wasSuccessful():
fail = True
diff --git a/meta/lib/oeqa/core/context.py b/meta/lib/oeqa/core/context.py
index 2628651..54958ad 100644
--- a/meta/lib/oeqa/core/context.py
+++ b/meta/lib/oeqa/core/context.py
@@ -7,10 +7,9 @@ import json
import time
import logging
import collections
-import re
from oeqa.core.loader import OETestLoader
-from oeqa.core.runner import OETestRunner, xmlEnabled
+from oeqa.core.runner import OETestRunner
class OETestContext(object):
loaderClass = OETestLoader
@@ -59,92 +58,6 @@ class OETestContext(object):
return result
- def logSummary(self, result, component, context_msg=''):
- self.logger.info("SUMMARY:")
- self.logger.info("%s (%s) - Ran %d test%s in %.3fs" % (component,
- context_msg, result.testsRun, result.testsRun != 1 and "s" or "",
- (self._run_end_time - self._run_start_time)))
-
- if result.wasSuccessful():
- msg = "%s - OK - All required tests passed" % component
- else:
- msg = "%s - FAIL - Required tests failed" % component
- skipped = len(self._results['skipped'])
- if skipped:
- msg += " (skipped=%d)" % skipped
- self.logger.info(msg)
-
- def _getDetailsNotPassed(self, case, type, desc):
- found = False
-
- for (scase, msg) in self._results[type]:
- # XXX: When XML reporting is enabled scase is
- # xmlrunner.result._TestInfo instance instead of
- # string.
- if xmlEnabled:
- if case.id() == scase.test_id:
- found = True
- break
- scase_str = scase.test_id
- else:
- if case == scase:
- found = True
- break
- scase_str = str(scase)
-
- # When fails at module or class level the class name is passed as string
- # so figure out to see if match
- m = re.search("^setUpModule \((?P<module_name>.*)\)$", scase_str)
- if m:
- if case.__class__.__module__ == m.group('module_name'):
- found = True
- break
-
- m = re.search("^setUpClass \((?P<class_name>.*)\)$", scase_str)
- if m:
- class_name = "%s.%s" % (case.__class__.__module__,
- case.__class__.__name__)
-
- if class_name == m.group('class_name'):
- found = True
- break
-
- if found:
- return (found, msg)
-
- return (found, None)
-
- def logDetails(self):
- self.logger.info("RESULTS:")
- for case_name in self._registry['cases']:
- case = self._registry['cases'][case_name]
-
- result_types = ['failures', 'errors', 'skipped', 'expectedFailures']
- result_desc = ['FAILED', 'ERROR', 'SKIPPED', 'EXPECTEDFAIL']
-
- fail = False
- desc = None
- for idx, name in enumerate(result_types):
- (fail, msg) = self._getDetailsNotPassed(case, result_types[idx],
- result_desc[idx])
- if fail:
- desc = result_desc[idx]
- break
-
- oeid = -1
- for d in case.decorators:
- if hasattr(d, 'oeid'):
- oeid = d.oeid
-
- if fail:
- self.logger.info("RESULTS - %s - Testcase %s: %s" % (case.id(),
- oeid, desc))
- if msg:
- self.logger.info(msg)
- else:
- self.logger.info("RESULTS - %s - Testcase %s: %s" % (case.id(),
- oeid, 'PASSED'))
-
class OETestContextExecutor(object):
_context_class = OETestContext
@@ -227,8 +140,8 @@ class OETestContextExecutor(object):
self.tc = self._context_class(**self.tc_kwargs['init'])
self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
rc = self.tc.runTests(**self.tc_kwargs['run'])
- self.tc.logSummary(rc, self.name)
- self.tc.logDetails()
+ rc.logSummary(self.name)
+ rc.logDetails()
output_link = os.path.join(os.path.dirname(args.output_log),
"%s-results.log" % self.name)
diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py
index 0b04e8d..7505838 100644
--- a/meta/lib/oeqa/core/runner.py
+++ b/meta/lib/oeqa/core/runner.py
@@ -5,6 +5,7 @@ import os
import time
import unittest
import logging
+import re
xmlEnabled = False
try:
@@ -44,6 +45,93 @@ class OETestResult(_TestResult):
self.tc._results['skipped'] = self.skipped
self.tc._results['expectedFailures'] = self.expectedFailures
+ def logSummary(self, component, context_msg=''):
+ elapsed_time = self.tc._run_end_time - self.tc._run_start_time
+ self.tc.logger.info("SUMMARY:")
+ self.tc.logger.info("%s (%s) - Ran %d test%s in %.3fs" % (component,
+ context_msg, self.testsRun, self.testsRun != 1 and "s" or "",
+ elapsed_time))
+
+ if self.wasSuccessful():
+ msg = "%s - OK - All required tests passed" % component
+ else:
+ msg = "%s - FAIL - Required tests failed" % component
+ skipped = len(self.tc._results['skipped'])
+ if skipped:
+ msg += " (skipped=%d)" % skipped
+ self.tc.logger.info(msg)
+
+ def _getDetailsNotPassed(self, case, type, desc):
+ found = False
+
+ for (scase, msg) in self.tc._results[type]:
+ # XXX: When XML reporting is enabled scase is
+ # xmlrunner.result._TestInfo instance instead of
+ # string.
+ if xmlEnabled:
+ if case.id() == scase.test_id:
+ found = True
+ break
+ scase_str = scase.test_id
+ else:
+ if case == scase:
+ found = True
+ break
+ scase_str = str(scase)
+
+ # When fails at module or class level the class name is passed as string
+ # so figure out to see if match
+ m = re.search("^setUpModule \((?P<module_name>.*)\)$", scase_str)
+ if m:
+ if case.__class__.__module__ == m.group('module_name'):
+ found = True
+ break
+
+ m = re.search("^setUpClass \((?P<class_name>.*)\)$", scase_str)
+ if m:
+ class_name = "%s.%s" % (case.__class__.__module__,
+ case.__class__.__name__)
+
+ if class_name == m.group('class_name'):
+ found = True
+ break
+
+ if found:
+ return (found, msg)
+
+ return (found, None)
+
+ def logDetails(self):
+ self.tc.logger.info("RESULTS:")
+ for case_name in self.tc._registry['cases']:
+ case = self.tc._registry['cases'][case_name]
+
+ result_types = ['failures', 'errors', 'skipped', 'expectedFailures']
+ result_desc = ['FAILED', 'ERROR', 'SKIPPED', 'EXPECTEDFAIL']
+
+ fail = False
+ desc = None
+ for idx, name in enumerate(result_types):
+ (fail, msg) = self._getDetailsNotPassed(case, result_types[idx],
+ result_desc[idx])
+ if fail:
+ desc = result_desc[idx]
+ break
+
+ oeid = -1
+ for d in case.decorators:
+ if hasattr(d, 'oeid'):
+ oeid = d.oeid
+
+ if fail:
+ self.tc.logger.info("RESULTS - %s - Testcase %s: %s" % (case.id(),
+ oeid, desc))
+ if msg:
+ self.tc.logger.info(msg)
+ else:
+ self.tc.logger.info("RESULTS - %s - Testcase %s: %s" % (case.id(),
+ oeid, 'PASSED'))
+
class OETestRunner(_TestRunner):
streamLoggerClass = OEStreamLogger
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 07/17] oeqa/core/threaded: Add new module with OETestLoaderThreaded
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (5 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 06/17] oeqa/core: Move OETestContext.log{Summary, Details} into OETestResult Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 08/17] oeqa/core/threaded: Add OEStreamLoggerThreaded class Aníbal Limón
` (9 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
The threded module will implement Thread support into the OEQA
framework.
The OETestLoaderThreaded overrides discover to return a list of
suites every suite will be run by a separate Thread.
Some design considerations are:
- All the tests of a certain module needs to be run at one thread
because unittest framework needs to keep track of setUp{Module,
Class,} and tearDown{Module, Class,}.
- Tests that depends on other needs to be run at the same thread
because OEQA framework look at the status of dependant test to
decide if skip or not, this constraint can be change in the future
but a sync mechanishm is needed to implement between the Threads.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/threaded.py | 91 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 91 insertions(+)
create mode 100644 meta/lib/oeqa/core/threaded.py
diff --git a/meta/lib/oeqa/core/threaded.py b/meta/lib/oeqa/core/threaded.py
new file mode 100644
index 0000000..e6f2140
--- /dev/null
+++ b/meta/lib/oeqa/core/threaded.py
@@ -0,0 +1,91 @@
+# Copyright (C) 2017 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import multiprocessing
+
+from unittest.suite import TestSuite
+from oeqa.core.loader import OETestLoader
+
+class OETestLoaderThreaded(OETestLoader):
+ def __init__(self, tc, module_paths, modules, tests, modules_required,
+ filters, process_num=0, *args, **kwargs):
+ super(OETestLoaderThreaded, self).__init__(tc, module_paths, modules,
+ tests, modules_required, filters, *args, **kwargs)
+
+ self.process_num = process_num
+
+ def discover(self):
+ suite = super(OETestLoaderThreaded, self).discover()
+
+ if self.process_num <= 0:
+ self.process_num = min(multiprocessing.cpu_count(),
+ len(suite._tests))
+
+ suites = []
+ for _ in range(self.process_num):
+ suites.append(self.suiteClass())
+
+ def _search_for_module_idx(suites, case):
+ """
+ Cases in the same module needs to be run
+ in the same thread because PyUnit keeps track
+ of setUp{Module, Class,} and tearDown{Module, Class,}.
+ """
+
+ for idx in range(self.process_num):
+ suite = suites[idx]
+ for c in suite._tests:
+ if case.__module__ == c.__module__:
+ return idx
+
+ return -1
+
+ def _search_for_depend_idx(suites, depends):
+ """
+ Dependency cases needs to be run in the same
+ thread, because OEQA framework look at the state
+ of dependant test to figure out if skip or not.
+ """
+
+ for idx in range(self.process_num):
+ suite = suites[idx]
+
+ for case in suite._tests:
+ if case.id() in depends:
+ return idx
+ return -1
+
+ def _get_best_idx(suites):
+ sizes = [len(suite._tests) for suite in suites]
+ return sizes.index(min(sizes))
+
+ def _fill_suites(suite):
+ idx = -1
+ for case in suite:
+ if isinstance(case, TestSuite):
+ _fill_suites(case)
+ else:
+ idx = _search_for_module_idx(suites, case)
+
+ depends = {}
+ if 'depends' in self.tc._registry:
+ depends = self.tc._registry['depends']
+
+ if idx == -1 and case.id() in depends:
+ case_depends = depends[case.id()]
+ idx = _search_for_depend_idx(suites, case_depends)
+
+ if idx == -1:
+ idx = _get_best_idx(suites)
+
+ suites[idx].addTest(case)
+ _fill_suites(suite)
+
+ suites_tmp = suites
+ suites = []
+ for suite in suites_tmp:
+ if len(suite._tests) > 0:
+ suites.append(suite)
+
+ return suites
+
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 08/17] oeqa/core/threaded: Add OEStreamLoggerThreaded class
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (6 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 07/17] oeqa/core/threaded: Add new module with OETestLoaderThreaded Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 09/17] oeqa/core/runner: OETestResult add internal _tc_map_results Aníbal Limón
` (8 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
The OEStreamLoggerThreaded overrides OEStreamLogger to redirect
the PyUnit output to a logger.
Instead of log every line when comes the OEStreamLoggerThreaded
will buffer the PyUnit output and write everything at end of every
suite execution to don't have mixed suite outputs.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/threaded.py | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/meta/lib/oeqa/core/threaded.py b/meta/lib/oeqa/core/threaded.py
index e6f2140..73b7f2f 100644
--- a/meta/lib/oeqa/core/threaded.py
+++ b/meta/lib/oeqa/core/threaded.py
@@ -1,10 +1,13 @@
# Copyright (C) 2017 Intel Corporation
# Released under the MIT license (see COPYING.MIT)
+import threading
import multiprocessing
from unittest.suite import TestSuite
+
from oeqa.core.loader import OETestLoader
+from oeqa.core.runner import OEStreamLogger
class OETestLoaderThreaded(OETestLoader):
def __init__(self, tc, module_paths, modules, tests, modules_required,
@@ -89,3 +92,25 @@ class OETestLoaderThreaded(OETestLoader):
return suites
+class OEStreamLoggerThreaded(OEStreamLogger):
+ _lock = threading.Lock()
+ buffers = {}
+
+ def write(self, msg):
+ tid = threading.get_ident()
+
+ if not tid in self.buffers:
+ self.buffers[tid] = ""
+
+ if msg:
+ self.buffers[tid] += msg
+
+ def finish(self):
+ tid = threading.get_ident()
+
+ self._lock.acquire()
+ self.logger.info('THREAD: %d' % tid)
+ self.logger.info('-' * 70)
+ for line in self.buffers[tid].split('\n'):
+ self.logger.info(line)
+ self._lock.release()
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 09/17] oeqa/core/runner: OETestResult add internal _tc_map_results
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (7 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 08/17] oeqa/core/threaded: Add OEStreamLoggerThreaded class Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 10/17] oeqa/core/threaded: Add OETestResultThreaded{, Internal} classes Aníbal Limón
` (7 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
This method is to assign results into the TestContext, create
an internal one to support change implementation in Thread version.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/runner.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py
index 7505838..3ebffc7 100644
--- a/meta/lib/oeqa/core/runner.py
+++ b/meta/lib/oeqa/core/runner.py
@@ -39,7 +39,9 @@ class OETestResult(_TestResult):
super(OETestResult, self).__init__(*args, **kwargs)
self.tc = tc
+ self._tc_map_results()
+ def _tc_map_results(self):
self.tc._results['failures'] = self.failures
self.tc._results['errors'] = self.errors
self.tc._results['skipped'] = self.skipped
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 10/17] oeqa/core/threaded: Add OETestResultThreaded{, Internal} classes
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (8 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 09/17] oeqa/core/runner: OETestResult add internal _tc_map_results Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 11/17] oeqa/core/threaded: Add support of OETestRunnerThreaded Aníbal Limón
` (6 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
The OETestResultThreadedInternal extends OETestResult to stores
results by Thread.
The OETestResultThreaded is a simple class that provides the
implementation of interfaces needed by outside like wasSuccesful,
stop, logSummary, logDetails.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/threaded.py | 73 +++++++++++++++++++++++++++++++++++++++++-
1 file changed, 72 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oeqa/core/threaded.py b/meta/lib/oeqa/core/threaded.py
index 73b7f2f..f216685 100644
--- a/meta/lib/oeqa/core/threaded.py
+++ b/meta/lib/oeqa/core/threaded.py
@@ -7,7 +7,7 @@ import multiprocessing
from unittest.suite import TestSuite
from oeqa.core.loader import OETestLoader
-from oeqa.core.runner import OEStreamLogger
+from oeqa.core.runner import OEStreamLogger, OETestResult
class OETestLoaderThreaded(OETestLoader):
def __init__(self, tc, module_paths, modules, tests, modules_required,
@@ -114,3 +114,74 @@ class OEStreamLoggerThreaded(OEStreamLogger):
for line in self.buffers[tid].split('\n'):
self.logger.info(line)
self._lock.release()
+
+class OETestResultThreadedInternal(OETestResult):
+ def _tc_map_results(self):
+ tid = threading.get_ident()
+
+ # PyUnit generates a result for every test module run, test
+ # if the thread already has an entry to avoid lose the previous
+ # test module results.
+ if not tid in self.tc._results:
+ self.tc._results[tid] = {}
+ self.tc._results[tid]['failures'] = self.failures
+ self.tc._results[tid]['errors'] = self.errors
+ self.tc._results[tid]['skipped'] = self.skipped
+ self.tc._results[tid]['expectedFailures'] = self.expectedFailures
+
+class OETestResultThreaded(object):
+ _results = {}
+ _lock = threading.Lock()
+
+ def __init__(self, tc):
+ self.tc = tc
+
+ def _fill_tc_results(self):
+ tids = list(self.tc._results.keys())
+ fields = ['failures', 'errors', 'skipped', 'expectedFailures']
+
+ for tid in tids:
+ result = self.tc._results[tid]
+ for field in fields:
+ if not field in self.tc._results:
+ self.tc._results[field] = []
+ self.tc._results[field].extend(result[field])
+
+ def addResult(self, result, run_start_time, run_end_time):
+ tid = threading.get_ident()
+
+ self._lock.acquire()
+ self._results[tid] = {}
+ self._results[tid]['result'] = result
+ self._results[tid]['run_start_time'] = run_start_time
+ self._results[tid]['run_end_time'] = run_end_time
+ self._results[tid]['result'] = result
+ self._lock.release()
+
+ def wasSuccessful(self):
+ wasSuccessful = True
+ for tid in self._results.keys():
+ wasSuccessful = wasSuccessful and \
+ self._results[tid]['result'].wasSuccessful()
+ return wasSuccessful
+
+ def stop(self):
+ for tid in self._results.keys():
+ self._results[tid]['result'].stop()
+
+ def logSummary(self, component, context_msg=''):
+ elapsed_time = (self.tc._run_end_time - self.tc._run_start_time)
+
+ self.tc.logger.info("SUMMARY:")
+ self.tc.logger.info("%s (%s) - Ran %d tests in %.3fs" % (component,
+ context_msg, len(self.tc._registry['cases']), elapsed_time))
+ if self.wasSuccessful():
+ msg = "%s - OK - All required tests passed" % component
+ else:
+ msg = "%s - FAIL - Required tests failed" % component
+ self.tc.logger.info(msg)
+
+ def logDetails(self):
+ tid = list(self._results)[0]
+ result = self._results[tid]['result']
+ result.logDetails()
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 11/17] oeqa/core/threaded: Add support of OETestRunnerThreaded
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (9 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 10/17] oeqa/core/threaded: Add OETestResultThreaded{, Internal} classes Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 12/17] oeqa/core/threaded: Add OETestContextThreaded class Aníbal Limón
` (5 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
The OETestRunnerThreaded overrides the run method of OETestRunner
it recieves a list of suites to be executed by a ThreadPool.
The new run method handles the ThreadPool creation and the
OETestResultThreaded fill.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/threaded.py | 75 +++++++++++++++++++++++++++++++++++++++++-
1 file changed, 74 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oeqa/core/threaded.py b/meta/lib/oeqa/core/threaded.py
index f216685..81df340 100644
--- a/meta/lib/oeqa/core/threaded.py
+++ b/meta/lib/oeqa/core/threaded.py
@@ -3,11 +3,13 @@
import threading
import multiprocessing
+import queue
+import time
from unittest.suite import TestSuite
from oeqa.core.loader import OETestLoader
-from oeqa.core.runner import OEStreamLogger, OETestResult
+from oeqa.core.runner import OEStreamLogger, OETestResult, OETestRunner
class OETestLoaderThreaded(OETestLoader):
def __init__(self, tc, module_paths, modules, tests, modules_required,
@@ -185,3 +187,74 @@ class OETestResultThreaded(object):
tid = list(self._results)[0]
result = self._results[tid]['result']
result.logDetails()
+
+class _Worker(threading.Thread):
+ """Thread executing tasks from a given tasks queue"""
+ def __init__(self, tasks, result, stream):
+ threading.Thread.__init__(self)
+ self.tasks = tasks
+
+ self.result = result
+ self.stream = stream
+
+ def run(self):
+ while True:
+ try:
+ func, args, kargs = self.tasks.get(block=False)
+ except queue.Empty:
+ break
+
+ try:
+ run_start_time = time.time()
+ rc = func(*args, **kargs)
+ run_end_time = time.time()
+ self.result.addResult(rc, run_start_time, run_end_time)
+ self.stream.finish()
+ except Exception as e:
+ print(e)
+ finally:
+ self.tasks.task_done()
+
+class _ThreadedPool:
+ """Pool of threads consuming tasks from a queue"""
+ def __init__(self, num_workers, num_tasks, stream=None, result=None):
+ self.tasks = queue.Queue(num_tasks)
+ self.workers = []
+
+ for _ in range(num_workers):
+ worker = _Worker(self.tasks, result, stream)
+ self.workers.append(worker)
+
+ def start(self):
+ for worker in self.workers:
+ worker.start()
+
+ def add_task(self, func, *args, **kargs):
+ """Add a task to the queue"""
+ self.tasks.put((func, args, kargs))
+
+ def wait_completion(self):
+ """Wait for completion of all the tasks in the queue"""
+ self.tasks.join()
+ for worker in self.workers:
+ worker.join()
+
+class OETestRunnerThreaded(OETestRunner):
+ streamLoggerClass = OEStreamLoggerThreaded
+
+ def __init__(self, tc, *args, **kwargs):
+ super(OETestRunnerThreaded, self).__init__(tc, *args, **kwargs)
+ self.resultclass = OETestResultThreadedInternal # XXX: XML reporting overrides at __init__
+
+ def run(self, suites):
+ result = OETestResultThreaded(self.tc)
+
+ pool = _ThreadedPool(len(suites), len(suites), stream=self.stream,
+ result=result)
+ for s in suites:
+ pool.add_task(super(OETestRunnerThreaded, self).run, s)
+ pool.start()
+ pool.wait_completion()
+ result._fill_tc_results()
+
+ return result
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 12/17] oeqa/core/threaded: Add OETestContextThreaded class
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (10 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 11/17] oeqa/core/threaded: Add support of OETestRunnerThreaded Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 13/17] oeqa/core/decorator/depends: Add support for threading mode Aníbal Limón
` (4 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
This class sets the {loader,runner}Class to the threaded versions and
overrides loadTests method to be able specify the process_num.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/threaded.py | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/meta/lib/oeqa/core/threaded.py b/meta/lib/oeqa/core/threaded.py
index 81df340..c6058a3 100644
--- a/meta/lib/oeqa/core/threaded.py
+++ b/meta/lib/oeqa/core/threaded.py
@@ -10,6 +10,7 @@ from unittest.suite import TestSuite
from oeqa.core.loader import OETestLoader
from oeqa.core.runner import OEStreamLogger, OETestResult, OETestRunner
+from oeqa.core.context import OETestContext
class OETestLoaderThreaded(OETestLoader):
def __init__(self, tc, module_paths, modules, tests, modules_required,
@@ -258,3 +259,16 @@ class OETestRunnerThreaded(OETestRunner):
result._fill_tc_results()
return result
+
+class OETestContextThreaded(OETestContext):
+ loaderClass = OETestLoaderThreaded
+ runnerClass = OETestRunnerThreaded
+
+ def loadTests(self, module_paths, modules=[], tests=[],
+ modules_manifest="", modules_required=[], filters={}, process_num=0):
+ if modules_manifest:
+ modules = self._read_modules_from_manifest(modules_manifest)
+
+ self.loader = self.loaderClass(self, module_paths, modules, tests,
+ modules_required, filters, process_num)
+ self.suites = self.loader.discover()
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 13/17] oeqa/core/decorator/depends: Add support for threading mode
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (11 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 12/17] oeqa/core/threaded: Add OETestContextThreaded class Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 14/17] oeqa/core/decorator/oetimeout: Add support for OEQA threaded mode Aníbal Limón
` (3 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
The _skipTestDependency needs to know if the thread mode is
enabled because the _results are by thread.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/decorator/depends.py | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oeqa/core/decorator/depends.py b/meta/lib/oeqa/core/decorator/depends.py
index 195711c..baa0434 100644
--- a/meta/lib/oeqa/core/decorator/depends.py
+++ b/meta/lib/oeqa/core/decorator/depends.py
@@ -3,6 +3,7 @@
from unittest import SkipTest
+from oeqa.core.threaded import OETestRunnerThreaded
from oeqa.core.exception import OEQADependency
from . import OETestDiscover, registerDecorator
@@ -63,7 +64,12 @@ def _order_test_case_by_depends(cases, depends):
return [cases[case_id] for case_id in cases_ordered]
def _skipTestDependency(case, depends):
- results = case.tc._results
+ if isinstance(case.tc.runner, OETestRunnerThreaded):
+ import threading
+ results = case.tc._results[threading.get_ident()]
+ else:
+ results = case.tc._results
+
skipReasons = ['errors', 'failures', 'skipped']
for reason in skipReasons:
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 14/17] oeqa/core/decorator/oetimeout: Add support for OEQA threaded mode
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (12 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 13/17] oeqa/core/decorator/depends: Add support for threading mode Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 15/17] oeqa/core/tests: Add tests of OEQA Threaded mode Aníbal Limón
` (2 subsequent siblings)
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
In python signals are only allowed to delivery into the main thread,
to support the threading mode test if the runner is threaded and
use threading.Timer instead.
There are some considerations like SIGALRM interrupts the execution
after N seconds but the Timer only starts a Thread to notice the
timeout and the exception will be raised when the test run ends.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/decorator/oetimeout.py | 40 +++++++++++++++++++++++--------
1 file changed, 30 insertions(+), 10 deletions(-)
diff --git a/meta/lib/oeqa/core/decorator/oetimeout.py b/meta/lib/oeqa/core/decorator/oetimeout.py
index a247583..f85e7d9 100644
--- a/meta/lib/oeqa/core/decorator/oetimeout.py
+++ b/meta/lib/oeqa/core/decorator/oetimeout.py
@@ -1,8 +1,12 @@
# Copyright (C) 2016 Intel Corporation
# Released under the MIT license (see COPYING.MIT)
-import signal
from . import OETestDecorator, registerDecorator
+
+import signal
+from threading import Timer
+
+from oeqa.core.threaded import OETestRunnerThreaded
from oeqa.core.exception import OEQATimeoutError
@registerDecorator
@@ -10,16 +14,32 @@ class OETimeout(OETestDecorator):
attrs = ('oetimeout',)
def setUpDecorator(self):
- timeout = self.oetimeout
- def _timeoutHandler(signum, frame):
- raise OEQATimeoutError("Timed out after %s "
+ self.logger.debug("Setting up a %d second(s) timeout" % self.oetimeout)
+
+ if isinstance(self.case.tc.runner, OETestRunnerThreaded):
+ self.timeouted = False
+ def _timeoutHandler():
+ self.timeouted = True
+
+ self.timer = Timer(self.oetimeout, _timeoutHandler)
+ self.timer.start()
+ else:
+ timeout = self.oetimeout
+ def _timeoutHandler(signum, frame):
+ raise OEQATimeoutError("Timed out after %s "
"seconds of execution" % timeout)
- self.logger.debug("Setting up a %d second(s) timeout" % self.oetimeout)
- self.alarmSignal = signal.signal(signal.SIGALRM, _timeoutHandler)
- signal.alarm(self.oetimeout)
+ self.alarmSignal = signal.signal(signal.SIGALRM, _timeoutHandler)
+ signal.alarm(self.oetimeout)
def tearDownDecorator(self):
- signal.alarm(0)
- signal.signal(signal.SIGALRM, self.alarmSignal)
- self.logger.debug("Removed SIGALRM handler")
+ if isinstance(self.case.tc.runner, OETestRunnerThreaded):
+ self.timer.cancel()
+ self.logger.debug("Removed Timer handler")
+ if self.timeouted:
+ raise OEQATimeoutError("Timed out after %s "
+ "seconds of execution" % self.oetimeout)
+ else:
+ signal.alarm(0)
+ signal.signal(signal.SIGALRM, self.alarmSignal)
+ self.logger.debug("Removed SIGALRM handler")
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 15/17] oeqa/core/tests: Add tests of OEQA Threaded mode
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (13 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 14/17] oeqa/core/decorator/oetimeout: Add support for OEQA threaded mode Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 16/17] oeqa/sdkext/cases: Move sdk_update test into devtool module Aníbal Limón
2017-05-10 19:47 ` [PATCH 17/17] oeqa/sdk: Enable usage of OEQA thread mode Aníbal Limón
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
Add needed tests to validate the OEQA Threaded mode, the remaining
parts are tested by the OEQA without Threaded mode.
- test_loader.py: Add a test to validate rules when creating the
list of test suites.
- test_decorators.py: Add oetimeout test because the threaded mode
uses Timer instead of signal.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
.../core/tests/cases/loader/threaded/threaded.py | 12 +++++++++
.../tests/cases/loader/threaded/threaded_alone.py | 8 ++++++
.../cases/loader/threaded/threaded_depends.py | 10 ++++++++
.../tests/cases/loader/threaded/threaded_module.py | 12 +++++++++
meta/lib/oeqa/core/tests/common.py | 10 ++++++++
meta/lib/oeqa/core/tests/test_decorators.py | 12 +++++++++
meta/lib/oeqa/core/tests/test_loader.py | 30 +++++++++++++++++++++-
7 files changed, 93 insertions(+), 1 deletion(-)
create mode 100644 meta/lib/oeqa/core/tests/cases/loader/threaded/threaded.py
create mode 100644 meta/lib/oeqa/core/tests/cases/loader/threaded/threaded_alone.py
create mode 100644 meta/lib/oeqa/core/tests/cases/loader/threaded/threaded_depends.py
create mode 100644 meta/lib/oeqa/core/tests/cases/loader/threaded/threaded_module.py
diff --git a/meta/lib/oeqa/core/tests/cases/loader/threaded/threaded.py b/meta/lib/oeqa/core/tests/cases/loader/threaded/threaded.py
new file mode 100644
index 0000000..0fe4cb3
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/cases/loader/threaded/threaded.py
@@ -0,0 +1,12 @@
+# Copyright (C) 2017 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.case import OETestCase
+
+class ThreadedTest(OETestCase):
+ def test_threaded_no_depends(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+class ThreadedTest2(OETestCase):
+ def test_threaded_same_module(self):
+ self.assertTrue(True, msg='How is this possible?')
diff --git a/meta/lib/oeqa/core/tests/cases/loader/threaded/threaded_alone.py b/meta/lib/oeqa/core/tests/cases/loader/threaded/threaded_alone.py
new file mode 100644
index 0000000..905f397
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/cases/loader/threaded/threaded_alone.py
@@ -0,0 +1,8 @@
+# Copyright (C) 2017 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.case import OETestCase
+
+class ThreadedTestAlone(OETestCase):
+ def test_threaded_alone(self):
+ self.assertTrue(True, msg='How is this possible?')
diff --git a/meta/lib/oeqa/core/tests/cases/loader/threaded/threaded_depends.py b/meta/lib/oeqa/core/tests/cases/loader/threaded/threaded_depends.py
new file mode 100644
index 0000000..0c158d3
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/cases/loader/threaded/threaded_depends.py
@@ -0,0 +1,10 @@
+# Copyright (C) 2017 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.case import OETestCase
+from oeqa.core.decorator.depends import OETestDepends
+
+class ThreadedTest3(OETestCase):
+ @OETestDepends(['threaded.ThreadedTest.test_threaded_no_depends'])
+ def test_threaded_depends(self):
+ self.assertTrue(True, msg='How is this possible?')
diff --git a/meta/lib/oeqa/core/tests/cases/loader/threaded/threaded_module.py b/meta/lib/oeqa/core/tests/cases/loader/threaded/threaded_module.py
new file mode 100644
index 0000000..63d17e0
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/cases/loader/threaded/threaded_module.py
@@ -0,0 +1,12 @@
+# Copyright (C) 2017 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.case import OETestCase
+
+class ThreadedTestModule(OETestCase):
+ def test_threaded_module(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+class ThreadedTestModule2(OETestCase):
+ def test_threaded_module2(self):
+ self.assertTrue(True, msg='How is this possible?')
diff --git a/meta/lib/oeqa/core/tests/common.py b/meta/lib/oeqa/core/tests/common.py
index 52b18a1..1932323 100644
--- a/meta/lib/oeqa/core/tests/common.py
+++ b/meta/lib/oeqa/core/tests/common.py
@@ -33,3 +33,13 @@ class TestBase(unittest.TestCase):
tc.loadTests(self.cases_path, modules=modules, tests=tests,
filters=filters)
return tc
+
+ def _testLoaderThreaded(self, d={}, modules=[],
+ tests=[], filters={}):
+ from oeqa.core.threaded import OETestContextThreaded
+
+ tc = OETestContextThreaded(d, self.logger)
+ tc.loadTests(self.cases_path, modules=modules, tests=tests,
+ filters=filters)
+
+ return tc
diff --git a/meta/lib/oeqa/core/tests/test_decorators.py b/meta/lib/oeqa/core/tests/test_decorators.py
index f7d11e8..cf99e0d 100755
--- a/meta/lib/oeqa/core/tests/test_decorators.py
+++ b/meta/lib/oeqa/core/tests/test_decorators.py
@@ -131,5 +131,17 @@ class TestTimeoutDecorator(TestBase):
msg = "OETestTimeout didn't restore SIGALRM"
self.assertIs(alarm_signal, signal.getsignal(signal.SIGALRM), msg=msg)
+ def test_timeout_thread(self):
+ tests = ['timeout.TimeoutTest.testTimeoutPass']
+ msg = 'Failed to run test using OETestTimeout'
+ tc = self._testLoaderThreaded(modules=self.modules, tests=tests)
+ self.assertTrue(tc.runTests().wasSuccessful(), msg=msg)
+
+ def test_timeout_threaded_fail(self):
+ tests = ['timeout.TimeoutTest.testTimeoutFail']
+ msg = "OETestTimeout test didn't timeout as expected"
+ tc = self._testLoaderThreaded(modules=self.modules, tests=tests)
+ self.assertFalse(tc.runTests().wasSuccessful(), msg=msg)
+
if __name__ == '__main__':
unittest.main()
diff --git a/meta/lib/oeqa/core/tests/test_loader.py b/meta/lib/oeqa/core/tests/test_loader.py
index b79b8ba..e0d917d 100755
--- a/meta/lib/oeqa/core/tests/test_loader.py
+++ b/meta/lib/oeqa/core/tests/test_loader.py
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
-# Copyright (C) 2016 Intel Corporation
+# Copyright (C) 2016-2017 Intel Corporation
# Released under the MIT license (see COPYING.MIT)
import os
@@ -82,5 +82,33 @@ class TestLoader(TestBase):
msg = 'Expected modules from two different paths'
self.assertEqual(modules, expected_modules, msg=msg)
+ def test_loader_threaded(self):
+ cases_path = self.cases_path
+
+ self.cases_path = [os.path.join(self.cases_path, 'loader', 'threaded')]
+
+ tc = self._testLoaderThreaded()
+ self.assertEqual(len(tc.suites), 3, "Expected to be 3 suites")
+
+ case_ids = ['threaded.ThreadedTest.test_threaded_no_depends',
+ 'threaded.ThreadedTest2.test_threaded_same_module',
+ 'threaded_depends.ThreadedTest3.test_threaded_depends']
+ for case in tc.suites[0]._tests:
+ self.assertEqual(case.id(),
+ case_ids[tc.suites[0]._tests.index(case)])
+
+ case_ids = ['threaded_alone.ThreadedTestAlone.test_threaded_alone']
+ for case in tc.suites[1]._tests:
+ self.assertEqual(case.id(),
+ case_ids[tc.suites[1]._tests.index(case)])
+
+ case_ids = ['threaded_module.ThreadedTestModule.test_threaded_module',
+ 'threaded_module.ThreadedTestModule2.test_threaded_module2']
+ for case in tc.suites[2]._tests:
+ self.assertEqual(case.id(),
+ case_ids[tc.suites[2]._tests.index(case)])
+
+ self.cases_path = cases_path
+
if __name__ == '__main__':
unittest.main()
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 16/17] oeqa/sdkext/cases: Move sdk_update test into devtool module
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (14 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 15/17] oeqa/core/tests: Add tests of OEQA Threaded mode Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
2017-05-10 19:47 ` [PATCH 17/17] oeqa/sdk: Enable usage of OEQA thread mode Aníbal Limón
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
With the new OEQA thread support there are problems running
devtool twice at the same time because only one instance of
bitbake/devtool is allowed.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/sdkext/cases/devtool.py | 32 ++++++++++++++++++++++++++
meta/lib/oeqa/sdkext/cases/sdk_update.py | 39 --------------------------------
2 files changed, 32 insertions(+), 39 deletions(-)
delete mode 100644 meta/lib/oeqa/sdkext/cases/sdk_update.py
diff --git a/meta/lib/oeqa/sdkext/cases/devtool.py b/meta/lib/oeqa/sdkext/cases/devtool.py
index a01bc0b..ea90517 100644
--- a/meta/lib/oeqa/sdkext/cases/devtool.py
+++ b/meta/lib/oeqa/sdkext/cases/devtool.py
@@ -1,12 +1,14 @@
# Copyright (C) 2016 Intel Corporation
# Released under the MIT license (see COPYING.MIT)
+import os
import shutil
import subprocess
from oeqa.sdkext.case import OESDKExtTestCase
from oeqa.core.decorator.depends import OETestDepends
from oeqa.core.decorator.oeid import OETestID
+from oeqa.utils.httpserver import HTTPService
class DevtoolTest(OESDKExtTestCase):
@classmethod
@@ -95,3 +97,33 @@ class DevtoolTest(OESDKExtTestCase):
self._run('devtool build %s ' % package_nodejs)
finally:
self._run('devtool reset %s '% package_nodejs)
+
+class SdkUpdateTest(OESDKExtTestCase):
+ @classmethod
+ def setUpClass(self):
+ self.publish_dir = os.path.join(self.tc.sdk_dir, 'esdk_publish')
+ if os.path.exists(self.publish_dir):
+ shutil.rmtree(self.publish_dir)
+ os.mkdir(self.publish_dir)
+
+ base_tcname = "%s/%s" % (self.td.get("SDK_DEPLOY", ''),
+ self.td.get("TOOLCHAINEXT_OUTPUTNAME", ''))
+ tcname_new = "%s-new.sh" % base_tcname
+ if not os.path.exists(tcname_new):
+ tcname_new = "%s.sh" % base_tcname
+
+ cmd = 'oe-publish-sdk %s %s' % (tcname_new, self.publish_dir)
+ subprocess.check_output(cmd, shell=True)
+
+ self.http_service = HTTPService(self.publish_dir)
+ self.http_service.start()
+
+ self.http_url = "http://127.0.0.1:%d" % self.http_service.port
+
+ def test_sdk_update_http(self):
+ output = self._run("devtool sdk-update \"%s\"" % self.http_url)
+
+ @classmethod
+ def tearDownClass(self):
+ self.http_service.stop()
+ shutil.rmtree(self.publish_dir)
diff --git a/meta/lib/oeqa/sdkext/cases/sdk_update.py b/meta/lib/oeqa/sdkext/cases/sdk_update.py
deleted file mode 100644
index 2f8598b..0000000
--- a/meta/lib/oeqa/sdkext/cases/sdk_update.py
+++ /dev/null
@@ -1,39 +0,0 @@
-# Copyright (C) 2016 Intel Corporation
-# Released under the MIT license (see COPYING.MIT)
-
-import os
-import shutil
-import subprocess
-
-from oeqa.sdkext.case import OESDKExtTestCase
-from oeqa.utils.httpserver import HTTPService
-
-class SdkUpdateTest(OESDKExtTestCase):
- @classmethod
- def setUpClass(self):
- self.publish_dir = os.path.join(self.tc.sdk_dir, 'esdk_publish')
- if os.path.exists(self.publish_dir):
- shutil.rmtree(self.publish_dir)
- os.mkdir(self.publish_dir)
-
- base_tcname = "%s/%s" % (self.td.get("SDK_DEPLOY", ''),
- self.td.get("TOOLCHAINEXT_OUTPUTNAME", ''))
- tcname_new = "%s-new.sh" % base_tcname
- if not os.path.exists(tcname_new):
- tcname_new = "%s.sh" % base_tcname
-
- cmd = 'oe-publish-sdk %s %s' % (tcname_new, self.publish_dir)
- subprocess.check_output(cmd, shell=True)
-
- self.http_service = HTTPService(self.publish_dir)
- self.http_service.start()
-
- self.http_url = "http://127.0.0.1:%d" % self.http_service.port
-
- def test_sdk_update_http(self):
- output = self._run("devtool sdk-update \"%s\"" % self.http_url)
-
- @classmethod
- def tearDownClass(self):
- self.http_service.stop()
- shutil.rmtree(self.publish_dir)
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 17/17] oeqa/sdk: Enable usage of OEQA thread mode
2017-05-10 19:47 [PATCH 00/17] OEQA framework add support of threaded mode and enable in SDK Aníbal Limón
` (15 preceding siblings ...)
2017-05-10 19:47 ` [PATCH 16/17] oeqa/sdkext/cases: Move sdk_update test into devtool module Aníbal Limón
@ 2017-05-10 19:47 ` Aníbal Limón
16 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2017-05-10 19:47 UTC (permalink / raw)
To: openembedded-core; +Cc: paul.eggleton, juan.m.cruz.alcaraz
oeqa/sdk/context.py - Use OETestContextThreaded.
classes/testsdk.bbclass - Enable bb event thread mode to avoid
corrupt the PIPE when multiple threads writes.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/classes/testsdk.bbclass | 4 ++++
meta/lib/oeqa/sdk/context.py | 5 +++--
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
index edea89c..8a9e680 100644
--- a/meta/classes/testsdk.bbclass
+++ b/meta/classes/testsdk.bbclass
@@ -24,6 +24,8 @@ def testsdk_main(d):
from oeqa.sdk.context import OESDKTestContext, OESDKTestContextExecutor
from oeqa.utils import make_logger_bitbake_compatible
+ bb.event.enable_threadlock()
+
pn = d.getVar("PN")
logger = make_logger_bitbake_compatible(logging.getLogger("BitBake"))
@@ -97,6 +99,8 @@ def testsdkext_main(d):
from oeqa.utils import avoid_paths_in_environ, make_logger_bitbake_compatible, subprocesstweak
from oeqa.sdkext.context import OESDKExtTestContext, OESDKExtTestContextExecutor
+ bb.event.enable_threadlock()
+
pn = d.getVar("PN")
logger = make_logger_bitbake_compatible(logging.getLogger("BitBake"))
diff --git a/meta/lib/oeqa/sdk/context.py b/meta/lib/oeqa/sdk/context.py
index 82e4c19..b3d7c75 100644
--- a/meta/lib/oeqa/sdk/context.py
+++ b/meta/lib/oeqa/sdk/context.py
@@ -6,9 +6,10 @@ import sys
import glob
import re
-from oeqa.core.context import OETestContext, OETestContextExecutor
+from oeqa.core.context import OETestContextExecutor
+from oeqa.core.threaded import OETestContextThreaded
-class OESDKTestContext(OETestContext):
+class OESDKTestContext(OETestContextThreaded):
sdk_files_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "files")
def __init__(self, td=None, logger=None, sdk_dir=None, sdk_env=None,
--
2.1.4
^ permalink raw reply related [flat|nested] 18+ messages in thread