* [PATCH 01/55] oeqa/core: Add base OEQA framework
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 02/55] oeqa/core: Add utils module for " Aníbal Limón
` (55 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
case: Defines OETestCase base class that provides custom
methods/attrs defined by the framework.
Every OETestCase instance contains a reference to the test
data (d), the test context (tc) and the logger.
Also implements _oe{SetUp,TearDown}Class for make special
handling of OEQA decorators and validations.
runner: Defines OETestRunner/OETestResult with support for RAW
and XML result logs.
exception: Custom exceptions related to the OEQA framework based
on class OEQAException.
[YOCTO #10230]
[YOCTO #10233]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/__init__.py | 0
meta/lib/oeqa/core/case.py | 46 +++++++++++++++++++++++++
meta/lib/oeqa/core/exception.py | 14 ++++++++
meta/lib/oeqa/core/runner.py | 76 +++++++++++++++++++++++++++++++++++++++++
4 files changed, 136 insertions(+)
create mode 100644 meta/lib/oeqa/core/__init__.py
create mode 100644 meta/lib/oeqa/core/case.py
create mode 100644 meta/lib/oeqa/core/exception.py
create mode 100644 meta/lib/oeqa/core/runner.py
diff --git a/meta/lib/oeqa/core/__init__.py b/meta/lib/oeqa/core/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/meta/lib/oeqa/core/case.py b/meta/lib/oeqa/core/case.py
new file mode 100644
index 0000000..d2dbf20
--- /dev/null
+++ b/meta/lib/oeqa/core/case.py
@@ -0,0 +1,46 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import unittest
+
+from oeqa.core.exception import OEQAMissingVariable
+
+def _validate_td_vars(td, td_vars, type_msg):
+ if td_vars:
+ for v in td_vars:
+ if not v in td:
+ raise OEQAMissingVariable("Test %s need %s variable but"\
+ " isn't into td" % (type_msg, v))
+
+class OETestCase(unittest.TestCase):
+ # TestContext and Logger instance set by OETestLoader.
+ tc = None
+ logger = None
+
+ # td has all the variables needed by the test cases
+ # is the same across all the test cases.
+ td = None
+
+ # td_vars has the variables needed by a test class
+ # or test case instance, if some var isn't into td a
+ # OEMissingVariable exception is raised
+ td_vars = None
+
+ @classmethod
+ def _oeSetUpClass(clss):
+ _validate_td_vars(clss.td, clss.td_vars, "class")
+ clss.setUpClassMethod()
+
+ @classmethod
+ def _oeTearDownClass(clss):
+ clss.tearDownClassMethod()
+
+ def _oeSetUp(self):
+ for d in self.decorators:
+ d.setUpDecorator()
+ self.setUpMethod()
+
+ def _oeTearDown(self):
+ for d in self.decorators:
+ d.tearDownDecorator()
+ self.tearDownMethod()
diff --git a/meta/lib/oeqa/core/exception.py b/meta/lib/oeqa/core/exception.py
new file mode 100644
index 0000000..2dfd840
--- /dev/null
+++ b/meta/lib/oeqa/core/exception.py
@@ -0,0 +1,14 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+class OEQAException(Exception):
+ pass
+
+class OEQATimeoutError(OEQAException):
+ pass
+
+class OEQAMissingVariable(OEQAException):
+ pass
+
+class OEQADependency(OEQAException):
+ pass
diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py
new file mode 100644
index 0000000..8f5af57
--- /dev/null
+++ b/meta/lib/oeqa/core/runner.py
@@ -0,0 +1,76 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import time
+import unittest
+import logging
+
+xmlEnabled = False
+try:
+ import xmlrunner
+ from xmlrunner.result import _XMLTestResult as _TestResult
+ from xmlrunner.runner import XMLTestRunner as _TestRunner
+ xmlEnabled = True
+except ImportError:
+ # use the base runner instead
+ from unittest import TextTestResult as _TestResult
+ from unittest import TextTestRunner as _TestRunner
+
+class OEStreamLogger(object):
+ def __init__(self, logger):
+ self.logger = logger
+ self.buffer = ""
+
+ def write(self, msg):
+ if msg[-1] != '\n':
+ self.buffer += msg
+ else:
+ self.logger.log(logging.INFO, self.buffer.rstrip("\n"))
+ self.buffer = ""
+
+ def flush(self):
+ for handler in self.logger.handlers:
+ handler.flush()
+
+class OETestResult(_TestResult):
+ def __init__(self, tc, *args, **kwargs):
+ super(OETestResult, self).__init__(*args, **kwargs)
+
+ self.tc = tc
+
+ self.tc._results['failures'] = self.failures
+ self.tc._results['errors'] = self.errors
+ self.tc._results['skipped'] = self.skipped
+ self.tc._results['expectedFailures'] = self.expectedFailures
+
+ def startTest(self, test):
+ super(OETestResult, self).startTest(test)
+
+class OETestRunner(_TestRunner):
+ def __init__(self, tc, *args, **kwargs):
+ if xmlEnabled:
+ if not kwargs.get('output'):
+ kwargs['output'] = os.path.join(os.getcwd(),
+ 'TestResults_%s' % time.strftime("%Y%m%d%H%M%S"))
+
+ super(OETestRunner, self).__init__(*args, **kwargs)
+ self.tc = tc
+ self.resultclass = OETestResult
+
+ # XXX: The unittest-xml-reporting package defines _make_result method instead
+ # of _makeResult standard on unittest.
+ if xmlEnabled:
+ def _make_result(self):
+ """
+ Creates a TestResult object which will be used to store
+ information about the executed tests.
+ """
+ # override in subclasses if necessary.
+ return self.resultclass(self.tc,
+ self.stream, self.descriptions, self.verbosity, self.elapsed_times
+ )
+ else:
+ def _makeResult(self):
+ return self.resultclass(self.tc, self.stream, self.descriptions,
+ self.verbosity)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 02/55] oeqa/core: Add utils module for OEQA framework
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
2017-01-20 17:09 ` [PATCH 01/55] oeqa/core: Add base OEQA framework Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 03/55] oeqa/core: Add loader, context and decorator modules Aníbal Limón
` (54 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
misc: Functions for transform object to other types.
path: Functions for path handling.
test: Functions for operations related to test cases and suites.
[YOCTO #10232]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/utils/__init__.py | 0
meta/lib/oeqa/core/utils/misc.py | 37 +++++++++++++++++++
meta/lib/oeqa/core/utils/path.py | 19 ++++++++++
meta/lib/oeqa/core/utils/test.py | 71 ++++++++++++++++++++++++++++++++++++
4 files changed, 127 insertions(+)
create mode 100644 meta/lib/oeqa/core/utils/__init__.py
create mode 100644 meta/lib/oeqa/core/utils/misc.py
create mode 100644 meta/lib/oeqa/core/utils/path.py
create mode 100644 meta/lib/oeqa/core/utils/test.py
diff --git a/meta/lib/oeqa/core/utils/__init__.py b/meta/lib/oeqa/core/utils/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/meta/lib/oeqa/core/utils/misc.py b/meta/lib/oeqa/core/utils/misc.py
new file mode 100644
index 0000000..6ae58ad
--- /dev/null
+++ b/meta/lib/oeqa/core/utils/misc.py
@@ -0,0 +1,37 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+def toList(obj, obj_type, obj_name="Object"):
+ if isinstance(obj, obj_type):
+ return [obj]
+ elif isinstance(obj, list):
+ return obj
+ else:
+ raise TypeError("%s must be %s or list" % (obj_name, obj_type))
+
+def toSet(obj, obj_type, obj_name="Object"):
+ if isinstance(obj, obj_type):
+ return {obj}
+ elif isinstance(obj, list):
+ return set(obj)
+ elif isinstance(obj, set):
+ return obj
+ else:
+ raise TypeError("%s must be %s or set" % (obj_name, obj_type))
+
+def strToList(obj, obj_name="Object"):
+ return toList(obj, str, obj_name)
+
+def strToSet(obj, obj_name="Object"):
+ return toSet(obj, str, obj_name)
+
+def intToList(obj, obj_name="Object"):
+ return toList(obj, int, obj_name)
+
+def dataStoteToDict(d, variables):
+ data = {}
+
+ for v in variables:
+ data[v] = d.getVar(v, True)
+
+ return data
diff --git a/meta/lib/oeqa/core/utils/path.py b/meta/lib/oeqa/core/utils/path.py
new file mode 100644
index 0000000..a21caad
--- /dev/null
+++ b/meta/lib/oeqa/core/utils/path.py
@@ -0,0 +1,19 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import sys
+
+def findFile(file_name, directory):
+ """
+ Search for a file in directory and returns its complete path.
+ """
+ for r, d, f in os.walk(directory):
+ if file_name in f:
+ return os.path.join(r, file_name)
+ return None
+
+def remove_safe(path):
+ if os.path.exists(path):
+ os.remove(path)
+
diff --git a/meta/lib/oeqa/core/utils/test.py b/meta/lib/oeqa/core/utils/test.py
new file mode 100644
index 0000000..820b997
--- /dev/null
+++ b/meta/lib/oeqa/core/utils/test.py
@@ -0,0 +1,71 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import unittest
+
+def getSuiteCases(suite):
+ """
+ Returns individual test from a test suite.
+ """
+ tests = []
+ for item in suite:
+ if isinstance(item, unittest.suite.TestSuite):
+ tests.extend(getSuiteCases(item))
+ elif isinstance(item, unittest.TestCase):
+ tests.append(item)
+ return tests
+
+def getSuiteModules(suite):
+ """
+ Returns modules in a test suite.
+ """
+ modules = set()
+ for test in getSuiteCases(suite):
+ modules.add(getCaseModule(test))
+ return modules
+
+def getSuiteCasesInfo(suite, func):
+ """
+ Returns test case info from suite. Info is fetched from func.
+ """
+ tests = []
+ for test in getSuiteCases(suite):
+ tests.append(func(test))
+ return tests
+
+def getSuiteCasesNames(suite):
+ """
+ Returns test case names from suite.
+ """
+ return getSuiteCasesInfo(suite, getCaseMethod)
+
+def getSuiteCasesIDs(suite):
+ """
+ Returns test case ids from suite.
+ """
+ return getSuiteCasesInfo(suite, getCaseID)
+
+def getCaseModule(test_case):
+ """
+ Returns test case module name.
+ """
+ return test_case.__module__
+
+def getCaseClass(test_case):
+ """
+ Returns test case class name.
+ """
+ return test_case.__class__.__name__
+
+def getCaseID(test_case):
+ """
+ Returns test case complete id.
+ """
+ return test_case.id()
+
+def getCaseMethod(test_case):
+ """
+ Returns test case method name.
+ """
+ return getCaseID(test_case).split('.')[-1]
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 03/55] oeqa/core: Add loader, context and decorator modules
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
2017-01-20 17:09 ` [PATCH 01/55] oeqa/core: Add base OEQA framework Aníbal Limón
2017-01-20 17:09 ` [PATCH 02/55] oeqa/core: Add utils module for " Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 04/55] oeqa/core/decorator: Add support for OETestDepends Aníbal Limón
` (53 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
loader: Implements OETestLoader handling OETestDecorator
and filtering support when load tests. The OETestLoader is
responsible to set custom methods, attrs of the OEQA
frameowork.
[YOCTO #10231]
[YOCTO #10317]
[YOCTO #10353]
decorator: Add base class OETestDecorator to provide a common
way to define decorators to be used over OETestCase's, every
decorator has a method to be called when loading tests and
before test execution starts. Special decorators could be
implemented for filter tests on loading phase.
context: Provides HIGH level API for loadTests and runTests
of certain test component (i.e. runtime, sdk, selftest).
[YOCTO #10230]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/context.py | 148 +++++++++++++++++++
meta/lib/oeqa/core/decorator/__init__.py | 71 ++++++++++
meta/lib/oeqa/core/loader.py | 235 +++++++++++++++++++++++++++++++
3 files changed, 454 insertions(+)
create mode 100644 meta/lib/oeqa/core/context.py
create mode 100644 meta/lib/oeqa/core/decorator/__init__.py
create mode 100644 meta/lib/oeqa/core/loader.py
diff --git a/meta/lib/oeqa/core/context.py b/meta/lib/oeqa/core/context.py
new file mode 100644
index 0000000..cbab2f8
--- /dev/null
+++ b/meta/lib/oeqa/core/context.py
@@ -0,0 +1,148 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import sys
+import json
+import time
+import logging
+import collections
+import re
+
+from oeqa.core.loader import OETestLoader
+from oeqa.core.runner import OETestRunner, OEStreamLogger, xmlEnabled
+
+class OETestContext(object):
+ loaderClass = OETestLoader
+ runnerClass = OETestRunner
+ streamLoggerClass = OEStreamLogger
+
+ files_dir = os.path.abspath(os.path.join(os.path.dirname(
+ os.path.abspath(__file__)), "../files"))
+
+ def __init__(self, td=None, logger=None):
+ if not type(td) is dict:
+ raise TypeError("td isn't dictionary type")
+
+ self.td = td
+ self.logger = logger
+ self._registry = {}
+ self._registry['cases'] = collections.OrderedDict()
+ self._results = {}
+
+ def _read_modules_from_manifest(self, manifest):
+ if not os.path.exists(manifest):
+ raise
+
+ modules = []
+ for line in open(manifest).readlines():
+ line = line.strip()
+ if line and not line.startswith("#"):
+ modules.append(line)
+
+ return modules
+
+ def loadTests(self, module_paths, modules=[], tests=[],
+ modules_manifest="", modules_required=[], filters={}):
+ if modules_manifest:
+ modules = self._read_modules_from_manifest(modules_manifest)
+
+ self.loader = self.loaderClass(self, module_paths, modules, tests,
+ modules_required, filters)
+ self.suites = self.loader.discover()
+
+ def runTests(self):
+ streamLogger = self.streamLoggerClass(self.logger)
+ self.runner = self.runnerClass(self, stream=streamLogger, verbosity=2)
+
+ self._run_start_time = time.time()
+ result = self.runner.run(self.suites)
+ self._run_end_time = time.time()
+
+ return result
+
+ def logSummary(self, result, component, context_msg=''):
+ self.logger.info("SUMMARY:")
+ self.logger.info("%s (%s) - Ran %d test%s in %.3fs" % (component,
+ context_msg, result.testsRun, result.testsRun != 1 and "s" or "",
+ (self._run_end_time - self._run_start_time)))
+
+ if result.wasSuccessful():
+ msg = "%s - OK - All required tests passed" % component
+ else:
+ msg = "%s - FAIL - Required tests failed" % component
+ skipped = len(self._results['skipped'])
+ if skipped:
+ msg += " (skipped=%d)" % skipped
+ self.logger.info(msg)
+
+ def _getDetailsNotPassed(self, case, type, desc):
+ found = False
+
+ for (scase, msg) in self._results[type]:
+ # XXX: When XML reporting is enabled scase is
+ # xmlrunner.result._TestInfo instance instead of
+ # string.
+ if xmlEnabled:
+ if case.id() == scase.test_id:
+ found = True
+ break
+ scase_str = scase.test_id
+ else:
+ if case == scase:
+ found = True
+ break
+ scase_str = str(scase)
+
+ # When fails at module or class level the class name is passed as string
+ # so figure out to see if match
+ m = re.search("^setUpModule \((?P<module_name>.*)\)$", scase_str)
+ if m:
+ if case.__class__.__module__ == m.group('module_name'):
+ found = True
+ break
+
+ m = re.search("^setUpClass \((?P<class_name>.*)\)$", scase_str)
+ if m:
+ class_name = "%s.%s" % (case.__class__.__module__,
+ case.__class__.__name__)
+
+ if class_name == m.group('class_name'):
+ found = True
+ break
+
+ if found:
+ return (found, msg)
+
+ return (found, None)
+
+ def logDetails(self):
+ self.logger.info("RESULTS:")
+ for case_name in self._registry['cases']:
+ case = self._registry['cases'][case_name]
+
+ result_types = ['failures', 'errors', 'skipped', 'expectedFailures']
+ result_desc = ['FAILED', 'ERROR', 'SKIPPED', 'EXPECTEDFAIL']
+
+ fail = False
+ desc = None
+ for idx, name in enumerate(result_types):
+ (fail, msg) = self._getDetailsNotPassed(case, result_types[idx],
+ result_desc[idx])
+ if fail:
+ desc = result_desc[idx]
+ break
+
+ oeid = -1
+ for d in case.decorators:
+ if hasattr(d, 'oeid'):
+ oeid = d.oeid
+
+ if fail:
+ self.logger.info("RESULTS - %s - Testcase %s: %s" % (case.id(),
+ oeid, desc))
+ if msg:
+ self.logger.info(msg)
+ else:
+ self.logger.info("RESULTS - %s - Testcase %s: %s" % (case.id(),
+ oeid, 'PASSED'))
diff --git a/meta/lib/oeqa/core/decorator/__init__.py b/meta/lib/oeqa/core/decorator/__init__.py
new file mode 100644
index 0000000..855b6b9
--- /dev/null
+++ b/meta/lib/oeqa/core/decorator/__init__.py
@@ -0,0 +1,71 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from functools import wraps
+from abc import abstractmethod
+
+decoratorClasses = set()
+
+def registerDecorator(obj):
+ decoratorClasses.add(obj)
+ return obj
+
+class OETestDecorator(object):
+ case = None # Reference of OETestCase decorated
+ attrs = None # Attributes to be loaded by decorator implementation
+
+ def __init__(self, *args, **kwargs):
+ if not self.attrs:
+ return
+
+ for idx, attr in enumerate(self.attrs):
+ if attr in kwargs:
+ value = kwargs[attr]
+ else:
+ value = args[idx]
+ setattr(self, attr, value)
+
+ def __call__(self, func):
+ @wraps(func)
+ def wrapped_f(*args, **kwargs):
+ self.attrs = self.attrs # XXX: Enables OETestLoader discover
+ return func(*args, **kwargs)
+ return wrapped_f
+
+ # OETestLoader call it when is loading test cases.
+ # XXX: Most methods would change the registry for later
+ # processing; be aware that filtrate method needs to
+ # run later than bind, so there could be data (in the
+ # registry) of a cases that were filtered.
+ def bind(self, registry, case):
+ self.case = case
+ self.logger = case.tc.logger
+ self.case.decorators.append(self)
+
+ # OETestRunner call this method when tries to run
+ # the test case.
+ def setUpDecorator(self):
+ pass
+
+ # OETestRunner call it after a test method has been
+ # called even if the method raised an exception.
+ def tearDownDecorator(self):
+ pass
+
+class OETestDiscover(OETestDecorator):
+
+ # OETestLoader call it after discover test cases
+ # needs to return the cases to be run.
+ @staticmethod
+ def discover(registry):
+ return registry['cases']
+
+class OETestFilter(OETestDecorator):
+
+ # OETestLoader call it while loading the tests
+ # in loadTestsFromTestCase method, it needs to
+ # return a bool, True if needs to be filtered.
+ # This method must consume the filter used.
+ @abstractmethod
+ def filtrate(self, filters):
+ return False
diff --git a/meta/lib/oeqa/core/loader.py b/meta/lib/oeqa/core/loader.py
new file mode 100644
index 0000000..c73ef9a
--- /dev/null
+++ b/meta/lib/oeqa/core/loader.py
@@ -0,0 +1,235 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import sys
+import unittest
+
+from oeqa.core.utils.path import findFile
+from oeqa.core.utils.test import getSuiteModules, getCaseID
+
+from oeqa.core.case import OETestCase
+from oeqa.core.decorator import decoratorClasses, OETestDecorator, \
+ OETestFilter, OETestDiscover
+
+def _make_failed_test(classname, methodname, exception, suiteClass):
+ """
+ When loading tests unittest framework stores the exception in a new
+ class created for be displayed into run().
+
+ For our purposes will be better to raise the exception in loading
+ step instead of wait to run the test suite.
+ """
+ raise exception
+unittest.loader._make_failed_test = _make_failed_test
+
+def _find_duplicated_modules(suite, directory):
+ for module in getSuiteModules(suite):
+ path = findFile('%s.py' % module, directory)
+ if path:
+ raise ImportError("Duplicated %s module found in %s" % (module, path))
+
+class OETestLoader(unittest.TestLoader):
+ caseClass = OETestCase
+
+ kwargs_names = ['testMethodPrefix', 'sortTestMethodUsing', 'suiteClass',
+ '_top_level_dir']
+
+ def __init__(self, tc, module_paths, modules, tests, modules_required,
+ filters, *args, **kwargs):
+ self.tc = tc
+
+ self.modules = modules
+ self.tests = tests
+ self.modules_required = modules_required
+
+ self.filters = filters
+ self.decorator_filters = [d for d in decoratorClasses if \
+ issubclass(d, OETestFilter)]
+ self._validateFilters(self.filters, self.decorator_filters)
+ self.used_filters = [d for d in self.decorator_filters
+ for f in self.filters
+ if f in d.attrs]
+
+ if isinstance(module_paths, str):
+ module_paths = [module_paths]
+ elif not isinstance(module_paths, list):
+ raise TypeError('module_paths must be a str or a list of str')
+ self.module_paths = module_paths
+
+ for kwname in self.kwargs_names:
+ if kwname in kwargs:
+ setattr(self, kwname, kwargs[kwname])
+
+ self._patchCaseClass(self.caseClass)
+
+ def _patchCaseClass(self, testCaseClass):
+ # Adds custom attributes to the OETestCase class
+ setattr(testCaseClass, 'tc', self.tc)
+ setattr(testCaseClass, 'td', self.tc.td)
+ setattr(testCaseClass, 'logger', self.tc.logger)
+
+ def _validateFilters(self, filters, decorator_filters):
+ # Validate if filter isn't empty
+ for key,value in filters.items():
+ if not value:
+ raise TypeError("Filter %s specified is empty" % key)
+
+ # Validate unique attributes
+ attr_filters = [attr for clss in decorator_filters \
+ for attr in clss.attrs]
+ dup_attr = [attr for attr in attr_filters
+ if attr_filters.count(attr) > 1]
+ if dup_attr:
+ raise TypeError('Detected duplicated attribute(s) %s in filter'
+ ' decorators' % ' ,'.join(dup_attr))
+
+ # Validate if filter is supported
+ for f in filters:
+ if f not in attr_filters:
+ classes = ', '.join([d.__name__ for d in decorator_filters])
+ raise TypeError('Found "%s" filter but not declared in any of '
+ '%s decorators' % (f, classes))
+
+ def _registerTestCase(self, case):
+ case_id = case.id()
+ self.tc._registry['cases'][case_id] = case
+
+ def _handleTestCaseDecorators(self, case):
+ def _handle(obj):
+ if isinstance(obj, OETestDecorator):
+ if not obj.__class__ in decoratorClasses:
+ raise Exception("Decorator %s isn't registered" \
+ " in decoratorClasses." % obj.__name__)
+ obj.bind(self.tc._registry, case)
+
+ def _walk_closure(obj):
+ if hasattr(obj, '__closure__') and obj.__closure__:
+ for f in obj.__closure__:
+ obj = f.cell_contents
+ _handle(obj)
+ _walk_closure(obj)
+ method = getattr(case, case._testMethodName, None)
+ _walk_closure(method)
+
+ def _filterTest(self, case):
+ """
+ Returns True if test case must be filtered, False otherwise.
+ """
+ if self.filters:
+ filters = self.filters.copy()
+ case_decorators = [cd for cd in case.decorators
+ if cd.__class__ in self.used_filters]
+
+ # Iterate over case decorators to check if needs to be filtered.
+ for cd in case_decorators:
+ if cd.filtrate(filters):
+ return True
+
+ # Case is missing one or more decorators for all the filters
+ # being used, so filter test case.
+ if filters:
+ return True
+
+ return False
+
+ def _getTestCase(self, testCaseClass, tcName):
+ if not hasattr(testCaseClass, '__oeqa_loader'):
+ # In order to support data_vars validation
+ # monkey patch the default setUp/tearDown{Class} to use
+ # the ones provided by OETestCase
+ setattr(testCaseClass, 'setUpClassMethod',
+ getattr(testCaseClass, 'setUpClass'))
+ setattr(testCaseClass, 'tearDownClassMethod',
+ getattr(testCaseClass, 'tearDownClass'))
+ setattr(testCaseClass, 'setUpClass',
+ testCaseClass._oeSetUpClass)
+ setattr(testCaseClass, 'tearDownClass',
+ testCaseClass._oeTearDownClass)
+
+ # In order to support decorators initialization
+ # monkey patch the default setUp/tearDown to use
+ # a setUpDecorators/tearDownDecorators that methods
+ # will call setUp/tearDown original methods.
+ setattr(testCaseClass, 'setUpMethod',
+ getattr(testCaseClass, 'setUp'))
+ setattr(testCaseClass, 'tearDownMethod',
+ getattr(testCaseClass, 'tearDown'))
+ setattr(testCaseClass, 'setUp', testCaseClass._oeSetUp)
+ setattr(testCaseClass, 'tearDown', testCaseClass._oeTearDown)
+
+ setattr(testCaseClass, '__oeqa_loader', True)
+
+ case = testCaseClass(tcName)
+ setattr(case, 'decorators', [])
+
+ return case
+
+ def loadTestsFromTestCase(self, testCaseClass):
+ """
+ Returns a suite of all tests cases contained in testCaseClass.
+ """
+ if issubclass(testCaseClass, unittest.suite.TestSuite):
+ raise TypeError("Test cases should not be derived from TestSuite." \
+ " Maybe you meant to derive from TestCase?")
+ if not issubclass(testCaseClass, self.caseClass):
+ raise TypeError("Test cases need to be derived from %s" % \
+ caseClass.__name__)
+
+
+ testCaseNames = self.getTestCaseNames(testCaseClass)
+ if not testCaseNames and hasattr(testCaseClass, 'runTest'):
+ testCaseNames = ['runTest']
+
+ suite = []
+ for tcName in testCaseNames:
+ case = self._getTestCase(testCaseClass, tcName)
+ # Filer by case id
+ if not (self.tests and not 'all' in self.tests
+ and not getCaseID(case) in self.tests):
+ self._handleTestCaseDecorators(case)
+
+ # Filter by decorators
+ if not self._filterTest(case):
+ self._registerTestCase(case)
+ suite.append(case)
+
+ return self.suiteClass(suite)
+
+ def discover(self):
+ big_suite = self.suiteClass()
+ for path in self.module_paths:
+ _find_duplicated_modules(big_suite, path)
+ suite = super(OETestLoader, self).discover(path,
+ pattern='*.py', top_level_dir=path)
+ big_suite.addTests(suite)
+
+ cases = None
+ discover_classes = [clss for clss in decoratorClasses
+ if issubclass(clss, OETestDiscover)]
+ for clss in discover_classes:
+ cases = clss.discover(self.tc._registry)
+
+ return self.suiteClass(cases) if cases else big_suite
+
+ # XXX After Python 3.5, remove backward compatibility hacks for
+ # use_load_tests deprecation via *args and **kws. See issue 16662.
+ if sys.version_info >= (3,5):
+ def loadTestsFromModule(self, module, *args, pattern=None, **kws):
+ if not self.modules or "all" in self.modules or \
+ module.__name__ in self.modules:
+ return super(OETestLoader, self).loadTestsFromModule(
+ module, *args, pattern=pattern, **kws)
+ else:
+ return self.suiteClass()
+ else:
+ def loadTestsFromModule(self, module, use_load_tests=True):
+ """
+ Returns a suite of all tests cases contained in module.
+ """
+ if not self.modules or "all" in self.modules or \
+ module.__name__ in self.modules:
+ return super(OETestLoader, self).loadTestsFromModule(
+ module, use_load_tests)
+ else:
+ return self.suiteClass()
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 04/55] oeqa/core/decorator: Add support for OETestDepends
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (2 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 03/55] oeqa/core: Add loader, context and decorator modules Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 05/55] oeqa/core/decorator: Add support for OETestID and OETestTag Aníbal Limón
` (52 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
The OETestDepends decorator could be used over test cases to
define some dependency between them.
At loading time sorting the tests to grauntee that a test case
executes before also raise an exception if found a circular
dependency between test cases.
At before test case run reviews if the dependency if meet, in the
case of don't it skips the test case run.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/decorator/depends.py | 94 +++++++++++++++++++++++++++++++++
1 file changed, 94 insertions(+)
create mode 100644 meta/lib/oeqa/core/decorator/depends.py
diff --git a/meta/lib/oeqa/core/decorator/depends.py b/meta/lib/oeqa/core/decorator/depends.py
new file mode 100644
index 0000000..195711c
--- /dev/null
+++ b/meta/lib/oeqa/core/decorator/depends.py
@@ -0,0 +1,94 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from unittest import SkipTest
+
+from oeqa.core.exception import OEQADependency
+
+from . import OETestDiscover, registerDecorator
+
+def _add_depends(registry, case, depends):
+ module_name = case.__module__
+ class_name = case.__class__.__name__
+
+ case_id = case.id()
+
+ for depend in depends:
+ dparts = depend.split('.')
+
+ if len(dparts) == 1:
+ depend_id = ".".join((module_name, class_name, dparts[0]))
+ elif len(dparts) == 2:
+ depend_id = ".".join((module_name, dparts[0], dparts[1]))
+ else:
+ depend_id = depend
+
+ if not case_id in registry:
+ registry[case_id] = []
+ if not depend_id in registry[case_id]:
+ registry[case_id].append(depend_id)
+
+def _validate_test_case_depends(cases, depends):
+ for case in depends:
+ if not case in cases:
+ continue
+ for dep in depends[case]:
+ if not dep in cases:
+ raise OEQADependency("TestCase %s depends on %s and isn't available"\
+ ", cases available %s." % (case, dep, str(cases.keys())))
+
+def _order_test_case_by_depends(cases, depends):
+ def _dep_resolve(graph, node, resolved, seen):
+ seen.append(node)
+ for edge in graph[node]:
+ if edge not in resolved:
+ if edge in seen:
+ raise OEQADependency("Test cases %s and %s have a circular" \
+ " dependency." % (node, edge))
+ _dep_resolve(graph, edge, resolved, seen)
+ resolved.append(node)
+
+ dep_graph = {}
+ dep_graph['__root__'] = cases.keys()
+ for case in cases:
+ if case in depends:
+ dep_graph[case] = depends[case]
+ else:
+ dep_graph[case] = []
+
+ cases_ordered = []
+ _dep_resolve(dep_graph, '__root__', cases_ordered, [])
+ cases_ordered.remove('__root__')
+
+ return [cases[case_id] for case_id in cases_ordered]
+
+def _skipTestDependency(case, depends):
+ results = case.tc._results
+ skipReasons = ['errors', 'failures', 'skipped']
+
+ for reason in skipReasons:
+ for test, _ in results[reason]:
+ if test.id() in depends:
+ raise SkipTest("Test case %s depends on %s and was in %s." \
+ % (case.id(), test.id(), reason))
+
+@registerDecorator
+class OETestDepends(OETestDiscover):
+ attrs = ('depends',)
+
+ def bind(self, registry, case):
+ super(OETestDepends, self).bind(registry, case)
+ if not registry.get('depends'):
+ registry['depends'] = {}
+ _add_depends(registry['depends'], case, self.depends)
+
+ @staticmethod
+ def discover(registry):
+ if registry.get('depends'):
+ _validate_test_case_depends(registry['cases'], registry['depends'])
+ return _order_test_case_by_depends(registry['cases'], registry['depends'])
+ else:
+ return [registry['cases'][case_id] for case_id in registry['cases']]
+
+ def setUpDecorator(self):
+ _skipTestDependency(self.case, self.depends)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 05/55] oeqa/core/decorator: Add support for OETestID and OETestTag
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (3 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 04/55] oeqa/core/decorator: Add support for OETestDepends Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 06/55] oeqa/core/decorator: Add support for OETimeout decorator Aníbal Limón
` (51 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
These two decorators stores certain TAG or ID for the test case
also provides support for filtering in loading step.
[YOCTO #10236]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/decorator/oeid.py | 23 +++++++++++++++++++++++
meta/lib/oeqa/core/decorator/oetag.py | 24 ++++++++++++++++++++++++
2 files changed, 47 insertions(+)
create mode 100644 meta/lib/oeqa/core/decorator/oeid.py
create mode 100644 meta/lib/oeqa/core/decorator/oetag.py
diff --git a/meta/lib/oeqa/core/decorator/oeid.py b/meta/lib/oeqa/core/decorator/oeid.py
new file mode 100644
index 0000000..ea8017a
--- /dev/null
+++ b/meta/lib/oeqa/core/decorator/oeid.py
@@ -0,0 +1,23 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from . import OETestFilter, registerDecorator
+from oeqa.core.utils.misc import intToList
+
+def _idFilter(oeid, filters):
+ return False if oeid in filters else True
+
+@registerDecorator
+class OETestID(OETestFilter):
+ attrs = ('oeid',)
+
+ def bind(self, registry, case):
+ super(OETestID, self).bind(registry, case)
+
+ def filtrate(self, filters):
+ if filters.get('oeid'):
+ filterx = intToList(filters['oeid'], 'oeid')
+ del filters['oeid']
+ if _idFilter(self.oeid, filterx):
+ return True
+ return False
diff --git a/meta/lib/oeqa/core/decorator/oetag.py b/meta/lib/oeqa/core/decorator/oetag.py
new file mode 100644
index 0000000..ad38ab7
--- /dev/null
+++ b/meta/lib/oeqa/core/decorator/oetag.py
@@ -0,0 +1,24 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from . import OETestFilter, registerDecorator
+from oeqa.core.utils.misc import strToList
+
+def _tagFilter(tags, filters):
+ return False if set(tags) & set(filters) else True
+
+@registerDecorator
+class OETestTag(OETestFilter):
+ attrs = ('oetag',)
+
+ def bind(self, registry, case):
+ super(OETestTag, self).bind(registry, case)
+ self.oetag = strToList(self.oetag, 'oetag')
+
+ def filtrate(self, filters):
+ if filters.get('oetag'):
+ filterx = strToList(filters['oetag'], 'oetag')
+ del filters['oetag']
+ if _tagFilter(self.oetag, filterx):
+ return True
+ return False
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 06/55] oeqa/core/decorator: Add support for OETimeout decorator
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (4 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 05/55] oeqa/core/decorator: Add support for OETestID and OETestTag Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 07/55] oeqa/core/decorator: Add support for OETestDataDepends and skipIfDataVar Aníbal Limón
` (50 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
The OETimeout provides support for specify certain timeout
in seconds for a test case, if the timeout is reach the SIGALRM
is sent and an exception is raised to notify the timeout.
[YOCTO #10235]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/decorator/oetimeout.py | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
create mode 100644 meta/lib/oeqa/core/decorator/oetimeout.py
diff --git a/meta/lib/oeqa/core/decorator/oetimeout.py b/meta/lib/oeqa/core/decorator/oetimeout.py
new file mode 100644
index 0000000..a247583
--- /dev/null
+++ b/meta/lib/oeqa/core/decorator/oetimeout.py
@@ -0,0 +1,25 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import signal
+from . import OETestDecorator, registerDecorator
+from oeqa.core.exception import OEQATimeoutError
+
+@registerDecorator
+class OETimeout(OETestDecorator):
+ attrs = ('oetimeout',)
+
+ def setUpDecorator(self):
+ timeout = self.oetimeout
+ def _timeoutHandler(signum, frame):
+ raise OEQATimeoutError("Timed out after %s "
+ "seconds of execution" % timeout)
+
+ self.logger.debug("Setting up a %d second(s) timeout" % self.oetimeout)
+ self.alarmSignal = signal.signal(signal.SIGALRM, _timeoutHandler)
+ signal.alarm(self.oetimeout)
+
+ def tearDownDecorator(self):
+ signal.alarm(0)
+ signal.signal(signal.SIGALRM, self.alarmSignal)
+ self.logger.debug("Removed SIGALRM handler")
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 07/55] oeqa/core/decorator: Add support for OETestDataDepends and skipIfDataVar
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (5 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 06/55] oeqa/core/decorator: Add support for OETimeout decorator Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 08/55] oeqa/core: Add tests for the OEQA framework Aníbal Limón
` (49 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
The OETestDataDepends decorator skips a test case if a variable
isn't into test data (d).
The skipIfDataVar decorator skips a test case if a variable
has certain value.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/decorator/data.py | 36 ++++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
create mode 100644 meta/lib/oeqa/core/decorator/data.py
diff --git a/meta/lib/oeqa/core/decorator/data.py b/meta/lib/oeqa/core/decorator/data.py
new file mode 100644
index 0000000..73cca88
--- /dev/null
+++ b/meta/lib/oeqa/core/decorator/data.py
@@ -0,0 +1,36 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.exception import OEQAMissingVariable
+
+from . import OETestDecorator, registerDecorator
+
+@registerDecorator
+class skipIfDataVar(OETestDecorator):
+ """
+ Skip test based on value of a data store's variable.
+
+ It will get the info of var from the data store and will
+ check it against value; if are equal it will skip the test
+ with msg as the reason.
+ """
+
+ attrs = ('var', 'value', 'msg')
+
+ def setUpDecorator(self):
+ msg = 'Checking if %r value is %r to skip test' % (self.var, self.value)
+ self.logger.debug(msg)
+ if self.case.td.get(self.var) == self.value:
+ self.case.skipTest(self.msg)
+
+@registerDecorator
+class OETestDataDepends(OETestDecorator):
+ attrs = ('td_depends',)
+
+ def setUpDecorator(self):
+ for v in self.td_depends:
+ try:
+ value = self.case.td[v]
+ except KeyError:
+ raise OEQAMissingVariable("Test case need %s variable but"\
+ " isn't into td" % v)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 08/55] oeqa/core: Add tests for the OEQA framework
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (6 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 07/55] oeqa/core/decorator: Add support for OETestDataDepends and skipIfDataVar Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 09/55] scripts/oe-test: Add new oe-test script Aníbal Limón
` (48 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This test suite covers the current functionality for the OEQA
framework.
For run certain test suite,
$ cd meta/lib/oeqa/core/tests
$ ./test_data.py
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/tests/__init__.py | 0
meta/lib/oeqa/core/tests/cases/data.py | 20 +++
meta/lib/oeqa/core/tests/cases/depends.py | 38 ++++++
.../oeqa/core/tests/cases/loader/invalid/oeid.py | 15 +++
.../oeqa/core/tests/cases/loader/valid/another.py | 9 ++
meta/lib/oeqa/core/tests/cases/oeid.py | 18 +++
meta/lib/oeqa/core/tests/cases/oetag.py | 18 +++
meta/lib/oeqa/core/tests/cases/timeout.py | 18 +++
meta/lib/oeqa/core/tests/common.py | 35 ++++++
meta/lib/oeqa/core/tests/test_data.py | 51 ++++++++
meta/lib/oeqa/core/tests/test_decorators.py | 135 +++++++++++++++++++++
meta/lib/oeqa/core/tests/test_loader.py | 86 +++++++++++++
meta/lib/oeqa/core/tests/test_runner.py | 38 ++++++
13 files changed, 481 insertions(+)
create mode 100644 meta/lib/oeqa/core/tests/__init__.py
create mode 100644 meta/lib/oeqa/core/tests/cases/data.py
create mode 100644 meta/lib/oeqa/core/tests/cases/depends.py
create mode 100644 meta/lib/oeqa/core/tests/cases/loader/invalid/oeid.py
create mode 100644 meta/lib/oeqa/core/tests/cases/loader/valid/another.py
create mode 100644 meta/lib/oeqa/core/tests/cases/oeid.py
create mode 100644 meta/lib/oeqa/core/tests/cases/oetag.py
create mode 100644 meta/lib/oeqa/core/tests/cases/timeout.py
create mode 100644 meta/lib/oeqa/core/tests/common.py
create mode 100755 meta/lib/oeqa/core/tests/test_data.py
create mode 100755 meta/lib/oeqa/core/tests/test_decorators.py
create mode 100755 meta/lib/oeqa/core/tests/test_loader.py
create mode 100755 meta/lib/oeqa/core/tests/test_runner.py
diff --git a/meta/lib/oeqa/core/tests/__init__.py b/meta/lib/oeqa/core/tests/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/meta/lib/oeqa/core/tests/cases/data.py b/meta/lib/oeqa/core/tests/cases/data.py
new file mode 100644
index 0000000..88003a6
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/cases/data.py
@@ -0,0 +1,20 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.case import OETestCase
+from oeqa.core.decorator.oetag import OETestTag
+from oeqa.core.decorator.data import OETestDataDepends
+
+class DataTest(OETestCase):
+ data_vars = ['IMAGE', 'ARCH']
+
+ @OETestDataDepends(['MACHINE',])
+ @OETestTag('dataTestOk')
+ def testDataOk(self):
+ self.assertEqual(self.td.get('IMAGE'), 'core-image-minimal')
+ self.assertEqual(self.td.get('ARCH'), 'x86')
+ self.assertEqual(self.td.get('MACHINE'), 'qemuarm')
+
+ @OETestTag('dataTestFail')
+ def testDataFail(self):
+ pass
diff --git a/meta/lib/oeqa/core/tests/cases/depends.py b/meta/lib/oeqa/core/tests/cases/depends.py
new file mode 100644
index 0000000..17cdd90
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/cases/depends.py
@@ -0,0 +1,38 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.case import OETestCase
+from oeqa.core.decorator.depends import OETestDepends
+
+class DependsTest(OETestCase):
+
+ def testDependsFirst(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ @OETestDepends(['testDependsFirst'])
+ def testDependsSecond(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ @OETestDepends(['testDependsSecond'])
+ def testDependsThird(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ @OETestDepends(['testDependsSecond'])
+ def testDependsFourth(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ @OETestDepends(['testDependsThird', 'testDependsFourth'])
+ def testDependsFifth(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ @OETestDepends(['testDependsCircular3'])
+ def testDependsCircular1(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ @OETestDepends(['testDependsCircular1'])
+ def testDependsCircular2(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ @OETestDepends(['testDependsCircular2'])
+ def testDependsCircular3(self):
+ self.assertTrue(True, msg='How is this possible?')
diff --git a/meta/lib/oeqa/core/tests/cases/loader/invalid/oeid.py b/meta/lib/oeqa/core/tests/cases/loader/invalid/oeid.py
new file mode 100644
index 0000000..038d445
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/cases/loader/invalid/oeid.py
@@ -0,0 +1,15 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.case import OETestCase
+
+class AnotherIDTest(OETestCase):
+
+ def testAnotherIdGood(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ def testAnotherIdOther(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ def testAnotherIdNone(self):
+ self.assertTrue(True, msg='How is this possible?')
diff --git a/meta/lib/oeqa/core/tests/cases/loader/valid/another.py b/meta/lib/oeqa/core/tests/cases/loader/valid/another.py
new file mode 100644
index 0000000..c9ffd17
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/cases/loader/valid/another.py
@@ -0,0 +1,9 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.case import OETestCase
+
+class AnotherTest(OETestCase):
+
+ def testAnother(self):
+ self.assertTrue(True, msg='How is this possible?')
diff --git a/meta/lib/oeqa/core/tests/cases/oeid.py b/meta/lib/oeqa/core/tests/cases/oeid.py
new file mode 100644
index 0000000..c2d3d32
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/cases/oeid.py
@@ -0,0 +1,18 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.case import OETestCase
+from oeqa.core.decorator.oeid import OETestID
+
+class IDTest(OETestCase):
+
+ @OETestID(101)
+ def testIdGood(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ @OETestID(102)
+ def testIdOther(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ def testIdNone(self):
+ self.assertTrue(True, msg='How is this possible?')
diff --git a/meta/lib/oeqa/core/tests/cases/oetag.py b/meta/lib/oeqa/core/tests/cases/oetag.py
new file mode 100644
index 0000000..0cae02e
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/cases/oetag.py
@@ -0,0 +1,18 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.case import OETestCase
+from oeqa.core.decorator.oetag import OETestTag
+
+class TagTest(OETestCase):
+
+ @OETestTag('goodTag')
+ def testTagGood(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ @OETestTag('otherTag')
+ def testTagOther(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ def testTagNone(self):
+ self.assertTrue(True, msg='How is this possible?')
diff --git a/meta/lib/oeqa/core/tests/cases/timeout.py b/meta/lib/oeqa/core/tests/cases/timeout.py
new file mode 100644
index 0000000..870c3157
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/cases/timeout.py
@@ -0,0 +1,18 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from time import sleep
+
+from oeqa.core.case import OETestCase
+from oeqa.core.decorator.oetimeout import OETimeout
+
+class TimeoutTest(OETestCase):
+
+ @OETimeout(1)
+ def testTimeoutPass(self):
+ self.assertTrue(True, msg='How is this possible?')
+
+ @OETimeout(1)
+ def testTimeoutFail(self):
+ sleep(2)
+ self.assertTrue(True, msg='How is this possible?')
diff --git a/meta/lib/oeqa/core/tests/common.py b/meta/lib/oeqa/core/tests/common.py
new file mode 100644
index 0000000..52b18a1
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/common.py
@@ -0,0 +1,35 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import sys
+import os
+
+import unittest
+import logging
+import os
+
+logger = logging.getLogger("oeqa")
+logger.setLevel(logging.INFO)
+consoleHandler = logging.StreamHandler()
+formatter = logging.Formatter('OEQATest: %(message)s')
+consoleHandler.setFormatter(formatter)
+logger.addHandler(consoleHandler)
+
+def setup_sys_path():
+ directory = os.path.dirname(os.path.abspath(__file__))
+ oeqa_lib = os.path.realpath(os.path.join(directory, '../../../'))
+ if not oeqa_lib in sys.path:
+ sys.path.insert(0, oeqa_lib)
+
+class TestBase(unittest.TestCase):
+ def setUp(self):
+ self.logger = logger
+ directory = os.path.dirname(os.path.abspath(__file__))
+ self.cases_path = os.path.join(directory, 'cases')
+
+ def _testLoader(self, d={}, modules=[], tests=[], filters={}):
+ from oeqa.core.context import OETestContext
+ tc = OETestContext(d, self.logger)
+ tc.loadTests(self.cases_path, modules=modules, tests=tests,
+ filters=filters)
+ return tc
diff --git a/meta/lib/oeqa/core/tests/test_data.py b/meta/lib/oeqa/core/tests/test_data.py
new file mode 100755
index 0000000..320468c
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/test_data.py
@@ -0,0 +1,51 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import unittest
+import logging
+import os
+
+from common import setup_sys_path, TestBase
+setup_sys_path()
+
+from oeqa.core.exception import OEQAMissingVariable
+from oeqa.core.utils.test import getCaseMethod, getSuiteCasesNames
+
+class TestData(TestBase):
+ modules = ['data']
+
+ def test_data_fail_missing_variable(self):
+ expectedException = "oeqa.core.exception.OEQAMissingVariable"
+
+ tc = self._testLoader(modules=self.modules)
+ self.assertEqual(False, tc.runTests().wasSuccessful())
+ for test, data in tc._results['errors']:
+ expect = False
+ if expectedException in data:
+ expect = True
+
+ self.assertTrue(expect)
+
+ def test_data_fail_wrong_variable(self):
+ expectedError = 'AssertionError'
+ d = {'IMAGE' : 'core-image-sato', 'ARCH' : 'arm'}
+
+ tc = self._testLoader(d=d, modules=self.modules)
+ self.assertEqual(False, tc.runTests().wasSuccessful())
+ for test, data in tc._results['failures']:
+ expect = False
+ if expectedError in data:
+ expect = True
+
+ self.assertTrue(expect)
+
+ def test_data_ok(self):
+ d = {'IMAGE' : 'core-image-minimal', 'ARCH' : 'x86', 'MACHINE' : 'qemuarm'}
+
+ tc = self._testLoader(d=d, modules=self.modules)
+ self.assertEqual(True, tc.runTests().wasSuccessful())
+
+if __name__ == '__main__':
+ unittest.main()
diff --git a/meta/lib/oeqa/core/tests/test_decorators.py b/meta/lib/oeqa/core/tests/test_decorators.py
new file mode 100755
index 0000000..f7d11e8
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/test_decorators.py
@@ -0,0 +1,135 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import signal
+import unittest
+
+from common import setup_sys_path, TestBase
+setup_sys_path()
+
+from oeqa.core.exception import OEQADependency
+from oeqa.core.utils.test import getCaseMethod, getSuiteCasesNames, getSuiteCasesIDs
+
+class TestFilterDecorator(TestBase):
+
+ def _runFilterTest(self, modules, filters, expect, msg):
+ tc = self._testLoader(modules=modules, filters=filters)
+ test_loaded = set(getSuiteCasesNames(tc.suites))
+ self.assertEqual(expect, test_loaded, msg=msg)
+
+ def test_oetag(self):
+ # Get all cases without filtering.
+ filter_all = {}
+ test_all = {'testTagGood', 'testTagOther', 'testTagNone'}
+ msg_all = 'Failed to get all oetag cases without filtering.'
+
+ # Get cases with 'goodTag'.
+ filter_good = {'oetag':'goodTag'}
+ test_good = {'testTagGood'}
+ msg_good = 'Failed to get just one test filtering with "goodTag" oetag.'
+
+ # Get cases with an invalid tag.
+ filter_invalid = {'oetag':'invalidTag'}
+ test_invalid = set()
+ msg_invalid = 'Failed to filter all test using an invalid oetag.'
+
+ tests = ((filter_all, test_all, msg_all),
+ (filter_good, test_good, msg_good),
+ (filter_invalid, test_invalid, msg_invalid))
+
+ for test in tests:
+ self._runFilterTest(['oetag'], test[0], test[1], test[2])
+
+ def test_oeid(self):
+ # Get all cases without filtering.
+ filter_all = {}
+ test_all = {'testIdGood', 'testIdOther', 'testIdNone'}
+ msg_all = 'Failed to get all oeid cases without filtering.'
+
+ # Get cases with '101' oeid.
+ filter_good = {'oeid': 101}
+ test_good = {'testIdGood'}
+ msg_good = 'Failed to get just one tes filtering with "101" oeid.'
+
+ # Get cases with an invalid id.
+ filter_invalid = {'oeid':999}
+ test_invalid = set()
+ msg_invalid = 'Failed to filter all test using an invalid oeid.'
+
+ tests = ((filter_all, test_all, msg_all),
+ (filter_good, test_good, msg_good),
+ (filter_invalid, test_invalid, msg_invalid))
+
+ for test in tests:
+ self._runFilterTest(['oeid'], test[0], test[1], test[2])
+
+class TestDependsDecorator(TestBase):
+ modules = ['depends']
+
+ def test_depends_order(self):
+ tests = ['depends.DependsTest.testDependsFirst',
+ 'depends.DependsTest.testDependsSecond',
+ 'depends.DependsTest.testDependsThird',
+ 'depends.DependsTest.testDependsFourth',
+ 'depends.DependsTest.testDependsFifth']
+ tests2 = list(tests)
+ tests2[2], tests2[3] = tests[3], tests[2]
+ tc = self._testLoader(modules=self.modules, tests=tests)
+ test_loaded = getSuiteCasesIDs(tc.suites)
+ result = True if test_loaded == tests or test_loaded == tests2 else False
+ msg = 'Failed to order tests using OETestDepends decorator.\nTest order:'\
+ ' %s.\nExpected: %s\nOr: %s' % (test_loaded, tests, tests2)
+ self.assertTrue(result, msg=msg)
+
+ def test_depends_fail_missing_dependency(self):
+ expect = "TestCase depends.DependsTest.testDependsSecond depends on "\
+ "depends.DependsTest.testDependsFirst and isn't available"
+ tests = ['depends.DependsTest.testDependsSecond']
+ try:
+ # Must throw OEQADependency because missing 'testDependsFirst'
+ tc = self._testLoader(modules=self.modules, tests=tests)
+ self.fail('Expected OEQADependency exception')
+ except OEQADependency as e:
+ result = True if expect in str(e) else False
+ msg = 'Expected OEQADependency exception missing testDependsFirst test'
+ self.assertTrue(result, msg=msg)
+
+ def test_depends_fail_circular_dependency(self):
+ expect = 'have a circular dependency'
+ tests = ['depends.DependsTest.testDependsCircular1',
+ 'depends.DependsTest.testDependsCircular2',
+ 'depends.DependsTest.testDependsCircular3']
+ try:
+ # Must throw OEQADependency because circular dependency
+ tc = self._testLoader(modules=self.modules, tests=tests)
+ self.fail('Expected OEQADependency exception')
+ except OEQADependency as e:
+ result = True if expect in str(e) else False
+ msg = 'Expected OEQADependency exception having a circular dependency'
+ self.assertTrue(result, msg=msg)
+
+class TestTimeoutDecorator(TestBase):
+ modules = ['timeout']
+
+ def test_timeout(self):
+ tests = ['timeout.TimeoutTest.testTimeoutPass']
+ msg = 'Failed to run test using OETestTimeout'
+ alarm_signal = signal.getsignal(signal.SIGALRM)
+ tc = self._testLoader(modules=self.modules, tests=tests)
+ self.assertTrue(tc.runTests().wasSuccessful(), msg=msg)
+ msg = "OETestTimeout didn't restore SIGALRM"
+ self.assertIs(alarm_signal, signal.getsignal(signal.SIGALRM), msg=msg)
+
+ def test_timeout_fail(self):
+ tests = ['timeout.TimeoutTest.testTimeoutFail']
+ msg = "OETestTimeout test didn't timeout as expected"
+ alarm_signal = signal.getsignal(signal.SIGALRM)
+ tc = self._testLoader(modules=self.modules, tests=tests)
+ self.assertFalse(tc.runTests().wasSuccessful(), msg=msg)
+ msg = "OETestTimeout didn't restore SIGALRM"
+ self.assertIs(alarm_signal, signal.getsignal(signal.SIGALRM), msg=msg)
+
+if __name__ == '__main__':
+ unittest.main()
diff --git a/meta/lib/oeqa/core/tests/test_loader.py b/meta/lib/oeqa/core/tests/test_loader.py
new file mode 100755
index 0000000..b79b8ba
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/test_loader.py
@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import unittest
+
+from common import setup_sys_path, TestBase
+setup_sys_path()
+
+from oeqa.core.exception import OEQADependency
+from oeqa.core.utils.test import getSuiteModules, getSuiteCasesIDs
+
+class TestLoader(TestBase):
+
+ def test_fail_empty_filter(self):
+ filters = {'oetag' : ''}
+ expect = 'Filter oetag specified is empty'
+ msg = 'Expected TypeError exception for having invalid filter'
+ try:
+ # Must throw TypeError because empty filter
+ tc = self._testLoader(filters=filters)
+ self.fail(msg)
+ except TypeError as e:
+ result = True if expect in str(e) else False
+ self.assertTrue(result, msg=msg)
+
+ def test_fail_invalid_filter(self):
+ filters = {'invalid' : 'good'}
+ expect = 'filter but not declared in any of'
+ msg = 'Expected TypeError exception for having invalid filter'
+ try:
+ # Must throw TypeError because invalid filter
+ tc = self._testLoader(filters=filters)
+ self.fail(msg)
+ except TypeError as e:
+ result = True if expect in str(e) else False
+ self.assertTrue(result, msg=msg)
+
+ def test_fail_duplicated_module(self):
+ cases_path = self.cases_path
+ invalid_path = os.path.join(cases_path, 'loader', 'invalid')
+ self.cases_path = [self.cases_path, invalid_path]
+ expect = 'Duplicated oeid module found in'
+ msg = 'Expected ImportError exception for having duplicated module'
+ try:
+ # Must throw ImportEror because duplicated module
+ tc = self._testLoader()
+ self.fail(msg)
+ except ImportError as e:
+ result = True if expect in str(e) else False
+ self.assertTrue(result, msg=msg)
+ finally:
+ self.cases_path = cases_path
+
+ def test_filter_modules(self):
+ expected_modules = {'oeid', 'oetag'}
+ tc = self._testLoader(modules=expected_modules)
+ modules = getSuiteModules(tc.suites)
+ msg = 'Expected just %s modules' % ', '.join(expected_modules)
+ self.assertEqual(modules, expected_modules, msg=msg)
+
+ def test_filter_cases(self):
+ modules = ['oeid', 'oetag', 'data']
+ expected_cases = {'data.DataTest.testDataOk',
+ 'oetag.TagTest.testTagGood',
+ 'oeid.IDTest.testIdGood'}
+ tc = self._testLoader(modules=modules, tests=expected_cases)
+ cases = set(getSuiteCasesIDs(tc.suites))
+ msg = 'Expected just %s cases' % ', '.join(expected_cases)
+ self.assertEqual(cases, expected_cases, msg=msg)
+
+ def test_import_from_paths(self):
+ cases_path = self.cases_path
+ cases2_path = os.path.join(cases_path, 'loader', 'valid')
+ expected_modules = {'oeid', 'another'}
+ self.cases_path = [self.cases_path, cases2_path]
+ tc = self._testLoader(modules=expected_modules)
+ modules = getSuiteModules(tc.suites)
+ self.cases_path = cases_path
+ msg = 'Expected modules from two different paths'
+ self.assertEqual(modules, expected_modules, msg=msg)
+
+if __name__ == '__main__':
+ unittest.main()
diff --git a/meta/lib/oeqa/core/tests/test_runner.py b/meta/lib/oeqa/core/tests/test_runner.py
new file mode 100755
index 0000000..a3f3861
--- /dev/null
+++ b/meta/lib/oeqa/core/tests/test_runner.py
@@ -0,0 +1,38 @@
+#!/usr/bin/env python3
+
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import unittest
+import logging
+import tempfile
+
+from common import setup_sys_path, TestBase
+setup_sys_path()
+
+from oeqa.core.runner import OEStreamLogger
+
+class TestRunner(TestBase):
+ def test_stream_logger(self):
+ fp = tempfile.TemporaryFile(mode='w+')
+
+ logging.basicConfig(format='%(message)s', stream=fp)
+ logger = logging.getLogger()
+ logger.setLevel(logging.INFO)
+
+ oeSL = OEStreamLogger(logger)
+
+ lines = ['init', 'bigline_' * 65535, 'morebigline_' * 65535 * 4, 'end']
+ for line in lines:
+ oeSL.write(line)
+
+ fp.seek(0)
+ fp_lines = fp.readlines()
+ for i, fp_line in enumerate(fp_lines):
+ fp_line = fp_line.strip()
+ self.assertEqual(lines[i], fp_line)
+
+ fp.close()
+
+if __name__ == '__main__':
+ unittest.main()
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 09/55] scripts/oe-test: Add new oe-test script
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (7 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 08/55] oeqa/core: Add tests for the OEQA framework Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 10/55] oeqa/core/context: Add support of OETestContextExecutor Aníbal Limón
` (47 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
The new oe-test script will be use to run test components with
one single script.
The oe-test script search for test components inside meta/lib/oeqa,
the test components needs to implement OETestContextExecutor inside
context module in order to be supported by oe-test.
[YOCTO #10230]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
scripts/oe-test | 99 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 99 insertions(+)
create mode 100755 scripts/oe-test
diff --git a/scripts/oe-test b/scripts/oe-test
new file mode 100755
index 0000000..5731dff
--- /dev/null
+++ b/scripts/oe-test
@@ -0,0 +1,99 @@
+#!/usr/bin/env python3
+
+# OpenEmbedded test tool
+#
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import sys
+import argparse
+import importlib
+import logging
+
+scripts_path = os.path.dirname(os.path.realpath(__file__))
+lib_path = scripts_path + '/lib'
+sys.path = sys.path + [lib_path]
+import argparse_oe
+import scriptutils
+import scriptpath
+scriptpath.add_oe_lib_path()
+
+from oeqa.core.context import OETestContextExecutor
+
+logger = scriptutils.logger_create('oe-test')
+
+def _load_test_components(logger):
+ components = {}
+
+ for path in sys.path:
+ base_dir = os.path.join(path, 'oeqa')
+ if os.path.exists(base_dir) and os.path.isdir(base_dir):
+ for file in os.listdir(base_dir):
+ comp_name = file
+ comp_context = os.path.join(base_dir, file, 'context.py')
+ if os.path.exists(comp_context):
+ comp_plugin = importlib.import_module('oeqa.%s.%s' % \
+ (comp_name, 'context'))
+ try:
+ if not issubclass(comp_plugin._executor_class,
+ OETestContextExecutor):
+ raise TypeError("Component %s in %s, _executor_class "\
+ "isn't derived from OETestContextExecutor."\
+ % (comp_name, comp_context))
+
+ components[comp_name] = comp_plugin._executor_class()
+ except AttributeError:
+ raise AttributeError("Component %s in %s don't have "\
+ "_executor_class defined." % (comp_name, comp_context))
+
+ return components
+
+def main():
+ parser = argparse_oe.ArgumentParser(description="OpenEmbedded test tool",
+ add_help=False,
+ epilog="Use %(prog)s <subcommand> --help to get help on a specific command")
+ parser.add_argument('-d', '--debug', help='Enable debug output', action='store_true')
+ parser.add_argument('-q', '--quiet', help='Print only errors', action='store_true')
+ global_args, unparsed_args = parser.parse_known_args()
+
+ # Help is added here rather than via add_help=True, as we don't want it to
+ # be handled by parse_known_args()
+ parser.add_argument('-h', '--help', action='help', default=argparse.SUPPRESS,
+ help='show this help message and exit')
+
+ if global_args.debug:
+ logger.setLevel(logging.DEBUG)
+ elif global_args.quiet:
+ logger.setLevel(logging.ERROR)
+
+ components = _load_test_components(logger)
+
+ subparsers = parser.add_subparsers(dest="subparser_name", title='subcommands', metavar='<subcommand>')
+ subparsers.add_subparser_group('components', 'Test components')
+ subparsers.required = True
+ for comp_name in sorted(components.keys()):
+ comp = components[comp_name]
+ comp.register_commands(logger, subparsers)
+
+ try:
+ args = parser.parse_args(unparsed_args, namespace=global_args)
+ results = args.func(logger, args)
+ ret = 0 if results.wasSuccessful() else 1
+ except SystemExit as err:
+ if err.code != 0:
+ raise err
+ ret = err.code
+ except argparse_oe.ArgumentUsageError as ae:
+ parser.error_subcommand(ae.message, ae.subcommand)
+
+ return ret
+
+if __name__ == '__main__':
+ try:
+ ret = main()
+ except Exception:
+ ret = 1
+ import traceback
+ traceback.print_exc()
+ sys.exit(ret)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 10/55] oeqa/core/context: Add support of OETestContextExecutor
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (8 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 09/55] scripts/oe-test: Add new oe-test script Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 11/55] oeqa/core/cases: Add example test cases Aníbal Limón
` (46 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
The OETestContextExecutor class supports to use oe-test for run core
test component also is a base class for the other test components
(runtime, sdk, selftest).
Te principal functionality is to support cmdline parsing and execution
of OETestContext, the test components could extend the common options
to provide specific ones. The common options between test components
are test data file, output log and test cases path's to scan.
Also it initializes the logger to be passed to the whole OEQA framework.
[YOCTO #10230]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/context.py | 85 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 85 insertions(+)
diff --git a/meta/lib/oeqa/core/context.py b/meta/lib/oeqa/core/context.py
index cbab2f8..ba6ccf8 100644
--- a/meta/lib/oeqa/core/context.py
+++ b/meta/lib/oeqa/core/context.py
@@ -146,3 +146,88 @@ class OETestContext(object):
else:
self.logger.info("RESULTS - %s - Testcase %s: %s" % (case.id(),
oeid, 'PASSED'))
+
+class OETestContextExecutor(object):
+ _context_class = OETestContext
+
+ name = 'core'
+ help = 'core test component example'
+ description = 'executes core test suite example'
+
+ default_cases = [os.path.join(os.path.abspath(os.path.dirname(__file__)),
+ 'cases/example')]
+ default_test_data = os.path.join(default_cases[0], 'data.json')
+
+ def register_commands(self, logger, subparsers):
+ self.parser = subparsers.add_parser(self.name, help=self.help,
+ description=self.description, group='components')
+
+ self.default_output_log = '%s-results-%s.log' % (self.name,
+ time.strftime("%Y%m%d%H%M%S"))
+ self.parser.add_argument('--output-log', action='store',
+ default=self.default_output_log,
+ help="results output log, default: %s" % self.default_output_log)
+
+ if self.default_test_data:
+ self.parser.add_argument('--test-data-file', action='store',
+ default=self.default_test_data,
+ help="data file to load, default: %s" % self.default_test_data)
+ else:
+ self.parser.add_argument('--test-data-file', action='store',
+ help="data file to load")
+
+ if self.default_cases:
+ self.parser.add_argument('CASES_PATHS', action='store',
+ default=self.default_cases, nargs='*',
+ help="paths to directories with test cases, default: %s"\
+ % self.default_cases)
+ else:
+ self.parser.add_argument('CASES_PATHS', action='store',
+ nargs='+', help="paths to directories with test cases")
+
+ self.parser.set_defaults(func=self.run)
+
+ def _setup_logger(self, logger, args):
+ formatter = logging.Formatter('%(asctime)s - ' + self.name + \
+ ' - %(levelname)s - %(message)s')
+ sh = logger.handlers[0]
+ sh.setFormatter(formatter)
+ fh = logging.FileHandler(args.output_log)
+ fh.setFormatter(formatter)
+ logger.addHandler(fh)
+
+ return logger
+
+ def _process_args(self, logger, args):
+ self.tc_kwargs = {}
+ self.tc_kwargs['init'] = {}
+ self.tc_kwargs['load'] = {}
+ self.tc_kwargs['run'] = {}
+
+ self.tc_kwargs['init']['logger'] = self._setup_logger(logger, args)
+ if args.test_data_file:
+ self.tc_kwargs['init']['td'] = json.load(
+ open(args.test_data_file, "r"))
+ else:
+ self.tc_kwargs['init']['td'] = {}
+
+ self.module_paths = args.CASES_PATHS
+
+ def run(self, logger, args):
+ self._process_args(logger, args)
+
+ self.tc = self._context_class(**self.tc_kwargs['init'])
+ self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
+ rc = self.tc.runTests(**self.tc_kwargs['run'])
+ self.tc.logSummary(rc, self.name)
+ self.tc.logDetails()
+
+ output_link = os.path.join(os.path.dirname(args.output_log),
+ "%s-results.log" % self.name)
+ if os.path.exists(output_link):
+ os.remove(output_link)
+ os.symlink(args.output_log, output_link)
+
+ return rc
+
+_executor_class = OETestContextExecutor
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 11/55] oeqa/core/cases: Add example test cases
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (9 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 10/55] oeqa/core/context: Add support of OETestContextExecutor Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 12/55] oeqa/core: Add README Aníbal Limón
` (45 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
Serves as an first input of how to the OEQA framework works.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/cases/__init__.py | 0
meta/lib/oeqa/core/cases/example/data.json | 1 +
meta/lib/oeqa/core/cases/example/test_basic.py | 20 ++++++++++++++++++++
3 files changed, 21 insertions(+)
create mode 100644 meta/lib/oeqa/core/cases/__init__.py
create mode 100644 meta/lib/oeqa/core/cases/example/data.json
create mode 100644 meta/lib/oeqa/core/cases/example/test_basic.py
diff --git a/meta/lib/oeqa/core/cases/__init__.py b/meta/lib/oeqa/core/cases/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/meta/lib/oeqa/core/cases/example/data.json b/meta/lib/oeqa/core/cases/example/data.json
new file mode 100644
index 0000000..21d6b16
--- /dev/null
+++ b/meta/lib/oeqa/core/cases/example/data.json
@@ -0,0 +1 @@
+{"ARCH": "x86", "IMAGE": "core-image-minimal"}
\ No newline at end of file
diff --git a/meta/lib/oeqa/core/cases/example/test_basic.py b/meta/lib/oeqa/core/cases/example/test_basic.py
new file mode 100644
index 0000000..11cf380
--- /dev/null
+++ b/meta/lib/oeqa/core/cases/example/test_basic.py
@@ -0,0 +1,20 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.case import OETestCase
+from oeqa.core.decorator.depends import OETestDepends
+
+class OETestExample(OETestCase):
+ def test_example(self):
+ self.logger.info('IMAGE: %s' % self.td.get('IMAGE'))
+ self.assertEqual('core-image-minimal', self.td.get('IMAGE'))
+ self.logger.info('ARCH: %s' % self.td.get('ARCH'))
+ self.assertEqual('x86', self.td.get('ARCH'))
+
+class OETestExampleDepend(OETestCase):
+ @OETestDepends(['OETestExample.test_example'])
+ def test_example_depends(self):
+ pass
+
+ def test_example_no_depends(self):
+ pass
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 12/55] oeqa/core: Add README
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (10 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 11/55] oeqa/core/cases: Add example test cases Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 13/55] oe/data: Add export2json function Aníbal Limón
` (44 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
The README has an introduction and explains how to run the test suite
and creates a new Test component.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/README | 38 ++++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)
create mode 100644 meta/lib/oeqa/core/README
diff --git a/meta/lib/oeqa/core/README b/meta/lib/oeqa/core/README
new file mode 100644
index 0000000..0c859fd
--- /dev/null
+++ b/meta/lib/oeqa/core/README
@@ -0,0 +1,38 @@
+= OEQA Framework =
+
+== Introduction ==
+
+This is the new OEQA framework the base clases of the framework
+are in this module oeqa/core the subsequent components needs to
+extend this classes.
+
+A new/unique runner was created called oe-test and is under scripts/
+oe-test, this new runner scans over oeqa module searching for test
+components that supports OETestContextExecutor implemented in context
+module (i.e. oeqa/core/context.py).
+
+For execute an example:
+
+$ source oe-init-build-env
+$ oe-test core
+
+For list supported components:
+
+$ oe-test -h
+
+== Create new Test component ==
+
+Usally for add a new Test component the developer needs to extend
+OETestContext/OETestContextExecutor in context.py and OETestCase in
+case.py.
+
+== How to run the testing of the OEQA framework ==
+
+Run all tests:
+
+$ PATH=$PATH:../../ python3 -m unittest discover -s tests
+
+Run some test:
+
+$ cd tests/
+$ ./test_data.py
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 13/55] oe/data: Add export2json function
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (11 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 12/55] oeqa/core: Add README Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 14/55] classes/rootfs-postcommands: Add write_image_test_data Aníbal Limón
` (43 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
The export2json function export the variables contained in
the data store to JSON format, the main usage for now will be
to provide test data to QA framework.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oe/data.py | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/meta/lib/oe/data.py b/meta/lib/oe/data.py
index 4a67c45..3956733 100644
--- a/meta/lib/oe/data.py
+++ b/meta/lib/oe/data.py
@@ -1,3 +1,4 @@
+import json
import oe.maketype
def typed_value(key, d):
@@ -15,3 +16,30 @@ def typed_value(key, d):
return oe.maketype.create(d.getVar(key) or '', var_type, **flags)
except (TypeError, ValueError) as exc:
bb.msg.fatal("Data", "%s: %s" % (key, str(exc)))
+
+def export2json(d, json_file, expand=True):
+ data2export = {}
+ keys2export = []
+
+ for key in d.keys():
+ if key.startswith("_"):
+ continue
+ elif key.startswith("BB"):
+ continue
+ elif key.startswith("B_pn"):
+ continue
+ elif key.startswith("do_"):
+ continue
+ elif d.getVarFlag(key, "func", True):
+ continue
+
+ keys2export.append(key)
+
+ for key in keys2export:
+ try:
+ data2export[key] = d.getVar(key, expand)
+ except bb.data_smart.ExpansionError:
+ data2export[key] = ''
+
+ with open(json_file, "w") as f:
+ json.dump(data2export, f, skipkeys=True, indent=4, sort_keys=True)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 14/55] classes/rootfs-postcommands: Add write_image_test_data
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (12 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 13/55] oe/data: Add export2json function Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 15/55] classes/populate_sdk_base: Add write_sdk_test_data to postprocess Aníbal Limón
` (42 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
This function will generates testdata.json by image type.
[YOCTO #10231]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/classes/rootfs-postcommands.bbclass | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/meta/classes/rootfs-postcommands.bbclass b/meta/classes/rootfs-postcommands.bbclass
index 53a4fda..c8f18c6 100644
--- a/meta/classes/rootfs-postcommands.bbclass
+++ b/meta/classes/rootfs-postcommands.bbclass
@@ -14,6 +14,9 @@ ROOTFS_POSTPROCESS_COMMAND += "rootfs_update_timestamp ; "
# Tweak the mount options for rootfs in /etc/fstab if read-only-rootfs is enabled
ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs", "read_only_rootfs_hook; ", "",d)}'
+# Generates test data file with data store variables expanded in json format
+ROOTFS_POSTPROCESS_COMMAND += "write_image_test_data ; "
+
# Write manifest
IMAGE_MANIFEST = "${IMGDEPLOYDIR}/${IMAGE_NAME}.rootfs.manifest"
ROOTFS_POSTUNINSTALL_COMMAND =+ "write_image_manifest ; "
@@ -300,3 +303,18 @@ rootfs_check_host_user_contaminated () {
rootfs_sysroot_relativelinks () {
sysroot-relativelinks.py ${SDK_OUTPUT}/${SDKTARGETSYSROOT}
}
+
+# Generated test data json file
+python write_image_test_data() {
+ from oe.data import export2json
+
+ testdata = "%s/%s.testdata.json" % (d.getVar('DEPLOY_DIR_IMAGE', True), d.getVar('IMAGE_NAME', True))
+ testdata_link = "%s/%s.testdata.json" % (d.getVar('DEPLOY_DIR_IMAGE', True), d.getVar('IMAGE_LINK_NAME', True))
+
+ bb.utils.mkdirhier(os.path.dirname(testdata))
+ export2json(d, testdata)
+
+ if os.path.lexists(testdata_link):
+ os.remove(testdata_link)
+ os.symlink(os.path.basename(testdata), testdata_link)
+}
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 15/55] classes/populate_sdk_base: Add write_sdk_test_data to postprocess
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (13 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 14/55] classes/rootfs-postcommands: Add write_image_test_data Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 16/55] oeqa/utils/__init__.py: Adds compatibility with bitbake logger Aníbal Limón
` (41 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
This function will generates testdata.json per SDK type.
[YOCTO #10231]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/classes/populate_sdk_base.bbclass | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index 8d117a9..5ad08c6 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -70,6 +70,13 @@ python write_target_sdk_manifest () {
output.write(format_pkg_list(pkgs, 'ver'))
}
+python write_sdk_test_data() {
+ from oe.data import export2json
+ testdata = "%s/%s.testdata.json" % (d.getVar('SDKDEPLOYDIR', True), d.getVar('TOOLCHAIN_OUTPUTNAME', True))
+ bb.utils.mkdirhier(os.path.dirname(testdata))
+ export2json(d, testdata)
+}
+
python write_host_sdk_manifest () {
from oe.sdk import sdk_list_installed_packages
from oe.utils import format_pkg_list
@@ -81,7 +88,7 @@ python write_host_sdk_manifest () {
output.write(format_pkg_list(pkgs, 'ver'))
}
-POPULATE_SDK_POST_TARGET_COMMAND_append = " write_target_sdk_manifest ; "
+POPULATE_SDK_POST_TARGET_COMMAND_append = " write_target_sdk_manifest ; write_sdk_test_data ; "
POPULATE_SDK_POST_HOST_COMMAND_append = " write_host_sdk_manifest; "
SDK_PACKAGING_COMMAND = "${@'${SDK_PACKAGING_FUNC};' if '${SDK_PACKAGING_FUNC}' else ''}"
SDK_POSTPROCESS_COMMAND = " create_sdk_files; check_sdk_sysroots; tar_sdk; ${SDK_PACKAGING_COMMAND} "
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 16/55] oeqa/utils/__init__.py: Adds compatibility with bitbake logger
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (14 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 15/55] classes/populate_sdk_base: Add write_sdk_test_data to postprocess Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 17/55] oeqa: Move common files to oeqa/files instead of runtime only Aníbal Limón
` (40 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
The bitbake logger changes the way debug is logged and adds
different levels within debug, this is passed as argument
to the function and breaks compatibility with vanilla loggers.
This implements a way to handle this adding a new function for
debug, that will dispatch the correct logging method signature.
Also overrides info method to use logging.INFO + 1 in order to
see plain data.
Also this commit fix the issue of not showing the test summary
and results when running from bitbake.
[YOCTO #10686]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/utils/__init__.py | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
diff --git a/meta/lib/oeqa/utils/__init__.py b/meta/lib/oeqa/utils/__init__.py
index 8f706f3..485de03 100644
--- a/meta/lib/oeqa/utils/__init__.py
+++ b/meta/lib/oeqa/utils/__init__.py
@@ -36,3 +36,33 @@ def avoid_paths_in_environ(paths):
new_path = new_path[:-1]
return new_path
+
+def make_logger_bitbake_compatible(logger):
+ import logging
+
+ """
+ Bitbake logger redifines debug() in order to
+ set a level within debug, this breaks compatibility
+ with vainilla logging, so we neeed to redifine debug()
+ method again also add info() method with INFO + 1 level.
+ """
+ def _bitbake_log_debug(*args, **kwargs):
+ lvl = logging.DEBUG
+
+ if isinstance(args[0], int):
+ lvl = args[0]
+ msg = args[1]
+ args = args[2:]
+ else:
+ msg = args[0]
+ args = args[1:]
+
+ logger.log(lvl, msg, *args, **kwargs)
+
+ def _bitbake_log_info(msg, *args, **kwargs):
+ logger.log(logging.INFO + 1, msg, *args, **kwargs)
+
+ logger.debug = _bitbake_log_debug
+ logger.info = _bitbake_log_info
+
+ return logger
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 17/55] oeqa: Move common files to oeqa/files instead of runtime only
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (15 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 16/55] oeqa/utils/__init__.py: Adds compatibility with bitbake logger Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 18/55] oeqa/sdk: Move test cases inside cases directory Aníbal Limón
` (39 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
Those files are used by runtime and sdk test cases, so move to
base directory of oeqa module.
[YOCTO #10599]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/{runtime => }/files/test.c | 0
meta/lib/oeqa/{runtime => }/files/test.cpp | 0
meta/lib/oeqa/{runtime => }/files/test.pl | 0
meta/lib/oeqa/{runtime => }/files/test.py | 0
4 files changed, 0 insertions(+), 0 deletions(-)
rename meta/lib/oeqa/{runtime => }/files/test.c (100%)
rename meta/lib/oeqa/{runtime => }/files/test.cpp (100%)
rename meta/lib/oeqa/{runtime => }/files/test.pl (100%)
rename meta/lib/oeqa/{runtime => }/files/test.py (100%)
diff --git a/meta/lib/oeqa/runtime/files/test.c b/meta/lib/oeqa/files/test.c
similarity index 100%
rename from meta/lib/oeqa/runtime/files/test.c
rename to meta/lib/oeqa/files/test.c
diff --git a/meta/lib/oeqa/runtime/files/test.cpp b/meta/lib/oeqa/files/test.cpp
similarity index 100%
rename from meta/lib/oeqa/runtime/files/test.cpp
rename to meta/lib/oeqa/files/test.cpp
diff --git a/meta/lib/oeqa/runtime/files/test.pl b/meta/lib/oeqa/files/test.pl
similarity index 100%
rename from meta/lib/oeqa/runtime/files/test.pl
rename to meta/lib/oeqa/files/test.pl
diff --git a/meta/lib/oeqa/runtime/files/test.py b/meta/lib/oeqa/files/test.py
similarity index 100%
rename from meta/lib/oeqa/runtime/files/test.py
rename to meta/lib/oeqa/files/test.py
--
2.1.4
^ permalink raw reply [flat|nested] 59+ messages in thread* [PATCH 18/55] oeqa/sdk: Move test cases inside cases directory
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (16 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 17/55] oeqa: Move common files to oeqa/files instead of runtime only Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 19/55] oeqa/{runtime, sdk}/files: Move testsdkmakefile from runtime to sdk module Aníbal Limón
` (38 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
For match with the new structure of the OEQA framework.
In the new framework Test component base directory in this case
sdk module will contain case and context implementations.
[YOCTO #10599]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/sdk/{ => cases}/__init__.py | 0
meta/lib/oeqa/sdk/{ => cases}/buildcvs.py | 0
meta/lib/oeqa/sdk/{ => cases}/buildgalculator.py | 0
meta/lib/oeqa/sdk/{ => cases}/buildiptables.py | 0
meta/lib/oeqa/sdk/{ => cases}/gcc.py | 0
meta/lib/oeqa/sdk/{ => cases}/perl.py | 0
meta/lib/oeqa/sdk/{ => cases}/python.py | 0
7 files changed, 0 insertions(+), 0 deletions(-)
rename meta/lib/oeqa/sdk/{ => cases}/__init__.py (100%)
rename meta/lib/oeqa/sdk/{ => cases}/buildcvs.py (100%)
rename meta/lib/oeqa/sdk/{ => cases}/buildgalculator.py (100%)
rename meta/lib/oeqa/sdk/{ => cases}/buildiptables.py (100%)
rename meta/lib/oeqa/sdk/{ => cases}/gcc.py (100%)
rename meta/lib/oeqa/sdk/{ => cases}/perl.py (100%)
rename meta/lib/oeqa/sdk/{ => cases}/python.py (100%)
diff --git a/meta/lib/oeqa/sdk/__init__.py b/meta/lib/oeqa/sdk/cases/__init__.py
similarity index 100%
rename from meta/lib/oeqa/sdk/__init__.py
rename to meta/lib/oeqa/sdk/cases/__init__.py
diff --git a/meta/lib/oeqa/sdk/buildcvs.py b/meta/lib/oeqa/sdk/cases/buildcvs.py
similarity index 100%
rename from meta/lib/oeqa/sdk/buildcvs.py
rename to meta/lib/oeqa/sdk/cases/buildcvs.py
diff --git a/meta/lib/oeqa/sdk/buildgalculator.py b/meta/lib/oeqa/sdk/cases/buildgalculator.py
similarity index 100%
rename from meta/lib/oeqa/sdk/buildgalculator.py
rename to meta/lib/oeqa/sdk/cases/buildgalculator.py
diff --git a/meta/lib/oeqa/sdk/buildiptables.py b/meta/lib/oeqa/sdk/cases/buildiptables.py
similarity index 100%
rename from meta/lib/oeqa/sdk/buildiptables.py
rename to meta/lib/oeqa/sdk/cases/buildiptables.py
diff --git a/meta/lib/oeqa/sdk/gcc.py b/meta/lib/oeqa/sdk/cases/gcc.py
similarity index 100%
rename from meta/lib/oeqa/sdk/gcc.py
rename to meta/lib/oeqa/sdk/cases/gcc.py
diff --git a/meta/lib/oeqa/sdk/perl.py b/meta/lib/oeqa/sdk/cases/perl.py
similarity index 100%
rename from meta/lib/oeqa/sdk/perl.py
rename to meta/lib/oeqa/sdk/cases/perl.py
diff --git a/meta/lib/oeqa/sdk/python.py b/meta/lib/oeqa/sdk/cases/python.py
similarity index 100%
rename from meta/lib/oeqa/sdk/python.py
rename to meta/lib/oeqa/sdk/cases/python.py
--
2.1.4
^ permalink raw reply [flat|nested] 59+ messages in thread* [PATCH 19/55] oeqa/{runtime, sdk}/files: Move testsdkmakefile from runtime to sdk module
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (17 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 18/55] oeqa/sdk: Move test cases inside cases directory Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 20/55] oeqa/sdk: Add case and context modules for the SDK component Aníbal Limón
` (37 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
It doesn't make sense to have files related to sdk module into runtime
module.
[YOCTO #10599]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/{runtime => sdk}/files/testsdkmakefile | 0
1 file changed, 0 insertions(+), 0 deletions(-)
rename meta/lib/oeqa/{runtime => sdk}/files/testsdkmakefile (100%)
diff --git a/meta/lib/oeqa/runtime/files/testsdkmakefile b/meta/lib/oeqa/sdk/files/testsdkmakefile
similarity index 100%
rename from meta/lib/oeqa/runtime/files/testsdkmakefile
rename to meta/lib/oeqa/sdk/files/testsdkmakefile
--
2.1.4
^ permalink raw reply [flat|nested] 59+ messages in thread* [PATCH 20/55] oeqa/sdk: Add case and context modules for the SDK component
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (18 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 19/55] oeqa/{runtime, sdk}/files: Move testsdkmakefile from runtime to sdk module Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 21/55] classes/testsdk: Migrates testsdk.bbclass to use new OESDKTestContext Aníbal Limón
` (36 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
Adds case and context modules for SDK based on oetest.py old code.
Enables SDK Test component usage with oe-test, the SDK Test component
adds command line options for specify sdk installed dir, sdk environment
and target/hosts maniftest.
[YOCTO #10599]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/oetest.py | 51 ----------------
meta/lib/oeqa/sdk/__init__.py | 0
meta/lib/oeqa/sdk/case.py | 12 ++++
meta/lib/oeqa/sdk/context.py | 133 ++++++++++++++++++++++++++++++++++++++++++
4 files changed, 145 insertions(+), 51 deletions(-)
create mode 100644 meta/lib/oeqa/sdk/__init__.py
create mode 100644 meta/lib/oeqa/sdk/case.py
create mode 100644 meta/lib/oeqa/sdk/context.py
diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index d1aef96..d7c3588 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -145,20 +145,6 @@ class OETestCalledProcessError(subprocess.CalledProcessError):
subprocess.CalledProcessError = OETestCalledProcessError
-class oeSDKTest(oeTest):
- def __init__(self, methodName='runTest'):
- self.sdktestdir = oeSDKTest.tc.sdktestdir
- super(oeSDKTest, self).__init__(methodName)
-
- @classmethod
- def hasHostPackage(self, pkg):
- if re.search(pkg, oeTest.tc.hostpkgmanifest):
- return True
- return False
-
- def _run(self, cmd):
- return subprocess.check_output(". %s > /dev/null; %s;" % (self.tc.sdkenv, cmd), shell=True, stderr=subprocess.STDOUT).decode("utf-8")
-
class oeSDKExtTest(oeSDKTest):
def _run(self, cmd):
# extensible sdk shows a warning if found bitbake in the path
@@ -657,43 +643,6 @@ class ExportTestContext(RuntimeTestContext):
pkg_dir = os.path.join(export_dir, extracted_dir)
super(ExportTestContext, self).install_uninstall_packages(test_id, pkg_dir, install)
-class SDKTestContext(TestContext):
- def __init__(self, d, sdktestdir, sdkenv, tcname, *args):
- super(SDKTestContext, self).__init__(d)
-
- self.sdktestdir = sdktestdir
- self.sdkenv = sdkenv
- self.tcname = tcname
-
- if not hasattr(self, 'target_manifest'):
- self.target_manifest = d.getVar("SDK_TARGET_MANIFEST")
- try:
- self.pkgmanifest = {}
- with open(self.target_manifest) as f:
- for line in f:
- (pkg, arch, version) = line.strip().split()
- self.pkgmanifest[pkg] = (version, arch)
- except IOError as e:
- bb.fatal("No package manifest file found. Did you build the sdk image?\n%s" % e)
-
- if not hasattr(self, 'host_manifest'):
- self.host_manifest = d.getVar("SDK_HOST_MANIFEST")
- try:
- with open(self.host_manifest) as f:
- self.hostpkgmanifest = f.read()
- except IOError as e:
- bb.fatal("No host package manifest file found. Did you build the sdk image?\n%s" % e)
-
- def _get_test_namespace(self):
- return "sdk"
-
- def _get_test_suites(self):
- return (self.d.getVar("TEST_SUITES_SDK") or "auto").split()
-
- def _get_test_suites_required(self):
- return [t for t in (self.d.getVar("TEST_SUITES_SDK") or \
- "auto").split() if t != "auto"]
-
class SDKExtTestContext(SDKTestContext):
def __init__(self, d, sdktestdir, sdkenv, tcname, *args):
self.target_manifest = d.getVar("SDK_EXT_TARGET_MANIFEST")
diff --git a/meta/lib/oeqa/sdk/__init__.py b/meta/lib/oeqa/sdk/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/meta/lib/oeqa/sdk/case.py b/meta/lib/oeqa/sdk/case.py
new file mode 100644
index 0000000..782db8b
--- /dev/null
+++ b/meta/lib/oeqa/sdk/case.py
@@ -0,0 +1,12 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import subprocess
+
+from oeqa.core.case import OETestCase
+
+class OESDKTestCase(OETestCase):
+ def _run(self, cmd):
+ return subprocess.check_output(". %s > /dev/null; %s;" % \
+ (self.tc.sdk_env, cmd), shell=True,
+ stderr=subprocess.STDOUT).decode("utf-8")
diff --git a/meta/lib/oeqa/sdk/context.py b/meta/lib/oeqa/sdk/context.py
new file mode 100644
index 0000000..0189ed8
--- /dev/null
+++ b/meta/lib/oeqa/sdk/context.py
@@ -0,0 +1,133 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import sys
+import glob
+import re
+
+from oeqa.core.context import OETestContext, OETestContextExecutor
+
+class OESDKTestContext(OETestContext):
+ sdk_files_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "files")
+
+ def __init__(self, td=None, logger=None, sdk_dir=None, sdk_env=None,
+ target_pkg_manifest=None, host_pkg_manifest=None):
+ super(OESDKTestContext, self).__init__(td, logger)
+
+ self.sdk_dir = sdk_dir
+ self.sdk_env = sdk_env
+ self.target_pkg_manifest = target_pkg_manifest
+ self.host_pkg_manifest = host_pkg_manifest
+
+ def _hasPackage(self, manifest, pkg):
+ for host_pkg in manifest.keys():
+ if re.search(pkg, host_pkg):
+ return True
+ return False
+
+ def hasHostPackage(self, pkg):
+ return self._hasPackage(self.host_pkg_manifest, pkg)
+
+ def hasTargetPackage(self, pkg):
+ return self._hasPackage(self.target_pkg_manifest, pkg)
+
+class OESDKTestContextExecutor(OETestContextExecutor):
+ _context_class = OESDKTestContext
+
+ name = 'sdk'
+ help = 'sdk test component'
+ description = 'executes sdk tests'
+
+ default_cases = [os.path.join(os.path.abspath(os.path.dirname(__file__)),
+ 'cases')]
+ default_test_data = None
+
+ def register_commands(self, logger, subparsers):
+ import argparse_oe
+
+ super(OESDKTestContextExecutor, self).register_commands(logger, subparsers)
+
+ sdk_group = self.parser.add_argument_group('sdk options')
+ sdk_group.add_argument('--sdk-env', action='store',
+ help='sdk environment')
+ sdk_group.add_argument('--target-manifest', action='store',
+ help='sdk target manifest')
+ sdk_group.add_argument('--host-manifest', action='store',
+ help='sdk host manifest')
+
+ sdk_dgroup = self.parser.add_argument_group('sdk display options')
+ sdk_dgroup.add_argument('--list-sdk-env', action='store_true',
+ default=False, help='sdk list available environment')
+
+ # XXX this option is required but argparse_oe has a bug handling
+ # required options, seems that don't keep track of already parsed
+ # options
+ sdk_rgroup = self.parser.add_argument_group('sdk required options')
+ sdk_rgroup.add_argument('--sdk-dir', required=False, action='store',
+ help='sdk installed directory')
+
+ @staticmethod
+ def _load_manifest(manifest):
+ pkg_manifest = {}
+ if manifest:
+ with open(manifest) as f:
+ for line in f:
+ (pkg, arch, version) = line.strip().split()
+ pkg_manifest[pkg] = (version, arch)
+
+ return pkg_manifest
+
+ def _process_args(self, logger, args):
+ super(OESDKTestContextExecutor, self)._process_args(logger, args)
+
+ self.tc_kwargs['init']['sdk_dir'] = args.sdk_dir
+ self.tc_kwargs['init']['sdk_env'] = self.sdk_env
+ self.tc_kwargs['init']['target_pkg_manifest'] = \
+ OESDKTestContextExecutor._load_manifest(args.target_manifest)
+ self.tc_kwargs['init']['host_pkg_manifest'] = \
+ OESDKTestContextExecutor._load_manifest(args.host_manifest)
+
+ @staticmethod
+ def _get_sdk_environs(sdk_dir):
+ sdk_env = {}
+
+ environ_pattern = sdk_dir + '/environment-setup-*'
+ full_sdk_env = glob.glob(sdk_dir + '/environment-setup-*')
+ for env in full_sdk_env:
+ m = re.search('environment-setup-(.*)', env)
+ if m:
+ sdk_env[m.group(1)] = env
+
+ return sdk_env
+
+ def _display_sdk_envs(self, log, args, sdk_envs):
+ log("Available SDK environments at directory %s:" \
+ % args.sdk_dir)
+ log("")
+ for env in sdk_envs:
+ log(env)
+
+ def run(self, logger, args):
+ if not args.sdk_dir:
+ raise argparse_oe.ArgumentUsageError("No SDK directory "\
+ "specified please do, --sdk-dir SDK_DIR", self.name)
+
+ sdk_envs = OESDKTestContextExecutor._get_sdk_environs(args.sdk_dir)
+ if not sdk_envs:
+ raise argparse_oe.ArgumentUsageError("No available SDK "\
+ "enviroments found at %s" % args.sdk_dir, self.name)
+
+ if args.list_sdk_env:
+ self._display_sdk_envs(logger.info, args, sdk_envs)
+ sys.exit(0)
+
+ if not args.sdk_env in sdk_envs:
+ self._display_sdk_envs(logger.error, args, sdk_envs)
+ raise argparse_oe.ArgumentUsageError("No valid SDK "\
+ "environment (%s) specified" % args.sdk_env, self.name)
+
+ self.sdk_env = sdk_envs[args.sdk_env]
+ super(OESDKTestContextExecutor, self).run(logger, args)
+
+_executor_class = OESDKTestContextExecutor
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 21/55] classes/testsdk: Migrates testsdk.bbclass to use new OESDKTestContext
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (19 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 20/55] oeqa/sdk: Add case and context modules for the SDK component Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 22/55] oeqa/utils: Move targetbuild to buildproject module Aníbal Limón
` (35 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
The functionality provided is the same with imporvements on code
reuse and better interfaces.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/classes/testsdk.bbclass | 75 +++++++++++++++++++++++++++++++++++---------
1 file changed, 60 insertions(+), 15 deletions(-)
diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
index 063b908..7304129 100644
--- a/meta/classes/testsdk.bbclass
+++ b/meta/classes/testsdk.bbclass
@@ -50,30 +50,75 @@ def run_test_context(CTestContext, d, testdir, tcname, pn, *args):
def testsdk_main(d):
import os
- import oeqa.sdk
import subprocess
- from oeqa.oetest import SDKTestContext
+ import json
+ import logging
- pn = d.getVar("PN")
- bb.utils.mkdirhier(d.getVar("TEST_LOG_DIR"))
+ from bb.utils import export_proxies
+ from oeqa.core.runner import OEStreamLogger
+ from oeqa.sdk.context import OESDKTestContext, OESDKTestContextExecutor
+ from oeqa.utils import make_logger_bitbake_compatible
+
+ pn = d.getVar("PN", True)
+ logger = make_logger_bitbake_compatible(logging.getLogger("BitBake"))
+
+ # sdk use network for download projects for build
+ export_proxies(d)
+
+ test_log_dir = d.getVar("TEST_LOG_DIR", True)
+
+ bb.utils.mkdirhier(test_log_dir)
tcname = d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh")
if not os.path.exists(tcname):
bb.fatal("The toolchain %s is not built. Build it before running the tests: 'bitbake <image> -c populate_sdk' ." % tcname)
- sdktestdir = d.expand("${WORKDIR}/testimage-sdk/")
- bb.utils.remove(sdktestdir, True)
- bb.utils.mkdirhier(sdktestdir)
+ tdname = d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.testdata.json")
+ test_data = json.load(open(tdname, "r"))
+ test_data['TEST_LOG_DIR'] = test_log_dir
+
+ target_pkg_manifest = OESDKTestContextExecutor._load_manifest(
+ d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.target.manifest"))
+ host_pkg_manifest = OESDKTestContextExecutor._load_manifest(
+ d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.host.manifest"))
+
+ sdk_dir = d.expand("${WORKDIR}/testimage-sdk/")
+ bb.utils.remove(sdk_dir, True)
+ bb.utils.mkdirhier(sdk_dir)
try:
- subprocess.check_output("cd %s; %s <<EOF\n./tc\nY\nEOF" % (sdktestdir, tcname), shell=True)
+ subprocess.check_output("cd %s; %s <<EOF\n./\nY\nEOF" % (sdk_dir, tcname), shell=True)
except subprocess.CalledProcessError as e:
bb.fatal("Couldn't install the SDK:\n%s" % e.output.decode("utf-8"))
- try:
- run_test_context(SDKTestContext, d, sdktestdir, tcname, pn)
- finally:
- bb.utils.remove(sdktestdir, True)
+ fail = False
+ sdk_envs = OESDKTestContextExecutor._get_sdk_environs(sdk_dir)
+ for s in sdk_envs:
+ sdk_env = sdk_envs[s]
+ bb.plain("SDK testing environment: %s" % s)
+ tc = OESDKTestContext(td=test_data, logger=logger, sdk_dir=sdk_dir,
+ sdk_env=sdk_env, target_pkg_manifest=target_pkg_manifest,
+ host_pkg_manifest=host_pkg_manifest)
+ try:
+ tc.loadTests(OESDKTestContextExecutor.default_cases)
+ except Exception as e:
+ import traceback
+ bb.fatal("Loading tests failed:\n%s" % traceback.format_exc())
+
+ result = tc.runTests()
+
+ component = "%s %s" % (pn, OESDKTestContextExecutor.name)
+ context_msg = "%s:%s" % (os.path.basename(tcname), os.path.basename(sdk_env))
+
+ tc.logSummary(result, component, context_msg)
+ tc.logDetails()
+
+ if not result.wasSuccessful():
+ fail = True
+
+ if fail:
+ bb.fatal("%s - FAILED - check the task log and the commands log" % pn)
+
testsdk_main[vardepsexclude] =+ "BB_ORIGENV"
python do_testsdk() {
@@ -88,13 +133,13 @@ TESTSDKEXTLOCK = "${TMPDIR}/testsdkext.lock"
def testsdkext_main(d):
import os
- import oeqa.sdkext
+ import json
import subprocess
+ import logging
+
from bb.utils import export_proxies
- from oeqa.oetest import SDKTestContext, SDKExtTestContext
from oeqa.utils import avoid_paths_in_environ
-
# extensible sdk use network
export_proxies(d)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 22/55] oeqa/utils: Move targetbuild to buildproject module
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (20 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 21/55] classes/testsdk: Migrates testsdk.bbclass to use new OESDKTestContext Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 23/55] oeqa/utils: {Target, SDK, }BuildProject remove dependency of bb Aníbal Limón
` (34 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
The new buildproject module will contain only BuildProject class
a helper class for build source code.
The remaining classes TargetBuildProject and SDKBuildProject was
move to runtime and sdk respectively.
[YOCTO #10599]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/runtime/utils/__init__.py | 0
meta/lib/oeqa/runtime/utils/targetbuildproject.py | 33 +++++++++++
meta/lib/oeqa/sdk/utils/__init__.py | 0
meta/lib/oeqa/sdk/utils/sdkbuildproject.py | 45 +++++++++++++++
meta/lib/oeqa/utils/buildproject.py | 69 +++++++++++++++++++++++
5 files changed, 147 insertions(+)
create mode 100644 meta/lib/oeqa/runtime/utils/__init__.py
create mode 100644 meta/lib/oeqa/runtime/utils/targetbuildproject.py
create mode 100644 meta/lib/oeqa/sdk/utils/__init__.py
create mode 100644 meta/lib/oeqa/sdk/utils/sdkbuildproject.py
create mode 100644 meta/lib/oeqa/utils/buildproject.py
diff --git a/meta/lib/oeqa/runtime/utils/__init__.py b/meta/lib/oeqa/runtime/utils/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/meta/lib/oeqa/runtime/utils/targetbuildproject.py b/meta/lib/oeqa/runtime/utils/targetbuildproject.py
new file mode 100644
index 0000000..138b5ef
--- /dev/null
+++ b/meta/lib/oeqa/runtime/utils/targetbuildproject.py
@@ -0,0 +1,33 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.utils.buildproject import BuildProject
+
+class TargetBuildProject(BuildProject):
+
+ def __init__(self, target, d, uri, foldername=None):
+ self.target = target
+ self.targetdir = "~/"
+ BuildProject.__init__(self, d, uri, foldername, tmpdir="/tmp")
+
+ def download_archive(self):
+
+ self._download_archive()
+
+ (status, output) = self.target.copy_to(self.localarchive, self.targetdir)
+ if status != 0:
+ raise Exception("Failed to copy archive to target, output: %s" % output)
+
+ (status, output) = self.target.run('tar xf %s%s -C %s' % (self.targetdir, self.archive, self.targetdir))
+ if status != 0:
+ raise Exception("Failed to extract archive, output: %s" % output)
+
+ #Change targetdir to project folder
+ self.targetdir = self.targetdir + self.fname
+
+ # The timeout parameter of target.run is set to 0 to make the ssh command
+ # run with no timeout.
+ def _run(self, cmd):
+ return self.target.run(cmd, 0)[0]
+
+
diff --git a/meta/lib/oeqa/sdk/utils/__init__.py b/meta/lib/oeqa/sdk/utils/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/meta/lib/oeqa/sdk/utils/sdkbuildproject.py b/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
new file mode 100644
index 0000000..1eebd3a
--- /dev/null
+++ b/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
@@ -0,0 +1,45 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import subprocess
+
+from oeqa.utils.buildproject import BuildProject
+
+class SDKBuildProject(BuildProject):
+
+ def __init__(self, testpath, sdkenv, d, uri, foldername=None):
+ self.sdkenv = sdkenv
+ self.testdir = testpath
+ self.targetdir = testpath
+ bb.utils.mkdirhier(testpath)
+ self.datetime = d.getVar('DATETIME')
+ self.testlogdir = d.getVar("TEST_LOG_DIR")
+ bb.utils.mkdirhier(self.testlogdir)
+ self.logfile = os.path.join(self.testlogdir, "sdk_target_log.%s" % self.datetime)
+ BuildProject.__init__(self, d, uri, foldername, tmpdir=testpath)
+
+ def download_archive(self):
+
+ self._download_archive()
+
+ cmd = 'tar xf %s%s -C %s' % (self.targetdir, self.archive, self.targetdir)
+ subprocess.check_call(cmd, shell=True)
+
+ #Change targetdir to project folder
+ self.targetdir = os.path.join(self.targetdir, self.fname)
+
+ def run_configure(self, configure_args='', extra_cmds=' gnu-configize; '):
+ return super(SDKBuildProject, self).run_configure(configure_args=(configure_args or '$CONFIGURE_FLAGS'), extra_cmds=extra_cmds)
+
+ def run_install(self, install_args=''):
+ return super(SDKBuildProject, self).run_install(install_args=(install_args or "DESTDIR=%s/../install" % self.targetdir))
+
+ def log(self, msg):
+ if self.logfile:
+ with open(self.logfile, "a") as f:
+ f.write("%s\n" % msg)
+
+ def _run(self, cmd):
+ self.log("Running . %s; " % self.sdkenv + cmd)
+ return subprocess.call(". %s; " % self.sdkenv + cmd, shell=True)
diff --git a/meta/lib/oeqa/utils/buildproject.py b/meta/lib/oeqa/utils/buildproject.py
new file mode 100644
index 0000000..1ed9624
--- /dev/null
+++ b/meta/lib/oeqa/utils/buildproject.py
@@ -0,0 +1,69 @@
+# Copyright (C) 2013-2016 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+# Provides a class for automating build tests for projects
+
+import os
+import re
+import bb.utils
+import subprocess
+from abc import ABCMeta, abstractmethod
+
+class BuildProject(metaclass=ABCMeta):
+
+ def __init__(self, d, uri, foldername=None, tmpdir="/tmp/"):
+ self.d = d
+ self.uri = uri
+ self.archive = os.path.basename(uri)
+ self.localarchive = os.path.join(tmpdir,self.archive)
+ if foldername:
+ self.fname = foldername
+ else:
+ self.fname = re.sub(r'\.tar\.bz2$|\.tar\.gz$|\.tar\.xz$', '', self.archive)
+
+ # Download self.archive to self.localarchive
+ def _download_archive(self):
+
+ dl_dir = self.d.getVar("DL_DIR")
+ if dl_dir and os.path.exists(os.path.join(dl_dir, self.archive)):
+ bb.utils.copyfile(os.path.join(dl_dir, self.archive), self.localarchive)
+ return
+
+ exportvars = ['HTTP_PROXY', 'http_proxy',
+ 'HTTPS_PROXY', 'https_proxy',
+ 'FTP_PROXY', 'ftp_proxy',
+ 'FTPS_PROXY', 'ftps_proxy',
+ 'NO_PROXY', 'no_proxy',
+ 'ALL_PROXY', 'all_proxy',
+ 'SOCKS5_USER', 'SOCKS5_PASSWD']
+
+ cmd = ''
+ for var in exportvars:
+ val = self.d.getVar(var)
+ if val:
+ cmd = 'export ' + var + '=\"%s\"; %s' % (val, cmd)
+
+ cmd = cmd + "wget -O %s %s" % (self.localarchive, self.uri)
+ subprocess.check_call(cmd, shell=True)
+
+ # This method should provide a way to run a command in the desired environment.
+ @abstractmethod
+ def _run(self, cmd):
+ pass
+
+ # The timeout parameter of target.run is set to 0 to make the ssh command
+ # run with no timeout.
+ def run_configure(self, configure_args='', extra_cmds=''):
+ return self._run('cd %s; %s ./configure %s' % (self.targetdir, extra_cmds, configure_args))
+
+ def run_make(self, make_args=''):
+ return self._run('cd %s; make %s' % (self.targetdir, make_args))
+
+ def run_install(self, install_args=''):
+ return self._run('cd %s; make install %s' % (self.targetdir, install_args))
+
+ def clean(self):
+ self._run('rm -rf %s' % self.targetdir)
+ subprocess.call('rm -f %s' % self.localarchive, shell=True)
+ pass
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 23/55] oeqa/utils: {Target, SDK, }BuildProject remove dependency of bb
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (21 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 22/55] oeqa/utils: Move targetbuild to buildproject module Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 24/55] oeqa/sdk/cases: Migrate tests to the new OEQA framework Aníbal Limón
` (33 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
Don't use bitbake references inside utils modules, in order todo
that changes getVar calls for arguments in the __init__ method like
dl_dir for all the classes and testlogdir, builddatetime in
SDKBUildProject.
Also don't export proxies inside _download_archive method, a good
practice is to setup the proxies at init of the process instead of
do it in this helper module.
[YOCTO #10231]
[YOCTO #10599]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/runtime/utils/targetbuildproject.py | 6 ++---
meta/lib/oeqa/sdk/utils/sdkbuildproject.py | 14 +++++-----
meta/lib/oeqa/utils/buildproject.py | 31 +++++------------------
3 files changed, 17 insertions(+), 34 deletions(-)
diff --git a/meta/lib/oeqa/runtime/utils/targetbuildproject.py b/meta/lib/oeqa/runtime/utils/targetbuildproject.py
index 138b5ef..006d4d4 100644
--- a/meta/lib/oeqa/runtime/utils/targetbuildproject.py
+++ b/meta/lib/oeqa/runtime/utils/targetbuildproject.py
@@ -5,13 +5,13 @@ from oeqa.utils.buildproject import BuildProject
class TargetBuildProject(BuildProject):
- def __init__(self, target, d, uri, foldername=None):
+ def __init__(self, target, uri, foldername=None, dl_dir=None):
self.target = target
self.targetdir = "~/"
- BuildProject.__init__(self, d, uri, foldername, tmpdir="/tmp")
+ BuildProject.__init__(self, uri, foldername, tmpdir="/tmp",
+ dl_dir=dl_dir)
def download_archive(self):
-
self._download_archive()
(status, output) = self.target.copy_to(self.localarchive, self.targetdir)
diff --git a/meta/lib/oeqa/sdk/utils/sdkbuildproject.py b/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
index 1eebd3a..cc34e0c 100644
--- a/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
+++ b/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
@@ -7,17 +7,17 @@ import subprocess
from oeqa.utils.buildproject import BuildProject
class SDKBuildProject(BuildProject):
-
- def __init__(self, testpath, sdkenv, d, uri, foldername=None):
+ def __init__(self, testpath, sdkenv, uri, testlogdir, builddatetime,
+ foldername=None, dl_dir=None):
self.sdkenv = sdkenv
self.testdir = testpath
self.targetdir = testpath
- bb.utils.mkdirhier(testpath)
- self.datetime = d.getVar('DATETIME')
- self.testlogdir = d.getVar("TEST_LOG_DIR")
- bb.utils.mkdirhier(self.testlogdir)
+ os.makedirs(testpath, exist_ok=True)
+ self.datetime = builddatetime
+ self.testlogdir = testlogdir
+ os.makedirs(self.testlogdir, exist_ok=True)
self.logfile = os.path.join(self.testlogdir, "sdk_target_log.%s" % self.datetime)
- BuildProject.__init__(self, d, uri, foldername, tmpdir=testpath)
+ BuildProject.__init__(self, uri, foldername, tmpdir=testpath, dl_dir=dl_dir)
def download_archive(self):
diff --git a/meta/lib/oeqa/utils/buildproject.py b/meta/lib/oeqa/utils/buildproject.py
index 1ed9624..386a927 100644
--- a/meta/lib/oeqa/utils/buildproject.py
+++ b/meta/lib/oeqa/utils/buildproject.py
@@ -6,17 +6,17 @@
import os
import re
-import bb.utils
import subprocess
+import shutil
+
from abc import ABCMeta, abstractmethod
class BuildProject(metaclass=ABCMeta):
-
- def __init__(self, d, uri, foldername=None, tmpdir="/tmp/"):
- self.d = d
+ def __init__(self, uri, foldername=None, tmpdir="/tmp/", dl_dir=None):
self.uri = uri
self.archive = os.path.basename(uri)
self.localarchive = os.path.join(tmpdir,self.archive)
+ self.dl_dir = dl_dir
if foldername:
self.fname = foldername
else:
@@ -24,27 +24,11 @@ class BuildProject(metaclass=ABCMeta):
# Download self.archive to self.localarchive
def _download_archive(self):
-
- dl_dir = self.d.getVar("DL_DIR")
- if dl_dir and os.path.exists(os.path.join(dl_dir, self.archive)):
- bb.utils.copyfile(os.path.join(dl_dir, self.archive), self.localarchive)
+ if self.dl_dir and os.path.exists(os.path.join(self.dl_dir, self.archive)):
+ shutil.copyfile(os.path.join(self.dl_dir, self.archive), self.localarchive)
return
- exportvars = ['HTTP_PROXY', 'http_proxy',
- 'HTTPS_PROXY', 'https_proxy',
- 'FTP_PROXY', 'ftp_proxy',
- 'FTPS_PROXY', 'ftps_proxy',
- 'NO_PROXY', 'no_proxy',
- 'ALL_PROXY', 'all_proxy',
- 'SOCKS5_USER', 'SOCKS5_PASSWD']
-
- cmd = ''
- for var in exportvars:
- val = self.d.getVar(var)
- if val:
- cmd = 'export ' + var + '=\"%s\"; %s' % (val, cmd)
-
- cmd = cmd + "wget -O %s %s" % (self.localarchive, self.uri)
+ cmd = "wget -O %s %s" % (self.localarchive, self.uri)
subprocess.check_call(cmd, shell=True)
# This method should provide a way to run a command in the desired environment.
@@ -66,4 +50,3 @@ class BuildProject(metaclass=ABCMeta):
def clean(self):
self._run('rm -rf %s' % self.targetdir)
subprocess.call('rm -f %s' % self.localarchive, shell=True)
- pass
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 24/55] oeqa/sdk/cases: Migrate tests to the new OEQA framework
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (22 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 23/55] oeqa/utils: {Target, SDK, }BuildProject remove dependency of bb Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 25/55] classes/testsdk: Remove the need of TEST_LOG_DIR variable Aníbal Limón
` (32 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
Summary of the changes:
- Remove auto extend_path using pkgutil at __init__, is not needed.
- Change base class to OESDKTestCase.
- Add td_vars attr to set dependencies of certain variables in test
data.
- Change skips from module level to class level because Test context
(tc)
now isn't at module level.
- Variable names changes to be consistent (i.e. sdktestdir ->
sdk_dir).
[YOCTO #10599]
- Don't use bb.utils functions use instead remove_safe and shutil
for copy files.
- SDKBuildProject pass test data variables instead of call getVar
inside.
[YOCTO #10231]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/sdk/cases/__init__.py | 3 ---
meta/lib/oeqa/sdk/cases/buildcvs.py | 15 ++++++-----
meta/lib/oeqa/sdk/cases/buildgalculator.py | 28 ++++++++++++--------
meta/lib/oeqa/sdk/cases/buildiptables.py | 16 +++++++-----
meta/lib/oeqa/sdk/cases/gcc.py | 41 +++++++++++++++++-------------
meta/lib/oeqa/sdk/cases/perl.py | 25 +++++++++---------
meta/lib/oeqa/sdk/cases/python.py | 25 +++++++++---------
7 files changed, 84 insertions(+), 69 deletions(-)
delete mode 100644 meta/lib/oeqa/sdk/cases/__init__.py
diff --git a/meta/lib/oeqa/sdk/cases/__init__.py b/meta/lib/oeqa/sdk/cases/__init__.py
deleted file mode 100644
index 4cf3fa7..0000000
--- a/meta/lib/oeqa/sdk/cases/__init__.py
+++ /dev/null
@@ -1,3 +0,0 @@
-# Enable other layers to have tests in the same named directory
-from pkgutil import extend_path
-__path__ = extend_path(__path__, __name__)
diff --git a/meta/lib/oeqa/sdk/cases/buildcvs.py b/meta/lib/oeqa/sdk/cases/buildcvs.py
index c7146fa..ee7fb73 100644
--- a/meta/lib/oeqa/sdk/cases/buildcvs.py
+++ b/meta/lib/oeqa/sdk/cases/buildcvs.py
@@ -1,13 +1,16 @@
-from oeqa.oetest import oeSDKTest, skipModule
-from oeqa.utils.decorators import *
-from oeqa.utils.targetbuild import SDKBuildProject
+from oeqa.sdk.case import OESDKTestCase
+from oeqa.sdk.utils.sdkbuildproject import SDKBuildProject
-class BuildCvsTest(oeSDKTest):
+class BuildCvsTest(OESDKTestCase):
+ td_vars = ['TEST_LOG_DIR', 'DATETIME']
@classmethod
def setUpClass(self):
- self.project = SDKBuildProject(oeSDKTest.tc.sdktestdir + "/cvs/", oeSDKTest.tc.sdkenv, oeSDKTest.tc.d,
- "http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2")
+ dl_dir = self.td.get('DL_DIR', None)
+
+ self.project = SDKBuildProject(self.tc.sdk_dir + "/cvs/", self.tc.sdk_env,
+ "http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2",
+ self.td['TEST_LOG_DIR'], self.td['DATETIME'], dl_dir=dl_dir)
self.project.download_archive()
def test_cvs(self):
diff --git a/meta/lib/oeqa/sdk/cases/buildgalculator.py b/meta/lib/oeqa/sdk/cases/buildgalculator.py
index dc2fa9c..d2c1189 100644
--- a/meta/lib/oeqa/sdk/cases/buildgalculator.py
+++ b/meta/lib/oeqa/sdk/cases/buildgalculator.py
@@ -1,17 +1,25 @@
-from oeqa.oetest import oeSDKTest, skipModule
-from oeqa.utils.decorators import *
-from oeqa.utils.targetbuild import SDKBuildProject
+import unittest
-def setUpModule():
- if not (oeSDKTest.hasPackage("gtk+3") or oeSDKTest.hasPackage("libgtk-3.0")):
- skipModule("Image doesn't have gtk+3 in manifest")
+from oeqa.sdk.case import OESDKTestCase
+from oeqa.sdk.utils.sdkbuildproject import SDKBuildProject
+
+class GalculatorTest(OESDKTestCase):
+ td_vars = ['TEST_LOG_DIR', 'DATETIME']
+
+ @classmethod
+ def setUpClass(self):
+ if not (self.tc.hasTargetPackage("gtk+3") or\
+ self.tc.hasTargetPackage("libgtk-3.0")):
+ raise unittest.SkipTest("%s class: SDK don't support gtk+3" % self.__name__)
-class GalculatorTest(oeSDKTest):
def test_galculator(self):
+ dl_dir = self.td.get('DL_DIR', None)
+ project = None
try:
- project = SDKBuildProject(oeSDKTest.tc.sdktestdir + "/galculator/",
- oeSDKTest.tc.sdkenv, oeSDKTest.tc.d,
- "http://galculator.mnim.org/downloads/galculator-2.1.4.tar.bz2")
+ project = SDKBuildProject(self.tc.sdk_dir + "/galculator/",
+ self.tc.sdk_env,
+ "http://galculator.mnim.org/downloads/galculator-2.1.4.tar.bz2",
+ self.td['TEST_LOG_DIR'], self.td['DATETIME'], dl_dir=dl_dir)
project.download_archive()
diff --git a/meta/lib/oeqa/sdk/cases/buildiptables.py b/meta/lib/oeqa/sdk/cases/buildiptables.py
index f0cb8a4..a50fb5d 100644
--- a/meta/lib/oeqa/sdk/cases/buildiptables.py
+++ b/meta/lib/oeqa/sdk/cases/buildiptables.py
@@ -1,14 +1,16 @@
-from oeqa.oetest import oeSDKTest
-from oeqa.utils.decorators import *
-from oeqa.utils.targetbuild import SDKBuildProject
+from oeqa.sdk.case import OESDKTestCase
+from oeqa.sdk.utils.sdkbuildproject import SDKBuildProject
-
-class BuildIptablesTest(oeSDKTest):
+class BuildIptablesTest(OESDKTestCase):
+ td_vars = ['TEST_LOG_DIR', 'DATETIME']
@classmethod
def setUpClass(self):
- self.project = SDKBuildProject(oeSDKTest.tc.sdktestdir + "/iptables/", oeSDKTest.tc.sdkenv, oeSDKTest.tc.d,
- "http://downloads.yoctoproject.org/mirror/sources/iptables-1.4.13.tar.bz2")
+ dl_dir = self.td.get('DL_DIR', None)
+
+ self.project = SDKBuildProject(self.tc.sdk_dir + "/iptables/", self.tc.sdk_env,
+ "http://downloads.yoctoproject.org/mirror/sources/iptables-1.4.13.tar.bz2",
+ self.td['TEST_LOG_DIR'], self.td['DATETIME'], dl_dir=dl_dir)
self.project.download_archive()
def test_iptables(self):
diff --git a/meta/lib/oeqa/sdk/cases/gcc.py b/meta/lib/oeqa/sdk/cases/gcc.py
index f3f4341..e06af4c 100644
--- a/meta/lib/oeqa/sdk/cases/gcc.py
+++ b/meta/lib/oeqa/sdk/cases/gcc.py
@@ -1,36 +1,43 @@
-import unittest
import os
import shutil
-from oeqa.oetest import oeSDKTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- machine = oeSDKTest.tc.d.getVar("MACHINE")
- if not oeSDKTest.hasHostPackage("packagegroup-cross-canadian-" + machine):
- skipModule("SDK doesn't contain a cross-canadian toolchain")
+import unittest
+from oeqa.core.utils.path import remove_safe
+from oeqa.sdk.case import OESDKTestCase
-class GccCompileTest(oeSDKTest):
+class GccCompileTest(OESDKTestCase):
+ td_vars = ['MACHINE']
@classmethod
def setUpClass(self):
- for f in ['test.c', 'test.cpp', 'testsdkmakefile']:
- shutil.copyfile(os.path.join(self.tc.filesdir, f), self.tc.sdktestdir + f)
+ files = {'test.c' : self.tc.files_dir, 'test.cpp' : self.tc.files_dir,
+ 'testsdkmakefile' : self.tc.sdk_files_dir}
+ for f in files:
+ shutil.copyfile(os.path.join(files[f], f),
+ os.path.join(self.tc.sdk_dir, f))
+
+ def setUp(self):
+ machine = self.td.get("MACHINE")
+ if not self.tc.hasHostPackage("packagegroup-cross-canadian-%s" % machine):
+ raise unittest.SkipTest("%s class: SDK doesn't contain a cross-canadian toolchain",
+ self.__name__)
def test_gcc_compile(self):
- self._run('$CC %s/test.c -o %s/test -lm' % (self.tc.sdktestdir, self.tc.sdktestdir))
+ self._run('$CC %s/test.c -o %s/test -lm' % (self.tc.sdk_dir, self.tc.sdk_dir))
def test_gpp_compile(self):
- self._run('$CXX %s/test.c -o %s/test -lm' % (self.tc.sdktestdir, self.tc.sdktestdir))
+ self._run('$CXX %s/test.c -o %s/test -lm' % (self.tc.sdk_dir, self.tc.sdk_dir))
def test_gpp2_compile(self):
- self._run('$CXX %s/test.cpp -o %s/test -lm' % (self.tc.sdktestdir, self.tc.sdktestdir))
+ self._run('$CXX %s/test.cpp -o %s/test -lm' % (self.tc.sdk_dir, self.tc.sdk_dir))
def test_make(self):
- self._run('cd %s; make -f testsdkmakefile' % self.tc.sdktestdir)
+ self._run('cd %s; make -f testsdkmakefile' % self.tc.sdk_dir)
@classmethod
def tearDownClass(self):
- files = [self.tc.sdktestdir + f for f in ['test.c', 'test.cpp', 'test.o', 'test', 'testsdkmakefile']]
+ files = [os.path.join(self.tc.sdk_dir, f) \
+ for f in ['test.c', 'test.cpp', 'test.o', 'test',
+ 'testsdkmakefile']]
for f in files:
- bb.utils.remove(f)
+ remove_safe(f)
diff --git a/meta/lib/oeqa/sdk/cases/perl.py b/meta/lib/oeqa/sdk/cases/perl.py
index 45f422e..e1bded2 100644
--- a/meta/lib/oeqa/sdk/cases/perl.py
+++ b/meta/lib/oeqa/sdk/cases/perl.py
@@ -1,28 +1,27 @@
-import unittest
import os
import shutil
-from oeqa.oetest import oeSDKTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeSDKTest.hasHostPackage("nativesdk-perl"):
- skipModule("No perl package in the SDK")
-
+import unittest
-class PerlTest(oeSDKTest):
+from oeqa.core.utils.path import remove_safe
+from oeqa.sdk.case import OESDKTestCase
+class PerlTest(OESDKTestCase):
@classmethod
def setUpClass(self):
+ if not self.tc.hasHostPackage("nativesdk-perl"):
+ raise unittest.SkipTest("No perl package in the SDK")
+
for f in ['test.pl']:
- shutil.copyfile(os.path.join(self.tc.filesdir, f), self.tc.sdktestdir + f)
- self.testfile = self.tc.sdktestdir + "test.pl"
+ shutil.copyfile(os.path.join(self.tc.files_dir, f),
+ os.path.join(self.tc.sdk_dir, f))
+ self.testfile = os.path.join(self.tc.sdk_dir, "test.pl")
def test_perl_exists(self):
self._run('which perl')
def test_perl_works(self):
- self._run('perl %s/test.pl' % self.tc.sdktestdir)
+ self._run('perl %s' % self.testfile)
@classmethod
def tearDownClass(self):
- bb.utils.remove("%s/test.pl" % self.tc.sdktestdir)
+ remove_safe(self.testfile)
diff --git a/meta/lib/oeqa/sdk/cases/python.py b/meta/lib/oeqa/sdk/cases/python.py
index 896fab4..94a296f 100644
--- a/meta/lib/oeqa/sdk/cases/python.py
+++ b/meta/lib/oeqa/sdk/cases/python.py
@@ -1,26 +1,25 @@
-import unittest
import os
import shutil
-from oeqa.oetest import oeSDKTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeSDKTest.hasHostPackage("nativesdk-python"):
- skipModule("No python package in the SDK")
-
+import unittest
-class PythonTest(oeSDKTest):
+from oeqa.core.utils.path import remove_safe
+from oeqa.sdk.case import OESDKTestCase
+class PythonTest(OESDKTestCase):
@classmethod
def setUpClass(self):
+ if not self.tc.hasHostPackage("nativesdk-python"):
+ raise unittest.SkipTest("No python package in the SDK")
+
for f in ['test.py']:
- shutil.copyfile(os.path.join(self.tc.filesdir, f), self.tc.sdktestdir + f)
+ shutil.copyfile(os.path.join(self.tc.files_dir, f),
+ os.path.join(self.tc.sdk_dir, f))
def test_python_exists(self):
self._run('which python')
def test_python_stdout(self):
- output = self._run('python %s/test.py' % self.tc.sdktestdir)
+ output = self._run('python %s/test.py' % self.tc.sdk_dir)
self.assertEqual(output.strip(), "the value of a is 0.01", msg="Incorrect output: %s" % output)
def test_python_testfile(self):
@@ -28,5 +27,5 @@ class PythonTest(oeSDKTest):
@classmethod
def tearDownClass(self):
- bb.utils.remove("%s/test.py" % self.tc.sdktestdir)
- bb.utils.remove("/tmp/testfile.python")
+ remove_safe("%s/test.py" % self.tc.sdk_dir)
+ remove_safe("/tmp/testfile.python")
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 25/55] classes/testsdk: Remove the need of TEST_LOG_DIR variable
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (23 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 24/55] oeqa/sdk/cases: Migrate tests to the new OEQA framework Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 26/55] oeqa/sdkext: Move test cases inside cases directory Aníbal Limón
` (31 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
The TEST_LOG_DIR was used for store sdk_target_log this log
contains the output of the run of build commands now that information
could be found also on log.do_testsdk under WORKDIR.
The log will continue to store into SDK_DIR instead of TEST_LOG_DIR.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/classes/testsdk.bbclass | 6 ------
meta/lib/oeqa/sdk/cases/buildcvs.py | 4 ++--
meta/lib/oeqa/sdk/cases/buildgalculator.py | 4 ++--
meta/lib/oeqa/sdk/cases/buildiptables.py | 4 ++--
4 files changed, 6 insertions(+), 12 deletions(-)
diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
index 7304129..176dafd 100644
--- a/meta/classes/testsdk.bbclass
+++ b/meta/classes/testsdk.bbclass
@@ -14,7 +14,6 @@
#
# where "<image-name>" is an image like core-image-sato.
-TEST_LOG_DIR ?= "${WORKDIR}/testimage"
TESTSDKLOCK = "${TMPDIR}/testsdk.lock"
def run_test_context(CTestContext, d, testdir, tcname, pn, *args):
@@ -65,17 +64,12 @@ def testsdk_main(d):
# sdk use network for download projects for build
export_proxies(d)
- test_log_dir = d.getVar("TEST_LOG_DIR", True)
-
- bb.utils.mkdirhier(test_log_dir)
-
tcname = d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh")
if not os.path.exists(tcname):
bb.fatal("The toolchain %s is not built. Build it before running the tests: 'bitbake <image> -c populate_sdk' ." % tcname)
tdname = d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.testdata.json")
test_data = json.load(open(tdname, "r"))
- test_data['TEST_LOG_DIR'] = test_log_dir
target_pkg_manifest = OESDKTestContextExecutor._load_manifest(
d.expand("${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.target.manifest"))
diff --git a/meta/lib/oeqa/sdk/cases/buildcvs.py b/meta/lib/oeqa/sdk/cases/buildcvs.py
index ee7fb73..6222a8e 100644
--- a/meta/lib/oeqa/sdk/cases/buildcvs.py
+++ b/meta/lib/oeqa/sdk/cases/buildcvs.py
@@ -2,7 +2,7 @@ from oeqa.sdk.case import OESDKTestCase
from oeqa.sdk.utils.sdkbuildproject import SDKBuildProject
class BuildCvsTest(OESDKTestCase):
- td_vars = ['TEST_LOG_DIR', 'DATETIME']
+ td_vars = ['DATETIME']
@classmethod
def setUpClass(self):
@@ -10,7 +10,7 @@ class BuildCvsTest(OESDKTestCase):
self.project = SDKBuildProject(self.tc.sdk_dir + "/cvs/", self.tc.sdk_env,
"http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2",
- self.td['TEST_LOG_DIR'], self.td['DATETIME'], dl_dir=dl_dir)
+ self.tc.sdk_dir, self.td['DATETIME'], dl_dir=dl_dir)
self.project.download_archive()
def test_cvs(self):
diff --git a/meta/lib/oeqa/sdk/cases/buildgalculator.py b/meta/lib/oeqa/sdk/cases/buildgalculator.py
index d2c1189..bdc8b6a 100644
--- a/meta/lib/oeqa/sdk/cases/buildgalculator.py
+++ b/meta/lib/oeqa/sdk/cases/buildgalculator.py
@@ -4,7 +4,7 @@ from oeqa.sdk.case import OESDKTestCase
from oeqa.sdk.utils.sdkbuildproject import SDKBuildProject
class GalculatorTest(OESDKTestCase):
- td_vars = ['TEST_LOG_DIR', 'DATETIME']
+ td_vars = ['DATETIME']
@classmethod
def setUpClass(self):
@@ -19,7 +19,7 @@ class GalculatorTest(OESDKTestCase):
project = SDKBuildProject(self.tc.sdk_dir + "/galculator/",
self.tc.sdk_env,
"http://galculator.mnim.org/downloads/galculator-2.1.4.tar.bz2",
- self.td['TEST_LOG_DIR'], self.td['DATETIME'], dl_dir=dl_dir)
+ self.tc.sdk_dir, self.td['DATETIME'], dl_dir=dl_dir)
project.download_archive()
diff --git a/meta/lib/oeqa/sdk/cases/buildiptables.py b/meta/lib/oeqa/sdk/cases/buildiptables.py
index a50fb5d..532b5de 100644
--- a/meta/lib/oeqa/sdk/cases/buildiptables.py
+++ b/meta/lib/oeqa/sdk/cases/buildiptables.py
@@ -2,7 +2,7 @@ from oeqa.sdk.case import OESDKTestCase
from oeqa.sdk.utils.sdkbuildproject import SDKBuildProject
class BuildIptablesTest(OESDKTestCase):
- td_vars = ['TEST_LOG_DIR', 'DATETIME']
+ td_vars = ['DATETIME']
@classmethod
def setUpClass(self):
@@ -10,7 +10,7 @@ class BuildIptablesTest(OESDKTestCase):
self.project = SDKBuildProject(self.tc.sdk_dir + "/iptables/", self.tc.sdk_env,
"http://downloads.yoctoproject.org/mirror/sources/iptables-1.4.13.tar.bz2",
- self.td['TEST_LOG_DIR'], self.td['DATETIME'], dl_dir=dl_dir)
+ self.tc.sdk_dir, self.td['DATETIME'], dl_dir=dl_dir)
self.project.download_archive()
def test_iptables(self):
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 26/55] oeqa/sdkext: Move test cases inside cases directory
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (24 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 25/55] classes/testsdk: Remove the need of TEST_LOG_DIR variable Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 27/55] oeqa/sdkext: Adds case and context modules Aníbal Limón
` (30 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
For match with the new structure of the OEQA framework.
In the new framework Test component base directory in this case
sdk module will contain case and context implementations.
[YOCTO #10599]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/sdkext/__init__.py | 3 ---
meta/lib/oeqa/sdkext/{ => cases}/devtool.py | 0
meta/lib/oeqa/sdkext/{ => cases}/sdk_update.py | 0
3 files changed, 3 deletions(-)
rename meta/lib/oeqa/sdkext/{ => cases}/devtool.py (100%)
rename meta/lib/oeqa/sdkext/{ => cases}/sdk_update.py (100%)
diff --git a/meta/lib/oeqa/sdkext/__init__.py b/meta/lib/oeqa/sdkext/__init__.py
index 4cf3fa7..e69de29 100644
--- a/meta/lib/oeqa/sdkext/__init__.py
+++ b/meta/lib/oeqa/sdkext/__init__.py
@@ -1,3 +0,0 @@
-# Enable other layers to have tests in the same named directory
-from pkgutil import extend_path
-__path__ = extend_path(__path__, __name__)
diff --git a/meta/lib/oeqa/sdkext/devtool.py b/meta/lib/oeqa/sdkext/cases/devtool.py
similarity index 100%
rename from meta/lib/oeqa/sdkext/devtool.py
rename to meta/lib/oeqa/sdkext/cases/devtool.py
diff --git a/meta/lib/oeqa/sdkext/sdk_update.py b/meta/lib/oeqa/sdkext/cases/sdk_update.py
similarity index 100%
rename from meta/lib/oeqa/sdkext/sdk_update.py
rename to meta/lib/oeqa/sdkext/cases/sdk_update.py
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 27/55] oeqa/sdkext: Adds case and context modules.
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (25 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 26/55] oeqa/sdkext: Move test cases inside cases directory Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:09 ` [PATCH 28/55] classes/testsdk: Migrate to use the new OESDKExtTestContext Aníbal Limón
` (29 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
The extensible sdk context and case modules extends the sdk ones,
this means that the tests from sdk are run also the sdkext tests.
Enables support in context for use oe-test esdk command for run
the test suites, the same options of sdk are required for run esdk tests.
Removes old related to case and context inside oetest.py.
[YOCTO #10599]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/oetest.py | 40 ----------------------------------------
meta/lib/oeqa/sdkext/case.py | 21 +++++++++++++++++++++
meta/lib/oeqa/sdkext/context.py | 21 +++++++++++++++++++++
3 files changed, 42 insertions(+), 40 deletions(-)
create mode 100644 meta/lib/oeqa/sdkext/case.py
create mode 100644 meta/lib/oeqa/sdkext/context.py
diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index d7c3588..503f6fc 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -27,7 +27,6 @@ try:
except ImportError:
pass
from oeqa.utils.decorators import LogResults, gettag, getResults
-from oeqa.utils import avoid_paths_in_environ
logger = logging.getLogger("BitBake")
@@ -145,18 +144,6 @@ class OETestCalledProcessError(subprocess.CalledProcessError):
subprocess.CalledProcessError = OETestCalledProcessError
-class oeSDKExtTest(oeSDKTest):
- def _run(self, cmd):
- # extensible sdk shows a warning if found bitbake in the path
- # because can cause contamination, i.e. use devtool from
- # poky/scripts instead of eSDK one.
- env = os.environ.copy()
- paths_to_avoid = ['bitbake/bin', 'poky/scripts']
- env['PATH'] = avoid_paths_in_environ(paths_to_avoid)
-
- return subprocess.check_output(". %s > /dev/null;"\
- " %s;" % (self.tc.sdkenv, cmd), stderr=subprocess.STDOUT, shell=True, env=env).decode("utf-8")
-
def getmodule(pos=2):
# stack returns a list of tuples containg frame information
# First element of the list the is current frame, caller is 1
@@ -642,30 +629,3 @@ class ExportTestContext(RuntimeTestContext):
extracted_dir = self.d.getVar("TEST_EXPORT_EXTRACTED_DIR")
pkg_dir = os.path.join(export_dir, extracted_dir)
super(ExportTestContext, self).install_uninstall_packages(test_id, pkg_dir, install)
-
-class SDKExtTestContext(SDKTestContext):
- def __init__(self, d, sdktestdir, sdkenv, tcname, *args):
- self.target_manifest = d.getVar("SDK_EXT_TARGET_MANIFEST")
- self.host_manifest = d.getVar("SDK_EXT_HOST_MANIFEST")
- if args:
- self.cm = args[0] # Compatibility mode for run SDK tests
- else:
- self.cm = False
-
- super(SDKExtTestContext, self).__init__(d, sdktestdir, sdkenv, tcname)
-
- self.sdkextfilesdir = os.path.join(os.path.dirname(os.path.abspath(
- oeqa.sdkext.__file__)), "files")
-
- def _get_test_namespace(self):
- if self.cm:
- return "sdk"
- else:
- return "sdkext"
-
- def _get_test_suites(self):
- return (self.d.getVar("TEST_SUITES_SDK_EXT") or "auto").split()
-
- def _get_test_suites_required(self):
- return [t for t in (self.d.getVar("TEST_SUITES_SDK_EXT") or \
- "auto").split() if t != "auto"]
diff --git a/meta/lib/oeqa/sdkext/case.py b/meta/lib/oeqa/sdkext/case.py
new file mode 100644
index 0000000..6f708aa
--- /dev/null
+++ b/meta/lib/oeqa/sdkext/case.py
@@ -0,0 +1,21 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import subprocess
+
+from oeqa.utils import avoid_paths_in_environ
+from oeqa.sdk.case import OESDKTestCase
+
+class OESDKExtTestCase(OESDKTestCase):
+ def _run(self, cmd):
+ # extensible sdk shows a warning if found bitbake in the path
+ # because can cause contamination, i.e. use devtool from
+ # poky/scripts instead of eSDK one.
+ env = os.environ.copy()
+ paths_to_avoid = ['bitbake/bin', 'poky/scripts']
+ env['PATH'] = avoid_paths_in_environ(paths_to_avoid)
+
+ return subprocess.check_output(". %s > /dev/null;"\
+ " %s;" % (self.tc.sdk_env, cmd), stderr=subprocess.STDOUT,
+ shell=True, env=env).decode("utf-8")
diff --git a/meta/lib/oeqa/sdkext/context.py b/meta/lib/oeqa/sdkext/context.py
new file mode 100644
index 0000000..8dbcd80
--- /dev/null
+++ b/meta/lib/oeqa/sdkext/context.py
@@ -0,0 +1,21 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+from oeqa.sdk.context import OESDKTestContext, OESDKTestContextExecutor
+
+class OESDKExtTestContext(OESDKTestContext):
+ esdk_files_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "files")
+
+class OESDKExtTestContextExecutor(OESDKTestContextExecutor):
+ _context_class = OESDKExtTestContext
+
+ name = 'esdk'
+ help = 'esdk test component'
+ description = 'executes esdk tests'
+
+ default_cases = [OESDKTestContextExecutor.default_cases[0],
+ os.path.join(os.path.abspath(os.path.dirname(__file__)), 'cases')]
+ default_test_data = None
+
+_executor_class = OESDKExtTestContextExecutor
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 28/55] classes/testsdk: Migrate to use the new OESDKExtTestContext
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (26 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 27/55] oeqa/sdkext: Adds case and context modules Aníbal Limón
@ 2017-01-20 17:09 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 29/55] oeqa/sdkext/cases: Migrate test case to new OEQA framework Aníbal Limón
` (28 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:09 UTC (permalink / raw)
To: openembedded-core
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/classes/testsdk.bbclass | 98 ++++++++++++++++++++------------------------
1 file changed, 45 insertions(+), 53 deletions(-)
diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
index 176dafd..69689a1 100644
--- a/meta/classes/testsdk.bbclass
+++ b/meta/classes/testsdk.bbclass
@@ -16,37 +16,6 @@
TESTSDKLOCK = "${TMPDIR}/testsdk.lock"
-def run_test_context(CTestContext, d, testdir, tcname, pn, *args):
- import glob
- import time
-
- targets = glob.glob(d.expand(testdir + "/tc/environment-setup-*"))
- for sdkenv in targets:
- bb.plain("Testing %s" % sdkenv)
- tc = CTestContext(d, testdir, sdkenv, tcname, args)
-
- # this is a dummy load of tests
- # we are doing that to find compile errors in the tests themselves
- # before booting the image
- try:
- tc.loadTests()
- except Exception as e:
- import traceback
- bb.fatal("Loading tests failed:\n%s" % traceback.format_exc())
-
- starttime = time.time()
- result = tc.runTests()
- stoptime = time.time()
- if result.wasSuccessful():
- bb.plain("%s SDK(%s):%s - Ran %d test%s in %.3fs" % (pn, os.path.basename(tcname), os.path.basename(sdkenv),result.testsRun, result.testsRun != 1 and "s" or "", stoptime - starttime))
- msg = "%s - OK - All required tests passed" % pn
- skipped = len(result.skipped)
- if skipped:
- msg += " (skipped=%d)" % skipped
- bb.plain(msg)
- else:
- bb.fatal("%s - FAILED - check the task log and the commands log" % pn)
-
def testsdk_main(d):
import os
import subprocess
@@ -122,7 +91,6 @@ addtask testsdk
do_testsdk[nostamp] = "1"
do_testsdk[lockfiles] += "${TESTSDKLOCK}"
-TEST_LOG_SDKEXT_DIR ?= "${WORKDIR}/testsdkext"
TESTSDKEXTLOCK = "${TMPDIR}/testsdkext.lock"
def testsdkext_main(d):
@@ -132,7 +100,11 @@ def testsdkext_main(d):
import logging
from bb.utils import export_proxies
- from oeqa.utils import avoid_paths_in_environ
+ from oeqa.utils import avoid_paths_in_environ, make_logger_bitbake_compatible
+ from oeqa.sdkext.context import OESDKExtTestContext, OESDKExtTestContextExecutor
+
+ pn = d.getVar("PN", True)
+ logger = make_logger_bitbake_compatible(logging.getLogger("BitBake"))
# extensible sdk use network
export_proxies(d)
@@ -143,23 +115,27 @@ def testsdkext_main(d):
d.getVar('BASE_WORKDIR')]
os.environ['PATH'] = avoid_paths_in_environ(paths_to_avoid)
- pn = d.getVar("PN")
- bb.utils.mkdirhier(d.getVar("TEST_LOG_SDKEXT_DIR"))
-
tcname = d.expand("${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}.sh")
if not os.path.exists(tcname):
bb.fatal("The toolchain ext %s is not built. Build it before running the" \
" tests: 'bitbake <image> -c populate_sdk_ext' ." % tcname)
- testdir = d.expand("${WORKDIR}/testsdkext/")
- bb.utils.remove(testdir, True)
- bb.utils.mkdirhier(testdir)
- sdkdir = os.path.join(testdir, 'tc')
+ tdname = d.expand("${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}.testdata.json")
+ test_data = json.load(open(tdname, "r"))
+
+ target_pkg_manifest = OESDKExtTestContextExecutor._load_manifest(
+ d.expand("${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}.target.manifest"))
+ host_pkg_manifest = OESDKExtTestContextExecutor._load_manifest(
+ d.expand("${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}.host.manifest"))
+
+ sdk_dir = d.expand("${WORKDIR}/testsdkext/")
+ bb.utils.remove(sdk_dir, True)
+ bb.utils.mkdirhier(sdk_dir)
try:
- subprocess.check_output("%s -y -d %s" % (tcname, sdkdir), shell=True)
+ subprocess.check_output("%s -y -d %s" % (tcname, sdk_dir), shell=True)
except subprocess.CalledProcessError as e:
msg = "Couldn't install the extensible SDK:\n%s" % e.output.decode("utf-8")
- logfn = os.path.join(sdkdir, 'preparing_build_system.log')
+ logfn = os.path.join(sdk_dir, 'preparing_build_system.log')
if os.path.exists(logfn):
msg += '\n\nContents of preparing_build_system.log:\n'
with open(logfn, 'r') as f:
@@ -167,19 +143,35 @@ def testsdkext_main(d):
msg += line
bb.fatal(msg)
- try:
- bb.plain("Running SDK Compatibility tests ...")
- run_test_context(SDKExtTestContext, d, testdir, tcname, pn, True)
- finally:
- pass
+ fail = False
+ sdk_envs = OESDKExtTestContextExecutor._get_sdk_environs(sdk_dir)
+ for s in sdk_envs:
+ bb.plain("Extensible SDK testing environment: %s" % s)
- try:
- bb.plain("Running Extensible SDK tests ...")
- run_test_context(SDKExtTestContext, d, testdir, tcname, pn)
- finally:
- pass
+ sdk_env = sdk_envs[s]
+ tc = OESDKExtTestContext(td=test_data, logger=logger, sdk_dir=sdk_dir,
+ sdk_env=sdk_env, target_pkg_manifest=target_pkg_manifest,
+ host_pkg_manifest=host_pkg_manifest)
+
+ try:
+ tc.loadTests(OESDKExtTestContextExecutor.default_cases)
+ except Exception as e:
+ import traceback
+ bb.fatal("Loading tests failed:\n%s" % traceback.format_exc())
+
+ result = tc.runTests()
- bb.utils.remove(testdir, True)
+ component = "%s %s" % (pn, OESDKExtTestContextExecutor.name)
+ context_msg = "%s:%s" % (os.path.basename(tcname), os.path.basename(sdk_env))
+
+ tc.logSummary(result, component, context_msg)
+ tc.logDetails()
+
+ if not result.wasSuccessful():
+ fail = True
+
+ if fail:
+ bb.fatal("%s - FAILED - check the task log and the commands log" % pn)
testsdkext_main[vardepsexclude] =+ "BB_ORIGENV"
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 29/55] oeqa/sdkext/cases: Migrate test case to new OEQA framework
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (27 preceding siblings ...)
2017-01-20 17:09 ` [PATCH 28/55] classes/testsdk: Migrate to use the new OESDKExtTestContext Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 30/55] oeqa/runtime: Fix TargetBuildProject instances Aníbal Limón
` (27 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
Summary,
- Changes base case class to OESDKExtTest.
- Changes decorator classes to new ones.
- Chnages variable names sdktestdir -> sdk_dir.
- Added missing license to MIT.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/sdk/cases/buildgalculator.py | 2 +-
meta/lib/oeqa/sdk/cases/gcc.py | 3 +-
meta/lib/oeqa/sdkext/cases/devtool.py | 49 ++++++++++++++++--------------
meta/lib/oeqa/sdkext/cases/sdk_update.py | 17 ++++++-----
4 files changed, 38 insertions(+), 33 deletions(-)
diff --git a/meta/lib/oeqa/sdk/cases/buildgalculator.py b/meta/lib/oeqa/sdk/cases/buildgalculator.py
index bdc8b6a..42e8ddb 100644
--- a/meta/lib/oeqa/sdk/cases/buildgalculator.py
+++ b/meta/lib/oeqa/sdk/cases/buildgalculator.py
@@ -10,7 +10,7 @@ class GalculatorTest(OESDKTestCase):
def setUpClass(self):
if not (self.tc.hasTargetPackage("gtk+3") or\
self.tc.hasTargetPackage("libgtk-3.0")):
- raise unittest.SkipTest("%s class: SDK don't support gtk+3" % self.__name__)
+ raise unittest.SkipTest("GalculatorTest class: SDK don't support gtk+3")
def test_galculator(self):
dl_dir = self.td.get('DL_DIR', None)
diff --git a/meta/lib/oeqa/sdk/cases/gcc.py b/meta/lib/oeqa/sdk/cases/gcc.py
index e06af4c..74ad2a2 100644
--- a/meta/lib/oeqa/sdk/cases/gcc.py
+++ b/meta/lib/oeqa/sdk/cases/gcc.py
@@ -19,8 +19,7 @@ class GccCompileTest(OESDKTestCase):
def setUp(self):
machine = self.td.get("MACHINE")
if not self.tc.hasHostPackage("packagegroup-cross-canadian-%s" % machine):
- raise unittest.SkipTest("%s class: SDK doesn't contain a cross-canadian toolchain",
- self.__name__)
+ raise unittest.SkipTest("GccCompileTest class: SDK doesn't contain a cross-canadian toolchain")
def test_gcc_compile(self):
self._run('$CC %s/test.c -o %s/test -lm' % (self.tc.sdk_dir, self.tc.sdk_dir))
diff --git a/meta/lib/oeqa/sdkext/cases/devtool.py b/meta/lib/oeqa/sdkext/cases/devtool.py
index 65f41f6..da0050c 100644
--- a/meta/lib/oeqa/sdkext/cases/devtool.py
+++ b/meta/lib/oeqa/sdkext/cases/devtool.py
@@ -1,18 +1,22 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
import shutil
import subprocess
-import urllib.request
-from oeqa.oetest import oeSDKExtTest
-from oeqa.utils.decorators import *
-class DevtoolTest(oeSDKExtTest):
+from oeqa.sdkext.case import OESDKExtTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+
+class DevtoolTest(OESDKExtTestCase):
@classmethod
def setUpClass(self):
- self.myapp_src = os.path.join(self.tc.sdkextfilesdir, "myapp")
- self.myapp_dst = os.path.join(self.tc.sdktestdir, "myapp")
+ self.myapp_src = os.path.join(self.tc.esdk_files_dir, "myapp")
+ self.myapp_dst = os.path.join(self.tc.sdk_dir, "myapp")
shutil.copytree(self.myapp_src, self.myapp_dst)
- self.myapp_cmake_src = os.path.join(self.tc.sdkextfilesdir, "myapp_cmake")
- self.myapp_cmake_dst = os.path.join(self.tc.sdktestdir, "myapp_cmake")
+ self.myapp_cmake_src = os.path.join(self.tc.esdk_files_dir, "myapp_cmake")
+ self.myapp_cmake_dst = os.path.join(self.tc.sdk_dir, "myapp_cmake")
shutil.copytree(self.myapp_cmake_src, self.myapp_cmake_dst)
def _test_devtool_build(self, directory):
@@ -37,31 +41,31 @@ class DevtoolTest(oeSDKExtTest):
def test_devtool_location(self):
output = self._run('which devtool')
- self.assertEqual(output.startswith(self.tc.sdktestdir), True, \
+ self.assertEqual(output.startswith(self.tc.sdk_dir), True, \
msg="Seems that devtool isn't the eSDK one: %s" % output)
- @skipUnlessPassed('test_devtool_location')
+ @OETestDepends(['test_devtool_location'])
def test_devtool_add_reset(self):
self._run('devtool add myapp %s' % self.myapp_dst)
self._run('devtool reset myapp')
- @testcase(1473)
- @skipUnlessPassed('test_devtool_location')
+ @OETestID(1473)
+ @OETestDepends(['test_devtool_location'])
def test_devtool_build_make(self):
self._test_devtool_build(self.myapp_dst)
- @testcase(1474)
- @skipUnlessPassed('test_devtool_location')
+ @OETestID(1474)
+ @OETestDepends(['test_devtool_location'])
def test_devtool_build_esdk_package(self):
self._test_devtool_build_package(self.myapp_dst)
- @testcase(1479)
- @skipUnlessPassed('test_devtool_location')
+ @OETestID(1479)
+ @OETestDepends(['test_devtool_location'])
def test_devtool_build_cmake(self):
self._test_devtool_build(self.myapp_cmake_dst)
- @testcase(1482)
- @skipUnlessPassed('test_devtool_location')
+ @OETestID(1482)
+ @OETestDepends(['test_devtool_location'])
def test_extend_autotools_recipe_creation(self):
req = 'https://github.com/rdfa/librdfa'
recipe = "bbexample"
@@ -74,8 +78,8 @@ class DevtoolTest(oeSDKExtTest):
raise e
self._run('devtool reset %s' % recipe)
- @testcase(1484)
- @skipUnlessPassed('test_devtool_location')
+ @OETestID(1484)
+ @OETestDepends(['test_devtool_location'])
def test_devtool_kernelmodule(self):
docfile = 'https://github.com/umlaeute/v4l2loopback.git'
recipe = 'v4l2loopback-driver'
@@ -88,8 +92,8 @@ class DevtoolTest(oeSDKExtTest):
raise e
self._run('devtool reset %s' % recipe)
- @testcase(1478)
- @skipUnlessPassed('test_devtool_location')
+ @OETestID(1478)
+ @OETestDepends(['test_devtool_location'])
def test_recipes_for_nodejs(self):
package_nodejs = "npm://registry.npmjs.org;name=winston;version=2.2.0"
self._run('devtool add %s ' % package_nodejs)
@@ -101,7 +105,6 @@ class DevtoolTest(oeSDKExtTest):
raise e
self._run('devtool reset %s '% package_nodejs)
-
@classmethod
def tearDownClass(self):
shutil.rmtree(self.myapp_dst)
diff --git a/meta/lib/oeqa/sdkext/cases/sdk_update.py b/meta/lib/oeqa/sdkext/cases/sdk_update.py
index 2ade839..2f8598b 100644
--- a/meta/lib/oeqa/sdkext/cases/sdk_update.py
+++ b/meta/lib/oeqa/sdkext/cases/sdk_update.py
@@ -1,23 +1,26 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
import os
import shutil
import subprocess
-from oeqa.oetest import oeSDKExtTest
+from oeqa.sdkext.case import OESDKExtTestCase
from oeqa.utils.httpserver import HTTPService
-class SdkUpdateTest(oeSDKExtTest):
-
+class SdkUpdateTest(OESDKExtTestCase):
@classmethod
def setUpClass(self):
- self.publish_dir = os.path.join(self.tc.sdktestdir, 'esdk_publish')
+ self.publish_dir = os.path.join(self.tc.sdk_dir, 'esdk_publish')
if os.path.exists(self.publish_dir):
shutil.rmtree(self.publish_dir)
os.mkdir(self.publish_dir)
- tcname_new = self.tc.d.expand(
- "${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}-new.sh")
+ base_tcname = "%s/%s" % (self.td.get("SDK_DEPLOY", ''),
+ self.td.get("TOOLCHAINEXT_OUTPUTNAME", ''))
+ tcname_new = "%s-new.sh" % base_tcname
if not os.path.exists(tcname_new):
- tcname_new = self.tc.tcname
+ tcname_new = "%s.sh" % base_tcname
cmd = 'oe-publish-sdk %s %s' % (tcname_new, self.publish_dir)
subprocess.check_output(cmd, shell=True)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 30/55] oeqa/runtime: Fix TargetBuildProject instances
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (28 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 29/55] oeqa/sdkext/cases: Migrate test case to new OEQA framework Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 31/55] oeqa: Fix files handling on runtime tests Aníbal Limón
` (26 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
TargetBuildProject was refactored to avoid bitbake dependency so
the instance don't allow to pass data store anymore.
classes/testimage: Export proxies before run tests
The TargetBuildProject based tests download archives from network.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/classes/testimage.bbclass | 4 ++++
meta/lib/oeqa/runtime/buildcvs.py | 9 +++++----
meta/lib/oeqa/runtime/buildgalculator.py | 8 +++++---
meta/lib/oeqa/runtime/buildiptables.py | 8 +++++---
4 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 7eb4038..6fed408 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -129,11 +129,15 @@ def testimage_main(d):
from oeqa.oetest import ImageTestContext
from oeqa.targetcontrol import get_target_controller
from oeqa.utils.dump import get_host_dumper
+ from bb.utils import export_proxies
pn = d.getVar("PN")
bb.utils.mkdirhier(d.getVar("TEST_LOG_DIR"))
test_create_extract_dirs(d)
+ # runtime use network for download projects for build
+ export_proxies(d)
+
# we need the host dumper in test context
host_dumper = get_host_dumper(d)
diff --git a/meta/lib/oeqa/runtime/buildcvs.py b/meta/lib/oeqa/runtime/buildcvs.py
index fe6cbfb..a5ca3a5 100644
--- a/meta/lib/oeqa/runtime/buildcvs.py
+++ b/meta/lib/oeqa/runtime/buildcvs.py
@@ -1,6 +1,6 @@
from oeqa.oetest import oeRuntimeTest, skipModule
from oeqa.utils.decorators import *
-from oeqa.utils.targetbuild import TargetBuildProject
+from oeqa.runtime.utils.targetbuildproject import TargetBuildProject
def setUpModule():
if not oeRuntimeTest.hasFeature("tools-sdk"):
@@ -10,9 +10,10 @@ class BuildCvsTest(oeRuntimeTest):
@classmethod
def setUpClass(self):
- self.project = TargetBuildProject(oeRuntimeTest.tc.target, oeRuntimeTest.tc.d,
- "http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2")
- self.project.download_archive()
+ dl_dir = oeRuntimeTest.tc.d.getVar('DL_DIR', True)
+ self.project = TargetBuildProject(oeRuntimeTest.tc.target,
+ "http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2",
+ dl_dir=dl_dir)
@testcase(205)
@skipUnlessPassed("test_ssh")
diff --git a/meta/lib/oeqa/runtime/buildgalculator.py b/meta/lib/oeqa/runtime/buildgalculator.py
index 220101d..20f0a79 100644
--- a/meta/lib/oeqa/runtime/buildgalculator.py
+++ b/meta/lib/oeqa/runtime/buildgalculator.py
@@ -1,6 +1,6 @@
from oeqa.oetest import oeRuntimeTest, skipModule
from oeqa.utils.decorators import *
-from oeqa.utils.targetbuild import TargetBuildProject
+from oeqa.runtime.utils.targetbuildproject import TargetBuildProject
def setUpModule():
if not oeRuntimeTest.hasFeature("tools-sdk"):
@@ -10,9 +10,11 @@ class GalculatorTest(oeRuntimeTest):
@testcase(1526)
@skipUnlessPassed("test_ssh")
def test_galculator(self):
+ dl_dir = oeRuntimeTest.tc.d.getVar('DL_DIR', True)
try:
- project = TargetBuildProject(oeRuntimeTest.tc.target, oeRuntimeTest.tc.d,
- "http://galculator.mnim.org/downloads/galculator-2.1.4.tar.bz2")
+ project = TargetBuildProject(oeRuntimeTest.tc.target,
+ "http://galculator.mnim.org/downloads/galculator-2.1.4.tar.bz2",
+ dl_dir=dl_dir)
project.download_archive()
self.assertEqual(project.run_configure(), 0,
diff --git a/meta/lib/oeqa/runtime/buildiptables.py b/meta/lib/oeqa/runtime/buildiptables.py
index bc75d0a..a0e82f0 100644
--- a/meta/lib/oeqa/runtime/buildiptables.py
+++ b/meta/lib/oeqa/runtime/buildiptables.py
@@ -1,6 +1,6 @@
from oeqa.oetest import oeRuntimeTest, skipModule
from oeqa.utils.decorators import *
-from oeqa.utils.targetbuild import TargetBuildProject
+from oeqa.runtime.utils.targetbuildproject import TargetBuildProject
def setUpModule():
if not oeRuntimeTest.hasFeature("tools-sdk"):
@@ -10,8 +10,10 @@ class BuildIptablesTest(oeRuntimeTest):
@classmethod
def setUpClass(self):
- self.project = TargetBuildProject(oeRuntimeTest.tc.target, oeRuntimeTest.tc.d,
- "http://downloads.yoctoproject.org/mirror/sources/iptables-1.4.13.tar.bz2")
+ dl_dir = oeRuntimeTest.tc.d.getVar('DL_DIR', True)
+ self.project = TargetBuildProject(oeRuntimeTest.tc.target,
+ "http://downloads.yoctoproject.org/mirror/sources/iptables-1.4.13.tar.bz2",
+ dl_dir=dl_dir)
self.project.download_archive()
@testcase(206)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 31/55] oeqa: Fix files handling on runtime tests.
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (29 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 30/55] oeqa/runtime: Fix TargetBuildProject instances Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 32/55] oeqa/runtime: Move to runtime_cases Aníbal Limón
` (25 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
Common files was move to oeqa/files from oeqa/runtime/files
because the same files are used across Runtime,SDK,eSDK tests.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/classes/testexport.bbclass | 5 +++++
meta/lib/oeqa/oetest.py | 1 +
meta/lib/oeqa/runtime/gcc.py | 4 ++--
meta/lib/oeqa/runtime/perl.py | 2 +-
meta/lib/oeqa/runtime/python.py | 2 +-
5 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/meta/classes/testexport.bbclass b/meta/classes/testexport.bbclass
index e287f5a..00cf24e 100644
--- a/meta/classes/testexport.bbclass
+++ b/meta/classes/testexport.bbclass
@@ -87,6 +87,7 @@ def exportTests(d,tc):
# - the contents of oeqa/utils and oeqa/runtime/files
# - oeqa/oetest.py and oeqa/runexport.py (this will get copied to exportpath not exportpath/oeqa)
# - __init__.py files
+ bb.utils.mkdirhier(os.path.join(exportpath, "oeqa/files"))
bb.utils.mkdirhier(os.path.join(exportpath, "oeqa/runtime/files"))
bb.utils.mkdirhier(os.path.join(exportpath, "oeqa/utils"))
# copy test modules, this should cover tests in other layers too
@@ -124,6 +125,10 @@ def exportTests(d,tc):
for f in files:
if f.endswith(".py"):
shutil.copy2(os.path.join(root, f), os.path.join(exportpath, "oeqa/utils"))
+ # copy oeqa/files/*
+ for root, dirs, files in os.walk(os.path.join(oeqadir, "files")):
+ for f in files:
+ shutil.copy2(os.path.join(root, f), os.path.join(exportpath, "oeqa/files"))
# copy oeqa/runtime/files/*
for root, dirs, files in os.walk(os.path.join(oeqadir, "runtime/files")):
for f in files:
diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index 503f6fc..b886130 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -201,6 +201,7 @@ class TestContext(object):
self.testsrequired = self._get_test_suites_required()
self.filesdir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "runtime/files")
+ self.corefilesdir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "files")
self.imagefeatures = d.getVar("IMAGE_FEATURES").split()
self.distrofeatures = d.getVar("DISTRO_FEATURES").split()
diff --git a/meta/lib/oeqa/runtime/gcc.py b/meta/lib/oeqa/runtime/gcc.py
index d90cd17..6edb89f 100644
--- a/meta/lib/oeqa/runtime/gcc.py
+++ b/meta/lib/oeqa/runtime/gcc.py
@@ -12,9 +12,9 @@ class GccCompileTest(oeRuntimeTest):
@classmethod
def setUpClass(self):
- oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.filesdir, "test.c"), "/tmp/test.c")
+ oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.corefilesdir, "test.c"), "/tmp/test.c")
oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.filesdir, "testmakefile"), "/tmp/testmakefile")
- oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.filesdir, "test.cpp"), "/tmp/test.cpp")
+ oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.corefilesdir, "test.cpp"), "/tmp/test.cpp")
@testcase(203)
def test_gcc_compile(self):
diff --git a/meta/lib/oeqa/runtime/perl.py b/meta/lib/oeqa/runtime/perl.py
index e044d0a..6bf98f1 100644
--- a/meta/lib/oeqa/runtime/perl.py
+++ b/meta/lib/oeqa/runtime/perl.py
@@ -12,7 +12,7 @@ class PerlTest(oeRuntimeTest):
@classmethod
def setUpClass(self):
- oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.filesdir, "test.pl"), "/tmp/test.pl")
+ oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.corefilesdir, "test.pl"), "/tmp/test.pl")
@testcase(1141)
def test_perl_exists(self):
diff --git a/meta/lib/oeqa/runtime/python.py b/meta/lib/oeqa/runtime/python.py
index 29a231c..93e822c 100644
--- a/meta/lib/oeqa/runtime/python.py
+++ b/meta/lib/oeqa/runtime/python.py
@@ -12,7 +12,7 @@ class PythonTest(oeRuntimeTest):
@classmethod
def setUpClass(self):
- oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.filesdir, "test.py"), "/tmp/test.py")
+ oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.corefilesdir, "test.py"), "/tmp/test.py")
@testcase(1145)
def test_python_exists(self):
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 32/55] oeqa/runtime: Move to runtime_cases
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (30 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 31/55] oeqa: Fix files handling on runtime tests Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 33/55] oeqa/core/target Add OESSHTarget to sent commands to targets using SSH Aníbal Limón
` (24 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
The new oeqa core framework will modify the structure of the runtime
folder the new runtime folder will have python code inside to support
runtime test cases.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/{runtime => runtime_cases}/_ptest.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/_qemutiny.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/buildcvs.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/buildgalculator.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/buildiptables.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/connman.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/date.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/df.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/files/hellomod.c | 0
meta/lib/oeqa/{runtime => runtime_cases}/files/hellomod_makefile | 0
meta/lib/oeqa/{runtime => runtime_cases}/files/testmakefile | 0
meta/lib/oeqa/{runtime => runtime_cases}/gcc.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/kernelmodule.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/ldd.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/logrotate.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/multilib.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/pam.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/parselogs.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/perl.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/ping.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/python.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/rpm.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/scanelf.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/scp.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/skeletoninit.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/smart.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/ssh.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/syslog.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/systemd.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/x32lib.py | 0
meta/lib/oeqa/{runtime => runtime_cases}/xorg.py | 0
31 files changed, 0 insertions(+), 0 deletions(-)
rename meta/lib/oeqa/{runtime => runtime_cases}/_ptest.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/_qemutiny.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/buildcvs.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/buildgalculator.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/buildiptables.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/connman.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/date.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/df.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/files/hellomod.c (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/files/hellomod_makefile (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/files/testmakefile (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/gcc.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/kernelmodule.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/ldd.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/logrotate.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/multilib.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/pam.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/parselogs.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/perl.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/ping.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/python.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/rpm.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/scanelf.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/scp.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/skeletoninit.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/smart.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/ssh.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/syslog.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/systemd.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/x32lib.py (100%)
rename meta/lib/oeqa/{runtime => runtime_cases}/xorg.py (100%)
diff --git a/meta/lib/oeqa/runtime/_ptest.py b/meta/lib/oeqa/runtime_cases/_ptest.py
similarity index 100%
rename from meta/lib/oeqa/runtime/_ptest.py
rename to meta/lib/oeqa/runtime_cases/_ptest.py
diff --git a/meta/lib/oeqa/runtime/_qemutiny.py b/meta/lib/oeqa/runtime_cases/_qemutiny.py
similarity index 100%
rename from meta/lib/oeqa/runtime/_qemutiny.py
rename to meta/lib/oeqa/runtime_cases/_qemutiny.py
diff --git a/meta/lib/oeqa/runtime/buildcvs.py b/meta/lib/oeqa/runtime_cases/buildcvs.py
similarity index 100%
rename from meta/lib/oeqa/runtime/buildcvs.py
rename to meta/lib/oeqa/runtime_cases/buildcvs.py
diff --git a/meta/lib/oeqa/runtime/buildgalculator.py b/meta/lib/oeqa/runtime_cases/buildgalculator.py
similarity index 100%
rename from meta/lib/oeqa/runtime/buildgalculator.py
rename to meta/lib/oeqa/runtime_cases/buildgalculator.py
diff --git a/meta/lib/oeqa/runtime/buildiptables.py b/meta/lib/oeqa/runtime_cases/buildiptables.py
similarity index 100%
rename from meta/lib/oeqa/runtime/buildiptables.py
rename to meta/lib/oeqa/runtime_cases/buildiptables.py
diff --git a/meta/lib/oeqa/runtime/connman.py b/meta/lib/oeqa/runtime_cases/connman.py
similarity index 100%
rename from meta/lib/oeqa/runtime/connman.py
rename to meta/lib/oeqa/runtime_cases/connman.py
diff --git a/meta/lib/oeqa/runtime/date.py b/meta/lib/oeqa/runtime_cases/date.py
similarity index 100%
rename from meta/lib/oeqa/runtime/date.py
rename to meta/lib/oeqa/runtime_cases/date.py
diff --git a/meta/lib/oeqa/runtime/df.py b/meta/lib/oeqa/runtime_cases/df.py
similarity index 100%
rename from meta/lib/oeqa/runtime/df.py
rename to meta/lib/oeqa/runtime_cases/df.py
diff --git a/meta/lib/oeqa/runtime/files/hellomod.c b/meta/lib/oeqa/runtime_cases/files/hellomod.c
similarity index 100%
rename from meta/lib/oeqa/runtime/files/hellomod.c
rename to meta/lib/oeqa/runtime_cases/files/hellomod.c
diff --git a/meta/lib/oeqa/runtime/files/hellomod_makefile b/meta/lib/oeqa/runtime_cases/files/hellomod_makefile
similarity index 100%
rename from meta/lib/oeqa/runtime/files/hellomod_makefile
rename to meta/lib/oeqa/runtime_cases/files/hellomod_makefile
diff --git a/meta/lib/oeqa/runtime/files/testmakefile b/meta/lib/oeqa/runtime_cases/files/testmakefile
similarity index 100%
rename from meta/lib/oeqa/runtime/files/testmakefile
rename to meta/lib/oeqa/runtime_cases/files/testmakefile
diff --git a/meta/lib/oeqa/runtime/gcc.py b/meta/lib/oeqa/runtime_cases/gcc.py
similarity index 100%
rename from meta/lib/oeqa/runtime/gcc.py
rename to meta/lib/oeqa/runtime_cases/gcc.py
diff --git a/meta/lib/oeqa/runtime/kernelmodule.py b/meta/lib/oeqa/runtime_cases/kernelmodule.py
similarity index 100%
rename from meta/lib/oeqa/runtime/kernelmodule.py
rename to meta/lib/oeqa/runtime_cases/kernelmodule.py
diff --git a/meta/lib/oeqa/runtime/ldd.py b/meta/lib/oeqa/runtime_cases/ldd.py
similarity index 100%
rename from meta/lib/oeqa/runtime/ldd.py
rename to meta/lib/oeqa/runtime_cases/ldd.py
diff --git a/meta/lib/oeqa/runtime/logrotate.py b/meta/lib/oeqa/runtime_cases/logrotate.py
similarity index 100%
rename from meta/lib/oeqa/runtime/logrotate.py
rename to meta/lib/oeqa/runtime_cases/logrotate.py
diff --git a/meta/lib/oeqa/runtime/multilib.py b/meta/lib/oeqa/runtime_cases/multilib.py
similarity index 100%
rename from meta/lib/oeqa/runtime/multilib.py
rename to meta/lib/oeqa/runtime_cases/multilib.py
diff --git a/meta/lib/oeqa/runtime/pam.py b/meta/lib/oeqa/runtime_cases/pam.py
similarity index 100%
rename from meta/lib/oeqa/runtime/pam.py
rename to meta/lib/oeqa/runtime_cases/pam.py
diff --git a/meta/lib/oeqa/runtime/parselogs.py b/meta/lib/oeqa/runtime_cases/parselogs.py
similarity index 100%
rename from meta/lib/oeqa/runtime/parselogs.py
rename to meta/lib/oeqa/runtime_cases/parselogs.py
diff --git a/meta/lib/oeqa/runtime/perl.py b/meta/lib/oeqa/runtime_cases/perl.py
similarity index 100%
rename from meta/lib/oeqa/runtime/perl.py
rename to meta/lib/oeqa/runtime_cases/perl.py
diff --git a/meta/lib/oeqa/runtime/ping.py b/meta/lib/oeqa/runtime_cases/ping.py
similarity index 100%
rename from meta/lib/oeqa/runtime/ping.py
rename to meta/lib/oeqa/runtime_cases/ping.py
diff --git a/meta/lib/oeqa/runtime/python.py b/meta/lib/oeqa/runtime_cases/python.py
similarity index 100%
rename from meta/lib/oeqa/runtime/python.py
rename to meta/lib/oeqa/runtime_cases/python.py
diff --git a/meta/lib/oeqa/runtime/rpm.py b/meta/lib/oeqa/runtime_cases/rpm.py
similarity index 100%
rename from meta/lib/oeqa/runtime/rpm.py
rename to meta/lib/oeqa/runtime_cases/rpm.py
diff --git a/meta/lib/oeqa/runtime/scanelf.py b/meta/lib/oeqa/runtime_cases/scanelf.py
similarity index 100%
rename from meta/lib/oeqa/runtime/scanelf.py
rename to meta/lib/oeqa/runtime_cases/scanelf.py
diff --git a/meta/lib/oeqa/runtime/scp.py b/meta/lib/oeqa/runtime_cases/scp.py
similarity index 100%
rename from meta/lib/oeqa/runtime/scp.py
rename to meta/lib/oeqa/runtime_cases/scp.py
diff --git a/meta/lib/oeqa/runtime/skeletoninit.py b/meta/lib/oeqa/runtime_cases/skeletoninit.py
similarity index 100%
rename from meta/lib/oeqa/runtime/skeletoninit.py
rename to meta/lib/oeqa/runtime_cases/skeletoninit.py
diff --git a/meta/lib/oeqa/runtime/smart.py b/meta/lib/oeqa/runtime_cases/smart.py
similarity index 100%
rename from meta/lib/oeqa/runtime/smart.py
rename to meta/lib/oeqa/runtime_cases/smart.py
diff --git a/meta/lib/oeqa/runtime/ssh.py b/meta/lib/oeqa/runtime_cases/ssh.py
similarity index 100%
rename from meta/lib/oeqa/runtime/ssh.py
rename to meta/lib/oeqa/runtime_cases/ssh.py
diff --git a/meta/lib/oeqa/runtime/syslog.py b/meta/lib/oeqa/runtime_cases/syslog.py
similarity index 100%
rename from meta/lib/oeqa/runtime/syslog.py
rename to meta/lib/oeqa/runtime_cases/syslog.py
diff --git a/meta/lib/oeqa/runtime/systemd.py b/meta/lib/oeqa/runtime_cases/systemd.py
similarity index 100%
rename from meta/lib/oeqa/runtime/systemd.py
rename to meta/lib/oeqa/runtime_cases/systemd.py
diff --git a/meta/lib/oeqa/runtime/x32lib.py b/meta/lib/oeqa/runtime_cases/x32lib.py
similarity index 100%
rename from meta/lib/oeqa/runtime/x32lib.py
rename to meta/lib/oeqa/runtime_cases/x32lib.py
diff --git a/meta/lib/oeqa/runtime/xorg.py b/meta/lib/oeqa/runtime_cases/xorg.py
similarity index 100%
rename from meta/lib/oeqa/runtime/xorg.py
rename to meta/lib/oeqa/runtime_cases/xorg.py
--
2.1.4
^ permalink raw reply [flat|nested] 59+ messages in thread* [PATCH 33/55] oeqa/core/target Add OESSHTarget to sent commands to targets using SSH
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (31 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 32/55] oeqa/runtime: Move to runtime_cases Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 34/55] oeqa/runtime: Add case, context and loader classes for runtime testing Aníbal Limón
` (23 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
With this commit now it is possible to add targets with SSH for testing.
Most of it was imported for existing code, with improvements in log
handling.
[YOCTO #10234]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/target/__init__.py | 33 +++++
meta/lib/oeqa/core/target/ssh.py | 266 ++++++++++++++++++++++++++++++++++
2 files changed, 299 insertions(+)
create mode 100644 meta/lib/oeqa/core/target/__init__.py
create mode 100644 meta/lib/oeqa/core/target/ssh.py
diff --git a/meta/lib/oeqa/core/target/__init__.py b/meta/lib/oeqa/core/target/__init__.py
new file mode 100644
index 0000000..d2468bc
--- /dev/null
+++ b/meta/lib/oeqa/core/target/__init__.py
@@ -0,0 +1,33 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from abc import abstractmethod
+
+class OETarget(object):
+
+ def __init__(self, logger, *args, **kwargs):
+ self.logger = logger
+
+ @abstractmethod
+ def start(self):
+ pass
+
+ @abstractmethod
+ def stop(self):
+ pass
+
+ @abstractmethod
+ def run(self, cmd, timeout=None):
+ pass
+
+ @abstractmethod
+ def copyTo(self, localSrc, remoteDst):
+ pass
+
+ @abstractmethod
+ def copyFrom(self, remoteSrc, localDst):
+ pass
+
+ @abstractmethod
+ def copyDirTo(self, localSrc, remoteDst):
+ pass
diff --git a/meta/lib/oeqa/core/target/ssh.py b/meta/lib/oeqa/core/target/ssh.py
new file mode 100644
index 0000000..b80939c
--- /dev/null
+++ b/meta/lib/oeqa/core/target/ssh.py
@@ -0,0 +1,266 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import time
+import select
+import logging
+import subprocess
+
+from . import OETarget
+
+class OESSHTarget(OETarget):
+ def __init__(self, logger, ip, server_ip, timeout=300, user='root',
+ port=None, **kwargs):
+ if not logger:
+ logger = logging.getLogger('target')
+ logger.setLevel(logging.INFO)
+ filePath = os.path.join(os.getcwd(), 'remoteTarget.log')
+ fileHandler = logging.FileHandler(filePath, 'w', 'utf-8')
+ formatter = logging.Formatter(
+ '%(asctime)s.%(msecs)03d %(levelname)s: %(message)s',
+ '%H:%M:%S')
+ fileHandler.setFormatter(formatter)
+ logger.addHandler(fileHandler)
+
+ super(OESSHTarget, self).__init__(logger)
+ self.ip = ip
+ self.server_ip = server_ip
+ self.timeout = timeout
+ self.user = user
+ ssh_options = [
+ '-o', 'UserKnownHostsFile=/dev/null',
+ '-o', 'StrictHostKeyChecking=no',
+ '-o', 'LogLevel=ERROR'
+ ]
+ self.ssh = ['ssh', '-l', self.user ] + ssh_options
+ self.scp = ['scp'] + ssh_options
+ if port:
+ self.ssh = self.ssh + [ '-p', port ]
+ self.scp = self.scp + [ '-P', port ]
+
+ def start(self, **kwargs):
+ pass
+
+ def stop(self, **kwargs):
+ pass
+
+ def _run(self, command, timeout=None, ignore_status=True):
+ """
+ Runs command in target using SSHProcess.
+ """
+ self.logger.debug("[Running]$ %s" % " ".join(command))
+
+ starttime = time.time()
+ status, output = SSHCall(command, self.logger, timeout)
+ self.logger.debug("[Command returned '%d' after %.2f seconds]"
+ "" % (status, time.time() - starttime))
+
+ if status and not ignore_status:
+ raise AssertionError("Command '%s' returned non-zero exit "
+ "status %d:\n%s" % (command, status, output))
+
+ return (status, output)
+
+ def run(self, command, timeout=None):
+ """
+ Runs command in target.
+
+ command: Command to run on target.
+ timeout: <value>: Kill command after <val> seconds.
+ None: Kill command default value seconds.
+ 0: No timeout, runs until return.
+ """
+ targetCmd = 'export PATH=/usr/sbin:/sbin:/usr/bin:/bin; %s' % command
+ sshCmd = self.ssh + [self.ip, targetCmd]
+
+ if timeout:
+ processTimeout = timeout
+ elif timeout==0:
+ processTimeout = None
+ else:
+ processTimeout = self.timeout
+
+ status, output = self._run(sshCmd, processTimeout, True)
+ self.logger.info('\nCommand: %s\nOutput: %s\n' % (command, output))
+ return (status, output)
+
+ def copyTo(self, localSrc, remoteDst):
+ """
+ Copy file to target.
+
+ If local file is symlink, recreate symlink in target.
+ """
+ if os.path.islink(localSrc):
+ link = os.readlink(localSrc)
+ dstDir, dstBase = os.path.split(remoteDst)
+ sshCmd = 'cd %s; ln -s %s %s' % (dstDir, link, dstBase)
+ return self.run(sshCmd)
+
+ else:
+ remotePath = '%s@%s:%s' % (self.user, self.ip, remoteDst)
+ scpCmd = self.scp + [localSrc, remotePath]
+ return self._run(scpCmd, ignore_status=False)
+
+ def copyFrom(self, remoteSrc, localDst):
+ """
+ Copy file from target.
+ """
+ remotePath = '%s@%s:%s' % (self.user, self.ip, remoteSrc)
+ scpCmd = self.scp + [remotePath, localDst]
+ return self._run(scpCmd, ignore_status=False)
+
+ def copyDirTo(self, localSrc, remoteDst):
+ """
+ Copy recursively localSrc directory to remoteDst in target.
+ """
+
+ for root, dirs, files in os.walk(localSrc):
+ # Create directories in the target as needed
+ for d in dirs:
+ tmpDir = os.path.join(root, d).replace(localSrc, "")
+ newDir = os.path.join(remoteDst, tmpDir.lstrip("/"))
+ cmd = "mkdir -p %s" % newDir
+ self.run(cmd)
+
+ # Copy files into the target
+ for f in files:
+ tmpFile = os.path.join(root, f).replace(localSrc, "")
+ dstFile = os.path.join(remoteDst, tmpFile.lstrip("/"))
+ srcFile = os.path.join(root, f)
+ self.copyTo(srcFile, dstFile)
+
+ def deleteFiles(self, remotePath, files):
+ """
+ Deletes files in target's remotePath.
+ """
+
+ cmd = "rm"
+ if not isinstance(files, list):
+ files = [files]
+
+ for f in files:
+ cmd = "%s %s" % (cmd, os.path.join(remotePath, f))
+
+ self.run(cmd)
+
+
+ def deleteDir(self, remotePath):
+ """
+ Deletes target's remotePath directory.
+ """
+
+ cmd = "rmdir %s" % remotePath
+ self.run(cmd)
+
+
+ def deleteDirStructure(self, localPath, remotePath):
+ """
+ Delete recursively localPath structure directory in target's remotePath.
+
+ This function is very usefult to delete a package that is installed in
+ the DUT and the host running the test has such package extracted in tmp
+ directory.
+
+ Example:
+ pwd: /home/user/tmp
+ tree: .
+ └── work
+ ├── dir1
+ │ └── file1
+ └── dir2
+
+ localpath = "/home/user/tmp" and remotepath = "/home/user"
+
+ With the above variables this function will try to delete the
+ directory in the DUT in this order:
+ /home/user/work/dir1/file1
+ /home/user/work/dir1 (if dir is empty)
+ /home/user/work/dir2 (if dir is empty)
+ /home/user/work (if dir is empty)
+ """
+
+ for root, dirs, files in os.walk(localPath, topdown=False):
+ # Delete files first
+ tmpDir = os.path.join(root).replace(localPath, "")
+ remoteDir = os.path.join(remotePath, tmpDir.lstrip("/"))
+ self.deleteFiles(remoteDir, files)
+
+ # Remove dirs if empty
+ for d in dirs:
+ tmpDir = os.path.join(root, d).replace(localPath, "")
+ remoteDir = os.path.join(remotePath, tmpDir.lstrip("/"))
+ self.deleteDir(remoteDir)
+
+def SSHCall(command, logger, timeout=None, **opts):
+
+ def run():
+ nonlocal output
+ nonlocal process
+ starttime = time.time()
+ process = subprocess.Popen(command, **options)
+ if timeout:
+ endtime = starttime + timeout
+ eof = False
+ while time.time() < endtime and not eof:
+ logger.debug('time: %s, endtime: %s' % (time.time(), endtime))
+ try:
+ if select.select([process.stdout], [], [], 5)[0] != []:
+ data = os.read(process.stdout.fileno(), 1024)
+ if not data:
+ process.stdout.close()
+ eof = True
+ else:
+ data = data.decode("utf-8")
+ output += data
+ logger.debug('Partial data from SSH call: %s' % data)
+ endtime = time.time() + timeout
+ except InterruptedError:
+ continue
+
+ # process hasn't returned yet
+ if not eof:
+ process.terminate()
+ time.sleep(5)
+ try:
+ process.kill()
+ except OSError:
+ pass
+ endtime = time.time() - starttime
+ lastline = ("\nProcess killed - no output for %d seconds. Total"
+ " running time: %d seconds." % (timeout, endtime))
+ logger.debug('Received data from SSH call %s ' % lastline)
+ output += lastline
+
+ else:
+ output = process.communicate()[0].decode("utf-8")
+ logger.debug('Data from SSH call: %s' % output.rstrip())
+
+ options = {
+ "stdout": subprocess.PIPE,
+ "stderr": subprocess.STDOUT,
+ "stdin": None,
+ "shell": False,
+ "bufsize": -1,
+ "preexec_fn": os.setsid,
+ }
+ options.update(opts)
+ output = ''
+ process = None
+
+ # Unset DISPLAY which means we won't trigger SSH_ASKPASS
+ env = os.environ.copy()
+ if "DISPLAY" in env:
+ del env['DISPLAY']
+ options['env'] = env
+
+ try:
+ run()
+ except:
+ # Need to guard against a SystemExit or other exception ocurring
+ # whilst running and ensure we don't leave a process behind.
+ if process.poll() is None:
+ process.kill()
+ logger.debug('Something went wrong, killing SSH process')
+ raise
+ return (process.wait(), output.rstrip())
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 34/55] oeqa/runtime: Add case, context and loader classes for runtime testing
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (32 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 33/55] oeqa/core/target Add OESSHTarget to sent commands to targets using SSH Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 35/55] oeqa/runtime: Add OEHasPackage decorator Aníbal Limón
` (22 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This adds OERuntimeTestCase, OERuntimeTestContext, and OERuntimeTestLoader
to be used for runtime testing.
As expected there are some changes in runtime context:
- Adds the target to be used for runtime testing, the default
is a SSH connection to the device under test running a OE image.
- Runtime context requires image manifest because several
tests are skipped if a package is missing or installed.
- Several tests require the output of the ps command and it changes
its output and arguments if busybox o procps is installed, so the
case must use the correct ps command.
[YOCTO #10234]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/runtime/case.py | 8 +++++
meta/lib/oeqa/runtime/context.py | 69 ++++++++++++++++++++++++++++++++++++++++
meta/lib/oeqa/runtime/loader.py | 16 ++++++++++
3 files changed, 93 insertions(+)
create mode 100644 meta/lib/oeqa/runtime/case.py
create mode 100644 meta/lib/oeqa/runtime/context.py
create mode 100644 meta/lib/oeqa/runtime/loader.py
diff --git a/meta/lib/oeqa/runtime/case.py b/meta/lib/oeqa/runtime/case.py
new file mode 100644
index 0000000..43f1b2f
--- /dev/null
+++ b/meta/lib/oeqa/runtime/case.py
@@ -0,0 +1,8 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.case import OETestCase
+
+class OERuntimeTestCase(OETestCase):
+ # target instance set by OERuntimeTestLoader.
+ target = None
diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
new file mode 100644
index 0000000..496730d
--- /dev/null
+++ b/meta/lib/oeqa/runtime/context.py
@@ -0,0 +1,69 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+
+from oeqa.core.context import OETestContext, OETestContextExecutor
+from oeqa.core.target.ssh import OESSHTarget
+from oeqa.runtime.loader import OERuntimeTestLoader
+
+class OERuntimeTestContext(OETestContext):
+ loaderClass = OERuntimeTestLoader
+
+ def __init__(self, td, logger, target, packages_manifest):
+ super(OERuntimeTestContext, self).__init__(td, logger)
+ self.target = target
+ self.image_packages = self.readPackagesManifest(packages_manifest)
+ self._set_target_cmds()
+
+ def _set_target_cmds(self):
+ self.target_cmds = {}
+
+ self.target_cmds['ps'] = 'ps'
+ if 'procps' in self.image_packages:
+ self.target_cmds['ps'] = self.target_cmds['ps'] + ' -ef'
+
+ def readPackagesManifest(self, manifest):
+ if not os.path.exists(manifest):
+ raise OSError("Couldn't find manifest file: %s" % manifest)
+
+ image_packages = set()
+ with open(manifest, 'r') as f:
+ for line in f.readlines():
+ line = line.strip()
+ if line and not line.startswith("#"):
+ image_packages.add(line.split()[0])
+
+ return image_packages
+
+class OERuntimeTestContextExecutor(OETestContextExecutor):
+ _context_class = OERuntimeTestContext
+
+ name = 'runtime'
+ help = 'runtime test component'
+ description = 'executes runtime tests over targets'
+ default_cases = os.path.join(os.path.abspath(os.path.dirname(__file__)),
+ 'cases')
+ default_target_ip = '192.168.7.2'
+
+ def register_commands(self, logger, subparsers):
+ super(OERuntimeTestContextExecutor, self).register_commands(logger, subparsers)
+ self.parser.add_argument('--target-ip', action='store',
+ default=self.default_target_ip,
+ help="IP address of device under test, default: %s" \
+ % self.default_target_ip)
+ self.parser.add_argument('--packages-manifest', action='store',
+ help="Package manifest of the image under test")
+
+ def _process_args(self, logger, args):
+ if not args.packages_manifest:
+ raise TypeError('Manifest file not provided')
+
+ super(OERuntimeTestContextExecutor, self)._process_args(logger, args)
+ target = OESSHTarget(args.target_ip)
+ self.tc_kwargs['init']['target'] = target
+
+ packages_manifest = os.path.join(os.getcwd(), args.packages_manifest)
+ self.tc_kwargs['init']['packages_manifest'] = packages_manifest
+
+_executor_class = OERuntimeTestContextExecutor
diff --git a/meta/lib/oeqa/runtime/loader.py b/meta/lib/oeqa/runtime/loader.py
new file mode 100644
index 0000000..041ef97
--- /dev/null
+++ b/meta/lib/oeqa/runtime/loader.py
@@ -0,0 +1,16 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.loader import OETestLoader
+from oeqa.runtime.case import OERuntimeTestCase
+
+class OERuntimeTestLoader(OETestLoader):
+ caseClass = OERuntimeTestCase
+
+ def _getTestCase(self, testCaseClass, tcName):
+ case = super(OERuntimeTestLoader, self)._getTestCase(testCaseClass, tcName)
+
+ # Adds custom attributes to the OERuntimeTestCase
+ setattr(case, 'target', self.tc.target)
+
+ return case
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 35/55] oeqa/runtime: Add OEHasPackage decorator
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (33 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 34/55] oeqa/runtime: Add case, context and loader classes for runtime testing Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 36/55] oeqa/core/decorator/data.py: Add skipIfNotFeature decorator Aníbal Limón
` (21 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This new decorator will be used to skip the test
if the image under test doesn't have the required
packages installed.
[YOCTO #10234]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/runtime/decorator/package.py | 53 ++++++++++++++++++++++++++++++
1 file changed, 53 insertions(+)
create mode 100644 meta/lib/oeqa/runtime/decorator/package.py
diff --git a/meta/lib/oeqa/runtime/decorator/package.py b/meta/lib/oeqa/runtime/decorator/package.py
new file mode 100644
index 0000000..aa6ecb6
--- /dev/null
+++ b/meta/lib/oeqa/runtime/decorator/package.py
@@ -0,0 +1,53 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from oeqa.core.decorator import OETestDecorator, registerDecorator
+from oeqa.core.utils.misc import strToSet
+
+@registerDecorator
+class OEHasPackage(OETestDecorator):
+ """
+ Checks if image has packages (un)installed.
+
+ The argument must be a string, set, or list of packages that must be
+ installed or not present in the image.
+
+ The way to tell a package must not be in an image is using an
+ exclamation point ('!') before the name of the package.
+
+ If test depends on pkg1 or pkg2 you need to use:
+ @OEHasPackage({'pkg1', 'pkg2'})
+
+ If test depends on pkg1 and pkg2 you need to use:
+ @OEHasPackage('pkg1')
+ @OEHasPackage('pkg2')
+
+ If test depends on pkg1 but pkg2 must not be present use:
+ @OEHasPackage({'pkg1', '!pkg2'})
+ """
+
+ attrs = ('need_pkgs',)
+
+ def setUpDecorator(self):
+ need_pkgs = set()
+ unneed_pkgs = set()
+ pkgs = strToSet(self.need_pkgs)
+ for pkg in pkgs:
+ if pkg.startswith('!'):
+ unneed_pkgs.add(pkg[1:])
+ else:
+ need_pkgs.add(pkg)
+
+ if unneed_pkgs:
+ msg = 'Checking if %s is not installed' % ', '.join(unneed_pkgs)
+ self.logger.debug(msg)
+ if not self.case.tc.image_packages.isdisjoint(unneed_pkgs):
+ msg = "Test can't run with %s installed" % ', or'.join(unneed_pkgs)
+ self.case.skipTest(msg)
+
+ if need_pkgs:
+ msg = 'Checking if at least one of %s is installed' % ', '.join(need_pkgs)
+ self.logger.debug(msg)
+ if self.case.tc.image_packages.isdisjoint(need_pkgs):
+ msg = "Test requires %s to be installed" % ', or'.join(need_pkgs)
+ self.case.skipTest(msg)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 36/55] oeqa/core/decorator/data.py: Add skipIfNotFeature decorator
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (34 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 35/55] oeqa/runtime: Add OEHasPackage decorator Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 37/55] oeqa/runtime/files: Move runtime files from old directory Aníbal Limón
` (20 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This adds a new decorator to check if image under tests has
certain DISTRO_FEATURE or IMAGE_FEATURE.
[YOCTO #10234]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/decorator/data.py | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/meta/lib/oeqa/core/decorator/data.py b/meta/lib/oeqa/core/decorator/data.py
index 73cca88..fdeba9f 100644
--- a/meta/lib/oeqa/core/decorator/data.py
+++ b/meta/lib/oeqa/core/decorator/data.py
@@ -5,6 +5,16 @@ from oeqa.core.exception import OEQAMissingVariable
from . import OETestDecorator, registerDecorator
+def has_feature(td, feature):
+ """
+ Checks for feature in DISTRO_FEATURES or IMAGE_FEATURES.
+ """
+
+ if (feature in td.get('DISTRO_FEATURES', '') or
+ feature in td.get('IMAGE_FEATURES', '')):
+ return True
+ return False
+
@registerDecorator
class skipIfDataVar(OETestDecorator):
"""
@@ -34,3 +44,21 @@ class OETestDataDepends(OETestDecorator):
except KeyError:
raise OEQAMissingVariable("Test case need %s variable but"\
" isn't into td" % v)
+
+@registerDecorator
+class skipIfNotFeature(OETestDecorator):
+ """
+ Skip test based on DISTRO_FEATURES.
+
+ value must be in distro features or it will skip the test
+ with msg as the reason.
+ """
+
+ attrs = ('value', 'msg')
+
+ def setUpDecorator(self):
+ msg = ('Checking if %s is in DISTRO_FEATURES '
+ 'or IMAGE_FEATURES' % (self.value))
+ self.logger.debug(msg)
+ if not has_feature(self.case.td, self.value):
+ self.case.skipTest(self.msg)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 37/55] oeqa/runtime/files: Move runtime files from old directory
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (35 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 36/55] oeqa/core/decorator/data.py: Add skipIfNotFeature decorator Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 38/55] oeqa/runtime/utils/targetbuildproject.py: Don't use more than 80 characters per line Aníbal Limón
` (19 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
As part of the refactor we require to move the files used
in runtime testing to the new directory. This also adds
the path to the runtime test context.
[YOCTO #10234]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/runtime/context.py | 2 ++
meta/lib/oeqa/{runtime_cases => runtime}/files/hellomod.c | 0
meta/lib/oeqa/{runtime_cases => runtime}/files/hellomod_makefile | 0
meta/lib/oeqa/{runtime_cases => runtime}/files/testmakefile | 0
4 files changed, 2 insertions(+)
rename meta/lib/oeqa/{runtime_cases => runtime}/files/hellomod.c (100%)
rename meta/lib/oeqa/{runtime_cases => runtime}/files/hellomod_makefile (100%)
rename meta/lib/oeqa/{runtime_cases => runtime}/files/testmakefile (100%)
diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
index 496730d..10a8872 100644
--- a/meta/lib/oeqa/runtime/context.py
+++ b/meta/lib/oeqa/runtime/context.py
@@ -9,6 +9,8 @@ from oeqa.runtime.loader import OERuntimeTestLoader
class OERuntimeTestContext(OETestContext):
loaderClass = OERuntimeTestLoader
+ runtime_files_dir = os.path.join(
+ os.path.dirname(os.path.abspath(__file__)), "files")
def __init__(self, td, logger, target, packages_manifest):
super(OERuntimeTestContext, self).__init__(td, logger)
diff --git a/meta/lib/oeqa/runtime_cases/files/hellomod.c b/meta/lib/oeqa/runtime/files/hellomod.c
similarity index 100%
rename from meta/lib/oeqa/runtime_cases/files/hellomod.c
rename to meta/lib/oeqa/runtime/files/hellomod.c
diff --git a/meta/lib/oeqa/runtime_cases/files/hellomod_makefile b/meta/lib/oeqa/runtime/files/hellomod_makefile
similarity index 100%
rename from meta/lib/oeqa/runtime_cases/files/hellomod_makefile
rename to meta/lib/oeqa/runtime/files/hellomod_makefile
diff --git a/meta/lib/oeqa/runtime_cases/files/testmakefile b/meta/lib/oeqa/runtime/files/testmakefile
similarity index 100%
rename from meta/lib/oeqa/runtime_cases/files/testmakefile
rename to meta/lib/oeqa/runtime/files/testmakefile
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 38/55] oeqa/runtime/utils/targetbuildproject.py: Don't use more than 80 characters per line
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (36 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 37/55] oeqa/runtime/files: Move runtime files from old directory Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 39/55] oeqa/core/decorator: Add skipIfNotDataVar and skipIfNotInDataVar Aníbal Limón
` (18 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/runtime/utils/targetbuildproject.py | 29 +++++++++++++----------
1 file changed, 16 insertions(+), 13 deletions(-)
diff --git a/meta/lib/oeqa/runtime/utils/targetbuildproject.py b/meta/lib/oeqa/runtime/utils/targetbuildproject.py
index 006d4d4..1ed5789 100644
--- a/meta/lib/oeqa/runtime/utils/targetbuildproject.py
+++ b/meta/lib/oeqa/runtime/utils/targetbuildproject.py
@@ -14,20 +14,23 @@ class TargetBuildProject(BuildProject):
def download_archive(self):
self._download_archive()
- (status, output) = self.target.copy_to(self.localarchive, self.targetdir)
- if status != 0:
- raise Exception("Failed to copy archive to target, output: %s" % output)
-
- (status, output) = self.target.run('tar xf %s%s -C %s' % (self.targetdir, self.archive, self.targetdir))
- if status != 0:
- raise Exception("Failed to extract archive, output: %s" % output)
-
- #Change targetdir to project folder
+ status, output = self.target.copyTo(self.localarchive, self.targetdir)
+ if status:
+ raise Exception('Failed to copy archive to target, '
+ 'output: %s' % output)
+
+ cmd = 'tar xf %s%s -C %s' % (self.targetdir,
+ self.archive,
+ self.targetdir)
+ status, output = self.target.run(cmd)
+ if status:
+ raise Exception('Failed to extract archive, '
+ 'output: %s' % output)
+
+ # Change targetdir to project folder
self.targetdir = self.targetdir + self.fname
- # The timeout parameter of target.run is set to 0 to make the ssh command
- # run with no timeout.
+ # The timeout parameter of target.run is set to 0
+ # to make the ssh command run with no timeout.
def _run(self, cmd):
return self.target.run(cmd, 0)[0]
-
-
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 39/55] oeqa/core/decorator: Add skipIfNotDataVar and skipIfNotInDataVar
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (37 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 38/55] oeqa/runtime/utils/targetbuildproject.py: Don't use more than 80 characters per line Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 40/55] oeqa/core/context: Add option to select tests to run Aníbal Limón
` (17 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
skipIfNotDataVar will skip a test if a variable doesn't have certain value.
skipIfNotInDataVar will skip a test if a value is not in a certain variable.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/decorator/data.py | 36 +++++++++++++++++++++++++++++++++++-
1 file changed, 35 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oeqa/core/decorator/data.py b/meta/lib/oeqa/core/decorator/data.py
index fdeba9f..ff7bdd9 100644
--- a/meta/lib/oeqa/core/decorator/data.py
+++ b/meta/lib/oeqa/core/decorator/data.py
@@ -28,12 +28,46 @@ class skipIfDataVar(OETestDecorator):
attrs = ('var', 'value', 'msg')
def setUpDecorator(self):
- msg = 'Checking if %r value is %r to skip test' % (self.var, self.value)
+ msg = ('Checking if %r value is %r to skip test' %
+ (self.var, self.value))
self.logger.debug(msg)
if self.case.td.get(self.var) == self.value:
self.case.skipTest(self.msg)
@registerDecorator
+class skipIfNotDataVar(OETestDecorator):
+ """
+ Skip test based on value of a data store's variable.
+
+ It will get the info of var from the data store and will
+ check it against value; if are not equal it will skip the
+ test with msg as the reason.
+ """
+
+ attrs = ('var', 'value', 'msg')
+
+ def setUpDecorator(self):
+ msg = ('Checking if %r value is not %r to skip test' %
+ (self.var, self.value))
+ self.logger.debug(msg)
+ if not self.case.td.get(self.var) == self.value:
+ self.case.skipTest(self.msg)
+
+@registerDecorator
+class skipIfNotInDataVar(OETestDecorator):
+ """
+ Skip test if value is not in data store's variable.
+ """
+
+ attrs = ('var', 'value', 'msg')
+ def setUpDecorator(self):
+ msg = ('Checking if %r value is in %r to run '
+ 'the test' % (self.var, self.value))
+ self.logger.debug(msg)
+ if not self.value in self.case.td.get(self.var):
+ self.case.skipTest(self.msg)
+
+@registerDecorator
class OETestDataDepends(OETestDecorator):
attrs = ('td_depends',)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 40/55] oeqa/core/context: Add option to select tests to run
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (38 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 39/55] oeqa/core/decorator: Add skipIfNotDataVar and skipIfNotInDataVar Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 41/55] oeqa/runtime/context: Add runtime option group and options for target type and server ip Aníbal Limón
` (16 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This add the option to select what tests to run in the
<module>[.<class>[.<test>]] format.
Currently it just support modules
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/context.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/meta/lib/oeqa/core/context.py b/meta/lib/oeqa/core/context.py
index ba6ccf8..efed4e6 100644
--- a/meta/lib/oeqa/core/context.py
+++ b/meta/lib/oeqa/core/context.py
@@ -157,6 +157,7 @@ class OETestContextExecutor(object):
default_cases = [os.path.join(os.path.abspath(os.path.dirname(__file__)),
'cases/example')]
default_test_data = os.path.join(default_cases[0], 'data.json')
+ default_tests = None
def register_commands(self, logger, subparsers):
self.parser = subparsers.add_parser(self.name, help=self.help,
@@ -167,6 +168,9 @@ class OETestContextExecutor(object):
self.parser.add_argument('--output-log', action='store',
default=self.default_output_log,
help="results output log, default: %s" % self.default_output_log)
+ self.parser.add_argument('--run-tests', action='store',
+ default=self.default_tests,
+ help="tests to run in <module>[.<class>[.<name>]] format. Just works for modules now")
if self.default_test_data:
self.parser.add_argument('--test-data-file', action='store',
@@ -211,6 +215,8 @@ class OETestContextExecutor(object):
else:
self.tc_kwargs['init']['td'] = {}
+ self.tc_kwargs['load']['modules'] = args.run_tests.split()
+
self.module_paths = args.CASES_PATHS
def run(self, logger, args):
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 41/55] oeqa/runtime/context: Add runtime option group and options for target type and server ip.
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (39 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 40/55] oeqa/core/context: Add option to select tests to run Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 42/55] oeqa/runtime/context: Move helper functions for process args to executor Aníbal Limón
` (15 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/runtime/context.py | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
index 10a8872..e692bc0 100644
--- a/meta/lib/oeqa/runtime/context.py
+++ b/meta/lib/oeqa/runtime/context.py
@@ -44,17 +44,34 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
name = 'runtime'
help = 'runtime test component'
description = 'executes runtime tests over targets'
+
default_cases = os.path.join(os.path.abspath(os.path.dirname(__file__)),
'cases')
+ default_data = None
+
+ default_target_type = 'simpleremote'
+ default_server_ip = '192.168.7.1'
default_target_ip = '192.168.7.2'
def register_commands(self, logger, subparsers):
super(OERuntimeTestContextExecutor, self).register_commands(logger, subparsers)
- self.parser.add_argument('--target-ip', action='store',
+
+ runtime_group = self.parser.add_argument_group('runtime options')
+
+ runtime_group.add_argument('--target-type', action='store',
+ default=self.default_target_type, choices=['simpleremote', 'qemu'],
+ help="Target type of device under test, default: %s" \
+ % self.default_target_type)
+ runtime_group.add_argument('--target-ip', action='store',
default=self.default_target_ip,
help="IP address of device under test, default: %s" \
% self.default_target_ip)
- self.parser.add_argument('--packages-manifest', action='store',
+ runtime_group.add_argument('--server-ip', action='store',
+ default=self.default_target_ip,
+ help="IP address of device under test, default: %s" \
+ % self.default_server_ip)
+
+ runtime_group.add_argument('--packages-manifest', action='store',
help="Package manifest of the image under test")
def _process_args(self, logger, args):
@@ -63,6 +80,7 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
super(OERuntimeTestContextExecutor, self)._process_args(logger, args)
target = OESSHTarget(args.target_ip)
+
self.tc_kwargs['init']['target'] = target
packages_manifest = os.path.join(os.getcwd(), args.packages_manifest)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 42/55] oeqa/runtime/context: Move helper functions for process args to executor
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (40 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 41/55] oeqa/runtime/context: Add runtime option group and options for target type and server ip Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 43/55] oeqa/utils/dump: Move get_host_dumper to OERuntimeTestContextExecutor class Aníbal Limón
` (14 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/runtime/context.py | 56 ++++++++++++++++++++++++++--------------
1 file changed, 36 insertions(+), 20 deletions(-)
diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
index e692bc0..ec378bb 100644
--- a/meta/lib/oeqa/runtime/context.py
+++ b/meta/lib/oeqa/runtime/context.py
@@ -12,10 +12,10 @@ class OERuntimeTestContext(OETestContext):
runtime_files_dir = os.path.join(
os.path.dirname(os.path.abspath(__file__)), "files")
- def __init__(self, td, logger, target, packages_manifest):
+ def __init__(self, td, logger, target, image_packages):
super(OERuntimeTestContext, self).__init__(td, logger)
self.target = target
- self.image_packages = self.readPackagesManifest(packages_manifest)
+ self.image_packages = image_packages
self._set_target_cmds()
def _set_target_cmds(self):
@@ -25,19 +25,6 @@ class OERuntimeTestContext(OETestContext):
if 'procps' in self.image_packages:
self.target_cmds['ps'] = self.target_cmds['ps'] + ' -ef'
- def readPackagesManifest(self, manifest):
- if not os.path.exists(manifest):
- raise OSError("Couldn't find manifest file: %s" % manifest)
-
- image_packages = set()
- with open(manifest, 'r') as f:
- for line in f.readlines():
- line = line.strip()
- if line and not line.startswith("#"):
- image_packages.add(line.split()[0])
-
- return image_packages
-
class OERuntimeTestContextExecutor(OETestContextExecutor):
_context_class = OERuntimeTestContext
@@ -74,16 +61,45 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
runtime_group.add_argument('--packages-manifest', action='store',
help="Package manifest of the image under test")
+
+ @staticmethod
+ def getTarget(target_type, target_ip, server_ip):
+ target = None
+
+ if target_type == 'simpleremote':
+ target = OESSHTarget(target_ip)
+ elif target_type == 'qemu':
+ raise NotImplementedError("target_type %s isn't implemented yet" % \
+ target_type)
+ else:
+ raise TypeError("target_type %s isn't supported" % target_type)
+
+ return target
+
+ @staticmethod
+ def readPackagesManifest(manifest):
+ if not os.path.exists(manifest):
+ raise OSError("Manifest file not exists: %s" % manifest)
+
+ image_packages = set()
+ with open(manifest, 'r') as f:
+ for line in f.readlines():
+ line = line.strip()
+ if line and not line.startswith("#"):
+ image_packages.add(line.split()[0])
+
+ return image_packages
+
def _process_args(self, logger, args):
if not args.packages_manifest:
raise TypeError('Manifest file not provided')
super(OERuntimeTestContextExecutor, self)._process_args(logger, args)
- target = OESSHTarget(args.target_ip)
-
- self.tc_kwargs['init']['target'] = target
- packages_manifest = os.path.join(os.getcwd(), args.packages_manifest)
- self.tc_kwargs['init']['packages_manifest'] = packages_manifest
+ self.tc_kwargs['init']['target'] = \
+ OERuntimeTestContextExecutor.getTarget(args.target_type)
+ self.tc_kwargs['init']['image_packages'] = \
+ OERuntimeTestContextExecutor.readPackagesManifest(
+ args.packages_manifest)
_executor_class = OERuntimeTestContextExecutor
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 43/55] oeqa/utils/dump: Move get_host_dumper to OERuntimeTestContextExecutor class
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (41 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 42/55] oeqa/runtime/context: Move helper functions for process args to executor Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 44/55] oeqa/runtime/context.py: Prepare for qemu Aníbal Limón
` (13 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
To avoid getVar calls inside a utils module, also moves
get_host_dumper import inside testexport isn't needed.
[YOCTO #10231]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/classes/testexport.bbclass | 1 -
meta/classes/testimage.bbclass | 5 +++--
meta/lib/oeqa/runtime/context.py | 18 +++++++++++++++++-
meta/lib/oeqa/utils/dump.py | 11 ++---------
4 files changed, 22 insertions(+), 13 deletions(-)
diff --git a/meta/classes/testexport.bbclass b/meta/classes/testexport.bbclass
index 00cf24e..5398b3c 100644
--- a/meta/classes/testexport.bbclass
+++ b/meta/classes/testexport.bbclass
@@ -163,7 +163,6 @@ def exportTests(d,tc):
def testexport_main(d):
from oeqa.oetest import ExportTestContext
from oeqa.targetcontrol import get_target_controller
- from oeqa.utils.dump import get_host_dumper
test_create_extract_dirs(d)
export_dir = d.getVar("TEST_EXPORT_DIR")
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 6fed408..62ecaef 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -128,8 +128,8 @@ def testimage_main(d):
import signal
from oeqa.oetest import ImageTestContext
from oeqa.targetcontrol import get_target_controller
- from oeqa.utils.dump import get_host_dumper
from bb.utils import export_proxies
+ from oeqa.utils.dump import HostDumper
pn = d.getVar("PN")
bb.utils.mkdirhier(d.getVar("TEST_LOG_DIR"))
@@ -139,7 +139,8 @@ def testimage_main(d):
export_proxies(d)
# we need the host dumper in test context
- host_dumper = get_host_dumper(d)
+ host_dumper = HostDumper(d.getVar("testimage_dump_host", True),
+ d.getVar("TESTIMAGE_DUMP_DIR", True))
# the robot dance
target = get_target_controller(d)
diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
index ec378bb..ab7caa6 100644
--- a/meta/lib/oeqa/runtime/context.py
+++ b/meta/lib/oeqa/runtime/context.py
@@ -5,6 +5,8 @@ import os
from oeqa.core.context import OETestContext, OETestContextExecutor
from oeqa.core.target.ssh import OESSHTarget
+from oeqa.utils.dump import HostDumper
+
from oeqa.runtime.loader import OERuntimeTestLoader
class OERuntimeTestContext(OETestContext):
@@ -12,10 +14,11 @@ class OERuntimeTestContext(OETestContext):
runtime_files_dir = os.path.join(
os.path.dirname(os.path.abspath(__file__)), "files")
- def __init__(self, td, logger, target, image_packages):
+ def __init__(self, td, logger, target, host_dumper, image_packages):
super(OERuntimeTestContext, self).__init__(td, logger)
self.target = target
self.image_packages = image_packages
+ self.host_dumper = host_dumper
self._set_target_cmds()
def _set_target_cmds(self):
@@ -39,6 +42,7 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
default_target_type = 'simpleremote'
default_server_ip = '192.168.7.1'
default_target_ip = '192.168.7.2'
+ default_host_dumper_dir = '/tmp/oe-saved-tests'
def register_commands(self, logger, subparsers):
super(OERuntimeTestContextExecutor, self).register_commands(logger, subparsers)
@@ -58,6 +62,11 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
help="IP address of device under test, default: %s" \
% self.default_server_ip)
+ runtime_group.add_argument('--host-dumper-dir', action='store',
+ default=self.default_host_dumper_dir,
+ help="Directory where host status is dumped, if tests fails, default: %s" \
+ % self.default_host_dumper_dir)
+
runtime_group.add_argument('--packages-manifest', action='store',
help="Package manifest of the image under test")
@@ -90,6 +99,10 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
return image_packages
+ @staticmethod
+ def getHostDumper(cmds, directory):
+ return HostDumper(cmds, directory)
+
def _process_args(self, logger, args):
if not args.packages_manifest:
raise TypeError('Manifest file not provided')
@@ -98,6 +111,9 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
self.tc_kwargs['init']['target'] = \
OERuntimeTestContextExecutor.getTarget(args.target_type)
+ self.tc_kwargs['init']['host_dumper'] = \
+ OERuntimeTestContextExecutor.getHostDumper(None,
+ args.host_dumper_dir)
self.tc_kwargs['init']['image_packages'] = \
OERuntimeTestContextExecutor.readPackagesManifest(
args.packages_manifest)
diff --git a/meta/lib/oeqa/utils/dump.py b/meta/lib/oeqa/utils/dump.py
index 44037a9..5a7edc1 100644
--- a/meta/lib/oeqa/utils/dump.py
+++ b/meta/lib/oeqa/utils/dump.py
@@ -5,12 +5,6 @@ import datetime
import itertools
from .commands import runCmd
-def get_host_dumper(d):
- cmds = d.getVar("testimage_dump_host")
- parent_dir = d.getVar("TESTIMAGE_DUMP_DIR")
- return HostDumper(cmds, parent_dir)
-
-
class BaseDumper(object):
""" Base class to dump commands from host/target """
@@ -77,13 +71,12 @@ class HostDumper(BaseDumper):
result = runCmd(cmd, ignore_status=True)
self._write_dump(cmd.split()[0], result.output)
-
class TargetDumper(BaseDumper):
""" Class to get dumps from target, it only works with QemuRunner """
- def __init__(self, cmds, parent_dir, qemurunner):
+ def __init__(self, cmds, parent_dir, runner):
super(TargetDumper, self).__init__(cmds, parent_dir)
- self.runner = qemurunner
+ self.runner = runner
def dump_target(self, dump_dir=""):
if dump_dir:
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 44/55] oeqa/runtime/context.py: Prepare for qemu
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (42 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 43/55] oeqa/utils/dump: Move get_host_dumper to OERuntimeTestContextExecutor class Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 45/55] oeqa/runtime/context.py: Add logger to getTarget Aníbal Limón
` (12 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
This commit will prepare context to use qemu and not just
simpleremote.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/runtime/context.py | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
index ab7caa6..f0f6e62 100644
--- a/meta/lib/oeqa/runtime/context.py
+++ b/meta/lib/oeqa/runtime/context.py
@@ -5,6 +5,7 @@ import os
from oeqa.core.context import OETestContext, OETestContextExecutor
from oeqa.core.target.ssh import OESSHTarget
+from oeqa.core.target.qemu import OEQemuTarget
from oeqa.utils.dump import HostDumper
from oeqa.runtime.loader import OERuntimeTestLoader
@@ -16,6 +17,7 @@ class OERuntimeTestContext(OETestContext):
def __init__(self, td, logger, target, host_dumper, image_packages):
super(OERuntimeTestContext, self).__init__(td, logger)
+
self.target = target
self.image_packages = image_packages
self.host_dumper = host_dumper
@@ -70,17 +72,19 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
runtime_group.add_argument('--packages-manifest', action='store',
help="Package manifest of the image under test")
+ runtime_group.add_argument('--qemu-boot', action='store',
+ help="Qemu boot configuration, only needed when target_type is QEMU.")
@staticmethod
- def getTarget(target_type, target_ip, server_ip):
+ def getTarget(target_type, target_ip, server_ip, **kwargs):
target = None
if target_type == 'simpleremote':
- target = OESSHTarget(target_ip)
+ target = OESSHTarget(target_ip, server_ip, kwargs)
elif target_type == 'qemu':
- raise NotImplementedError("target_type %s isn't implemented yet" % \
- target_type)
+ target = OEQemuTarget(target_ip, server_ip, kwargs)
else:
+ # TODO: Implement custom target module loading
raise TypeError("target_type %s isn't supported" % target_type)
return target
@@ -109,8 +113,12 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
super(OERuntimeTestContextExecutor, self)._process_args(logger, args)
+ target_kwargs = {}
+ target_kwargs['qemuboot'] = args.qemu_boot
+
self.tc_kwargs['init']['target'] = \
- OERuntimeTestContextExecutor.getTarget(args.target_type)
+ OERuntimeTestContextExecutor.getTarget(args.target_type,
+ args.target_ip, args.server_ip, **target_kwargs)
self.tc_kwargs['init']['host_dumper'] = \
OERuntimeTestContextExecutor.getHostDumper(None,
args.host_dumper_dir)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 45/55] oeqa/runtime/context.py: Add logger to getTarget
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (43 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 44/55] oeqa/runtime/context.py: Prepare for qemu Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 46/55] core/target/qemu.py Adds qemu target Aníbal Limón
` (11 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
Current targets (ssh and qemu) require a logger in their
constructors, so in order to get a new target we need
to provide the logger.
[YOCTO #10686]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/runtime/context.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
index f0f6e62..bc8abd0 100644
--- a/meta/lib/oeqa/runtime/context.py
+++ b/meta/lib/oeqa/runtime/context.py
@@ -76,13 +76,13 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
help="Qemu boot configuration, only needed when target_type is QEMU.")
@staticmethod
- def getTarget(target_type, target_ip, server_ip, **kwargs):
+ def getTarget(target_type, logger, target_ip, server_ip, **kwargs):
target = None
if target_type == 'simpleremote':
- target = OESSHTarget(target_ip, server_ip, kwargs)
+ target = OESSHTarget(logger, target_ip, server_ip, **kwargs)
elif target_type == 'qemu':
- target = OEQemuTarget(target_ip, server_ip, kwargs)
+ target = OEQemuTarget(logger, target_ip, server_ip, **kwargs)
else:
# TODO: Implement custom target module loading
raise TypeError("target_type %s isn't supported" % target_type)
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 46/55] core/target/qemu.py Adds qemu target
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (44 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 45/55] oeqa/runtime/context.py: Add logger to getTarget Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 47/55] testimage.bbclass: Migrate class to use new runtime framework Aníbal Limón
` (10 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This adds qemu target to be used in testimage.
It uses the current QemuRunner class in order
to boot and control qemu.
[YOCTO #10231]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/target/qemu.py | 45 +++++++++++++++++++++++++++++++++++++++
1 file changed, 45 insertions(+)
create mode 100644 meta/lib/oeqa/core/target/qemu.py
diff --git a/meta/lib/oeqa/core/target/qemu.py b/meta/lib/oeqa/core/target/qemu.py
new file mode 100644
index 0000000..641dd6a
--- /dev/null
+++ b/meta/lib/oeqa/core/target/qemu.py
@@ -0,0 +1,45 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import sys
+import signal
+import time
+
+from .ssh import OESSHTarget
+from oeqa.utils.qemurunner import QemuRunner
+
+supported_fstypes = ['ext3', 'ext4', 'cpio.gz', 'wic']
+
+class OEQemuTarget(OESSHTarget):
+ def __init__(self, logger, ip, server_ip, timeout=300, user='root',
+ port=None, machine='', rootfs='', kernel='', kvm=False,
+ dump_dir='', dump_host_cmds='', display='', bootlog='',
+ tmpdir='', dir_image='', boottime=60):
+
+ super(OEQemuTarget, self).__init__(logger, ip, server_ip, timeout,
+ user, port)
+
+ self.ip = ip
+ self.server_ip = server_ip
+ self.machine = machine
+ self.rootfs = rootfs
+ self.kernel = kernel
+ self.kvm = kvm
+
+ self.runner = QemuRunner(machine=machine, rootfs=rootfs, tmpdir=tmpdir,
+ deploy_dir_image=dir_image, display=display,
+ logfile=bootlog, boottime=boottime,
+ use_kvm=kvm, dump_dir=dump_dir,
+ dump_host_cmds=dump_host_cmds)
+
+ def start(self, params=None, extra_bootparams=None):
+ if self.runner.start(params, extra_bootparams=extra_bootparams):
+ self.ip = self.runner.ip
+ self.server_ip = self.runner.server_ip
+ else:
+ self.stop()
+ raise RuntimeError("FAILED to start qemu - check the task log and the boot log")
+
+ def stop(self):
+ self.runner.stop()
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 47/55] testimage.bbclass: Migrate class to use new runtime framework
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (45 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 46/55] core/target/qemu.py Adds qemu target Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 48/55] oeqa/runtime/cases: Migrate runtime tests Aníbal Limón
` (9 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This migrates testimage class to use the new framework. Most of
the code added here is to get rid off the data store dependency.
[YOCTO #10234]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/classes/testimage.bbclass | 186 +++++++++++++++++++++++++++++++----------
1 file changed, 144 insertions(+), 42 deletions(-)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 62ecaef..016c1c1 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -77,6 +77,7 @@ TESTIMAGEDEPENDS_qemuall += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'creat
TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'python-smartpm-native:do_populate_sysroot', '', d)}"
TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'ipk', 'opkg-utils-native:do_populate_sysroot', '', d)}"
TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'deb', 'apt-native:do_populate_sysroot', '', d)}"
+TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'python-smartpm-native:do_populate_sysroot', '', d)}"
TESTIMAGELOCK = "${TMPDIR}/testimage.lock"
@@ -112,6 +113,9 @@ testimage_dump_host () {
}
python do_testimage() {
+
+ testimage_sanity(d)
+
testimage_main(d)
}
@@ -120,67 +124,165 @@ do_testimage[nostamp] = "1"
do_testimage[depends] += "${TESTIMAGEDEPENDS}"
do_testimage[lockfiles] += "${TESTIMAGELOCK}"
+def testimage_sanity(d):
+ if (d.getVar('TEST_TARGET') == 'simpleremote'
+ and (not d.getVar('TEST_TARGET_IP')
+ or not d.getVar('TEST_SERVER_IP'))):
+ bb.fatal('When TEST_TARGET is set to "simpleremote" '
+ 'TEST_TARGET_IP and TEST_SERVER_IP are needed too.')
+
def testimage_main(d):
- import unittest
import os
- import oeqa.runtime
- import time
import signal
- from oeqa.oetest import ImageTestContext
- from oeqa.targetcontrol import get_target_controller
- from bb.utils import export_proxies
- from oeqa.utils.dump import HostDumper
+ import json
+ import sys
+ import logging
+ import time
+ from bb.utils import export_proxies
+ from oeqa.runtime.context import OERuntimeTestContext
+ from oeqa.runtime.context import OERuntimeTestContextExecutor
+ from oeqa.core.target.qemu import supported_fstypes
+ from oeqa.utils import make_logger_bitbake_compatible
+
+ def sigterm_exception(signum, stackframe):
+ """
+ Catch SIGTERM from worker in order to stop qemu.
+ """
+ raise RuntimeError
+
+ logger = make_logger_bitbake_compatible(logging.getLogger("BitBake"))
pn = d.getVar("PN")
+
bb.utils.mkdirhier(d.getVar("TEST_LOG_DIR"))
- test_create_extract_dirs(d)
+ #test_create_extract_dirs(d)
+
+ image_name = ("%s/%s" % (d.getVar('DEPLOY_DIR_IMAGE'),
+ d.getVar('IMAGE_LINK_NAME')))
+
+ tdname = "%s.testdata.json" % image_name
+ td = json.load(open(tdname, "r"))
+
+ image_manifest = "%s.manifest" % image_name
+ image_packages = OERuntimeTestContextExecutor.readPackagesManifest(image_manifest)
+
+ # Get machine
+ machine = d.getVar("MACHINE")
+
+ # Get rootfs
+ fstypes = [fs for fs in d.getVar('IMAGE_FSTYPES').split(' ')
+ if fs in supported_fstypes]
+ if not fstypes:
+ bb.fatal('Unsupported image type built. Add a comptible image to '
+ 'IMAGE_FSTYPES. Supported types: %s' %
+ ', '.join(supported_fstypes))
+ rootfs = '%s.%s' % (image_name, fstypes[0])
+
+ # Get tmpdir (not really used, just for compatibility)
+ tmpdir = d.getVar("TMPDIR")
+
+ # Get deploy_dir_image (not really used, just for compatibility)
+ dir_image = d.getVar("DEPLOY_DIR_IMAGE")
+
+ # Get bootlog
+ bootlog = os.path.join(d.getVar("TEST_LOG_DIR"),
+ 'qemu_boot_log.%s' % d.getVar('DATETIME'))
+
+ # Get display
+ display = d.getVar("BB_ORIGENV").getVar("DISPLAY")
+
+ # Get kernel
+ kernel_name = ('%s-%s.bin' % (d.getVar("KERNEL_IMAGETYPE"), machine))
+ kernel = os.path.join(d.getVar("DEPLOY_DIR_IMAGE"), kernel_name)
+
+ # Get boottime
+ boottime = int(d.getVar("TEST_QEMUBOOT_TIMEOUT"))
+
+ # Get use_kvm
+ qemu_use_kvm = d.getVar("QEMU_USE_KVM")
+ if qemu_use_kvm and qemu_use_kvm == 'True' and 'x86' in machine:
+ kvm = True
+ else:
+ kvm = False
+
+ # TODO: We use the current implementatin of qemu runner because of
+ # time constrains, qemu runner really needs a refactor too.
+ target_kwargs = { 'machine' : machine,
+ 'rootfs' : rootfs,
+ 'tmpdir' : tmpdir,
+ 'dir_image' : dir_image,
+ 'display' : display,
+ 'kernel' : kernel,
+ 'boottime' : boottime,
+ 'bootlog' : bootlog,
+ 'kvm' : kvm,
+ }
# runtime use network for download projects for build
export_proxies(d)
# we need the host dumper in test context
- host_dumper = HostDumper(d.getVar("testimage_dump_host", True),
- d.getVar("TESTIMAGE_DUMP_DIR", True))
+ host_dumper = OERuntimeTestContextExecutor.getHostDumper(
+ d.getVar("testimage_dump_host"),
+ d.getVar("TESTIMAGE_DUMP_DIR"))
# the robot dance
- target = get_target_controller(d)
+ target = OERuntimeTestContextExecutor.getTarget(
+ d.getVar("TEST_TARGET"), None, d.getVar("TEST_TARGET_IP"),
+ d.getVar("TEST_SERVER_IP"), **target_kwargs)
# test context
- tc = ImageTestContext(d, target, host_dumper)
+ tc = OERuntimeTestContext(td, logger, target, host_dumper, image_packages)
- # this is a dummy load of tests
- # we are doing that to find compile errors in the tests themselves
- # before booting the image
- try:
- tc.loadTests()
- except Exception as e:
- import traceback
- bb.fatal("Loading tests failed:\n%s" % traceback.format_exc())
+ # Load tests before starting the target
+ test_paths = get_runtime_paths(d)
+ test_modules = d.getVar('TEST_SUITES')
+ tc.loadTests(test_paths, modules=test_modules)
+
+ bootparams = None
+ if d.getVar('VIRTUAL-RUNTIME_init_manager', '') == 'systemd':
+ bootparams = 'systemd.log_level=debug systemd.log_target=console'
- tc.extract_packages()
- target.deploy()
+ results = None
+ orig_sigterm_handler = signal.signal(signal.SIGTERM, sigterm_exception)
try:
- bootparams = None
- if d.getVar('VIRTUAL-RUNTIME_init_manager', '') == 'systemd':
- bootparams = 'systemd.log_level=debug systemd.log_target=console'
- target.start(extra_bootparams=bootparams)
- starttime = time.time()
- result = tc.runTests()
- stoptime = time.time()
- if result.wasSuccessful():
- bb.plain("%s - Ran %d test%s in %.3fs" % (pn, result.testsRun, result.testsRun != 1 and "s" or "", stoptime - starttime))
- msg = "%s - OK - All required tests passed" % pn
- skipped = len(result.skipped)
- if skipped:
- msg += " (skipped=%d)" % skipped
- bb.plain(msg)
+ # We need to check if runqemu ends unexpectedly
+ # or if the worker send us a SIGTERM
+ tc.target.start(extra_bootparams=bootparams)
+ results = tc.runTests()
+ except (RuntimeError, BlockingIOError) as err:
+ if isinstance(err, RuntimeError):
+ bb.error('testimage received SIGTERM, shutting down...')
else:
- bb.fatal("%s - FAILED - check the task log and the ssh log" % pn)
- except BlockingIOError as err:
- bb.error('runqemu failed, shutting down...')
+ bb.error('runqemu failed, shutting down...')
+ if results:
+ results.stop()
+ results = None
finally:
- signal.signal(signal.SIGTERM, tc.origsigtermhandler)
- target.stop()
+ signal.signal(signal.SIGTERM, orig_sigterm_handler)
+ tc.target.stop()
+
+ # Show results (if we have them)
+ if not results:
+ bb.fatal('%s - FAILED - tests were interrupted during execution' % pn)
+ tc.logSummary(results, pn)
+ tc.logDetails()
+ if not results.wasSuccessful():
+ bb.fatal('%s - FAILED - check the task log and the ssh log' % pn)
+
+def get_runtime_paths(d):
+ """
+ Returns a list of paths where runtime test must reside.
+
+ Runtime tests are expected in <LAYER_DIR>/lib/oeqa/runtime/cases/
+ """
+ paths = []
+
+ for layer in d.getVar('BBLAYERS').split():
+ path = os.path.join(layer, 'lib/oeqa/runtime/cases')
+ if os.path.isdir(path):
+ paths.append(path)
+ return paths
def test_create_extract_dirs(d):
install_path = d.getVar("TEST_INSTALL_TMP_DIR")
@@ -193,6 +295,6 @@ def test_create_extract_dirs(d):
bb.utils.mkdirhier(extracted_path)
-testimage_main[vardepsexclude] =+ "BB_ORIGENV"
+testimage_main[vardepsexclude] += "BB_ORIGENV DATETIME"
inherit testsdk
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 48/55] oeqa/runtime/cases: Migrate runtime tests.
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (46 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 47/55] testimage.bbclass: Migrate class to use new runtime framework Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 49/55] runtime/cases/smart.py: Migrate smart tests Aníbal Limón
` (8 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This migrates current runtime test suite to be used with the new framework.
[YOCTO #10234]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
.../{runtime_cases => runtime/cases}/_ptest.py | 0
.../{runtime_cases => runtime/cases}/_qemutiny.py | 0
meta/lib/oeqa/runtime/cases/buildcvs.py | 35 ++++
meta/lib/oeqa/runtime/cases/buildgalculator.py | 31 ++++
meta/lib/oeqa/runtime/cases/buildiptables.py | 39 +++++
meta/lib/oeqa/runtime/cases/connman.py | 30 ++++
meta/lib/oeqa/runtime/cases/date.py | 38 +++++
meta/lib/oeqa/runtime/cases/df.py | 13 ++
meta/lib/oeqa/runtime/cases/gcc.py | 76 +++++++++
meta/lib/oeqa/runtime/cases/kernelmodule.py | 40 +++++
meta/lib/oeqa/runtime/cases/ldd.py | 25 +++
meta/lib/oeqa/runtime/cases/logrotate.py | 42 +++++
meta/lib/oeqa/runtime/cases/multilib.py | 41 +++++
meta/lib/oeqa/runtime/cases/pam.py | 33 ++++
.../{runtime_cases => runtime/cases}/parselogs.py | 187 ++++++++++++---------
meta/lib/oeqa/runtime/cases/perl.py | 37 ++++
meta/lib/oeqa/runtime/cases/ping.py | 24 +++
meta/lib/oeqa/runtime/cases/python.py | 43 +++++
meta/lib/oeqa/runtime/cases/rpm.py | 141 ++++++++++++++++
meta/lib/oeqa/runtime/cases/scanelf.py | 26 +++
meta/lib/oeqa/runtime/cases/scp.py | 33 ++++
meta/lib/oeqa/runtime/cases/skeletoninit.py | 33 ++++
.../oeqa/{runtime_cases => runtime/cases}/smart.py | 0
meta/lib/oeqa/runtime/cases/ssh.py | 15 ++
meta/lib/oeqa/runtime/cases/syslog.py | 57 +++++++
.../{runtime_cases => runtime/cases}/systemd.py | 151 +++++++++--------
meta/lib/oeqa/runtime/cases/x32lib.py | 19 +++
meta/lib/oeqa/runtime/cases/xorg.py | 17 ++
meta/lib/oeqa/runtime_cases/buildcvs.py | 32 ----
meta/lib/oeqa/runtime_cases/buildgalculator.py | 26 ---
meta/lib/oeqa/runtime_cases/buildiptables.py | 33 ----
meta/lib/oeqa/runtime_cases/connman.py | 31 ----
meta/lib/oeqa/runtime_cases/date.py | 31 ----
meta/lib/oeqa/runtime_cases/df.py | 12 --
meta/lib/oeqa/runtime_cases/gcc.py | 47 ------
meta/lib/oeqa/runtime_cases/kernelmodule.py | 34 ----
meta/lib/oeqa/runtime_cases/ldd.py | 21 ---
meta/lib/oeqa/runtime_cases/logrotate.py | 30 ----
meta/lib/oeqa/runtime_cases/multilib.py | 42 -----
meta/lib/oeqa/runtime_cases/pam.py | 25 ---
meta/lib/oeqa/runtime_cases/perl.py | 30 ----
meta/lib/oeqa/runtime_cases/ping.py | 22 ---
meta/lib/oeqa/runtime_cases/python.py | 35 ----
meta/lib/oeqa/runtime_cases/rpm.py | 120 -------------
meta/lib/oeqa/runtime_cases/scanelf.py | 28 ---
meta/lib/oeqa/runtime_cases/scp.py | 22 ---
meta/lib/oeqa/runtime_cases/skeletoninit.py | 29 ----
meta/lib/oeqa/runtime_cases/ssh.py | 19 ---
meta/lib/oeqa/runtime_cases/syslog.py | 52 ------
meta/lib/oeqa/runtime_cases/x32lib.py | 18 --
meta/lib/oeqa/runtime_cases/xorg.py | 16 --
51 files changed, 1075 insertions(+), 906 deletions(-)
rename meta/lib/oeqa/{runtime_cases => runtime/cases}/_ptest.py (100%)
rename meta/lib/oeqa/{runtime_cases => runtime/cases}/_qemutiny.py (100%)
create mode 100644 meta/lib/oeqa/runtime/cases/buildcvs.py
create mode 100644 meta/lib/oeqa/runtime/cases/buildgalculator.py
create mode 100644 meta/lib/oeqa/runtime/cases/buildiptables.py
create mode 100644 meta/lib/oeqa/runtime/cases/connman.py
create mode 100644 meta/lib/oeqa/runtime/cases/date.py
create mode 100644 meta/lib/oeqa/runtime/cases/df.py
create mode 100644 meta/lib/oeqa/runtime/cases/gcc.py
create mode 100644 meta/lib/oeqa/runtime/cases/kernelmodule.py
create mode 100644 meta/lib/oeqa/runtime/cases/ldd.py
create mode 100644 meta/lib/oeqa/runtime/cases/logrotate.py
create mode 100644 meta/lib/oeqa/runtime/cases/multilib.py
create mode 100644 meta/lib/oeqa/runtime/cases/pam.py
rename meta/lib/oeqa/{runtime_cases => runtime/cases}/parselogs.py (64%)
create mode 100644 meta/lib/oeqa/runtime/cases/perl.py
create mode 100644 meta/lib/oeqa/runtime/cases/ping.py
create mode 100644 meta/lib/oeqa/runtime/cases/python.py
create mode 100644 meta/lib/oeqa/runtime/cases/rpm.py
create mode 100644 meta/lib/oeqa/runtime/cases/scanelf.py
create mode 100644 meta/lib/oeqa/runtime/cases/scp.py
create mode 100644 meta/lib/oeqa/runtime/cases/skeletoninit.py
rename meta/lib/oeqa/{runtime_cases => runtime/cases}/smart.py (100%)
create mode 100644 meta/lib/oeqa/runtime/cases/ssh.py
create mode 100644 meta/lib/oeqa/runtime/cases/syslog.py
rename meta/lib/oeqa/{runtime_cases => runtime/cases}/systemd.py (51%)
create mode 100644 meta/lib/oeqa/runtime/cases/x32lib.py
create mode 100644 meta/lib/oeqa/runtime/cases/xorg.py
delete mode 100644 meta/lib/oeqa/runtime_cases/buildcvs.py
delete mode 100644 meta/lib/oeqa/runtime_cases/buildgalculator.py
delete mode 100644 meta/lib/oeqa/runtime_cases/buildiptables.py
delete mode 100644 meta/lib/oeqa/runtime_cases/connman.py
delete mode 100644 meta/lib/oeqa/runtime_cases/date.py
delete mode 100644 meta/lib/oeqa/runtime_cases/df.py
delete mode 100644 meta/lib/oeqa/runtime_cases/gcc.py
delete mode 100644 meta/lib/oeqa/runtime_cases/kernelmodule.py
delete mode 100644 meta/lib/oeqa/runtime_cases/ldd.py
delete mode 100644 meta/lib/oeqa/runtime_cases/logrotate.py
delete mode 100644 meta/lib/oeqa/runtime_cases/multilib.py
delete mode 100644 meta/lib/oeqa/runtime_cases/pam.py
delete mode 100644 meta/lib/oeqa/runtime_cases/perl.py
delete mode 100644 meta/lib/oeqa/runtime_cases/ping.py
delete mode 100644 meta/lib/oeqa/runtime_cases/python.py
delete mode 100644 meta/lib/oeqa/runtime_cases/rpm.py
delete mode 100644 meta/lib/oeqa/runtime_cases/scanelf.py
delete mode 100644 meta/lib/oeqa/runtime_cases/scp.py
delete mode 100644 meta/lib/oeqa/runtime_cases/skeletoninit.py
delete mode 100644 meta/lib/oeqa/runtime_cases/ssh.py
delete mode 100644 meta/lib/oeqa/runtime_cases/syslog.py
delete mode 100644 meta/lib/oeqa/runtime_cases/x32lib.py
delete mode 100644 meta/lib/oeqa/runtime_cases/xorg.py
diff --git a/meta/lib/oeqa/runtime_cases/_ptest.py b/meta/lib/oeqa/runtime/cases/_ptest.py
similarity index 100%
rename from meta/lib/oeqa/runtime_cases/_ptest.py
rename to meta/lib/oeqa/runtime/cases/_ptest.py
diff --git a/meta/lib/oeqa/runtime_cases/_qemutiny.py b/meta/lib/oeqa/runtime/cases/_qemutiny.py
similarity index 100%
rename from meta/lib/oeqa/runtime_cases/_qemutiny.py
rename to meta/lib/oeqa/runtime/cases/_qemutiny.py
diff --git a/meta/lib/oeqa/runtime/cases/buildcvs.py b/meta/lib/oeqa/runtime/cases/buildcvs.py
new file mode 100644
index 0000000..341eb49
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/buildcvs.py
@@ -0,0 +1,35 @@
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfNotFeature
+
+from oeqa.runtime.utils.targetbuildproject import TargetBuildProject
+
+class BuildCvsTest(OERuntimeTestCase):
+
+ @classmethod
+ def setUpClass(cls):
+ uri = 'http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13'
+ uri = '%s/cvs-1.12.13.tar.bz2' % uri
+ cls.project = TargetBuildProject(cls.tc.target,
+ uri,
+ dl_dir = cls.tc.td['DL_DIR'])
+ cls.project.download_archive()
+
+ @classmethod
+ def tearDownClass(cls):
+ cls.project.clean()
+
+ @OETestID(205)
+ @skipIfNotFeature('tools-sdk',
+ 'Test requires tools-sdk to be in IMAGE_FEATURES')
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_cvs(self):
+ self.assertEqual(self.project.run_configure(), 0,
+ msg="Running configure failed")
+
+ self.assertEqual(self.project.run_make(), 0,
+ msg="Running make failed")
+
+ self.assertEqual(self.project.run_install(), 0,
+ msg="Running make install failed")
diff --git a/meta/lib/oeqa/runtime/cases/buildgalculator.py b/meta/lib/oeqa/runtime/cases/buildgalculator.py
new file mode 100644
index 0000000..0bd76f9
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/buildgalculator.py
@@ -0,0 +1,31 @@
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfNotFeature
+
+from oeqa.runtime.utils.targetbuildproject import TargetBuildProject
+
+class GalculatorTest(OERuntimeTestCase):
+
+ @classmethod
+ def setUpClass(cls):
+ uri = 'http://galculator.mnim.org/downloads/galculator-2.1.4.tar.bz2'
+ cls.project = TargetBuildProject(cls.tc.target,
+ uri,
+ dl_dir = cls.tc.td['DL_DIR'])
+ cls.project.download_archive()
+
+ @classmethod
+ def tearDownClass(cls):
+ cls.project.clean()
+
+ @OETestID(1526)
+ @skipIfNotFeature('tools-sdk',
+ 'Test requires tools-sdk to be in IMAGE_FEATURES')
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_galculator(self):
+ self.assertEqual(self.project.run_configure(), 0,
+ msg="Running configure failed")
+
+ self.assertEqual(self.project.run_make(), 0,
+ msg="Running make failed")
diff --git a/meta/lib/oeqa/runtime/cases/buildiptables.py b/meta/lib/oeqa/runtime/cases/buildiptables.py
new file mode 100644
index 0000000..bae8039
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/buildiptables.py
@@ -0,0 +1,39 @@
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfNotFeature
+
+from oeqa.runtime.utils.targetbuildproject import TargetBuildProject
+
+class BuildIptablesTest(OERuntimeTestCase):
+
+ @classmethod
+ def setUpClass(cls):
+ uri = 'http://downloads.yoctoproject.org/mirror/sources'
+ uri = '%s/iptables-1.4.13.tar.bz2' % uri
+ cls.project = TargetBuildProject(cls.tc.target,
+ uri,
+ dl_dir = cls.tc.td['DL_DIR'])
+ cls.project.download_archive()
+
+ @classmethod
+ def tearDownClass(cls):
+ cls.project.clean()
+
+ @OETestID(206)
+ @skipIfNotFeature('tools-sdk',
+ 'Test requires tools-sdk to be in IMAGE_FEATURES')
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_iptables(self):
+ self.assertEqual(self.project.run_configure(), 0,
+ msg="Running configure failed")
+
+ self.assertEqual(self.project.run_make(), 0,
+ msg="Running make failed")
+
+ self.assertEqual(self.project.run_install(), 0,
+ msg="Running make install failed")
+
+ @classmethod
+ def tearDownClass(self):
+ self.project.clean()
diff --git a/meta/lib/oeqa/runtime/cases/connman.py b/meta/lib/oeqa/runtime/cases/connman.py
new file mode 100644
index 0000000..8b47108
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/connman.py
@@ -0,0 +1,30 @@
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.runtime.decorator.package import OEHasPackage
+
+class ConnmanTest(OERuntimeTestCase):
+
+ def service_status(self, service):
+ if oeRuntimeTest.hasFeature("systemd"):
+ (_, output) = self.target.run('systemctl status -l %s' % service)
+ return output
+ else:
+ return "Unable to get status or logs for %s" % service
+
+ @OETestID(961)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ @OEHasPackage(["connman"])
+ def test_connmand_help(self):
+ (status, output) = self.target.run('/usr/sbin/connmand --help')
+ msg = 'Failed to get connman help. Output: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ @OETestID(221)
+ @OETestDepends(['connman.ConnmanTest.test_connmand_help'])
+ def test_connmand_running(self):
+ cmd = '%s | grep [c]onnmand' % self.tc.target_cmds['ps']
+ (status, output) = self.target.run(cmd)
+ if status != 0:
+ self.logger.info(self.service_status("connman"))
+ self.fail("No connmand process running")
diff --git a/meta/lib/oeqa/runtime/cases/date.py b/meta/lib/oeqa/runtime/cases/date.py
new file mode 100644
index 0000000..ece7338
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/date.py
@@ -0,0 +1,38 @@
+import re
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+
+class DateTest(OERuntimeTestCase):
+
+ def setUp(self):
+ if self.tc.td.get('VIRTUAL-RUNTIME_init_manager') == 'systemd':
+ self.logger.debug('Stopping systemd-timesyncd daemon')
+ self.target.run('systemctl stop systemd-timesyncd')
+
+ def tearDown(self):
+ if self.tc.td.get('VIRTUAL-RUNTIME_init_manager') == 'systemd':
+ self.logger.debug('Starting systemd-timesyncd daemon')
+ self.target.run('systemctl start systemd-timesyncd')
+
+ @OETestID(211)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_date(self):
+ (status, output) = self.target.run('date +"%Y-%m-%d %T"')
+ msg = 'Failed to get initial date, output: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+ oldDate = output
+
+ sampleDate = '"2016-08-09 10:00:00"'
+ (status, output) = self.target.run("date -s %s" % sampleDate)
+ self.assertEqual(status, 0, msg='Date set failed, output: %s' % output)
+
+ (status, output) = self.target.run("date -R")
+ p = re.match('Tue, 09 Aug 2016 10:00:.. \+0000', output)
+ msg = 'The date was not set correctly, output: %s' % output
+ self.assertTrue(p, msg=msg)
+
+ (status, output) = self.target.run('date -s "%s"' % oldDate)
+ msg = 'Failed to reset date, output: %s' % output
+ self.assertEqual(status, 0, msg=msg)
diff --git a/meta/lib/oeqa/runtime/cases/df.py b/meta/lib/oeqa/runtime/cases/df.py
new file mode 100644
index 0000000..aecc32d
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/df.py
@@ -0,0 +1,13 @@
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+
+class DfTest(OERuntimeTestCase):
+
+ @OETestID(234)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_df(self):
+ cmd = "df / | sed -n '2p' | awk '{print $4}'"
+ (status,output) = self.target.run(cmd)
+ msg = 'Not enough space on image. Current size is %s' % output
+ self.assertTrue(int(output)>5120, msg=msg)
diff --git a/meta/lib/oeqa/runtime/cases/gcc.py b/meta/lib/oeqa/runtime/cases/gcc.py
new file mode 100644
index 0000000..064fa49
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/gcc.py
@@ -0,0 +1,76 @@
+import os
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfNotFeature
+
+class GccCompileTest(OERuntimeTestCase):
+
+ @classmethod
+ def setUpClass(cls):
+ dst = '/tmp/'
+ src = os.path.join(cls.tc.files_dir, 'test.c')
+ #dst = '/tmp/test.c'
+ cls.tc.target.copyTo(src, dst)
+
+ src = os.path.join(cls.tc.runtime_files_dir, 'testmakefile')
+ #dst = '/tmp/testmakefile'
+ cls.tc.target.copyTo(src, dst)
+
+ src = os.path.join(cls.tc.files_dir, 'test.cpp')
+ #dst = '/tmp/test.cpp'
+ cls.tc.target.copyTo(src, dst)
+
+ @classmethod
+ def tearDownClass(cls):
+ files = '/tmp/test.c /tmp/test.o /tmp/test /tmp/testmakefile'
+ cls.tc.target.run('rm %s' % files)
+
+ @OETestID(203)
+ @skipIfNotFeature('tools-sdk',
+ 'Test requires tools-sdk to be in IMAGE_FEATURES')
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_gcc_compile(self):
+ status, output = self.target.run('gcc /tmp/test.c -o /tmp/test -lm')
+ msg = 'gcc compile failed, output: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ status, output = self.target.run('/tmp/test')
+ msg = 'running compiled file failed, output: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ @OETestID(200)
+ @skipIfNotFeature('tools-sdk',
+ 'Test requires tools-sdk to be in IMAGE_FEATURES')
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_gpp_compile(self):
+ status, output = self.target.run('g++ /tmp/test.c -o /tmp/test -lm')
+ msg = 'g++ compile failed, output: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ status, output = self.target.run('/tmp/test')
+ msg = 'running compiled file failed, output: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ @OETestID(1142)
+ @skipIfNotFeature('tools-sdk',
+ 'Test requires tools-sdk to be in IMAGE_FEATURES')
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_gpp2_compile(self):
+ status, output = self.target.run('g++ /tmp/test.cpp -o /tmp/test -lm')
+ msg = 'g++ compile failed, output: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ status, output = self.target.run('/tmp/test')
+ msg = 'running compiled file failed, output: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ @OETestID(204)
+ @skipIfNotFeature('tools-sdk',
+ 'Test requires tools-sdk to be in IMAGE_FEATURES')
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_make(self):
+ status, output = self.target.run('cd /tmp; make -f testmakefile')
+ msg = 'running make failed, output %s' % output
+ self.assertEqual(status, 0, msg=msg)
diff --git a/meta/lib/oeqa/runtime/cases/kernelmodule.py b/meta/lib/oeqa/runtime/cases/kernelmodule.py
new file mode 100644
index 0000000..11ad7b7
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/kernelmodule.py
@@ -0,0 +1,40 @@
+import os
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfNotFeature
+
+class KernelModuleTest(OERuntimeTestCase):
+
+ @classmethod
+ def setUpClass(cls):
+ src = os.path.join(cls.tc.runtime_files_dir, 'hellomod.c')
+ dst = '/tmp/hellomod.c'
+ cls.tc.target.copyTo(src, dst)
+
+ src = os.path.join(cls.tc.runtime_files_dir, 'hellomod_makefile')
+ dst = '/tmp/Makefile'
+ cls.tc.target.copyTo(src, dst)
+
+ @classmethod
+ def tearDownClass(cls):
+ files = '/tmp/Makefile /tmp/hellomod.c'
+ cls.tc.target.run('rm %s' % files)
+
+ @OETestID(1541)
+ @skipIfNotFeature('tools-sdk',
+ 'Test requires tools-sdk to be in IMAGE_FEATURES')
+ @OETestDepends(['gcc.GccCompileTest.test_gcc_compile'])
+ def test_kernel_module(self):
+ cmds = [
+ 'cd /usr/src/kernel && make scripts',
+ 'cd /tmp && make',
+ 'cd /tmp && insmod hellomod.ko',
+ 'lsmod | grep hellomod',
+ 'dmesg | grep Hello',
+ 'rmmod hellomod', 'dmesg | grep "Cleaning up hellomod"'
+ ]
+ for cmd in cmds:
+ status, output = self.target.run(cmd, 900)
+ self.assertEqual(status, 0, msg='\n'.join([cmd, output]))
diff --git a/meta/lib/oeqa/runtime/cases/ldd.py b/meta/lib/oeqa/runtime/cases/ldd.py
new file mode 100644
index 0000000..c6d92fd
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/ldd.py
@@ -0,0 +1,25 @@
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfNotFeature
+
+class LddTest(OERuntimeTestCase):
+
+ @OETestID(962)
+ @skipIfNotFeature('tools-sdk',
+ 'Test requires tools-sdk to be in IMAGE_FEATURES')
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_ldd_exists(self):
+ status, output = self.target.run('which ldd')
+ msg = 'ldd does not exist in PATH: which ldd: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ @OETestID(239)
+ @OETestDepends(['ldd.LddTest.test_ldd_exists'])
+ def test_ldd_rtldlist_check(self):
+ cmd = ('for i in $(which ldd | xargs cat | grep "^RTLDLIST"| '
+ 'cut -d\'=\' -f2|tr -d \'"\'); '
+ 'do test -f $i && echo $i && break; done')
+ status, output = self.target.run(cmd)
+ msg = "ldd path not correct or RTLDLIST files don't exist."
+ self.assertEqual(status, 0, msg=msg)
diff --git a/meta/lib/oeqa/runtime/cases/logrotate.py b/meta/lib/oeqa/runtime/cases/logrotate.py
new file mode 100644
index 0000000..992fef2
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/logrotate.py
@@ -0,0 +1,42 @@
+# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=289 testcase
+# Note that the image under test must have logrotate installed
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.runtime.decorator.package import OEHasPackage
+
+class LogrotateTest(OERuntimeTestCase):
+
+ @classmethod
+ def tearDownClass(cls):
+ cls.tc.target.run('rm -rf $HOME/logrotate_dir')
+
+ @OETestID(1544)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ @OEHasPackage(['logrotate'])
+ def test_1_logrotate_setup(self):
+ status, output = self.target.run('mkdir $HOME/logrotate_dir')
+ msg = 'Could not create logrotate_dir. Output: %s' % output
+ self.assertEqual(status, 0, msg = msg)
+
+ cmd = ('sed -i "s#wtmp {#wtmp {\\n olddir $HOME/logrotate_dir#"'
+ ' /etc/logrotate.conf')
+ status, output = self.target.run(cmd)
+ msg = ('Could not write to logrotate.conf file. Status and output: '
+ ' %s and %s' % (status, output))
+ self.assertEqual(status, 0, msg = msg)
+
+ @OETestID(1542)
+ @OETestDepends(['logrotate.LogrotateTest.test_1_logrotate_setup'])
+ def test_2_logrotate(self):
+ status, output = self.target.run('logrotate -f /etc/logrotate.conf')
+ msg = ('logrotate service could not be reloaded. Status and output: '
+ '%s and %s' % (status, output))
+ self.assertEqual(status, 0, msg = msg)
+
+ _, output = self.target.run('ls -la $HOME/logrotate_dir/ | wc -l')
+ msg = ('new logfile could not be created. List of files within log '
+ 'directory: %s' % (
+ self.target.run('ls -la $HOME/logrotate_dir')[1]))
+ self.assertTrue(int(output)>=3, msg = msg)
diff --git a/meta/lib/oeqa/runtime/cases/multilib.py b/meta/lib/oeqa/runtime/cases/multilib.py
new file mode 100644
index 0000000..8f6d2b2
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/multilib.py
@@ -0,0 +1,41 @@
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfNotInDataVar
+from oeqa.runtime.decorator.package import OEHasPackage
+
+class MultilibTest(OERuntimeTestCase):
+
+ def archtest(self, binary, arch):
+ """
+ Check that ``binary`` has the ELF class ``arch`` (e.g. ELF32/ELF64).
+ """
+
+ status, output = self.target.run('readelf -h %s' % binary)
+ self.assertEqual(status, 0, 'Failed to readelf %s' % binary)
+
+ l = [l.split()[1] for l in output.split('\n') if "Class:" in l]
+ if l:
+ theclass = l[0]
+ else:
+ self.fail('Cannot parse readelf. Output:\n%s' % output)
+
+ msg = "%s isn't %s (is %s)" % (binary, arch, theclass)
+ self.assertEqual(theclass, arch, msg=msg)
+
+ @skipIfNotInDataVar('MULTILIBS', 'multilib:lib32',
+ "This isn't a multilib:lib32 image")
+ @OETestID(201)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_check_multilib_libc(self):
+ """
+ Check that a multilib image has both 32-bit and 64-bit libc in.
+ """
+ self.archtest("/lib/libc.so.6", "ELF32")
+ self.archtest("/lib64/libc.so.6", "ELF64")
+
+ @OETestID(279)
+ @OETestDepends(['multilib.MultilibTest.test_check_multilib_libc'])
+ @OEHasPackage(['lib32-connman'])
+ def test_file_connman(self):
+ self.archtest("/usr/sbin/connmand", "ELF32")
diff --git a/meta/lib/oeqa/runtime/cases/pam.py b/meta/lib/oeqa/runtime/cases/pam.py
new file mode 100644
index 0000000..3654cdc
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/pam.py
@@ -0,0 +1,33 @@
+# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=287 testcase
+# Note that the image under test must have "pam" in DISTRO_FEATURES
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfNotFeature
+
+class PamBasicTest(OERuntimeTestCase):
+
+ @OETestID(1543)
+ @skipIfNotFeature('pam', 'Test requires pam to be in DISTRO_FEATURES')
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_pam(self):
+ status, output = self.target.run('login --help')
+ msg = ('login command does not work as expected. '
+ 'Status and output:%s and %s' % (status, output))
+ self.assertEqual(status, 1, msg = msg)
+
+ status, output = self.target.run('passwd --help')
+ msg = ('passwd command does not work as expected. '
+ 'Status and output:%s and %s' % (status, output))
+ self.assertEqual(status, 0, msg = msg)
+
+ status, output = self.target.run('su --help')
+ msg = ('su command does not work as expected. '
+ 'Status and output:%s and %s' % (status, output))
+ self.assertEqual(status, 0, msg = msg)
+
+ status, output = self.target.run('useradd --help')
+ msg = ('useradd command does not work as expected. '
+ 'Status and output:%s and %s' % (status, output))
+ self.assertEqual(status, 0, msg = msg)
diff --git a/meta/lib/oeqa/runtime_cases/parselogs.py b/meta/lib/oeqa/runtime/cases/parselogs.py
similarity index 64%
rename from meta/lib/oeqa/runtime_cases/parselogs.py
rename to meta/lib/oeqa/runtime/cases/parselogs.py
index cc2d061..a53a360 100644
--- a/meta/lib/oeqa/runtime_cases/parselogs.py
+++ b/meta/lib/oeqa/runtime/cases/parselogs.py
@@ -1,8 +1,12 @@
import os
-import unittest
-import subprocess
-from oeqa.oetest import oeRuntimeTest
-from oeqa.utils.decorators import *
+
+from subprocess import check_output
+from shutil import rmtree
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfDataVar
+from oeqa.runtime.decorator.package import OEHasPackage
#in the future these lists could be moved outside of module
errors = ["error", "cannot", "can\'t", "failed"]
@@ -168,144 +172,173 @@ ignore_errors = {
log_locations = ["/var/log/","/var/log/dmesg", "/tmp/dmesg_output.log"]
-class ParseLogsTest(oeRuntimeTest):
+class ParseLogsTest(OERuntimeTestCase):
@classmethod
- def setUpClass(self):
- self.errors = errors
+ def setUpClass(cls):
+ cls.errors = errors
# When systemd is enabled we need to notice errors on
# circular dependencies in units.
- if self.hasFeature("systemd"):
- self.errors.extend([
+ if 'systemd' in cls.td.get('DISTRO_FEATURES', ''):
+ cls.errors.extend([
'Found ordering cycle on',
'Breaking ordering cycle by deleting job',
'deleted to break ordering cycle',
'Ordering cycle found, skipping',
])
- self.ignore_errors = ignore_errors
- self.log_locations = log_locations
- self.msg = ""
- (is_lsb, location) = oeRuntimeTest.tc.target.run("which LSB_Test.sh")
+ cls.ignore_errors = ignore_errors
+ cls.log_locations = log_locations
+ cls.msg = ''
+ is_lsb, _ = cls.tc.target.run("which LSB_Test.sh")
if is_lsb == 0:
- for machine in self.ignore_errors:
- self.ignore_errors[machine] = self.ignore_errors[machine] + video_related
+ for machine in cls.ignore_errors:
+ cls.ignore_errors[machine] = cls.ignore_errors[machine] \
+ + video_related
def getMachine(self):
- return oeRuntimeTest.tc.d.getVar("MACHINE")
+ return self.td.get('MACHINE', '')
def getWorkdir(self):
- return oeRuntimeTest.tc.d.getVar("WORKDIR")
+ return self.td.get('WORKDIR', '')
- #get some information on the CPU of the machine to display at the beginning of the output. This info might be useful in some cases.
+ # Get some information on the CPU of the machine to display at the
+ # beginning of the output. This info might be useful in some cases.
def getHardwareInfo(self):
hwi = ""
- (status, cpu_name) = self.target.run("cat /proc/cpuinfo | grep \"model name\" | head -n1 | awk 'BEGIN{FS=\":\"}{print $2}'")
- (status, cpu_physical_cores) = self.target.run("cat /proc/cpuinfo | grep \"cpu cores\" | head -n1 | awk {'print $4'}")
- (status, cpu_logical_cores) = self.target.run("cat /proc/cpuinfo | grep \"processor\" | wc -l")
- (status, cpu_arch) = self.target.run("uname -m")
- hwi += "Machine information: \n"
- hwi += "*******************************\n"
- hwi += "Machine name: "+self.getMachine()+"\n"
- hwi += "CPU: "+str(cpu_name)+"\n"
- hwi += "Arch: "+str(cpu_arch)+"\n"
- hwi += "Physical cores: "+str(cpu_physical_cores)+"\n"
- hwi += "Logical cores: "+str(cpu_logical_cores)+"\n"
- hwi += "*******************************\n"
+ cmd = ('cat /proc/cpuinfo | grep "model name" | head -n1 | '
+ " awk 'BEGIN{FS=\":\"}{print $2}'")
+ _, cpu_name = self.target.run(cmd)
+
+ cmd = ('cat /proc/cpuinfo | grep "cpu cores" | head -n1 | '
+ "awk {'print $4'}")
+ _, cpu_physical_cores = self.target.run(cmd)
+
+ cmd = 'cat /proc/cpuinfo | grep "processor" | wc -l'
+ _, cpu_logical_cores = self.target.run(cmd)
+
+ _, cpu_arch = self.target.run('uname -m')
+
+ hwi += 'Machine information: \n'
+ hwi += '*******************************\n'
+ hwi += 'Machine name: ' + self.getMachine() + '\n'
+ hwi += 'CPU: ' + str(cpu_name) + '\n'
+ hwi += 'Arch: ' + str(cpu_arch)+ '\n'
+ hwi += 'Physical cores: ' + str(cpu_physical_cores) + '\n'
+ hwi += 'Logical cores: ' + str(cpu_logical_cores) + '\n'
+ hwi += '*******************************\n'
+
return hwi
- #go through the log locations provided and if it's a folder create a list with all the .log files in it, if it's a file just add
- #it to that list
+ # Go through the log locations provided and if it's a folder
+ # create a list with all the .log files in it, if it's a file
+ # just add it to that list.
def getLogList(self, log_locations):
logs = []
for location in log_locations:
- (status, output) = self.target.run("test -f "+str(location))
- if (status == 0):
+ status, _ = self.target.run('test -f ' + str(location))
+ if status == 0:
logs.append(str(location))
else:
- (status, output) = self.target.run("test -d "+str(location))
- if (status == 0):
- (status, output) = self.target.run("find "+str(location)+"/*.log -maxdepth 1 -type f")
- if (status == 0):
+ status, _ = self.target.run('test -d ' + str(location))
+ if status == 0:
+ cmd = 'find ' + str(location) + '/*.log -maxdepth 1 -type f'
+ status, output = self.target.run(cmd)
+ if status == 0:
output = output.splitlines()
for logfile in output:
- logs.append(os.path.join(location,str(logfile)))
+ logs.append(os.path.join(location, str(logfile)))
return logs
- #copy the log files to be parsed locally
+ # Copy the log files to be parsed locally
def transfer_logs(self, log_list):
workdir = self.getWorkdir()
self.target_logs = workdir + '/' + 'target_logs'
target_logs = self.target_logs
- if not os.path.exists(target_logs):
- os.makedirs(target_logs)
- bb.utils.remove(self.target_logs + "/*")
+ if os.path.exists(target_logs):
+ rmtree(self.target_logs)
+ os.makedirs(target_logs)
for f in log_list:
- self.target.copy_from(f, target_logs)
+ self.target.copyFrom(str(f), target_logs)
- #get the local list of logs
+ # Get the local list of logs
def get_local_log_list(self, log_locations):
self.transfer_logs(self.getLogList(log_locations))
- logs = [ os.path.join(self.target_logs, f) for f in os.listdir(self.target_logs) if os.path.isfile(os.path.join(self.target_logs, f)) ]
+ list_dir = os.listdir(self.target_logs)
+ dir_files = [os.path.join(self.target_logs, f) for f in list_dir]
+ logs = [f for f in dir_files if os.path.isfile(f)]
return logs
- #build the grep command to be used with filters and exclusions
+ # Build the grep command to be used with filters and exclusions
def build_grepcmd(self, errors, ignore_errors, log):
- grepcmd = "grep "
- grepcmd +="-Ei \""
+ grepcmd = 'grep '
+ grepcmd += '-Ei "'
for error in errors:
- grepcmd += error+"|"
+ grepcmd += error + '|'
grepcmd = grepcmd[:-1]
- grepcmd += "\" "+str(log)+" | grep -Eiv \'"
+ grepcmd += '" ' + str(log) + " | grep -Eiv \'"
+
try:
errorlist = ignore_errors[self.getMachine()]
except KeyError:
- self.msg += "No ignore list found for this machine, using default\n"
+ self.msg += 'No ignore list found for this machine, using default\n'
errorlist = ignore_errors['default']
+
for ignore_error in errorlist:
- ignore_error = ignore_error.replace("(", "\(")
- ignore_error = ignore_error.replace(")", "\)")
- ignore_error = ignore_error.replace("'", ".")
- ignore_error = ignore_error.replace("?", "\?")
- ignore_error = ignore_error.replace("[", "\[")
- ignore_error = ignore_error.replace("]", "\]")
- ignore_error = ignore_error.replace("*", "\*")
- ignore_error = ignore_error.replace("0-9", "[0-9]")
- grepcmd += ignore_error+"|"
+ ignore_error = ignore_error.replace('(', '\(')
+ ignore_error = ignore_error.replace(')', '\)')
+ ignore_error = ignore_error.replace("'", '.')
+ ignore_error = ignore_error.replace('?', '\?')
+ ignore_error = ignore_error.replace('[', '\[')
+ ignore_error = ignore_error.replace(']', '\]')
+ ignore_error = ignore_error.replace('*', '\*')
+ ignore_error = ignore_error.replace('0-9', '[0-9]')
+ grepcmd += ignore_error + '|'
grepcmd = grepcmd[:-1]
grepcmd += "\'"
+
return grepcmd
- #grep only the errors so that their context could be collected. Default context is 10 lines before and after the error itself
- def parse_logs(self, errors, ignore_errors, logs, lines_before = 10, lines_after = 10):
+ # Grep only the errors so that their context could be collected.
+ # Default context is 10 lines before and after the error itself
+ def parse_logs(self, errors, ignore_errors, logs,
+ lines_before = 10, lines_after = 10):
results = {}
rez = []
grep_output = ''
+
for log in logs:
result = None
thegrep = self.build_grepcmd(errors, ignore_errors, log)
+
try:
- result = subprocess.check_output(thegrep, shell=True).decode("utf-8")
+ result = check_output(thegrep, shell=True).decode('utf-8')
except:
pass
- if (result is not None):
+
+ if result is not None:
results[log.replace('target_logs/','')] = {}
rez = result.splitlines()
+
for xrez in rez:
try:
- grep_output = subprocess.check_output(['grep', '-F', xrez, '-B', str(lines_before), '-A', str(lines_after), log]).decode("utf-8")
+ cmd = ['grep', '-F', xrez, '-B', str(lines_before)]
+ cmd += ['-A', str(lines_after), log]
+ grep_output = check_output(cmd).decode('utf-8')
except:
pass
results[log.replace('target_logs/','')][xrez]=grep_output
+
return results
- #get the output of dmesg and write it in a file. This file is added to log_locations.
+ # Get the output of dmesg and write it in a file.
+ # This file is added to log_locations.
def write_dmesg(self):
- (status, dmesg) = self.target.run("dmesg > /tmp/dmesg_output.log")
+ (status, dmesg) = self.target.run('dmesg > /tmp/dmesg_output.log')
- @testcase(1059)
- @skipUnlessPassed('test_ssh')
+ @OETestID(1059)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
def test_parselogs(self):
self.write_dmesg()
log_list = self.get_local_log_list(self.log_locations)
@@ -313,13 +346,13 @@ class ParseLogsTest(oeRuntimeTest):
print(self.getHardwareInfo())
errcount = 0
for log in result:
- self.msg += "Log: "+log+"\n"
- self.msg += "-----------------------\n"
+ self.msg += 'Log: ' + log + '\n'
+ self.msg += '-----------------------\n'
for error in result[log]:
errcount += 1
- self.msg += "Central error: "+str(error)+"\n"
- self.msg += "***********************\n"
- self.msg += result[str(log)][str(error)]+"\n"
- self.msg += "***********************\n"
- self.msg += "%s errors found in logs." % errcount
+ self.msg += 'Central error: ' + str(error) + '\n'
+ self.msg += '***********************\n'
+ self.msg += result[str(log)][str(error)] + '\n'
+ self.msg += '***********************\n'
+ self.msg += '%s errors found in logs.' % errcount
self.assertEqual(errcount, 0, msg=self.msg)
diff --git a/meta/lib/oeqa/runtime/cases/perl.py b/meta/lib/oeqa/runtime/cases/perl.py
new file mode 100644
index 0000000..d0b7e8e
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/perl.py
@@ -0,0 +1,37 @@
+import os
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.runtime.decorator.package import OEHasPackage
+
+class PerlTest(OERuntimeTestCase):
+
+ @classmethod
+ def setUpClass(cls):
+ src = os.path.join(cls.tc.files_dir, 'test.pl')
+ dst = '/tmp/test.pl'
+ cls.tc.target.copyTo(src, dst)
+
+ @classmethod
+ def tearDownClass(cls):
+ dst = '/tmp/test.pl'
+ cls.tc.target.run('rm %s' % dst)
+
+ @OETestID(1141)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ @OEHasPackage(['perl'])
+ def test_perl_exists(self):
+ status, output = self.target.run('which perl')
+ msg = 'Perl binary not in PATH or not on target.'
+ self.assertEqual(status, 0, msg=msg)
+
+ @OETestID(208)
+ @OETestDepends(['perl.PerlTest.test_perl_exists'])
+ def test_perl_works(self):
+ status, output = self.target.run('perl /tmp/test.pl')
+ msg = 'Exit status was not 0. Output: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ msg = 'Incorrect output: %s' % output
+ self.assertEqual(output, "the value of a is 0.01", msg=msg)
diff --git a/meta/lib/oeqa/runtime/cases/ping.py b/meta/lib/oeqa/runtime/cases/ping.py
new file mode 100644
index 0000000..02f580a
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/ping.py
@@ -0,0 +1,24 @@
+from subprocess import Popen, PIPE
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.oetimeout import OETimeout
+
+class PingTest(OERuntimeTestCase):
+
+ @OETimeout(30)
+ @OETestID(964)
+ def test_ping(self):
+ output = ''
+ count = 0
+ while count < 5:
+ cmd = 'ping -c 1 %s' % self.target.ip
+ proc = Popen(cmd, shell=True, stdout=PIPE)
+ output += proc.communicate()[0].decode('utf-8')
+ if proc.poll() == 0:
+ count += 1
+ else:
+ count = 0
+ msg = ('Expected 5 consecutive, got %d.\n'
+ 'ping output is:\n%s' % (count,output))
+ self.assertEqual(count, 5, msg = msg)
diff --git a/meta/lib/oeqa/runtime/cases/python.py b/meta/lib/oeqa/runtime/cases/python.py
new file mode 100644
index 0000000..bf3e179
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/python.py
@@ -0,0 +1,43 @@
+import os
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.runtime.decorator.package import OEHasPackage
+
+class PythonTest(OERuntimeTestCase):
+
+ @classmethod
+ def setUpClass(cls):
+ src = os.path.join(cls.tc.files_dir, 'test.py')
+ dst = '/tmp/test.py'
+ cls.tc.target.copyTo(src, dst)
+
+ @classmethod
+ def tearDownClass(cls):
+ dst = '/tmp/test.py'
+ cls.tc.target.run('rm %s' % dst)
+
+ @OETestID(1145)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ @OEHasPackage(['python-core'])
+ def test_python_exists(self):
+ status, output = self.target.run('which python')
+ msg = 'Python binary not in PATH or not on target.'
+ self.assertEqual(status, 0, msg=msg)
+
+ @OETestID(965)
+ @OETestDepends(['python.PythonTest.test_python_exists'])
+ def test_python_stdout(self):
+ status, output = self.target.run('python /tmp/test.py')
+ msg = 'Exit status was not 0. Output: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ msg = 'Incorrect output: %s' % output
+ self.assertEqual(output, "the value of a is 0.01", msg=msg)
+
+ @OETestID(1146)
+ @OETestDepends(['python.PythonTest.test_python_stdout'])
+ def test_python_testfile(self):
+ status, output = self.target.run('ls /tmp/testfile.python')
+ self.assertEqual(status, 0, msg='Python test file generate failed.')
diff --git a/meta/lib/oeqa/runtime/cases/rpm.py b/meta/lib/oeqa/runtime/cases/rpm.py
new file mode 100644
index 0000000..532fbf8
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/rpm.py
@@ -0,0 +1,141 @@
+import os
+import fnmatch
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfDataVar
+from oeqa.runtime.decorator.package import OEHasPackage
+from oeqa.core.utils.path import findFile
+
+class RpmBasicTest(OERuntimeTestCase):
+
+ @classmethod
+ def setUpClass(cls):
+ if cls.tc.td['PACKAGE_CLASSES'].split()[0] != 'package_rpm':
+ cls.skipTest('Tests require image to be build from rpm')
+
+ @OETestID(960)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_rpm_help(self):
+ status, output = self.target.run('rpm --help')
+ msg = 'status and output: %s and %s' % (status, output)
+ self.assertEqual(status, 0, msg=msg)
+
+ @OETestID(191)
+ @OETestDepends(['rpm.RpmBasicTest.test_rpm_help'])
+ def test_rpm_query(self):
+ status, output = self.target.run('rpm -q rpm')
+ msg = 'status and output: %s and %s' % (status, output)
+ self.assertEqual(status, 0, msg=msg)
+
+class RpmInstallRemoveTest(OERuntimeTestCase):
+
+ @classmethod
+ def setUpClass(cls):
+ if cls.tc.td['PACKAGE_CLASSES'].split()[0] != 'package_rpm':
+ cls.skipTest('Tests require image to be build from rpm')
+
+ pkgarch = cls.td['TUNE_PKGARCH'].replace('-', '_')
+ rpmdir = os.path.join(cls.tc.td['DEPLOY_DIR'], 'rpm', pkgarch)
+ # Pick rpm-doc as a test file to get installed, because it's small
+ # and it will always be built for standard targets
+ rpm_doc = 'rpm-doc-*.%s.rpm' % pkgarch
+ for f in fnmatch.filter(os.listdir(rpmdir), rpm_doc):
+ test_file = os.path.join(rpmdir, f)
+ dst = '/tmp/rpm-doc.rpm'
+ cls.tc.target.copyTo(test_file, dst)
+
+ @classmethod
+ def tearDownClass(cls):
+ dst = '/tmp/rpm-doc.rpm'
+ cls.tc.target.run('rm -f %s' % dst)
+
+ @OETestID(192)
+ @OETestDepends(['rpm.RpmBasicTest.test_rpm_help'])
+ def test_rpm_install(self):
+ status, output = self.target.run('rpm -ivh /tmp/rpm-doc.rpm')
+ msg = 'Failed to install rpm-doc package: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ @OETestID(194)
+ @OETestDepends(['rpm.RpmInstallRemoveTest.test_rpm_install'])
+ def test_rpm_remove(self):
+ status,output = self.target.run('rpm -e rpm-doc')
+ msg = 'Failed to remove rpm-doc package: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ @OETestID(1096)
+ @OETestDepends(['rpm.RpmBasicTest.test_rpm_query'])
+ def test_rpm_query_nonroot(self):
+
+ def set_up_test_user(u):
+ status, output = self.target.run('id -u %s' % u)
+ if status:
+ status, output = self.target.run('useradd %s' % u)
+ msg = 'Failed to create new user: %s' % output
+ self.assertTrue(status == 0, msg=msg)
+
+ def exec_as_test_user(u):
+ status, output = self.target.run('su -c id %s' % u)
+ msg = 'Failed to execute as new user'
+ self.assertTrue("({0})".format(u) in output, msg=msg)
+
+ status, output = self.target.run('su -c "rpm -qa" %s ' % u)
+ msg = 'status: %s. Cannot run rpm -qa: %s' % (status, output)
+ self.assertEqual(status, 0, msg=msg)
+
+ def unset_up_test_user(u):
+ status, output = self.target.run('userdel -r %s' % u)
+ msg = 'Failed to erase user: %s' % output
+ self.assertTrue(status == 0, msg=msg)
+
+ tuser = 'test1'
+
+ try:
+ set_up_test_user(tuser)
+ exec_as_test_user(tuser)
+ finally:
+ unset_up_test_user(tuser)
+
+ @OETestID(195)
+ @OETestDepends(['rpm.RpmInstallRemoveTest.test_rpm_remove'])
+ def test_check_rpm_install_removal_log_file_size(self):
+ """
+ Summary: Check rpm install/removal log file size
+ Expected: There should be some method to keep rpm log in a small size .
+ Product: BSPs
+ Author: Alexandru Georgescu <alexandru.c.georgescu@intel.com>
+ AutomatedBy: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
+ """
+ db_files_cmd = 'ls /var/lib/rpm/__db.*'
+ get_log_size_cmd = "du /var/lib/rpm/log/log.* | awk '{print $1}'"
+
+ # Make sure that some database files are under /var/lib/rpm as '__db.xxx'
+ status, output = self.target.run(db_files_cmd)
+ msg = 'Failed to find database files under /var/lib/rpm/ as __db.xxx'
+ self.assertEqual(0, status, msg=msg)
+
+ # Remove the package just in case
+ self.target.run('rpm -e rpm-doc')
+
+ # Install/Remove a package 10 times
+ for i in range(10):
+ status, output = self.target.run('rpm -ivh /tmp/rpm-doc.rpm')
+ msg = 'Failed to install rpm-doc package. Reason: {}'.format(output)
+ self.assertEqual(0, status, msg=msg)
+
+ status, output = self.target.run('rpm -e rpm-doc')
+ msg = 'Failed to remove rpm-doc package. Reason: {}'.format(output)
+ self.assertEqual(0, status, msg=msg)
+
+ # Get the size of log file
+ status, output = self.target.run(get_log_size_cmd)
+ msg = 'Failed to get the final size of the log file.'
+ self.assertEqual(0, status, msg=msg)
+
+ # Compare each log size
+ for log_file_size in output:
+ msg = ('Log file size is greater that expected (~10MB), '
+ 'found {} bytes'.format(log_file_size))
+ self.assertLessEqual(int(log_file_size), 11264, msg=msg)
diff --git a/meta/lib/oeqa/runtime/cases/scanelf.py b/meta/lib/oeqa/runtime/cases/scanelf.py
new file mode 100644
index 0000000..3ba1f78
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/scanelf.py
@@ -0,0 +1,26 @@
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.runtime.decorator.package import OEHasPackage
+
+class ScanelfTest(OERuntimeTestCase):
+ scancmd = 'scanelf --quiet --recursive --mount --ldpath --path'
+
+ @OETestID(966)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ @OEHasPackage(['pax-utils'])
+ def test_scanelf_textrel(self):
+ # print TEXTREL information
+ cmd = '%s --textrel' % self.scancmd
+ status, output = self.target.run(cmd)
+ msg = '\n'.join([cmd, output])
+ self.assertEqual(output.strip(), '', msg=msg)
+
+ @OETestID(967)
+ @OETestDepends(['scanelf.ScanelfTest.test_scanelf_textrel'])
+ def test_scanelf_rpath(self):
+ # print RPATH information
+ cmd = '%s --textrel --rpath' % self.scancmd
+ status, output = self.target.run(cmd)
+ msg = '\n'.join([cmd, output])
+ self.assertEqual(output.strip(), '', msg=msg)
diff --git a/meta/lib/oeqa/runtime/cases/scp.py b/meta/lib/oeqa/runtime/cases/scp.py
new file mode 100644
index 0000000..f488a61
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/scp.py
@@ -0,0 +1,33 @@
+import os
+from tempfile import mkstemp
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+
+class ScpTest(OERuntimeTestCase):
+
+ @classmethod
+ def setUpClass(cls):
+ cls.tmp_fd, cls.tmp_path = mkstemp()
+ with os.fdopen(cls.tmp_fd, 'w') as f:
+ f.seek(2 ** 22 -1)
+ f.write(os.linesep)
+
+ @classmethod
+ def tearDownClass(cls):
+ os.remove(cls.tmp_path)
+
+ @OETestID(220)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_scp_file(self):
+ dst = '/tmp/test_scp_file'
+
+ (status, output) = self.target.copyTo(self.tmp_path, dst)
+ msg = 'File could not be copied. Output: %s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ (status, output) = self.target.run('ls -la %s' % dst)
+ self.assertEqual(status, 0, msg = 'SCP test failed')
+
+ self.target.run('rm %s' % dst)
diff --git a/meta/lib/oeqa/runtime/cases/skeletoninit.py b/meta/lib/oeqa/runtime/cases/skeletoninit.py
new file mode 100644
index 0000000..e2dbbb7
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/skeletoninit.py
@@ -0,0 +1,33 @@
+# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=284
+# testcase. Image under test must have meta-skeleton layer in bblayers and
+# IMAGE_INSTALL_append = " service" in local.conf
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfDataVar
+from oeqa.runtime.decorator.package import OEHasPackage
+
+class SkeletonBasicTest(OERuntimeTestCase):
+
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ @OEHasPackage(['service'])
+ @skipIfDataVar('VIRTUAL-RUNTIME_init_manager', 'systemd',
+ 'Not appropiate for systemd image')
+ def test_skeleton_availability(self):
+ status, output = self.target.run('ls /etc/init.d/skeleton')
+ msg = 'skeleton init script not found. Output:\n%s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ status, output = self.target.run('ls /usr/sbin/skeleton-test')
+ msg = 'skeleton-test not found. Output:\n%s' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ @OETestID(284)
+ @OETestDepends(['skeletoninit.SkeletonBasicTest.test_skeleton_availability'])
+ def test_skeleton_script(self):
+ output1 = self.target.run("/etc/init.d/skeleton start")[1]
+ cmd = '%s | grep [s]keleton-test' % self.tc.target_cmds['ps']
+ status, output2 = self.target.run(cmd)
+ cmd = ('Skeleton script could not be started:'
+ '\n%s\n%s' % (output1, output2))
+ self.assertEqual(status, 0, msg=msg)
diff --git a/meta/lib/oeqa/runtime_cases/smart.py b/meta/lib/oeqa/runtime/cases/smart.py
similarity index 100%
rename from meta/lib/oeqa/runtime_cases/smart.py
rename to meta/lib/oeqa/runtime/cases/smart.py
diff --git a/meta/lib/oeqa/runtime/cases/ssh.py b/meta/lib/oeqa/runtime/cases/ssh.py
new file mode 100644
index 0000000..eca1679
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/ssh.py
@@ -0,0 +1,15 @@
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+
+class SSHTest(OERuntimeTestCase):
+
+ @OETestID(224)
+ @OETestDepends(['ping.PingTest.test_ping'])
+ def test_ssh(self):
+ (status, output) = self.target.run('uname -a')
+ self.assertEqual(status, 0, msg='SSH Test failed: %s' % output)
+ (status, output) = self.target.run('cat /etc/masterimage')
+ msg = "This isn't the right image - /etc/masterimage " \
+ "shouldn't be here %s" % output
+ self.assertEqual(status, 1, msg=msg)
diff --git a/meta/lib/oeqa/runtime/cases/syslog.py b/meta/lib/oeqa/runtime/cases/syslog.py
new file mode 100644
index 0000000..1016e67
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/syslog.py
@@ -0,0 +1,57 @@
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfDataVar
+from oeqa.runtime.decorator.package import OEHasPackage
+
+class SyslogTest(OERuntimeTestCase):
+
+ @OETestID(201)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ @OEHasPackage(["busybox-syslog", "sysklogd"])
+ def test_syslog_running(self):
+ cmd = '%s | grep -i [s]yslogd' % self.tc.target_cmds['ps']
+ status, output = self.target.run(cmd)
+ msg = "No syslogd process; ps output: %s" % output
+ self.assertEqual(status, 0, msg=msg)
+
+class SyslogTestConfig(OERuntimeTestCase):
+
+ @OETestID(1149)
+ @OETestDepends(['syslog.SyslogTest.test_syslog_running'])
+ def test_syslog_logger(self):
+ status, output = self.target.run('logger foobar')
+ msg = "Can't log into syslog. Output: %s " % output
+ self.assertEqual(status, 0, msg=msg)
+
+ status, output = self.target.run('grep foobar /var/log/messages')
+ if status != 0:
+ if self.tc.td.get("VIRTUAL-RUNTIME_init_manager") == "systemd":
+ status, output = self.target.run('journalctl -o cat | grep foobar')
+ else:
+ status, output = self.target.run('logread | grep foobar')
+ msg = ('Test log string not found in /var/log/messages or logread.'
+ ' Output: %s ' % output)
+ self.assertEqual(status, 0, msg=msg)
+
+ @OETestID(202)
+ @OETestDepends(['syslog.SyslogTestConfig.test_syslog_logger'])
+ @OEHasPackage(["!sysklogd", "busybox"])
+ @skipIfDataVar('VIRTUAL-RUNTIME_init_manager', 'systemd',
+ 'Not appropiate for systemd image')
+ def test_syslog_startup_config(self):
+ cmd = 'echo "LOGFILE=/var/log/test" >> /etc/syslog-startup.conf'
+ self.target.run(cmd)
+ status, output = self.target.run('/etc/init.d/syslog restart')
+ msg = ('Could not restart syslog service. Status and output:'
+ ' %s and %s' % (status,output))
+ self.assertEqual(status, 0, msg)
+
+ cmd = 'logger foobar && grep foobar /var/log/test'
+ status,output = self.target.run(cmd)
+ msg = 'Test log string not found. Output: %s ' % output
+ self.assertEqual(status, 0, msg=msg)
+
+ cmd = "sed -i 's#LOGFILE=/var/log/test##' /etc/syslog-startup.conf"
+ self.target.run(cmd)
+ self.target.run('/etc/init.d/syslog restart')
diff --git a/meta/lib/oeqa/runtime_cases/systemd.py b/meta/lib/oeqa/runtime/cases/systemd.py
similarity index 51%
rename from meta/lib/oeqa/runtime_cases/systemd.py
rename to meta/lib/oeqa/runtime/cases/systemd.py
index 52feb1b..db69384 100644
--- a/meta/lib/oeqa/runtime_cases/systemd.py
+++ b/meta/lib/oeqa/runtime/cases/systemd.py
@@ -1,28 +1,27 @@
-import unittest
import re
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
+import time
-def setUpModule():
- if not oeRuntimeTest.hasFeature("systemd"):
- skipModule("target doesn't have systemd in DISTRO_FEATURES")
- if "systemd" != oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager"):
- skipModule("systemd is not the init manager for this image")
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfDataVar, skipIfNotDataVar
+from oeqa.runtime.decorator.package import OEHasPackage
+from oeqa.core.decorator.data import skipIfNotFeature
+class SystemdTest(OERuntimeTestCase):
-class SystemdTest(oeRuntimeTest):
-
- def systemctl(self, action = '', target = '', expected = 0, verbose = False):
+ def systemctl(self, action='', target='', expected=0, verbose=False):
command = 'systemctl %s %s' % (action, target)
status, output = self.target.run(command)
message = '\n'.join([command, output])
if status != expected and verbose:
- message += self.target.run('systemctl status --full %s' % target)[1]
+ cmd = 'systemctl status --full %s' % target
+ message += self.target.run(cmd)[1]
self.assertEqual(status, expected, message)
return output
#TODO: use pyjournalctl instead
- def journalctl(self, args='',l_match_units=[]):
+ def journalctl(self, args='',l_match_units=None):
"""
Request for the journalctl output to the current target system
@@ -36,31 +35,23 @@ class SystemdTest(oeRuntimeTest):
-ValueError, on a journalctl call with filtering by l_match_units that
returned no entries
"""
- query_units=""
- if len(l_match_units):
+
+ query_units=''
+ if l_match_units:
query_units = ['_SYSTEMD_UNIT='+unit for unit in l_match_units]
- query_units = " ".join(query_units)
+ query_units = ' '.join(query_units)
command = 'journalctl %s %s' %(args, query_units)
status, output = self.target.run(command)
if status:
- raise AssertionError("Command '%s' returned non-zero exit \
- code %d:\n%s" % (command, status, output))
+ raise AssertionError("Command '%s' returned non-zero exit "
+ 'code %d:\n%s' % (command, status, output))
if len(output) == 1 and "-- No entries --" in output:
- raise ValueError("List of units to match: %s, returned no entries"
+ raise ValueError('List of units to match: %s, returned no entries'
% l_match_units)
return output
class SystemdBasicTests(SystemdTest):
- @skipUnlessPassed('test_ssh')
- def test_systemd_basic(self):
- self.systemctl('--version')
-
- @testcase(551)
- @skipUnlessPassed('test_systemd_basic')
- def test_systemd_list(self):
- self.systemctl('list-unit-files')
-
def settle(self):
"""
Block until systemd has finished activating any units being activated,
@@ -70,7 +61,6 @@ class SystemdBasicTests(SystemdTest):
activating, or (False, message string) if there are still units
activating (generally, failing units that restart).
"""
- import time
endtime = time.time() + (60 * 2)
while True:
status, output = self.target.run('systemctl --state=activating')
@@ -80,55 +70,65 @@ class SystemdBasicTests(SystemdTest):
return (False, output)
time.sleep(10)
- @testcase(550)
- @skipUnlessPassed('test_systemd_basic')
+ @skipIfNotFeature('systemd',
+ 'Test requires systemd to be in DISTRO_FEATURES')
+ @skipIfNotDataVar('VIRTUAL-RUNTIME_init_manager', 'systemd',
+ 'systemd is not the init manager for this image')
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_systemd_basic(self):
+ self.systemctl('--version')
+
+ @OETestID(551)
+ @OETestDepends(['systemd.SystemdBasicTests.test_systemd_basic'])
+ def test_systemd_list(self):
+ self.systemctl('list-unit-files')
+
+ @OETestID(550)
+ @OETestDepends(['systemd.SystemdBasicTests.test_systemd_basic'])
def test_systemd_failed(self):
settled, output = self.settle()
- self.assertTrue(settled, msg="Timed out waiting for systemd to settle:\n" + output)
+ msg = "Timed out waiting for systemd to settle:\n%s" % output
+ self.assertTrue(settled, msg=msg)
output = self.systemctl('list-units', '--failed')
- match = re.search("0 loaded units listed", output)
+ match = re.search('0 loaded units listed', output)
if not match:
output += self.systemctl('status --full --failed')
- self.assertTrue(match, msg="Some systemd units failed:\n%s" % output)
+ self.assertTrue(match, msg='Some systemd units failed:\n%s' % output)
class SystemdServiceTests(SystemdTest):
- def check_for_avahi(self):
- if not self.hasPackage('avahi-daemon'):
- raise unittest.SkipTest("Testcase dependency not met: need avahi-daemon installed on target")
-
- @skipUnlessPassed('test_systemd_basic')
+ @OEHasPackage(['avahi-daemon'])
+ @OETestDepends(['systemd.SystemdBasicTests.test_systemd_basic'])
def test_systemd_status(self):
- self.check_for_avahi()
self.systemctl('status --full', 'avahi-daemon.service')
- @testcase(695)
- @skipUnlessPassed('test_systemd_status')
+ @OETestID(695)
+ @OETestDepends(['systemd.SystemdServiceTests.test_systemd_status'])
def test_systemd_stop_start(self):
- self.check_for_avahi()
self.systemctl('stop', 'avahi-daemon.service')
- self.systemctl('is-active', 'avahi-daemon.service', expected=3, verbose=True)
+ self.systemctl('is-active', 'avahi-daemon.service',
+ expected=3, verbose=True)
self.systemctl('start','avahi-daemon.service')
self.systemctl('is-active', 'avahi-daemon.service', verbose=True)
- @testcase(696)
- @skipUnlessPassed('test_systemd_basic')
+ @OETestID(696)
+ @OETestDepends(['systemd.SystemdServiceTests.test_systemd_status'])
def test_systemd_disable_enable(self):
- self.check_for_avahi()
self.systemctl('disable', 'avahi-daemon.service')
self.systemctl('is-enabled', 'avahi-daemon.service', expected=1)
self.systemctl('enable', 'avahi-daemon.service')
self.systemctl('is-enabled', 'avahi-daemon.service')
class SystemdJournalTests(SystemdTest):
- @skipUnlessPassed('test_ssh')
+
+ @OETestDepends(['systemd.SystemdBasicTests.test_systemd_basic'])
def test_systemd_journal(self):
- (status, output) = self.target.run('journalctl')
+ status, output = self.target.run('journalctl')
self.assertEqual(status, 0, output)
- @skipUnlessPassed('test_systemd_basic')
+ @OETestDepends(['systemd.SystemdBasicTests.test_systemd_basic'])
def test_systemd_boot_time(self, systemd_TimeoutStartSec=90):
"""
Get the target boot time from journalctl and log it
@@ -138,41 +138,44 @@ class SystemdJournalTests(SystemdTest):
unit start timeout to compare against
"""
- # the expression chain that uniquely identifies the time boot message
- expr_items=["Startup finished","kernel", "userspace","\.$"]
+ # The expression chain that uniquely identifies the time boot message.
+ expr_items=['Startup finished', 'kernel', 'userspace','\.$']
try:
- output = self.journalctl(args="-o cat --reverse")
+ output = self.journalctl(args='-o cat --reverse')
except AssertionError:
- self.fail("Error occurred while calling journalctl")
+ self.fail('Error occurred while calling journalctl')
if not len(output):
- self.fail("Error, unable to get startup time from systemd journal")
+ self.fail('Error, unable to get startup time from systemd journal')
- # check for the regular expression items that match the startup time
+ # Check for the regular expression items that match the startup time.
for line in output.split('\n'):
- check_match = "".join(re.findall(".*".join(expr_items), line))
- if check_match: break
- # put the startup time in the test log
+ check_match = ''.join(re.findall('.*'.join(expr_items), line))
+ if check_match:
+ break
+ # Put the startup time in the test log
if check_match:
- print("%s" % check_match)
+ self.tc.logger.info('%s' % check_match)
else:
- self.skipTest("Error at obtaining the boot time from journalctl")
+ self.skipTest('Error at obtaining the boot time from journalctl')
boot_time_sec = 0
- # get the numeric values from the string and convert them to seconds
- # same data will be placed in list and string for manipulation
- l_boot_time = check_match.split(" ")[-2:]
- s_boot_time = " ".join(l_boot_time)
+ # Get the numeric values from the string and convert them to seconds
+ # same data will be placed in list and string for manipulation.
+ l_boot_time = check_match.split(' ')[-2:]
+ s_boot_time = ' '.join(l_boot_time)
try:
- # Obtain the minutes it took to boot
+ # Obtain the minutes it took to boot.
if l_boot_time[0].endswith('min') and l_boot_time[0][0].isdigit():
- boot_time_min = s_boot_time.split("min")[0]
- # convert to seconds and accumulate it
+ boot_time_min = s_boot_time.split('min')[0]
+ # Convert to seconds and accumulate it.
boot_time_sec += int(boot_time_min) * 60
- # Obtain the seconds it took to boot and accumulate
- boot_time_sec += float(l_boot_time[1].split("s")[0])
+ # Obtain the seconds it took to boot and accumulate.
+ boot_time_sec += float(l_boot_time[1].split('s')[0])
except ValueError:
- self.skipTest("Error when parsing time from boot string")
- #Assert the target boot time against systemd's unit start timeout
+ self.skipTest('Error when parsing time from boot string')
+
+ # Assert the target boot time against systemd's unit start timeout.
if boot_time_sec > systemd_TimeoutStartSec:
- print("Target boot time %s exceeds systemd's TimeoutStartSec %s"\
- %(boot_time_sec, systemd_TimeoutStartSec))
+ msg = ("Target boot time %s exceeds systemd's TimeoutStartSec %s"
+ % (boot_time_sec, systemd_TimeoutStartSec))
+ self.tc.logger.info(msg)
diff --git a/meta/lib/oeqa/runtime/cases/x32lib.py b/meta/lib/oeqa/runtime/cases/x32lib.py
new file mode 100644
index 0000000..8da0154
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/x32lib.py
@@ -0,0 +1,19 @@
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfNotInDataVar
+
+class X32libTest(OERuntimeTestCase):
+
+ @skipIfNotInDataVar('DEFAULTTUNE', 'x86-64-x32',
+ 'DEFAULTTUNE is not set to x86-64-x32')
+ @OETestID(281)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_x32_file(self):
+ cmd = 'readelf -h /bin/ls | grep Class | grep ELF32'
+ status1 = self.target.run(cmd)[0]
+ cmd = 'readelf -h /bin/ls | grep Machine | grep X86-64'
+ status2 = self.target.run(cmd)[0]
+ msg = ("/bin/ls isn't an X86-64 ELF32 binary. readelf says: %s" %
+ self.target.run("readelf -h /bin/ls")[1])
+ self.assertTrue(status1 == 0 and status2 == 0, msg=msg)
diff --git a/meta/lib/oeqa/runtime/cases/xorg.py b/meta/lib/oeqa/runtime/cases/xorg.py
new file mode 100644
index 0000000..2124813
--- /dev/null
+++ b/meta/lib/oeqa/runtime/cases/xorg.py
@@ -0,0 +1,17 @@
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfNotFeature
+
+class XorgTest(OERuntimeTestCase):
+
+ @OETestID(1151)
+ @skipIfNotFeature('x11-base',
+ 'Test requires x11 to be in IMAGE_FEATURES')
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
+ def test_xorg_running(self):
+ cmd ='%s | grep -v xinit | grep [X]org' % self.tc.target_cmds['ps']
+ status, output = self.target.run(cmd)
+ msg = ('Xorg does not appear to be running %s' %
+ self.target.run(self.tc.target_cmds['ps'])[1])
+ self.assertEqual(status, 0, msg=msg)
diff --git a/meta/lib/oeqa/runtime_cases/buildcvs.py b/meta/lib/oeqa/runtime_cases/buildcvs.py
deleted file mode 100644
index a5ca3a5..0000000
--- a/meta/lib/oeqa/runtime_cases/buildcvs.py
+++ /dev/null
@@ -1,32 +0,0 @@
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-from oeqa.runtime.utils.targetbuildproject import TargetBuildProject
-
-def setUpModule():
- if not oeRuntimeTest.hasFeature("tools-sdk"):
- skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
-
-class BuildCvsTest(oeRuntimeTest):
-
- @classmethod
- def setUpClass(self):
- dl_dir = oeRuntimeTest.tc.d.getVar('DL_DIR', True)
- self.project = TargetBuildProject(oeRuntimeTest.tc.target,
- "http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2",
- dl_dir=dl_dir)
-
- @testcase(205)
- @skipUnlessPassed("test_ssh")
- def test_cvs(self):
- self.assertEqual(self.project.run_configure(), 0,
- msg="Running configure failed")
-
- self.assertEqual(self.project.run_make(), 0,
- msg="Running make failed")
-
- self.assertEqual(self.project.run_install(), 0,
- msg="Running make install failed")
-
- @classmethod
- def tearDownClass(self):
- self.project.clean()
diff --git a/meta/lib/oeqa/runtime_cases/buildgalculator.py b/meta/lib/oeqa/runtime_cases/buildgalculator.py
deleted file mode 100644
index 20f0a79..0000000
--- a/meta/lib/oeqa/runtime_cases/buildgalculator.py
+++ /dev/null
@@ -1,26 +0,0 @@
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-from oeqa.runtime.utils.targetbuildproject import TargetBuildProject
-
-def setUpModule():
- if not oeRuntimeTest.hasFeature("tools-sdk"):
- skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
-
-class GalculatorTest(oeRuntimeTest):
- @testcase(1526)
- @skipUnlessPassed("test_ssh")
- def test_galculator(self):
- dl_dir = oeRuntimeTest.tc.d.getVar('DL_DIR', True)
- try:
- project = TargetBuildProject(oeRuntimeTest.tc.target,
- "http://galculator.mnim.org/downloads/galculator-2.1.4.tar.bz2",
- dl_dir=dl_dir)
- project.download_archive()
-
- self.assertEqual(project.run_configure(), 0,
- msg="Running configure failed")
-
- self.assertEqual(project.run_make(), 0,
- msg="Running make failed")
- finally:
- project.clean()
diff --git a/meta/lib/oeqa/runtime_cases/buildiptables.py b/meta/lib/oeqa/runtime_cases/buildiptables.py
deleted file mode 100644
index a0e82f0..0000000
--- a/meta/lib/oeqa/runtime_cases/buildiptables.py
+++ /dev/null
@@ -1,33 +0,0 @@
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-from oeqa.runtime.utils.targetbuildproject import TargetBuildProject
-
-def setUpModule():
- if not oeRuntimeTest.hasFeature("tools-sdk"):
- skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
-
-class BuildIptablesTest(oeRuntimeTest):
-
- @classmethod
- def setUpClass(self):
- dl_dir = oeRuntimeTest.tc.d.getVar('DL_DIR', True)
- self.project = TargetBuildProject(oeRuntimeTest.tc.target,
- "http://downloads.yoctoproject.org/mirror/sources/iptables-1.4.13.tar.bz2",
- dl_dir=dl_dir)
- self.project.download_archive()
-
- @testcase(206)
- @skipUnlessPassed("test_ssh")
- def test_iptables(self):
- self.assertEqual(self.project.run_configure(), 0,
- msg="Running configure failed")
-
- self.assertEqual(self.project.run_make(), 0,
- msg="Running make failed")
-
- self.assertEqual(self.project.run_install(), 0,
- msg="Running make install failed")
-
- @classmethod
- def tearDownClass(self):
- self.project.clean()
diff --git a/meta/lib/oeqa/runtime_cases/connman.py b/meta/lib/oeqa/runtime_cases/connman.py
deleted file mode 100644
index 003fefe..0000000
--- a/meta/lib/oeqa/runtime_cases/connman.py
+++ /dev/null
@@ -1,31 +0,0 @@
-import unittest
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeRuntimeTest.hasPackage("connman"):
- skipModule("No connman package in image")
-
-
-class ConnmanTest(oeRuntimeTest):
-
- def service_status(self, service):
- if oeRuntimeTest.hasFeature("systemd"):
- (status, output) = self.target.run('systemctl status -l %s' % service)
- return output
- else:
- return "Unable to get status or logs for %s" % service
-
- @testcase(961)
- @skipUnlessPassed('test_ssh')
- def test_connmand_help(self):
- (status, output) = self.target.run('/usr/sbin/connmand --help')
- self.assertEqual(status, 0, msg="status and output: %s and %s" % (status,output))
-
- @testcase(221)
- @skipUnlessPassed('test_connmand_help')
- def test_connmand_running(self):
- (status, output) = self.target.run(oeRuntimeTest.pscmd + ' | grep [c]onnmand')
- if status != 0:
- print(self.service_status("connman"))
- self.fail("No connmand process running")
diff --git a/meta/lib/oeqa/runtime_cases/date.py b/meta/lib/oeqa/runtime_cases/date.py
deleted file mode 100644
index 6f3516a..0000000
--- a/meta/lib/oeqa/runtime_cases/date.py
+++ /dev/null
@@ -1,31 +0,0 @@
-from oeqa.oetest import oeRuntimeTest
-from oeqa.utils.decorators import *
-import re
-
-class DateTest(oeRuntimeTest):
-
- def setUpLocal(self):
- if oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager") == "systemd":
- self.target.run('systemctl stop systemd-timesyncd')
-
- def tearDownLocal(self):
- if oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager") == "systemd":
- self.target.run('systemctl start systemd-timesyncd')
-
- @testcase(211)
- @skipUnlessPassed("test_ssh")
- def test_date(self):
- (status, output) = self.target.run('date +"%Y-%m-%d %T"')
- self.assertEqual(status, 0, msg="Failed to get initial date, output: %s" % output)
- oldDate = output
-
- sampleDate = '"2016-08-09 10:00:00"'
- (status, output) = self.target.run("date -s %s" % sampleDate)
- self.assertEqual(status, 0, msg="Date set failed, output: %s" % output)
-
- (status, output) = self.target.run("date -R")
- p = re.match('Tue, 09 Aug 2016 10:00:.. \+0000', output)
- self.assertTrue(p, msg="The date was not set correctly, output: %s" % output)
-
- (status, output) = self.target.run('date -s "%s"' % oldDate)
- self.assertEqual(status, 0, msg="Failed to reset date, output: %s" % output)
diff --git a/meta/lib/oeqa/runtime_cases/df.py b/meta/lib/oeqa/runtime_cases/df.py
deleted file mode 100644
index 09569d5..0000000
--- a/meta/lib/oeqa/runtime_cases/df.py
+++ /dev/null
@@ -1,12 +0,0 @@
-import unittest
-from oeqa.oetest import oeRuntimeTest
-from oeqa.utils.decorators import *
-
-
-class DfTest(oeRuntimeTest):
-
- @testcase(234)
- @skipUnlessPassed("test_ssh")
- def test_df(self):
- (status,output) = self.target.run("df / | sed -n '2p' | awk '{print $4}'")
- self.assertTrue(int(output)>5120, msg="Not enough space on image. Current size is %s" % output)
diff --git a/meta/lib/oeqa/runtime_cases/gcc.py b/meta/lib/oeqa/runtime_cases/gcc.py
deleted file mode 100644
index 6edb89f..0000000
--- a/meta/lib/oeqa/runtime_cases/gcc.py
+++ /dev/null
@@ -1,47 +0,0 @@
-import unittest
-import os
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeRuntimeTest.hasFeature("tools-sdk"):
- skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
-
-
-class GccCompileTest(oeRuntimeTest):
-
- @classmethod
- def setUpClass(self):
- oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.corefilesdir, "test.c"), "/tmp/test.c")
- oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.filesdir, "testmakefile"), "/tmp/testmakefile")
- oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.corefilesdir, "test.cpp"), "/tmp/test.cpp")
-
- @testcase(203)
- def test_gcc_compile(self):
- (status, output) = self.target.run('gcc /tmp/test.c -o /tmp/test -lm')
- self.assertEqual(status, 0, msg="gcc compile failed, output: %s" % output)
- (status, output) = self.target.run('/tmp/test')
- self.assertEqual(status, 0, msg="running compiled file failed, output %s" % output)
-
- @testcase(200)
- def test_gpp_compile(self):
- (status, output) = self.target.run('g++ /tmp/test.c -o /tmp/test -lm')
- self.assertEqual(status, 0, msg="g++ compile failed, output: %s" % output)
- (status, output) = self.target.run('/tmp/test')
- self.assertEqual(status, 0, msg="running compiled file failed, output %s" % output)
-
- @testcase(1142)
- def test_gpp2_compile(self):
- (status, output) = self.target.run('g++ /tmp/test.cpp -o /tmp/test -lm')
- self.assertEqual(status, 0, msg="g++ compile failed, output: %s" % output)
- (status, output) = self.target.run('/tmp/test')
- self.assertEqual(status, 0, msg="running compiled file failed, output %s" % output)
-
- @testcase(204)
- def test_make(self):
- (status, output) = self.target.run('cd /tmp; make -f testmakefile')
- self.assertEqual(status, 0, msg="running make failed, output %s" % output)
-
- @classmethod
- def tearDownClass(self):
- oeRuntimeTest.tc.target.run("rm /tmp/test.c /tmp/test.o /tmp/test /tmp/testmakefile")
diff --git a/meta/lib/oeqa/runtime_cases/kernelmodule.py b/meta/lib/oeqa/runtime_cases/kernelmodule.py
deleted file mode 100644
index 2ac1bc9..0000000
--- a/meta/lib/oeqa/runtime_cases/kernelmodule.py
+++ /dev/null
@@ -1,34 +0,0 @@
-import unittest
-import os
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeRuntimeTest.hasFeature("tools-sdk"):
- skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
-
-
-class KernelModuleTest(oeRuntimeTest):
-
- def setUpLocal(self):
- self.target.copy_to(os.path.join(oeRuntimeTest.tc.filesdir, "hellomod.c"), "/tmp/hellomod.c")
- self.target.copy_to(os.path.join(oeRuntimeTest.tc.filesdir, "hellomod_makefile"), "/tmp/Makefile")
-
- @testcase('1541')
- @skipUnlessPassed('test_ssh')
- @skipUnlessPassed('test_gcc_compile')
- def test_kernel_module(self):
- cmds = [
- 'cd /usr/src/kernel && make scripts',
- 'cd /tmp && make',
- 'cd /tmp && insmod hellomod.ko',
- 'lsmod | grep hellomod',
- 'dmesg | grep Hello',
- 'rmmod hellomod', 'dmesg | grep "Cleaning up hellomod"'
- ]
- for cmd in cmds:
- (status, output) = self.target.run(cmd, 900)
- self.assertEqual(status, 0, msg="\n".join([cmd, output]))
-
- def tearDownLocal(self):
- self.target.run('rm -f /tmp/Makefile /tmp/hellomod.c')
diff --git a/meta/lib/oeqa/runtime_cases/ldd.py b/meta/lib/oeqa/runtime_cases/ldd.py
deleted file mode 100644
index 47b3885..0000000
--- a/meta/lib/oeqa/runtime_cases/ldd.py
+++ /dev/null
@@ -1,21 +0,0 @@
-import unittest
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeRuntimeTest.hasFeature("tools-sdk"):
- skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
-
-class LddTest(oeRuntimeTest):
-
- @testcase(962)
- @skipUnlessPassed('test_ssh')
- def test_ldd_exists(self):
- (status, output) = self.target.run('which ldd')
- self.assertEqual(status, 0, msg = "ldd does not exist in PATH: which ldd: %s" % output)
-
- @testcase(239)
- @skipUnlessPassed('test_ldd_exists')
- def test_ldd_rtldlist_check(self):
- (status, output) = self.target.run('for i in $(which ldd | xargs cat | grep "^RTLDLIST"|cut -d\'=\' -f2|tr -d \'"\'); do test -f $i && echo $i && break; done')
- self.assertEqual(status, 0, msg = "ldd path not correct or RTLDLIST files don't exist. ")
diff --git a/meta/lib/oeqa/runtime_cases/logrotate.py b/meta/lib/oeqa/runtime_cases/logrotate.py
deleted file mode 100644
index 063280b5..0000000
--- a/meta/lib/oeqa/runtime_cases/logrotate.py
+++ /dev/null
@@ -1,30 +0,0 @@
-# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=289 testcase
-# Note that the image under test must have logrotate installed
-
-import unittest
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeRuntimeTest.hasPackage("logrotate"):
- skipModule("No logrotate package in image")
-
-
-class LogrotateTest(oeRuntimeTest):
-
- @testcase(1544)
- @skipUnlessPassed("test_ssh")
- def test_1_logrotate_setup(self):
- (status, output) = self.target.run('mkdir $HOME/logrotate_dir')
- self.assertEqual(status, 0, msg = "Could not create logrotate_dir. Output: %s" % output)
- (status, output) = self.target.run("sed -i \"s#wtmp {#wtmp {\\n olddir $HOME/logrotate_dir#\" /etc/logrotate.conf")
- self.assertEqual(status, 0, msg = "Could not write to logrotate.conf file. Status and output: %s and %s)" % (status, output))
-
- @testcase(1542)
- @skipUnlessPassed("test_1_logrotate_setup")
- def test_2_logrotate(self):
- (status, output) = self.target.run('logrotate -f /etc/logrotate.conf')
- self.assertEqual(status, 0, msg = "logrotate service could not be reloaded. Status and output: %s and %s" % (status, output))
- output = self.target.run('ls -la $HOME/logrotate_dir/ | wc -l')[1]
- self.assertTrue(int(output)>=3, msg = "new logfile could not be created. List of files within log directory: %s" %(self.target.run('ls -la $HOME/logrotate_dir')[1]))
- self.target.run('rm -rf $HOME/logrotate_dir')
diff --git a/meta/lib/oeqa/runtime_cases/multilib.py b/meta/lib/oeqa/runtime_cases/multilib.py
deleted file mode 100644
index 5cce24f..0000000
--- a/meta/lib/oeqa/runtime_cases/multilib.py
+++ /dev/null
@@ -1,42 +0,0 @@
-import unittest
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- multilibs = oeRuntimeTest.tc.d.getVar("MULTILIBS") or ""
- if "multilib:lib32" not in multilibs:
- skipModule("this isn't a multilib:lib32 image")
-
-
-class MultilibTest(oeRuntimeTest):
-
- def archtest(self, binary, arch):
- """
- Check that ``binary`` has the ELF class ``arch`` (e.g. ELF32/ELF64).
- """
-
- (status, output) = self.target.run("readelf -h %s" % binary)
- self.assertEqual(status, 0, "Failed to readelf %s" % binary)
-
- l = [l.split()[1] for l in output.split('\n') if "Class:" in l]
- if l:
- theclass = l[0]
- else:
- self.fail("Cannot parse readelf output\n" + s)
-
- self.assertEqual(theclass, arch, msg="%s isn't %s (is %s)" % (binary, arch, theclass))
-
- @skipUnlessPassed('test_ssh')
- def test_check_multilib_libc(self):
- """
- Check that a multilib image has both 32-bit and 64-bit libc in.
- """
- self.archtest("/lib/libc.so.6", "ELF32")
- self.archtest("/lib64/libc.so.6", "ELF64")
-
- @testcase('279')
- @skipUnlessPassed('test_check_multilib_libc')
- def test_file_connman(self):
- self.assertTrue(oeRuntimeTest.hasPackage('lib32-connman'), msg="This test assumes lib32-connman is installed")
-
- self.archtest("/usr/sbin/connmand", "ELF32")
diff --git a/meta/lib/oeqa/runtime_cases/pam.py b/meta/lib/oeqa/runtime_cases/pam.py
deleted file mode 100644
index b7f2dfa..0000000
--- a/meta/lib/oeqa/runtime_cases/pam.py
+++ /dev/null
@@ -1,25 +0,0 @@
-# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=287 testcase
-# Note that the image under test must have "pam" in DISTRO_FEATURES
-
-import unittest
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeRuntimeTest.hasFeature("pam"):
- skipModule("target doesn't have 'pam' in DISTRO_FEATURES")
-
-
-class PamBasicTest(oeRuntimeTest):
-
- @testcase(1543)
- @skipUnlessPassed('test_ssh')
- def test_pam(self):
- (status, output) = self.target.run('login --help')
- self.assertEqual(status, 1, msg = "login command does not work as expected. Status and output:%s and %s" %(status, output))
- (status, output) = self.target.run('passwd --help')
- self.assertEqual(status, 0, msg = "passwd command does not work as expected. Status and output:%s and %s" %(status, output))
- (status, output) = self.target.run('su --help')
- self.assertEqual(status, 0, msg = "su command does not work as expected. Status and output:%s and %s" %(status, output))
- (status, output) = self.target.run('useradd --help')
- self.assertEqual(status, 0, msg = "useradd command does not work as expected. Status and output:%s and %s" %(status, output))
diff --git a/meta/lib/oeqa/runtime_cases/perl.py b/meta/lib/oeqa/runtime_cases/perl.py
deleted file mode 100644
index 6bf98f1..0000000
--- a/meta/lib/oeqa/runtime_cases/perl.py
+++ /dev/null
@@ -1,30 +0,0 @@
-import unittest
-import os
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeRuntimeTest.hasPackage("perl"):
- skipModule("No perl package in the image")
-
-
-class PerlTest(oeRuntimeTest):
-
- @classmethod
- def setUpClass(self):
- oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.corefilesdir, "test.pl"), "/tmp/test.pl")
-
- @testcase(1141)
- def test_perl_exists(self):
- (status, output) = self.target.run('which perl')
- self.assertEqual(status, 0, msg="Perl binary not in PATH or not on target.")
-
- @testcase(208)
- def test_perl_works(self):
- (status, output) = self.target.run('perl /tmp/test.pl')
- self.assertEqual(status, 0, msg="Exit status was not 0. Output: %s" % output)
- self.assertEqual(output, "the value of a is 0.01", msg="Incorrect output: %s" % output)
-
- @classmethod
- def tearDownClass(self):
- oeRuntimeTest.tc.target.run("rm /tmp/test.pl")
diff --git a/meta/lib/oeqa/runtime_cases/ping.py b/meta/lib/oeqa/runtime_cases/ping.py
deleted file mode 100644
index 0f27447..0000000
--- a/meta/lib/oeqa/runtime_cases/ping.py
+++ /dev/null
@@ -1,22 +0,0 @@
-import subprocess
-import unittest
-import sys
-import time
-from oeqa.oetest import oeRuntimeTest
-from oeqa.utils.decorators import *
-
-class PingTest(oeRuntimeTest):
-
- @testcase(964)
- def test_ping(self):
- output = ''
- count = 0
- endtime = time.time() + 60
- while count < 5 and time.time() < endtime:
- proc = subprocess.Popen("ping -c 1 %s" % self.target.ip, shell=True, stdout=subprocess.PIPE)
- output += proc.communicate()[0].decode("utf-8")
- if proc.poll() == 0:
- count += 1
- else:
- count = 0
- self.assertEqual(count, 5, msg = "Expected 5 consecutive replies, got %d.\nping output is:\n%s" % (count,output))
diff --git a/meta/lib/oeqa/runtime_cases/python.py b/meta/lib/oeqa/runtime_cases/python.py
deleted file mode 100644
index 93e822c..0000000
--- a/meta/lib/oeqa/runtime_cases/python.py
+++ /dev/null
@@ -1,35 +0,0 @@
-import unittest
-import os
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeRuntimeTest.hasPackage("python-core"):
- skipModule("No python package in the image")
-
-
-class PythonTest(oeRuntimeTest):
-
- @classmethod
- def setUpClass(self):
- oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.corefilesdir, "test.py"), "/tmp/test.py")
-
- @testcase(1145)
- def test_python_exists(self):
- (status, output) = self.target.run('which python')
- self.assertEqual(status, 0, msg="Python binary not in PATH or not on target.")
-
- @testcase(965)
- def test_python_stdout(self):
- (status, output) = self.target.run('python /tmp/test.py')
- self.assertEqual(status, 0, msg="Exit status was not 0. Output: %s" % output)
- self.assertEqual(output, "the value of a is 0.01", msg="Incorrect output: %s" % output)
-
- @testcase(1146)
- def test_python_testfile(self):
- (status, output) = self.target.run('ls /tmp/testfile.python')
- self.assertEqual(status, 0, msg="Python test file generate failed.")
-
- @classmethod
- def tearDownClass(self):
- oeRuntimeTest.tc.target.run("rm /tmp/test.py /tmp/testfile.python")
diff --git a/meta/lib/oeqa/runtime_cases/rpm.py b/meta/lib/oeqa/runtime_cases/rpm.py
deleted file mode 100644
index f1c4763..0000000
--- a/meta/lib/oeqa/runtime_cases/rpm.py
+++ /dev/null
@@ -1,120 +0,0 @@
-import unittest
-import os
-import fnmatch
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeRuntimeTest.hasFeature("package-management"):
- skipModule("rpm module skipped: target doesn't have package-management in IMAGE_FEATURES")
- if "package_rpm" != oeRuntimeTest.tc.d.getVar("PACKAGE_CLASSES").split()[0]:
- skipModule("rpm module skipped: target doesn't have rpm as primary package manager")
-
-
-class RpmBasicTest(oeRuntimeTest):
-
- @testcase(960)
- @skipUnlessPassed('test_ssh')
- def test_rpm_help(self):
- (status, output) = self.target.run('rpm --help')
- self.assertEqual(status, 0, msg="status and output: %s and %s" % (status,output))
-
- @testcase(191)
- @skipUnlessPassed('test_rpm_help')
- def test_rpm_query(self):
- (status, output) = self.target.run('rpm -q rpm')
- self.assertEqual(status, 0, msg="status and output: %s and %s" % (status,output))
-
-class RpmInstallRemoveTest(oeRuntimeTest):
-
- @classmethod
- def setUpClass(self):
- pkgarch = oeRuntimeTest.tc.d.getVar('TUNE_PKGARCH').replace("-", "_")
- rpmdir = os.path.join(oeRuntimeTest.tc.d.getVar('DEPLOY_DIR'), "rpm", pkgarch)
- # pick rpm-doc as a test file to get installed, because it's small and it will always be built for standard targets
- for f in fnmatch.filter(os.listdir(rpmdir), "rpm-doc-*.%s.rpm" % pkgarch):
- testrpmfile = f
- oeRuntimeTest.tc.target.copy_to(os.path.join(rpmdir,testrpmfile), "/tmp/rpm-doc.rpm")
-
- @testcase(192)
- @skipUnlessPassed('test_rpm_help')
- def test_rpm_install(self):
- (status, output) = self.target.run('rpm -ivh /tmp/rpm-doc.rpm')
- self.assertEqual(status, 0, msg="Failed to install rpm-doc package: %s" % output)
-
- @testcase(194)
- @skipUnlessPassed('test_rpm_install')
- def test_rpm_remove(self):
- (status,output) = self.target.run('rpm -e rpm-doc')
- self.assertEqual(status, 0, msg="Failed to remove rpm-doc package: %s" % output)
-
- @testcase(1096)
- @skipUnlessPassed('test_ssh')
- def test_rpm_query_nonroot(self):
-
- def set_up_test_user(u):
- (status, output) = self.target.run("id -u %s" % u)
- if status == 0:
- pass
- else:
- (status, output) = self.target.run("useradd %s" % u)
- self.assertTrue(status == 0, msg="Failed to create new user: " + output)
-
- def exec_as_test_user(u):
- (status, output) = self.target.run("su -c id %s" % u)
- self.assertTrue("({0})".format(u) in output, msg="Failed to execute as new user")
- (status, output) = self.target.run("su -c \"rpm -qa\" %s " % u)
- self.assertEqual(status, 0, msg="status: %s. Cannot run rpm -qa: %s" % (status, output))
-
- def unset_up_test_user(u):
- (status, output) = self.target.run("userdel -r %s" % u)
- self.assertTrue(status == 0, msg="Failed to erase user: %s" % output)
-
- tuser = 'test1'
-
- try:
- set_up_test_user(tuser)
- exec_as_test_user(tuser)
- finally:
- unset_up_test_user(tuser)
-
- @testcase(195)
- @skipUnlessPassed('test_rpm_install')
- def test_check_rpm_install_removal_log_file_size(self):
- """
- Summary: Check rpm install/removal log file size
- Expected: There should be some method to keep rpm log in a small size .
- Product: BSPs
- Author: Alexandru Georgescu <alexandru.c.georgescu@intel.com>
- AutomatedBy: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
- """
- db_files_cmd = 'ls /var/lib/rpm/__db.*'
- get_log_size_cmd = "du /var/lib/rpm/log/log.* | awk '{print $1}'"
-
- # Make sure that some database files are under /var/lib/rpm as '__db.xxx'
- (status, output) = self.target.run(db_files_cmd)
- self.assertEqual(0, status, 'Failed to find database files under /var/lib/rpm/ as __db.xxx')
-
- # Remove the package just in case
- self.target.run('rpm -e rpm-doc')
-
- # Install/Remove a package 10 times
- for i in range(10):
- (status, output) = self.target.run('rpm -ivh /tmp/rpm-doc.rpm')
- self.assertEqual(0, status, "Failed to install rpm-doc package. Reason: {}".format(output))
-
- (status, output) = self.target.run('rpm -e rpm-doc')
- self.assertEqual(0, status, "Failed to remove rpm-doc package. Reason: {}".format(output))
-
- # Get the size of log file
- (status, output) = self.target.run(get_log_size_cmd)
- self.assertEqual(0, status, 'Failed to get the final size of the log file.')
-
- # Compare each log size
- for log_file_size in output:
- self.assertLessEqual(int(log_file_size), 11264,
- 'Log file size is greater that expected (~10MB), found {} bytes'.format(log_file_size))
-
- @classmethod
- def tearDownClass(self):
- oeRuntimeTest.tc.target.run('rm -f /tmp/rpm-doc.rpm')
diff --git a/meta/lib/oeqa/runtime_cases/scanelf.py b/meta/lib/oeqa/runtime_cases/scanelf.py
deleted file mode 100644
index 67e02ff..0000000
--- a/meta/lib/oeqa/runtime_cases/scanelf.py
+++ /dev/null
@@ -1,28 +0,0 @@
-import unittest
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeRuntimeTest.hasPackage("pax-utils"):
- skipModule("pax-utils package not installed")
-
-class ScanelfTest(oeRuntimeTest):
-
- def setUpLocal(self):
- self.scancmd = 'scanelf --quiet --recursive --mount --ldpath --path'
-
- @testcase(966)
- @skipUnlessPassed('test_ssh')
- def test_scanelf_textrel(self):
- # print TEXTREL information
- self.scancmd += " --textrel"
- (status, output) = self.target.run(self.scancmd)
- self.assertEqual(output.strip(), "", "\n".join([self.scancmd, output]))
-
- @testcase(967)
- @skipUnlessPassed('test_ssh')
- def test_scanelf_rpath(self):
- # print RPATH information
- self.scancmd += " --rpath"
- (status, output) = self.target.run(self.scancmd)
- self.assertEqual(output.strip(), "", "\n".join([self.scancmd, output]))
diff --git a/meta/lib/oeqa/runtime_cases/scp.py b/meta/lib/oeqa/runtime_cases/scp.py
deleted file mode 100644
index cf36cfa..0000000
--- a/meta/lib/oeqa/runtime_cases/scp.py
+++ /dev/null
@@ -1,22 +0,0 @@
-import os
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import skipUnlessPassed, testcase
-
-def setUpModule():
- if not (oeRuntimeTest.hasPackage("dropbear") or oeRuntimeTest.hasPackage("openssh-sshd")):
- skipModule("No ssh package in image")
-
-class ScpTest(oeRuntimeTest):
-
- @testcase(220)
- @skipUnlessPassed('test_ssh')
- def test_scp_file(self):
- test_log_dir = oeRuntimeTest.tc.d.getVar("TEST_LOG_DIR")
- test_file_path = os.path.join(test_log_dir, 'test_scp_file')
- with open(test_file_path, 'w') as test_scp_file:
- test_scp_file.seek(2 ** 22 - 1)
- test_scp_file.write(os.linesep)
- (status, output) = self.target.copy_to(test_file_path, '/tmp/test_scp_file')
- self.assertEqual(status, 0, msg = "File could not be copied. Output: %s" % output)
- (status, output) = self.target.run("ls -la /tmp/test_scp_file")
- self.assertEqual(status, 0, msg = "SCP test failed")
diff --git a/meta/lib/oeqa/runtime_cases/skeletoninit.py b/meta/lib/oeqa/runtime_cases/skeletoninit.py
deleted file mode 100644
index cb0cb9b..0000000
--- a/meta/lib/oeqa/runtime_cases/skeletoninit.py
+++ /dev/null
@@ -1,29 +0,0 @@
-# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=284 testcase
-# Note that the image under test must have meta-skeleton layer in bblayers and IMAGE_INSTALL_append = " service" in local.conf
-
-import unittest
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeRuntimeTest.hasPackage("service"):
- skipModule("No service package in image")
-
-
-class SkeletonBasicTest(oeRuntimeTest):
-
- @skipUnlessPassed('test_ssh')
- @unittest.skipIf("systemd" == oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager", False), "Not appropiate for systemd image")
- def test_skeleton_availability(self):
- (status, output) = self.target.run('ls /etc/init.d/skeleton')
- self.assertEqual(status, 0, msg = "skeleton init script not found. Output:\n%s " % output)
- (status, output) = self.target.run('ls /usr/sbin/skeleton-test')
- self.assertEqual(status, 0, msg = "skeleton-test not found. Output:\n%s" % output)
-
- @testcase(284)
- @skipUnlessPassed('test_skeleton_availability')
- @unittest.skipIf("systemd" == oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager", False), "Not appropiate for systemd image")
- def test_skeleton_script(self):
- output1 = self.target.run("/etc/init.d/skeleton start")[1]
- (status, output2) = self.target.run(oeRuntimeTest.pscmd + ' | grep [s]keleton-test')
- self.assertEqual(status, 0, msg = "Skeleton script could not be started:\n%s\n%s" % (output1, output2))
diff --git a/meta/lib/oeqa/runtime_cases/ssh.py b/meta/lib/oeqa/runtime_cases/ssh.py
deleted file mode 100644
index 0e76d5d..0000000
--- a/meta/lib/oeqa/runtime_cases/ssh.py
+++ /dev/null
@@ -1,19 +0,0 @@
-import subprocess
-import unittest
-import sys
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not (oeRuntimeTest.hasPackage("dropbear") or oeRuntimeTest.hasPackage("openssh")):
- skipModule("No ssh package in image")
-
-class SshTest(oeRuntimeTest):
-
- @testcase(224)
- @skipUnlessPassed('test_ping')
- def test_ssh(self):
- (status, output) = self.target.run('uname -a')
- self.assertEqual(status, 0, msg="SSH Test failed: %s" % output)
- (status, output) = self.target.run('cat /etc/masterimage')
- self.assertEqual(status, 1, msg="This isn't the right image - /etc/masterimage shouldn't be here %s" % output)
diff --git a/meta/lib/oeqa/runtime_cases/syslog.py b/meta/lib/oeqa/runtime_cases/syslog.py
deleted file mode 100644
index 8f55032..0000000
--- a/meta/lib/oeqa/runtime_cases/syslog.py
+++ /dev/null
@@ -1,52 +0,0 @@
-import unittest
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not (oeRuntimeTest.hasPackage("busybox-syslog") or oeRuntimeTest.hasPackage("sysklogd")):
- skipModule("No syslog package in image")
-
-class SyslogTest(oeRuntimeTest):
-
- @testcase(201)
- def test_syslog_running(self):
- (status,output) = self.target.run(oeRuntimeTest.pscmd + ' | grep -i [s]yslogd')
- self.assertEqual(status, 0, msg="no syslogd process, ps output: %s" % self.target.run(oeRuntimeTest.pscmd)[1])
-
-class SyslogTestConfig(oeRuntimeTest):
-
- @testcase(1149)
- @skipUnlessPassed("test_syslog_running")
- def test_syslog_logger(self):
- (status, output) = self.target.run('logger foobar')
- self.assertEqual(status, 0, msg="Can't log into syslog. Output: %s " % output)
-
- (status, output) = self.target.run('grep foobar /var/log/messages')
- if status != 0:
- if oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager", "") == "systemd":
- (status, output) = self.target.run('journalctl -o cat | grep foobar')
- else:
- (status, output) = self.target.run('logread | grep foobar')
- self.assertEqual(status, 0, msg="Test log string not found in /var/log/messages or logread. Output: %s " % output)
-
- @testcase(1150)
- @skipUnlessPassed("test_syslog_running")
- def test_syslog_restart(self):
- if "systemd" != oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager", False):
- (status,output) = self.target.run('/etc/init.d/syslog restart')
- else:
- (status,output) = self.target.run('systemctl restart syslog.service')
-
- @testcase(202)
- @skipUnlessPassed("test_syslog_restart")
- @skipUnlessPassed("test_syslog_logger")
- @unittest.skipIf("systemd" == oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager", False), "Not appropiate for systemd image")
- @unittest.skipIf(oeRuntimeTest.hasPackage("sysklogd") or not oeRuntimeTest.hasPackage("busybox"), "Non-busybox syslog")
- def test_syslog_startup_config(self):
- self.target.run('echo "LOGFILE=/var/log/test" >> /etc/syslog-startup.conf')
- (status,output) = self.target.run('/etc/init.d/syslog restart')
- self.assertEqual(status, 0, msg="Could not restart syslog service. Status and output: %s and %s" % (status,output))
- (status,output) = self.target.run('logger foobar && grep foobar /var/log/test')
- self.assertEqual(status, 0, msg="Test log string not found. Output: %s " % output)
- self.target.run("sed -i 's#LOGFILE=/var/log/test##' /etc/syslog-startup.conf")
- self.target.run('/etc/init.d/syslog restart')
diff --git a/meta/lib/oeqa/runtime_cases/x32lib.py b/meta/lib/oeqa/runtime_cases/x32lib.py
deleted file mode 100644
index 2f98dbf..0000000
--- a/meta/lib/oeqa/runtime_cases/x32lib.py
+++ /dev/null
@@ -1,18 +0,0 @@
-import unittest
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- #check if DEFAULTTUNE is set and it's value is: x86-64-x32
- defaulttune = oeRuntimeTest.tc.d.getVar("DEFAULTTUNE")
- if "x86-64-x32" not in defaulttune:
- skipModule("DEFAULTTUNE is not set to x86-64-x32")
-
-class X32libTest(oeRuntimeTest):
-
- @testcase(281)
- @skipUnlessPassed("test_ssh")
- def test_x32_file(self):
- status1 = self.target.run("readelf -h /bin/ls | grep Class | grep ELF32")[0]
- status2 = self.target.run("readelf -h /bin/ls | grep Machine | grep X86-64")[0]
- self.assertTrue(status1 == 0 and status2 == 0, msg="/bin/ls isn't an X86-64 ELF32 binary. readelf says: %s" % self.target.run("readelf -h /bin/ls")[1])
diff --git a/meta/lib/oeqa/runtime_cases/xorg.py b/meta/lib/oeqa/runtime_cases/xorg.py
deleted file mode 100644
index 12bcd37..0000000
--- a/meta/lib/oeqa/runtime_cases/xorg.py
+++ /dev/null
@@ -1,16 +0,0 @@
-import unittest
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
-
-def setUpModule():
- if not oeRuntimeTest.hasFeature("x11-base"):
- skipModule("target doesn't have x11 in IMAGE_FEATURES")
-
-
-class XorgTest(oeRuntimeTest):
-
- @testcase(1151)
- @skipUnlessPassed('test_ssh')
- def test_xorg_running(self):
- (status, output) = self.target.run(oeRuntimeTest.pscmd + ' | grep -v xinit | grep [X]org')
- self.assertEqual(status, 0, msg="Xorg does not appear to be running %s" % self.target.run(oeRuntimeTest.pscmd)[1])
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 49/55] runtime/cases/smart.py: Migrate smart tests
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (47 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 48/55] oeqa/runtime/cases: Migrate runtime tests Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 50/55] oeqa/core/utils/test.py: Add functions to get module path Aníbal Limón
` (7 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This migrates the smart test from the old framework to
the new one. This has its own commit because smart
test was using bb and oe libraries that are available
when exporting the test cases to run in a different host.
Because of the removal of bb and oe libraries index and
packages feeds creation will be managed in testimage bbclass.
[YOCTO #10234]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/classes/testimage.bbclass | 53 +++++++++++
meta/lib/oeqa/runtime/cases/smart.py | 172 +++++++++++++++--------------------
2 files changed, 128 insertions(+), 97 deletions(-)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 016c1c1..abcecca 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -116,6 +116,10 @@ python do_testimage() {
testimage_sanity(d)
+ if (d.getVar('IMAGE_PKGTYPE') == 'rpm'
+ and 'smart' in d.getVar('TEST_SUITES')):
+ create_rpm_index(d)
+
testimage_main(d)
}
@@ -284,6 +288,55 @@ def get_runtime_paths(d):
paths.append(path)
return paths
+def create_index(arg):
+ import subprocess
+
+ index_cmd = arg
+ try:
+ bb.note("Executing '%s' ..." % index_cmd)
+ result = subprocess.check_output(index_cmd,
+ stderr=subprocess.STDOUT,
+ shell=True)
+ result = result.decode('utf-8')
+ except subprocess.CalledProcessError as e:
+ return("Index creation command '%s' failed with return code "
+ '%d:\n%s' % (e.cmd, e.returncode, e.output.decode("utf-8")))
+ if result:
+ bb.note(result)
+ return None
+
+def create_rpm_index(d):
+ # Index RPMs
+ rpm_createrepo = bb.utils.which(os.getenv('PATH'), "createrepo")
+ index_cmds = []
+ archs = (d.getVar('ALL_MULTILIB_PACKAGE_ARCHS') or '').replace('-', '_')
+
+ for arch in archs.split():
+ rpm_dir = os.path.join(d.getVar('DEPLOY_DIR_RPM'), arch)
+ idx_path = os.path.join(d.getVar('WORKDIR'), 'rpm', arch)
+ db_path = os.path.join(d.getVar('WORKDIR'), 'rpmdb', arch)
+
+ if not os.path.isdir(rpm_dir):
+ continue
+ if os.path.exists(db_path):
+ bb.utils.remove(dbpath, True)
+
+ lockfilename = os.path.join(d.getVar('DEPLOY_DIR_RPM'), 'rpm.lock')
+ lf = bb.utils.lockfile(lockfilename, False)
+ oe.path.copyhardlinktree(rpm_dir, idx_path)
+ # Full indexes overload a 256MB image so reduce the number of rpms
+ # in the feed. Filter to p* since we use the psplash packages and
+ # this leaves some allarch and machine arch packages too.
+ bb.utils.remove(idx_path + "*/[a-oq-z]*.rpm")
+ bb.utils.unlockfile(lf)
+ cmd = '%s --dbpath %s --update -q %s' % (rpm_createrepo,
+ db_path, idx_path)
+
+ # Create repodata
+ result = create_index(cmd)
+ if result:
+ bb.fatal('%s' % ('\n'.join(result)))
+
def test_create_extract_dirs(d):
install_path = d.getVar("TEST_INSTALL_TMP_DIR")
package_path = d.getVar("TEST_PACKAGED_DIR")
diff --git a/meta/lib/oeqa/runtime/cases/smart.py b/meta/lib/oeqa/runtime/cases/smart.py
index dde1c4d..9b4d0d2 100644
--- a/meta/lib/oeqa/runtime/cases/smart.py
+++ b/meta/lib/oeqa/runtime/cases/smart.py
@@ -1,166 +1,142 @@
-import unittest
+import os
import re
-import oe
import subprocess
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.decorators import *
from oeqa.utils.httpserver import HTTPService
-def setUpModule():
- if not oeRuntimeTest.hasFeature("package-management"):
- skipModule("Image doesn't have package management feature")
- if not oeRuntimeTest.hasPackage("smartpm"):
- skipModule("Image doesn't have smart installed")
- if "package_rpm" != oeRuntimeTest.tc.d.getVar("PACKAGE_CLASSES").split()[0]:
- skipModule("Rpm is not the primary package manager")
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.core.decorator.oeid import OETestID
+from oeqa.core.decorator.data import skipIfNotDataVar, skipIfNotFeature
+from oeqa.runtime.decorator.package import OEHasPackage
-class SmartTest(oeRuntimeTest):
+class SmartTest(OERuntimeTestCase):
- @skipUnlessPassed('test_smart_help')
def smart(self, command, expected = 0):
command = 'smart %s' % command
status, output = self.target.run(command, 1500)
message = os.linesep.join([command, output])
self.assertEqual(status, expected, message)
- self.assertFalse("Cannot allocate memory" in output, message)
+ self.assertFalse('Cannot allocate memory' in output, message)
return output
class SmartBasicTest(SmartTest):
- @testcase(716)
- @skipUnlessPassed('test_ssh')
+ @skipIfNotFeature('package-management',
+ 'Test requires package-management to be in IMAGE_FEATURES')
+ @skipIfNotDataVar('PACKAGE_CLASSES', 'package_rpm',
+ 'RPM is not the primary package manager')
+ @OEHasPackage(['smartpm'])
+ @OETestID(716)
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
def test_smart_help(self):
self.smart('--help')
- @testcase(968)
+ @OETestID(968)
+ @OETestDepends(['smart.SmartBasicTest.test_smart_help'])
def test_smart_version(self):
self.smart('--version')
- @testcase(721)
+ @OETestID(721)
+ @OETestDepends(['smart.SmartBasicTest.test_smart_help'])
def test_smart_info(self):
self.smart('info python-smartpm')
- @testcase(421)
+ @OETestID(421)
+ @OETestDepends(['smart.SmartBasicTest.test_smart_help'])
def test_smart_query(self):
self.smart('query python-smartpm')
- @testcase(720)
+ @OETestID(720)
+ @OETestDepends(['smart.SmartBasicTest.test_smart_help'])
def test_smart_search(self):
self.smart('search python-smartpm')
- @testcase(722)
+ @OETestID(722)
+ @OETestDepends(['smart.SmartBasicTest.test_smart_help'])
def test_smart_stats(self):
self.smart('stats')
class SmartRepoTest(SmartTest):
@classmethod
- def create_index(self, arg):
- index_cmd = arg
- try:
- bb.note("Executing '%s' ..." % index_cmd)
- result = subprocess.check_output(index_cmd, stderr=subprocess.STDOUT, shell=True).decode("utf-8")
- except subprocess.CalledProcessError as e:
- return("Index creation command '%s' failed with return code %d:\n%s" %
- (e.cmd, e.returncode, e.output.decode("utf-8")))
- if result:
- bb.note(result)
- return None
+ def setUpClass(cls):
+ cls.repolist = []
+ cls.repo_server = HTTPService(cls.tc.td['WORKDIR'],
+ cls.tc.target.server_ip)
+ cls.repo_server.start()
@classmethod
- def setUpClass(self):
- self.repolist = []
-
- # Index RPMs
- rpm_createrepo = bb.utils.which(os.getenv('PATH'), "createrepo")
- index_cmds = []
- rpm_dirs_found = False
- archs = (oeRuntimeTest.tc.d.getVar('ALL_MULTILIB_PACKAGE_ARCHS') or "").replace('-', '_').split()
- for arch in archs:
- rpm_dir = os.path.join(oeRuntimeTest.tc.d.getVar('DEPLOY_DIR_RPM'), arch)
- idx_path = os.path.join(oeRuntimeTest.tc.d.getVar('WORKDIR'), 'rpm', arch)
- db_path = os.path.join(oeRuntimeTest.tc.d.getVar('WORKDIR'), 'rpmdb', arch)
- if not os.path.isdir(rpm_dir):
- continue
- if os.path.exists(db_path):
- bb.utils.remove(dbpath, True)
- lockfilename = oeRuntimeTest.tc.d.getVar('DEPLOY_DIR_RPM') + "/rpm.lock"
- lf = bb.utils.lockfile(lockfilename, False)
- oe.path.copyhardlinktree(rpm_dir, idx_path)
- # Full indexes overload a 256MB image so reduce the number of rpms
- # in the feed. Filter to p* since we use the psplash packages and
- # this leaves some allarch and machine arch packages too.
- bb.utils.remove(idx_path + "*/[a-oq-z]*.rpm")
- bb.utils.unlockfile(lf)
- index_cmds.append("%s --dbpath %s --update -q %s" % (rpm_createrepo, db_path, idx_path))
- rpm_dirs_found = True
- # Create repodata¬
- result = oe.utils.multiprocess_exec(index_cmds, self.create_index)
- if result:
- bb.fatal('%s' % ('\n'.join(result)))
- self.repo_server = HTTPService(oeRuntimeTest.tc.d.getVar('WORKDIR'), oeRuntimeTest.tc.target.server_ip)
- self.repo_server.start()
-
- @classmethod
- def tearDownClass(self):
- self.repo_server.stop()
- for i in self.repolist:
- oeRuntimeTest.tc.target.run('smart channel -y --remove '+str(i))
+ def tearDownClass(cls):
+ cls.repo_server.stop()
+ for repo in cls.repolist:
+ cls.tc.target.run('smart channel -y --remove %s' % repo)
- @testcase(1143)
+ @OETestID(1143)
+ @OETestDepends(['smart.SmartBasicTest.test_smart_help'])
def test_smart_channel(self):
self.smart('channel', 1)
- @testcase(719)
+ @OETestID(719)
+ @OETestDepends(['smart.SmartBasicTest.test_smart_help'])
def test_smart_channel_add(self):
- image_pkgtype = self.tc.d.getVar('IMAGE_PKGTYPE')
- deploy_url = 'http://%s:%s/%s' %(self.target.server_ip, self.repo_server.port, image_pkgtype)
- pkgarchs = self.tc.d.getVar('PACKAGE_ARCHS').replace("-","_").split()
- for arch in os.listdir('%s/%s' % (self.repo_server.root_dir, image_pkgtype)):
+ image_pkgtype = self.tc.td['IMAGE_PKGTYPE']
+ deploy_url = 'http://%s:%s/%s' % (self.target.server_ip,
+ self.repo_server.port,
+ image_pkgtype)
+ pkgarchs = self.tc.td['PACKAGE_ARCHS'].replace("-","_").split()
+ archs = os.listdir(os.path.join(self.repo_server.root_dir,
+ image_pkgtype))
+ for arch in archs:
if arch in pkgarchs:
- self.smart('channel -y --add {a} type=rpm-md baseurl={u}/{a}'.format(a=arch, u=deploy_url))
+ cmd = ('channel -y --add {a} type=rpm-md '
+ 'baseurl={u}/{a}'.format(a=arch, u=deploy_url))
+ self.smart(cmd)
self.repolist.append(arch)
self.smart('update')
- @testcase(969)
+ @OETestID(969)
+ @OETestDepends(['smart.SmartBasicTest.test_smart_help'])
def test_smart_channel_help(self):
self.smart('channel --help')
- @testcase(970)
+ @OETestID(970)
+ @OETestDepends(['smart.SmartBasicTest.test_smart_help'])
def test_smart_channel_list(self):
self.smart('channel --list')
- @testcase(971)
+ @OETestID(971)
+ @OETestDepends(['smart.SmartBasicTest.test_smart_help'])
def test_smart_channel_show(self):
self.smart('channel --show')
- @testcase(717)
+ @OETestID(717)
+ @OETestDepends(['smart.SmartBasicTest.test_smart_help'])
def test_smart_channel_rpmsys(self):
self.smart('channel --show rpmsys')
self.smart('channel --disable rpmsys')
self.smart('channel --enable rpmsys')
- @testcase(1144)
- @skipUnlessPassed('test_smart_channel_add')
+ @OETestID(1144)
+ @OETestDepends(['smart.SmartRepoTest.test_smart_channel_add'])
def test_smart_install(self):
self.smart('remove -y psplash-default')
self.smart('install -y psplash-default')
- @testcase(728)
- @skipUnlessPassed('test_smart_install')
+ @OETestID(728)
+ @OETestDepends(['smart.SmartRepoTest.test_smart_install'])
def test_smart_install_dependency(self):
self.smart('remove -y psplash')
self.smart('install -y psplash-default')
- @testcase(723)
- @skipUnlessPassed('test_smart_channel_add')
+ @OETestID(723)
+ @OETestDepends(['smart.SmartRepoTest.test_smart_channel_add'])
def test_smart_install_from_disk(self):
self.smart('remove -y psplash-default')
self.smart('download psplash-default')
self.smart('install -y ./psplash-default*')
- @testcase(725)
- @skipUnlessPassed('test_smart_channel_add')
+ @OETestID(725)
+ @OETestDepends(['smart.SmartRepoTest.test_smart_channel_add'])
def test_smart_install_from_http(self):
output = self.smart('download --urls psplash-default')
url = re.search('(http://.*/psplash-default.*\.rpm)', output)
@@ -168,19 +144,20 @@ class SmartRepoTest(SmartTest):
self.smart('remove -y psplash-default')
self.smart('install -y %s' % url.group(0))
- @testcase(729)
- @skipUnlessPassed('test_smart_install')
+ @OETestID(729)
+ @OETestDepends(['smart.SmartRepoTest.test_smart_install'])
def test_smart_reinstall(self):
self.smart('reinstall -y psplash-default')
- @testcase(727)
- @skipUnlessPassed('test_smart_channel_add')
+ @OETestID(727)
+ @OETestDepends(['smart.SmartRepoTest.test_smart_channel_add'])
def test_smart_remote_repo(self):
self.smart('update')
self.smart('install -y psplash')
self.smart('remove -y psplash')
- @testcase(726)
+ @OETestID(726)
+ @OETestDepends(['smart.SmartBasicTest.test_smart_help'])
def test_smart_local_dir(self):
self.target.run('mkdir /tmp/myrpmdir')
self.smart('channel --add myrpmdir type=rpm-dir path=/tmp/myrpmdir -y')
@@ -198,7 +175,8 @@ class SmartRepoTest(SmartTest):
self.smart('channel --remove myrpmdir -y')
self.target.run("rm -rf /tmp/myrpmdir")
- @testcase(718)
+ @OETestID(718)
+ @OETestDepends(['smart.SmartBasicTest.test_smart_help'])
def test_smart_add_rpmdir(self):
self.target.run('mkdir /tmp/myrpmdir')
self.smart('channel --add myrpmdir type=rpm-dir path=/tmp/myrpmdir -y')
@@ -211,8 +189,8 @@ class SmartRepoTest(SmartTest):
self.smart('channel --remove myrpmdir -y')
self.target.run("rm -rf /tmp/myrpmdir")
- @testcase(731)
- @skipUnlessPassed('test_smart_channel_add')
+ @OETestID(731)
+ @OETestDepends(['smart.SmartRepoTest.test_smart_channel_add'])
def test_smart_remove_package(self):
self.smart('install -y psplash')
self.smart('remove -y psplash')
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 50/55] oeqa/core/utils/test.py: Add functions to get module path
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (48 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 49/55] runtime/cases/smart.py: Migrate smart tests Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 51/55] testimage.bbclass: Add support for package extraction Aníbal Limón
` (6 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This will add functions to get module file path from a test
case or a complete suite.
[YOCTO #10234]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/utils/test.py | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/meta/lib/oeqa/core/utils/test.py b/meta/lib/oeqa/core/utils/test.py
index 820b997..88d1765 100644
--- a/meta/lib/oeqa/core/utils/test.py
+++ b/meta/lib/oeqa/core/utils/test.py
@@ -2,6 +2,7 @@
# Released under the MIT license (see COPYING.MIT)
import os
+import inspect
import unittest
def getSuiteCases(suite):
@@ -46,6 +47,12 @@ def getSuiteCasesIDs(suite):
"""
return getSuiteCasesInfo(suite, getCaseID)
+def getSuiteCasesFiles(suite):
+ """
+ Returns test case files paths from suite.
+ """
+ return getSuiteCasesInfo(suite, getCaseFile)
+
def getCaseModule(test_case):
"""
Returns test case module name.
@@ -64,6 +71,12 @@ def getCaseID(test_case):
"""
return test_case.id()
+def getCaseFile(test_case):
+ """
+ Returns test case file path.
+ """
+ return inspect.getsourcefile(test_case.__class__)
+
def getCaseMethod(test_case):
"""
Returns test case method name.
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 51/55] testimage.bbclass: Add support for package extraction
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (49 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 50/55] oeqa/core/utils/test.py: Add functions to get module path Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 52/55] testimage.bbclass: Add package install feature Aníbal Limón
` (5 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
testimage support the installation of packages without a package
manager in the target. This adds support for package extraction
required to support the installation feature.
[YOCTO #10234]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/classes/testimage.bbclass | 12 +++
meta/lib/oeqa/utils/package_manager.py | 161 +++++++++++++++++++++++++++++++++
2 files changed, 173 insertions(+)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index abcecca..1dfbc49 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -243,6 +243,8 @@ def testimage_main(d):
test_modules = d.getVar('TEST_SUITES')
tc.loadTests(test_paths, modules=test_modules)
+ package_extraction(d, tc.suites)
+
bootparams = None
if d.getVar('VIRTUAL-RUNTIME_init_manager', '') == 'systemd':
bootparams = 'systemd.log_level=debug systemd.log_target=console'
@@ -337,12 +339,22 @@ def create_rpm_index(d):
if result:
bb.fatal('%s' % ('\n'.join(result)))
+def package_extraction(d, test_suites):
+ from oeqa.utils.package_manager import find_packages_to_extract
+ from oeqa.utils.package_manager import extract_packages
+
+ test_create_extract_dirs(d)
+ packages = find_packages_to_extract(test_suites)
+ extract_packages(d, packages)
+
def test_create_extract_dirs(d):
install_path = d.getVar("TEST_INSTALL_TMP_DIR")
package_path = d.getVar("TEST_PACKAGED_DIR")
extracted_path = d.getVar("TEST_EXTRACTED_DIR")
bb.utils.mkdirhier(d.getVar("TEST_LOG_DIR"))
+ bb.utils.remove(install_path, recurse=True)
bb.utils.remove(package_path, recurse=True)
+ bb.utils.remove(extracted_path, recurse=True)
bb.utils.mkdirhier(install_path)
bb.utils.mkdirhier(package_path)
bb.utils.mkdirhier(extracted_path)
diff --git a/meta/lib/oeqa/utils/package_manager.py b/meta/lib/oeqa/utils/package_manager.py
index 0f6bdbc..ab0d3e5 100644
--- a/meta/lib/oeqa/utils/package_manager.py
+++ b/meta/lib/oeqa/utils/package_manager.py
@@ -1,3 +1,8 @@
+import bb
+import json
+import shutil
+
+
def get_package_manager(d, root_path):
"""
Returns an OE package manager that can install packages in root_path.
@@ -27,3 +32,159 @@ def get_package_manager(d, root_path):
pm.update()
return pm
+
+def find_packages_to_extract(test_suite):
+ """
+ Returns packages to extract required by runtime tests.
+ """
+ from oeqa.core.utils.test import getSuiteCasesFiles
+
+ needed_packages = {}
+ files = getSuiteCasesFiles(test_suite)
+
+ for f in set(files):
+ json_file = _get_json_file(f)
+ if json_file:
+ needed_packages.update(_get_needed_packages(json_file))
+
+ return needed_packages
+
+def _get_json_file(module_path):
+ """
+ Returns the path of the JSON file for a module, empty if doesn't exitst.
+ """
+
+ json_file = '%s.json' % module_path.rsplit('.', 1)[0]
+ if os.path.isfile(module_path) and os.path.isfile(json_file):
+ return json_file
+ else:
+ return ''
+
+def _get_needed_packages(json_file, test=None):
+ """
+ Returns a dict with needed packages based on a JSON file.
+
+ If a test is specified it will return the dict just for that test.
+ """
+ needed_packages = {}
+
+ with open(json_file) as f:
+ test_packages = json.load(f)
+ for key,value in test_packages.items():
+ needed_packages[key] = value
+
+ if test:
+ if test in needed_packages:
+ needed_packages = needed_packages[test]
+ else:
+ needed_packages = {}
+
+ return needed_packages
+
+def extract_packages(d, needed_packages):
+ """
+ Extract packages that will be needed during runtime.
+ """
+
+ import oe.path
+
+ extracted_path = d.getVar('TEST_EXTRACTED_DIR')
+
+ for key,value in needed_packages.items():
+ packages = ()
+ if isinstance(value, dict):
+ packages = (value, )
+ elif isinstance(value, list):
+ packages = value
+ else:
+ bb.fatal('Failed to process needed packages for %s; '
+ 'Value must be a dict or list' % key)
+
+ for package in packages:
+ pkg = package['pkg']
+ rm = package.get('rm', False)
+ extract = package.get('extract', True)
+
+ if extract:
+ #logger.debug(1, 'Extracting %s' % pkg)
+ dst_dir = os.path.join(extracted_path, pkg)
+ # Same package used for more than one test,
+ # don't need to extract again.
+ if os.path.exists(dst_dir):
+ continue
+
+ # Extract package and copy it to TEST_EXTRACTED_DIR
+ pkg_dir = _extract_in_tmpdir(d, pkg)
+ oe.path.copytree(pkg_dir, dst_dir)
+ shutil.rmtree(pkg_dir)
+
+ else:
+ #logger.debug(1, 'Copying %s' % pkg)
+ _copy_package(d, pkg)
+
+def _extract_in_tmpdir(d, pkg):
+ """"
+ Returns path to a temp directory where the package was
+ extracted without dependencies.
+ """
+
+ from oeqa.utils.package_manager import get_package_manager
+
+ pkg_path = os.path.join(d.getVar('TEST_INSTALL_TMP_DIR'), pkg)
+ pm = get_package_manager(d, pkg_path)
+ extract_dir = pm.extract(pkg)
+ shutil.rmtree(pkg_path)
+
+ return extract_dir
+
+def _copy_package(d, pkg):
+ """
+ Copy the RPM, DEB or IPK package to dst_dir
+ """
+
+ from oeqa.utils.package_manager import get_package_manager
+
+ pkg_path = os.path.join(d.getVar('TEST_INSTALL_TMP_DIR'), pkg)
+ dst_dir = d.getVar('TEST_PACKAGED_DIR')
+ pm = get_package_manager(d, pkg_path)
+ pkg_info = pm.package_info(pkg)
+ file_path = pkg_info[pkg]['filepath']
+ shutil.copy2(file_path, dst_dir)
+ shutil.rmtree(pkg_path)
+
+def install_uninstall_packages(self, test_id, pkg_dir, install):
+ """
+ Check if the test requires a package and Install/Unistall it in the DUT
+ """
+
+ test = test_id.split('.')[4]
+ module = self.getModulefromID(test_id)
+ json = self._getJsonFile(module)
+ if json:
+ needed_packages = self._getNeededPackages(json, test)
+ if needed_packages:
+ self._install_uninstall_packages(needed_packages, pkg_dir, install)
+
+def _install_uninstall_packages(self, needed_packages, pkg_dir, install=True):
+ """
+ Install/Unistall packages in the DUT without using a package manager
+ """
+
+ if isinstance(needed_packages, dict):
+ packages = [needed_packages]
+ elif isinstance(needed_packages, list):
+ packages = needed_packages
+
+ for package in packages:
+ pkg = package['pkg']
+ rm = package.get('rm', False)
+ extract = package.get('extract', True)
+ src_dir = os.path.join(pkg_dir, pkg)
+
+ # Install package
+ if install and extract:
+ self.target.connection.copy_dir_to(src_dir, '/')
+
+ # Unistall package
+ elif not install and rm:
+ self.target.connection.delete_dir_structure(src_dir, '/')
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 52/55] testimage.bbclass: Add package install feature
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (50 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 51/55] testimage.bbclass: Add support for package extraction Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 53/55] oeqa/runtime/context.py: Add defaults for runtime context Aníbal Limón
` (4 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This allows to use the package install feature with
the new OEQA framework.
[YOCTO #10234]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/classes/testimage.bbclass | 27 ++++++++------------
meta/lib/oeqa/core/utils/test.py | 10 +++++---
meta/lib/oeqa/runtime/case.py | 9 +++++++
meta/lib/oeqa/runtime/context.py | 12 ++++++++-
meta/lib/oeqa/utils/package_manager.py | 46 ++++++++++++++++++++++++----------
5 files changed, 69 insertions(+), 35 deletions(-)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 1dfbc49..96e41c6 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -159,7 +159,6 @@ def testimage_main(d):
pn = d.getVar("PN")
bb.utils.mkdirhier(d.getVar("TEST_LOG_DIR"))
- #test_create_extract_dirs(d)
image_name = ("%s/%s" % (d.getVar('DEPLOY_DIR_IMAGE'),
d.getVar('IMAGE_LINK_NAME')))
@@ -170,6 +169,8 @@ def testimage_main(d):
image_manifest = "%s.manifest" % image_name
image_packages = OERuntimeTestContextExecutor.readPackagesManifest(image_manifest)
+ extract_dir = d.getVar("TEST_EXTRACTED_DIR")
+
# Get machine
machine = d.getVar("MACHINE")
@@ -236,7 +237,8 @@ def testimage_main(d):
d.getVar("TEST_SERVER_IP"), **target_kwargs)
# test context
- tc = OERuntimeTestContext(td, logger, target, host_dumper, image_packages)
+ tc = OERuntimeTestContext(td, logger, target, host_dumper,
+ image_packages, extract_dir)
# Load tests before starting the target
test_paths = get_runtime_paths(d)
@@ -343,22 +345,13 @@ def package_extraction(d, test_suites):
from oeqa.utils.package_manager import find_packages_to_extract
from oeqa.utils.package_manager import extract_packages
- test_create_extract_dirs(d)
+ bb.utils.remove(d.getVar("TEST_NEEDED_PACKAGES_DIR"), recurse=True)
packages = find_packages_to_extract(test_suites)
- extract_packages(d, packages)
-
-def test_create_extract_dirs(d):
- install_path = d.getVar("TEST_INSTALL_TMP_DIR")
- package_path = d.getVar("TEST_PACKAGED_DIR")
- extracted_path = d.getVar("TEST_EXTRACTED_DIR")
- bb.utils.mkdirhier(d.getVar("TEST_LOG_DIR"))
- bb.utils.remove(install_path, recurse=True)
- bb.utils.remove(package_path, recurse=True)
- bb.utils.remove(extracted_path, recurse=True)
- bb.utils.mkdirhier(install_path)
- bb.utils.mkdirhier(package_path)
- bb.utils.mkdirhier(extracted_path)
-
+ if packages:
+ bb.utils.mkdirhier(d.getVar("TEST_INSTALL_TMP_DIR"))
+ bb.utils.mkdirhier(d.getVar("TEST_PACKAGED_DIR"))
+ bb.utils.mkdirhier(d.getVar("TEST_EXTRACTED_DIR"))
+ extract_packages(d, packages)
testimage_main[vardepsexclude] += "BB_ORIGENV DATETIME"
diff --git a/meta/lib/oeqa/core/utils/test.py b/meta/lib/oeqa/core/utils/test.py
index 88d1765..88d5d13 100644
--- a/meta/lib/oeqa/core/utils/test.py
+++ b/meta/lib/oeqa/core/utils/test.py
@@ -10,11 +10,13 @@ def getSuiteCases(suite):
Returns individual test from a test suite.
"""
tests = []
- for item in suite:
- if isinstance(item, unittest.suite.TestSuite):
+
+ if isinstance(suite, unittest.TestCase):
+ tests.append(suite)
+ elif isinstance(suite, unittest.suite.TestSuite):
+ for item in suite:
tests.extend(getSuiteCases(item))
- elif isinstance(item, unittest.TestCase):
- tests.append(item)
+
return tests
def getSuiteModules(suite):
diff --git a/meta/lib/oeqa/runtime/case.py b/meta/lib/oeqa/runtime/case.py
index 43f1b2f..c1485c9 100644
--- a/meta/lib/oeqa/runtime/case.py
+++ b/meta/lib/oeqa/runtime/case.py
@@ -2,7 +2,16 @@
# Released under the MIT license (see COPYING.MIT)
from oeqa.core.case import OETestCase
+from oeqa.utils.package_manager import install_package, uninstall_package
class OERuntimeTestCase(OETestCase):
# target instance set by OERuntimeTestLoader.
target = None
+
+ def _oeSetUp(self):
+ super(OERuntimeTestCase, self)._oeSetUp()
+ install_package(self)
+
+ def _oeTearDown(self):
+ super(OERuntimeTestCase, self)._oeTearDown()
+ uninstall_package(self)
diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
index bc8abd0..10b8b54 100644
--- a/meta/lib/oeqa/runtime/context.py
+++ b/meta/lib/oeqa/runtime/context.py
@@ -15,12 +15,14 @@ class OERuntimeTestContext(OETestContext):
runtime_files_dir = os.path.join(
os.path.dirname(os.path.abspath(__file__)), "files")
- def __init__(self, td, logger, target, host_dumper, image_packages):
+ def __init__(self, td, logger, target,
+ host_dumper, image_packages, extract_dir):
super(OERuntimeTestContext, self).__init__(td, logger)
self.target = target
self.image_packages = image_packages
self.host_dumper = host_dumper
+ self.extract_dir = extract_dir
self._set_target_cmds()
def _set_target_cmds(self):
@@ -45,6 +47,7 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
default_server_ip = '192.168.7.1'
default_target_ip = '192.168.7.2'
default_host_dumper_dir = '/tmp/oe-saved-tests'
+ default_extract_dir = 'extract_dir'
def register_commands(self, logger, subparsers):
super(OERuntimeTestContextExecutor, self).register_commands(logger, subparsers)
@@ -72,6 +75,9 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
runtime_group.add_argument('--packages-manifest', action='store',
help="Package manifest of the image under test")
+ runtime_group.add_argument('--extract-dir', action='store',
+ help='Directory where extracted packages reside')
+
runtime_group.add_argument('--qemu-boot', action='store',
help="Qemu boot configuration, only needed when target_type is QEMU.")
@@ -126,4 +132,8 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
OERuntimeTestContextExecutor.readPackagesManifest(
args.packages_manifest)
+ self.tc_kwargs['init']['extract_dir'] = \
+ OERuntimeTestContextExecutor.readPackagesManifest(
+ args.extract_dir)
+
_executor_class = OERuntimeTestContextExecutor
diff --git a/meta/lib/oeqa/utils/package_manager.py b/meta/lib/oeqa/utils/package_manager.py
index ab0d3e5..9264648 100644
--- a/meta/lib/oeqa/utils/package_manager.py
+++ b/meta/lib/oeqa/utils/package_manager.py
@@ -1,7 +1,8 @@
-import bb
+import os
import json
import shutil
+from oeqa.core.utils.test import getCaseFile, getCaseMethod
def get_package_manager(d, root_path):
"""
@@ -86,6 +87,7 @@ def extract_packages(d, needed_packages):
Extract packages that will be needed during runtime.
"""
+ import bb
import oe.path
extracted_path = d.getVar('TEST_EXTRACTED_DIR')
@@ -152,20 +154,38 @@ def _copy_package(d, pkg):
shutil.copy2(file_path, dst_dir)
shutil.rmtree(pkg_path)
-def install_uninstall_packages(self, test_id, pkg_dir, install):
+def install_package(test_case):
"""
- Check if the test requires a package and Install/Unistall it in the DUT
+ Installs package in DUT if required.
"""
+ needed_packages = test_needs_package(test_case)
+ if needed_packages:
+ _install_uninstall_packages(needed_packages, test_case, True)
- test = test_id.split('.')[4]
- module = self.getModulefromID(test_id)
- json = self._getJsonFile(module)
- if json:
- needed_packages = self._getNeededPackages(json, test)
+def uninstall_package(test_case):
+ """
+ Uninstalls package in DUT if required.
+ """
+ needed_packages = test_needs_package(test_case)
+ if needed_packages:
+ _install_uninstall_packages(needed_packages, test_case, False)
+
+def test_needs_package(test_case):
+ """
+ Checks if a test case requires to install/uninstall packages.
+ """
+ test_file = getCaseFile(test_case)
+ json_file = _get_json_file(test_file)
+
+ if json_file:
+ test_method = getCaseMethod(test_case)
+ needed_packages = _get_needed_packages(json_file, test_method)
if needed_packages:
- self._install_uninstall_packages(needed_packages, pkg_dir, install)
+ return needed_packages
+
+ return None
-def _install_uninstall_packages(self, needed_packages, pkg_dir, install=True):
+def _install_uninstall_packages(needed_packages, test_case, install=True):
"""
Install/Unistall packages in the DUT without using a package manager
"""
@@ -179,12 +199,12 @@ def _install_uninstall_packages(self, needed_packages, pkg_dir, install=True):
pkg = package['pkg']
rm = package.get('rm', False)
extract = package.get('extract', True)
- src_dir = os.path.join(pkg_dir, pkg)
+ src_dir = os.path.join(test_case.tc.extract_dir, pkg)
# Install package
if install and extract:
- self.target.connection.copy_dir_to(src_dir, '/')
+ test_case.tc.target.copyDirTo(src_dir, '/')
# Unistall package
elif not install and rm:
- self.target.connection.delete_dir_structure(src_dir, '/')
+ test_case.tc.target.deleteDirStructure(src_dir, '/')
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 53/55] oeqa/runtime/context.py: Add defaults for runtime context
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (51 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 52/55] testimage.bbclass: Add package install feature Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 54/55] testexport.bbclass: Migrate testexport to use new framework Aníbal Limón
` (3 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This adds default values to OERuntimeTestContextExecutor class in
order to make easier the execution of exported test that were
generated with testexport class.
[YOCTO #10686]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/runtime/context.py | 22 +++++++++++++---------
1 file changed, 13 insertions(+), 9 deletions(-)
diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
index 10b8b54..e5e0141 100644
--- a/meta/lib/oeqa/runtime/context.py
+++ b/meta/lib/oeqa/runtime/context.py
@@ -42,12 +42,15 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
default_cases = os.path.join(os.path.abspath(os.path.dirname(__file__)),
'cases')
default_data = None
+ default_test_data = 'data/testdata.json'
+ default_tests = ''
default_target_type = 'simpleremote'
+ default_manifest = 'data/manifest'
default_server_ip = '192.168.7.1'
default_target_ip = '192.168.7.2'
default_host_dumper_dir = '/tmp/oe-saved-tests'
- default_extract_dir = 'extract_dir'
+ default_extract_dir = 'packages/extracted'
def register_commands(self, logger, subparsers):
super(OERuntimeTestContextExecutor, self).register_commands(logger, subparsers)
@@ -73,10 +76,14 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
% self.default_host_dumper_dir)
runtime_group.add_argument('--packages-manifest', action='store',
- help="Package manifest of the image under test")
+ default=self.default_manifest,
+ help="Package manifest of the image under testi, default: %s" \
+ % self.default_manifest)
runtime_group.add_argument('--extract-dir', action='store',
- help='Directory where extracted packages reside')
+ default=self.default_extract_dir,
+ help='Directory where extracted packages reside, default: %s' \
+ % self.default_extract_dir)
runtime_group.add_argument('--qemu-boot', action='store',
help="Qemu boot configuration, only needed when target_type is QEMU.")
@@ -97,7 +104,7 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
@staticmethod
def readPackagesManifest(manifest):
- if not os.path.exists(manifest):
+ if not manifest or not os.path.exists(manifest):
raise OSError("Manifest file not exists: %s" % manifest)
image_packages = set()
@@ -124,16 +131,13 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
self.tc_kwargs['init']['target'] = \
OERuntimeTestContextExecutor.getTarget(args.target_type,
- args.target_ip, args.server_ip, **target_kwargs)
+ None, args.target_ip, args.server_ip, **target_kwargs)
self.tc_kwargs['init']['host_dumper'] = \
OERuntimeTestContextExecutor.getHostDumper(None,
args.host_dumper_dir)
self.tc_kwargs['init']['image_packages'] = \
OERuntimeTestContextExecutor.readPackagesManifest(
args.packages_manifest)
-
- self.tc_kwargs['init']['extract_dir'] = \
- OERuntimeTestContextExecutor.readPackagesManifest(
- args.extract_dir)
+ self.tc_kwargs['init']['extract_dir'] = args.extract_dir
_executor_class = OERuntimeTestContextExecutor
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 54/55] testexport.bbclass: Migrate testexport to use new framework
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (52 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 53/55] oeqa/runtime/context.py: Add defaults for runtime context Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:10 ` [PATCH 55/55] selftest/runtime-test.py: Adapt test to use new runtime framework Aníbal Limón
` (2 subsequent siblings)
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This migrates current testexport implmentation to use the
new OEQA framework.
[YOCTO #10686]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/classes/testexport.bbclass | 215 +++++++++++++++++-----------------------
scripts/oe-test | 10 +-
2 files changed, 99 insertions(+), 126 deletions(-)
diff --git a/meta/classes/testexport.bbclass b/meta/classes/testexport.bbclass
index 5398b3c..56edda9 100644
--- a/meta/classes/testexport.bbclass
+++ b/meta/classes/testexport.bbclass
@@ -33,113 +33,111 @@ TEST_EXPORT_DEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'cpio-nativ
TEST_EXPORT_DEPENDS += "${@bb.utils.contains('TEST_EXPORT_SDK_ENABLED', '1', 'testexport-tarball:do_populate_sdk', '', d)}"
TEST_EXPORT_LOCK = "${TMPDIR}/testimage.lock"
-python do_testexport() {
- testexport_main(d)
-}
-
addtask testexport
do_testexport[nostamp] = "1"
do_testexport[depends] += "${TEST_EXPORT_DEPENDS} ${TESTIMAGEDEPENDS}"
do_testexport[lockfiles] += "${TEST_EXPORT_LOCK}"
-def exportTests(d,tc):
+python do_testexport() {
+ testexport_main(d)
+}
+
+def testexport_main(d):
import json
+ import logging
+
+ from oeqa.runtime.context import OERuntimeTestContext
+ from oeqa.runtime.context import OERuntimeTestContextExecutor
+
+ image_name = ("%s/%s" % (d.getVar('DEPLOY_DIR_IMAGE'),
+ d.getVar('IMAGE_LINK_NAME')))
+
+ tdname = "%s.testdata.json" % image_name
+ td = json.load(open(tdname, "r"))
+
+ logger = logging.getLogger("BitBake")
+
+ target = OERuntimeTestContextExecutor.getTarget(
+ d.getVar("TEST_TARGET"), None, d.getVar("TEST_TARGET_IP"),
+ d.getVar("TEST_SERVER_IP"))
+
+ host_dumper = OERuntimeTestContextExecutor.getHostDumper(
+ d.getVar("testimage_dump_host"), d.getVar("TESTIMAGE_DUMP_DIR"))
+
+ image_manifest = "%s.manifest" % image_name
+ image_packages = OERuntimeTestContextExecutor.readPackagesManifest(image_manifest)
+
+ extract_dir = d.getVar("TEST_EXTRACTED_DIR")
+
+ tc = OERuntimeTestContext(td, logger, target, host_dumper,
+ image_packages, extract_dir)
+
+ copy_needed_files(d, tc)
+
+def copy_needed_files(d, tc):
import shutil
- import pkgutil
- import re
import oe.path
- exportpath = d.getVar("TEST_EXPORT_DIR")
-
- savedata = {}
- savedata["d"] = {}
- savedata["target"] = {}
- savedata["target"]["ip"] = tc.target.ip or d.getVar("TEST_TARGET_IP")
- savedata["target"]["server_ip"] = tc.target.server_ip or d.getVar("TEST_SERVER_IP")
-
- keys = [ key for key in d.keys() if not key.startswith("_") and not key.startswith("BB") \
- and not key.startswith("B_pn") and not key.startswith("do_") and not d.getVarFlag(key, "func")]
- for key in keys:
- try:
- savedata["d"][key] = d.getVar(key)
- except bb.data_smart.ExpansionError:
- # we don't care about those anyway
- pass
-
- json_file = os.path.join(exportpath, "testdata.json")
- with open(json_file, "w") as f:
- json.dump(savedata, f, skipkeys=True, indent=4, sort_keys=True)
-
- # Replace absolute path with relative in the file
- exclude_path = os.path.join(d.getVar("COREBASE"),'meta','lib','oeqa')
- f1 = open(json_file,'r').read()
- f2 = open(json_file,'w')
- m = f1.replace(exclude_path,'oeqa')
- f2.write(m)
- f2.close()
-
- # now start copying files
- # we'll basically copy everything under meta/lib/oeqa, with these exceptions
- # - oeqa/targetcontrol.py - not needed
- # - oeqa/selftest - something else
- # That means:
- # - all tests from oeqa/runtime defined in TEST_SUITES (including from other layers)
- # - the contents of oeqa/utils and oeqa/runtime/files
- # - oeqa/oetest.py and oeqa/runexport.py (this will get copied to exportpath not exportpath/oeqa)
- # - __init__.py files
- bb.utils.mkdirhier(os.path.join(exportpath, "oeqa/files"))
- bb.utils.mkdirhier(os.path.join(exportpath, "oeqa/runtime/files"))
- bb.utils.mkdirhier(os.path.join(exportpath, "oeqa/utils"))
- # copy test modules, this should cover tests in other layers too
- bbpath = d.getVar("BBPATH").split(':')
- for t in tc.testslist:
- isfolder = False
- if re.search("\w+\.\w+\.test_\S+", t):
- t = '.'.join(t.split('.')[:3])
- mod = pkgutil.get_loader(t)
- # More depth than usual?
- if (t.count('.') > 2):
- for p in bbpath:
- foldername = os.path.join(p, 'lib', os.sep.join(t.split('.')).rsplit(os.sep, 1)[0])
- if os.path.isdir(foldername):
- isfolder = True
- target_folder = os.path.join(exportpath, "oeqa", "runtime", os.path.basename(foldername))
- if not os.path.exists(target_folder):
- oe.path.copytree(foldername, target_folder)
- if not isfolder:
- shutil.copy2(mod.path, os.path.join(exportpath, "oeqa/runtime"))
- json_file = "%s.json" % mod.path.rsplit(".", 1)[0]
- if os.path.isfile(json_file):
- shutil.copy2(json_file, os.path.join(exportpath, "oeqa/runtime"))
- # Get meta layer
- for layer in d.getVar("BBLAYERS").split():
- if os.path.basename(layer) == "meta":
- meta_layer = layer
- break
- # copy oeqa/oetest.py and oeqa/runexported.py
- oeqadir = os.path.join(meta_layer, "lib/oeqa")
- shutil.copy2(os.path.join(oeqadir, "oetest.py"), os.path.join(exportpath, "oeqa"))
- shutil.copy2(os.path.join(oeqadir, "runexported.py"), exportpath)
- # copy oeqa/utils/*.py
- for root, dirs, files in os.walk(os.path.join(oeqadir, "utils")):
- for f in files:
- if f.endswith(".py"):
- shutil.copy2(os.path.join(root, f), os.path.join(exportpath, "oeqa/utils"))
- # copy oeqa/files/*
- for root, dirs, files in os.walk(os.path.join(oeqadir, "files")):
- for f in files:
- shutil.copy2(os.path.join(root, f), os.path.join(exportpath, "oeqa/files"))
- # copy oeqa/runtime/files/*
- for root, dirs, files in os.walk(os.path.join(oeqadir, "runtime/files")):
- for f in files:
- shutil.copy2(os.path.join(root, f), os.path.join(exportpath, "oeqa/runtime/files"))
+ from oeqa.utils.package_manager import _get_json_file
+ from oeqa.core.utils.test import getSuiteCasesFiles
+
+ export_path = d.getVar('TEST_EXPORT_DIR')
+ corebase_path = d.getVar('COREBASE')
+
+ # Clean everything before starting
+ oe.path.remove(export_path)
+ bb.utils.mkdirhier(os.path.join(export_path, 'lib', 'oeqa'))
+
+ # The source of files to copy are relative to 'COREBASE' directory
+ # The destination is relative to 'TEST_EXPORT_DIR'
+ # Because we are squashing the libraries, we need to remove
+ # the layer/script directory
+ files_to_copy = [ os.path.join('meta', 'lib', 'oeqa', 'core'),
+ os.path.join('meta', 'lib', 'oeqa', 'runtime'),
+ os.path.join('meta', 'lib', 'oeqa', 'files'),
+ os.path.join('meta', 'lib', 'oeqa', 'utils'),
+ os.path.join('scripts', 'oe-test'),
+ os.path.join('scripts', 'lib', 'argparse_oe.py'),
+ os.path.join('scripts', 'lib', 'scriptutils.py'), ]
+
+ for f in files_to_copy:
+ src = os.path.join(corebase_path, f)
+ dst = os.path.join(export_path, f.split('/', 1)[-1])
+ if os.path.isdir(src):
+ oe.path.copytree(src, dst)
+ else:
+ shutil.copy2(src, dst)
+
+ # Remove cases and just copy the ones specified
+ cases_path = os.path.join(export_path, 'lib', 'oeqa', 'runtime', 'cases')
+ oe.path.remove(cases_path)
+ bb.utils.mkdirhier(cases_path)
+ test_paths = get_runtime_paths(d)
+ test_modules = d.getVar('TEST_SUITES')
+ tc.loadTests(test_paths, modules=test_modules)
+ for f in getSuiteCasesFiles(tc.suites):
+ shutil.copy2(f, cases_path)
+ json_file = _get_json_file(f)
+ if json_file:
+ shutil.copy2(json_file, cases_path)
+
+ # Copy test data
+ image_name = ("%s/%s" % (d.getVar('DEPLOY_DIR_IMAGE'),
+ d.getVar('IMAGE_LINK_NAME')))
+ image_manifest = "%s.manifest" % image_name
+ tdname = "%s.testdata.json" % image_name
+ test_data_path = os.path.join(export_path, 'data')
+ bb.utils.mkdirhier(test_data_path)
+ shutil.copy2(image_manifest, os.path.join(test_data_path, 'manifest'))
+ shutil.copy2(tdname, os.path.join(test_data_path, 'testdata.json'))
# Create tar file for common parts of testexport
create_tarball(d, "testexport.tar.gz", d.getVar("TEST_EXPORT_DIR"))
# Copy packages needed for runtime testing
+ package_extraction(d, tc.suites)
test_pkg_dir = d.getVar("TEST_NEEDED_PACKAGES_DIR")
- if os.listdir(test_pkg_dir):
+ if os.path.isdir(test_pkg_dir) and os.listdir(test_pkg_dir):
export_pkg_dir = os.path.join(d.getVar("TEST_EXPORT_DIR"), "packages")
oe.path.copytree(test_pkg_dir, export_pkg_dir)
# Create tar file for packages needed by the DUT
@@ -158,35 +156,7 @@ def exportTests(d,tc):
# Create tar file for the sdk
create_tarball(d, "testexport_sdk_%s.tar.gz" % d.getVar("SDK_ARCH"), export_sdk_dir)
- bb.plain("Exported tests to: %s" % exportpath)
-
-def testexport_main(d):
- from oeqa.oetest import ExportTestContext
- from oeqa.targetcontrol import get_target_controller
-
- test_create_extract_dirs(d)
- export_dir = d.getVar("TEST_EXPORT_DIR")
- bb.utils.mkdirhier(d.getVar("TEST_LOG_DIR"))
- bb.utils.remove(export_dir, recurse=True)
- bb.utils.mkdirhier(export_dir)
-
- # the robot dance
- target = get_target_controller(d)
-
- # test context
- tc = ExportTestContext(d, target)
-
- # this is a dummy load of tests
- # we are doing that to find compile errors in the tests themselves
- # before booting the image
- try:
- tc.loadTests()
- except Exception as e:
- import traceback
- bb.fatal("Loading tests failed:\n%s" % traceback.format_exc())
-
- tc.extract_packages()
- exportTests(d,tc)
+ bb.plain("Exported tests to: %s" % export_path)
def create_tarball(d, tar_name, src_dir):
@@ -204,7 +174,4 @@ def create_tarball(d, tar_name, src_dir):
tar.close()
os.chdir(current_dir)
-
-testexport_main[vardepsexclude] =+ "BB_ORIGENV"
-
inherit testimage
diff --git a/scripts/oe-test b/scripts/oe-test
index 5731dff..a1d282d 100755
--- a/scripts/oe-test
+++ b/scripts/oe-test
@@ -16,8 +16,14 @@ lib_path = scripts_path + '/lib'
sys.path = sys.path + [lib_path]
import argparse_oe
import scriptutils
-import scriptpath
-scriptpath.add_oe_lib_path()
+
+# oe-test is used for testexport and it doesn't have oe lib
+# so we just skip adding these libraries (not used in testexport)
+try:
+ import scriptpath
+ scriptpath.add_oe_lib_path()
+except ImportError:
+ pass
from oeqa.core.context import OETestContextExecutor
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* [PATCH 55/55] selftest/runtime-test.py: Adapt test to use new runtime framework
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (53 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 54/55] testexport.bbclass: Migrate testexport to use new framework Aníbal Limón
@ 2017-01-20 17:10 ` Aníbal Limón
2017-01-20 17:23 ` ✗ patchtest: failure for OEQA Framework Refactor & Improvements (rev4) Patchwork
2017-01-21 12:28 ` [PATCH 00/55] OEQA Framework Refactor & Improvements Richard Purdie
56 siblings, 0 replies; 59+ messages in thread
From: Aníbal Limón @ 2017-01-20 17:10 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This adapt the current runtime selftest to use the new runtime
framework.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
.../lib/oeqa/runtime/{ => cases}/selftest.json | 0
.../lib/oeqa/runtime/{ => cases}/selftest.py | 34 +++---------------
meta/lib/oeqa/selftest/runtime-test.py | 42 +++++++++++++---------
3 files changed, 31 insertions(+), 45 deletions(-)
rename meta-selftest/lib/oeqa/runtime/{ => cases}/selftest.json (100%)
rename meta-selftest/lib/oeqa/runtime/{ => cases}/selftest.py (53%)
diff --git a/meta-selftest/lib/oeqa/runtime/selftest.json b/meta-selftest/lib/oeqa/runtime/cases/selftest.json
similarity index 100%
rename from meta-selftest/lib/oeqa/runtime/selftest.json
rename to meta-selftest/lib/oeqa/runtime/cases/selftest.json
diff --git a/meta-selftest/lib/oeqa/runtime/selftest.py b/meta-selftest/lib/oeqa/runtime/cases/selftest.py
similarity index 53%
rename from meta-selftest/lib/oeqa/runtime/selftest.py
rename to meta-selftest/lib/oeqa/runtime/cases/selftest.py
index a7e58ab3..329470f 100644
--- a/meta-selftest/lib/oeqa/runtime/selftest.py
+++ b/meta-selftest/lib/oeqa/runtime/cases/selftest.py
@@ -1,13 +1,9 @@
-import os
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
-from oeqa.oetest import oeRuntimeTest, skipModule
-from oeqa.utils.commands import runCmd
-from oeqa.utils.decorators import *
+class Selftest(OERuntimeTestCase):
-class Selftest(oeRuntimeTest):
-
- @skipUnlessPassed("test_ssh")
- @tag("selftest_package_install")
+ @OETestDepends(['ssh.SSHTest.test_ssh'])
def test_install_package(self):
"""
Summary: Check basic package installation functionality.
@@ -21,8 +17,7 @@ class Selftest(oeRuntimeTest):
(status, output) = self.target.run("socat -V")
self.assertEqual(status, 0, msg="socat is not installed")
- @skipUnlessPassed("test_install_package")
- @tag("selftest_package_install")
+ @OETestDepends(['selftest.Selftest.test_install_package'])
def test_verify_unistall(self):
"""
Summary: Check basic package installation functionality.
@@ -34,22 +29,3 @@ class Selftest(oeRuntimeTest):
(status, output) = self.target.run("socat -V")
self.assertNotEqual(status, 0, msg="socat is still installed")
-
- @tag("selftest_sdk")
- def test_sdk(self):
-
- result = runCmd("env -0")
- sdk_path = search_sdk_path(result.output)
- self.assertTrue(sdk_path, msg="Can't find SDK path")
-
- tar_cmd = os.path.join(sdk_path, "tar")
- result = runCmd("%s --help" % tar_cmd)
-
-def search_sdk_path(env):
- for line in env.split("\0"):
- (key, _, value) = line.partition("=")
- if key == "PATH":
- for path in value.split(":"):
- if "pokysdk" in path:
- return path
- return ""
diff --git a/meta/lib/oeqa/selftest/runtime-test.py b/meta/lib/oeqa/selftest/runtime-test.py
index a105f3f..9e2ecc6 100644
--- a/meta/lib/oeqa/selftest/runtime-test.py
+++ b/meta/lib/oeqa/selftest/runtime-test.py
@@ -6,6 +6,10 @@ import re
class TestExport(oeSelfTest):
+ @classmethod
+ def tearDownClass(cls):
+ runCmd("rm -rf /tmp/sdk")
+
def test_testexport_basic(self):
"""
Summary: Check basic testexport functionality with only ping test enabled.
@@ -34,15 +38,15 @@ class TestExport(oeSelfTest):
with runqemu('core-image-minimal') as qemu:
# Attempt to run runexported.py to perform ping test
- runexported_path = os.path.join(testexport_dir, "runexported.py")
- testdata_path = os.path.join(testexport_dir, "testdata.json")
- cmd = "%s -t %s -s %s %s" % (runexported_path, qemu.ip, qemu.server_ip, testdata_path)
+ test_path = os.path.join(testexport_dir, "oe-test")
+ data_file = os.path.join(testexport_dir, 'data', 'testdata.json')
+ manifest = os.path.join(testexport_dir, 'data', 'manifest')
+ cmd = ("%s runtime --test-data-file %s --packages-manifest %s "
+ "--target-ip %s --server-ip %s --quiet"
+ % (test_path, data_file, manifest, qemu.ip, qemu.server_ip))
result = runCmd(cmd)
- self.assertEqual(0, result.status, 'runexported.py returned a non 0 status')
-
# Verify ping test was succesful
- failure = True if 'FAIL' in result.output else False
- self.assertNotEqual(True, failure, 'ping test failed')
+ self.assertEqual(0, result.status, 'oe-test runtime returned a non 0 status')
def test_testexport_sdk(self):
"""
@@ -61,7 +65,6 @@ class TestExport(oeSelfTest):
features += 'TEST_SERVER_IP = "192.168.7.1"\n'
features += 'TEST_TARGET_IP = "192.168.7.1"\n'
features += 'TEST_SUITES = "ping"\n'
- features += 'TEST_SUITES_TAGS = "selftest_sdk"\n'
features += 'TEST_EXPORT_SDK_ENABLED = "1"\n'
features += 'TEST_EXPORT_SDK_PACKAGES = "nativesdk-tar"\n'
self.write_config(features)
@@ -75,14 +78,22 @@ class TestExport(oeSelfTest):
sdk_dir = get_bb_var('TEST_EXPORT_SDK_DIR', 'core-image-minimal')
tarball_name = "%s.sh" % get_bb_var('TEST_EXPORT_SDK_NAME', 'core-image-minimal')
tarball_path = os.path.join(testexport_dir, sdk_dir, tarball_name)
- self.assertEqual(os.path.isfile(tarball_path), True, "Couldn't find SDK tarball: %s" % tarball_path)
+ msg = "Couldn't find SDK tarball: %s" % tarball_path
+ self.assertEqual(os.path.isfile(tarball_path), True, msg)
+
+ # Extract SDK and run tar from SDK
+ result = runCmd("%s -y -d /tmp/sdk" % tarball_path)
+ self.assertEqual(0, result.status, "Couldn't extract SDK")
+
+ env_script = result.output.split()[-1]
+ result = runCmd(". %s; which tar" % env_script, shell=True)
+ self.assertEqual(0, result.status, "Couldn't setup SDK environment")
+ is_sdk_tar = True if "/tmp/sdk" in result.output else False
+ self.assertTrue(is_sdk_tar, "Couldn't setup SDK environment")
- # Run runexported.py
- runexported_path = os.path.join(testexport_dir, "runexported.py")
- testdata_path = os.path.join(testexport_dir, "testdata.json")
- cmd = "%s %s" % (runexported_path, testdata_path)
- result = runCmd(cmd)
- self.assertEqual(0, result.status, 'runexported.py returned a non 0 status')
+ tar_sdk = result.output
+ result = runCmd("%s --version" % tar_sdk)
+ self.assertEqual(0, result.status, "Couldn't run tar from SDK")
class TestImage(oeSelfTest):
@@ -100,7 +111,6 @@ class TestImage(oeSelfTest):
features = 'INHERIT += "testimage"\n'
features += 'TEST_SUITES = "ping ssh selftest"\n'
- features += 'TEST_SUITES_TAGS = "selftest_package_install"\n'
self.write_config(features)
# Build core-image-sato and testimage
--
2.1.4
^ permalink raw reply related [flat|nested] 59+ messages in thread* ✗ patchtest: failure for OEQA Framework Refactor & Improvements (rev4)
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (54 preceding siblings ...)
2017-01-20 17:10 ` [PATCH 55/55] selftest/runtime-test.py: Adapt test to use new runtime framework Aníbal Limón
@ 2017-01-20 17:23 ` Patchwork
2017-01-21 12:28 ` [PATCH 00/55] OEQA Framework Refactor & Improvements Richard Purdie
56 siblings, 0 replies; 59+ messages in thread
From: Patchwork @ 2017-01-20 17:23 UTC (permalink / raw)
To: Aníbal Limón; +Cc: openembedded-core
== Series Details ==
Series: OEQA Framework Refactor & Improvements (rev4)
Revision: 4
URL : https://patchwork.openembedded.org/series/4272/
State : failure
== Summary ==
Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:
* Patch [33/55] oeqa/core/target Add OESSHTarget to sent commands to targets using SSH
Issue Shortlog does not follow expected format [test_shortlog_format]
Suggested fix Commit shortlog (first line of commit message) should follow the format "<target>: <summary>"
* Issue Series does not apply on top of target branch [test_series_merge_on_head]
Suggested fix Rebase your series on top of targeted branch
Targeted branch master (currently at ce5c7075d5)
If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).
---
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [PATCH 00/55] OEQA Framework Refactor & Improvements
2017-01-20 17:09 [PATCH 00/55] OEQA Framework Refactor & Improvements Aníbal Limón
` (55 preceding siblings ...)
2017-01-20 17:23 ` ✗ patchtest: failure for OEQA Framework Refactor & Improvements (rev4) Patchwork
@ 2017-01-21 12:28 ` Richard Purdie
2017-01-23 14:20 ` Richard Purdie
56 siblings, 1 reply; 59+ messages in thread
From: Richard Purdie @ 2017-01-21 12:28 UTC (permalink / raw)
To: Aníbal Limón, openembedded-core
On Fri, 2017-01-20 at 11:09 -0600, Aníbal Limón wrote:
> This patchset is related to OEQA Framework for details read the RFC
> send to the
> Openembedded architecture ML.
>
> http://lists.openembedded.org/pipermail/openembedded-architecture/201
> 6-December/000351.html
>
> Also adds the migration of the runtime testing leaving only selftest
> remains
> to migrate.
>
> The testing was made using GDC Autobuilder building in different
> archs and running selftest.
Thanks, I'm testing this on the autobuilder, it seems mostly good but
so far we've seen this issue:
https://autobuilder.yocto.io/builders/nightly-qa-skeleton/builds/145/st
eps/Running%20Sanity%20Tests/logs/stdio
Please send any further patches on top of this series.
Cheers,
Richard
^ permalink raw reply [flat|nested] 59+ messages in thread* Re: [PATCH 00/55] OEQA Framework Refactor & Improvements
2017-01-21 12:28 ` [PATCH 00/55] OEQA Framework Refactor & Improvements Richard Purdie
@ 2017-01-23 14:20 ` Richard Purdie
0 siblings, 0 replies; 59+ messages in thread
From: Richard Purdie @ 2017-01-23 14:20 UTC (permalink / raw)
To: Aníbal Limón, openembedded-core
On Sat, 2017-01-21 at 12:28 +0000, Richard Purdie wrote:
> On Fri, 2017-01-20 at 11:09 -0600, Aníbal Limón wrote:
> >
> > This patchset is related to OEQA Framework for details read the RFC
> > send to the
> > Openembedded architecture ML.
> >
> > http://lists.openembedded.org/pipermail/openembedded-architecture/2
> > 01
> > 6-December/000351.html
> >
> > Also adds the migration of the runtime testing leaving only
> > selftest
> > remains
> > to migrate.
> >
> > The testing was made using GDC Autobuilder building in different
> > archs and running selftest.
> Thanks, I'm testing this on the autobuilder, it seems mostly good but
> so far we've seen this issue:
>
> https://autobuilder.yocto.io/builders/nightly-qa-skeleton/builds/145/
> st
> eps/Running%20Sanity%20Tests/logs/stdio
>
> Please send any further patches on top of this series.
To update, the fix was trivial and the only issue uncovered by testing
so I've merged a fix into the appropriate commit as I merged the
series.
Cheers,
Richard
^ permalink raw reply [flat|nested] 59+ messages in thread