* [PATCHv2 02/32] oeqa/core: Add utils module for OEQA framework
From: Aníbal Limón @ 2016-12-07 20:32 UTC (permalink / raw)
To: openembedded-core
In-Reply-To: <cover.1481142664.git.anibal.limon@linux.intel.com>
From: Mariano Lopez <mariano.lopez@linux.intel.com>
misc: Functions for transform object to other types.
path: Functions for path handling.
test: Functions for operations related to test cases and suites.
[YOCTO #10232]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
meta/lib/oeqa/core/utils/__init__.py | 0
meta/lib/oeqa/core/utils/misc.py | 37 +++++++++++++++++++
meta/lib/oeqa/core/utils/path.py | 14 +++++++
meta/lib/oeqa/core/utils/test.py | 71 ++++++++++++++++++++++++++++++++++++
4 files changed, 122 insertions(+)
create mode 100644 meta/lib/oeqa/core/utils/__init__.py
create mode 100644 meta/lib/oeqa/core/utils/misc.py
create mode 100644 meta/lib/oeqa/core/utils/path.py
create mode 100644 meta/lib/oeqa/core/utils/test.py
diff --git a/meta/lib/oeqa/core/utils/__init__.py b/meta/lib/oeqa/core/utils/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/meta/lib/oeqa/core/utils/misc.py b/meta/lib/oeqa/core/utils/misc.py
new file mode 100644
index 0000000..6ae58ad
--- /dev/null
+++ b/meta/lib/oeqa/core/utils/misc.py
@@ -0,0 +1,37 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+def toList(obj, obj_type, obj_name="Object"):
+ if isinstance(obj, obj_type):
+ return [obj]
+ elif isinstance(obj, list):
+ return obj
+ else:
+ raise TypeError("%s must be %s or list" % (obj_name, obj_type))
+
+def toSet(obj, obj_type, obj_name="Object"):
+ if isinstance(obj, obj_type):
+ return {obj}
+ elif isinstance(obj, list):
+ return set(obj)
+ elif isinstance(obj, set):
+ return obj
+ else:
+ raise TypeError("%s must be %s or set" % (obj_name, obj_type))
+
+def strToList(obj, obj_name="Object"):
+ return toList(obj, str, obj_name)
+
+def strToSet(obj, obj_name="Object"):
+ return toSet(obj, str, obj_name)
+
+def intToList(obj, obj_name="Object"):
+ return toList(obj, int, obj_name)
+
+def dataStoteToDict(d, variables):
+ data = {}
+
+ for v in variables:
+ data[v] = d.getVar(v, True)
+
+ return data
diff --git a/meta/lib/oeqa/core/utils/path.py b/meta/lib/oeqa/core/utils/path.py
new file mode 100644
index 0000000..cb06523
--- /dev/null
+++ b/meta/lib/oeqa/core/utils/path.py
@@ -0,0 +1,14 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import sys
+
+def findFile(file_name, directory):
+ """
+ Search for a file in directory and returns its complete path.
+ """
+ for r, d, f in os.walk(directory):
+ if file_name in f:
+ return os.path.join(r, file_name)
+ return None
diff --git a/meta/lib/oeqa/core/utils/test.py b/meta/lib/oeqa/core/utils/test.py
new file mode 100644
index 0000000..820b997
--- /dev/null
+++ b/meta/lib/oeqa/core/utils/test.py
@@ -0,0 +1,71 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import unittest
+
+def getSuiteCases(suite):
+ """
+ Returns individual test from a test suite.
+ """
+ tests = []
+ for item in suite:
+ if isinstance(item, unittest.suite.TestSuite):
+ tests.extend(getSuiteCases(item))
+ elif isinstance(item, unittest.TestCase):
+ tests.append(item)
+ return tests
+
+def getSuiteModules(suite):
+ """
+ Returns modules in a test suite.
+ """
+ modules = set()
+ for test in getSuiteCases(suite):
+ modules.add(getCaseModule(test))
+ return modules
+
+def getSuiteCasesInfo(suite, func):
+ """
+ Returns test case info from suite. Info is fetched from func.
+ """
+ tests = []
+ for test in getSuiteCases(suite):
+ tests.append(func(test))
+ return tests
+
+def getSuiteCasesNames(suite):
+ """
+ Returns test case names from suite.
+ """
+ return getSuiteCasesInfo(suite, getCaseMethod)
+
+def getSuiteCasesIDs(suite):
+ """
+ Returns test case ids from suite.
+ """
+ return getSuiteCasesInfo(suite, getCaseID)
+
+def getCaseModule(test_case):
+ """
+ Returns test case module name.
+ """
+ return test_case.__module__
+
+def getCaseClass(test_case):
+ """
+ Returns test case class name.
+ """
+ return test_case.__class__.__name__
+
+def getCaseID(test_case):
+ """
+ Returns test case complete id.
+ """
+ return test_case.id()
+
+def getCaseMethod(test_case):
+ """
+ Returns test case method name.
+ """
+ return getCaseID(test_case).split('.')[-1]
--
2.1.4
^ permalink raw reply related
* [PATCHv2 04/32] oeqa/core/decorator: Add support for OETestDepends
From: Aníbal Limón @ 2016-12-07 20:32 UTC (permalink / raw)
To: openembedded-core
In-Reply-To: <cover.1481142664.git.anibal.limon@linux.intel.com>
The OETestDepends decorator could be used over test cases to
define some dependency between them.
At loading time sorting the tests to grauntee that a test case
executes before also raise an exception if found a circular
dependency between test cases.
At before test case run reviews if the dependency if meet, in the
case of don't it skips the test case run.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/decorator/depends.py | 94 +++++++++++++++++++++++++++++++++
1 file changed, 94 insertions(+)
create mode 100644 meta/lib/oeqa/core/decorator/depends.py
diff --git a/meta/lib/oeqa/core/decorator/depends.py b/meta/lib/oeqa/core/decorator/depends.py
new file mode 100644
index 0000000..195711c
--- /dev/null
+++ b/meta/lib/oeqa/core/decorator/depends.py
@@ -0,0 +1,94 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from unittest import SkipTest
+
+from oeqa.core.exception import OEQADependency
+
+from . import OETestDiscover, registerDecorator
+
+def _add_depends(registry, case, depends):
+ module_name = case.__module__
+ class_name = case.__class__.__name__
+
+ case_id = case.id()
+
+ for depend in depends:
+ dparts = depend.split('.')
+
+ if len(dparts) == 1:
+ depend_id = ".".join((module_name, class_name, dparts[0]))
+ elif len(dparts) == 2:
+ depend_id = ".".join((module_name, dparts[0], dparts[1]))
+ else:
+ depend_id = depend
+
+ if not case_id in registry:
+ registry[case_id] = []
+ if not depend_id in registry[case_id]:
+ registry[case_id].append(depend_id)
+
+def _validate_test_case_depends(cases, depends):
+ for case in depends:
+ if not case in cases:
+ continue
+ for dep in depends[case]:
+ if not dep in cases:
+ raise OEQADependency("TestCase %s depends on %s and isn't available"\
+ ", cases available %s." % (case, dep, str(cases.keys())))
+
+def _order_test_case_by_depends(cases, depends):
+ def _dep_resolve(graph, node, resolved, seen):
+ seen.append(node)
+ for edge in graph[node]:
+ if edge not in resolved:
+ if edge in seen:
+ raise OEQADependency("Test cases %s and %s have a circular" \
+ " dependency." % (node, edge))
+ _dep_resolve(graph, edge, resolved, seen)
+ resolved.append(node)
+
+ dep_graph = {}
+ dep_graph['__root__'] = cases.keys()
+ for case in cases:
+ if case in depends:
+ dep_graph[case] = depends[case]
+ else:
+ dep_graph[case] = []
+
+ cases_ordered = []
+ _dep_resolve(dep_graph, '__root__', cases_ordered, [])
+ cases_ordered.remove('__root__')
+
+ return [cases[case_id] for case_id in cases_ordered]
+
+def _skipTestDependency(case, depends):
+ results = case.tc._results
+ skipReasons = ['errors', 'failures', 'skipped']
+
+ for reason in skipReasons:
+ for test, _ in results[reason]:
+ if test.id() in depends:
+ raise SkipTest("Test case %s depends on %s and was in %s." \
+ % (case.id(), test.id(), reason))
+
+@registerDecorator
+class OETestDepends(OETestDiscover):
+ attrs = ('depends',)
+
+ def bind(self, registry, case):
+ super(OETestDepends, self).bind(registry, case)
+ if not registry.get('depends'):
+ registry['depends'] = {}
+ _add_depends(registry['depends'], case, self.depends)
+
+ @staticmethod
+ def discover(registry):
+ if registry.get('depends'):
+ _validate_test_case_depends(registry['cases'], registry['depends'])
+ return _order_test_case_by_depends(registry['cases'], registry['depends'])
+ else:
+ return [registry['cases'][case_id] for case_id in registry['cases']]
+
+ def setUpDecorator(self):
+ _skipTestDependency(self.case, self.depends)
--
2.1.4
^ permalink raw reply related
* [PATCHv2 03/32] oeqa/core: Add loader, context and decorator modules
From: Aníbal Limón @ 2016-12-07 20:32 UTC (permalink / raw)
To: openembedded-core
In-Reply-To: <cover.1481142664.git.anibal.limon@linux.intel.com>
loader: Implements OETestLoader handling OETestDecorator
and filtering support when load tests. The OETestLoader is
responsible to set custom methods, attrs of the OEQA
frameowork.
[YOCTO #10231]
[YOCTO #10317]
[YOCTO #10353]
decorator: Add base class OETestDecorator to provide a common
way to define decorators to be used over OETestCase's, every
decorator has a method to be called when loading tests and
before test execution starts. Special decorators could be
implemented for filter tests on loading phase.
context: Provides HIGH level API for loadTests and runTests
of certain test component (i.e. runtime, sdk, selftest).
[YOCTO #10230]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/context.py | 225 +++++++++++++++++++++++++++++
meta/lib/oeqa/core/decorator/__init__.py | 71 ++++++++++
meta/lib/oeqa/core/loader.py | 235 +++++++++++++++++++++++++++++++
3 files changed, 531 insertions(+)
create mode 100644 meta/lib/oeqa/core/context.py
create mode 100644 meta/lib/oeqa/core/decorator/__init__.py
create mode 100644 meta/lib/oeqa/core/loader.py
diff --git a/meta/lib/oeqa/core/context.py b/meta/lib/oeqa/core/context.py
new file mode 100644
index 0000000..d5caf53
--- /dev/null
+++ b/meta/lib/oeqa/core/context.py
@@ -0,0 +1,225 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import sys
+import json
+import time
+import logging
+import collections
+import re
+
+from oeqa.core.loader import OETestLoader
+from oeqa.core.runner import OETestRunner, OEStreamLogger, xmlEnabled
+
+class OETestContext(object):
+ loaderClass = OETestLoader
+ runnerClass = OETestRunner
+ streamLoggerClass = OEStreamLogger
+
+ files_dir = os.path.abspath(os.path.join(os.path.dirname(
+ os.path.abspath(__file__)), "../files"))
+
+ def __init__(self, d=None, logger=None):
+ if not type(d) is dict:
+ raise TypeError("d isn't dictionary type")
+
+ self.d = d
+ self.logger = logger
+ self._registry = {}
+ self._registry['cases'] = collections.OrderedDict()
+ self._results = {}
+
+ def _read_modules_from_manifest(self, manifest):
+ if not os.path.exists(manifest):
+ raise
+
+ modules = []
+ for line in open(manifest).readlines():
+ line = line.strip()
+ if line and not line.startswith("#"):
+ modules.append(line)
+
+ return modules
+
+ def loadTests(self, module_paths, modules=[], tests=[],
+ modules_manifest="", modules_required=[], filters={}):
+ if modules_manifest:
+ modules = self._read_modules_from_manifest(modules_manifest)
+
+ self.loader = self.loaderClass(self, module_paths, modules, tests,
+ modules_required, filters)
+ self.suites = self.loader.discover()
+
+ def runTests(self):
+ streamLogger = self.streamLoggerClass(self.logger)
+ self.runner = self.runnerClass(self, stream=streamLogger, verbosity=2)
+
+ self._run_start_time = time.time()
+ result = self.runner.run(self.suites)
+ self._run_end_time = time.time()
+
+ return result
+
+ def logSummary(self, result, component, context_msg=''):
+ self.logger.info("SUMMARY:")
+ self.logger.info("%s (%s) - Ran %d test%s in %.3fs" % (component,
+ context_msg, result.testsRun, result.testsRun != 1 and "s" or "",
+ (self._run_end_time - self._run_start_time)))
+
+ if result.wasSuccessful():
+ msg = "%s - OK - All required tests passed" % component
+ else:
+ msg = "%s - FAIL - Required tests failed" % component
+ skipped = len(self._results['skipped'])
+ if skipped:
+ msg += " (skipped=%d)" % skipped
+ self.logger.info(msg)
+
+ def _logDetailsNotPassed(self, case, type, desc):
+ found = False
+
+ for (scase, msg) in self._results[type]:
+ # XXX: When XML reporting is enabled scase is
+ # xmlrunner.result._TestInfo instance instead of
+ # string.
+ if xmlEnabled:
+ if case.id() == scase.test_id:
+ found = True
+ break
+ scase_str = scase.test_id
+ else:
+ if case == scase:
+ found = True
+ break
+ scase_str = str(scase)
+
+ # When fails at module or class level the class name is passed as string
+ # so figure out to see if match
+ m = re.search("^setUpModule \((?P<module_name>.*)\)$", scase_str)
+ if m:
+ if case.__class__.__module__ == m.group('module_name'):
+ found = True
+ break
+
+ m = re.search("^setUpClass \((?P<class_name>.*)\)$", scase_str)
+ if m:
+ class_name = "%s.%s" % (case.__class__.__module__,
+ case.__class__.__name__)
+
+ if class_name == m.group('class_name'):
+ found = True
+ break
+
+ if found:
+ self.logger.info("RESULTS - %s - Testcase %s: %s" % (case.id(),
+ case.oe_id if hasattr(case, 'oeid') else '-1', desc))
+ if msg:
+ self.logger.info(msg)
+
+ return found
+
+ def logDetails(self):
+ self.logger.info("RESULTS:")
+ for case_name in self._registry['cases']:
+ case = self._registry['cases'][case_name]
+
+ result_types = ['failures', 'errors', 'skipped', 'expectedFailures']
+ result_desc = ['FAILED', 'ERROR', 'SKIPPED', 'EXPECTEDFAIL']
+
+ match = False
+ for idx, name in enumerate(result_types):
+ match = self._logDetailsNotPassed(case, result_types[idx],
+ result_desc[idx])
+ if match:
+ break
+
+ if not match:
+ self.logger.info("RESULTS - %s - Testcase %s: %s" % (case.id(),
+ case.oe_id if hasattr(case, 'oeid') else '-1',
+ 'PASSED'))
+
+class OETestContextExecutor(object):
+ _context_class = OETestContext
+
+ name = 'core'
+ help = 'core test component example'
+ description = 'executes core test suite example'
+
+ default_cases = [os.path.join(os.path.abspath(os.path.dirname(__file__)),
+ 'cases/example')]
+ default_test_data = os.path.join(default_cases[0], 'data.json')
+
+ def register_commands(self, logger, subparsers):
+ self.parser = subparsers.add_parser(self.name, help=self.help,
+ description=self.description, group='components')
+
+ self.default_output_log = '%s-results-%s.log' % (self.name,
+ time.strftime("%Y%m%d%H%M%S"))
+ self.parser.add_argument('--output-log', action='store',
+ default=self.default_output_log,
+ help="results output log, default: %s" % self.default_output_log)
+
+ if self.default_test_data:
+ self.parser.add_argument('--test-data-file', action='store',
+ default=self.default_test_data,
+ help="data file to load, default: %s" % self.default_test_data)
+ else:
+ self.parser.add_argument('--test-data-file', action='store',
+ help="data file to load")
+
+ if self.default_cases:
+ self.parser.add_argument('CASES_PATHS', action='store',
+ default=self.default_cases, nargs='*',
+ help="paths to directories with test cases, default: %s"\
+ % self.default_cases)
+ else:
+ self.parser.add_argument('CASES_PATHS', action='store',
+ nargs='+', help="paths to directories with test cases")
+
+ self.parser.set_defaults(func=self.run)
+
+ def _setup_logger(self, logger, args):
+ formatter = logging.Formatter('%(asctime)s - ' + self.name + \
+ ' - %(levelname)s - %(message)s')
+ sh = logger.handlers[0]
+ sh.setFormatter(formatter)
+ fh = logging.FileHandler(args.output_log)
+ fh.setFormatter(formatter)
+ logger.addHandler(fh)
+
+ return logger
+
+ def _process_args(self, logger, args):
+ self.tc_kwargs = {}
+ self.tc_kwargs['init'] = {}
+ self.tc_kwargs['load'] = {}
+ self.tc_kwargs['run'] = {}
+
+ self.tc_kwargs['init']['logger'] = self._setup_logger(logger, args)
+ if args.test_data_file:
+ self.tc_kwargs['init']['td'] = json.load(
+ open(args.test_data_file, "r"))
+ else:
+ self.tc_kwargs['init']['td'] = {}
+
+ self.module_paths = args.CASES_PATHS
+
+ def run(self, logger, args):
+ self._process_args(logger, args)
+
+ self.tc = self._context_class(**self.tc_kwargs['init'])
+ self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
+ rc = self.tc.runTests(**self.tc_kwargs['run'])
+ self.tc.logSummary(rc, self.name)
+ self.tc.logDetails()
+
+ output_link = os.path.join(os.path.dirname(args.output_log),
+ "%s-results.log" % self.name)
+ if os.path.exists(output_link):
+ os.remove(output_link)
+ os.symlink(args.output_log, output_link)
+
+ return rc
+
+_executor_class = OETestContextExecutor
diff --git a/meta/lib/oeqa/core/decorator/__init__.py b/meta/lib/oeqa/core/decorator/__init__.py
new file mode 100644
index 0000000..855b6b9
--- /dev/null
+++ b/meta/lib/oeqa/core/decorator/__init__.py
@@ -0,0 +1,71 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+from functools import wraps
+from abc import abstractmethod
+
+decoratorClasses = set()
+
+def registerDecorator(obj):
+ decoratorClasses.add(obj)
+ return obj
+
+class OETestDecorator(object):
+ case = None # Reference of OETestCase decorated
+ attrs = None # Attributes to be loaded by decorator implementation
+
+ def __init__(self, *args, **kwargs):
+ if not self.attrs:
+ return
+
+ for idx, attr in enumerate(self.attrs):
+ if attr in kwargs:
+ value = kwargs[attr]
+ else:
+ value = args[idx]
+ setattr(self, attr, value)
+
+ def __call__(self, func):
+ @wraps(func)
+ def wrapped_f(*args, **kwargs):
+ self.attrs = self.attrs # XXX: Enables OETestLoader discover
+ return func(*args, **kwargs)
+ return wrapped_f
+
+ # OETestLoader call it when is loading test cases.
+ # XXX: Most methods would change the registry for later
+ # processing; be aware that filtrate method needs to
+ # run later than bind, so there could be data (in the
+ # registry) of a cases that were filtered.
+ def bind(self, registry, case):
+ self.case = case
+ self.logger = case.tc.logger
+ self.case.decorators.append(self)
+
+ # OETestRunner call this method when tries to run
+ # the test case.
+ def setUpDecorator(self):
+ pass
+
+ # OETestRunner call it after a test method has been
+ # called even if the method raised an exception.
+ def tearDownDecorator(self):
+ pass
+
+class OETestDiscover(OETestDecorator):
+
+ # OETestLoader call it after discover test cases
+ # needs to return the cases to be run.
+ @staticmethod
+ def discover(registry):
+ return registry['cases']
+
+class OETestFilter(OETestDecorator):
+
+ # OETestLoader call it while loading the tests
+ # in loadTestsFromTestCase method, it needs to
+ # return a bool, True if needs to be filtered.
+ # This method must consume the filter used.
+ @abstractmethod
+ def filtrate(self, filters):
+ return False
diff --git a/meta/lib/oeqa/core/loader.py b/meta/lib/oeqa/core/loader.py
new file mode 100644
index 0000000..94f71ba
--- /dev/null
+++ b/meta/lib/oeqa/core/loader.py
@@ -0,0 +1,235 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import sys
+import unittest
+
+from oeqa.core.utils.path import findFile
+from oeqa.core.utils.test import getSuiteModules, getCaseID
+
+from oeqa.core.case import OETestCase
+from oeqa.core.decorator import decoratorClasses, OETestDecorator, \
+ OETestFilter, OETestDiscover
+
+def _make_failed_test(classname, methodname, exception, suiteClass):
+ """
+ When loading tests unittest framework stores the exception in a new
+ class created for be displayed into run().
+
+ For our purposes will be better to raise the exception in loading
+ step instead of wait to run the test suite.
+ """
+ raise exception
+unittest.loader._make_failed_test = _make_failed_test
+
+def _find_duplicated_modules(suite, directory):
+ for module in getSuiteModules(suite):
+ path = findFile('%s.py' % module, directory)
+ if path:
+ raise ImportError("Duplicated %s module found in %s" % (module, path))
+
+class OETestLoader(unittest.TestLoader):
+ caseClass = OETestCase
+
+ kwargs_names = ['testMethodPrefix', 'sortTestMethodUsing', 'suiteClass',
+ '_top_level_dir']
+
+ def __init__(self, tc, module_paths, modules, tests, modules_required,
+ filters, *args, **kwargs):
+ self.tc = tc
+
+ self.modules = modules
+ self.tests = tests
+ self.modules_required = modules_required
+
+ self.filters = filters
+ self.decorator_filters = [d for d in decoratorClasses if \
+ issubclass(d, OETestFilter)]
+ self._validateFilters(self.filters, self.decorator_filters)
+ self.used_filters = [d for d in self.decorator_filters
+ for f in self.filters
+ if f in d.attrs]
+
+ if isinstance(module_paths, str):
+ module_paths = [module_paths]
+ elif not isinstance(module_paths, list):
+ raise TypeError('module_paths must be a str or a list of str')
+ self.module_paths = module_paths
+
+ for kwname in self.kwargs_names:
+ if kwname in kwargs:
+ setattr(self, kwname, kwargs[kwname])
+
+ self._patchCaseClass(self.caseClass)
+
+ def _patchCaseClass(self, testCaseClass):
+ # Adds custom attributes to the OETestCase class
+ setattr(testCaseClass, 'tc', self.tc)
+ setattr(testCaseClass, 'd', self.tc.d)
+ setattr(testCaseClass, 'logger', self.tc.logger)
+
+ def _validateFilters(self, filters, decorator_filters):
+ # Validate if filter isn't empty
+ for key,value in filters.items():
+ if not value:
+ raise TypeError("Filter %s specified is empty" % key)
+
+ # Validate unique attributes
+ attr_filters = [attr for clss in decorator_filters \
+ for attr in clss.attrs]
+ dup_attr = [attr for attr in attr_filters
+ if attr_filters.count(attr) > 1]
+ if dup_attr:
+ raise TypeError('Detected duplicated attribute(s) %s in filter'
+ ' decorators' % ' ,'.join(dup_attr))
+
+ # Validate if filter is supported
+ for f in filters:
+ if f not in attr_filters:
+ classes = ', '.join([d.__name__ for d in decorator_filters])
+ raise TypeError('Found "%s" filter but not declared in any of '
+ '%s decorators' % (f, classes))
+
+ def _registerTestCase(self, case):
+ case_id = case.id()
+ self.tc._registry['cases'][case_id] = case
+
+ def _handleTestCaseDecorators(self, case):
+ def _handle(obj):
+ if isinstance(obj, OETestDecorator):
+ if not obj.__class__ in decoratorClasses:
+ raise Exception("Decorator %s isn't registered" \
+ " in decoratorClasses." % obj.__name__)
+ obj.bind(self.tc._registry, case)
+
+ def _walk_closure(obj):
+ if hasattr(obj, '__closure__') and obj.__closure__:
+ for f in obj.__closure__:
+ obj = f.cell_contents
+ _handle(obj)
+ _walk_closure(obj)
+ method = getattr(case, case._testMethodName, None)
+ _walk_closure(method)
+
+ def _filterTest(self, case):
+ """
+ Returns True if test case must be filtered, False otherwise.
+ """
+ if self.filters:
+ filters = self.filters.copy()
+ case_decorators = [cd for cd in case.decorators
+ if cd.__class__ in self.used_filters]
+
+ # Iterate over case decorators to check if needs to be filtered.
+ for cd in case_decorators:
+ if cd.filtrate(filters):
+ return True
+
+ # Case is missing one or more decorators for all the filters
+ # being used, so filter test case.
+ if filters:
+ return True
+
+ return False
+
+ def _getTestCase(self, testCaseClass, tcName):
+ if not hasattr(testCaseClass, '__oeqa_loader'):
+ # In order to support data_vars validation
+ # monkey patch the default setUp/tearDown{Class} to use
+ # the ones provided by OETestCase
+ setattr(testCaseClass, 'setUpClassMethod',
+ getattr(testCaseClass, 'setUpClass'))
+ setattr(testCaseClass, 'tearDownClassMethod',
+ getattr(testCaseClass, 'tearDownClass'))
+ setattr(testCaseClass, 'setUpClass',
+ testCaseClass._oeSetUpClass)
+ setattr(testCaseClass, 'tearDownClass',
+ testCaseClass._oeTearDownClass)
+
+ # In order to support decorators initialization
+ # monkey patch the default setUp/tearDown to use
+ # a setUpDecorators/tearDownDecorators that methods
+ # will call setUp/tearDown original methods.
+ setattr(testCaseClass, 'setUpMethod',
+ getattr(testCaseClass, 'setUp'))
+ setattr(testCaseClass, 'tearDownMethod',
+ getattr(testCaseClass, 'tearDown'))
+ setattr(testCaseClass, 'setUp', testCaseClass._oeSetUp)
+ setattr(testCaseClass, 'tearDown', testCaseClass._oeTearDown)
+
+ setattr(testCaseClass, '__oeqa_loader', True)
+
+ case = testCaseClass(tcName)
+ setattr(case, 'decorators', [])
+
+ return case
+
+ def loadTestsFromTestCase(self, testCaseClass):
+ """
+ Returns a suite of all tests cases contained in testCaseClass.
+ """
+ if issubclass(testCaseClass, unittest.suite.TestSuite):
+ raise TypeError("Test cases should not be derived from TestSuite." \
+ " Maybe you meant to derive from TestCase?")
+ if not issubclass(testCaseClass, self.caseClass):
+ raise TypeError("Test cases need to be derived from %s" % \
+ caseClass.__name__)
+
+
+ testCaseNames = self.getTestCaseNames(testCaseClass)
+ if not testCaseNames and hasattr(testCaseClass, 'runTest'):
+ testCaseNames = ['runTest']
+
+ suite = []
+ for tcName in testCaseNames:
+ case = self._getTestCase(testCaseClass, tcName)
+ # Filer by case id
+ if not (self.tests and not 'all' in self.tests
+ and not getCaseID(case) in self.tests):
+ self._handleTestCaseDecorators(case)
+
+ # Filter by decorators
+ if not self._filterTest(case):
+ self._registerTestCase(case)
+ suite.append(case)
+
+ return self.suiteClass(suite)
+
+ def discover(self):
+ big_suite = self.suiteClass()
+ for path in self.module_paths:
+ _find_duplicated_modules(big_suite, path)
+ suite = super(OETestLoader, self).discover(path,
+ pattern='*.py', top_level_dir=path)
+ big_suite.addTests(suite)
+
+ cases = None
+ discover_classes = [clss for clss in decoratorClasses
+ if issubclass(clss, OETestDiscover)]
+ for clss in discover_classes:
+ cases = clss.discover(self.tc._registry)
+
+ return self.suiteClass(cases) if cases else big_suite
+
+ # XXX After Python 3.5, remove backward compatibility hacks for
+ # use_load_tests deprecation via *args and **kws. See issue 16662.
+ if sys.version_info >= (3,5):
+ def loadTestsFromModule(self, module, *args, pattern=None, **kws):
+ if not self.modules or "all" in self.modules or \
+ module.__name__ in self.modules:
+ return super(OETestLoader, self).loadTestsFromModule(
+ module, *args, pattern=pattern, **kws)
+ else:
+ return self.suiteClass()
+ else:
+ def loadTestsFromModule(self, module, use_load_tests=True):
+ """
+ Returns a suite of all tests cases contained in module.
+ """
+ if not self.modules or "all" in self.modules or \
+ module.__name__ in self.modules:
+ return super(OETestLoader, self).loadTestsFromModule(
+ module, use_load_tests)
+ else:
+ return self.suiteClass()
--
2.1.4
^ permalink raw reply related
* [PATCHv2 01/32] oeqa/core: Add base OEQA framework
From: Aníbal Limón @ 2016-12-07 20:32 UTC (permalink / raw)
To: openembedded-core
In-Reply-To: <cover.1481142664.git.anibal.limon@linux.intel.com>
case: Defines OETestCase base class that provides custom
methods/attrs defined by the framework.
Every OETestCase instance contains a reference to the test
data (d), the test context (tc) and the logger.
Also implements _oe{SetUp,TearDown}Class for make special
handling of OEQA decorators and validations.
runner: Defines OETestRunner/OETestResult with support for RAW
and XML result logs.
exception: Custom exceptions related to the OEQA framework based
on class OEQAException.
[YOCTO #10230]
[YOCTO #10233]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/lib/oeqa/core/__init__.py | 0
meta/lib/oeqa/core/case.py | 46 +++++++++++++++++++++++++
meta/lib/oeqa/core/exception.py | 14 ++++++++
meta/lib/oeqa/core/runner.py | 76 +++++++++++++++++++++++++++++++++++++++++
4 files changed, 136 insertions(+)
create mode 100644 meta/lib/oeqa/core/__init__.py
create mode 100644 meta/lib/oeqa/core/case.py
create mode 100644 meta/lib/oeqa/core/exception.py
create mode 100644 meta/lib/oeqa/core/runner.py
diff --git a/meta/lib/oeqa/core/__init__.py b/meta/lib/oeqa/core/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/meta/lib/oeqa/core/case.py b/meta/lib/oeqa/core/case.py
new file mode 100644
index 0000000..a47cb19
--- /dev/null
+++ b/meta/lib/oeqa/core/case.py
@@ -0,0 +1,46 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import unittest
+
+from oeqa.core.exception import OEQAMissingVariable
+
+def _validate_data_vars(d, data_vars, type_msg):
+ if data_vars:
+ for v in data_vars:
+ if not v in d:
+ raise OEQAMissingVariable("Test %s need %s variable but"\
+ " isn't into d" % (type_msg, v))
+
+class OETestCase(unittest.TestCase):
+ # TestContext and Logger instance set by OETestLoader.
+ tc = None
+ logger = None
+
+ # d has all the variables needed by the test cases
+ # is the same across all the test cases.
+ d = None
+
+ # data_vars has the variables needed by a test class
+ # or test case instance, if some var isn't into d a
+ # OEMissingVariable exception is raised
+ data_vars = None
+
+ @classmethod
+ def _oeSetUpClass(clss):
+ _validate_data_vars(clss.d, clss.data_vars, "class")
+ clss.setUpClassMethod()
+
+ @classmethod
+ def _oeTearDownClass(clss):
+ clss.tearDownClassMethod()
+
+ def _oeSetUp(self):
+ for d in self.decorators:
+ d.setUpDecorator()
+ self.setUpMethod()
+
+ def _oeTearDown(self):
+ for d in self.decorators:
+ d.tearDownDecorator()
+ self.tearDownMethod()
diff --git a/meta/lib/oeqa/core/exception.py b/meta/lib/oeqa/core/exception.py
new file mode 100644
index 0000000..2dfd840
--- /dev/null
+++ b/meta/lib/oeqa/core/exception.py
@@ -0,0 +1,14 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+class OEQAException(Exception):
+ pass
+
+class OEQATimeoutError(OEQAException):
+ pass
+
+class OEQAMissingVariable(OEQAException):
+ pass
+
+class OEQADependency(OEQAException):
+ pass
diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py
new file mode 100644
index 0000000..8f5af57
--- /dev/null
+++ b/meta/lib/oeqa/core/runner.py
@@ -0,0 +1,76 @@
+# Copyright (C) 2016 Intel Corporation
+# Released under the MIT license (see COPYING.MIT)
+
+import os
+import time
+import unittest
+import logging
+
+xmlEnabled = False
+try:
+ import xmlrunner
+ from xmlrunner.result import _XMLTestResult as _TestResult
+ from xmlrunner.runner import XMLTestRunner as _TestRunner
+ xmlEnabled = True
+except ImportError:
+ # use the base runner instead
+ from unittest import TextTestResult as _TestResult
+ from unittest import TextTestRunner as _TestRunner
+
+class OEStreamLogger(object):
+ def __init__(self, logger):
+ self.logger = logger
+ self.buffer = ""
+
+ def write(self, msg):
+ if msg[-1] != '\n':
+ self.buffer += msg
+ else:
+ self.logger.log(logging.INFO, self.buffer.rstrip("\n"))
+ self.buffer = ""
+
+ def flush(self):
+ for handler in self.logger.handlers:
+ handler.flush()
+
+class OETestResult(_TestResult):
+ def __init__(self, tc, *args, **kwargs):
+ super(OETestResult, self).__init__(*args, **kwargs)
+
+ self.tc = tc
+
+ self.tc._results['failures'] = self.failures
+ self.tc._results['errors'] = self.errors
+ self.tc._results['skipped'] = self.skipped
+ self.tc._results['expectedFailures'] = self.expectedFailures
+
+ def startTest(self, test):
+ super(OETestResult, self).startTest(test)
+
+class OETestRunner(_TestRunner):
+ def __init__(self, tc, *args, **kwargs):
+ if xmlEnabled:
+ if not kwargs.get('output'):
+ kwargs['output'] = os.path.join(os.getcwd(),
+ 'TestResults_%s' % time.strftime("%Y%m%d%H%M%S"))
+
+ super(OETestRunner, self).__init__(*args, **kwargs)
+ self.tc = tc
+ self.resultclass = OETestResult
+
+ # XXX: The unittest-xml-reporting package defines _make_result method instead
+ # of _makeResult standard on unittest.
+ if xmlEnabled:
+ def _make_result(self):
+ """
+ Creates a TestResult object which will be used to store
+ information about the executed tests.
+ """
+ # override in subclasses if necessary.
+ return self.resultclass(self.tc,
+ self.stream, self.descriptions, self.verbosity, self.elapsed_times
+ )
+ else:
+ def _makeResult(self):
+ return self.resultclass(self.tc, self.stream, self.descriptions,
+ self.verbosity)
--
2.1.4
^ permalink raw reply related
* [PATCHv2 00/32] OEQA Framework Refactor & Improvements
From: Aníbal Limón @ 2016-12-07 20:32 UTC (permalink / raw)
To: openembedded-core
This patchset is related to OEQA Framework for details read the RFC send to the
Openembedded architecture ML.
http://lists.openembedded.org/pipermail/openembedded-architecture/2016-December/000351.html
This v2 fixes sdk extensible test case sdk_update.
The following changes since commit 9e63f81c78e284c9b325fe04a1b59e61c7ad8a1a:
bitbake: ast: remove BBVERSIONS support (2016-11-30 15:48:10 +0000)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib alimon/oeqa_sdk_migration
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=alimon/oeqa_sdk_migration
Aníbal Limón (28):
oeqa/core: Add base OEQA framework
oeqa/core: Add loader, context and decorator modules
oeqa/core/decorator: Add support for OETestDepends
oeqa/core/decorator: Add support for OETestDataDepends and
skipIfDataVar
scripts/oe-test: Add new oe-test script
oeqa/core/context: Add support of OETestContextExecutor
oeqa/core/cases: Add example test cases
oeqa/core: Add README
oe/data: Add export2json function
classes/rootfs-postcommands: Add write_image_test_data
classes/populate_sdk_base: Add write_sdk_test_data to postprocess
oeqa/core: Change name of d to td
oeqa/utils/path: Add remove_safe function
oeqa: Move common files to oeqa/files instead of runtime only
oeqa/sdk: Move test cases inside cases directory
oeqa/{runtime,sdk}/files: Move testsdkmakefile from runtime to sdk
module
oeqa/sdk: Add case and context modules for the SDK component
classes/testsdk: Migrates testsdk.bbclass to use new OESDKTestContext
oeqa/utils: Move targetbuild to buildproject module
oeqa/utils: {Target,SDK,}BuildProject remove dependency of bb
oeqa/sdk/cases: Migrate tests to the new OEQA framework
classes/testsdk: Remove the need of TEST_LOG_DIR variable
oeqa/sdkext: Move test cases inside cases directory
oeqa/sdkext: Adds case and context modules.
classes/testsdk: Migrate to use the new OESDKExtTestContext
oeqa/sdkext/cases: Migrate test case to new OEQA framework
oeqa/runtime: Fix TargetBuildProject instances
oeqa: Fix files handling on runtime tests.
Mariano Lopez (4):
oeqa/core: Add utils module for OEQA framework
oeqa/core/decorator: Add support for OETestID and OETestTag
oeqa/core/decorator: Add support for OETimeout decorator
oeqa/core: Add tests for the OEQA framework
meta/classes/populate_sdk_base.bbclass | 9 +-
meta/classes/rootfs-postcommands.bbclass | 18 ++
meta/classes/testexport.bbclass | 5 +
meta/classes/testimage.bbclass | 4 +
meta/classes/testsdk.bbclass | 162 ++++++++------
meta/lib/oe/data.py | 28 +++
meta/lib/oeqa/core/README | 38 ++++
meta/lib/oeqa/core/__init__.py | 0
meta/lib/oeqa/core/case.py | 46 ++++
meta/lib/oeqa/core/cases/__init__.py | 0
meta/lib/oeqa/core/cases/example/data.json | 1 +
meta/lib/oeqa/core/cases/example/test_basic.py | 20 ++
meta/lib/oeqa/core/context.py | 225 ++++++++++++++++++++
meta/lib/oeqa/core/decorator/__init__.py | 71 +++++++
meta/lib/oeqa/core/decorator/data.py | 36 ++++
meta/lib/oeqa/core/decorator/depends.py | 94 +++++++++
meta/lib/oeqa/core/decorator/oeid.py | 23 ++
meta/lib/oeqa/core/decorator/oetag.py | 24 +++
meta/lib/oeqa/core/decorator/oetimeout.py | 25 +++
meta/lib/oeqa/core/exception.py | 14 ++
meta/lib/oeqa/core/loader.py | 235 +++++++++++++++++++++
meta/lib/oeqa/core/runner.py | 76 +++++++
meta/lib/oeqa/core/tests/__init__.py | 0
meta/lib/oeqa/core/tests/cases/data.py | 20 ++
meta/lib/oeqa/core/tests/cases/depends.py | 38 ++++
.../oeqa/core/tests/cases/loader/invalid/oeid.py | 15 ++
.../oeqa/core/tests/cases/loader/valid/another.py | 9 +
meta/lib/oeqa/core/tests/cases/oeid.py | 18 ++
meta/lib/oeqa/core/tests/cases/oetag.py | 18 ++
meta/lib/oeqa/core/tests/cases/timeout.py | 18 ++
meta/lib/oeqa/core/tests/common.py | 35 +++
meta/lib/oeqa/core/tests/test_data.py | 51 +++++
meta/lib/oeqa/core/tests/test_decorators.py | 135 ++++++++++++
meta/lib/oeqa/core/tests/test_loader.py | 86 ++++++++
meta/lib/oeqa/core/tests/test_runner.py | 38 ++++
meta/lib/oeqa/core/utils/__init__.py | 0
meta/lib/oeqa/core/utils/misc.py | 37 ++++
meta/lib/oeqa/core/utils/path.py | 19 ++
meta/lib/oeqa/core/utils/test.py | 71 +++++++
meta/lib/oeqa/{runtime => }/files/test.c | 0
meta/lib/oeqa/{runtime => }/files/test.cpp | 0
meta/lib/oeqa/{runtime => }/files/test.pl | 0
meta/lib/oeqa/{runtime => }/files/test.py | 0
meta/lib/oeqa/oetest.py | 92 +-------
meta/lib/oeqa/runtime/buildcvs.py | 9 +-
meta/lib/oeqa/runtime/buildgalculator.py | 8 +-
meta/lib/oeqa/runtime/buildiptables.py | 8 +-
meta/lib/oeqa/runtime/gcc.py | 4 +-
meta/lib/oeqa/runtime/perl.py | 2 +-
meta/lib/oeqa/runtime/python.py | 2 +-
meta/lib/oeqa/runtime/utils/__init__.py | 0
meta/lib/oeqa/runtime/utils/targetbuildproject.py | 33 +++
meta/lib/oeqa/sdk/__init__.py | 3 -
meta/lib/oeqa/sdk/case.py | 12 ++
meta/lib/oeqa/sdk/{ => cases}/buildcvs.py | 15 +-
meta/lib/oeqa/sdk/{ => cases}/buildgalculator.py | 28 ++-
meta/lib/oeqa/sdk/{ => cases}/buildiptables.py | 16 +-
meta/lib/oeqa/sdk/cases/gcc.py | 43 ++++
meta/lib/oeqa/sdk/cases/perl.py | 27 +++
meta/lib/oeqa/sdk/{ => cases}/python.py | 25 ++-
meta/lib/oeqa/sdk/context.py | 133 ++++++++++++
.../oeqa/{runtime => sdk}/files/testsdkmakefile | 0
meta/lib/oeqa/sdk/gcc.py | 36 ----
meta/lib/oeqa/sdk/perl.py | 28 ---
meta/lib/oeqa/sdk/utils/__init__.py | 0
meta/lib/oeqa/sdk/utils/sdkbuildproject.py | 45 ++++
meta/lib/oeqa/sdkext/__init__.py | 3 -
meta/lib/oeqa/sdkext/case.py | 21 ++
meta/lib/oeqa/sdkext/{ => cases}/devtool.py | 49 +++--
meta/lib/oeqa/sdkext/{ => cases}/sdk_update.py | 17 +-
meta/lib/oeqa/sdkext/context.py | 21 ++
meta/lib/oeqa/utils/buildproject.py | 52 +++++
meta/lib/oeqa/utils/targetbuild.py | 135 ------------
scripts/oe-test | 98 +++++++++
74 files changed, 2284 insertions(+), 443 deletions(-)
create mode 100644 meta/lib/oeqa/core/README
create mode 100644 meta/lib/oeqa/core/__init__.py
create mode 100644 meta/lib/oeqa/core/case.py
create mode 100644 meta/lib/oeqa/core/cases/__init__.py
create mode 100644 meta/lib/oeqa/core/cases/example/data.json
create mode 100644 meta/lib/oeqa/core/cases/example/test_basic.py
create mode 100644 meta/lib/oeqa/core/context.py
create mode 100644 meta/lib/oeqa/core/decorator/__init__.py
create mode 100644 meta/lib/oeqa/core/decorator/data.py
create mode 100644 meta/lib/oeqa/core/decorator/depends.py
create mode 100644 meta/lib/oeqa/core/decorator/oeid.py
create mode 100644 meta/lib/oeqa/core/decorator/oetag.py
create mode 100644 meta/lib/oeqa/core/decorator/oetimeout.py
create mode 100644 meta/lib/oeqa/core/exception.py
create mode 100644 meta/lib/oeqa/core/loader.py
create mode 100644 meta/lib/oeqa/core/runner.py
create mode 100644 meta/lib/oeqa/core/tests/__init__.py
create mode 100644 meta/lib/oeqa/core/tests/cases/data.py
create mode 100644 meta/lib/oeqa/core/tests/cases/depends.py
create mode 100644 meta/lib/oeqa/core/tests/cases/loader/invalid/oeid.py
create mode 100644 meta/lib/oeqa/core/tests/cases/loader/valid/another.py
create mode 100644 meta/lib/oeqa/core/tests/cases/oeid.py
create mode 100644 meta/lib/oeqa/core/tests/cases/oetag.py
create mode 100644 meta/lib/oeqa/core/tests/cases/timeout.py
create mode 100644 meta/lib/oeqa/core/tests/common.py
create mode 100755 meta/lib/oeqa/core/tests/test_data.py
create mode 100755 meta/lib/oeqa/core/tests/test_decorators.py
create mode 100755 meta/lib/oeqa/core/tests/test_loader.py
create mode 100755 meta/lib/oeqa/core/tests/test_runner.py
create mode 100644 meta/lib/oeqa/core/utils/__init__.py
create mode 100644 meta/lib/oeqa/core/utils/misc.py
create mode 100644 meta/lib/oeqa/core/utils/path.py
create mode 100644 meta/lib/oeqa/core/utils/test.py
rename meta/lib/oeqa/{runtime => }/files/test.c (100%)
rename meta/lib/oeqa/{runtime => }/files/test.cpp (100%)
rename meta/lib/oeqa/{runtime => }/files/test.pl (100%)
rename meta/lib/oeqa/{runtime => }/files/test.py (100%)
create mode 100644 meta/lib/oeqa/runtime/utils/__init__.py
create mode 100644 meta/lib/oeqa/runtime/utils/targetbuildproject.py
create mode 100644 meta/lib/oeqa/sdk/case.py
rename meta/lib/oeqa/sdk/{ => cases}/buildcvs.py (59%)
rename meta/lib/oeqa/sdk/{ => cases}/buildgalculator.py (45%)
rename meta/lib/oeqa/sdk/{ => cases}/buildiptables.py (58%)
create mode 100644 meta/lib/oeqa/sdk/cases/gcc.py
create mode 100644 meta/lib/oeqa/sdk/cases/perl.py
rename meta/lib/oeqa/sdk/{ => cases}/python.py (46%)
create mode 100644 meta/lib/oeqa/sdk/context.py
rename meta/lib/oeqa/{runtime => sdk}/files/testsdkmakefile (100%)
delete mode 100644 meta/lib/oeqa/sdk/gcc.py
delete mode 100644 meta/lib/oeqa/sdk/perl.py
create mode 100644 meta/lib/oeqa/sdk/utils/__init__.py
create mode 100644 meta/lib/oeqa/sdk/utils/sdkbuildproject.py
create mode 100644 meta/lib/oeqa/sdkext/case.py
rename meta/lib/oeqa/sdkext/{ => cases}/devtool.py (73%)
rename meta/lib/oeqa/sdkext/{ => cases}/sdk_update.py (63%)
create mode 100644 meta/lib/oeqa/sdkext/context.py
create mode 100644 meta/lib/oeqa/utils/buildproject.py
delete mode 100644 meta/lib/oeqa/utils/targetbuild.py
create mode 100755 scripts/oe-test
--
2.1.4
^ permalink raw reply
* [PATCH] Revert "webkitgtk: drop patch 0001-WebKitMacros-Append-to-I-and-not-to-isystem.patch"
From: Carlos Alberto Lopez Perez @ 2016-12-07 20:17 UTC (permalink / raw)
To: openembedded-core
In-Reply-To: <CAMKF1srtVjVmEN0s2vKgw89g=LxJSG36eEkxFdhhouuxzg6KYQ@mail.gmail.com>
This reverts commit 812c52f654c1bccca033163100055e3a8b8cda6e.
Upstream fixed the issue with GCC. But the build still fails with Clang.
Therefore reintroduce this patch until a better solution is found.
Upstream bug: https://bugs.webkit.org/show_bug.cgi?id=161697
Signed-off-by: Carlos Alberto Lopez Perez <clopez@igalia.com>
---
...bKitMacros-Append-to-I-and-not-to-isystem.patch | 185 +++++++++++++++++++++
meta/recipes-sato/webkit/webkitgtk_2.14.2.bb | 1 +
2 files changed, 186 insertions(+)
create mode 100644 meta/recipes-sato/webkit/files/0001-WebKitMacros-Append-to-I-and-not-to-isystem.patch
diff --git a/meta/recipes-sato/webkit/files/0001-WebKitMacros-Append-to-I-and-not-to-isystem.patch b/meta/recipes-sato/webkit/files/0001-WebKitMacros-Append-to-I-and-not-to-isystem.patch
new file mode 100644
index 0000000..a4face2
--- /dev/null
+++ b/meta/recipes-sato/webkit/files/0001-WebKitMacros-Append-to-I-and-not-to-isystem.patch
@@ -0,0 +1,185 @@
+From 20ee11dd188e1538f8cdd17a289dc6f9c63a011e Mon Sep 17 00:00:00 2001
+From: Khem Raj <raj.khem@gmail.com>
+Date: Sun, 17 Apr 2016 12:35:41 -0700
+Subject: [PATCH] WebKitMacros: Append to -I and not to -isystem
+
+gcc-6 has now introduced stdlib.h in libstdc++ for better
+compliance and its including the C library stdlib.h using
+include_next which is sensitive to order of system header
+include paths. Its infact better to not tinker with the
+system header include paths at all. Since adding /usr/include
+to -system is redundant and compiler knows about it moreover
+now with gcc6 it interferes with compiler's functioning
+and ends up with compile errors e.g.
+
+/usr/include/c++/6.0.0/cstdlib:75:25: fatal error: stdlib.h: No such file or directory
+
+This is also an issue with clang (when using libstdc++ >= 6)
+
+Upstream bug: https://bugs.webkit.org/show_bug.cgi?id=161697
+
+Signed-off-by: Khem Raj <raj.khem@gmail.com>
+Upstream-Status: Pending
+---
+ Source/JavaScriptCore/shell/CMakeLists.txt | 2 +-
+ Source/WebCore/PlatformGTK.cmake | 6 +++---
+ Source/WebKit2/PlatformGTK.cmake | 2 +-
+ Source/cmake/WebKitMacros.cmake | 2 +-
+ Tools/DumpRenderTree/TestNetscapePlugIn/CMakeLists.txt | 2 +-
+ Tools/ImageDiff/CMakeLists.txt | 2 +-
+ Tools/MiniBrowser/gtk/CMakeLists.txt | 2 +-
+ Tools/TestWebKitAPI/PlatformGTK.cmake | 2 +-
+ Tools/TestWebKitAPI/Tests/WebKit2Gtk/CMakeLists.txt | 2 +-
+ Tools/WebKitTestRunner/CMakeLists.txt | 2 +-
+ 10 files changed, 12 insertions(+), 12 deletions(-)
+
+diff --git a/Source/JavaScriptCore/shell/CMakeLists.txt b/Source/JavaScriptCore/shell/CMakeLists.txt
+index 155c797..80fe22b 100644
+--- a/Source/JavaScriptCore/shell/CMakeLists.txt
++++ b/Source/JavaScriptCore/shell/CMakeLists.txt
+@@ -20,7 +20,7 @@ WEBKIT_INCLUDE_CONFIG_FILES_IF_EXISTS()
+
+ WEBKIT_WRAP_SOURCELIST(${JSC_SOURCES})
+ include_directories(./ ${JavaScriptCore_INCLUDE_DIRECTORIES})
+-include_directories(SYSTEM ${JavaScriptCore_SYSTEM_INCLUDE_DIRECTORIES})
++include_directories(${JavaScriptCore_SYSTEM_INCLUDE_DIRECTORIES})
+ add_executable(jsc ${JSC_SOURCES})
+ target_link_libraries(jsc ${JSC_LIBRARIES})
+
+diff --git a/Source/WebCore/PlatformGTK.cmake b/Source/WebCore/PlatformGTK.cmake
+index 567bd79..1fabea8 100644
+--- a/Source/WebCore/PlatformGTK.cmake
++++ b/Source/WebCore/PlatformGTK.cmake
+@@ -340,7 +340,7 @@ if (ENABLE_PLUGIN_PROCESS_GTK2)
+ ${GTK2_INCLUDE_DIRS}
+ ${GDK2_INCLUDE_DIRS}
+ )
+- target_include_directories(WebCorePlatformGTK2 SYSTEM PRIVATE
++ target_include_directories(WebCorePlatformGTK2 PRIVATE
+ ${WebCore_SYSTEM_INCLUDE_DIRECTORIES}
+ )
+ target_link_libraries(WebCorePlatformGTK2
+@@ -365,7 +365,7 @@ WEBKIT_SET_EXTRA_COMPILER_FLAGS(WebCorePlatformGTK)
+ target_include_directories(WebCorePlatformGTK PRIVATE
+ ${WebCore_INCLUDE_DIRECTORIES}
+ )
+-target_include_directories(WebCorePlatformGTK SYSTEM PRIVATE
++target_include_directories(WebCorePlatformGTK PRIVATE
+ ${WebCore_SYSTEM_INCLUDE_DIRECTORIES}
+ ${GTK_INCLUDE_DIRS}
+ ${GDK_INCLUDE_DIRS}
+@@ -383,7 +383,7 @@ include_directories(
+ "${DERIVED_SOURCES_GOBJECT_DOM_BINDINGS_DIR}"
+ )
+
+-include_directories(SYSTEM
++include_directories(
+ ${WebCore_SYSTEM_INCLUDE_DIRECTORIES}
+ )
+
+diff --git a/Source/WebKit2/PlatformGTK.cmake b/Source/WebKit2/PlatformGTK.cmake
+index e4805a4..c57df5d 100644
+--- a/Source/WebKit2/PlatformGTK.cmake
++++ b/Source/WebKit2/PlatformGTK.cmake
+@@ -822,7 +822,7 @@ if (ENABLE_PLUGIN_PROCESS_GTK2)
+ target_include_directories(WebKitPluginProcess2 PRIVATE
+ ${WebKit2CommonIncludeDirectories}
+ )
+- target_include_directories(WebKitPluginProcess2 SYSTEM PRIVATE
++ target_include_directories(WebKitPluginProcess2 PRIVATE
+ ${WebKit2CommonSystemIncludeDirectories}
+ ${GTK2_INCLUDE_DIRS}
+ ${GDK2_INCLUDE_DIRS}
+diff --git a/Source/cmake/WebKitMacros.cmake b/Source/cmake/WebKitMacros.cmake
+index 043e78e..8b3b642 100644
+--- a/Source/cmake/WebKitMacros.cmake
++++ b/Source/cmake/WebKitMacros.cmake
+@@ -224,7 +224,7 @@ endmacro()
+
+ macro(WEBKIT_FRAMEWORK _target)
+ include_directories(${${_target}_INCLUDE_DIRECTORIES})
+- include_directories(SYSTEM ${${_target}_SYSTEM_INCLUDE_DIRECTORIES})
++ include_directories(${${_target}_SYSTEM_INCLUDE_DIRECTORIES})
+ add_library(${_target} ${${_target}_LIBRARY_TYPE}
+ ${${_target}_HEADERS}
+ ${${_target}_SOURCES}
+diff --git a/Tools/DumpRenderTree/TestNetscapePlugIn/CMakeLists.txt b/Tools/DumpRenderTree/TestNetscapePlugIn/CMakeLists.txt
+index c431667..6dff244 100644
+--- a/Tools/DumpRenderTree/TestNetscapePlugIn/CMakeLists.txt
++++ b/Tools/DumpRenderTree/TestNetscapePlugIn/CMakeLists.txt
+@@ -42,7 +42,7 @@ set(WebKitTestNetscapePlugin_SYSTEM_INCLUDE_DIRECTORIES
+ )
+
+ include_directories(${WebKitTestNetscapePlugin_INCLUDE_DIRECTORIES})
+-include_directories(SYSTEM ${WebKitTestNetscapePlugin_SYSTEM_INCLUDE_DIRECTORIES})
++include_directories(${WebKitTestNetscapePlugin_SYSTEM_INCLUDE_DIRECTORIES})
+
+ set(WebKitTestNetscapePlugin_LIBRARIES
+ ${X11_LIBRARIES}
+diff --git a/Tools/ImageDiff/CMakeLists.txt b/Tools/ImageDiff/CMakeLists.txt
+index b15443f..87e03bf 100644
+--- a/Tools/ImageDiff/CMakeLists.txt
++++ b/Tools/ImageDiff/CMakeLists.txt
+@@ -14,6 +14,6 @@ set(IMAGE_DIFF_LIBRARIES
+ WEBKIT_INCLUDE_CONFIG_FILES_IF_EXISTS()
+
+ include_directories(${IMAGE_DIFF_INCLUDE_DIRECTORIES})
+-include_directories(SYSTEM ${IMAGE_DIFF_SYSTEM_INCLUDE_DIRECTORIES})
++include_directories(${IMAGE_DIFF_SYSTEM_INCLUDE_DIRECTORIES})
+ add_executable(ImageDiff ${IMAGE_DIFF_SOURCES})
+ target_link_libraries(ImageDiff ${IMAGE_DIFF_LIBRARIES})
+diff --git a/Tools/MiniBrowser/gtk/CMakeLists.txt b/Tools/MiniBrowser/gtk/CMakeLists.txt
+index 0704bc6..619e5a5 100644
+--- a/Tools/MiniBrowser/gtk/CMakeLists.txt
++++ b/Tools/MiniBrowser/gtk/CMakeLists.txt
+@@ -58,7 +58,7 @@ endif ()
+ add_definitions(-DGDK_VERSION_MIN_REQUIRED=GDK_VERSION_3_6)
+
+ include_directories(${MiniBrowser_INCLUDE_DIRECTORIES})
+-include_directories(SYSTEM ${MiniBrowser_SYSTEM_INCLUDE_DIRECTORIES})
++include_directories(${MiniBrowser_SYSTEM_INCLUDE_DIRECTORIES})
+ add_executable(MiniBrowser ${MiniBrowser_SOURCES})
+ target_link_libraries(MiniBrowser ${MiniBrowser_LIBRARIES})
+
+diff --git a/Tools/TestWebKitAPI/PlatformGTK.cmake b/Tools/TestWebKitAPI/PlatformGTK.cmake
+index 7552cc2..2eb44d5 100644
+--- a/Tools/TestWebKitAPI/PlatformGTK.cmake
++++ b/Tools/TestWebKitAPI/PlatformGTK.cmake
+@@ -20,7 +20,7 @@ include_directories(
+ ${WEBKIT2_DIR}/UIProcess/API/gtk
+ )
+
+-include_directories(SYSTEM
++include_directories(
+ ${GDK3_INCLUDE_DIRS}
+ ${GLIB_INCLUDE_DIRS}
+ ${GTK3_INCLUDE_DIRS}
+diff --git a/Tools/TestWebKitAPI/Tests/WebKit2Gtk/CMakeLists.txt b/Tools/TestWebKitAPI/Tests/WebKit2Gtk/CMakeLists.txt
+index b0b4739..434e4ca 100644
+--- a/Tools/TestWebKitAPI/Tests/WebKit2Gtk/CMakeLists.txt
++++ b/Tools/TestWebKitAPI/Tests/WebKit2Gtk/CMakeLists.txt
+@@ -23,7 +23,7 @@ include_directories(
+ ${TOOLS_DIR}/TestWebKitAPI/gtk/WebKit2Gtk
+ )
+
+-include_directories(SYSTEM
++include_directories(
+ ${ATSPI_INCLUDE_DIRS}
+ ${GLIB_INCLUDE_DIRS}
+ ${GSTREAMER_INCLUDE_DIRS}
+diff --git a/Tools/WebKitTestRunner/CMakeLists.txt b/Tools/WebKitTestRunner/CMakeLists.txt
+index 7db90f2..a4f917f 100644
+--- a/Tools/WebKitTestRunner/CMakeLists.txt
++++ b/Tools/WebKitTestRunner/CMakeLists.txt
+@@ -116,7 +116,7 @@ GENERATE_BINDINGS(WebKitTestRunner_SOURCES
+ WEBKIT_INCLUDE_CONFIG_FILES_IF_EXISTS()
+
+ include_directories(${WebKitTestRunner_INCLUDE_DIRECTORIES})
+-include_directories(SYSTEM ${WebKitTestRunner_SYSTEM_INCLUDE_DIRECTORIES})
++include_directories(${WebKitTestRunner_SYSTEM_INCLUDE_DIRECTORIES})
+
+ add_library(TestRunnerInjectedBundle SHARED ${WebKitTestRunnerInjectedBundle_SOURCES})
+ target_link_libraries(TestRunnerInjectedBundle ${WebKitTestRunner_LIBRARIES})
+--
+2.9.3
+
diff --git a/meta/recipes-sato/webkit/webkitgtk_2.14.2.bb b/meta/recipes-sato/webkit/webkitgtk_2.14.2.bb
index 280765c..1c327ba 100644
--- a/meta/recipes-sato/webkit/webkitgtk_2.14.2.bb
+++ b/meta/recipes-sato/webkit/webkitgtk_2.14.2.bb
@@ -18,6 +18,7 @@ SRC_URI = "http://www.webkitgtk.org/releases/${BPN}-${PV}.tar.xz \
file://ppc-musl-fix.patch \
file://0001-Fix-racy-parallel-build-of-WebKit2-4.0.gir.patch \
file://0001-Tweak-gtkdoc-settings-so-that-gtkdoc-generation-work.patch \
+ file://0001-WebKitMacros-Append-to-I-and-not-to-isystem.patch \
"
SRC_URI[md5sum] = "2fe3cadbc546d93ca68a13756c2be015"
--
2.1.4
^ permalink raw reply related
* Re: [PATCH] webkitgtk: drop patch 0001-WebKitMacros-Append-to-I-and-not-to-isystem.patch
From: Khem Raj @ 2016-12-07 19:47 UTC (permalink / raw)
To: Carlos Alberto Lopez Perez
Cc: Patches and discussions about the oe-core layer
In-Reply-To: <33b8cb66-7285-cfbb-6424-f86b319bf24b@igalia.com>
On Wed, Dec 7, 2016 at 11:46 AM, Carlos Alberto Lopez Perez
<clopez@igalia.com> wrote:
> On 05/12/16 23:56, Khem Raj wrote:
>> Carlos
>>
>> webkitgtk fails now e.g. see
>>
>> http://errors.yoctoproject.org/Errors/Details/111221/ <http://errors.yoctoproject.org/Errors/Details/111221/>
>>
>> fatal error: 'stdlib.h' file not found
>> #include_next <stdlib.h>
>>
>> This is when building with clang, I think its too early to drop this patch.
>
> Hi,
>
> I thought this was an issue only with GCC ...
>
> Some questions:
> * Which version of clang is used ?
> * The clang toolchain used is the one from the
> https://github.com/kraj/meta-clang layer ?
>
yes
>
> I don't have a fix for it at this moment, and investigating it would
> require time. So I think is fine to reintroduce the patch
> 0001-WebKitMacros-Append-to-I-and-not-to-isystem.patch (If possible,
> please change the commit message on the patch to tell that this an issue
> now only when building with clang).
Please send a revert then
>
>
> Thanks
>
^ permalink raw reply
* Re: [PATCH] gstreamer1.0-plugins-bad: Add PKG_CONFIG_SYSROOT_DIR to output of pkg-config
From: Khem Raj @ 2016-12-07 19:46 UTC (permalink / raw)
To: Jussi Kukkonen; +Cc: Patches and discussions about the oe-core layer
In-Reply-To: <CAHiDW_G52CHLn9A9Hk8Z-m=LvmA5yzUbHjC-Rx9B9oQeZZW4qQ@mail.gmail.com>
On Wed, Dec 7, 2016 at 4:25 AM, Jussi Kukkonen <jussi.kukkonen@intel.com> wrote:
>
>
> On 5 December 2016 at 21:08, Khem Raj <raj.khem@gmail.com> wrote:
>>
>>
>> On Dec 5, 2016, at 5:20 AM, Jussi Kukkonen <jussi.kukkonen@intel.com>
>> wrote:
>>
>>
>>
>> On 1 December 2016 at 10:37, Khem Raj <raj.khem@gmail.com> wrote:
>>>
>>> When configure pokes for wayland-protocols isntallations it ended up
>>> using the ones from host, which is because it did not account for sysroot
>>> prefix
>>>
>>> Signed-off-by: Khem Raj <raj.khem@gmail.com>
>>> ---
>>> .../gstreamer/gstreamer1.0-plugins-bad.inc | 2 +-
>>> ...G_CONFIG_SYSROOT_DIR-to-pkg-config-output.patch | 37
>>> ++++++++++++++++++++++
>>> .../gstreamer/gstreamer1.0-plugins-bad_1.10.1.bb | 1 +
>>> 3 files changed, 39 insertions(+), 1 deletion(-)
>>> create mode 100644
>>> meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0001-Prepend-PKG_CONFIG_SYSROOT_DIR-to-pkg-config-output.patch
>>>
>>> diff --git
>>> a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad.inc
>>> b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad.inc
>>> index d26a6a9..d3c5326 100644
>>> --- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad.inc
>>> +++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad.inc
>>> @@ -64,7 +64,7 @@ PACKAGECONFIG[srtp] =
>>> "--enable-srtp,--disable-srtp,libsrtp"
>>> PACKAGECONFIG[uvch264] =
>>> "--enable-uvch264,--disable-uvch264,libusb1 libgudev"
>>> PACKAGECONFIG[voaacenc] =
>>> "--enable-voaacenc,--disable-voaacenc,vo-aacenc"
>>> PACKAGECONFIG[voamrwbenc] =
>>> "--enable-voamrwbenc,--disable-voamrwbenc,vo-amrwbenc"
>>> -PACKAGECONFIG[wayland] =
>>> "--enable-wayland,--disable-wayland,wayland-native wayland"
>>> +PACKAGECONFIG[wayland] =
>>> "--enable-wayland,--disable-wayland,wayland-native wayland
>>> wayland-protocols"
>>> PACKAGECONFIG[webp] = "--enable-webp,--disable-webp,libwebp"
>>>
>>> # these plugins have not been ported to 1.0 (yet):
>>> diff --git
>>> a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0001-Prepend-PKG_CONFIG_SYSROOT_DIR-to-pkg-config-output.patch
>>> b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0001-Prepend-PKG_CONFIG_SYSROOT_DIR-to-pkg-config-output.patch
>>> new file mode 100644
>>> index 0000000..eb789df
>>> --- /dev/null
>>> +++
>>> b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0001-Prepend-PKG_CONFIG_SYSROOT_DIR-to-pkg-config-output.patch
>>> @@ -0,0 +1,37 @@
>>> +From c271503d7e233428ac0323c51d6517113e26bef7 Mon Sep 17 00:00:00 2001
>>> +From: Khem Raj <raj.khem@gmail.com>
>>> +Date: Thu, 1 Dec 2016 00:27:13 -0800
>>> +Subject: [PATCH] Prepend PKG_CONFIG_SYSROOT_DIR to pkg-config output
>>> +
>>> +In cross environment we have to prepend the sysroot to the path found by
>>> +pkgconfig since the path returned from pkgconfig does not have sysroot
>>> prefixed
>>> +it ends up using the files from host system. If build host has wayland
>>> installed
>>> +the build will succeed but if you dont have wayland-protocols installed
>>> on build host then
>>> +it wont find the files on build host
>>> +
>>> +This should work ok with non sysrooted builds too since in those cases
>>> PKG_CONFIG_SYSROOT_DIR
>>> +will be empty
>>> +
>>> +Upstream-Status: Pending
>>> +
>>> +Signed-off-by: Khem Raj <raj.khem@gmail.com>
>>> +---
>>> + configure.ac | 2 +-
>>> + 1 file changed, 1 insertion(+), 1 deletion(-)
>>> +
>>> +diff --git a/configure.ac b/configure.ac
>>> +index f8ac96b..dc87b08 100644
>>> +--- a/configure.ac
>>> ++++ b/configure.ac
>>> +@@ -2233,7 +2233,7 @@ AG_GST_CHECK_FEATURE(WAYLAND, [wayland sink],
>>> wayland , [
>>> + PKG_CHECK_MODULES(WAYLAND_PROTOCOLS, wayland-protocols >= 1.4, [
>>> + if test "x$wayland_scanner" != "x"; then
>>> + HAVE_WAYLAND="yes"
>>> +- AC_SUBST(WAYLAND_PROTOCOLS_DATADIR, `$PKG_CONFIG
>>> --variable=pkgdatadir wayland-protocols`)
>>> ++ AC_SUBST(WAYLAND_PROTOCOLS_DATADIR,
>>> ${PKG_CONFIG_SYSROOT_DIR}`$PKG_CONFIG --variable=pkgdatadir
>>> wayland-protocols`)
>>
>>
>> I believe this breaks multilib because wayland-protocols is allarch so
>> PKG_CONFIG_SYSROOT_DIR may then be incorrect.
>>
>>
>> Realized my last reply is too terse. expanding on it a bit more.
>> this is not really the case. Since PKG_CONFIG_SYSROOT_DIR is set to
>> STAGING_DIR_HOST which is then set to ${STAGING_DIR}/${MACHINE} so
>> effectively its just
>> reusing an existing var compared to your case where a new redundant
>> variable is introduced.
>>
>> having said that, there could be a multilib issue if pkgdatadir thats in
>> .pc file from wayland-protocols package is using variables like ${libdir}
>> which then are
>> multilib dependent, if this is the case then we need to fix .pc file.
>> however when I look at the
>> <target-sysroot>/usr/share/pkgconfig/wayland-protocols.pc it has
>>
>> prefix=/usr
>> datarootdir=${prefix}/share
>> pkgdatadir=/usr/share/wayland-protocols
>>
>> seems to be free of multilib deps.
>>
>
> It's not about the .pc file, the issue is that if you build
> lib32-gstreamer1.0-plugins-bad WAYLAND_PROTOCOLS_DATADIR will end up as (as
> an example)
> $TMPDIR/sysroots/lib32-qemux86-64/usr/share/wayland-protocols/
> but that's not where wayland protocols are. The actual path is
> $TMPDIR/sysroots/qemux86-64/usr/share/wayland-protocols/
>
> Maybe there should be a generic multilib fix for this but I can't figure out
> what it would be...
hmmm is seems to me that 'all' packages should be common between
all syroots, may be symlinks into multilib sysroots for all files could be
a solution, of just staging function for 'all' packages should install it in
both sysroots.
>
> Jussi
>
>
^ permalink raw reply
* Re: [PATCH] webkitgtk: drop patch 0001-WebKitMacros-Append-to-I-and-not-to-isystem.patch
From: Carlos Alberto Lopez Perez @ 2016-12-07 19:46 UTC (permalink / raw)
To: Khem Raj; +Cc: openembedded-core
In-Reply-To: <D1A701D4-6205-498C-9A83-C65F7542F91E@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 890 bytes --]
On 05/12/16 23:56, Khem Raj wrote:
> Carlos
>
> webkitgtk fails now e.g. see
>
> http://errors.yoctoproject.org/Errors/Details/111221/ <http://errors.yoctoproject.org/Errors/Details/111221/>
>
> fatal error: 'stdlib.h' file not found
> #include_next <stdlib.h>
>
> This is when building with clang, I think its too early to drop this patch.
Hi,
I thought this was an issue only with GCC ...
Some questions:
* Which version of clang is used ?
* The clang toolchain used is the one from the
https://github.com/kraj/meta-clang layer ?
I don't have a fix for it at this moment, and investigating it would
require time. So I think is fine to reintroduce the patch
0001-WebKitMacros-Append-to-I-and-not-to-isystem.patch (If possible,
please change the commit message on the patch to tell that this an issue
now only when building with clang).
Thanks
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]
^ permalink raw reply
* Re: [PATCH] liburi-perl: Uprev from 1.60 to 1.71 to pickup bugfixes and compatibilty
From: Jason Wessel @ 2016-12-07 18:32 UTC (permalink / raw)
To: Openembedded-core
In-Reply-To: <20161207183100.6260-1-jason.wessel@windriver.com>
Looks like someone beat me to it, but it had not been merged yet, so this can be ignored.
Cheers,
Jason.
On 12/07/2016 12:31 PM, Jason Wessel wrote:
> The exo recipe from meta-oe no longer compiles and some parts of
> URI::Escape no longer work due to changes in perl 5. The main reason
> for the uprev is to pickup the fix for this problem:
>
> ERROR: \C is in regex; marked by <-- HERE in m/(\C <-- HERE )/ at /perl5/URI/Escape.pm line 205.
>
> The perl 5.24 transitioned the \C regex from deprecated into a hard
> error.
>
> Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
> ---
> .../perl/{liburi-perl_1.60.bb => liburi-perl_1.71.bb} | 8 ++++----
> 1 file changed, 4 insertions(+), 4 deletions(-)
> rename meta/recipes-devtools/perl/{liburi-perl_1.60.bb => liburi-perl_1.71.bb} (66%)
>
> diff --git a/meta/recipes-devtools/perl/liburi-perl_1.60.bb b/meta/recipes-devtools/perl/liburi-perl_1.71.bb
> similarity index 66%
> rename from meta/recipes-devtools/perl/liburi-perl_1.60.bb
> rename to meta/recipes-devtools/perl/liburi-perl_1.71.bb
> index 8809a44..1806d93 100644
> --- a/meta/recipes-devtools/perl/liburi-perl_1.60.bb
> +++ b/meta/recipes-devtools/perl/liburi-perl_1.71.bb
> @@ -6,14 +6,14 @@ and manipulate the various components that make up these strings."
> SECTION = "libs"
> LICENSE = "Artistic-1.0 | GPL-1.0+"
>
> -LIC_FILES_CHKSUM = "file://README;beginline=26;endline=30;md5=6c33ae5c87fd1c4897714e122dd9c23d"
> +LIC_FILES_CHKSUM = "file://LICENSE;md5=c453e94fae672800f83bc1bd7a38b53f"
>
> DEPENDS += "perl"
>
> -SRC_URI = "http://www.cpan.org/authors/id/G/GA/GAAS/URI-${PV}.tar.gz"
> +SRC_URI = "http://www.cpan.org/authors/id/E/ET/ETHER/URI-${PV}.tar.gz"
>
> -SRC_URI[md5sum] = "70f739be8ce28b8baba7c5920ffee4dc"
> -SRC_URI[sha256sum] = "1f92d3dc64acb8845e9917c945e22b9a5275aeb9ff924eb7873c3b7a5c0d2377"
> +SRC_URI[md5sum] = "247c3da29a794f72730e01aa5a715daf"
> +SRC_URI[sha256sum] = "9c8eca0d7f39e74bbc14706293e653b699238eeb1a7690cc9c136fb8c2644115"
>
> S = "${WORKDIR}/URI-${PV}"
>
^ permalink raw reply
* [PATCH] liburi-perl: Uprev from 1.60 to 1.71 to pickup bugfixes and compatibilty
From: Jason Wessel @ 2016-12-07 18:31 UTC (permalink / raw)
To: Openembedded-core
The exo recipe from meta-oe no longer compiles and some parts of
URI::Escape no longer work due to changes in perl 5. The main reason
for the uprev is to pickup the fix for this problem:
ERROR: \C is in regex; marked by <-- HERE in m/(\C <-- HERE )/ at /perl5/URI/Escape.pm line 205.
The perl 5.24 transitioned the \C regex from deprecated into a hard
error.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
.../perl/{liburi-perl_1.60.bb => liburi-perl_1.71.bb} | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
rename meta/recipes-devtools/perl/{liburi-perl_1.60.bb => liburi-perl_1.71.bb} (66%)
diff --git a/meta/recipes-devtools/perl/liburi-perl_1.60.bb b/meta/recipes-devtools/perl/liburi-perl_1.71.bb
similarity index 66%
rename from meta/recipes-devtools/perl/liburi-perl_1.60.bb
rename to meta/recipes-devtools/perl/liburi-perl_1.71.bb
index 8809a44..1806d93 100644
--- a/meta/recipes-devtools/perl/liburi-perl_1.60.bb
+++ b/meta/recipes-devtools/perl/liburi-perl_1.71.bb
@@ -6,14 +6,14 @@ and manipulate the various components that make up these strings."
SECTION = "libs"
LICENSE = "Artistic-1.0 | GPL-1.0+"
-LIC_FILES_CHKSUM = "file://README;beginline=26;endline=30;md5=6c33ae5c87fd1c4897714e122dd9c23d"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=c453e94fae672800f83bc1bd7a38b53f"
DEPENDS += "perl"
-SRC_URI = "http://www.cpan.org/authors/id/G/GA/GAAS/URI-${PV}.tar.gz"
+SRC_URI = "http://www.cpan.org/authors/id/E/ET/ETHER/URI-${PV}.tar.gz"
-SRC_URI[md5sum] = "70f739be8ce28b8baba7c5920ffee4dc"
-SRC_URI[sha256sum] = "1f92d3dc64acb8845e9917c945e22b9a5275aeb9ff924eb7873c3b7a5c0d2377"
+SRC_URI[md5sum] = "247c3da29a794f72730e01aa5a715daf"
+SRC_URI[sha256sum] = "9c8eca0d7f39e74bbc14706293e653b699238eeb1a7690cc9c136fb8c2644115"
S = "${WORKDIR}/URI-${PV}"
--
2.10.1
^ permalink raw reply related
* Re: [PATCH 3/4] kern-tools: fix processing for no branch meta-data
From: Trevor Woerner @ 2016-12-07 18:05 UTC (permalink / raw)
To: Bruce Ashfield; +Cc: Patches and discussions about the oe-core layer
In-Reply-To: <78eef00e-4156-bdbf-cda1-688ae0495414@windriver.com>
On Wed, Dec 7, 2016 at 11:18 AM, Bruce Ashfield
<bruce.ashfield@windriver.com> wrote:
> With the attached patch, I see nothing else that is named in /tmp/
>
> If you have the cycles, can you give it a try and let me know ?
Yes, I'm giving it a whirl right now. Thanks!
^ permalink raw reply
* Re: [PATCH V2 5/6] runqemu: fixes for slirp, network device and hostfwd
From: Randy Witt @ 2016-12-07 17:58 UTC (permalink / raw)
To: Robert Yang, openembedded-core
In-Reply-To: <2ecbdf8cca844fb9b76df48f80b0762811dd1b04.1481014270.git.liezhi.yang@windriver.com>
> def setup_slirp(self):
> """Setup user networking"""
>
> if self.fstype == 'nfs':
> self.setup_nfs()
> self.kernel_cmdline_script += ' ip=dhcp'
> - self.set('NETWORK_CMD', self.get('QB_SLIRP_OPT'))
> + # Port mapping
> + hostfwd = ",hostfwd=tcp::2222-:22,hostfwd=tcp::2323-:23"
> + qb_slirp_opt_default = "-netdev user,id=net0%s" % hostfwd
> + qb_slirp_opt = self.get('QB_SLIRP_OPT') or qb_slirp_opt_default
> + # Figure out the port
> + ports = re.findall('hostfwd=[^-]*:([0-9]+)-[^,-]*', qb_slirp_opt)
> + ports = [int(i) for i in ports]
> + mac = 2
> + # Find a free port to avoid conflicts
> + for p in ports[:]:
> + p_new = p
> + while not check_free_port('localhost', p_new):
> + p_new += 1
> + mac += 1
> + while p_new in ports:
> + p_new += 1
> + mac += 1
> + if p != p_new:
> + ports.append(p_new)
> + qb_slirp_opt = re.sub(':%s-' % p, ':%s-' % p_new, qb_slirp_opt)
> + logger.info("Port forward changed: %s -> %s" % (p, p_new))
Regardless if the port is changed or not, so that things like tests have an
easier time of figuring out the port mappings, would it be good add a flag that
prints out the entire list of ports used, or always do it? i.e. "Port forwarding
2222:22 2333:23.... It's not necessary of course, you can always look at the
command line and then check to see if anything is remapped. But that's a bit
more work.
> + mac = "%s%02x" % (self.mac_slirp, mac)
> + self.set('NETWORK_CMD', '%s %s' % (self.network_device.replace('@MAC@', mac), qb_slirp_opt))
^ permalink raw reply
* Re: [PATCH 1/3] cve-check: allow recipes to override the product name
From: Mariano Lopez @ 2016-12-07 17:05 UTC (permalink / raw)
To: Ross Burton, openembedded-core
In-Reply-To: <1481129438-28306-1-git-send-email-ross.burton@intel.com>
On 07/12/16 10:50, Ross Burton wrote:
> Add a new variable CVE_PRODUCT for the product name to look up in the NVD
> database. Default this to BPN, but allow recipes such as tiff (which is libtiff
> in NVD) to override it.
>
> Signed-off-by: Ross Burton <ross.burton@intel.com>
>
I like the idea to be able to override the name that cve-check-tool
checks. The only drawback would be the burden of adding these to needed
recipes. This is still better to have to guess the correct name, or to
check PROVIDES or RPROVIDES, there are just too much corner cases. So
this solution has my approval.
^ permalink raw reply
* [PATCH 3/3] curl: set CVE_PRODUCT
From: Ross Burton @ 2016-12-07 16:50 UTC (permalink / raw)
To: openembedded-core
In-Reply-To: <1481129438-28306-1-git-send-email-ross.burton@intel.com>
This is 'libcurl' in NVD.
Signed-off-by: Ross Burton <ross.burton@intel.com>
---
meta/recipes-support/curl/curl_7.51.0.bb | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-support/curl/curl_7.51.0.bb b/meta/recipes-support/curl/curl_7.51.0.bb
index e1a996b..a9589b8 100644
--- a/meta/recipes-support/curl/curl_7.51.0.bb
+++ b/meta/recipes-support/curl/curl_7.51.0.bb
@@ -17,6 +17,7 @@ SRC_URI += " file://configure_ac.patch"
SRC_URI[md5sum] = "09a7c5769a7eae676d5e2c86d51f167e"
SRC_URI[sha256sum] = "7f8240048907e5030f67be0a6129bc4b333783b9cca1391026d700835a788dde"
+CVE_PRODUCT = "libcurl"
inherit autotools pkgconfig binconfig multilib_header
PACKAGECONFIG ??= "${@bb.utils.contains("DISTRO_FEATURES", "ipv6", "ipv6", "", d)} gnutls proxy zlib"
--
2.8.1
^ permalink raw reply related
* [PATCH 2/3] tiff: set CVE_PRODUCT
From: Ross Burton @ 2016-12-07 16:50 UTC (permalink / raw)
To: openembedded-core
In-Reply-To: <1481129438-28306-1-git-send-email-ross.burton@intel.com>
This is 'libtiff' in NVD.
Signed-off-by: Ross Burton <ross.burton@intel.com>
---
meta/recipes-multimedia/libtiff/tiff_4.0.6.bb | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-multimedia/libtiff/tiff_4.0.6.bb b/meta/recipes-multimedia/libtiff/tiff_4.0.6.bb
index 5fccde9..963d4b3 100644
--- a/meta/recipes-multimedia/libtiff/tiff_4.0.6.bb
+++ b/meta/recipes-multimedia/libtiff/tiff_4.0.6.bb
@@ -2,6 +2,8 @@ SUMMARY = "Provides support for the Tag Image File Format (TIFF)"
LICENSE = "BSD-2-Clause"
LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=34da3db46fab7501992f9615d7e158cf"
+CVE_PRODUCT = "libtiff"
+
SRC_URI = "http://download.osgeo.org/libtiff/tiff-${PV}.tar.gz \
file://libtool2.patch \
file://CVE-2015-8665_8683.patch \
--
2.8.1
^ permalink raw reply related
* [PATCH 1/3] cve-check: allow recipes to override the product name
From: Ross Burton @ 2016-12-07 16:50 UTC (permalink / raw)
To: openembedded-core
Add a new variable CVE_PRODUCT for the product name to look up in the NVD
database. Default this to BPN, but allow recipes such as tiff (which is libtiff
in NVD) to override it.
Signed-off-by: Ross Burton <ross.burton@intel.com>
---
meta/classes/cve-check.bbclass | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index b0febfb..75b8fa9 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -20,6 +20,10 @@
# the only method to check against CVEs. Running this tool
# doesn't guarantee your packages are free of CVEs.
+# The product name that the CVE database uses. Defaults to BPN, but may need to
+# be overriden per recipe (for example tiff.bb sets CVE_PRODUCT=libtiff).
+CVE_PRODUCT ?= "${BPN}"
+
CVE_CHECK_DB_DIR ?= "${DL_DIR}/CVE_CHECK"
CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvd.db"
@@ -144,7 +148,7 @@ def check_cves(d, patched_cves):
cves_patched = []
cves_unpatched = []
- bpn = d.getVar("BPN", True)
+ bpn = d.getVar("CVE_PRODUCT")
pv = d.getVar("PV", True).split("git+")[0]
cves = " ".join(patched_cves)
cve_db_dir = d.getVar("CVE_CHECK_DB_DIR", True)
--
2.8.1
^ permalink raw reply related
* Re: [PATCH V2 0/6] runqemu: fix for slirp, network device and hostfwd
From: Nathan Rossi @ 2016-12-07 16:50 UTC (permalink / raw)
To: Robert Yang; +Cc: openembedded-core
In-Reply-To: <b54c6e1b-dfb8-2753-74ff-ce42b65db95b@windriver.com>
On 6 December 2016 at 19:01, Robert Yang <liezhi.yang@windriver.com> wrote:
>
>
> On 12/06/2016 04:55 PM, Robert Yang wrote:
>>
>> * V2
>> - Add QB_NETWORK_DEVICE to set network device for both slirp and tap,
>> the idea is from Randy and Nathan.
>
>
> Add Randy and Nathan in the loop, I had added them in git send-email,
> but there were not in the CC list, look strange.
Hi Robert
I tested the cpio.gz and network device parts of this series with the
meta-xilinx qemu machines, they work well. Will implement the
QB_NETWORK_DEVICE use in meta-xilinx once this series gets applied.
Thanks,
Nathan
>
> // Robert
>
>
>> - Use different mac sections for slirp and tap to fix conflicts when
>> running both of them on the same host.
>>
>> * V1
>> - Initial version
>>
>> The following changes since commit
>> 11063a01d4511b2688ea7ba2d7359e4e07328c66:
>>
>> ruby: upgrade to 2.3.1 (2016-11-30 15:47:17 +0000)
>>
>> are available in the git repository at:
>>
>> git://git.openembedded.org/openembedded-core-contrib rbt/runqemu
>>
>> http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/runqemu
>>
>> Robert Yang (6):
>> scripts/runqemu: fix checking for <file>.cpio.gz
>> qemuboot.bbclass: use IMGDEPLOYDIR
>> runqemu-export-rootfs: fix inconsistent var names
>> runqemu: support mutiple qemus running when nfs
>> runqemu: fixes for slirp, network device and hostfwd
>> qemuboot.bbclass: add blank lines in comments
>>
>> meta/classes/qemuboot.bbclass | 34 ++++++--
>> meta/conf/machine/include/qemuboot-x86.inc | 1 -
>> meta/conf/machine/qemuarm64.conf | 1 -
>> scripts/runqemu | 133
>> ++++++++++++++++++++++-------
>> scripts/runqemu-export-rootfs | 21 ++---
>> 5 files changed, 140 insertions(+), 50 deletions(-)
>>
>
^ permalink raw reply
* Re: [PATCH 3/4] kern-tools: fix processing for no branch meta-data
From: Bruce Ashfield @ 2016-12-07 16:18 UTC (permalink / raw)
To: Trevor Woerner, Paul Barker
Cc: Patches and discussions about the oe-core layer
In-Reply-To: <CAHUNapQ847n7e=acLQ+hkLqziqaP8EQg5_0wrw5oJ9i9+Tw6tg@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 3683 bytes --]
On 2016-12-07 10:11 AM, Trevor Woerner wrote:
> This patch does fix the raspberrypi issue, and it has been pushed, thankfully.
>
> But I'm still seeing hard-coded filenames in /tmp that are messing up
> the ability for more than one person to use a given build machine.
> E.g.:
>
> /tmp/patch.standard.arm-versatile-926ejs.queue
> /tmp/patch.standard.mti-malta32.queue
> /tmp/patch.standard.mti-malta64.queue
> /tmp/patch.standard.preempt-rt.base.queue
> /tmp/patch.standard.preempt-rt.queue
> /tmp/patch.standard.queue
> /tmp/patch.unused.queue
>
> I think the malta ones are used for qemumips64, which is still failing
> for this reason (multiple users)
I tracked this down to some of the sub commands using the dirname
of the global patch queue .. and since that is in /tmp/, they are
following along.
With the attached patch, I see nothing else that is named in /tmp/
If you have the cycles, can you give it a try and let me know ?
Bruce
>
> On Tue, Dec 6, 2016 at 6:02 PM, Paul Barker <paul@paulbarker.me.uk> wrote:
>> On Tue, 06 Dec 2016 22:14:41 +0100
>> Patrick Ohly <patrick.ohly@intel.com> wrote:
>>
>>> On Fri, 2016-12-02 at 16:09 -0500, Bruce Ashfield wrote:
>>>> Lernel meta-data that has patches, but no branches, can trigger an
>>>> error due to no branch specific patch queue.
>>>>
>>>> This error then cascades to more issues since the tools are using
>>>> a named file in /tmp to store and display error messages to the
>>>> user.
>>>>
>>>> We fix both issues though the following kern tools tweaks:
>>>>
>>>> commit bd9e1d6c9b0a34ff3e19a06999aaf57ffadfd04c
>>>> Author: Bruce Ashfield <bruce.ashfield@windriver.com>
>>>> Date: Fri Dec 2 13:09:40 2016 -0500
>>>>
>>>> scc: use mktemp for consolidated output capture
>>>>
>>>> To provide useful error messages the tools dump pre-processed
>>>> files and messages to a temporary file. If multiple users are
>>>> doing builds, this means they either race, or can have permissions
>>>> issues.
>>>>
>>>> By creating the temporary file via mktemp, we avoid both issues.
>>>> (We also make sure to clean these up on exit, or /tmp will get
>>>> polluted quickly).
>>>>
>>>> commit a287da4bfe0b4acb8f2b0627bd8e7abd1a1dde26
>>>> Author: Bruce Ashfield <bruce.ashfield@windriver.com>
>>>> Date: Fri Dec 2 13:08:08 2016 -0500
>>>>
>>>> patch: do not assume a branch specific patch queue is needed
>>>>
>>>> When processing input files per-branch and global patch queues are
>>>> generated. If the meta-data has not created any branches in the
>>>> repo, no branch specific queue is required.
>>>>
>>>> The tools assumed that one is always valid, and hence would throw a
>>>> non-zero exit code and stop processing.
>>>>
>>>> By testing for a named per-branch queue, we avoid this issue.
>>>
>>> Ostro OS runs into the problem while trying to use current OE-core
>>> master:
>>>
>>> .../patch.cmd: line 29: : No such file or directory
>>>
>>> | ERROR: Function failed: do_kernel_metadata (log file is located ...)
>>>
>>> This commit here fixed it for me. I see that it is already in Ross' mut2
>>> branch, so hopefully that'll land in master soon.
>>>
>>
>> Ditto for meta-raspberrypi, the kernel doesn't build with current
>> oe-core master as discussed on the yocto@ list. This patch is needed
>> to fix things.
>>
>> Thanks,
>> Paul Barker
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
[-- Attachment #2: 0001-kern-tools-ensure-that-no-shared-directories-are-use.patch --]
[-- Type: text/x-patch, Size: 1287 bytes --]
From 707978ff47533c705eac42619988793f07a66b80 Mon Sep 17 00:00:00 2001
From: Bruce Ashfield <bruce.ashfield@windriver.com>
Date: Wed, 7 Dec 2016 11:14:34 -0500
Subject: [PATCH] kern-tools: ensure that no shared directories are used
We need to avoid using shared/common directories for any files that are
part of specific build, since permissions issues in multi user
environments will cause issues.
Integrating the following commit to solve the issue:
scc: move unused patch queue under output dir
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
index 0f8a786176f9..3ca6ca22ef20 100644
--- a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
+++ b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://git/tools/kgit;beginline=5;endline=9;md5=a6c2fa8aef1b
DEPENDS = "git-native"
-SRCREV = "bd9e1d6c9b0a34ff3e19a06999aaf57ffadfd04c"
+SRCREV = "626ceac135fa66277c2fa53197be33cc9d4d7614"
PR = "r12"
PV = "0.2+git${SRCPV}"
--
2.5.0
^ permalink raw reply related
* Re: [PATCH 00/33] Accumulated patches for deb packaging
From: Alexander Kanavin @ 2016-12-07 16:02 UTC (permalink / raw)
To: Andreas Oberritter, openembedded-core
In-Reply-To: <fb6aaabe-3c7a-7e56-4e82-f36dd288c366@opendreambox.org>
On 12/07/2016 05:30 PM, Andreas Oberritter wrote:
>>> These are most of my patches which accumulated since our distro
>>> switched from opkg to dpkg and apt two years ago. They were tested
>>> thoroughly on dora and krogoth and just a little bit on master.
>>
>> How about updating apt and dpkg to latest upstream releases at the same
>> time?
>
> I don't see an advantage over updating at a later point in time.
The advantage is that oe-core does not accumulate technical debt that way.
Specifically, it's better to do many incremental version updates than
few massive updates (or avoid updates altogether until the ongoing
maintenance becomes unbearable, in the worst case). There is less
changes in upstream code to deal with, less dependency fixing, and less
resistance from community. This is actually the preferred mode of
operation in oe-core. We are constantly fighting back against the
increase in debt, and you could really help us here a bit.
Regards,
Alex
^ permalink raw reply
* Re: [PATCH 00/33] Accumulated patches for deb packaging
From: Andreas Oberritter @ 2016-12-07 15:30 UTC (permalink / raw)
To: Alexander Kanavin, openembedded-core
In-Reply-To: <5106886a-56c8-79f3-bd7a-aee780fd590b@linux.intel.com>
Hi Alex,
On 07.12.2016 13:25, Alexander Kanavin wrote:
> On 12/06/2016 01:49 PM, Andreas Oberritter wrote:
>> These are most of my patches which accumulated since our distro
>> switched from opkg to dpkg and apt two years ago. They were tested
>> thoroughly on dora and krogoth and just a little bit on master.
>
> How about updating apt and dpkg to latest upstream releases at the same
> time?
I don't see an advantage over updating at a later point in time.
Regards,
Andreas
^ permalink raw reply
* Re: [PATCH 3/4] kern-tools: fix processing for no branch meta-data
From: Bruce Ashfield @ 2016-12-07 15:14 UTC (permalink / raw)
To: Trevor Woerner, Paul Barker
Cc: Patches and discussions about the oe-core layer
In-Reply-To: <CAHUNapQ847n7e=acLQ+hkLqziqaP8EQg5_0wrw5oJ9i9+Tw6tg@mail.gmail.com>
On 2016-12-07 10:11 AM, Trevor Woerner wrote:
> This patch does fix the raspberrypi issue, and it has been pushed, thankfully.
>
> But I'm still seeing hard-coded filenames in /tmp that are messing up
> the ability for more than one person to use a given build machine.
> E.g.:
>
> /tmp/patch.standard.arm-versatile-926ejs.queue
> /tmp/patch.standard.mti-malta32.queue
> /tmp/patch.standard.mti-malta64.queue
> /tmp/patch.standard.preempt-rt.base.queue
> /tmp/patch.standard.preempt-rt.queue
> /tmp/patch.standard.queue
> /tmp/patch.unused.queue
>
> I think the malta ones are used for qemumips64, which is still failing
> for this reason (multiple users)
Those are never supposed to go to /tmp/, they are targeted to ${S}
of the kernel build.
It must be a side effect of the change I just did where the branch
isn't defined.
I'll whip up another patch.
Bruce
>
> On Tue, Dec 6, 2016 at 6:02 PM, Paul Barker <paul@paulbarker.me.uk> wrote:
>> On Tue, 06 Dec 2016 22:14:41 +0100
>> Patrick Ohly <patrick.ohly@intel.com> wrote:
>>
>>> On Fri, 2016-12-02 at 16:09 -0500, Bruce Ashfield wrote:
>>>> Lernel meta-data that has patches, but no branches, can trigger an
>>>> error due to no branch specific patch queue.
>>>>
>>>> This error then cascades to more issues since the tools are using
>>>> a named file in /tmp to store and display error messages to the
>>>> user.
>>>>
>>>> We fix both issues though the following kern tools tweaks:
>>>>
>>>> commit bd9e1d6c9b0a34ff3e19a06999aaf57ffadfd04c
>>>> Author: Bruce Ashfield <bruce.ashfield@windriver.com>
>>>> Date: Fri Dec 2 13:09:40 2016 -0500
>>>>
>>>> scc: use mktemp for consolidated output capture
>>>>
>>>> To provide useful error messages the tools dump pre-processed
>>>> files and messages to a temporary file. If multiple users are
>>>> doing builds, this means they either race, or can have permissions
>>>> issues.
>>>>
>>>> By creating the temporary file via mktemp, we avoid both issues.
>>>> (We also make sure to clean these up on exit, or /tmp will get
>>>> polluted quickly).
>>>>
>>>> commit a287da4bfe0b4acb8f2b0627bd8e7abd1a1dde26
>>>> Author: Bruce Ashfield <bruce.ashfield@windriver.com>
>>>> Date: Fri Dec 2 13:08:08 2016 -0500
>>>>
>>>> patch: do not assume a branch specific patch queue is needed
>>>>
>>>> When processing input files per-branch and global patch queues are
>>>> generated. If the meta-data has not created any branches in the
>>>> repo, no branch specific queue is required.
>>>>
>>>> The tools assumed that one is always valid, and hence would throw a
>>>> non-zero exit code and stop processing.
>>>>
>>>> By testing for a named per-branch queue, we avoid this issue.
>>>
>>> Ostro OS runs into the problem while trying to use current OE-core
>>> master:
>>>
>>> .../patch.cmd: line 29: : No such file or directory
>>>
>>> | ERROR: Function failed: do_kernel_metadata (log file is located ...)
>>>
>>> This commit here fixed it for me. I see that it is already in Ross' mut2
>>> branch, so hopefully that'll land in master soon.
>>>
>>
>> Ditto for meta-raspberrypi, the kernel doesn't build with current
>> oe-core master as discussed on the yocto@ list. This patch is needed
>> to fix things.
>>
>> Thanks,
>> Paul Barker
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply
* Re: [PATCH 3/4] kern-tools: fix processing for no branch meta-data
From: Trevor Woerner @ 2016-12-07 15:11 UTC (permalink / raw)
To: Paul Barker
Cc: Bruce Ashfield, Patches and discussions about the oe-core layer
In-Reply-To: <20161206230222.5411d7b6@nuc.betafive.co.uk>
This patch does fix the raspberrypi issue, and it has been pushed, thankfully.
But I'm still seeing hard-coded filenames in /tmp that are messing up
the ability for more than one person to use a given build machine.
E.g.:
/tmp/patch.standard.arm-versatile-926ejs.queue
/tmp/patch.standard.mti-malta32.queue
/tmp/patch.standard.mti-malta64.queue
/tmp/patch.standard.preempt-rt.base.queue
/tmp/patch.standard.preempt-rt.queue
/tmp/patch.standard.queue
/tmp/patch.unused.queue
I think the malta ones are used for qemumips64, which is still failing
for this reason (multiple users)
On Tue, Dec 6, 2016 at 6:02 PM, Paul Barker <paul@paulbarker.me.uk> wrote:
> On Tue, 06 Dec 2016 22:14:41 +0100
> Patrick Ohly <patrick.ohly@intel.com> wrote:
>
>> On Fri, 2016-12-02 at 16:09 -0500, Bruce Ashfield wrote:
>> > Lernel meta-data that has patches, but no branches, can trigger an
>> > error due to no branch specific patch queue.
>> >
>> > This error then cascades to more issues since the tools are using
>> > a named file in /tmp to store and display error messages to the
>> > user.
>> >
>> > We fix both issues though the following kern tools tweaks:
>> >
>> > commit bd9e1d6c9b0a34ff3e19a06999aaf57ffadfd04c
>> > Author: Bruce Ashfield <bruce.ashfield@windriver.com>
>> > Date: Fri Dec 2 13:09:40 2016 -0500
>> >
>> > scc: use mktemp for consolidated output capture
>> >
>> > To provide useful error messages the tools dump pre-processed
>> > files and messages to a temporary file. If multiple users are
>> > doing builds, this means they either race, or can have permissions
>> > issues.
>> >
>> > By creating the temporary file via mktemp, we avoid both issues.
>> > (We also make sure to clean these up on exit, or /tmp will get
>> > polluted quickly).
>> >
>> > commit a287da4bfe0b4acb8f2b0627bd8e7abd1a1dde26
>> > Author: Bruce Ashfield <bruce.ashfield@windriver.com>
>> > Date: Fri Dec 2 13:08:08 2016 -0500
>> >
>> > patch: do not assume a branch specific patch queue is needed
>> >
>> > When processing input files per-branch and global patch queues are
>> > generated. If the meta-data has not created any branches in the
>> > repo, no branch specific queue is required.
>> >
>> > The tools assumed that one is always valid, and hence would throw a
>> > non-zero exit code and stop processing.
>> >
>> > By testing for a named per-branch queue, we avoid this issue.
>>
>> Ostro OS runs into the problem while trying to use current OE-core
>> master:
>>
>> .../patch.cmd: line 29: : No such file or directory
>>
>> | ERROR: Function failed: do_kernel_metadata (log file is located ...)
>>
>> This commit here fixed it for me. I see that it is already in Ross' mut2
>> branch, so hopefully that'll land in master soon.
>>
>
> Ditto for meta-raspberrypi, the kernel doesn't build with current
> oe-core master as discussed on the yocto@ list. This patch is needed
> to fix things.
>
> Thanks,
> Paul Barker
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply
* [PATCH] wic: Create a logical partition only when it is really mandatory
From: Alessio Igor Bogani @ 2016-12-07 14:00 UTC (permalink / raw)
To: openembedded-core; +Cc: Alessio Igor Bogani
Don't worth bother with logical partition on MBR partition type (aka
msdos) if disk image generated by wic should have 4 partitions.
Signed-off-by: Alessio Igor Bogani <alessio.bogani@elettra.eu>
---
scripts/lib/wic/utils/partitionedfs.py | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/scripts/lib/wic/utils/partitionedfs.py b/scripts/lib/wic/utils/partitionedfs.py
index cb03009..ccc9700 100644
--- a/scripts/lib/wic/utils/partitionedfs.py
+++ b/scripts/lib/wic/utils/partitionedfs.py
@@ -201,9 +201,10 @@ class Image():
part['num'] = 0
if disk['ptable_format'] == "msdos":
- if disk['realpart'] > 3:
- part['type'] = 'logical'
- part['num'] = disk['realpart'] + 1
+ if len(self.partitions) > 4:
+ if disk['realpart'] > 3:
+ part['type'] = 'logical'
+ part['num'] = disk['realpart'] + 1
disk['partitions'].append(num)
msger.debug("Assigned %s to %s%d, sectors range %d-%d size %d "
--
2.10.2
^ permalink raw reply related
* Re: what is the closest alternative to red hat's ABRT in OE?
From: Burton, Ross @ 2016-12-07 13:43 UTC (permalink / raw)
To: Robert P. J. Day; +Cc: OE Core mailing list
In-Reply-To: <alpine.LFD.2.20.1612070835310.2008@ca624034.mitel.com>
[-- Attachment #1: Type: text/plain, Size: 302 bytes --]
On 7 December 2016 at 13:37, Robert P. J. Day <rpjday@crashcourse.ca> wrote:
> and pretty sure i asked this before ... is there some reasonable
> equivalent to "sosreport" for OE?
>
For people who don't run Fedora, you might have better luck saying what
functionality you're after.
Ross
[-- Attachment #2: Type: text/html, Size: 758 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox