* [PATCH 01/14] oeqa/core/loader.py: support the 'auto' keyword
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
@ 2018-06-01 5:02 ` Chen Qi
2018-06-01 5:02 ` [PATCH 02/14] testimage.bbclass: fix behavior of empty TEST_SUITES Chen Qi
` (12 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:02 UTC (permalink / raw)
To: openembedded-core
In previous OEQA, having 'auto' in TEST_SUITES results in executing
as many test cases as possible.
This behaviour is broken for now. From the codes in core/loader.py,
I can see that it tries to use another keyword 'all'. But in fact,
it does not work.
I've checked the current manual. The manual says using 'auto'.
Below is the current information in manual.
"""
Alternatively, you can provide the "auto" option to have all applicable
tests run against the image.
TEST_SUITES_append = " auto"
"""
So we should restore this behaviour. This patch does so.
Also, output warning message is some module is named as 'auto', as this
is a reserved keyword.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/lib/oeqa/core/loader.py | 17 +++++++++++++----
1 file changed, 13 insertions(+), 4 deletions(-)
diff --git a/meta/lib/oeqa/core/loader.py b/meta/lib/oeqa/core/loader.py
index a4744de..98fc0f6 100644
--- a/meta/lib/oeqa/core/loader.py
+++ b/meta/lib/oeqa/core/loader.py
@@ -155,7 +155,16 @@ class OETestLoader(unittest.TestLoader):
class_name = case.__class__.__name__
test_name = case._testMethodName
- if self.modules:
+ # 'auto' is a reserved key word to run test cases automatically
+ # warn users if their test case belong to a module named 'auto'
+ if module_name_small == "auto":
+ bb.warn("'auto' is a reserved key word for TEST_SUITES. "
+ "But test case '%s' is detected to belong to auto module. "
+ "Please condier using a new name for your module." % str(case))
+
+ # check if case belongs to any specified module
+ # if 'auto' is specified, such check is skipped
+ if self.modules and not 'auto' in self.modules:
module = None
try:
module = self.modules[module_name_small]
@@ -245,7 +254,7 @@ class OETestLoader(unittest.TestLoader):
for tcName in testCaseNames:
case = self._getTestCase(testCaseClass, tcName)
# Filer by case id
- if not (self.tests and not 'all' in self.tests
+ if not (self.tests and not 'auto' in self.tests
and not getCaseID(case) in self.tests):
self._handleTestCaseDecorators(case)
@@ -309,14 +318,14 @@ class OETestLoader(unittest.TestLoader):
module_name = module.__name__
# Normal test modules are loaded if no modules were specified,
- # if module is in the specified module list or if 'all' is in
+ # if module is in the specified module list or if 'auto' is in
# module list.
# Underscore modules are loaded only if specified in module list.
load_module = True if not module_name.startswith('_') \
and (not self.modules \
or module_name in self.modules \
or module_name_small in self.modules \
- or 'all' in self.modules) \
+ or 'auto' in self.modules) \
else False
load_underscore = True if module_name.startswith('_') \
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH 02/14] testimage.bbclass: fix behavior of empty TEST_SUITES
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
2018-06-01 5:02 ` [PATCH 01/14] oeqa/core/loader.py: support the 'auto' keyword Chen Qi
@ 2018-06-01 5:02 ` Chen Qi
2018-06-01 5:03 ` [PATCH 03/14] oeqa/core/decorator/__init__.py: set metaclass to ABCMeta Chen Qi
` (11 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:02 UTC (permalink / raw)
To: openembedded-core
The current behaviour of TEST_SUITES is very confusing.
setting: TEST_SUITES = ""
result: Execute all test cases.
setting: TEST_SUITES = "some_case_not_exist"
result: Error out with 'Empty test suite' message.
The expected behaviour of TEST_SUITES should be:
1. when 'auto' is in it, execute as many test cases as possible
2. when no valid test case is specified, error out and give user message
The first one is implemented by a previous patch.
The second one is fixed in this patch.
Also add debug message to show test cases to be executed. This is
for easier debugging.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/classes/testimage.bbclass | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index bb688b0..d97da67 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -260,10 +260,16 @@ def testimage_main(d):
# Load tests before starting the target
test_paths = get_runtime_paths(d)
test_modules = d.getVar('TEST_SUITES').split()
+ if not test_modules:
+ bb.fatal('Empty test suite, please verify TEST_SUITES variable')
+
tc.loadTests(test_paths, modules=test_modules)
- if not getSuiteCases(tc.suites):
+ suitecases = getSuiteCases(tc.suites)
+ if not suitecases:
bb.fatal('Empty test suite, please verify TEST_SUITES variable')
+ else:
+ bb.debug(2, 'test suites:\n\t%s' % '\n\t'.join([str(c) for c in suitecases]))
package_extraction(d, tc.suites)
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH 03/14] oeqa/core/decorator/__init__.py: set metaclass to ABCMeta
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
2018-06-01 5:02 ` [PATCH 01/14] oeqa/core/loader.py: support the 'auto' keyword Chen Qi
2018-06-01 5:02 ` [PATCH 02/14] testimage.bbclass: fix behavior of empty TEST_SUITES Chen Qi
@ 2018-06-01 5:03 ` Chen Qi
2018-06-01 5:03 ` [PATCH 04/14] oeqa/core/decorator/__init__.py: use 'cls' instead of 'obj' Chen Qi
` (10 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:03 UTC (permalink / raw)
To: openembedded-core
OETestFilter is a subclass of OETestDecorator. It wants to make
use of @abstractmethod decorator. But such decorator requires
metaclass to be ABCMeta to have effect. So add it now to achieve
the designed behaviour.
Comments from python's manual:
"""
Using this decorator requires that the class's metaclass is ABCMeta
or is derived from it.
"""
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/lib/oeqa/core/decorator/__init__.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/lib/oeqa/core/decorator/__init__.py b/meta/lib/oeqa/core/decorator/__init__.py
index 855b6b9..6ca0acf 100644
--- a/meta/lib/oeqa/core/decorator/__init__.py
+++ b/meta/lib/oeqa/core/decorator/__init__.py
@@ -2,7 +2,7 @@
# Released under the MIT license (see COPYING.MIT)
from functools import wraps
-from abc import abstractmethod
+from abc import abstractmethod, ABCMeta
decoratorClasses = set()
@@ -10,7 +10,7 @@ def registerDecorator(obj):
decoratorClasses.add(obj)
return obj
-class OETestDecorator(object):
+class OETestDecorator(object, metaclass=ABCMeta):
case = None # Reference of OETestCase decorated
attrs = None # Attributes to be loaded by decorator implementation
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH 04/14] oeqa/core/decorator/__init__.py: use 'cls' instead of 'obj'
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
` (2 preceding siblings ...)
2018-06-01 5:03 ` [PATCH 03/14] oeqa/core/decorator/__init__.py: set metaclass to ABCMeta Chen Qi
@ 2018-06-01 5:03 ` Chen Qi
2018-06-01 5:03 ` [PATCH 05/14] oeqa/runtime/cases/dnf_runtime.py: skip test if PACKAGE_FEED_URIS is not set Chen Qi
` (9 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:03 UTC (permalink / raw)
To: openembedded-core
Use 'cls' instead of 'obj' to better reflect that registerDecorator
actually serves as a class decorator.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/lib/oeqa/core/decorator/__init__.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/meta/lib/oeqa/core/decorator/__init__.py b/meta/lib/oeqa/core/decorator/__init__.py
index 6ca0acf..14d7bfc 100644
--- a/meta/lib/oeqa/core/decorator/__init__.py
+++ b/meta/lib/oeqa/core/decorator/__init__.py
@@ -6,9 +6,9 @@ from abc import abstractmethod, ABCMeta
decoratorClasses = set()
-def registerDecorator(obj):
- decoratorClasses.add(obj)
- return obj
+def registerDecorator(cls):
+ decoratorClasses.add(cls)
+ return cls
class OETestDecorator(object, metaclass=ABCMeta):
case = None # Reference of OETestCase decorated
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH 05/14] oeqa/runtime/cases/dnf_runtime.py: skip test if PACKAGE_FEED_URIS is not set
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
` (3 preceding siblings ...)
2018-06-01 5:03 ` [PATCH 04/14] oeqa/core/decorator/__init__.py: use 'cls' instead of 'obj' Chen Qi
@ 2018-06-01 5:03 ` Chen Qi
2018-06-01 5:03 ` [PATCH 06/14] oeqa/runtime/cases/selftest.py: rename to _selftest.py Chen Qi
` (8 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:03 UTC (permalink / raw)
To: openembedded-core
This test is to test the behaviour of PACKAGE_FEED_URIS is correct or not.
If it's not even set, it makes no sense to do such test. So skip this
test if PACKAGE_FEED_URIS is not set.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py b/meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py
index 81c50ed..1aa7274 100644
--- a/meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py
+++ b/meta-selftest/lib/oeqa/runtime/cases/dnf_runtime.py
@@ -1,6 +1,7 @@
from oeqa.core.decorator.depends import OETestDepends
from oeqa.runtime.cases.dnf import DnfTest
from oeqa.utils.httpserver import HTTPService
+from oeqa.core.decorator.data import skipIfDataVar
class DnfSelftest(DnfTest):
@@ -18,6 +19,8 @@ class DnfSelftest(DnfTest):
cls.temp_dir.cleanup()
@OETestDepends(['dnf.DnfBasicTest.test_dnf_help'])
+ @skipIfDataVar('PACKAGE_FEED_URIS', None,
+ 'Not suitable as PACKAGE_FEED_URIS is not set')
def test_verify_package_feeds(self):
"""
Summary: Check correct setting of PACKAGE_FEED_URIS var
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH 06/14] oeqa/runtime/cases/selftest.py: rename to _selftest.py
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
` (4 preceding siblings ...)
2018-06-01 5:03 ` [PATCH 05/14] oeqa/runtime/cases/dnf_runtime.py: skip test if PACKAGE_FEED_URIS is not set Chen Qi
@ 2018-06-01 5:03 ` Chen Qi
2018-06-03 22:42 ` Richard Purdie
2018-06-01 5:03 ` [PATCH 07/14] testimage.bbclass: also check 'auto' to create rpm index Chen Qi
` (7 subsequent siblings)
13 siblings, 1 reply; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:03 UTC (permalink / raw)
To: openembedded-core
This test modules is designed to be invoked only by selftest. It's
not meant to be tested by normal runtime test. So it should be renamed
with '_' prefix, so that it will not be automatically loaded by normal
runtime tests when 'auto' is in TEST_SUITES.
The failure message is as below.
RESULTS - selftest.Selftest.test_install_package - Testcase -1: FAILED
Also, change selftest/cases/runtime_test.py to use '_selftest' accordingly.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta-selftest/lib/oeqa/runtime/cases/{selftest.py => _selftest.py} | 0
meta/lib/oeqa/selftest/cases/runtime_test.py | 2 +-
2 files changed, 1 insertion(+), 1 deletion(-)
rename meta-selftest/lib/oeqa/runtime/cases/{selftest.py => _selftest.py} (100%)
diff --git a/meta-selftest/lib/oeqa/runtime/cases/selftest.py b/meta-selftest/lib/oeqa/runtime/cases/_selftest.py
similarity index 100%
rename from meta-selftest/lib/oeqa/runtime/cases/selftest.py
rename to meta-selftest/lib/oeqa/runtime/cases/_selftest.py
diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py
index 9c9b4b3..def3819 100644
--- a/meta/lib/oeqa/selftest/cases/runtime_test.py
+++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
@@ -122,7 +122,7 @@ class TestImage(OESelftestTestCase):
self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
features = 'INHERIT += "testimage"\n'
- features += 'TEST_SUITES = "ping ssh selftest"\n'
+ features += 'TEST_SUITES = "ping ssh _selftest"\n'
self.write_config(features)
# Build core-image-sato and testimage
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* Re: [PATCH 06/14] oeqa/runtime/cases/selftest.py: rename to _selftest.py
2018-06-01 5:03 ` [PATCH 06/14] oeqa/runtime/cases/selftest.py: rename to _selftest.py Chen Qi
@ 2018-06-03 22:42 ` Richard Purdie
2018-06-04 3:06 ` ChenQi
0 siblings, 1 reply; 17+ messages in thread
From: Richard Purdie @ 2018-06-03 22:42 UTC (permalink / raw)
To: Chen Qi, openembedded-core
On Fri, 2018-06-01 at 13:03 +0800, Chen Qi wrote:
> This test modules is designed to be invoked only by selftest. It's
> not meant to be tested by normal runtime test. So it should be
> renamed
> with '_' prefix, so that it will not be automatically loaded by
> normal
> runtime tests when 'auto' is in TEST_SUITES.
>
> The failure message is as below.
>
> RESULTS - selftest.Selftest.test_install_package - Testcase -1:
> FAILED
>
> Also, change selftest/cases/runtime_test.py to use '_selftest'
> accordingly.
>
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
This seems to cause the second failure in
https://autobuilder.yocto.io/builders/nightly-oe-selftest/builds/1096/steps/Running%20oe-selftest/logs/stdio
Cheers,
Richard
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 06/14] oeqa/runtime/cases/selftest.py: rename to _selftest.py
2018-06-03 22:42 ` Richard Purdie
@ 2018-06-04 3:06 ` ChenQi
0 siblings, 0 replies; 17+ messages in thread
From: ChenQi @ 2018-06-04 3:06 UTC (permalink / raw)
To: Richard Purdie, openembedded-core
On 06/04/2018 06:42 AM, Richard Purdie wrote:
> On Fri, 2018-06-01 at 13:03 +0800, Chen Qi wrote:
>> This test modules is designed to be invoked only by selftest. It's
>> not meant to be tested by normal runtime test. So it should be
>> renamed
>> with '_' prefix, so that it will not be automatically loaded by
>> normal
>> runtime tests when 'auto' is in TEST_SUITES.
>>
>> The failure message is as below.
>>
>> RESULTS - selftest.Selftest.test_install_package - Testcase -1:
>> FAILED
>>
>> Also, change selftest/cases/runtime_test.py to use '_selftest'
>> accordingly.
>>
>> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> This seems to cause the second failure in
>
> https://autobuilder.yocto.io/builders/nightly-oe-selftest/builds/1096/steps/Running%20oe-selftest/logs/stdio
>
> Cheers,
>
> Richard
>
Hi Richard,
I've fixed this issue and updated the remote branch.
I'll send out V3 soon.
Best Regards,
Chen Qi
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH 07/14] testimage.bbclass: also check 'auto' to create rpm index
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
` (5 preceding siblings ...)
2018-06-01 5:03 ` [PATCH 06/14] oeqa/runtime/cases/selftest.py: rename to _selftest.py Chen Qi
@ 2018-06-01 5:03 ` Chen Qi
2018-06-01 5:03 ` [PATCH 08/14] oeqa/core/decorator/data.py: fix skipIfNotInDataVar Chen Qi
` (6 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:03 UTC (permalink / raw)
To: openembedded-core
Having 'auto' in TEST_SUITES will also run the 'dnf' test cases,
so also check it to determine whether to create rpm index or not.
This is to fix the following error when TEST_SUITES = "auto".
RESULTS - dnf.DnfRepoTest.test_dnf_makecache - Testcase 1744: ERROR
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/classes/testimage.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index d97da67..14252ff 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -121,7 +121,7 @@ python do_testimage() {
testimage_sanity(d)
if (d.getVar('IMAGE_PKGTYPE') == 'rpm'
- and 'dnf' in d.getVar('TEST_SUITES')):
+ and ('dnf' in d.getVar('TEST_SUITES') or 'auto' in d.getVar('TEST_SUITES'))):
create_rpm_index(d)
testimage_main(d)
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH 08/14] oeqa/core/decorator/data.py: fix skipIfNotInDataVar
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
` (6 preceding siblings ...)
2018-06-01 5:03 ` [PATCH 07/14] testimage.bbclass: also check 'auto' to create rpm index Chen Qi
@ 2018-06-01 5:03 ` Chen Qi
2018-06-01 5:03 ` [PATCH 09/14] oeqa/runtime/cases/rpm.py: skip if rpm not available Chen Qi
` (5 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:03 UTC (permalink / raw)
To: openembedded-core
The var might not be set, resulting in unexpected error.
RESULTS - multilib.MultilibTest.test_check_multilib_libc - Testcase 1593: ERROR
The above error is due to MULTILIBS being not set, which is the default
for OE. This patch fixes this problem.
Also, the debugging message in skipIfNotInDataVar is currently confusing.
Instead of
DEBUG: Checking if 'MULTILIBS' value is in 'multilib:lib32' to run the test
it should be
DEBUG: Checking if 'MULTILIBS' value contains 'multilib:lib32' to run the test
This patch also fixes it.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/lib/oeqa/core/decorator/data.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/lib/oeqa/core/decorator/data.py b/meta/lib/oeqa/core/decorator/data.py
index ff7bdd9..31c6dd6 100644
--- a/meta/lib/oeqa/core/decorator/data.py
+++ b/meta/lib/oeqa/core/decorator/data.py
@@ -61,10 +61,10 @@ class skipIfNotInDataVar(OETestDecorator):
attrs = ('var', 'value', 'msg')
def setUpDecorator(self):
- msg = ('Checking if %r value is in %r to run '
+ msg = ('Checking if %r value contains %r to run '
'the test' % (self.var, self.value))
self.logger.debug(msg)
- if not self.value in self.case.td.get(self.var):
+ if not self.value in (self.case.td.get(self.var) or ""):
self.case.skipTest(self.msg)
@registerDecorator
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH 09/14] oeqa/runtime/cases/rpm.py: skip if rpm not available
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
` (7 preceding siblings ...)
2018-06-01 5:03 ` [PATCH 08/14] oeqa/core/decorator/data.py: fix skipIfNotInDataVar Chen Qi
@ 2018-06-01 5:03 ` Chen Qi
2018-06-01 5:03 ` [PATCH 10/14] oeqa/runtime/cases/multilib.py: skip if needed packages are " Chen Qi
` (4 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:03 UTC (permalink / raw)
To: openembedded-core
This test case should only run when rpm package is installed.
So skip it if rpm package is not installed. This fixes:
RESULTS - rpm.RpmBasicTest.test_rpm_help - Testcase 1059: FAILED
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/lib/oeqa/runtime/cases/rpm.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/lib/oeqa/runtime/cases/rpm.py b/meta/lib/oeqa/runtime/cases/rpm.py
index 05b94c7..84c59a6 100644
--- a/meta/lib/oeqa/runtime/cases/rpm.py
+++ b/meta/lib/oeqa/runtime/cases/rpm.py
@@ -16,6 +16,7 @@ class RpmBasicTest(OERuntimeTestCase):
cls.skipTest('Tests require image to be build from rpm')
@OETestID(960)
+ @OEHasPackage(['rpm'])
@OETestDepends(['ssh.SSHTest.test_ssh'])
def test_rpm_help(self):
status, output = self.target.run('rpm --help')
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH 10/14] oeqa/runtime/cases/multilib.py: skip if needed packages are not available
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
` (8 preceding siblings ...)
2018-06-01 5:03 ` [PATCH 09/14] oeqa/runtime/cases/rpm.py: skip if rpm not available Chen Qi
@ 2018-06-01 5:03 ` Chen Qi
2018-06-01 5:03 ` [PATCH 11/14] oeqa/runtime/cases/multilib.py: fix test_file_connman skipping logic Chen Qi
` (3 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:03 UTC (permalink / raw)
To: openembedded-core
1) The test cases use 'readelf' command to do the check. This command
is from binutils. So skip the test if the needed binutils package is
not installed.
The related error message in log.do_testimage is like below.
Output: sh: readelf: not found
2) The test case tests /lib/libc.so.6 from lib32-libc6. So skip the test
if lib32-libc6 is not installed.
The related error message in log.do_testimage is like below.
Output: readelf: Error: 'lib/libc.so.6': No such file
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/lib/oeqa/runtime/cases/multilib.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/lib/oeqa/runtime/cases/multilib.py b/meta/lib/oeqa/runtime/cases/multilib.py
index 8c167f1..970f676 100644
--- a/meta/lib/oeqa/runtime/cases/multilib.py
+++ b/meta/lib/oeqa/runtime/cases/multilib.py
@@ -27,6 +27,8 @@ class MultilibTest(OERuntimeTestCase):
@skipIfNotInDataVar('MULTILIBS', 'multilib:lib32',
"This isn't a multilib:lib32 image")
@OETestDepends(['ssh.SSHTest.test_ssh'])
+ @OEHasPackage(['binutils'])
+ @OEHasPackage(['lib32-libc6'])
def test_check_multilib_libc(self):
"""
Check that a multilib image has both 32-bit and 64-bit libc in.
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH 11/14] oeqa/runtime/cases/multilib.py: fix test_file_connman skipping logic
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
` (9 preceding siblings ...)
2018-06-01 5:03 ` [PATCH 10/14] oeqa/runtime/cases/multilib.py: skip if needed packages are " Chen Qi
@ 2018-06-01 5:03 ` Chen Qi
2018-06-01 5:03 ` [PATCH 12/14] bitbake.conf: fix HOSTTOOLS setting related to image testing Chen Qi
` (2 subsequent siblings)
13 siblings, 0 replies; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:03 UTC (permalink / raw)
To: openembedded-core
The test_file_connman should be executed only when 'lib32-connman' is
installed and 'connman' is not installed.
When lib32-connman and connman are both installed, the /usr/sbin/connmand
could be from connman or lib32-connman, depending on the installation
order. What we want to check is the connmand command from lib32-connman,
so we need to make sure that connman is not there to cause chaos.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/lib/oeqa/runtime/cases/multilib.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/lib/oeqa/runtime/cases/multilib.py b/meta/lib/oeqa/runtime/cases/multilib.py
index 970f676..8902038 100644
--- a/meta/lib/oeqa/runtime/cases/multilib.py
+++ b/meta/lib/oeqa/runtime/cases/multilib.py
@@ -38,6 +38,6 @@ class MultilibTest(OERuntimeTestCase):
@OETestID(279)
@OETestDepends(['multilib.MultilibTest.test_check_multilib_libc'])
- @OEHasPackage(['lib32-connman'])
+ @OEHasPackage(['lib32-connman', '!connman'])
def test_file_connman(self):
self.archtest("/usr/sbin/connmand", "ELF32")
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH 12/14] bitbake.conf: fix HOSTTOOLS setting related to image testing
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
` (10 preceding siblings ...)
2018-06-01 5:03 ` [PATCH 11/14] oeqa/runtime/cases/multilib.py: fix test_file_connman skipping logic Chen Qi
@ 2018-06-01 5:03 ` Chen Qi
2018-06-01 5:03 ` [PATCH 13/14] testimage.bbclass: move codes into testimage_main Chen Qi
2018-06-01 5:03 ` [PATCH 14/14] oeqa/core/target/ssh.py: increase maximum read bytes from 1024 to 4096 Chen Qi
13 siblings, 0 replies; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:03 UTC (permalink / raw)
To: openembedded-core
A list of tools are added to HOSTTOOLS depending on if we inherit
testimage or not. Unfortunately, if we use TEST_IMAGE variable to
automate the test, these tools are not added to HOSTTOOLS.
Modify the condition to also check TEST_IMAGE to fix the above problem.
Also, change to use if...else... instead of list index for such setting.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/conf/bitbake.conf | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 6ecf78b..4bac7ea 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -487,7 +487,7 @@ HOSTTOOLS += " \
"
# Tools needed to run testimage runtime image testing
-HOSTTOOLS += "${@['', 'ip ping ps scp ssh stty'][bb.data.inherits_class('testimage', d)]}"
+HOSTTOOLS += "${@'ip ping ps scp ssh stty' if (bb.data.inherits_class('testimage', d) or d.getVar('TEST_IMAGE') == '1') else ''}"
# Link to these if present
HOSTTOOLS_NONFATAL += "aws ccache gcc-ar gpg ld.bfd ld.gold nc sftp socat ssh sudo"
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH 13/14] testimage.bbclass: move codes into testimage_main
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
` (11 preceding siblings ...)
2018-06-01 5:03 ` [PATCH 12/14] bitbake.conf: fix HOSTTOOLS setting related to image testing Chen Qi
@ 2018-06-01 5:03 ` Chen Qi
2018-06-01 5:03 ` [PATCH 14/14] oeqa/core/target/ssh.py: increase maximum read bytes from 1024 to 4096 Chen Qi
13 siblings, 0 replies; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:03 UTC (permalink / raw)
To: openembedded-core
testimage-auto is expected to run testimage task's codes automatically.
But in fact, it's currently missing some codes, including testimage_sanity
and create_rpm_index.
This leads to the problem of unexpected runtime failure of test_dnf_makecache.
The error message is as below.
RESULTS - dnf.DnfRepoTest.test_dnf_makecache - Testcase 1744: ERROR
This error is caused by the fact that create_rpm_index is not executed
before running the tests.
There's no reason why such codes should not be in testimage_main, so
move them into it.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/classes/testimage.bbclass | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 14252ff..c17a7c6 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -117,13 +117,6 @@ testimage_dump_host () {
}
python do_testimage() {
-
- testimage_sanity(d)
-
- if (d.getVar('IMAGE_PKGTYPE') == 'rpm'
- and ('dnf' in d.getVar('TEST_SUITES') or 'auto' in d.getVar('TEST_SUITES'))):
- create_rpm_index(d)
-
testimage_main(d)
}
@@ -159,6 +152,12 @@ def testimage_main(d):
"""
raise RuntimeError
+ testimage_sanity(d)
+
+ if (d.getVar('IMAGE_PKGTYPE') == 'rpm'
+ and ('dnf' in d.getVar('TEST_SUITES') or 'auto' in d.getVar('TEST_SUITES'))):
+ create_rpm_index(d)
+
logger = make_logger_bitbake_compatible(logging.getLogger("BitBake"))
pn = d.getVar("PN")
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH 14/14] oeqa/core/target/ssh.py: increase maximum read bytes from 1024 to 4096
2018-06-01 5:02 [PATCH V2 00/14] testimage: restore 'auto' behaviour and a few other fixes Chen Qi
` (12 preceding siblings ...)
2018-06-01 5:03 ` [PATCH 13/14] testimage.bbclass: move codes into testimage_main Chen Qi
@ 2018-06-01 5:03 ` Chen Qi
13 siblings, 0 replies; 17+ messages in thread
From: Chen Qi @ 2018-06-01 5:03 UTC (permalink / raw)
To: openembedded-core
When running testimage task for core-image-sato-sdk, the following
error appeared.
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x82 at position 0: invalid start byte
Checking the codes, I found it's caused by setting a 1024 limit for the
read method of the StreamReader object.
Comments from the manual:
"""
The chars argument indicates the number of decoded code points or bytes to
return. The read() method will never return more data than requested, but
it might return less, if there is not enough available.
"""
When running `systemctl status --full' on target, this error occurs.
This patch increase the bytes limit to 4096 to fix the error.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
meta/lib/oeqa/core/target/ssh.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/lib/oeqa/core/target/ssh.py b/meta/lib/oeqa/core/target/ssh.py
index 151b99a..8ff1f6c 100644
--- a/meta/lib/oeqa/core/target/ssh.py
+++ b/meta/lib/oeqa/core/target/ssh.py
@@ -208,7 +208,7 @@ def SSHCall(command, logger, timeout=None, **opts):
try:
if select.select([process.stdout], [], [], 5)[0] != []:
reader = codecs.getreader('utf-8')(process.stdout)
- data = reader.read(1024, 1024)
+ data = reader.read(1024, 4096)
if not data:
process.stdout.close()
eof = True
--
1.9.1
^ permalink raw reply related [flat|nested] 17+ messages in thread