[PATCH 0/1] oeqa/runtime/cases/rpm.py: change to use base-passwd-doc

[PATCH 0/1] oeqa/runtime/cases/rpm.py: change to use base-passwd-doc

[Chen Qi]

The following changes since commit ddbd7b0cd6580ee26e11aa87e426fb294b372d64:

  meta-yocto-bsp: bump to the latest linux stable kernel for edgerouter/beaglebone (2018-06-12 23:34:40 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib ChenQi/rpm_test_case
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=ChenQi/rpm_test_case

Chen Qi (1):
  oeqa/runtime/cases/rpm.py: change to use base-passwd-doc

 meta/lib/oeqa/runtime/cases/rpm.py | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

-- 
1.9.1

[PATCH 1/1] oeqa/runtime/cases/rpm.py: change to use base-passwd-doc

[Chen Qi]

The test cases assume that rpm-doc is built out, as it says it its
comment. This is not always true. And it sometimes results in
following error.

  |     cls.tc.target.copyTo(test_file, dst)
  | UnboundLocalError: local variable 'test_file' referenced before assignment

Change to use base-passwd-doc, as this package is more likely to be
built out than rpm-doc.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/lib/oeqa/runtime/cases/rpm.py | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/meta/lib/oeqa/runtime/cases/rpm.py b/meta/lib/oeqa/runtime/cases/rpm.py
index 58df28b..1e5e463 100644
--- a/meta/lib/oeqa/runtime/cases/rpm.py
+++ b/meta/lib/oeqa/runtime/cases/rpm.py
@@ -39,31 +39,31 @@ class RpmInstallRemoveTest(OERuntimeTestCase):
 
         pkgarch = cls.td['TUNE_PKGARCH'].replace('-', '_')
         rpmdir = os.path.join(cls.tc.td['DEPLOY_DIR'], 'rpm', pkgarch)
-        # Pick rpm-doc as a test file to get installed, because it's small
+        # Pick base-passwd-doc as a test file to get installed, because it's small
         # and it will always be built for standard targets
-        rpm_doc = 'rpm-doc-*.%s.rpm' % pkgarch
+        rpm_doc = 'base-passwd-doc-*.%s.rpm' % pkgarch
         for f in fnmatch.filter(os.listdir(rpmdir), rpm_doc):
             test_file = os.path.join(rpmdir, f)
-        dst = '/tmp/rpm-doc.rpm'
+        dst = '/tmp/base-passwd-doc.rpm'
         cls.tc.target.copyTo(test_file, dst)
 
     @classmethod
     def tearDownClass(cls):
-        dst = '/tmp/rpm-doc.rpm'
+        dst = '/tmp/base-passwd-doc.rpm'
         cls.tc.target.run('rm -f %s' % dst)
 
     @OETestID(192)
     @OETestDepends(['rpm.RpmBasicTest.test_rpm_help'])
     def test_rpm_install(self):
-        status, output = self.target.run('rpm -ivh /tmp/rpm-doc.rpm')
-        msg = 'Failed to install rpm-doc package: %s' % output
+        status, output = self.target.run('rpm -ivh /tmp/base-passwd-doc.rpm')
+        msg = 'Failed to install base-passwd-doc package: %s' % output
         self.assertEqual(status, 0, msg=msg)
 
     @OETestID(194)
     @OETestDepends(['rpm.RpmInstallRemoveTest.test_rpm_install'])
     def test_rpm_remove(self):
-        status,output = self.target.run('rpm -e rpm-doc')
-        msg = 'Failed to remove rpm-doc package: %s' % output
+        status,output = self.target.run('rpm -e base-passwd-doc')
+        msg = 'Failed to remove base-passwd-doc package: %s' % output
         self.assertEqual(status, 0, msg=msg)
 
     @OETestID(1096)
@@ -119,16 +119,16 @@ class RpmInstallRemoveTest(OERuntimeTestCase):
         self.assertEqual(0, status, msg=msg)
 
         # Remove the package just in case
-        self.target.run('rpm -e rpm-doc')
+        self.target.run('rpm -e base-passwd-doc')
 
         # Install/Remove a package 10 times
         for i in range(10):
-            status, output = self.target.run('rpm -ivh /tmp/rpm-doc.rpm')
-            msg = 'Failed to install rpm-doc package. Reason: {}'.format(output)
+            status, output = self.target.run('rpm -ivh /tmp/base-passwd-doc.rpm')
+            msg = 'Failed to install base-passwd-doc package. Reason: {}'.format(output)
             self.assertEqual(0, status, msg=msg)
 
-            status, output = self.target.run('rpm -e rpm-doc')
-            msg = 'Failed to remove rpm-doc package. Reason: {}'.format(output)
+            status, output = self.target.run('rpm -e base-passwd-doc')
+            msg = 'Failed to remove base-passwd-doc package. Reason: {}'.format(output)
             self.assertEqual(0, status, msg=msg)
 
         # if using systemd this should ensure all entries are flushed to /var
-- 
1.9.1

Re: [PATCH 1/1] oeqa/runtime/cases/rpm.py: change to use base-passwd-doc

[Alexander Kanavin]

2018-06-14 7:52 GMT+03:00 Chen Qi <Qi.Chen@windriver.com>:
> The test cases assume that rpm-doc is built out, as it says it its
> comment. This is not always true. And it sometimes results in
> following error.
>
>   |     cls.tc.target.copyTo(test_file, dst)
>   | UnboundLocalError: local variable 'test_file' referenced before assignment

Wait, you need to explain this further. What is the situation when it
isn't true?

Alex

Re: [PATCH 1/1] oeqa/runtime/cases/rpm.py: change to use base-passwd-doc

[ChenQi]

On 06/14/2018 02:57 PM, Alexander Kanavin wrote:
> 2018-06-14 7:52 GMT+03:00 Chen Qi <Qi.Chen@windriver.com>:
>> The test cases assume that rpm-doc is built out, as it says it its
>> comment. This is not always true. And it sometimes results in
>> following error.
>>
>>    |     cls.tc.target.copyTo(test_file, dst)
>>    | UnboundLocalError: local variable 'test_file' referenced before assignment
> Wait, you need to explain this further. What is the situation when it
> isn't true?
>
> Alex
>

Hi Alex,

For any image that does not have rpm installed, rpm package is not 
likely to be built out.
Having PACKAGE_CLASSES set to "package_rpm" will only ensure rpm-native 
to be built out.

Best Regards,
Chen Qi

Re: [PATCH 1/1] oeqa/runtime/cases/rpm.py: change to use base-passwd-doc

[Alexander Kanavin]

Hello Chen,

But the test still attempts to run rpm on the target. I can't
understand how target rpm can be available on the image, but at the
same time target rpm-doc package is not available in rpm deploy dir.

The issue is probably that the test should simply be skipped (in the
same way that dnf testcases are skipped - please refer to them), if
there is no package management on target? Like this:

    @skipIfNotFeature('package-management',
                      'Test requires package-management to be in
IMAGE_FEATURES')


Alex

2018-06-14 10:29 GMT+03:00 ChenQi <Qi.Chen@windriver.com>:
> On 06/14/2018 02:57 PM, Alexander Kanavin wrote:
>>
>> 2018-06-14 7:52 GMT+03:00 Chen Qi <Qi.Chen@windriver.com>:
>>>
>>> The test cases assume that rpm-doc is built out, as it says it its
>>> comment. This is not always true. And it sometimes results in
>>> following error.
>>>
>>>    |     cls.tc.target.copyTo(test_file, dst)
>>>    | UnboundLocalError: local variable 'test_file' referenced before
>>> assignment
>>
>> Wait, you need to explain this further. What is the situation when it
>> isn't true?
>>
>> Alex
>>
>
> Hi Alex,
>
> For any image that does not have rpm installed, rpm package is not likely to
> be built out.
> Having PACKAGE_CLASSES set to "package_rpm" will only ensure rpm-native to
> be built out.
>
> Best Regards,
> Chen Qi

Re: [PATCH 1/1] oeqa/runtime/cases/rpm.py: change to use base-passwd-doc

[ChenQi]

Hi Alex,

In short, it's about setUpClass.
See details below.

In fact, if rpm is not installed, these test cases have been skipped.

See codes below from rpm.py:

     @OETestID(960)
     @OEHasPackage(['rpm'])
     @OETestDepends(['ssh.SSHTest.test_ssh'])
     def test_rpm_help(self):
         status, output = self.target.run('rpm --help')
         msg = 'status and output: %s and %s' % (status, output)
         self.assertEqual(status, 0, msg=msg)

     @OETestID(192)
     @OETestDepends(['rpm.RpmBasicTest.test_rpm_help'])
     def test_rpm_install(self):
         status, output = self.target.run('rpm -ivh 
/tmp/base-passwd-doc.rpm')
         msg = 'Failed to install base-passwd-doc package: %s' % output
         self.assertEqual(status, 0, msg=msg)

You can see that if 'rpm' is not installed on target, the test cases are 
skipped.
The '@skipIfNotFeature' decorator you suggested will achieve the same 
effect that the above logic has already achieved.

The key problem here is that the copying action happens in setUpClass().
Even if the testcase is determined to be skipped via the decorator 
logic, this setUpClass function will still be called.

For the current OEQA's runtime testing design, we need to ensure 
setUpClass do not generate 'error'.

For the problem I described in this email, the simplest approach I see 
here is to change rpm-doc to base-passwd-doc.
Another apporach is to check whether 'rpm' is installed in setUpClass() 
function and if not skip. But this is more complex and kind of weird as 
we've checked it via @HasPackage and the test cases dependencies are 
correct.

You can do a quick test with the following configuration to reproduce 
the problem.
local.conf:
IMAGE_INSTALL_append = " openssh"
TEST_IMAGE = "1"
TEST_SUITES = "auto"

bitbake core-image-minimal

Best Regards,
Chen Qi


On 06/14/2018 04:55 PM, Alexander Kanavin wrote:
> Hello Chen,
>
> But the test still attempts to run rpm on the target. I can't
> understand how target rpm can be available on the image, but at the
> same time target rpm-doc package is not available in rpm deploy dir.
>
> The issue is probably that the test should simply be skipped (in the
> same way that dnf testcases are skipped - please refer to them), if
> there is no package management on target? Like this:
>
>      @skipIfNotFeature('package-management',
>                        'Test requires package-management to be in
> IMAGE_FEATURES')
>
>
> Alex
>
> 2018-06-14 10:29 GMT+03:00 ChenQi <Qi.Chen@windriver.com>:
>> On 06/14/2018 02:57 PM, Alexander Kanavin wrote:
>>> 2018-06-14 7:52 GMT+03:00 Chen Qi <Qi.Chen@windriver.com>:
>>>> The test cases assume that rpm-doc is built out, as it says it its
>>>> comment. This is not always true. And it sometimes results in
>>>> following error.
>>>>
>>>>     |     cls.tc.target.copyTo(test_file, dst)
>>>>     | UnboundLocalError: local variable 'test_file' referenced before
>>>> assignment
>>> Wait, you need to explain this further. What is the situation when it
>>> isn't true?
>>>
>>> Alex
>>>
>> Hi Alex,
>>
>> For any image that does not have rpm installed, rpm package is not likely to
>> be built out.
>> Having PACKAGE_CLASSES set to "package_rpm" will only ensure rpm-native to
>> be built out.
>>
>> Best Regards,
>> Chen Qi

Re: [PATCH 1/1] oeqa/runtime/cases/rpm.py: change to use base-passwd-doc

[Alexander Kanavin]

Right, I see. Thanks for explaining, the patch is fine then.

Alex

2018-06-14 12:31 GMT+03:00 ChenQi <Qi.Chen@windriver.com>:
> Hi Alex,
>
> In short, it's about setUpClass.
> See details below.
>
> In fact, if rpm is not installed, these test cases have been skipped.
>
> See codes below from rpm.py:
>
>     @OETestID(960)
>     @OEHasPackage(['rpm'])
>     @OETestDepends(['ssh.SSHTest.test_ssh'])
>     def test_rpm_help(self):
>         status, output = self.target.run('rpm --help')
>         msg = 'status and output: %s and %s' % (status, output)
>         self.assertEqual(status, 0, msg=msg)
>
>     @OETestID(192)
>     @OETestDepends(['rpm.RpmBasicTest.test_rpm_help'])
>     def test_rpm_install(self):
>         status, output = self.target.run('rpm -ivh
> /tmp/base-passwd-doc.rpm')
>         msg = 'Failed to install base-passwd-doc package: %s' % output
>         self.assertEqual(status, 0, msg=msg)
>
> You can see that if 'rpm' is not installed on target, the test cases are
> skipped.
> The '@skipIfNotFeature' decorator you suggested will achieve the same effect
> that the above logic has already achieved.
>
> The key problem here is that the copying action happens in setUpClass().
> Even if the testcase is determined to be skipped via the decorator logic,
> this setUpClass function will still be called.
>
> For the current OEQA's runtime testing design, we need to ensure setUpClass
> do not generate 'error'.
>
> For the problem I described in this email, the simplest approach I see here
> is to change rpm-doc to base-passwd-doc.
> Another apporach is to check whether 'rpm' is installed in setUpClass()
> function and if not skip. But this is more complex and kind of weird as
> we've checked it via @HasPackage and the test cases dependencies are
> correct.
>
> You can do a quick test with the following configuration to reproduce the
> problem.
> local.conf:
> IMAGE_INSTALL_append = " openssh"
> TEST_IMAGE = "1"
> TEST_SUITES = "auto"
>
> bitbake core-image-minimal
>
> Best Regards,
> Chen Qi
>
>
>
> On 06/14/2018 04:55 PM, Alexander Kanavin wrote:
>>
>> Hello Chen,
>>
>> But the test still attempts to run rpm on the target. I can't
>> understand how target rpm can be available on the image, but at the
>> same time target rpm-doc package is not available in rpm deploy dir.
>>
>> The issue is probably that the test should simply be skipped (in the
>> same way that dnf testcases are skipped - please refer to them), if
>> there is no package management on target? Like this:
>>
>>      @skipIfNotFeature('package-management',
>>                        'Test requires package-management to be in
>> IMAGE_FEATURES')
>>
>>
>> Alex
>>
>> 2018-06-14 10:29 GMT+03:00 ChenQi <Qi.Chen@windriver.com>:
>>>
>>> On 06/14/2018 02:57 PM, Alexander Kanavin wrote:
>>>>
>>>> 2018-06-14 7:52 GMT+03:00 Chen Qi <Qi.Chen@windriver.com>:
>>>>>
>>>>> The test cases assume that rpm-doc is built out, as it says it its
>>>>> comment. This is not always true. And it sometimes results in
>>>>> following error.
>>>>>
>>>>>     |     cls.tc.target.copyTo(test_file, dst)
>>>>>     | UnboundLocalError: local variable 'test_file' referenced before
>>>>> assignment
>>>>
>>>> Wait, you need to explain this further. What is the situation when it
>>>> isn't true?
>>>>
>>>> Alex
>>>>
>>> Hi Alex,
>>>
>>> For any image that does not have rpm installed, rpm package is not likely
>>> to
>>> be built out.
>>> Having PACKAGE_CLASSES set to "package_rpm" will only ensure rpm-native
>>> to
>>> be built out.
>>>
>>> Best Regards,
>>> Chen Qi
>
>
>