[PATCH 0/1] multiconfig.py: added testsuite for multiconfig

[PATCH 0/1] multiconfig.py: added testsuite for multiconfig

[Humberto Ibarra]

Multiconfig testsuite is being added to selftest, incluiding a globbing
testcase.

The following changes since commit d4c3ace09790a18e240c4aea655e41ecee786e1b:

  bitbake: bitbake-diffsigs: fix regression after recent server changes (2017-07-27 15:14:20 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib humberto/Multiconfig
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=humberto/Multiconfig

Humberto Ibarra (1):
  multiconfig.py: added testsuite for multiconfig

 meta/lib/oeqa/selftest/cases/multiconfig.py | 100 ++++++++++++++++++++++++++++
 1 file changed, 100 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/multiconfig.py

-- 
2.7.5

[PATCH 1/1] multiconfig.py: added testsuite for multiconfig

[Humberto Ibarra]

Multiconfig was not being tested at all. This adds a few basic
tests for multiconfig, including globbing.

[Yocto #11612]

Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
---
 meta/lib/oeqa/selftest/cases/multiconfig.py | 100 ++++++++++++++++++++++++++++
 1 file changed, 100 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/multiconfig.py

diff --git a/meta/lib/oeqa/selftest/cases/multiconfig.py b/meta/lib/oeqa/selftest/cases/multiconfig.py
new file mode 100644
index 0000000..67051b7
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/multiconfig.py
@@ -0,0 +1,100 @@
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_vars
+import os
+import shutil
+import glob
+
+
+class MulticonfigTests(OESelftestTestCase):
+
+    def setUpLocal(self):
+        self.bbvars = get_bb_vars(['TOPDIR', 'DEPLOY_DIR', 'MACHINE'])
+
+        self.machines = ['qemux86', 'qemuarm', 'qemumips']
+        self.recipes = ['core-image-minimal', 'core-image-sato', \
+                'core-image-minimal-dev']
+        self.test_recipe = 'core-image-sato'
+
+        self.conf_cmds = ["multiconfig:config%s:%s" % (x, self.recipes[x]) \
+                for x in range(3)]
+
+        self.mconf = os.path.join(self.bbvars['TOPDIR'], 'conf', 'multiconfig')
+        os.mkdir(self.mconf)
+        for n in range(3):
+            conf_file = os.path.join(self.mconf, 'config%s.conf' % n)
+            with open(conf_file, 'w') as f:
+                f.write('MACHINE = "%s"\n' % self.machines[n])
+        features = 'BBMULTICONFIG = "%s"' % " ".join(["config%s" % x \
+                for x in range(3)])
+        self.write_config(features)
+
+    def tearDownLocal(self):
+        shutil.rmtree(self.mconf)
+
+    def check_recipe(self, machine, recipe, unique=True):
+        built_tar = "%s-%s.tar.bz2" % (recipe, machine)
+        recipe_paths = glob.iglob(os.path.join(self.bbvars['DEPLOY_DIR'], \
+                    'images', '**', "%s-*.tar.bz2" % recipe))
+        recipe_tars = [os.path.basename(x) for x in recipe_paths]
+        hyphen_c = recipe.count('-') + machine.count('-') + 1
+        recipe_tars = [ x for x in recipe_tars if x.count('.') == 2 \
+                    and x.count('-') == hyphen_c]
+
+        self.assertLess(0,len(recipe_tars),"Recipe %s was not built for %s"\
+                    % (recipe, machine))
+        self.assertEqual(True, built_tar in recipe_tars, "Wrong machine was " \
+                    " built, expected %s." % machine)
+        if unique:
+            self.assertLess(len(recipe_tars), 2, "Recipe %s was built " \
+                "for machines other than %s" % (recipe, machine))
+
+    def clean_builds(self):
+        bitbake('-c clean %s' % " ".join(self.conf_cmds))
+        bitbake('-c clean %s' % " ".join(self.recipes))
+        bitbake('-c clean multiconfig:*:%s' % self.test_recipe)
+
+    def test_different_configs(self):
+        """
+        Summary: Check multiconfig using 2 diferent machines
+        Expected: 1. All recipes can be build using the right conf
+                  2. Recipes aren't build for more confs than intended
+        Product: oe-core
+        Author: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
+        """
+        self.clean_builds()
+        bitbake(" ".join(self.conf_cmds))
+
+        # Check that proper builds were done and only for the required recipes
+        for x in range(3):
+            self.check_recipe(self.machines[x], self.recipes[x])
+
+    def test_general_recipe(self):
+        """
+        Summary: Check multiconfig and a general recipe at the same time
+        Expected: 1. The multiconfig recipe can be build
+                  2. The other recipe can be build for the machine in local.conf
+        Product: oe-core
+        Author: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
+        """
+        self.clean_builds()
+        bitbake('%s %s' % (self.conf_cmds[0], self.test_recipe))
+
+        # Check that recipe[0] was only built for config0
+        self.check_recipe(self.machines[0], self.recipes[0])
+
+        # Check that core-image-sato was built for general config
+        self.check_recipe(self.bbvars['MACHINE'], self.test_recipe)
+
+    def test_multiconfig_globbing(self):
+        """
+        Summary: Check multiconfig globbing
+        Expected: 1. The globbing feature works for multiconfig
+        Product: oe-core
+        Author: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
+        """
+        self.clean_builds()
+        bitbake('multiconfig:*:%s' % self.test_recipe)
+
+        # Check that recipe was built for all configs
+        for machine in self.machines:
+            self.check_recipe(machine, self.test_recipe, False)
-- 
2.7.5

Re: [PATCH 1/1] multiconfig.py: added testsuite for multiconfig

[Richard Purdie]

On Thu, 2017-07-27 at 15:18 -0500, Humberto Ibarra wrote:
> Multiconfig was not being tested at all. This adds a few basic
> tests for multiconfig, including globbing.
> 
> [Yocto #11612]
> 
> Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>

If I run this selftest and I have some multiconfigs already in my build
directory, what happens to them?

Cheers,

Richard

Re: [PATCH 1/1] multiconfig.py: added testsuite for multiconfig

[Ibarra Lopez, Humberto]

This patch does not consider current multiconfig settings. I talked with Anibal about this and he said this wouldn't be an issue after #11429, which he is working on. Just checked that bug and is still in implementation, so I could put 11429 as a dependency for 11612?

Regards,

Humberto 

-----Original Message-----
From: Richard Purdie [mailto:richard.purdie@linuxfoundation.org] 
Sent: Thursday, July 27, 2017 4:12 PM
To: Ibarra Lopez, Humberto <humberto.ibarra.lopez@intel.com>; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 1/1] multiconfig.py: added testsuite for multiconfig

On Thu, 2017-07-27 at 15:18 -0500, Humberto Ibarra wrote:
> Multiconfig was not being tested at all. This adds a few basic tests 
> for multiconfig, including globbing.
> 
> [Yocto #11612]
> 
> Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>

If I run this selftest and I have some multiconfigs already in my build directory, what happens to them?

Cheers,

Richard