[PATCH 0/4] Some optimizations on top time selftest checks

[PATCH 0/4] Some optimizations on top time selftest checks

[leonardo.sandoval.gonzalez]

From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>

* On my local host test, the optimization went from 44 to 8 minutes with the following command

oe-selftest -r \
buildoptions.ImageOptionsTests.test_read_only_image \
bbtests.BitbakeTests.test_non_gplv3 \
buildoptions.ImageOptionsTests.test_ccache_tool \
buildoptions.ImageOptionsTests.test_read_only_image

  * Before series applied
  
  2017-01-31 15:41:20 - test_read_only_image (oeqa.selftest.buildoptions.ImageOptionsTests) ... ok
  2017-01-31 16:20:12 - test_non_gplv3 (oeqa.selftest.bbtests.BitbakeTests) ... ok
  2017-01-31 16:22:55 - test_ccache_tool (oeqa.selftest.buildoptions.ImageOptionsTests) ... ok
  2017-01-31 16:23:49 - test_read_only_image (oeqa.selftest.buildoptions.ImageOptionsTests) ... ok

  ----------------------------------------------------------------------
  Ran 4 tests in 2662.852s

  * After series applied
  
  2017-01-31 15:27:08 - test_read_only_image (oeqa.selftest.buildoptions.ImageOptionsTests) ... ok
  2017-01-31 15:27:14 - test_non_gplv3 (oeqa.selftest.bbtests.BitbakeTests) ... ok
  2017-01-31 15:33:42 - test_ccache_tool (oeqa.selftest.buildoptions.ImageOptionsTests) ... ok
  2017-01-31 15:34:30 - test_read_only_image (oeqa.selftest.buildoptions.ImageOptionsTests) ... ok

  ----------------------------------------------------------------------
  Ran 4 tests in 518.647s


[YOCTO #10874]

The following changes since commit d45d4a5a21ab4209b87331dddf515ecdb62367fa:

  staging: Handle stale sysroot rebuild issue (2017-01-28 23:32:16 +0000)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lsandov1/oe-selftest-optimizations
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lsandov1/oe-selftest-optimizations

Leonardo Sandoval (4):
  selftest/buildoptions: use a thinner image to test 'read-only-rootfs'
    feature
  selftest/buildoptions: force compile task instead of cleaning sstates
  selftest/bbtests: use write_config instead of local.conf file
  selftest/archiver: invalidate stamps instead of removing TMPDIR

 meta/lib/oeqa/selftest/archiver.py     | 4 ++--
 meta/lib/oeqa/selftest/bbtests.py      | 5 +----
 meta/lib/oeqa/selftest/buildoptions.py | 8 ++------
 3 files changed, 5 insertions(+), 12 deletions(-)

-- 
2.1.4

[PATCH 1/4] selftest/buildoptions: use a thinner image to test 'read-only-rootfs' feature

[leonardo.sandoval.gonzalez]

From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>

The minimal is much faster to build that sato, so use the former to test
read-only feature.

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
---
 meta/lib/oeqa/selftest/buildoptions.py | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/selftest/buildoptions.py b/meta/lib/oeqa/selftest/buildoptions.py
index d40eb00..004b2dd 100644
--- a/meta/lib/oeqa/selftest/buildoptions.py
+++ b/meta/lib/oeqa/selftest/buildoptions.py
@@ -44,11 +44,8 @@ class ImageOptionsTests(oeSelfTest):
 
     @testcase(1435)
     def test_read_only_image(self):
-        distro_features = get_bb_var('DISTRO_FEATURES')
-        if not ('x11' in distro_features and 'opengl' in distro_features):
-            self.skipTest('core-image-sato requires x11 and opengl in distro features')
         self.write_config('IMAGE_FEATURES += "read-only-rootfs"')
-        bitbake("core-image-sato")
+        bitbake("core-image-minimal")
         # do_image will fail if there are any pending postinsts
 
 class DiskMonTest(oeSelfTest):
-- 
2.1.4

[PATCH 2/4] selftest/buildoptions: force compile task instead of cleaning sstates

[leonardo.sandoval.gonzalez]

From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>

There is no need to clean m4 sstates, use force instead.

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
---
 meta/lib/oeqa/selftest/buildoptions.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/selftest/buildoptions.py b/meta/lib/oeqa/selftest/buildoptions.py
index 004b2dd..a83088f 100644
--- a/meta/lib/oeqa/selftest/buildoptions.py
+++ b/meta/lib/oeqa/selftest/buildoptions.py
@@ -36,8 +36,7 @@ class ImageOptionsTests(oeSelfTest):
         p = get_bb_var('SYSROOT_DESTDIR', 'ccache-native') + get_bb_var('bindir', 'ccache-native') + "/" + "ccache"
         self.assertTrue(os.path.isfile(p), msg = "No ccache found (%s)" % p)
         self.write_config('INHERIT += "ccache"')
-        bitbake("m4 -c cleansstate")
-        bitbake("m4 -c compile")
+        bitbake("m4 -c compile -f")
         self.addCleanup(bitbake, 'ccache-native -ccleansstate')
         res = runCmd("grep ccache %s" % (os.path.join(get_bb_var("WORKDIR","m4"),"temp/log.do_compile")), ignore_status=True)
         self.assertEqual(0, res.status, msg="No match for ccache in m4 log.do_compile. For further details: %s" % os.path.join(get_bb_var("WORKDIR","m4"),"temp/log.do_compile"))
-- 
2.1.4

[PATCH 3/4] selftest/bbtests: use write_config instead of local.conf file

[leonardo.sandoval.gonzalez]

From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>

Extra configuration data should be write using the write_config
method instead of manually appending to the local.conf file

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
---
 meta/lib/oeqa/selftest/bbtests.py | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/selftest/bbtests.py b/meta/lib/oeqa/selftest/bbtests.py
index c4e50cb..9dbac95 100644
--- a/meta/lib/oeqa/selftest/bbtests.py
+++ b/meta/lib/oeqa/selftest/bbtests.py
@@ -229,10 +229,7 @@ INHERIT_remove = \"report-error\"
 
     @testcase(1119)
     def test_non_gplv3(self):
-        data = 'INCOMPATIBLE_LICENSE = "GPLv3"'
-        conf = os.path.join(self.builddir, 'conf/local.conf')
-        ftools.append_file(conf ,data)
-        self.addCleanup(ftools.remove_from_file, conf ,data)
+        self.write_config('INCOMPATIBLE_LICENSE = "GPLv3"')
         result = bitbake('readline', ignore_status=True)
         self.assertEqual(result.status, 0, "Bitbake failed, exit code %s, output %s" % (result.status, result.output))
         lic_dir = get_bb_var('LICENSE_DIRECTORY')
-- 
2.1.4

[PATCH 4/4] selftest/archiver: invalidate stamps instead of removing TMPDIR

[leonardo.sandoval.gonzalez]

From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>

There is no need to remove the whole TMPDIR, instead just invalidate
stamps and build again the targets.

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
---
 meta/lib/oeqa/selftest/archiver.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/selftest/archiver.py b/meta/lib/oeqa/selftest/archiver.py
index 97b6f5b..c8c4533 100644
--- a/meta/lib/oeqa/selftest/archiver.py
+++ b/meta/lib/oeqa/selftest/archiver.py
@@ -28,8 +28,8 @@ class Archiver(oeSelfTest):
         features += 'COPYLEFT_PN_EXCLUDE = "%s"\n' % exclude_recipe
         self.write_config(features)
 
-        shutil.rmtree(get_bb_var('TMPDIR'))
-        bitbake("%s %s" % (include_recipe, exclude_recipe))
+        bitbake("-C fetch %s" % include_recipe)
+        bitbake("-C fetch %s" % exclude_recipe)
 
         src_path = os.path.join(get_bb_var('DEPLOY_DIR_SRC'), get_bb_var('TARGET_SYS'))
 
-- 
2.1.4

Re: [PATCH 2/4] selftest/buildoptions: force compile task instead of cleaning sstates

[Richard Purdie]

On Tue, 2017-01-31 at 16:50 -0600,
leonardo.sandoval.gonzalez@linux.intel.com wrote:
> From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
> 
> There is no need to clean m4 sstates, use force instead.
> 
> Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.in
> tel.com>
> ---
>  meta/lib/oeqa/selftest/buildoptions.py | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/meta/lib/oeqa/selftest/buildoptions.py
> b/meta/lib/oeqa/selftest/buildoptions.py
> index 004b2dd..a83088f 100644
> --- a/meta/lib/oeqa/selftest/buildoptions.py
> +++ b/meta/lib/oeqa/selftest/buildoptions.py
> @@ -36,8 +36,7 @@ class ImageOptionsTests(oeSelfTest):
>          p = get_bb_var('SYSROOT_DESTDIR', 'ccache-native') +
> get_bb_var('bindir', 'ccache-native') + "/" + "ccache"
>          self.assertTrue(os.path.isfile(p), msg = "No ccache found
> (%s)" % p)
>          self.write_config('INHERIT += "ccache"')
> -        bitbake("m4 -c cleansstate")
> -        bitbake("m4 -c compile")
> +        bitbake("m4 -c compile -f")
>          self.addCleanup(bitbake, 'ccache-native -ccleansstate')
>          res = runCmd("grep ccache %s" %
> (os.path.join(get_bb_var("WORKDIR","m4"),"temp/log.do_compile")),
> ignore_status=True)
>          self.assertEqual(0, res.status, msg="No match for ccache in
> m4 log.do_compile. For further details: %s" %
> os.path.join(get_bb_var("WORKDIR","m4"),"temp/log.do_compile"))

If we force something like this we really need a cleanup in there to
remove the taint too...

Cheers,

Richard

Re: [PATCH 1/4] selftest/buildoptions: use a thinner image to test 'read-only-rootfs' feature

[Richard Purdie]

On Tue, 2017-01-31 at 16:50 -0600,
leonardo.sandoval.gonzalez@linux.intel.com wrote:
> From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
> 
> The minimal is much faster to build that sato, so use the former to
> test
> read-only feature.
> 
> Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.in
> tel.com>
> ---
>  meta/lib/oeqa/selftest/buildoptions.py | 5 +----
>  1 file changed, 1 insertion(+), 4 deletions(-)
> 
> diff --git a/meta/lib/oeqa/selftest/buildoptions.py
> b/meta/lib/oeqa/selftest/buildoptions.py
> index d40eb00..004b2dd 100644
> --- a/meta/lib/oeqa/selftest/buildoptions.py
> +++ b/meta/lib/oeqa/selftest/buildoptions.py
> @@ -44,11 +44,8 @@ class ImageOptionsTests(oeSelfTest):
>  
>      @testcase(1435)
>      def test_read_only_image(self):
> -        distro_features = get_bb_var('DISTRO_FEATURES')
> -        if not ('x11' in distro_features and 'opengl' in
> distro_features):
> -            self.skipTest('core-image-sato requires x11 and opengl
> in distro features')
>          self.write_config('IMAGE_FEATURES += "read-only-rootfs"')
> -        bitbake("core-image-sato")
> +        bitbake("core-image-minimal")
>          # do_image will fail if there are any pending postinsts

Whilst this is certainly going to be a touch faster, I believe we do
want to test read only rootfs with a larger image like sato to make
sure the postinsts really do work with a read only system?

Cheers,

Richard

Re: [PATCH 1/4] selftest/buildoptions: use a thinner image to test 'read-only-rootfs' feature

[Leonardo Sandoval]

On 01/31/2017 05:16 PM, Richard Purdie wrote:
> On Tue, 2017-01-31 at 16:50 -0600,
> leonardo.sandoval.gonzalez@linux.intel.com wrote:
>> From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
>>
>> The minimal is much faster to build that sato, so use the former to
>> test
>> read-only feature.
>>
>> Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.in
>> tel.com>
>> ---
>>   meta/lib/oeqa/selftest/buildoptions.py | 5 +----
>>   1 file changed, 1 insertion(+), 4 deletions(-)
>>
>> diff --git a/meta/lib/oeqa/selftest/buildoptions.py
>> b/meta/lib/oeqa/selftest/buildoptions.py
>> index d40eb00..004b2dd 100644
>> --- a/meta/lib/oeqa/selftest/buildoptions.py
>> +++ b/meta/lib/oeqa/selftest/buildoptions.py
>> @@ -44,11 +44,8 @@ class ImageOptionsTests(oeSelfTest):
>>   
>>       @testcase(1435)
>>       def test_read_only_image(self):
>> -        distro_features = get_bb_var('DISTRO_FEATURES')
>> -        if not ('x11' in distro_features and 'opengl' in
>> distro_features):
>> -            self.skipTest('core-image-sato requires x11 and opengl
>> in distro features')
>>           self.write_config('IMAGE_FEATURES += "read-only-rootfs"')
>> -        bitbake("core-image-sato")
>> +        bitbake("core-image-minimal")
>>           # do_image will fail if there are any pending postinsts
> Whilst this is certainly going to be a touch faster, I believe we do
> want to test read only rootfs with a larger image like sato to make
> sure the postinsts really do work with a read only system?

I don't get it. What would make the test different using a larger image?
>
> Cheers,
>
> Richard
>

Re: [PATCH 1/4] selftest/buildoptions: use a thinner image to test 'read-only-rootfs' feature

[Richard Purdie]

On Wed, 2017-02-01 at 09:02 -0600, Leonardo Sandoval wrote:
> 
> On 01/31/2017 05:16 PM, Richard Purdie wrote:
> > 
> > On Tue, 2017-01-31 at 16:50 -0600,
> > leonardo.sandoval.gonzalez@linux.intel.com wrote:
> > > 
> > > From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.c
> > > om>
> > > 
> > > The minimal is much faster to build that sato, so use the former
> > > to
> > > test
> > > read-only feature.
> > > 
> > > Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linu
> > > x.in
> > > tel.com>
> > > ---
> > >   meta/lib/oeqa/selftest/buildoptions.py | 5 +----
> > >   1 file changed, 1 insertion(+), 4 deletions(-)
> > > 
> > > diff --git a/meta/lib/oeqa/selftest/buildoptions.py
> > > b/meta/lib/oeqa/selftest/buildoptions.py
> > > index d40eb00..004b2dd 100644
> > > --- a/meta/lib/oeqa/selftest/buildoptions.py
> > > +++ b/meta/lib/oeqa/selftest/buildoptions.py
> > > @@ -44,11 +44,8 @@ class ImageOptionsTests(oeSelfTest):
> > >   
> > >       @testcase(1435)
> > >       def test_read_only_image(self):
> > > -        distro_features = get_bb_var('DISTRO_FEATURES')
> > > -        if not ('x11' in distro_features and 'opengl' in
> > > distro_features):
> > > -            self.skipTest('core-image-sato requires x11 and
> > > opengl
> > > in distro features')
> > >           self.write_config('IMAGE_FEATURES += "read-only-
> > > rootfs"')
> > > -        bitbake("core-image-sato")
> > > +        bitbake("core-image-minimal")
> > >           # do_image will fail if there are any pending postinsts
> > Whilst this is certainly going to be a touch faster, I believe we
> > do
> > want to test read only rootfs with a larger image like sato to make
> > sure the postinsts really do work with a read only system?
> I don't get it. What would make the test different using a larger
> image?

The large image is more complex, it has more complex post-installs and
more utilities using update alternatives. The areas read-only rootfs
tends to break is in these areas as for example if certain postinstalls
get deferred to "on device at first boot", read-only rootfs breaks.

Cheers,

Richard

Re: [PATCH 1/4] selftest/buildoptions: use a thinner image to test 'read-only-rootfs' feature

[Patrick Ohly]

On Wed, 2017-02-01 at 09:02 -0600, Leonardo Sandoval wrote:
> 
> On 01/31/2017 05:16 PM, Richard Purdie wrote:
> > On Tue, 2017-01-31 at 16:50 -0600,
> > leonardo.sandoval.gonzalez@linux.intel.com wrote:
> >> From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
> >> -        bitbake("core-image-sato")
> >> +        bitbake("core-image-minimal")
> >>           # do_image will fail if there are any pending postinsts
> > Whilst this is certainly going to be a touch faster, I believe we do
> > want to test read only rootfs with a larger image like sato to make
> > sure the postinsts really do work with a read only system?
> 
> I don't get it. What would make the test different using a larger image?

The postinst of each component installed into the image must work
properly in a read-only rootfs configuration. So the test is partly for
image creation, partly for the components, and thus more comprehensive
when using a larger image.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: [PATCH 1/4] selftest/buildoptions: use a thinner image to test 'read-only-rootfs' feature

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 381 bytes --]

On 31 January 2017 at 22:50, <leonardo.sandoval.gonzalez@linux.intel.com>
wrote:

> The minimal is much faster to build that sato, so use the former to test
> read-only feature.
>

What Patrick and Richard said.  In particular, it's things like the
gdk-pixbuf and fontconfig postinsts which we need to ensure are not
delayed, and these are not in minimal images.

Ross

[-- Attachment #2: Type: text/html, Size: 815 bytes --]