[PATCH 0/4] toaster: eventreplay tests

[PATCH 0/4] toaster: eventreplay tests

[Ed Bartosh]

Hi,

This patchset adds 2 tests that use eventreplay script to "replay" core-image-minimal
and zlib build events.

Note that tests require event files that are not included into the patchset due to
their big size. Please, have a look at README file for the instructions how to produce
event files.

The following changes since commit 7c3a5891cb4f8c07bb3092dd6270459816ee9a14:

  toaster: tests builds test_core_image_min Clean ups (2016-10-31 11:29:15 +0200)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib ed/toaster/tests/eventreplay
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/tests/eventreplay

Ed Bartosh (4):
  toaster: use current directory if BUILDDIR is not set
  toaster: add eventreplay test case for core-image-minimal
  toaster: add eventreplay test case for zlib
  toaster: add tests/eventreplay/README

 bitbake/lib/toaster/orm/models.py                 |  2 +-
 bitbake/lib/toaster/tests/eventreplay/README      | 22 +++++
 bitbake/lib/toaster/tests/eventreplay/__init__.py | 97 +++++++++++++++++++++++
 3 files changed, 120 insertions(+), 1 deletion(-)
 create mode 100644 bitbake/lib/toaster/tests/eventreplay/README
 create mode 100644 bitbake/lib/toaster/tests/eventreplay/__init__.py

--
Regards,
Ed

[PATCH 1/4] toaster: use current directory if BUILDDIR is not set

[Ed Bartosh]

If BUILDDIR environment variable is not set signal_runbuilds function
throws TypeError as os.getenv('BUILDDIR') returns None:

ERROR: unsupported operand type(s) for +=: 'NoneType' and 'str'
Traceback (most recent call last):
  File "bitbake/lib/bb/ui/toasterui.py", line
391, in main
    buildinfohelper.update_build_information(event, errors, warnings,
taskfailures)
  File "bitbake/lib/bb/ui/buildinfohelper.py",
line 1184, in update_build_information
    self.internal_state['build'], errors, warnings, taskfailures)
  File "bitbake/lib/bb/ui/buildinfohelper.py",
line 238, in update_build_stats_and_outcome
    signal_runbuilds()
  File "bitbake/lib/toaster/orm/models.py", line
1746, in signal_runbuilds
    '.runbuilds.pid')) as pidf:
  File "/usr/lib64/python3.4/posixpath.py", line 82, in join
    path += b
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

Used os.getenv('BUILDIR', '.') to make it always return meaningful
directory path. Current directory '.' will be used if BUILDDIR is
not set.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/toaster/orm/models.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bitbake/lib/toaster/orm/models.py b/bitbake/lib/toaster/orm/models.py
index 9a4db1e..4f8510c 100644
--- a/bitbake/lib/toaster/orm/models.py
+++ b/bitbake/lib/toaster/orm/models.py
@@ -1742,7 +1742,7 @@ def invalidate_cache(**kwargs):
 def signal_runbuilds():
     """Send SIGUSR1 to runbuilds process"""
     try:
-        with open(os.path.join(os.getenv('BUILDDIR'),
+        with open(os.path.join(os.getenv('BUILDDIR', '.'),
                                '.runbuilds.pid')) as pidf:
             os.kill(int(pidf.read()), SIGUSR1)
     except FileNotFoundError:
-- 
2.1.4

[PATCH 2/4] toaster: add eventreplay test case for core-image-minimal

[Ed Bartosh]

Run toaster-eventreplay with core-image-minimal.events and
test if all required packages present in Target_Installed_Package
table.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/toaster/tests/eventreplay/__init__.py | 87 +++++++++++++++++++++++
 1 file changed, 87 insertions(+)
 create mode 100644 bitbake/lib/toaster/tests/eventreplay/__init__.py

diff --git a/bitbake/lib/toaster/tests/eventreplay/__init__.py b/bitbake/lib/toaster/tests/eventreplay/__init__.py
new file mode 100644
index 0000000..792f07d
--- /dev/null
+++ b/bitbake/lib/toaster/tests/eventreplay/__init__.py
@@ -0,0 +1,87 @@
+#! /usr/bin/env python
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# BitBake Toaster Implementation
+#
+# Copyright (C) 2016 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+# Tests were part of openembedded-core oe selftest Authored by: Lucian Musat
+# Ionut Chisanovici, Paul Eggleton and Cristian Iorga
+
+"""
+Test toaster backend by playing build event log files
+using toaster-eventreplay script
+"""
+
+import os
+
+from subprocess import getstatusoutput
+from pathlib import Path
+
+from django.test import TestCase
+
+from orm.models import Target_Installed_Package
+
+class EventReplay(TestCase):
+    """Base class for eventreplay test cases"""
+
+    def setUp(self):
+        """
+        Setup build environment:
+            - set self.script to toaster-eventreplay path
+            - set self.eventplay_dir to the value of EVENTPLAY_DIR env variable
+        """
+        bitbake_dir = Path(__file__.split('lib/toaster')[0])
+        self.script = bitbake_dir /  'bin' / 'toaster-eventreplay'
+        self.assertTrue(self.script.exists(), "%s doesn't exist")
+        self.eventplay_dir = os.getenv("EVENTREPLAY_DIR")
+        self.assertTrue(self.eventplay_dir,
+                        "Environment variable EVENTREPLAY_DIR is not set")
+
+    def _replay(self, eventfile):
+        """Run toaster-eventplay <eventfile>"""
+        eventpath = Path(self.eventplay_dir) / eventfile
+        status, output = getstatusoutput('%s %s' % (self.script, eventpath))
+        if status:
+            print(output)
+
+        self.assertEqual(status, 0)
+
+class CoreImageMinimalEventReplay(EventReplay):
+    """Replay core-image-minimal events"""
+
+    def test_installed_packages(self):
+        """Test if all required packages have been installed"""
+
+        self._replay('core-image-minimal.events')
+
+        # test installed packages
+        packages = sorted(Target_Installed_Package.objects.\
+                          values_list('package__name', flat=True))
+        self.assertEqual(packages, ['base-files', 'base-passwd', 'busybox',
+                                    'busybox-hwclock', 'busybox-syslog',
+                                    'busybox-udhcpc', 'eudev', 'glibc',
+                                    'init-ifupdown', 'initscripts',
+                                    'initscripts-functions', 'kernel-base',
+                                    'kernel-module-uvesafb', 'libkmod',
+                                    'modutils-initscripts', 'netbase',
+                                    'packagegroup-core-boot', 'run-postinsts',
+                                    'sysvinit', 'sysvinit-inittab',
+                                    'sysvinit-pidof', 'udev-cache',
+                                    'update-alternatives-opkg',
+                                    'update-rc.d', 'util-linux-libblkid',
+                                    'util-linux-libuuid', 'v86d', 'zlib'])
-- 
2.1.4

[PATCH 3/4] toaster: add eventreplay test case for zlib

[Ed Bartosh]

Run toaster-eventreplay with zlib.events.
Check if zlib build and package present in Toaster database.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/toaster/tests/eventreplay/__init__.py | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/toaster/tests/eventreplay/__init__.py b/bitbake/lib/toaster/tests/eventreplay/__init__.py
index 792f07d..6956619 100644
--- a/bitbake/lib/toaster/tests/eventreplay/__init__.py
+++ b/bitbake/lib/toaster/tests/eventreplay/__init__.py
@@ -34,7 +34,7 @@ from pathlib import Path
 
 from django.test import TestCase
 
-from orm.models import Target_Installed_Package
+from orm.models import Target_Installed_Package, Package, Build
 
 class EventReplay(TestCase):
     """Base class for eventreplay test cases"""
@@ -85,3 +85,13 @@ class CoreImageMinimalEventReplay(EventReplay):
                                     'update-alternatives-opkg',
                                     'update-rc.d', 'util-linux-libblkid',
                                     'util-linux-libuuid', 'v86d', 'zlib'])
+
+class ZlibEventReplay(EventReplay):
+    """Replay zlib events"""
+
+    def test_replay_zlib(self):
+        """Test if zlib build and package are in the database"""
+        self._replay("zlib.events")
+
+        self.assertEqual(Build.objects.last().target_set.last().target, "zlib")
+        self.assertTrue('zlib' in Package.objects.values_list('name', flat=True))
-- 
2.1.4

[PATCH 4/4] toaster: add tests/eventreplay/README

[Ed Bartosh]

Put instructions on how to prepare event log files
and run eventreplay tests.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/toaster/tests/eventreplay/README | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100644 bitbake/lib/toaster/tests/eventreplay/README

diff --git a/bitbake/lib/toaster/tests/eventreplay/README b/bitbake/lib/toaster/tests/eventreplay/README
new file mode 100644
index 0000000..8c5bb64
--- /dev/null
+++ b/bitbake/lib/toaster/tests/eventreplay/README
@@ -0,0 +1,22 @@
+# Running eventreplay tests
+
+These tests use event log files produced by bitbake <target> -w <event log file>
+You need to have event log files produced before running this tests.
+
+At the moment of writing this document tests use 2 event log files: zlib.events
+and core-image-minimal.events. They're not provided with the tests due to their
+significant size.
+
+Here is how to produce them:
+
+$ . oe-init-build-env
+$ rm -r tmp sstate-cache
+$ bitbake core-image-minimal -w core-image-minimal.events
+$ rm -rf tmp sstate-cache
+$ bitbake zlib -w zlib.events
+
+After that it should be possible to run eventreplay tests this way:
+
+$ EVENTREPLAY_DIR=./ DJANGO_SETTINGS_MODULE=toastermain.settings_test ../bitbake/lib/toaster/manage.py test -v2 tests.eventreplay
+
+Note that environment variable EVENTREPLAY_DIR should point to the directory with event log files.
-- 
2.1.4

Re: [PATCH 4/4] toaster: add tests/eventreplay/README

[Michael Wood]

Thanks. Applied to toaster-next and sent bitbake-devel

On 31/10/16 10:16, Ed Bartosh wrote:
> Put instructions on how to prepare event log files
> and run eventreplay tests.
>
> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> ---
>   bitbake/lib/toaster/tests/eventreplay/README | 22 ++++++++++++++++++++++
>   1 file changed, 22 insertions(+)
>   create mode 100644 bitbake/lib/toaster/tests/eventreplay/README
>
> diff --git a/bitbake/lib/toaster/tests/eventreplay/README b/bitbake/lib/toaster/tests/eventreplay/README
> new file mode 100644
> index 0000000..8c5bb64
> --- /dev/null
> +++ b/bitbake/lib/toaster/tests/eventreplay/README
> @@ -0,0 +1,22 @@
> +# Running eventreplay tests
> +
> +These tests use event log files produced by bitbake <target> -w <event log file>
> +You need to have event log files produced before running this tests.
> +
> +At the moment of writing this document tests use 2 event log files: zlib.events
> +and core-image-minimal.events. They're not provided with the tests due to their
> +significant size.
> +
> +Here is how to produce them:
> +
> +$ . oe-init-build-env
> +$ rm -r tmp sstate-cache
> +$ bitbake core-image-minimal -w core-image-minimal.events
> +$ rm -rf tmp sstate-cache
> +$ bitbake zlib -w zlib.events
> +
> +After that it should be possible to run eventreplay tests this way:
> +
> +$ EVENTREPLAY_DIR=./ DJANGO_SETTINGS_MODULE=toastermain.settings_test ../bitbake/lib/toaster/manage.py test -v2 tests.eventreplay
> +
> +Note that environment variable EVENTREPLAY_DIR should point to the directory with event log files.

[PATCH 0/4] toaster: eventreplay tests

[Brian Avery]

[-- Attachment #1: Type: text/plain, Size: 2109 bytes --]

Hi,

There was interest in knowing how to replay bitbake events through a
toaster instance in an offline mode.  I'm reforwarding the test eventreplay
commit to the list as it has the relevant information.  The README (
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=ed/toaster/tests/eventreplay&id=92dd8d7561657c769d9d3b460cdce6f1f638547f)
has the instructions.

-bavery
an intel employee


---------- Forwarded message ----------
From: Ed Bartosh <ed.bartosh@linux.intel.com>
Date: Mon, Oct 31, 2016 at 3:16 AM
Subject: [Toaster] [PATCH 0/4] toaster: eventreplay tests
To: toaster@yoctoproject.org


Hi,

This patchset adds 2 tests that use eventreplay script to "replay"
core-image-minimal
and zlib build events.

Note that tests require event files that are not included into the patchset
due to
their big size. Please, have a look at README file for the instructions how
to produce
event files.

The following changes since commit 7c3a5891cb4f8c07bb3092dd6270459816ee9a14:

  toaster: tests builds test_core_image_min Clean ups (2016-10-31 11:29:15
+0200)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib ed/toaster/tests/eventreplay
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=
ed/toaster/tests/eventreplay

Ed Bartosh (4):
  toaster: use current directory if BUILDDIR is not set
  toaster: add eventreplay test case for core-image-minimal
  toaster: add eventreplay test case for zlib
  toaster: add tests/eventreplay/README

 bitbake/lib/toaster/orm/models.py                 |  2 +-
 bitbake/lib/toaster/tests/eventreplay/README      | 22 +++++
 bitbake/lib/toaster/tests/eventreplay/__init__.py | 97
+++++++++++++++++++++++
 3 files changed, 120 insertions(+), 1 deletion(-)
 create mode 100644 bitbake/lib/toaster/tests/eventreplay/README
 create mode 100644 bitbake/lib/toaster/tests/eventreplay/__init__.py

--
Regards,
Ed

--
_______________________________________________
toaster mailing list
toaster@yoctoproject.org
https://lists.yoctoproject.org/listinfo/toaster

[-- Attachment #2: Type: text/html, Size: 3240 bytes --]