[Qemu-devel] [PATCH v2 0/3] qemu-iotests: add image streaming tests

[Qemu-devel] [PATCH v2 0/3] qemu-iotests: add image streaming tests

[Stefan Hajnoczi]

This series adds the image streaming test suite to qemu-iotests.  It covers the
'block_stream', 'block_job_cancel', 'block_job_set_speed', and
'query-block-jobs' QMP interfaces.

Since these tests involve QMP it is no longer convenient to write them in bash.
Instead these tests are written in Python and make use of the existing
QMP/qmp.py module.  In order to achieve this, it adds an iotests Python module
that handles interaction with the qemu-iotests framework.

If you want to review using a top-down approach, I suggest reading this series
backwards, starting from Patch 3 which introduces 030, the image streaming test
suite.

Or if you like the bottom-up approach:

* Patch 1 exports TEST_DIR from qemu-iotests so test executables can learn the
  temporary directory path.

* Patch 2 adds the iotests.py module, which brings together qemu-iotests,
  unittest, and QEMU in a way that is easy to consume in Python.

* Patch 3 adds 030, the image streaming test suite.

Tests pass successfully with both qcow2 and qed on qemu.git/master.

v2:
 * Python 2.6 compatibility, use public unittest.TextTestRunner [Kevin]

Stefan Hajnoczi (3):
  qemu-iotests: export TEST_DIR for non-bash tests
  qemu-iotests: add iotests Python module
  test: add image streaming tests

 tests/qemu-iotests/030           |  151 +++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/030.out       |    5 +
 tests/qemu-iotests/common.config |    2 +
 tests/qemu-iotests/group         |    1 +
 tests/qemu-iotests/iotests.py    |  164 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 323 insertions(+), 0 deletions(-)
 create mode 100755 tests/qemu-iotests/030
 create mode 100644 tests/qemu-iotests/030.out
 create mode 100644 tests/qemu-iotests/iotests.py

-- 
1.7.9

[Qemu-devel] [PATCH v2 1/3] qemu-iotests: export TEST_DIR for non-bash tests

[Stefan Hajnoczi]

Since qemu-iotests may need to create large image files it is possible
to specify the test directory.  The TEST_DIR variable needs to be
exported so non-bash tests can make use of it.

Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
 tests/qemu-iotests/common.config |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.config
index d07f435..a220684 100644
--- a/tests/qemu-iotests/common.config
+++ b/tests/qemu-iotests/common.config
@@ -121,6 +121,8 @@ if [ ! -d "$TEST_DIR" ]; then
     exit 1
 fi
 
+export TEST_DIR
+
 _readlink()
 {
     if [ $# -ne 1 ]; then
-- 
1.7.9

[Qemu-devel] [PATCH v2 2/3] qemu-iotests: add iotests Python module

[Stefan Hajnoczi]

Block layer tests that involve QMP commands rather than qemu-img or
qemu-io are not well-suited for shell scripting.  This patch adds a
Python module which allows tests to be written in Python instead.

The basic API is:

  VM          - class for launching and interacting with a VM
  QMPTestCase - abstract base class for tests that use QMP
  qemu_img()  - wrapper function for invoking qemu-img
  qemu_io()   - wrapper function for invoking qemu-io
  imgfmt      - the image format under test (e.g. qcow2, qed)
  test_dir    - scratch directory path for temporary files
  main()      - entry point for running tests

Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
 tests/qemu-iotests/iotests.py |  164 +++++++++++++++++++++++++++++++++++++++++
 1 files changed, 164 insertions(+), 0 deletions(-)
 create mode 100644 tests/qemu-iotests/iotests.py

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
new file mode 100644
index 0000000..ec1a86a
--- /dev/null
+++ b/tests/qemu-iotests/iotests.py
@@ -0,0 +1,164 @@
+# Common utilities and Python wrappers for qemu-iotests
+#
+# Copyright (C) 2012 IBM Corp.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import re
+import subprocess
+import unittest
+import sys; sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'QMP'))
+import qmp
+
+__all__ = ['imgfmt', 'imgproto', 'test_dir' 'qemu_img', 'qemu_io',
+           'VM', 'QMPTestCase', 'notrun', 'main']
+
+# This will not work if arguments or path contain spaces but is necessary if we
+# want to support the override options that ./check supports.
+qemu_img_args = os.environ.get('QEMU_IMG', 'qemu-img').split(' ')
+qemu_io_args = os.environ.get('QEMU_IO', 'qemu-io').split(' ')
+qemu_args = os.environ.get('QEMU', 'qemu').split(' ')
+
+imgfmt = os.environ.get('IMGFMT', 'raw')
+imgproto = os.environ.get('IMGPROTO', 'file')
+test_dir = os.environ.get('TEST_DIR', '/var/tmp')
+
+def qemu_img(*args):
+    '''Run qemu-img and return the exit code'''
+    devnull = open('/dev/null', 'r+')
+    return subprocess.call(qemu_img_args + list(args), stdin=devnull, stdout=devnull)
+
+def qemu_io(*args):
+    '''Run qemu-io and return the stdout data'''
+    args = qemu_io_args + list(args)
+    return subprocess.Popen(args, stdout=subprocess.PIPE).communicate()[0]
+
+class VM(object):
+    '''A QEMU VM'''
+
+    def __init__(self):
+        self._monitor_path = os.path.join(test_dir, 'qemu-mon.%d' % os.getpid())
+        self._qemu_log_path = os.path.join(test_dir, 'qemu-log.%d' % os.getpid())
+        self._args = qemu_args + ['-chardev',
+                     'socket,id=mon,path=' + self._monitor_path,
+                     '-mon', 'chardev=mon,mode=control', '-nographic']
+        self._num_drives = 0
+
+    def add_drive(self, path, opts=''):
+        '''Add a virtio-blk drive to the VM'''
+        options = ['if=virtio',
+                   'format=%s' % imgfmt,
+                   'cache=none',
+                   'file=%s' % path,
+                   'id=drive%d' % self._num_drives]
+        if opts:
+            options.append(opts)
+
+        self._args.append('-drive')
+        self._args.append(','.join(options))
+        self._num_drives += 1
+        return self
+
+    def launch(self):
+        '''Launch the VM and establish a QMP connection'''
+        devnull = open('/dev/null', 'rb')
+        qemulog = open(self._qemu_log_path, 'wb')
+        try:
+            self._qmp = qmp.QEMUMonitorProtocol(self._monitor_path, server=True)
+            self._popen = subprocess.Popen(self._args, stdin=devnull, stdout=qemulog,
+                                           stderr=subprocess.STDOUT)
+            self._qmp.accept()
+        except:
+            os.remove(self._monitor_path)
+            raise
+
+    def shutdown(self):
+        '''Terminate the VM and clean up'''
+        self._qmp.cmd('quit')
+        self._popen.wait()
+        os.remove(self._monitor_path)
+        os.remove(self._qemu_log_path)
+
+    def qmp(self, cmd, **args):
+        '''Invoke a QMP command and return the result dict'''
+        return self._qmp.cmd(cmd, args=args)
+
+    def get_qmp_events(self, wait=False):
+        '''Poll for queued QMP events and return a list of dicts'''
+        events = self._qmp.get_events(wait=wait)
+        self._qmp.clear_events()
+        return events
+
+index_re = re.compile(r'([^\[]+)\[([^\]]+)\]')
+
+class QMPTestCase(unittest.TestCase):
+    '''Abstract base class for QMP test cases'''
+
+    def dictpath(self, d, path):
+        '''Traverse a path in a nested dict'''
+        for component in path.split('/'):
+            m = index_re.match(component)
+            if m:
+                component, idx = m.groups()
+                idx = int(idx)
+
+            if not isinstance(d, dict) or component not in d:
+                self.fail('failed path traversal for "%s" in "%s"' % (path, str(d)))
+            d = d[component]
+
+            if m:
+                if not isinstance(d, list):
+                    self.fail('path component "%s" in "%s" is not a list in "%s"' % (component, path, str(d)))
+                try:
+                    d = d[idx]
+                except IndexError:
+                    self.fail('invalid index "%s" in path "%s" in "%s"' % (idx, path, str(d)))
+        return d
+
+    def assert_qmp(self, d, path, value):
+        '''Assert that the value for a specific path in a QMP dict matches'''
+        result = self.dictpath(d, path)
+        self.assertEqual(result, value, 'values not equal "%s" and "%s"' % (str(result), str(value)))
+
+def notrun(reason):
+    '''Skip this test suite'''
+    # Each test in qemu-iotests has a number ("seq")
+    seq = os.path.basename(sys.argv[0])
+
+    open('%s.notrun' % seq, 'wb').write(reason + '\n')
+    print '%s not run: %s' % (seq, reason)
+    sys.exit(0)
+
+def main(supported_fmts=[]):
+    '''Run tests'''
+
+    if supported_fmts and (imgfmt not in supported_fmts):
+        notrun('not suitable for this image format: %s' % imgfmt)
+
+    # We need to filter out the time taken from the output so that qemu-iotest
+    # can reliably diff the results against master output.
+    import StringIO
+    output = StringIO.StringIO()
+
+    class MyTestRunner(unittest.TextTestRunner):
+        def __init__(self, stream=output, descriptions=True, verbosity=1):
+            unittest.TextTestRunner.__init__(self, stream, descriptions, verbosity)
+
+    # unittest.main() will use sys.exit() so expect a SystemExit exception
+    try:
+        unittest.main(testRunner=MyTestRunner)
+    finally:
+        sys.stderr.write(re.sub(r'Ran (\d+) test[s] in [\d.]+s', r'Ran \1 tests', output.getvalue()))
-- 
1.7.9

[Qemu-devel] [PATCH v2 3/3] test: add image streaming tests

[Stefan Hajnoczi]

This patch adds a test suite for the image streaming feature.  It
exercises the 'block_stream', 'block_job_cancel', 'block_job_set_speed',
and 'query-block-jobs' QMP commands.

Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
 tests/qemu-iotests/030     |  151 ++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/030.out |    5 ++
 tests/qemu-iotests/group   |    1 +
 3 files changed, 157 insertions(+), 0 deletions(-)
 create mode 100755 tests/qemu-iotests/030
 create mode 100644 tests/qemu-iotests/030.out

diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030
new file mode 100755
index 0000000..1faf984
--- /dev/null
+++ b/tests/qemu-iotests/030
@@ -0,0 +1,151 @@
+#!/usr/bin/env python
+#
+# Tests for image streaming.
+#
+# Copyright (C) 2012 IBM Corp.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+from iotests import qemu_img, qemu_io
+
+backing_img = os.path.join(iotests.test_dir, 'backing.img')
+test_img = os.path.join(iotests.test_dir, 'test.img')
+
+class ImageStreamingTestCase(iotests.QMPTestCase):
+    '''Abstract base class for image streaming test cases'''
+
+    def assert_no_active_streams(self):
+        result = self.vm.qmp('query-block-jobs')
+        self.assert_qmp(result, 'return', [])
+
+class TestSingleDrive(ImageStreamingTestCase):
+    image_len = 1 * 1024 * 1024 # MB
+
+    def setUp(self):
+        qemu_img('create', backing_img, str(TestSingleDrive.image_len))
+        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
+        self.vm = iotests.VM().add_drive(test_img)
+        self.vm.launch()
+
+    def tearDown(self):
+        self.vm.shutdown()
+        os.remove(test_img)
+        os.remove(backing_img)
+
+    def test_stream(self):
+        self.assert_no_active_streams()
+
+        result = self.vm.qmp('block_stream', device='drive0')
+        self.assert_qmp(result, 'return', {})
+
+        completed = False
+        while not completed:
+            for event in self.vm.get_qmp_events(wait=True):
+                if event['event'] == 'BLOCK_JOB_COMPLETED':
+                    self.assert_qmp(event, 'data/type', 'stream')
+                    self.assert_qmp(event, 'data/device', 'drive0')
+                    self.assert_qmp(event, 'data/offset', self.image_len)
+                    self.assert_qmp(event, 'data/len', self.image_len)
+                    completed = True
+
+        self.assert_no_active_streams()
+
+        self.assertFalse('sectors not allocated' in qemu_io('-c', 'map', test_img),
+                         'image file not fully populated after streaming')
+
+    def test_device_not_found(self):
+        result = self.vm.qmp('block_stream', device='nonexistent')
+        self.assert_qmp(result, 'error/class', 'DeviceNotFound')
+
+class TestStreamStop(ImageStreamingTestCase):
+    image_len = 8 * 1024 * 1024 * 1024 # GB
+
+    def setUp(self):
+        qemu_img('create', backing_img, str(TestStreamStop.image_len))
+        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
+        self.vm = iotests.VM().add_drive(test_img)
+        self.vm.launch()
+
+    def tearDown(self):
+        self.vm.shutdown()
+        os.remove(test_img)
+        os.remove(backing_img)
+
+    def test_stream_stop(self):
+        import time
+
+        self.assert_no_active_streams()
+
+        result = self.vm.qmp('block_stream', device='drive0')
+        self.assert_qmp(result, 'return', {})
+
+        time.sleep(1)
+        events = self.vm.get_qmp_events(wait=False)
+        self.assertEqual(events, [], 'unexpected QMP event: %s' % events)
+
+        self.vm.qmp('block_job_cancel', device='drive0')
+        self.assert_qmp(result, 'return', {})
+
+        cancelled = False
+        while not cancelled:
+            for event in self.vm.get_qmp_events(wait=True):
+                if event['event'] == 'BLOCK_JOB_CANCELLED':
+                    self.assert_qmp(event, 'data/type', 'stream')
+                    self.assert_qmp(event, 'data/device', 'drive0')
+                    cancelled = True
+
+        self.assert_no_active_streams()
+
+# This is a short performance test which is not run by default.
+# Invoke "IMGFMT=qed ./030 TestSetSpeed.perf_test_set_speed"
+class TestSetSpeed(ImageStreamingTestCase):
+    image_len = 80 * 1024 * 1024 # MB
+
+    def setUp(self):
+        qemu_img('create', backing_img, str(TestSetSpeed.image_len))
+        qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
+        self.vm = iotests.VM().add_drive(test_img)
+        self.vm.launch()
+
+    def tearDown(self):
+        self.vm.shutdown()
+        os.remove(test_img)
+        os.remove(backing_img)
+
+    def perf_test_set_speed(self):
+        self.assert_no_active_streams()
+
+        result = self.vm.qmp('block_stream', device='drive0')
+        self.assert_qmp(result, 'return', {})
+
+        result = self.vm.qmp('block_job_set_speed', device='drive0', value=8 * 1024 * 1024)
+        self.assert_qmp(result, 'return', {})
+
+        completed = False
+        while not completed:
+            for event in self.vm.get_qmp_events(wait=True):
+                if event['event'] == 'BLOCK_JOB_COMPLETED':
+                    self.assert_qmp(event, 'data/type', 'stream')
+                    self.assert_qmp(event, 'data/device', 'drive0')
+                    self.assert_qmp(event, 'data/offset', self.image_len)
+                    self.assert_qmp(event, 'data/len', self.image_len)
+                    completed = True
+
+        self.assert_no_active_streams()
+
+if __name__ == '__main__':
+    iotests.main(supported_fmts=['qcow2', 'qed'])
diff --git a/tests/qemu-iotests/030.out b/tests/qemu-iotests/030.out
new file mode 100644
index 0000000..8d7e996
--- /dev/null
+++ b/tests/qemu-iotests/030.out
@@ -0,0 +1,5 @@
+...
+----------------------------------------------------------------------
+Ran 3 tests
+
+OK
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 0a5c866..fcf869d 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -36,3 +36,4 @@
 027 rw auto
 028 rw backing auto
 029 rw auto
+030 rw auto
-- 
1.7.9

Re: [Qemu-devel] [PATCH v2 0/3] qemu-iotests: add image streaming tests

[Kevin Wolf]

Am 29.02.2012 14:25, schrieb Stefan Hajnoczi:
> This series adds the image streaming test suite to qemu-iotests.  It covers the
> 'block_stream', 'block_job_cancel', 'block_job_set_speed', and
> 'query-block-jobs' QMP interfaces.
> 
> Since these tests involve QMP it is no longer convenient to write them in bash.
> Instead these tests are written in Python and make use of the existing
> QMP/qmp.py module.  In order to achieve this, it adds an iotests Python module
> that handles interaction with the qemu-iotests framework.
> 
> If you want to review using a top-down approach, I suggest reading this series
> backwards, starting from Patch 3 which introduces 030, the image streaming test
> suite.
> 
> Or if you like the bottom-up approach:
> 
> * Patch 1 exports TEST_DIR from qemu-iotests so test executables can learn the
>   temporary directory path.
> 
> * Patch 2 adds the iotests.py module, which brings together qemu-iotests,
>   unittest, and QEMU in a way that is easy to consume in Python.
> 
> * Patch 3 adds 030, the image streaming test suite.
> 
> Tests pass successfully with both qcow2 and qed on qemu.git/master.
> 
> v2:
>  * Python 2.6 compatibility, use public unittest.TextTestRunner [Kevin]
> 
> Stefan Hajnoczi (3):
>   qemu-iotests: export TEST_DIR for non-bash tests
>   qemu-iotests: add iotests Python module
>   test: add image streaming tests
> 
>  tests/qemu-iotests/030           |  151 +++++++++++++++++++++++++++++++++++
>  tests/qemu-iotests/030.out       |    5 +
>  tests/qemu-iotests/common.config |    2 +
>  tests/qemu-iotests/group         |    1 +
>  tests/qemu-iotests/iotests.py    |  164 ++++++++++++++++++++++++++++++++++++++
>  5 files changed, 323 insertions(+), 0 deletions(-)
>  create mode 100755 tests/qemu-iotests/030
>  create mode 100644 tests/qemu-iotests/030.out
>  create mode 100644 tests/qemu-iotests/iotests.py

Thanks, applied to the block branch.

Kevin