* [Qemu-devel] [PATCH v2 1/3] qemu-iotests: export TEST_DIR for non-bash tests
2012-02-29 13:25 [Qemu-devel] [PATCH v2 0/3] qemu-iotests: add image streaming tests Stefan Hajnoczi
@ 2012-02-29 13:25 ` Stefan Hajnoczi
2012-02-29 13:25 ` [Qemu-devel] [PATCH v2 2/3] qemu-iotests: add iotests Python module Stefan Hajnoczi
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Stefan Hajnoczi @ 2012-02-29 13:25 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi
Since qemu-iotests may need to create large image files it is possible
to specify the test directory. The TEST_DIR variable needs to be
exported so non-bash tests can make use of it.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
tests/qemu-iotests/common.config | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.config
index d07f435..a220684 100644
--- a/tests/qemu-iotests/common.config
+++ b/tests/qemu-iotests/common.config
@@ -121,6 +121,8 @@ if [ ! -d "$TEST_DIR" ]; then
exit 1
fi
+export TEST_DIR
+
_readlink()
{
if [ $# -ne 1 ]; then
--
1.7.9
^ permalink raw reply related [flat|nested] 5+ messages in thread* [Qemu-devel] [PATCH v2 2/3] qemu-iotests: add iotests Python module
2012-02-29 13:25 [Qemu-devel] [PATCH v2 0/3] qemu-iotests: add image streaming tests Stefan Hajnoczi
2012-02-29 13:25 ` [Qemu-devel] [PATCH v2 1/3] qemu-iotests: export TEST_DIR for non-bash tests Stefan Hajnoczi
@ 2012-02-29 13:25 ` Stefan Hajnoczi
2012-02-29 13:25 ` [Qemu-devel] [PATCH v2 3/3] test: add image streaming tests Stefan Hajnoczi
2012-02-29 13:35 ` [Qemu-devel] [PATCH v2 0/3] qemu-iotests: " Kevin Wolf
3 siblings, 0 replies; 5+ messages in thread
From: Stefan Hajnoczi @ 2012-02-29 13:25 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi
Block layer tests that involve QMP commands rather than qemu-img or
qemu-io are not well-suited for shell scripting. This patch adds a
Python module which allows tests to be written in Python instead.
The basic API is:
VM - class for launching and interacting with a VM
QMPTestCase - abstract base class for tests that use QMP
qemu_img() - wrapper function for invoking qemu-img
qemu_io() - wrapper function for invoking qemu-io
imgfmt - the image format under test (e.g. qcow2, qed)
test_dir - scratch directory path for temporary files
main() - entry point for running tests
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
tests/qemu-iotests/iotests.py | 164 +++++++++++++++++++++++++++++++++++++++++
1 files changed, 164 insertions(+), 0 deletions(-)
create mode 100644 tests/qemu-iotests/iotests.py
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
new file mode 100644
index 0000000..ec1a86a
--- /dev/null
+++ b/tests/qemu-iotests/iotests.py
@@ -0,0 +1,164 @@
+# Common utilities and Python wrappers for qemu-iotests
+#
+# Copyright (C) 2012 IBM Corp.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import re
+import subprocess
+import unittest
+import sys; sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'QMP'))
+import qmp
+
+__all__ = ['imgfmt', 'imgproto', 'test_dir' 'qemu_img', 'qemu_io',
+ 'VM', 'QMPTestCase', 'notrun', 'main']
+
+# This will not work if arguments or path contain spaces but is necessary if we
+# want to support the override options that ./check supports.
+qemu_img_args = os.environ.get('QEMU_IMG', 'qemu-img').split(' ')
+qemu_io_args = os.environ.get('QEMU_IO', 'qemu-io').split(' ')
+qemu_args = os.environ.get('QEMU', 'qemu').split(' ')
+
+imgfmt = os.environ.get('IMGFMT', 'raw')
+imgproto = os.environ.get('IMGPROTO', 'file')
+test_dir = os.environ.get('TEST_DIR', '/var/tmp')
+
+def qemu_img(*args):
+ '''Run qemu-img and return the exit code'''
+ devnull = open('/dev/null', 'r+')
+ return subprocess.call(qemu_img_args + list(args), stdin=devnull, stdout=devnull)
+
+def qemu_io(*args):
+ '''Run qemu-io and return the stdout data'''
+ args = qemu_io_args + list(args)
+ return subprocess.Popen(args, stdout=subprocess.PIPE).communicate()[0]
+
+class VM(object):
+ '''A QEMU VM'''
+
+ def __init__(self):
+ self._monitor_path = os.path.join(test_dir, 'qemu-mon.%d' % os.getpid())
+ self._qemu_log_path = os.path.join(test_dir, 'qemu-log.%d' % os.getpid())
+ self._args = qemu_args + ['-chardev',
+ 'socket,id=mon,path=' + self._monitor_path,
+ '-mon', 'chardev=mon,mode=control', '-nographic']
+ self._num_drives = 0
+
+ def add_drive(self, path, opts=''):
+ '''Add a virtio-blk drive to the VM'''
+ options = ['if=virtio',
+ 'format=%s' % imgfmt,
+ 'cache=none',
+ 'file=%s' % path,
+ 'id=drive%d' % self._num_drives]
+ if opts:
+ options.append(opts)
+
+ self._args.append('-drive')
+ self._args.append(','.join(options))
+ self._num_drives += 1
+ return self
+
+ def launch(self):
+ '''Launch the VM and establish a QMP connection'''
+ devnull = open('/dev/null', 'rb')
+ qemulog = open(self._qemu_log_path, 'wb')
+ try:
+ self._qmp = qmp.QEMUMonitorProtocol(self._monitor_path, server=True)
+ self._popen = subprocess.Popen(self._args, stdin=devnull, stdout=qemulog,
+ stderr=subprocess.STDOUT)
+ self._qmp.accept()
+ except:
+ os.remove(self._monitor_path)
+ raise
+
+ def shutdown(self):
+ '''Terminate the VM and clean up'''
+ self._qmp.cmd('quit')
+ self._popen.wait()
+ os.remove(self._monitor_path)
+ os.remove(self._qemu_log_path)
+
+ def qmp(self, cmd, **args):
+ '''Invoke a QMP command and return the result dict'''
+ return self._qmp.cmd(cmd, args=args)
+
+ def get_qmp_events(self, wait=False):
+ '''Poll for queued QMP events and return a list of dicts'''
+ events = self._qmp.get_events(wait=wait)
+ self._qmp.clear_events()
+ return events
+
+index_re = re.compile(r'([^\[]+)\[([^\]]+)\]')
+
+class QMPTestCase(unittest.TestCase):
+ '''Abstract base class for QMP test cases'''
+
+ def dictpath(self, d, path):
+ '''Traverse a path in a nested dict'''
+ for component in path.split('/'):
+ m = index_re.match(component)
+ if m:
+ component, idx = m.groups()
+ idx = int(idx)
+
+ if not isinstance(d, dict) or component not in d:
+ self.fail('failed path traversal for "%s" in "%s"' % (path, str(d)))
+ d = d[component]
+
+ if m:
+ if not isinstance(d, list):
+ self.fail('path component "%s" in "%s" is not a list in "%s"' % (component, path, str(d)))
+ try:
+ d = d[idx]
+ except IndexError:
+ self.fail('invalid index "%s" in path "%s" in "%s"' % (idx, path, str(d)))
+ return d
+
+ def assert_qmp(self, d, path, value):
+ '''Assert that the value for a specific path in a QMP dict matches'''
+ result = self.dictpath(d, path)
+ self.assertEqual(result, value, 'values not equal "%s" and "%s"' % (str(result), str(value)))
+
+def notrun(reason):
+ '''Skip this test suite'''
+ # Each test in qemu-iotests has a number ("seq")
+ seq = os.path.basename(sys.argv[0])
+
+ open('%s.notrun' % seq, 'wb').write(reason + '\n')
+ print '%s not run: %s' % (seq, reason)
+ sys.exit(0)
+
+def main(supported_fmts=[]):
+ '''Run tests'''
+
+ if supported_fmts and (imgfmt not in supported_fmts):
+ notrun('not suitable for this image format: %s' % imgfmt)
+
+ # We need to filter out the time taken from the output so that qemu-iotest
+ # can reliably diff the results against master output.
+ import StringIO
+ output = StringIO.StringIO()
+
+ class MyTestRunner(unittest.TextTestRunner):
+ def __init__(self, stream=output, descriptions=True, verbosity=1):
+ unittest.TextTestRunner.__init__(self, stream, descriptions, verbosity)
+
+ # unittest.main() will use sys.exit() so expect a SystemExit exception
+ try:
+ unittest.main(testRunner=MyTestRunner)
+ finally:
+ sys.stderr.write(re.sub(r'Ran (\d+) test[s] in [\d.]+s', r'Ran \1 tests', output.getvalue()))
--
1.7.9
^ permalink raw reply related [flat|nested] 5+ messages in thread* [Qemu-devel] [PATCH v2 3/3] test: add image streaming tests
2012-02-29 13:25 [Qemu-devel] [PATCH v2 0/3] qemu-iotests: add image streaming tests Stefan Hajnoczi
2012-02-29 13:25 ` [Qemu-devel] [PATCH v2 1/3] qemu-iotests: export TEST_DIR for non-bash tests Stefan Hajnoczi
2012-02-29 13:25 ` [Qemu-devel] [PATCH v2 2/3] qemu-iotests: add iotests Python module Stefan Hajnoczi
@ 2012-02-29 13:25 ` Stefan Hajnoczi
2012-02-29 13:35 ` [Qemu-devel] [PATCH v2 0/3] qemu-iotests: " Kevin Wolf
3 siblings, 0 replies; 5+ messages in thread
From: Stefan Hajnoczi @ 2012-02-29 13:25 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi
This patch adds a test suite for the image streaming feature. It
exercises the 'block_stream', 'block_job_cancel', 'block_job_set_speed',
and 'query-block-jobs' QMP commands.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
tests/qemu-iotests/030 | 151 ++++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/030.out | 5 ++
tests/qemu-iotests/group | 1 +
3 files changed, 157 insertions(+), 0 deletions(-)
create mode 100755 tests/qemu-iotests/030
create mode 100644 tests/qemu-iotests/030.out
diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030
new file mode 100755
index 0000000..1faf984
--- /dev/null
+++ b/tests/qemu-iotests/030
@@ -0,0 +1,151 @@
+#!/usr/bin/env python
+#
+# Tests for image streaming.
+#
+# Copyright (C) 2012 IBM Corp.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+from iotests import qemu_img, qemu_io
+
+backing_img = os.path.join(iotests.test_dir, 'backing.img')
+test_img = os.path.join(iotests.test_dir, 'test.img')
+
+class ImageStreamingTestCase(iotests.QMPTestCase):
+ '''Abstract base class for image streaming test cases'''
+
+ def assert_no_active_streams(self):
+ result = self.vm.qmp('query-block-jobs')
+ self.assert_qmp(result, 'return', [])
+
+class TestSingleDrive(ImageStreamingTestCase):
+ image_len = 1 * 1024 * 1024 # MB
+
+ def setUp(self):
+ qemu_img('create', backing_img, str(TestSingleDrive.image_len))
+ qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
+ self.vm = iotests.VM().add_drive(test_img)
+ self.vm.launch()
+
+ def tearDown(self):
+ self.vm.shutdown()
+ os.remove(test_img)
+ os.remove(backing_img)
+
+ def test_stream(self):
+ self.assert_no_active_streams()
+
+ result = self.vm.qmp('block_stream', device='drive0')
+ self.assert_qmp(result, 'return', {})
+
+ completed = False
+ while not completed:
+ for event in self.vm.get_qmp_events(wait=True):
+ if event['event'] == 'BLOCK_JOB_COMPLETED':
+ self.assert_qmp(event, 'data/type', 'stream')
+ self.assert_qmp(event, 'data/device', 'drive0')
+ self.assert_qmp(event, 'data/offset', self.image_len)
+ self.assert_qmp(event, 'data/len', self.image_len)
+ completed = True
+
+ self.assert_no_active_streams()
+
+ self.assertFalse('sectors not allocated' in qemu_io('-c', 'map', test_img),
+ 'image file not fully populated after streaming')
+
+ def test_device_not_found(self):
+ result = self.vm.qmp('block_stream', device='nonexistent')
+ self.assert_qmp(result, 'error/class', 'DeviceNotFound')
+
+class TestStreamStop(ImageStreamingTestCase):
+ image_len = 8 * 1024 * 1024 * 1024 # GB
+
+ def setUp(self):
+ qemu_img('create', backing_img, str(TestStreamStop.image_len))
+ qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
+ self.vm = iotests.VM().add_drive(test_img)
+ self.vm.launch()
+
+ def tearDown(self):
+ self.vm.shutdown()
+ os.remove(test_img)
+ os.remove(backing_img)
+
+ def test_stream_stop(self):
+ import time
+
+ self.assert_no_active_streams()
+
+ result = self.vm.qmp('block_stream', device='drive0')
+ self.assert_qmp(result, 'return', {})
+
+ time.sleep(1)
+ events = self.vm.get_qmp_events(wait=False)
+ self.assertEqual(events, [], 'unexpected QMP event: %s' % events)
+
+ self.vm.qmp('block_job_cancel', device='drive0')
+ self.assert_qmp(result, 'return', {})
+
+ cancelled = False
+ while not cancelled:
+ for event in self.vm.get_qmp_events(wait=True):
+ if event['event'] == 'BLOCK_JOB_CANCELLED':
+ self.assert_qmp(event, 'data/type', 'stream')
+ self.assert_qmp(event, 'data/device', 'drive0')
+ cancelled = True
+
+ self.assert_no_active_streams()
+
+# This is a short performance test which is not run by default.
+# Invoke "IMGFMT=qed ./030 TestSetSpeed.perf_test_set_speed"
+class TestSetSpeed(ImageStreamingTestCase):
+ image_len = 80 * 1024 * 1024 # MB
+
+ def setUp(self):
+ qemu_img('create', backing_img, str(TestSetSpeed.image_len))
+ qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, test_img)
+ self.vm = iotests.VM().add_drive(test_img)
+ self.vm.launch()
+
+ def tearDown(self):
+ self.vm.shutdown()
+ os.remove(test_img)
+ os.remove(backing_img)
+
+ def perf_test_set_speed(self):
+ self.assert_no_active_streams()
+
+ result = self.vm.qmp('block_stream', device='drive0')
+ self.assert_qmp(result, 'return', {})
+
+ result = self.vm.qmp('block_job_set_speed', device='drive0', value=8 * 1024 * 1024)
+ self.assert_qmp(result, 'return', {})
+
+ completed = False
+ while not completed:
+ for event in self.vm.get_qmp_events(wait=True):
+ if event['event'] == 'BLOCK_JOB_COMPLETED':
+ self.assert_qmp(event, 'data/type', 'stream')
+ self.assert_qmp(event, 'data/device', 'drive0')
+ self.assert_qmp(event, 'data/offset', self.image_len)
+ self.assert_qmp(event, 'data/len', self.image_len)
+ completed = True
+
+ self.assert_no_active_streams()
+
+if __name__ == '__main__':
+ iotests.main(supported_fmts=['qcow2', 'qed'])
diff --git a/tests/qemu-iotests/030.out b/tests/qemu-iotests/030.out
new file mode 100644
index 0000000..8d7e996
--- /dev/null
+++ b/tests/qemu-iotests/030.out
@@ -0,0 +1,5 @@
+...
+----------------------------------------------------------------------
+Ran 3 tests
+
+OK
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 0a5c866..fcf869d 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -36,3 +36,4 @@
027 rw auto
028 rw backing auto
029 rw auto
+030 rw auto
--
1.7.9
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [Qemu-devel] [PATCH v2 0/3] qemu-iotests: add image streaming tests
2012-02-29 13:25 [Qemu-devel] [PATCH v2 0/3] qemu-iotests: add image streaming tests Stefan Hajnoczi
` (2 preceding siblings ...)
2012-02-29 13:25 ` [Qemu-devel] [PATCH v2 3/3] test: add image streaming tests Stefan Hajnoczi
@ 2012-02-29 13:35 ` Kevin Wolf
3 siblings, 0 replies; 5+ messages in thread
From: Kevin Wolf @ 2012-02-29 13:35 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: qemu-devel
Am 29.02.2012 14:25, schrieb Stefan Hajnoczi:
> This series adds the image streaming test suite to qemu-iotests. It covers the
> 'block_stream', 'block_job_cancel', 'block_job_set_speed', and
> 'query-block-jobs' QMP interfaces.
>
> Since these tests involve QMP it is no longer convenient to write them in bash.
> Instead these tests are written in Python and make use of the existing
> QMP/qmp.py module. In order to achieve this, it adds an iotests Python module
> that handles interaction with the qemu-iotests framework.
>
> If you want to review using a top-down approach, I suggest reading this series
> backwards, starting from Patch 3 which introduces 030, the image streaming test
> suite.
>
> Or if you like the bottom-up approach:
>
> * Patch 1 exports TEST_DIR from qemu-iotests so test executables can learn the
> temporary directory path.
>
> * Patch 2 adds the iotests.py module, which brings together qemu-iotests,
> unittest, and QEMU in a way that is easy to consume in Python.
>
> * Patch 3 adds 030, the image streaming test suite.
>
> Tests pass successfully with both qcow2 and qed on qemu.git/master.
>
> v2:
> * Python 2.6 compatibility, use public unittest.TextTestRunner [Kevin]
>
> Stefan Hajnoczi (3):
> qemu-iotests: export TEST_DIR for non-bash tests
> qemu-iotests: add iotests Python module
> test: add image streaming tests
>
> tests/qemu-iotests/030 | 151 +++++++++++++++++++++++++++++++++++
> tests/qemu-iotests/030.out | 5 +
> tests/qemu-iotests/common.config | 2 +
> tests/qemu-iotests/group | 1 +
> tests/qemu-iotests/iotests.py | 164 ++++++++++++++++++++++++++++++++++++++
> 5 files changed, 323 insertions(+), 0 deletions(-)
> create mode 100755 tests/qemu-iotests/030
> create mode 100644 tests/qemu-iotests/030.out
> create mode 100644 tests/qemu-iotests/iotests.py
Thanks, applied to the block branch.
Kevin
^ permalink raw reply [flat|nested] 5+ messages in thread