* [Qemu-devel] [PATCH 0/2] Add qemu-test for automated testing
@ 2008-10-15 19:53 Ryan Harper
2008-10-15 19:53 ` [Qemu-devel] [PATCH 1/2] " Ryan Harper
2008-10-15 19:53 ` [Qemu-devel] [PATCH 2/2] Integrate qemu-test into make system Ryan Harper
0 siblings, 2 replies; 5+ messages in thread
From: Ryan Harper @ 2008-10-15 19:53 UTC (permalink / raw)
To: qemu-devel; +Cc: Anthony Liguori, Ryan Harper, kvm
This patch provides a relatively simple method for exercising various
features of qemu by interacting with the guest via serial and qemu
via the monitor that individual developers can use to validate that
their changes haven't broken feature/function. Such tests
need to be very accessable and easy-to-use to encourge there use by
users/developers. As a bonus, in-tree make test is trivially integrated
into higher-level test frameworks such as autotest.
Using such interfaces the guest requirements are very low (serial login)
and thus we can execute tests across a large number of differing
guests. Writing new test cases should also be fairly simple, for
example, the host_shutdown.py test logic is as follows:
def run_test(vm):
vm.wait_for_boot()
vm.login()
vm.guest_command("echo pressing power button")
output = vm.monitor_command('system_powerdown')
if vm.wait_for_shutdown(): return 1
There is a configuration file (tests/qemu-test/CONFIG) which can be used to
provide per-distro overrides for various functions. The current defaults have
been tested against: RHEL5, openSUSE 11, Ubuntu 8.04, and Fedora 9 w.r.t the
regular expressions used for login, and prompt.
Getting started is relatively simple for someone familiar with
configuring a guest for serial login and boot output via serial. Once
completed, add a file to tests/qemu-test/images like the following:
% cat images/rhel5.2
DISK="/home/rharper/work/images/rhel5.2_x86_64_10G.qcow2"
COMMAND="qemu-system-x86_64 -m 512 -smp 2 -net tap -net nic,model=e1000
-net nic,model=rtl8139 -drive file=${DISK},if=ide,boot=on
-vnc none"
You can omit the COMMAND variable and a default qemu command will be run
instead (see run.sh). For additional guest image or config just add a
new file in the images dir.
To run the tests, at the toplevel run:
% make qemu-test
Some additional parameters can be passed through to run.sh to control how we
execute qemu-test: TESTS, ARGS, and IMAGEDIR. By default, run.sh will execute
each test (*.py) in tests/qemu-test/. If you want to restrict that list to
something more specific, you can pass a space separated list to the make
command:
% make qemu-test TESTS="reboot.py networking.py"
ARGS allows you to specific what parameters are passed to the qemu-system
command. You can either modify the default args in tests/qemu-test/run.sh or
override them via the command line
IMAGEDIR is a relative path where the image input files are located. It
defaults to 'images' in tests/qemu-test, but if you have a specific directory
you want to specify, for example tests/qemu-test/images.i386, this can be done
with:
% make qemu-test IMAGEDIR="images.i386"
Output of the run will be logged to qemu/${TARGET_ARCH}_test_results.out.
This includes all data generated during the run and the results, each test run
will include whether it passed or failed, the testname, and the name of the
image in which the test was run. Some sample output:
% head -n 10 i386-softmmu_test_results.out
Running Test:migrate.py on Image:ubuntu-6.06
Using default command: ../../i386-softmmu/qemu -L ../../pc-bios -m 128 -net tap -net nic,model=e1000 -net nic,model=rtl8139 -snapshot -vnc none -drive file=/home/rharper/work/images/ubuntu_6.06_i386_10G.qcow2,if=ide -boot c
[42949372.960000] Linux version 2.6.15-51-server (buildd@rothera) (gcc version 4.0.3 (Ubuntu 4.0.3-1ubuntu5)) #1 SMP Thu Dec 6 21:37:18 UTC 2007
[42949372.960000] BIOS-provided physical RAM map:
[42949372.960000] BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
[42949372.960000] BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
[42949372.960000] BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
[42949372.960000] BIOS-e820: 0000000000100000 - 0000000007ff0000 (usable)
[42949372.960000] BIOS-e820: 0000000007ff0000 - 0000000008000000 (ACPI data)
[42949372.960000] BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
% tail -n 7 i386-softmmu_test_results.out
0
root@vm1:~#
************************************************************
Results: 2 passed, 0 FAILED
passed: migrate.py:ubuntu-6.06
passed: migrate_networking.py:ubuntu-6.06
************************************************************
^ permalink raw reply [flat|nested] 5+ messages in thread
* [Qemu-devel] [PATCH 1/2] Add qemu-test for automated testing
2008-10-15 19:53 [Qemu-devel] [PATCH 0/2] Add qemu-test for automated testing Ryan Harper
@ 2008-10-15 19:53 ` Ryan Harper
2008-10-15 22:59 ` Paul Brook
2008-10-15 19:53 ` [Qemu-devel] [PATCH 2/2] Integrate qemu-test into make system Ryan Harper
1 sibling, 1 reply; 5+ messages in thread
From: Ryan Harper @ 2008-10-15 19:53 UTC (permalink / raw)
To: qemu-devel; +Cc: Anthony Liguori, Ryan Harper, kvm
This patch places the qemu-test framework and tests into the qemu source tree.
There are a number of components to this patch:
- Python-based framework for interacting with the qemu process and the guest
via serial console and qemu monitor.
- Several intial tests cases for exercising various qemu features/functions
- Bash script (run.sh) for running a set of tests on a set of images and
collecting results.
This is a self-contained patch, the framework, tests and script are
self-reliant, no further qemu integration is needed to realize the value of
automatically testing qemu features against a set of images.
Patch 2/2 provides Makefile integration into qemu to execute qemu-test against
the current build of the tree. This can be done without the Makefile
integration, but simplifies the execution.
Signed-off-by: Ryan Harper <ryanh@us.ibm.com>
diff --git a/tests/qemu-test/CONFIG b/tests/qemu-test/CONFIG
new file mode 100644
index 0000000..0da0947
--- /dev/null
+++ b/tests/qemu-test/CONFIG
@@ -0,0 +1,58 @@
+###############################################################################
+# QEMU-TEST options and defaults
+#
+###############################################################################
+[options]
+debug=1
+logfile=qemu-test.log
+# Uncomment if you want to redirect guest serial output to a file instead
+# of stdout
+#serialfile=console.log
+
+###############################################################################
+# !!!!WARNING!!!!: don't change the defaults, instead add an override for your
+# distro, see below
+###############################################################################
+[defaults]
+username=root
+password=
+login_prompt=^(\S+) login:
+shell_prompt=(%(username)s@)*%(hostname)s.*#\ $
+
+###############################################################################
+# OVERRIDES
+#
+# Distro overrides section, to override the default values as specified in the
+# [defaults] section, add the same option name and a new value, e.g, overriding
+# the default username:
+#
+# [rhel]
+# username=foobar
+#
+#
+# Fields:
+# username:
+# password:
+# login_prompt:
+# shell_prompt:
+# banner:
+# add a regular expression to match the output from your distro's welcome banner
+# (/etc/issue) the option name will be used for overriding the default section.
+# For example, if you add the following option:
+#
+###############################################################################
+
+[rhel]
+banner=^Red Hat Enterprise Linux
+shell_prompt=^\[%(username)s@%(hostname)s.*\]#\ $
+
+[opensuse]
+banner=^Welcome to openSUSE
+shell_prompt=%(hostname)s:.*#\ $
+
+[ubuntu]
+banner=^Ubuntu
+shell_prompt=^%(username)s@%(hostname)s:.*#\ $
+
+[fedora]
+banner=^Fedora
diff --git a/tests/qemu-test/HOWTO b/tests/qemu-test/HOWTO
new file mode 100644
index 0000000..1bac6bc
--- /dev/null
+++ b/tests/qemu-test/HOWTO
@@ -0,0 +1,30 @@
+Configure your Guest
+--------------------
+1. Configure guest to support serial login
+
+
+2. Create an image file in qemu-test/images:
+
+For example:
+
+% cat images/rhel5.2
+DISK="/home/rharper/work/images/rhel5.2_x86_64_10G.qcow2"
+COMMAND="qemu-system-x86_64 -m 512 -smp 2 -net tap -net nic,model=e1000 -net nic,model=rtl8139 -drive file=${DISK},if=ide,boot=on -vnc none"
+
+You can omit the COMMAND variable and a default qemu command will be run instead
+(see run.sh). For each guest image or config add additional files in
+qemu-test/images.
+
+3. Run the test and observe the results:
+
+% make && sudo make test
+<snip massive amount of output>
+************************************************************
+Results: 5 passed, 4 FAILED
+passed: e820_memory.py:fedora-9-x86_64
+passed: host_shutdown.py:fedora-9-x86_64
+passed: networking.py:fedora-9-x86_64
+passed: reboot.py:fedora-9-x86_64
+FAILED: Test:host_reset.py,Image:fedora-9-x86_64
+FAILED: Test:timedrift.py,Image:fedora-9-x86_64
+************************************************************
diff --git a/tests/qemu-test/Makefile b/tests/qemu-test/Makefile
new file mode 100644
index 0000000..76c340f
--- /dev/null
+++ b/tests/qemu-test/Makefile
@@ -0,0 +1,24 @@
+-include ../../config-host.mak
+
+TARGET_ARCH := $(shell grep TARGET_ARCH $(QEMU)/config.mak | sed -e s/TARGET_ARCH=//)
+ifeq ($(TARGET_ARCH), i386)
+QEMU_PROG=qemu$(EXESUF)
+else
+QEMU_PROG=qemu-system-$(TARGET_ARCH)$(EXESUF)
+endif
+
+setup:
+ cd ../../pc-bios && ln -sf ../keymaps
+ chmod a+x ./run.sh
+
+clean:
+ rm -f ../../pc-bios/keymaps
+
+test: all
+
+.PHONY: all test clean setup
+all: setup
+ ./run.sh QEMU="$(QEMU)/$(QEMU_PROG) -L ../../pc-bios" \
+ TESTS="$(TESTS)" ARGS="$(ARGS)" IMAGEDIR="$(IMAGEDIR)" |\
+ tee $(QEMU)_test_results.out
+
diff --git a/tests/qemu-test/THEORY b/tests/qemu-test/THEORY
new file mode 100644
index 0000000..16d5c48
--- /dev/null
+++ b/tests/qemu-test/THEORY
@@ -0,0 +1,4 @@
+Each test should work with as many guests as possible with the least number
+of parameters required.
+
+Minimum guest requirements (only configuration)
diff --git a/tests/qemu-test/TODO b/tests/qemu-test/TODO
new file mode 100644
index 0000000..4a927f8
--- /dev/null
+++ b/tests/qemu-test/TODO
@@ -0,0 +1,33 @@
+* check guest image for commands/utilities needed for each benchmark
+* improve failure detection
+ o oops/soft lockup detection
+ o freeze detection
+ - hard lockup detection (100% cpu usage, min timeout expired)
+ - soft lockup detection (0% cpu usage, min timeout expired)
+ - qemu freeze detection (timeout in monitor command)
+* lots more tests
+ o device validation
+ o disk emu/paravirt verification
+ o pause/resume
+ o savevm/loadvm
+* randomize sockets
+* rewrite networking.py to parse info out of sysfs
+* pass test name to launch for logging
+
+
+#### Completed #####
+2008-08-01
+* write up on how to configure various distros to be ready to run any test
+* redirect serial output to file, print to screen as option
+* detect guest type
+ o automagically select appropriate defaults for prompt
+2008-07-23
+* make check|test integration into kvm-userspace tree
+* fedora guest support
+
+2008-07-16
+* integrate config file
+ o support per-distro overrides
+* ubuntu guest support
+* rhel5 guest support
+* opensuse geust support
diff --git a/tests/qemu-test/boottime.py b/tests/qemu-test/boottime.py
new file mode 100644
index 0000000..f2b2020
--- /dev/null
+++ b/tests/qemu-test/boottime.py
@@ -0,0 +1,38 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <ryanh@us.ibm.com
+
+from qemu.test import launch, TimeoutException
+import sys, time
+
+def run_test(vm):
+ start = time.time()
+ vm.wait_for_boot()
+ end = time.time()
+ vm.login()
+ vm.shutdown()
+ vm.wait_for_shutdown()
+ print '\n************'
+ print 'Time to boot: %s seconds'%(end-start)
+ print '************'
+
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm)
+ finally:
+ vm.quit()
+
+ return err
+
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/e820_memory.py b/tests/qemu-test/e820_memory.py
new file mode 100644
index 0000000..028e5bb
--- /dev/null
+++ b/tests/qemu-test/e820_memory.py
@@ -0,0 +1,117 @@
+# Copyright IBM Corp. 2008
+# Authors: Anthony Liguori <aliguori@us.ibm.com>
+# : Ryan Harper <ryanh@us.ibm.com
+#
+
+from qemu.test import launch, TimeoutException
+import sys
+import ConfigParser, os
+
+
+def check_table_entry(e820, entry, field, value):
+ v1 = e820[entry][field]
+ v2 = value
+ # third field is a string, so strip to trim newlines and such
+ if field == 2:
+ v1 = v1.strip()
+ v2 = v2.strip()
+ if v1 != v2:
+ print 'e820 entry %d field %d should be "%s", found [%s]' %(
+ entry, field, str(value), e820[entry][field])
+ return 1
+ print 'e820[%d][%d] OK' %(entry, field)
+ return 0
+
+
+def compare_table_entry(e820_a, e820_b, entry, field):
+ v1 = e820_a[entry][field]
+ v2 = e820_b[entry][field]
+ # third field is a string, so strip to trim newlines and such
+ if field == 2:
+ v1 = v1.strip()
+ v2 = v2.strip()
+ if v1 != v2:
+ print 'e820 table entry %d field %d do not match' %(entry, field)
+ print '[%s] != [%s]' %(v1, v2)
+ return 1
+
+ print 'e820[%d][%d] match' %(entry, field)
+ return 0
+
+
+def run_test(vm, memory_size):
+ vm.wait_for_boot()
+
+ vm.login()
+
+ e820 = []
+ output = vm.guest_command('dmesg | grep BIOS-e820')
+ for line in output.split('\n'):
+ # handle timestamps in dmesg
+ if line.startswith('['):
+ _, line = line.split('] ', 1)
+
+ words = line.split(' ', 5)
+ start = long('0x%s' % words[2], 0)
+ end = long('0x%s' % words[4], 0)
+ e820.append((start, end, words[5]))
+
+ if len(e820) != 6:
+ print 'Unusual number of e820 entries: %d' % len(e820)
+ return 1
+
+ e820_static = [(0x00000, 0x09fc00, '(usable)'),
+ (0x9fc00, 0x0a0000, '(reserved)'),
+ (0xe8000, 0x100000, '(reserved)')]
+
+ for i in range(3):
+ if compare_table_entry(e820_static, e820, i, 0): return 1
+ if compare_table_entry(e820_static, e820, i, 1): return 1
+ if compare_table_entry(e820_static, e820, i, 2): return 1
+
+ if check_table_entry(e820, 3, 0, 0x100000): return 1
+ if check_table_entry(e820, 3, 2, '(usable)'): return 1
+ if check_table_entry(e820, 4, 2, '(ACPI data)'): return 1
+
+ if (e820[4][1] - e820[4][0]) != (64 << 10):
+ print 'e820 entry 4 has invalid size'
+ return 1
+
+ if e820[4][1] != memory_size:
+ print 'e820 entry 4 has invalid start address'
+ return 1
+
+ # KVM e820 5, 0: 0xffbd000
+ # QEMU e820 5, 0: 0xffc0000
+ if check_table_entry(e820, 5, 0, 0xfffbd000):
+ if check_table_entry(e820, 5, 0, 0xfffc0000): return 1
+ if check_table_entry(e820, 5, 1, 0x100000000): return 1
+ if check_table_entry(e820, 5, 2, '(reserved)'): return 1
+
+ return 0
+
+
+def main(args):
+ memory_size = 128 << 20
+ for i in range(len(args)):
+ if args[i] == '-m':
+ memory_size = long(args[i + 1]) << 20
+ break
+
+ vm = launch(*args)
+ err = 1
+ try:
+ err = run_test(vm, memory_size)
+ vm.shutdown()
+ vm.wait_for_shutdown()
+ finally:
+ vm.quit()
+
+ return err
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/host_reset.py b/tests/qemu-test/host_reset.py
new file mode 100644
index 0000000..6a5e630
--- /dev/null
+++ b/tests/qemu-test/host_reset.py
@@ -0,0 +1,39 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <ryanh@us.ibm.com
+#
+
+from qemu.test import launch, TimeoutException
+import sys
+
+def run_test(vm):
+ vm.wait_for_boot()
+ vm.login()
+ vm.guest_command("echo pressing reset button")
+ output = vm.monitor_command('system_reset')
+ vm.wait_for_restart()
+ vm.wait_for_boot()
+ vm.login()
+
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm)
+ vm.shutdown()
+ vm.wait_for_shutdown()
+ finally:
+ vm.quit()
+
+ return err
+
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/host_shutdown.py b/tests/qemu-test/host_shutdown.py
new file mode 100644
index 0000000..9b180c1
--- /dev/null
+++ b/tests/qemu-test/host_shutdown.py
@@ -0,0 +1,34 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <ryanh@us.ibm.com
+
+from qemu.test import launch, TimeoutException
+import sys
+
+def run_test(vm):
+ vm.wait_for_boot()
+ vm.login()
+ vm.guest_command("echo pressing power button")
+ output = vm.monitor_command('system_powerdown')
+ if vm.wait_for_shutdown(): return 1
+
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm)
+ finally:
+ vm.quit()
+
+ return err
+
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/iperf.py b/tests/qemu-test/iperf.py
new file mode 100644
index 0000000..0e102c4
--- /dev/null
+++ b/tests/qemu-test/iperf.py
@@ -0,0 +1,75 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <ryanh@us.ibm.com
+#
+
+from qemu.test import launch, TimeoutException
+import sys, os.path, popen2
+
+
+def run_test(vm, args):
+ configured_nics = []
+ iperf_url = "http://internap.dl.sourceforge.net/sourceforge/iperf/iperf-2.0.4.tar.gz"
+ build_iperf = "cd /tmp && wget %s && tar xzf iperf-2.0.4.tar.gz && cd iperf-2.0.4 && ./configure && make"%(iperf_url)
+ iperf_bin = "/tmp/iperf-2.0.4/src/iperf"
+ iperf_port = "12345"
+
+ vm.wait_for_boot()
+ vm.login()
+
+ for i in range(len(args)):
+ if args[i].startswith('nic') and 'model=' in args[i]:
+ configured_nics.append(args[i].split('model=')[1].split(',')[0])
+
+ if len(configured_nics) < 1:
+ print "this test requires a configured nic"
+ print "try again with: '-net nic,model=<ethernet device>'"
+ return 1
+
+ gw_info = vm.get_gateway_info()
+ dev_info = vm.get_netinfo(gw_info['device'])
+
+ output = vm.guest_command(build_iperf)
+ output = vm.guest_command('%s --help'%(iperf_bin))
+ output = vm.guest_command('%s -D -s -p %s; echo Launching iperf Daemon'%(iperf_bin, iperf_port))
+
+ vm.log('building iperf on host')
+ (output, input) = popen2.popen2(build_iperf)
+ # dump output on console for user
+ for l in output.readlines():
+ print "host: %s" %(l.strip())
+
+ print "host: running benchmark"
+ results = []
+ for x in range(0,3):
+ cmd = '%s -c %s -p %s' %(iperf_bin, dev_info['address'], iperf_port)
+ vm.log('running host command: %s'%(cmd))
+ (output, input) = popen2.popen2(cmd)
+ o = output.readlines()
+ r = "".join(o)
+ results.append(r)
+ print "host: benchmark complete"
+
+ vm.shutdown()
+ vm.wait_for_shutdown()
+
+ print "".join(results)
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm, args)
+ finally:
+ vm.quit()
+
+ return err
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/migrate.py b/tests/qemu-test/migrate.py
new file mode 100644
index 0000000..270097b
--- /dev/null
+++ b/tests/qemu-test/migrate.py
@@ -0,0 +1,67 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <ryanh@us.ibm.com
+# : Anthony Liguori <aliguori@us.ibm.com>
+
+from qemu.test import launch, TimeoutException
+import sys, time
+
+def test_alive(vm):
+ rv = vm.guest_command("true; echo $?")
+ if rv != "0":
+ return False
+
+ return True
+
+def main(args):
+ A = launch(*args)
+ A.wait_for_boot()
+ A.login()
+ B = ""
+
+ try:
+ for i in range(10):
+ B = launch(*(args + ['-incoming', 'tcp:localhost:1025']))
+ A.guest_command('echo ------- migration %s -------' %(i))
+
+ B.booted = A.booted
+ B.logged_in = A.logged_in
+ B.hostname = A.hostname
+ B.username = A.username
+ B.password = A.password
+ B.prompt = A.prompt
+
+ if not test_alive(A):
+ print 'Alive test failed before migration'
+ A.quit()
+ B.quit()
+ return 1
+
+ # migrate from A to B
+ A.guest_command('echo starting migration')
+ A.monitor_command('migrate tcp:localhost:1025')
+ A.quit()
+ # test if guest is still alive
+ B.guest_command('echo migration complete')
+ if not test_alive(B):
+ B.log('Alive test failed after migration')
+ B.quit()
+ return 1
+
+ # swap A and B, A is now origin, B migration target
+ A = B
+ B = None
+
+ finally:
+ if A:
+ A.quit()
+ if B:
+ B.quit()
+
+ return 0
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/migrate_networking.py b/tests/qemu-test/migrate_networking.py
new file mode 100644
index 0000000..64f7da4
--- /dev/null
+++ b/tests/qemu-test/migrate_networking.py
@@ -0,0 +1,73 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <ryanh@us.ibm.com
+# : Anthony Liguori <aliguori@us.ibm.com>
+
+from qemu.test import launch, TimeoutException
+import sys, time
+
+def test_ping(vm):
+ gw_info = vm.get_gateway_info()
+ if vm.ping(gw_info['address']):
+ return False
+
+ return True
+
+def main(args):
+ A = launch(*args)
+ A.wait_for_boot()
+ A.login()
+ B = ""
+
+ try:
+ for i in range(10):
+ B = launch(*(args + ['-incoming', 'tcp:localhost:1025']))
+ A.guest_command('echo ------- migration %s -------' %(i))
+
+ B.booted = A.booted
+ B.logged_in = A.logged_in
+ B.hostname = A.hostname
+ B.username = A.username
+ B.password = A.password
+ B.prompt = A.prompt
+
+ if not test_ping(A):
+ print 'Ping test failed before migration'
+ A.quit()
+ B.quit()
+ return 1
+
+ # migrate from A to B
+ A.guest_command('echo starting migration')
+ A.monitor_command('migrate tcp:localhost:1025')
+ A.quit()
+ B.guest_command('echo migration complete')
+
+ # after migration, it takes a bit for arp tables to sync
+ B.guest_command('echo waiting for arp to sync')
+ for x in range(0,10):
+ B.guest_command('arp -a && sleep 1')
+
+ # test outgoing network connection
+ if not test_ping(B):
+ B.log('Ping test failed after migration')
+ B.quit()
+ return 1
+
+ # swap A and B, A is now origin, B migration target
+ A = B
+ B = None
+
+ finally:
+ if A:
+ A.quit()
+ if B:
+ B.quit()
+
+ return 0
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/networking.py b/tests/qemu-test/networking.py
new file mode 100644
index 0000000..d0ee2bd
--- /dev/null
+++ b/tests/qemu-test/networking.py
@@ -0,0 +1,80 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <ryanh@us.ibm.com
+#
+
+from qemu.test import launch, TimeoutException
+import sys
+
+def run_test(vm, args):
+ configured_nics = []
+ detected_nics = []
+ nic_static = { 'Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139 C+ (rev 20)': 'rtl8319',
+ 'Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 20)': 'rtl8319',
+ 'Intel Corporation 82540EM Gigabit Ethernet Controller (rev 03)': 'e1000' }
+ drivers_static = { 'e1000': 'e1000', '8139cp': 'rtl8139' }
+ devices = []
+
+ vm.wait_for_boot()
+ vm.login()
+
+ for i in range(len(args)):
+ if args[i].startswith('nic') and 'model=' in args[i]:
+ configured_nics.append(args[i].split('model=')[1].split(',')[0])
+
+ if len(configured_nics) < 1:
+ print "this test requires a configured nic"
+ print "try again with: '-net nic,model=<ethernet device>'"
+ return 1
+
+ output = vm.guest_command('lspci -v | grep Ethernet')
+ for line in output.split('\n'):
+ device = " ".join(line.split()[3:]).replace('\r','')
+ if device in nic_static.keys():
+ detected_nics.append(nic_static[device])
+
+ if len(detected_nics) != len(configured_nics):
+ print 'Failed to detect all configured nics'
+ print 'Configured nics: %s' %(configured_nics)
+ print 'Detected nics: %s' %(detected_nics)
+ return 1
+
+ # build interface to driver mapping array
+ output = vm.guest_command('for d in `ls -1 /sys/class/net/ | grep eth`; do driver=$(basename `readlink /sys/class/net/$d/device/driver/module`); echo ${driver}:$d; done');
+ for line in output.split('\n'):
+ (driver,device) = line.split(':')
+ device = device.replace('\r','').replace('\n','')
+ devices.append((device,driver))
+
+ gw_info = vm.get_gateway_info()
+
+ # if device is configured and connected to a gateway, do a ping check
+ # FIXME multi-gateway setups not tested
+ for (device,driver) in devices:
+ dev_info = vm.get_netinfo(device)
+ if dev_info['state'] == 'up':
+ if vm.ping(gw_info['address'], iface=device): return 1
+ else:
+ print "device %s, driver %s not enabled in guest, skipping..." %(device, driver)
+
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm, args)
+ vm.shutdown()
+ vm.wait_for_shutdown()
+ finally:
+ vm.quit()
+
+ return err
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/qemu/__init__.py b/tests/qemu-test/qemu/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qemu-test/qemu/test.py b/tests/qemu-test/qemu/test.py
new file mode 100644
index 0000000..d7d19aa
--- /dev/null
+++ b/tests/qemu-test/qemu/test.py
@@ -0,0 +1,433 @@
+# Copyright IBM Corp. 2008
+# Authors: Anthony Liguori <aliguori@us.ibm.com>
+# : Ryan Harper <ryanh@us.ibm.com
+#
+
+import os, sys, socket, re, time, select, ConfigParser
+
+class TimeoutException(Exception):
+ def __init__(self, message):
+ Exception.__init__(self)
+ self.message = message
+
+
+class QEMUInstance(object):
+ def __init__(self, pid, serial_fd, monitor_fd, configfile=None):
+ self.configfile = None
+ self.DEBUG = None
+ self.logfile = None
+ self.logfd = None
+ self.pid = pid
+ self.serial = serial_fd
+ self.serialfile = None
+ self.outfd = sys.stdout
+ self.monitor = monitor_fd
+ self.booted = False
+ self.logged_in = False
+ self.distro = None
+ self.login_prompt = None
+ self.shell_prompt = None
+ self.username = None
+ self.hostname = None
+ self.prompt = None
+
+ # parse config file, check if custom path has been provided
+ if configfile:
+ self.configfile = configfile
+ else:
+ # look for CONFIG file in current dir
+ self.configfile = os.path.join(os.getcwd(),'CONFIG')
+
+ # fire up the parser
+ try:
+ self.config = ConfigParser.ConfigParser()
+ self.config.readfp(open(self.configfile))
+ except Exception, e:
+ print 'Failed to parse config: %s'%(self.configfile)
+ print e.message
+ self.quit()
+
+ # fetch common options from config
+ if self.config.has_option("options", "debug"):
+ self.DEBUG = self.config.get("options", "debug")
+ if self.config.has_option("options", "logfile"):
+ self.logfile = self.config.get("options", "logfile")
+ if self.config.has_option("options", "serialfile"):
+ self.serialfile = self.config.get("options", "serialfile")
+
+ # dump header if we're debuggin
+ if self.DEBUG:
+ self.logfd = open(self.logfile, "a")
+ self.log("****************************************************************")
+ self.log("qemu-test v0.1 init")
+ self.log("****************************************************************")
+
+ if self.serialfile:
+ self.outfd = open(self.serialfile, "a")
+ self.log("Writing serial output to %s"%(self.serialfile))
+
+
+
+ # wait for qemu monitor to come up -- ensure it's working
+ self.wait_for_monitor_prompt()
+
+
+ def log(self, message):
+ if self.DEBUG:
+ self.logfd.write('[PID=%s][%s] %s\n' %(self.pid, time.time(), message))
+ self.logfd.flush()
+
+ ###########################################################
+ # waiting methods
+ ###########################################################
+
+ # base method which reads from the serial line checking to see
+ # the current input matches a designated regular expression
+ # also provide two timers, total (timeout) and activity timeout
+ # (min_activity). The latter is specified when the caller
+ # wants to ensure some output is being generated with some
+ # frequency. Return 'message' in the exception if either
+ # timer fails
+ def wait_for_rexpr(self, rexpr, timeout=None, min_activity=None,
+ message=None):
+ output = ''
+ m = None
+ self.log("wait_for_rexpr(%s) ->"%(rexpr))
+
+ if message == None:
+ message = "rexpr `%s'\n" % rexpr
+
+ if timeout != None:
+ end_time = time.time() + timeout
+ else:
+ end_time = None
+
+ if min_activity == None:
+ min_activity = timeout
+
+ while m == None:
+ now = time.time()
+ if end_time != None and end_time < now:
+ raise TimeoutException(message)
+
+ if end_time != None or min_activity != None:
+ if end_time == None:
+ wait = min_activity
+ else:
+ wait = min(min_activity, end_time - now)
+
+ rdfds, _, __ = select.select([self.serial], [], [], wait)
+ if self.serial not in rdfds:
+ raise TimeoutException(message)
+
+ ch = self.serial.recv(1)
+ if ch == '':
+ break
+ self.outfd.write(ch)
+ self.outfd.flush()
+ output += ch
+ if output.find('\n') != -1:
+ _, line = output.rsplit('\n', 1)
+ #self.log("searching line: [%s] -> [%s]"%(rexpr, line))
+ m = re.search(rexpr, line)
+ self.log("wait_for_rexpr() <-")
+ return output, m
+
+
+ def wait_for_monitor_prompt(self):
+ info = ''
+ self.log("wait_for_monitor_prompt() ->")
+ while not info.endswith('\n(qemu) '):
+ ch = self.monitor.recv(1)
+ if ch == '':
+ break
+ #self.log("got: %s"%(ch))
+ info += ch
+
+ self.log("wait_for_monitor_prompt() <-")
+ return info
+
+
+ def wait_for_linux_password(self, rexpr='^Password: ', timeout=None):
+ self.wait_for_rexpr(rexpr, timeout=timeout)
+
+
+ def wait_for_boot(self, timeout=None):
+ try:
+ self.log("wait_for_boot() ->")
+ self.detect_distro()
+ self.log("after distro detect")
+ if not self.login_prompt:
+ self.login_prompt = self.getconfig("login_prompt")
+ output, m = self.wait_for_rexpr(self.login_prompt, timeout, min_activity=120,
+ message="Linux login prompt")
+ self.hostname = m.group(1)
+ self.booted = True
+ self.log("wait_for_boot() <-")
+ except TimeoutException, e:
+ print "wait_for_boot error: %s" % e.message
+ self.log("wait_for_boot error: %s" % e.message)
+
+
+ def detect_distro(self, timeout=None):
+ self.log("detect_distro() ->")
+ if self.distro == None:
+ match_map = []
+ self.log("distro not set yet")
+ # list potential distros to check for based on config overrides
+ distros = filter(lambda x: self.config.has_option(x, 'banner'),
+ self.config.sections())
+ self.log("distros in config: %s"%(distros))
+
+ # keep a position mapping
+ for d in distros:
+ match_map.append(d)
+
+ banner = "|".join(map(lambda y: "(%s)" % self.config.get(y, 'banner',
+ raw=True), distros))
+ self.log("distro rexpr: [%s]"%(banner))
+ try:
+ output, m = self.wait_for_rexpr(banner, timeout=None, min_activity=120,
+ message="Detect distro banner")
+
+ # groups returns a tuple of which parts of the regular expression
+ # were involved in the match, extracting which group was
+ # not None will give us an index into our regrexpress for each
+ # distro and we can determine which distro we detected
+ matchlist = list(m.groups())
+ for x in range(len(matchlist)):
+ if matchlist[x] != None:
+ self.distro = match_map[x]
+
+ self.log("detected distro: %s"%(self.distro))
+ except TimeoutException, e:
+ self.log("waiting for distro rexpr FAIL: %s"%(e.message))
+ self.distro = 'defaults'
+ # kick the console to spit out a new login prompt
+ # as we probably ate it looking for a banner
+ self.send_to_guest("\n")
+
+
+ self.log("detect_distro() <-")
+
+ def wait_for_shutdown(self, rexpr='(^Power down.|^System halted.)', timeout=360):
+ output, m = self.wait_for_rexpr(rexpr, timeout, min_activity=120,
+ message="Linux shutdown message")
+ self.booted = False
+
+
+ def wait_for_restart(self, rexpr='.*Restarting system.$', timeout=120):
+ try:
+ output, m = self.wait_for_rexpr(rexpr, timeout, min_activity=10,
+ message="Linux shutdown message")
+ except TimeoutException, e:
+ self.log("WARN: didn't detect system restart")
+ pass
+ self.booted = False
+
+
+ def wait_for_quit(self):
+ self.log("wait_for_quit: waitpid() on PID=%d"%(self.pid))
+ os.waitpid(self.pid, 0)
+
+
+ def getconfig(self, field):
+ def __getconfig(self, section, field):
+ self.log("getconfig: fetching section:%s field:%s "%(section,
+ field))
+ try:
+ return self.config.get(section, field, raw=True)
+ except Exception, e:
+ self.log("getconfig: %s"%(e.message))
+ pass
+ return None
+
+ self.log("getconfig() ->")
+ # try looking in distro override section, otherwise defaults
+ value = __getconfig(self, self.distro, field)
+ if value == None:
+ value = __getconfig(self, "defaults", field)
+
+ self.log("getconfig() <-")
+ return value
+
+
+ def login(self):
+ self.log("login() ->")
+ if not self.prompt:
+ username = self.getconfig("username")
+ password = self.getconfig("password")
+ prompt = self.getconfig("shell_prompt")
+
+ if username != 'root':
+ raise Exception("login requires username=root");
+
+ if not self.booted:
+ self.wait_for_boot()
+
+ self.username = username
+ self.password = password
+ self.prompt = prompt % {'username': username, 'hostname': self.hostname}
+
+
+ # XXX: maybe I should check for self.booted here as well
+ self.send_to_guest('%s\n' % self.username)
+ self.wait_for_rexpr('^Password: ')
+ # There seems to be a race with the password entry
+ time.sleep(1)
+ self.send_to_guest('%s\n' % self.password)
+ self.wait_for_rexpr(self.prompt, timeout=10)
+ self.logged_in = True
+
+
+ def send_to_guest(self, data):
+ self.serial.sendall(data)
+
+
+ def __sanitize(self, string):
+ return string.replace('\r','')
+
+
+ # common networking helpers
+ def get_gateway_info(self):
+ cmd = "route -n | grep ^0 | awk '{print $8\":\"$2}'"
+ output = self.__sanitize(self.guest_command(cmd))
+ # FIXME: there may be multiple gateways, we're only going to look at the first
+ # one
+ output = output.split('\n')[0].split(':')
+ return { 'device':output[0], 'address': output[1] }
+
+
+ def get_netinfo(self, devicename):
+ d = self.__sanitize(devicename)
+ netinfo = {}
+ c = "ifconfig %s | grep [A-Za-z]" %(d)
+ ifconfig_raw = self.guest_command(c).split('\n')
+ if 'error' in ifconfig_raw:
+ return netinfo
+
+ netinfo['device'] = d
+ netinfo['macaddr'] = ifconfig_raw[0].split()[-1:]
+ #FIXME: counting # of lines of ifconfig output isn't a good way
+ # to determine if the device is configured
+ if len(ifconfig_raw) >= 8:
+ netinfo['address'] = ifconfig_raw[1].split()[1].split(':')[1]
+ netinfo['netmask'] = ifconfig_raw[1].split()[-1:][0].split(':')[1]
+ if 'UP' in ifconfig_raw[3]:
+ netinfo['state'] = 'up'
+ else:
+ netinfo['state'] = 'down'
+ netinfo['mtu'] = ifconfig_raw[3].split()[4].split(':')[1]
+ else:
+ # device not configured
+ netinfo['state'] = 'down'
+
+ return netinfo
+
+
+ def ping(self, address, iface=None):
+ address = self.__sanitize(address)
+ cmd = "ping -c 5 %s -w 10" %(address)
+ if iface:
+ iface = self.__sanitize(iface)
+ cmd = "%s -I %s" %(cmd, iface)
+ cmd = "%s; echo $?" %(cmd)
+ raw = self.guest_command(cmd).split('\n')
+ # extract the return code, dropping any empty entries from the list
+ rc = filter(lambda x: len(x) > 0, raw)[-1]
+ return int(rc)
+
+
+ def shutdown(self):
+ self.send_to_guest('halt\n')
+
+
+ def quit(self):
+ # might be racy with guest shutting down, so catch broken pipe
+ # exceptions
+ try:
+ self.monitor_command("quit")
+ except Exception, e:
+ self.log("exception: %s"%(e.message))
+ pass
+ finally:
+ self.wait_for_quit()
+
+ def guest_command(self, command, timeout=300):
+ self.log("guest_command(command=%s) ->"%(command))
+ self.send_to_guest('%s\n' % command)
+ output, _ = self.wait_for_rexpr('%s' % self.prompt, timeout,
+ message="Guest command: %s"%(command))
+ try:
+ output, _ = output.rsplit('\n', 1)
+ _, output = output.split('\n', 1)
+ except Exception, e:
+ pass
+ self.log("guest_command() <-")
+ return output.rsplit('\r', 1)[0]
+
+
+ def monitor_command(self, command):
+ self.log("monitor_command(command=%s) ->"%(command))
+ self.monitor.sendall('%s\n' % command)
+ info = self.wait_for_monitor_prompt()
+ self.log("info = %s"%(info))
+ index = info.find('\n')
+ if index != -1:
+ info = info[index+1:]
+ index = info.rfind('\n')
+ if index != -1:
+ info = info[0:index]
+ self.log("monitor_command() <-")
+ return info
+
+
+def launch(executable, *args):
+ def __extract_config(args):
+ if 'config=' in args:
+ configfile = args.split("config=")[1].split()[0]
+ newargs = " ".join(args.split("config=")[1].split()[1:])
+ return (configfile, newargs)
+
+ return (None, args)
+
+
+ (config, args) = __extract_config(args)
+
+ serial_path = os.tmpnam()
+ monitor_path = os.tmpnam()
+
+ try:
+ os.unlink(serial_path)
+ except Exception, e:
+ pass
+ try:
+ os.unlink(monitor_path)
+ except Exception, e:
+ pass
+
+ serial = socket.socket(socket.AF_UNIX)
+ serial.bind(serial_path)
+ serial.listen(1)
+
+ monitor = socket.socket(socket.AF_UNIX)
+ monitor.bind(monitor_path)
+ monitor.listen(1)
+
+ pid = os.fork()
+ if pid == 0:
+ os.execvp(executable, (executable, '-serial', 'unix:%s' % serial_path,
+ '-monitor', 'unix:%s' % monitor_path) + args)
+ sys.exit(1)
+
+ serial_fd, _ = serial.accept()
+ monitor_fd, _ = monitor.accept()
+
+ os.unlink(serial_path)
+ os.unlink(monitor_path)
+
+ serial.close()
+ monitor.close()
+
+ return QEMUInstance(pid, serial_fd, monitor_fd, config)
+
diff --git a/tests/qemu-test/reboot.py b/tests/qemu-test/reboot.py
new file mode 100644
index 0000000..6585c01
--- /dev/null
+++ b/tests/qemu-test/reboot.py
@@ -0,0 +1,42 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <ryanh@us.ibm.com
+#
+
+from qemu.test import launch, TimeoutException
+import sys
+
+def run_test(vm):
+ reboots=10
+ vm.wait_for_boot()
+ vm.login()
+
+ for x in range(1,reboots+1):
+ # background the reboot command so we match the prompt string
+ vm.guest_command("echo Reboot %s; reboot&"%(x))
+ vm.wait_for_restart()
+ vm.wait_for_boot()
+ vm.login()
+
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm)
+ vm.shutdown()
+ vm.wait_for_shutdown()
+ finally:
+ vm.quit()
+
+ return err
+
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
diff --git a/tests/qemu-test/run.sh b/tests/qemu-test/run.sh
new file mode 100755
index 0000000..4064c63
--- /dev/null
+++ b/tests/qemu-test/run.sh
@@ -0,0 +1,69 @@
+#!/bin/bash
+#
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <ryanh@us.ibm.com
+#
+
+while [ $# -gt 0 ]; do
+ var=${1%=*}
+ value=${1#*=}
+ export ${var}="${value}"
+ shift;
+done
+
+[ -z "${QEMU}" ] && { QEMU="qemu-system-x86_64"; }
+[ -z "${TESTS}" ] && { TESTS="`ls *.py`"; }
+[ -z "${ARGS}" ] && { ARGS="-m 128 -net tap -net nic,model=e1000 -net nic,model=rtl8139 -snapshot -vnc none"; }
+[ -z "${IMAGEDIR}" ] && { IMAGEDIR=$PWD/images; }
+
+# source repositories don't like empty files very well
+# just in case, touch the file for qemu to ensure the module
+# gets compiled
+touch qemu/__init__.py
+
+mkdir -p ${IMAGEDIR}
+DISKS=( `ls ${IMAGEDIR}` )
+# error out if we don't have any images defined
+[ ${#DISKS[@]} -le 0 ] && {
+ echo "No test images defined in ${IMAGEDIR}"
+ exit 0;
+}
+
+declare -a PASS
+declare -a FAIL
+for t in ${TESTS[@]}; do
+ for d in ${DISKS[@]}; do
+ . ${IMAGEDIR}/${d}
+ echo "Running Test:${t} on Image:${d}"
+ if [ -z "${COMMAND}" ]; then
+ COMMAND="${QEMU} ${ARGS} -drive file=${DISK},if=ide -boot c"
+ echo "Using default command: ${COMMAND}"
+ else
+ # filter out QEMU from COMMAND, image files should use
+ # QEMU if they need to specify, and force -vnc none
+ COMMAND="${QEMU} $(echo $COMMAND | fmt -w 1 | grep -v ${QEMU} | fmt -w 255) -vnc none"
+ echo "Using override command: ${COMMAND}"
+ fi
+ sleep 2
+ sudo python ${t} ${COMMAND}
+ rv="$?"
+ if [ "${rv}" == "0" ]; then
+ PASS[${#PASS[@]}]="${t}:${d}"
+ else
+ FAIL[${#FAIL[@]}]="Test:${t},Image:${d}"
+ fi
+ # reset command for next image
+ COMMAND=""
+ done
+done
+
+echo ""
+echo "************************************************************"
+echo "Results: ${#PASS[@]} passed, ${#FAIL[@]} FAILED"
+for f in ${PASS[@]}; do
+ echo "passed: ${f}"
+done
+for f in ${FAIL[@]}; do
+ echo "FAILED: ${f}"
+done
+echo "************************************************************"
diff --git a/tests/qemu-test/timedrift.py b/tests/qemu-test/timedrift.py
new file mode 100644
index 0000000..4c8ab04
--- /dev/null
+++ b/tests/qemu-test/timedrift.py
@@ -0,0 +1,67 @@
+# Copyright IBM Corp. 2008
+# Authors: Ryan Harper <ryanh@us.ibm.com
+#
+
+from qemu.test import launch, TimeoutException
+import sys, time
+
+def run_test(vm):
+ badclocks = []
+ vm.wait_for_boot()
+ vm.login()
+
+ # calculate how much time it takes to run a very simple guest command
+ length = 600
+ now = time.time()
+ vm.guest_command("echo true")
+ end = time.time()
+ overhead = int(end - now)
+ # obviously it takes *some* amount of time to issue the command
+ if overhead == 0:
+ overhead = 1
+ vm.guest_command("echo calculated overhead: %s seconds"%(overhead))
+
+ available = vm.guest_command("cat /sys/devices/system/clocksource/clocksource0/available_clocksource")
+ clocksource = vm.guest_command("cat /sys/devices/system/clocksource/clocksource0/current_clocksource")
+ for clock in available.split():
+ if clock == "jiffies":
+ vm.guest_command("echo skipping crappy clock=jiffies")
+ continue
+ vm.guest_command("echo switching clocksource to %s"%(clock))
+ vm.guest_command("echo %s | dd of=/sys/devices/system/clocksource/clocksource0/current_clocksource" %(clock))
+ now = time.time()
+ vm.guest_command("date -u +%s; sleep 600; date -u +%s", timeout=1200)
+ end = time.time()
+ delta = int(end - now)
+ vm.guest_command("echo %s took %s seconds to complete"%(clock, delta))
+
+ if (delta-length) > overhead:
+ badclocks.append(clock)
+
+ if len(badclocks) > 0:
+ print "Bad clocks: %s"%(badclocks)
+ return 1
+
+ return 0
+
+
+def main(args):
+ err = 1
+ vm = launch(*args)
+
+ try:
+ err = run_test(vm)
+ vm.shutdown()
+ vm.wait_for_shutdown()
+ finally:
+ vm.quit()
+
+ return err
+
+
+if __name__ == '__main__':
+ try:
+ sys.exit(main(sys.argv[1:]))
+ except TimeoutException, e:
+ print 'Timeout occurred waiting for %s' % e.message
+ sys.exit(1)
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [Qemu-devel] [PATCH 2/2] Integrate qemu-test into make system
2008-10-15 19:53 [Qemu-devel] [PATCH 0/2] Add qemu-test for automated testing Ryan Harper
2008-10-15 19:53 ` [Qemu-devel] [PATCH 1/2] " Ryan Harper
@ 2008-10-15 19:53 ` Ryan Harper
1 sibling, 0 replies; 5+ messages in thread
From: Ryan Harper @ 2008-10-15 19:53 UTC (permalink / raw)
To: qemu-devel; +Cc: Anthony Liguori, Ryan Harper, kvm
This patch integrates qemu-test into the Make system for qemu. Providing a new
target at the toplevel and invoking the included Makefile in qemu-test to
execute testing. Invoking make qemu-test from the toplevel will run qemu-test
across each configured target, for example, if qemu was configured with:
./configure --target-list="i386-softmmu x86_64-softmmu"
then qemu-test will execute the tests against the above targets.
Signed-off-by: Ryan Harper <ryanh@us.ibm.com>
diff --git a/Makefile b/Makefile
index 36b36cd..0a40827 100644
--- a/Makefile
+++ b/Makefile
@@ -252,6 +252,9 @@ endif
test speed: all
$(MAKE) -C tests $@
+qemu-test: all
+ $(MAKE) -C tests $@
+
TAGS:
etags *.[ch] tests/*.[ch]
diff --git a/tests/Makefile b/tests/Makefile
index 326b733..d769662 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -103,6 +103,13 @@ hello-mipsel: hello-mips.c
test-cris:
$(MAKE) -C cris check
+SUBDIR_RULES=$(patsubst %,qemu-%, $(TARGET_DIRS))
+qemu-%:
+ $(MAKE) -C qemu-test QEMU=../../$(subst qemu-,,$@) all
+recurse-all: $(SUBDIR_RULES)
+.PHONY: qemu-test
+qemu-test: recurse-all
+
clean:
rm -f *~ *.o test-i386.out test-i386.ref \
test-x86_64.log test-x86_64.ref qruncom $(TESTS)
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [PATCH 1/2] Add qemu-test for automated testing
2008-10-15 19:53 ` [Qemu-devel] [PATCH 1/2] " Ryan Harper
@ 2008-10-15 22:59 ` Paul Brook
2008-10-16 4:25 ` Anthony Liguori
0 siblings, 1 reply; 5+ messages in thread
From: Paul Brook @ 2008-10-15 22:59 UTC (permalink / raw)
To: qemu-devel; +Cc: Anthony Liguori, Ryan Harper, kvm
On Wednesday 15 October 2008, Ryan Harper wrote:
> This patch places the qemu-test framework and tests into the qemu source
> tree. There are a number of components to this patch:
Is there any point having this in the qemu repository?
AFAICS it gains nothing from being "integrated" with qemu. It should work
equally well with other hypervisors, and even real hardware.
I'm not saying this is a bad thing, just that is seems like it should be a
separate project, and I'd be surprised if such projects don't already exist.
Paul
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [PATCH 1/2] Add qemu-test for automated testing
2008-10-15 22:59 ` Paul Brook
@ 2008-10-16 4:25 ` Anthony Liguori
0 siblings, 0 replies; 5+ messages in thread
From: Anthony Liguori @ 2008-10-16 4:25 UTC (permalink / raw)
To: Paul Brook; +Cc: Ryan Harper, qemu-devel, kvm
Paul Brook wrote:
> On Wednesday 15 October 2008, Ryan Harper wrote:
>
>> This patch places the qemu-test framework and tests into the qemu source
>> tree. There are a number of components to this patch:
>>
>
> Is there any point having this in the qemu repository?
>
In tree unit testing. The ideal model would be to have something that
anyone could run via a "make check" that could confirm whether something
you've done introduces regressions in things you don't normally test.
> AFAICS it gains nothing from being "integrated" with qemu. It should work
> equally well with other hypervisors, and even real hardware.
>
If we're just talking about booting a guest, yeah, I agree. But even
the tests introduced here are validating specific QEMU features (like
migration). Ideally, this is the type of test that would be added.
Things like verifying that an emulated USB disk still works, that USB
hotplug works, that SCSI disks can be mounted in the guest, etc. If we
can find a place to host images, it could be automated in such a way
that images could be easily downloaded. This would be extremely good
for doing quick sniff tests of whether or not a change breaks SPARC
emulation (not that any of us ever do that :-)).
Basically, I think we all end up building this sort of infrastructure on
our own to be able to do testing so it's a good idea if we all work on
the same set of tests.
> I'm not saying this is a bad thing, just that is seems like it should be a
> separate project, and I'd be surprised if such projects don't already exist.
>
There are separate, more generic test programs like autotest that do
support QEMU (at least, KVM). The focus here is more QEMU specific
unit/regression testing.
Regards,
Anthony Liguori
> Paul
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2008-10-16 4:26 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-10-15 19:53 [Qemu-devel] [PATCH 0/2] Add qemu-test for automated testing Ryan Harper
2008-10-15 19:53 ` [Qemu-devel] [PATCH 1/2] " Ryan Harper
2008-10-15 22:59 ` Paul Brook
2008-10-16 4:25 ` Anthony Liguori
2008-10-15 19:53 ` [Qemu-devel] [PATCH 2/2] Integrate qemu-test into make system Ryan Harper
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).