[Qemu-devel] [PATCH 0/1] integrate qemu-test into kvm-userspace

[Ryan Harper <ryanh@us.ibm.com>]

This patch provides a relatively simple method for exercising various
features of qemu/kvm by interacting with the guest via serial and qemu
via the monitor that individual developers can use to validate that
their changes having broken fundamental feature/function.  Such tests
need to be very accessable and easy-to-use to encourge there use by
users/developers.  As a bonus, in-tree make test is trivially integrated
into higher-level test frameworks such as autotest.

Using such interfaces the guest requirements are very low (serial login)
and thus we can execute tests across a large number of differing
guests.  Writing new test cases should also be fairly simple,  for
example, the host_shutdown.py test logic is as follows:

def run_test(vm):
        vm.wait_for_boot()
        vm.login()
        vm.guest_command("echo pressing power button")
        output = vm.monitor_command('system_powerdown')
        if vm.wait_for_shutdown(): return 1

There is a configuration file (test/CONFIG) which can be used to provide
per-distro overrides for various functions.  The current defaults have
been tested against: RHEL5, openSUSE 11, Ubuntu 8.04, and Fedora 9 w.r.t
the regular expressions used for login, and prompt.

Getting started is relatively simple for someone familiar with
configuring a guest for serial login and boot output via serial.  Once
completed, add a file to $(TOPDIR)/test/images like the following:

% cat images/rhel5.2
DISK="/home/rharper/work/images/rhel5.2_x86_64_10G.qcow2"
COMMAND="qemu-system-x86_64 -m 512 -smp 2 -net tap -net nic,model=e1000
         -net nic,model=rtl8139 -drive file=${DISK},if=ide,boot=on
         -vnc none"

You can omit the COMMAND variable and a default qemu command will be run
instead (see run.sh).  For additional guest image or config just add a
new file in the images dir.

To run the tests, at the toplevel run:

% make && sudo make test
<snip massive amount of output>
************************************************************
Results: 5 passed, 4 FAILED
passed: e820_memory.py:fedora-9-x86_64
passed: host_shutdown.py:fedora-9-x86_64
passed: networking.py:fedora-9-x86_64
passed: reboot.py:fedora-9-x86_64
passed: smp_hotplug.py:fedora-9-x86_64
FAILED: Test:host_reset.py,Image:fedora-9-x86_64
FAILED: Test:migrate.py,Image:fedora-9-x86_64
FAILED: Test:timedrift.py,Image:fedora-9-x86_64
FAILED: Test:writeverify.py,Image:fedora-9-x86_64
************************************************************


Ultimately, I'd like to push this into qemu since nothing we're doing
here is arch or kvm specific.  To do so, I'd move the test bits into qemu/tests
and invoke run.sh from there, iterating over the various QEMU_PROG names
available for each configured TARGET_DIRS so that we support running
when qemu has been configured with multiple targets.

The current kvm-userspace integration handles removing and loading the
compiled modules in the kernel dir and some additional symlink-fu to
ensure we run qemu from the build tree and don't need to have run make
install.  This integration would stay in the kvm-userspace tree and
change to recursing into qemu and invoking make test.

There is still plenty to do on the framework as well as the test, but I
wanted to get feedback on the current setup before investing significant
efforts.