[ANNOUNCE] kvm-autotest

[Uri Lublin <uril@qumranet.com>]

Hello,

     We are happy to announce the availability of kvm-autotest, a test framework 
for KVM based on autotest. Naturally, the purpose is to make KVM stable, 
find/fix bugs faster and prevent regressions. It is to serve developers, to test 
if their new code breaks anything, as well as users, to verify that the KVM they 
are using is stable in their own environment.

     There are many things to improve. Currently supported are only a few tests
(mainly reboot/migration) and a single guest (Fedora 8 i386). This will change 
soon as we add tests and guests to the test matrix.

     We've tried to be very flexible with configuration of tests, so there are
quite a few options to choose from (check the documentation). For example
one may choose to download and install the latest (or a specific) KVM release,
the latest (or a specific) daily snapshot, use git, or use other source.
Almost all configuration is located in kvm.cfg (under client directory).

     By default kvm-autotest creates a machine-local subnet with a
local dhcp server. The dhcp server is configured to ignore unknown clients and
only serves KVM guests. One may disable it by changing kvm.cfg (See documentation)

     The test framework is based on autotest ( http://test.kernel.org/autotest ).
Currently we only using client tests, later we may want to use server tests
for more complicated tests.

To download the source:
     cd /usr/local ( or another root-writeable directory )
     git clone git://git.kernel.org/pub/scm/virt/kvm/kvm-autotest.git
     cd kvm-autotest/client
To read documentation look at http://kvm.qumranet.com/kvmwiki/KVM_RegressionTest

The simplest way to use it is:
0. login as root (not necessary but simpler, as tests must be run as root)
1. Download kvm-autotest (see a few lines above)
2. Change the following in kvm.cfg according to your preference (or leave the 
default settings):
        local_kvm_dir   -- "base"   directory
        local_guest_dir -- "images" directroy (may take a lot of disk space)
        local_iso_dir   -- "isos"   directroy (may take a lot of disk space)
        bridge          -- change if your local network conflicts with
                               192.168.1/24 subnet
                           change bridge name if conflicts with a currently used
                               bridge.
        dhcp_net        -- change accordingly (e.g. use 10.0.1.2)
3. Make sure you have enough disk space for isos and images (df -h /var/local)
4. The default guest-installation test requires Fedora-8-i386-DVD.iso.
    It is configured to fetch it automatically from the web unless it already
      exists in the local_iso_dir (default path is
      /var/local/kvm_regression/isos/linux/Fedora-8-i386-DVD.iso).
    If you already have that iso image locally, create a link or change
      local_iso_dir.
    Alternatively, in tests/kvm_guest_install/Fedora-8-i386/Fedora-8-i386.ini
      set iso=/path/to/Fedora-8-i386-DVD.iso.
5. Default machine-local subnet is 192.168.1/24. If it conflicts with your
    network, you need change network configuration in kvm.cfg
6. Run the tests as root (./bin/autotest tests/kvm_runtest/control)
7. Checkout the results at results/default/

Caveats/Issues:
- Upon failure check results/defualt/debug/error-*
- After first download of KVM sources (and building + loading the modules),
     one needs to either manually remove /var/local/kvm_regression/{src,build}
     or set kvm_install=no in kvm.cfg. Otherwise the test will fail.
     The reason for it is that we do not want to remove directories
     automatically.
- The message "Guest installation completed but guest is not responsive",
     usually means that the guest installation failed.
- The tests must be run as root.
- When a guest is installing, do not move its mouse or type its keyboard.
   It confuses the guest installation algorithm (when watching its progress, e.g.
   with vncviewer).
- Guest installation is currently very sensitive to timing. If installation
   fails, sometimes it's helpfull to enlarge sleep time before the reboot and
   between steps. Do it by modifying
   tests/kvm_guest_install/Fedora-8-i386/Fedora-8-i386.steps
- After the guest is installed and you want to rerun the other tests set
     guest_install=no in kvm.cfg
   Also if you want to test an existing image, create a new vm-class (guest
   configuration) and set guest_install=no in kvm.cfg

     We hope many will test KVM on their systems and report results to the
community. We plan to have a web page with results. We hope many will add tests 
and improve KVM regression tests.

Thanks,
     Uri.