From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marcelo Tosatti Subject: Re: [ANNOUNCE] kvm-autotest Date: Wed, 9 Jul 2008 12:43:57 -0300 Message-ID: <20080709154357.GA6217@dmt.cnet> References: <48709B6D.6030300@qumranet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: kvm@vger.kernel.org, Dror Russo To: Uri Lublin Return-path: Received: from mx1.redhat.com ([66.187.233.31]:43172 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754054AbYGIPoi (ORCPT ); Wed, 9 Jul 2008 11:44:38 -0400 Content-Disposition: inline In-Reply-To: <48709B6D.6030300@qumranet.com> Sender: kvm-owner@vger.kernel.org List-ID: On Sun, Jul 06, 2008 at 01:16:13PM +0300, Uri Lublin wrote: > Hello, > > We are happy to announce the availability of kvm-autotest, a test > framework for KVM based on autotest. Naturally, the purpose is to make > KVM stable, find/fix bugs faster and prevent regressions. It is to serve > developers, to test if their new code breaks anything, as well as users, > to verify that the KVM they are using is stable in their own environment. > > There are many things to improve. Currently supported are only a few tests > (mainly reboot/migration) and a single guest (Fedora 8 i386). This will > change soon as we add tests and guests to the test matrix. > > We've tried to be very flexible with configuration of tests, so there are > quite a few options to choose from (check the documentation). For example > one may choose to download and install the latest (or a specific) KVM release, > the latest (or a specific) daily snapshot, use git, or use other source. > Almost all configuration is located in kvm.cfg (under client directory). > > By default kvm-autotest creates a machine-local subnet with a > local dhcp server. The dhcp server is configured to ignore unknown clients and > only serves KVM guests. One may disable it by changing kvm.cfg (See documentation) > > The test framework is based on autotest ( http://test.kernel.org/autotest ). > Currently we only using client tests, later we may want to use server tests > for more complicated tests. This is looking great. Easy to use and fast. A few comments: - As you mention, it should reuse the server/client model for running tests inside guests. I hacked up a "kvm_autotest" test that basically does: tests = ["linus_stress", "bash_shared_mapping", "rmaptest", "tsc", "scrashme", "isic", "sleeptest", "libhugetlbfs", "..."] vm.ssh.scp_to_remote(autotest_tarball, '/root') (s,o) = vm.ssh.ssh('tar zvxf kvm-autotest.tar.gz') for i in range(0, len(tests)): (s,o) = vm.ssh.ssh('kvm-autotest/client/bin/autotest ' + 'kvm-autotest/client/tests/' + tests[i] + '/control') print(o) Which poorly replicates what the client/server infrastructure already provides. IMO its a waste of time to write specialized client tests (other than virt specific ones). - On top of the client tests, we want to be able to force "special stress conditions", such as: - host memory pressure (to test mmu shrink and mmu notifiers) - CPU starvation (lower the priority of the guest process with higher prio CPU hungry tasks running on the host). - synthetic cpu migration stress. ChrisW wrote this script sometime ago to test clock stability: PID=$1 while : do for x in 0x01 0x02 0x04 0x08 0x10 0x20 0x40 0x80 do taskset -p $x $PID > /dev/null 2>&1 usleep 5 done done - migration (as it stands now there is no transparent way to schedule client tests to run _while_ migration is in progress, one needs to manually change the migration test for that). - Currently its difficult to debug client test failure inside guests, since the VM and its image are destroyed. Perhaps the client/server model handles error handling/reporting much more nicely. Great work!