From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1KqCRF-0008Ou-04 for qemu-devel@nongnu.org; Wed, 15 Oct 2008 15:53:45 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1KqCRC-0008NA-4N for qemu-devel@nongnu.org; Wed, 15 Oct 2008 15:53:43 -0400 Received: from [199.232.76.173] (port=54243 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1KqCRB-0008Mx-Rt for qemu-devel@nongnu.org; Wed, 15 Oct 2008 15:53:41 -0400 Received: from e2.ny.us.ibm.com ([32.97.182.142]:36949) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1KqCRB-0008Jl-4D for qemu-devel@nongnu.org; Wed, 15 Oct 2008 15:53:41 -0400 Received: from d01relay02.pok.ibm.com (d01relay02.pok.ibm.com [9.56.227.234]) by e2.ny.us.ibm.com (8.13.8/8.13.8) with ESMTP id m9FJra5x009958 for ; Wed, 15 Oct 2008 15:53:36 -0400 Received: from d01av04.pok.ibm.com (d01av04.pok.ibm.com [9.56.224.64]) by d01relay02.pok.ibm.com (8.13.8/8.13.8/NCO v9.1) with ESMTP id m9FJrZJq050610 for ; Wed, 15 Oct 2008 15:53:35 -0400 Received: from d01av04.pok.ibm.com (loopback [127.0.0.1]) by d01av04.pok.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id m9FJrZAT004245 for ; Wed, 15 Oct 2008 15:53:35 -0400 From: Ryan Harper Date: Wed, 15 Oct 2008 14:53:32 -0500 Message-Id: <1224100414-14925-1-git-send-email-ryanh@us.ibm.com> Subject: [Qemu-devel] [PATCH 0/2] Add qemu-test for automated testing Reply-To: qemu-devel@nongnu.org List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Anthony Liguori , Ryan Harper , kvm@vger.kernel.org This patch provides a relatively simple method for exercising various features of qemu by interacting with the guest via serial and qemu via the monitor that individual developers can use to validate that their changes haven't broken feature/function. Such tests need to be very accessable and easy-to-use to encourge there use by users/developers. As a bonus, in-tree make test is trivially integrated into higher-level test frameworks such as autotest. Using such interfaces the guest requirements are very low (serial login) and thus we can execute tests across a large number of differing guests. Writing new test cases should also be fairly simple, for example, the host_shutdown.py test logic is as follows: def run_test(vm): vm.wait_for_boot() vm.login() vm.guest_command("echo pressing power button") output = vm.monitor_command('system_powerdown') if vm.wait_for_shutdown(): return 1 There is a configuration file (tests/qemu-test/CONFIG) which can be used to provide per-distro overrides for various functions. The current defaults have been tested against: RHEL5, openSUSE 11, Ubuntu 8.04, and Fedora 9 w.r.t the regular expressions used for login, and prompt. Getting started is relatively simple for someone familiar with configuring a guest for serial login and boot output via serial. Once completed, add a file to tests/qemu-test/images like the following: % cat images/rhel5.2 DISK="/home/rharper/work/images/rhel5.2_x86_64_10G.qcow2" COMMAND="qemu-system-x86_64 -m 512 -smp 2 -net tap -net nic,model=e1000 -net nic,model=rtl8139 -drive file=${DISK},if=ide,boot=on -vnc none" You can omit the COMMAND variable and a default qemu command will be run instead (see run.sh). For additional guest image or config just add a new file in the images dir. To run the tests, at the toplevel run: % make qemu-test Some additional parameters can be passed through to run.sh to control how we execute qemu-test: TESTS, ARGS, and IMAGEDIR. By default, run.sh will execute each test (*.py) in tests/qemu-test/. If you want to restrict that list to something more specific, you can pass a space separated list to the make command: % make qemu-test TESTS="reboot.py networking.py" ARGS allows you to specific what parameters are passed to the qemu-system command. You can either modify the default args in tests/qemu-test/run.sh or override them via the command line IMAGEDIR is a relative path where the image input files are located. It defaults to 'images' in tests/qemu-test, but if you have a specific directory you want to specify, for example tests/qemu-test/images.i386, this can be done with: % make qemu-test IMAGEDIR="images.i386" Output of the run will be logged to qemu/${TARGET_ARCH}_test_results.out. This includes all data generated during the run and the results, each test run will include whether it passed or failed, the testname, and the name of the image in which the test was run. Some sample output: % head -n 10 i386-softmmu_test_results.out Running Test:migrate.py on Image:ubuntu-6.06 Using default command: ../../i386-softmmu/qemu -L ../../pc-bios -m 128 -net tap -net nic,model=e1000 -net nic,model=rtl8139 -snapshot -vnc none -drive file=/home/rharper/work/images/ubuntu_6.06_i386_10G.qcow2,if=ide -boot c [42949372.960000] Linux version 2.6.15-51-server (buildd@rothera) (gcc version 4.0.3 (Ubuntu 4.0.3-1ubuntu5)) #1 SMP Thu Dec 6 21:37:18 UTC 2007 [42949372.960000] BIOS-provided physical RAM map: [42949372.960000] BIOS-e820: 0000000000000000 - 000000000009fc00 (usable) [42949372.960000] BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved) [42949372.960000] BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved) [42949372.960000] BIOS-e820: 0000000000100000 - 0000000007ff0000 (usable) [42949372.960000] BIOS-e820: 0000000007ff0000 - 0000000008000000 (ACPI data) [42949372.960000] BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved) % tail -n 7 i386-softmmu_test_results.out 0 root@vm1:~# ************************************************************ Results: 2 passed, 0 FAILED passed: migrate.py:ubuntu-6.06 passed: migrate_networking.py:ubuntu-6.06 ************************************************************