Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU

[Lucas Meneghel Rodrigues <lmr@redhat.com>]

On 12/29/2011 12:38 PM, Dor Laor wrote:
>  From the recent threads it looks to me that the 2 advantages of
> qemu-test over kvm-autotest are:
>
> 1. python is not used within the guest

^ Just a (relatively small) subset of KVM autotest tests do require 
python in the guest (the ones that execute the autotest client inside 
the guest, virtio-console, and ksm overcommit), so if you don't want py 
on the guest, just execute the tests that do not require it. For 
example, we don't rely on any python for the Windows tests.

> 2. qemu-test is smaller and simpler
>
> Except for the above 2, kvm autotest is superior. My main motivation to
> merge qemu-test and kvm autotest is that I fear that qemu-test will be
> the only requirement for committing stuff for qemu and developers will
> be excused from their 'duty' by writing a 50 LOC shell script and assume
> they done w/ testing. In addition to that, the 50 LOC will be too basic
> and only provide value for basic functionality tests and kvm autotest
> folks will need to rewrite all of the matching content and beyond.

Requiring people to writing tests for qemu-test is already a improvement 
over the current situation. It might make people more interested and 
engaged on test driven development, some of them might want to take the 
time and learn how to use and develop tests for autotest. We gotta start 
with something :)

> Those above 2 advantages can be solve:
>
> 1. python is not used within the guest
> - One option is just to use the same shell code in kvm autotest w/o
> no python code and use the same kernel/initramfs.
> - Another way is to use a plain linux distro w/ python and boot it
> into single user test mode and potentially use a S4 guest snapshot
> or external snapshot to shorten its boot time.
> You'll get faster boot time and friendlier code.

Yes, slowly we are making autotest and the kvm tests faster by disabling 
by default some settings that make more sense to be enabled on automated 
test grids (drop memory caches, take a list of all packages installed on 
the test machine *between* tests, things like that). Hopefully it'll 
improve the experience of using it quite a lot.

Then, on the virt test side, we can find ways to cut the time to start 
the functional tests like you suggested (small, pre-installed images, 
snapshots, you name it). In any case, I believe that qemu-test is 
positive, since it lowers the barrier and gets people interested in 
writing tests. As long as we are careful and avoid overlapping features 
as much as possible, it's fine.