Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU

[Avi Kivity <avi@redhat.com>]

On 12/29/2011 06:39 PM, Anthony Liguori wrote:
>
> qemu-test builds a custom guest, entirely from source.  This makes it
> possible to efficiently test non-native architectures.  The tests are
> written for this minimal environment (which is not straight forward to
> do).  This tests will never run on Windows as they are 100% tied to
> their environment.
>
> autotest (let's be clear, there is no such thing as kvm autotest
> anymore) is a framework for executing third party tests.  It's got
> fancy magic to execute in client mode vs. server mode.  There is a
> suite of tests that are integrated in autotest that exercise various
> types of virtualization functionality.  This suite of tests use a
> special config format to determine what they do.
>
> Included in this, is the ability to create a guest from an ISO either
> using step files or via a guest-specific mechanism (kickstart, etc.). 
> The tests are written to be mostly guest agnostic and are therefore
> written in Python in a cross platform way.
>
> There is essentially no overlap between the two and calling kvm
> autotest superior is like calling the Sun superior to the Moon.

Why not extend autotest do the same thing.  Instead of downloading a
fedora iso it would download a kernel tarball and (cross-)build it.

>
>> My main motivation to merge
>> qemu-test and kvm autotest is that I fear that qemu-test will be the
>> only
>> requirement for committing stuff for qemu and developers will be
>> excused from
>> their 'duty' by writing a 50 LOC shell script and assume they done w/
>> testing.
>
> There is no requirement to write autotest tests to get things merged
> into QEMU.  That's is how things are today.
>
> And I don't think there will ever a requirement to write anything that
> isn't merged directly into qemu.git.  I'm not going to hold up merging
> a feature until another project merges something into their tree.

What about virtio features (we used to depend on the kernel, now on the
spec)?  Seabios?

>
> So unless we're going to merge KVM autotest into qemu.git, I don't
> think there's every going to be a requirement to write a KVM autotest
> test in order to get something merged into qemu.git.

Let's consider postcopy live migration being proposed.  In addition to
various unit tests, it also wants an integration test.  So now we need
to write a qemu-test postcopy live migration test, and also autotest
test that tests non-Linux guests, migration under load, with memory
hotplug, etc.

>
> But since autotest is a framework for running third-party tests, it
> seems to make sense for qemu.git to have a test framework that
> autotest can execute.
>
> And since what we call KVM autotest actually tests a heck of a lot
> more than just QEMU, it makes sense for KVM autotest to continue to
> focus on full stack testing where QEMU is but one of the many
> components that it tests.

It might have made sense to split the kvm-testing functionality of
autotest, and have autotest drive that.  We could even have called it
qemu-test.

-- 
error compiling committee.c: too many arguments to function