Re: [Qemu-devel] [RFC] Future goals for autotest and virtualization tests

[Anthony Liguori <anthony@codemonkey.ws>]

On 03/08/2012 08:49 AM, Ademar Reis wrote:
> On Thu, Mar 08, 2012 at 07:36:11AM -0600, Anthony Liguori wrote:
>> On 03/07/2012 10:00 PM, Lucas Meneghel Rodrigues wrote:
>>> Virt/qemu tests: Minimal guest images
>>> -------------------------------------
>>>
>>> In order to make development level test possible, we need the tests to run fast.
>>> In order to do that, a set of minimal guest images is being developed and we
>>> have a version for x86_64 ready and functional:
>>>
>>> https://github.com/autotest/buildroot-autotest
>>
>> I'm really not a fan of buildroot.  Note that in order to ship
>> binaries, full source needs to be provided in order to comply with
>> the GPL.  The FSF at least states that referring to another website
>> for source that's not under your control doesn't satisfy the
>> requirements of the GPL.
>>
>> Just out of curiosity, did you try to use qemu-test?  Is there a
>> reason you created something different?
>>
>> I think it's good that you're thinking about how to make writing
>> tests easier, but we have a growing test infrastructure in QEMU and
>> that's what I'd prefer people focused on.
>>
>
> You probably remember the long thread we had back in December on
> qemu-devel on this topic. Back then our message was "we have a
> growing test infrastructure in s/QEMU/autotest/ and that's what
> we'd prefer people focused on". :-)
>
>  From Dor:
>
> (http://lists.nongnu.org/archive/html/qemu-devel/2011-12/msg03024.html)
>
> """
> If you wish, you can challenge Lucas and Cleber w/ these type of
> requirements and we'll all improve as a result.
> """
>
> Your response was:
>
> """
> Well consider qemu-test the challenge. It's an existence proof
> that we can have a very easy to use framework for testing that
> runs extremely fast and is very easy to write tests for.
> """
>
> http://knowyourmeme.com/memes/challenge-accepted ;-)
>
> I particularly agreed with basically everything you said on that
> discussion regarding test simplification (I had just joined the
> team back then). To me, autotest has been focusing on QE-level,
> leaving the developer-level test requirements out. Now we're
> attacking this new front, and a lot of the requirements are
> indeed from that discussion.

If you want to talk about this in terms of "requirements", my requirement is for 
"developer-level" tests to live in qemu.git and be integrated into make check.

Just as we've been discussing and working on since the previous set of discussions.

> By simplifying the design and bringing barriers down, we hope to
> reach a broader audience and help developers write and maintain
> tests, benefiting from all the instrumentation that autotest
> brings. It's not going to be just about qemu (check the new test
> examples).
>
> We have a team fully dedicated to autotest and it's used not only
> by Qemu but also libvirt, Google, Xen, Fedora, Twitter, etc, etc
> (these all have code contributions in autotest)
>
> That said, the current qemu-tests will probably be easily
> integrated into (the new) autotest and we hope that, given enough
> time, autotest will be good enough to relieve qemu from the
> framework maintenance and code duplication with other projects.

autotest should not be the focal point for integration.  qemu.git should be.

I'd be perfectly happy to review patches submitting the test infrastructure from 
kvm-autotest into qemu.git (provided it didn't have unreasonable external 
dependencies and fit into QEMU).

Developer-level tests need to live where the developers live.  The developers 
live in qemu.git.  See my other response on this thread for the explanation of 
why this is so important.

Regards,

Anthony Liguori