Re: [Qemu-devel] [RFC] Future goals for autotest and virtualization tests

[Anthony Liguori <anthony@codemonkey.ws>]

On 03/09/2012 08:30 AM, Paolo Bonzini wrote:
> Il 09/03/2012 15:01, Anthony Liguori ha scritto:
>>> How do you handle out-of-tree patches with submodules (as is the case
>>> when working on new code)?
>>
>> It's very easy to update .gitmodules to point to a different tree on
>> your local system and then update the ref to a local commit.
>>
>> So from a development perspective, it's simple.  The harder question is
>> what to do about qemu-test HEAD on qemu.org
>
> Yep.  So you would commit patches to qemu.git with some kind of IOU for
> tests, that will be committed as soon as kernel support hits the mainline?

Probably commit the test and mark it as an expected failure until support hits 
mainline.

>
>>> You want to require tests in order to commit to qemu, but this (for
>>> tests where using qtest is not feasible for any reason) requires all
>>> drivers to be upstream and accessible to qemu-jeos.
>>
>> We're really just talking about virtio here, no?  Maybe we can convince
>> Rusty to have a proper virtio-next.git...
>>
>> I think this ends up being outside the scope of qemu-test.  Perfect is
>> the enemy of good here.
>
> No, this ends up being outside the scope of Anthony, who only cares
> about testing KVM/x86.  We add one virtio device every couple years,
> while new ARM boards are added every couple months.  It kind of works
> because a large part of the QEMU development community is testing on
> KVM/x86, but not entirely.

qemu-test is a specific solution to a specific problem.  The problem is: how can 
you leverage an existing kernel to simplify writing device model tests without 
having a full blown libOS in qtest.

Linux is the only part that matters here.  The userspace in qemu-jeos is just 
there to give a small environment for Linux to function properly in.

> It is not a problem. of course: you're human, you're one person and one
> would need a dozen to set up everything properly.  OTOH, it's too easy
> to dismiss buildroot when you are catering to a much smaller objective.
> The complete objective would have a much bigger overlap with
> buildroot's, and likely the design would too.

You have a different objective than I do.  You want to build a reproducible 
guest that runs on a wider variety of architectures than common distributions do 
with a good enough userspace to write integration tests with.  That's a good 
goal if you care about the full stack.

I'm skeptical of the value of this from a QEMU point of view.  Do we really care 
if the buildroot version of udev propagates hotplug events correct in buildroot 
for ARM in a beagleboard machine?

I'm sure someone cares about that, but it's ultimately not a QEMU problem.  I 
think we need to focus on getting our house in order (in terms of quality) 
before we start trying to tackle a larger scope.

Regards,

Anthony Liguori

>
> Paolo