[Qemu-devel] Re: spice in kvm-autotest [was: Re: KVM call minutes for Apr 5]

[Lucas Meneghel Rodrigues <lmr@redhat.com>]

On Tue, 2011-04-05 at 19:01 +0300, Alon Levy wrote:
> screenshots are already there, and they are a great start. But you can't
> really do testing if you aren't recreating the same environment, and having
> a client server where there is no client, while being a good test, doesn't
> cover the case where the client is connected :)
> 
> So I was basically talking about the added requirement of creating a client
> connection (one or more) to a single vm.

Yeah, I think now I've got the idea. We can certainly think of some
strategy to coordinate test execution in vms in one host, and have
another bare metal machine that opens clients to the vms on that host.

Autotest has infrastructure to coordinate tests among multiple bare
metal machines, we call that infrastructure 'barriers'. It works
basically through TCP socket communication, we have an autotest library
for that, and an API that allows coordination among multiple machines.
We use that for example, to coordinate cross host migration tests.

So, it might be tricky (specially setting up the environment on the
machine that will open displays and handle all the client opening
part[1]), but certainly doable IMHO.

[1] This is the part where the LDTP thing would come to place - we need
infrastructure to support opening the display, starting the graphical
clients and interacting with them in an automated fashion, LDTP and
dogtail are the means to do that.

> Note that I wasn't asking anyone to develop this - I'm just asking if patches
> in that direction would be accepted/interesting. We (well, Swapna, cc-ed) are
> still working on deciding exactly how to do automated testing as a project.

Sure, absolutely, I've been talking to Swapna about this.

> Regarding the dogtail/LDTP issue, they are about specific tests run inside the
> guest, and they are certainly something we would leverage. But like I mentioned
> on the call, there is a whole suite of whql tests that are display specific,
> and don't require anything new. In fact, a few months ago I added support for
> autotest to run one of them, resizing to all the possible modes - so I know I
> don't need dogtail for significant portions of our testing. (sorry, no git link
> - I'll clean it up and post, it's been done 10 months ago so probably won't
> cleanly apply :)

The thing about WHQL is that it has its own test suite coordination
program (DTM), that has to run on a separate machine/VM. So they have
all infrastructure in place there. If we need graphical clients being
started and controlled on a linux bare metal machine, I am afraid we'll
have to resort to those GUI test automation frameworks I mentioned. if
our test is geared more towards windows clients, then they won't be
needed, sure.

> For some ideas about what we are interested in see
> http://spice-space.org/page/SpiceAutomatedTesting. (just the Requirements section).

I took a look at it