Re: [ANNOUNCE] kvm-autotest

[Uri Lublin <uril@qumranet.com>]

Marcelo Tosatti wrote:
> On Thu, Jul 10, 2008 at 03:09:36PM +0300, Uri Lublin wrote:
> 
> The big problem with writing client tests is that you have to rewrite
> what autotest already provides, diverging from mainline. That means work
> to be done everytime a new test is added in autotest, which could come
> for free if guests were treated as clients and the host treated as a
> server.

We can have a generic function like
run-on-guest(vm, run_path_on_guest, hdir, gdir, hresult, gresult) which
- copies a directory from host to guest
- runs a script on the guest (we may add an option to pass params to the script)
- copies results directory from guest to host

That would work for any autotest test (the run script will run bin/autotest, 
newly added tests will be added to the run script, or be passed as args) and 
should work for non-Linux guests.

Look at server/tests and client/tests; Most of the tests reside in the client side.

> 
> And the other way around is also true: new OS tests added to
> kvm-autotest will not make their way naturally into autotest mainline.

They may be accepted as client test(s). If we put all tests under client/kvm_* 
or even better all tests under client/kvm_runtest, than it would be just another 
(massive) client test.

> 
> Other than Windows tests, of course, but most Linux OS tests should run
> fine in *NIX.
> 
I'm not yet sure how we are going to implement access to windows-kvm-guest but 
we sure want to have windows guests in our test matrix.


Basically I don't mind trying the server-side solution.
We can have a little exercise, trying to write a kvm-test on the server and on 
the client and compare the effort.
I think we should mostly focus on adding tests and guests.


>>> - Currently its difficult to debug client test failure inside guests,
>>>   since the VM and its image are destroyed. Perhaps the client/server
>>>   model handles error handling/reporting much more nicely.
>> Agreed, we thought of that, but it's not cooked yet. Currently we always  
>> cleanup. We should not remove the (temporary) image if the test fails. 
>> Should we keep the guest running upon failure ? Currently we continue to 
>> test the next guest. We should probably have a configuration flag for 
>> that too.
> 
> Yes, it should be dependant on a configuration flag. For developer
> use, stopping the run makes sense so the failure can be analyzed. For
> stress/coverage, you want to know results of the entire test battery.

We'll add such a flag.

Thanks,
Uri