Re: [ANNOUNCE] kvm-autotest

[Marcelo Tosatti <mtosatti@redhat.com>]

Hi Uri,

On Thu, Jul 10, 2008 at 03:09:36PM +0300, Uri Lublin wrote:
> You see guests as clients and the host as the server.
> We were thinking of the host as a client and multi-host operations to be 
> done by a server. guest-operations would be done using ssh (for linux 
> guests) as your example above. You make a good point that we can use 
> server/client infrastructure for guest operations. As it is simpler to 
> write autotest client tests, and we thought most of the tests would be 
> run as client tests, we want to postpone the server tests and focus on 
> adding tests and guests to the matrix.

Perhaps you need a main server on top for that, to coordinate various
hosts.

The big problem with writing client tests is that you have to rewrite
what autotest already provides, diverging from mainline. That means work
to be done everytime a new test is added in autotest, which could come
for free if guests were treated as clients and the host treated as a
server.

And the other way around is also true: new OS tests added to
kvm-autotest will not make their way naturally into autotest mainline.

Other than Windows tests, of course, but most Linux OS tests should run
fine in *NIX.

>> - On top of the client tests, we want to be able to force "special
>> stress conditions", such as:
>>     - host memory pressure (to test mmu shrink and mmu notifiers)
> Similar is how many guests (or what to memory over-subscription 
> percentage) we can start while all of them are responsive.
>>     - CPU starvation (lower the priority of the guest process with      
>> higher prio CPU hungry tasks running on the host).
>>     - synthetic cpu migration stress. ChrisW wrote this script sometime
>>       ago to test clock stability:
>>
>>      PID=$1
>>      while :
>>      do
>>         for x in 0x01 0x02 0x04 0x08 0x10 0x20 0x40 0x80
>>         do
>>             taskset -p $x $PID > /dev/null 2>&1
>>             usleep 5
>>         done
>>      done
>>     - migration (as it stands now there is no transparent way to 
>> schedule       client tests to run _while_ migration is in progress, 
>> one needs to
>>       manually change the migration test for that).
>
> You probably do not need to change the migration test. I think we need to 
> run some load on the guest (we'd probably have many load options, and the 
> user will choose/configure which one to use) before migration starts and 
> keep it running during the migration process.

Right.

>> - Currently its difficult to debug client test failure inside guests,
>>   since the VM and its image are destroyed. Perhaps the client/server
>>   model handles error handling/reporting much more nicely.
>
> Agreed, we thought of that, but it's not cooked yet. Currently we always  
> cleanup. We should not remove the (temporary) image if the test fails. 
> Should we keep the guest running upon failure ? Currently we continue to 
> test the next guest. We should probably have a configuration flag for 
> that too.

Yes, it should be dependant on a configuration flag. For developer
use, stopping the run makes sense so the failure can be analyzed. For
stress/coverage, you want to know results of the entire test battery.