Re: [ANNOUNCE] kvm-autotest

[Uri Lublin <uril@qumranet.com>]

Marcelo Tosatti wrote:
> On Sun, Jul 06, 2008 at 01:16:13PM +0300, Uri Lublin wrote:
>>
>>     The test framework is based on autotest ( http://test.kernel.org/autotest ).
>> Currently we only using client tests, later we may want to use server tests
>> for more complicated tests.
> 
> This is looking great. Easy to use and fast. A few comments:
> 
> - As you mention, it should reuse the server/client model for running
>   tests inside guests. I hacked up a "kvm_autotest" test that
>   basically does:
> 
> tests = ["linus_stress", "bash_shared_mapping", "rmaptest", "tsc",
> "scrashme", "isic", "sleeptest", "libhugetlbfs", "..."]
> 
> vm.ssh.scp_to_remote(autotest_tarball, '/root')
> (s,o) = vm.ssh.ssh('tar zvxf kvm-autotest.tar.gz')
> for i in range(0, len(tests)):
>     (s,o) = vm.ssh.ssh('kvm-autotest/client/bin/autotest ' +
>                        'kvm-autotest/client/tests/' + tests[i] +
>                        '/control')
>     print(o)
> 
> Which poorly replicates what the client/server infrastructure already
> provides. IMO its a waste of time to write specialized client
> tests (other than virt specific ones).
> 

You see guests as clients and the host as the server.
We were thinking of the host as a client and multi-host operations to be done by 
a server. guest-operations would be done using ssh (for linux guests) as your 
example above. You make a good point that we can use server/client 
infrastructure for guest operations. As it is simpler to write autotest client 
tests, and we thought most of the tests would be run as client tests, we want to 
postpone the server tests and focus on adding tests and guests to the matrix.

> - On top of the client tests, we want to be able to force "special
> stress conditions", such as:
>     - host memory pressure (to test mmu shrink and mmu notifiers)
Similar is how many guests (or what to memory over-subscription percentage) we 
can start while all of them are responsive.
>     - CPU starvation (lower the priority of the guest process with 
>      higher prio CPU hungry tasks running on the host).
>     - synthetic cpu migration stress. ChrisW wrote this script sometime
>       ago to test clock stability:
> 
>      PID=$1
>      while :
>      do
>         for x in 0x01 0x02 0x04 0x08 0x10 0x20 0x40 0x80
>         do
>             taskset -p $x $PID > /dev/null 2>&1
>             usleep 5
>         done
>      done
>     - migration (as it stands now there is no transparent way to schedule 
>       client tests to run _while_ migration is in progress, one needs to
>       manually change the migration test for that).

You probably do not need to change the migration test. I think we need to run 
some load on the guest (we'd probably have many load options, and the user will 
choose/configure which one to use) before migration starts and keep it running 
during the migration process.

> 
> - Currently its difficult to debug client test failure inside guests,
>   since the VM and its image are destroyed. Perhaps the client/server
>   model handles error handling/reporting much more nicely.

Agreed, we thought of that, but it's not cooked yet. Currently we always 
cleanup. We should not remove the (temporary) image if the test fails. Should we 
keep the guest running upon failure ? Currently we continue to test the next 
guest. We should probably have a configuration flag for that too.

Thanks,
Uri.