Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU

[Lucas Meneghel Rodrigues <lmr@redhat.com>]

On 12/29/2011 03:02 PM, Anthony Liguori wrote:
> On 12/29/2011 10:53 AM, Avi Kivity wrote:
>> On 12/29/2011 06:39 PM, Anthony Liguori wrote:
>>
>> It might have made sense to split the kvm-testing functionality of
>> autotest, and have autotest drive that. We could even have called it
>> qemu-test.
>
> I specifically advocated this during Lucas' KVM Forum talk and he was
> strongly opposed to it.

Ok, this is a point where I have failed in communicating things.

I've watched the presentation on video just to see if my memory was not 
betraying me, and I wouldn't say I was 'strongly opposed', it's just 
that there are some practical problems, not impossible to overcome of 
course, but worth thinking.

  * We rely on a fair bit of autotest libraries/API, so by extracting 
the APIs we rely on to start a new project we'd be creating some code 
duplication, and updates/bugfixes on the autotest API wouldn't be easily 
carried on to the new project.
  * I'm not sure that this per se would fix the perceived problems with 
the tests. It'd be nearly the same code, just outside the autotest tree, 
I fail to see how that would be much better, and therefore, justify the 
work on doing this. Of course I might be wrong.

> I think kvm autotest would get a lot more interest if the test cases
> were pulled out of autotest, made more stand alone. They also should be
> more Unix like being divided into individual executables with
> independent command line options over a single do-everything
> configuration file.

You have a point with regards to having the test cases more stand alone, 
but it's not as easy as one would think mainly because of the large 
amount of user configurable options that we have to pass to the tests. 
This is a place where the config file format shines, since we just have 
to override some parameters on different variants to have a test working 
among Windows and Linux guests, for example, all expressed in a concise 
way. The config files might look enormous, but they are actually pretty 
small compared with the amount of test combinations we can get out of 
it. This is nothing I'd call insurmountable as well.

If I get more feedback of people saying "Ok, this would make things 
better for me and make me more interested", then we'll go that route. 
I'm sorry if I gave the impression I was strongly opposed to your idea.