Re: [Qemu-devel] [KVM-AUTOTEST] [RFC] Future goals for autotest and virtualization tests

[Osier Yang <jyang@redhat.com>]

On 2012年03月08日 20:17, Ademar Reis wrote:
> On Thu, Mar 08, 2012 at 11:54:31AM +0000, Stefan Hajnoczi wrote:
>> On Thu, Mar 8, 2012 at 11:44 AM, Stefan Hajnoczi<stefanha@gmail.com>  wrote:
>>> On Thu, Mar 8, 2012 at 4:00 AM, Lucas Meneghel Rodrigues<lmr@redhat.com>  wrote:
>>>> One of our main goals is to provide useful tools for the qemu community,
>>>> since we have a good number of tests and libraries written to perform
>>>> integration/QA testing for that tool, being successfuly used by a number of
>>>> QA teams that work on qemu. Also, we recently provided a subset of that
>>>> infrastructure to test libvirt, one of our virtualization projects of
>>>> interest.
>>>
>>> Thanks for sharing.
>>
>> One thing I should have added is that my message is about what it
>> would take for me to use autotest and contribute tests.  But now I
>> realize that you might be going for a different model:
>>
>> If you're aiming for a different model where autotest integrates
>> external test suites (i.e. tests wouldn't be written in autotest.git,
>> instead autotest.git would contain snapshots of external test suites),
>> then this proposal seems fine.  Upstream projects like QEMU would
>> develop their own test suite and it would be dropped into autotest or
>> a specific autotest instance.
>>
>
> Yes, that's the idea. We want autotest to be the framework, not
> (just a) collection of tests. We also want each development team
> to implement and maintain their own set of tests, using (or not)
> the goodies from autotest at their discretion.
>
> In summary, autotest is (or is going to be) a framework that
> provides:
>
>   - A test runner, with grid/cluster support and advanced
>     instrumentation
>   - A devel library and set of utilities for test writers
>   - A set of pre-built images (JeOS – Just Enough OS) for
>     test writers
>
> (attached is a picture showing what we want to achieve)
>
> If a project has an internal library or set of utilities that can
> be of general use, they can be submitted to autotest.git for
> inclusion, thus reaching a broader audience.
>
> A short summary of the plans:
>
>   - Tests can live anywhere and each devel team implements and
>     maintains their own set of tests
>   - Usage of the autotest library by test writers is optional
>   - Tests are scripts returning 0 or error (any language)
>   - Tests can be run individually or in sets
>   - Tests should run fast, our target is seconds or a few minutes
>   - The test runner is smart and “just works” by default
>     - Trivial standard output (FAIL, PASS, SKIPPED)
>     - Collect logs, OS data and other stuff (e.g. --record-video!)

Is there a way to collect the logs generated by external test
suite itself? It could be in the specific path.

Osier