Re: [Qemu-devel] [RFC] Future goals for autotest and virtualization tests

[Anthony Liguori <anthony@codemonkey.ws>]

On 03/08/2012 09:57 AM, Ademar Reis wrote:
> On Thu, Mar 08, 2012 at 04:46:09PM +0100, Kevin Wolf wrote:
>> Am 08.03.2012 15:56, schrieb Anthony Liguori:
>>>> I particularly agreed with basically everything you said on that
>>>> discussion regarding test simplification (I had just joined the
>>>> team back then). To me, autotest has been focusing on QE-level,
>>>> leaving the developer-level test requirements out. Now we're
>>>> attacking this new front, and a lot of the requirements are
>>>> indeed from that discussion.
>>>
>>> If you want to talk about this in terms of "requirements", my requirement is for
>>> "developer-level" tests to live in qemu.git and be integrated into make check.
>>
>> Actually, I think make check should be something that runs _really_
>> quickly. We'll probably want another make check-full or something that
>> runs more extensive tests.
>
> I like make check because it's part of make, but I would suggest:
>
> $ test-runner<dir>  |<file>  ...
>
> This would allow use-cases such as:
>
> [qemu]$ test-runner tests.d/block/
>    (run all block tests)

make check-block

>
> [qemu]$ test-runner tests.d/core/whatever-i-want-to-test.sh
>    (run one individual test)

./tests/core/whatever-i-want-to-test

>
> [qemu]$ test-runner tests.d/
>    (run all tests)

make check

>
> [qemu]$ test-runner tests.d/devices/*pci*
>    (you get the idea)

make check-pci

>
> And, of course:
> [qemu]$ test-runner --remote=autotest.qemu.org tests.d/block

I don't understand what this would do.

Are you proposing test-runner as a replacement for gtest?

Regards,

Anthony Liguori

> (a test path/name is its ID to both the local test runner and the
> remote server)
>
> Cheers,
>    - Ademar
>
>>
>> That's the reason why the patch I proposed earlier today runs only a few
>> qemu-iotests cases during make check. The whole thing would already take
>> too long for everyone to use it. If people run some quick sanity tests
>> for all of qemu plus the full tests in the subsystem in which they're
>> making changes, I think that would be the best we can realistically expect.
>>
>> Kevin
>