Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU

[Anthony Liguori <anthony@codemonkey.ws>]

On 12/28/2011 09:28 AM, Avi Kivity wrote:
> On 12/28/2011 05:01 PM, Avi Kivity wrote:
>> I'd say that running a ping test is a weak version of kvm-autotest's
>> system tests.  Running a synthetic test that pokes values into memory
>> and mmio and sees a packet coming out is a unit test (the latter can in
>> fact be executed without a guest at all, just a table driving calls to
>> the memory and irq APIs).
>>
>
> Consider
>    98d23704138e0
>    7b4252e83f6f7d
>    f7e80adf3cc4
>    c16ada980f43
>    4abf12f4ea8
>
> (found by looking for 'fix' in the commit log and filtering out the
> commits that don't support my case)
>
> how can you reject such patches on the grounds that they're not
> accompanied by unit tests?

That's why I've also proposed qtest.  But having written quite a few qtest unit 
tests by now, you hit the limits of this type of testing pretty quickly.

> only by making it easy to add tests for
> them.  I think it would be hard/impossible to test them with
> linux-as-a-guest, since they fix edge cases that linux doesn't invoke.
> But by having our own driver (often just using qmp to poke at memory),
> we can easily generate the sequence that triggers the error.
>
> We'd probably need a library to support setting up a pci device's BARs,
> but that's easy with qmp/python integration.  You can even poke a small
> executable into memory and execute it directly, if you really need guest
> cpu interaction.

Please review the qtest series.  I think it offers a pretty good approach to 
writing this style of test.  But as I mentioned, you hit the limits pretty quickly.

Regards,

Anthony Liguori

>