Re: [PATCH] kvm-unit-test: more documentation and runner script

[Lucas Meneghel Rodrigues <lmr@redhat.com>]

On 02/28/2013 06:51 AM, Gleb Natapov wrote:
> On Thu, Feb 28, 2013 at 11:45:33AM +0200, Michael S. Tsirkin wrote:
>> On Thu, Feb 28, 2013 at 11:06:20AM +0200, Gleb Natapov wrote:
>>> On Wed, Feb 27, 2013 at 11:40:37PM +0200, Michael S. Tsirkin wrote:
>>>> On Wed, Feb 27, 2013 at 06:03:41PM -0300, Lucas Meneghel Rodrigues wrote:
>>>>> On 02/27/2013 05:44 PM, Michael S. Tsirkin wrote:
>>>>>>>> +Using qemu (supported since qemu 1.3):
>>>>>>>> +qemu-system-x86_64 -enable-kvm -device pc-testdev -serial stdio -device isa-debug-exit,iobase=0xf4,iosize=0x4 -kernel ./x86/msr.flat
>>>>>>>
>>>>>>> I think it is worth here to point out that with new qemu, after the
>>>>>>> unittest is done, the exit status of qemu is 1, different from the
>>>>>>> 'old style', whose exit status in successful completion is 0.
>>>>>
>>>>> ^ "comment above"
>>>>>
>>>>>>>
>>>>>>>> +exec ${command} "$@"
>>>>>>>
>>>>>>> ^ What about checking the exit status of qemu here and print
>>>>>>> something like "test $@ PASS" or "test $@ FAIL"?
>>>>>>
>>>>>> How do we know how to interpret it?
>>>>>> Overall I think it's best to rely on test output
>>>>>> than on return status.
>>>>>
>>>>> See comment above. Well, test output may be good for humans, but it
>>>>> is really not good for machines [1], that's why when the test suite
>>>>> was developed, the convention was to make qemu to exit with a given
>>>>> return code on success and others on failure.
>>>>
>>>> Right but given a qemu binary, how do I find out what it is on success
>>>> and what it is on failure?
>>>>
>>> Since you know what device you are using you know expected value for
>>> successful/failure.
>>
>> So exit status is 1 for success 0 for failure?

For qemu-kvm, 0 success, non 0 for failure
For qemu upstream, shift that 1 bit.

> As Lucas said above upstream is different from qemu-kvm unfortunately.
> On upstream you need to shift return value right for 1 bit to get to
> the tests return value (why?) and then 0 will be success otherwise failure.

On the why, It was Anthony's call. See, as the upstream test device 
doesn't have any ports we can write to to make qemu exit, we used 
another device that can do that. I vaguely remember he argued for the 
exit code to be shifted by 1 on that other device, and said it wasn't a 
problem for test suites, since they can define what is a success and 
what is a failure.