Re: [kvm-unit-tests PATCH 1/3] run_tests.sh: reduce return code ambiguity

["Radim Krčmář" <rkrcmar@redhat.com>]

2015-12-21 13:35-0600, Andrew Jones:
> On Mon, Dec 21, 2015 at 05:31:24PM +0100, Radim Krčmář wrote:
> > 2015-12-17 14:10-0600, Andrew Jones:
>>  > 128 = exited because of signal $? - 128
>>  *     = unit-test failed
>> 
>> (Signal 0 is not used, so we could map 128 to mean "debug-exit probably
>>  wasn't called", but others might not understand our signal convention.
> 
> I think we want 128 to be the beginning of signal space, which goes all
> the way up to 255, in order to allow exit code masking to work.
> 
>>  Anyway, it'd be best for us to start at 200, for `case $? in 2??)` ...)
> 
> Start what at 200?

Signals, signal = $? - 200.  Shell default to decimal representation of
numbers, so using binary steps doesn't give an advantage and 55 is still
a plenty of space.  (I deplore elif cascade on the same variable, but we
can always convert the $? to binary/octal/hex, for `case` decoding. :])

>                    I think we have everything covered above. The mapping
> looks like this
> 
> 0	= success
> 1-63	= unit test failure code
> 64-127	= test suite failure code

77 is not that easy to categorize -- we want to return it from both.

> 128-255	= signal
> 
> which sounds good to me.

To me as well.

>> > Signed-off-by: Andrew Jones <drjones@redhat.com>
>> > ---
>> > diff --git a/run_tests.sh b/run_tests.sh
>> > @@ -54,10 +55,32 @@ function run()
>> >  
>> >      # extra_params in the config file may contain backticks that need to be
>> >      # expanded, so use eval to start qemu
>> > -    eval $cmdline >> test.log
>> > +    errlog=$(mktemp)
>> > +    eval $cmdline >> test.log 2> $errlog
>> | [...]
>> |      cat $errlog >> test.log
>> 
>> This assumes that stderr is always after stdout,
> 
> True. I'm not sure that matters when the unit test, which only uses stdout
> will always output stuff serially with qemu, which could output a mix. But
> your version below is fine by me if we want to pick up the need for the
> pipe and tee.

Yeah, I assume that QEMU can warn during the test, or interact with its
own stdout in an ordered manner.  I don't think it matter much, but
there isn't a significant draw-back.

>>   eval $cmdline 2>&1 >> test.log | tee $errlog >> test.log
>> 
>> has a chance to print lines in wrong order too, but I think it's going
>> to be closer to the original.
> 
> I'll play with it and send a v2 soon.

Thanks, though I am quite distracted during the end of the year, so
"soon" won't be truly appreciated. :)