On 01/12/2018 05:43 AM, Vladimir Sementsov-Ogievskiy wrote:

>>> +++ b/tests/qemu-iotests/201.out
>>> @@ -0,0 +1,5 @@
>>> +.......
>>> +----------------------------------------------------------------------
>>> +Ran 7 tests
>> I'm not a fan of python tests that are difficult to debug.  Your
>> additions to 147 in patch 4/6 are okay (hard to debug, but an
>> incremental addition); but is it possible to rewrite this test in a bit
>> more verbose manner?  See test 194 and this message for more details:
>> https://lists.gnu.org/archive/html/qemu-devel/2017-12/msg00234.html
> 
> hmm, what do you mean by "difficult to debug"? This is a usual python
> unittest based test.

And the list archives show several threads of people complaining that
./check failing with a diff that merely shows:

-.....
+..E..

makes it rather hard to see WHAT test 2 was doing that caused an error
instead of a pass, let alone set up a reproduction scenario on JUST the
failing test.  Yes, a lot of existing iotests use this unittest layout,
and on that grounds, I'm not opposed to adding another one; but test 194
really IS easier to debug when something goes wrong.

> And there 3 test cases, sharing same setUp. Do not you say that unittest
> becomes
> deprecated in qemu? I think, if we have only one testcase, we may use
> 194-like approach,
> but if we have more, it's better to use unittest.

Yes, I think a nice goal for improved testing is to write more
python-based iotests in the style that uses actual output, and not just
the unittest framework, in the test log.  It's not a hard requirement as
long as no one has converted existing tests, but is food for thought.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org