Re: [PATCH v8 0/5] Rework iotests/check

[Kevin Wolf <kwolf@redhat.com>]

Am 23.01.2021 um 22:04 hat Vladimir Sementsov-Ogievskiy geschrieben:
> v8:
> 
> about linters:
> 
> I didn't modify 297, as Max already staged 297 modifications to test all files.
> 
> Also, now I have two complains:
> +************* Module testenv
> +testenv.py:158:4: R0915: Too many statements (53/50) (too-many-statements)
> +************* Module testrunner
> +testrunner.py:222:4: R0911: Too many return statements (7/6) (too-many-return-statements)
>  Success: no issues found in 5 source files
> 
> And I feel, I'm tired to refactor it now.. Probably we can ignore them in 297. Probably I can
> do some refactoring as a follow-up.

I don't think these warning are very helpful, I would agree with
disabling them (even globally).

When testing this with the other image formats, I found some problems.

1. The first one probably means that we have changed the order of some
   checks: 150 and 178 have reference outputs for raw and qcow2, but no
   other formats.

   Previously, the _supported_fmt line in the test would just skip the test:

   $ build/check -vhdx 150 178
   150      not run    [16:45:46] [16:45:46]                    not suitable for this image format: vhdx
   178      not run    [16:45:46] [16:45:46]                    not suitable for this image format: vhdx

   Now we seem to test first if a reference output exists and fail:

   150   fail       [16:49:18] [16:49:18]   ...                  No qualified output (expected /home/kwolf/source/qemu/tests/qemu-iotests/150.out)
   178   fail       [16:49:18] [16:49:18]   ...                  No qualified output (expected /home/kwolf/source/qemu/tests/qemu-iotests/178.out)

2. Test case 146 for vpc passed previously, it fails now. This seems to
   be because of whitespace that is checked now.

3. Skipped tests display either "..." or "0.1s" as the elapsed time. The
   old check implementation didn't display any time for them. I don't
   really mind either of the three ways, but a consistent result would
   be nice.

Kevin