[PATCH for-5.1?] iotests: Adjust which tests are quick

[PATCH for-5.1?] iotests: Adjust which tests are quick

[Eric Blake]

A quick run of './check -qcow2 -g migration' shows that test 169 is
NOT quick, but meanwhile several other tests ARE quick.  Let's adjust
the test designations accordingly.

Signed-off-by: Eric Blake <eblake@redhat.com>
---

I noticed this while working on my pending pull request that includes
Vladimir's massive speedup of 199 (but even with his speedup, that test
is still not quick).

 tests/qemu-iotests/group | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 1d0252e1f051..806044642c69 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -112,7 +112,7 @@
 088 rw quick
 089 rw auto quick
 090 rw auto quick
-091 rw migration
+091 rw migration quick
 092 rw quick
 093 throttle
 094 rw quick
@@ -186,7 +186,7 @@
 162 quick
 163 rw
 165 rw quick
-169 rw quick migration
+169 rw migration
 170 rw auto quick
 171 rw quick
 172 auto
@@ -197,9 +197,9 @@
 177 rw auto quick
 178 img
 179 rw auto quick
-181 rw auto migration
+181 rw auto migration quick
 182 rw quick
-183 rw migration
+183 rw migration quick
 184 rw auto quick
 185 rw
 186 rw auto
@@ -216,9 +216,9 @@
 198 rw
 199 rw migration
 200 rw
-201 rw migration
+201 rw migration quick
 202 rw quick
-203 rw auto migration
+203 rw auto migration quick
 204 rw quick
 205 rw quick
 206 rw
-- 
2.27.0

Re: [PATCH for-5.1?] iotests: Adjust which tests are quick

[Vladimir Sementsov-Ogievskiy]

27.07.2020 22:51, Eric Blake wrote:
> A quick run of './check -qcow2 -g migration' shows that test 169 is
> NOT quick, but meanwhile several other tests ARE quick.  Let's adjust
> the test designations accordingly.
> 
> Signed-off-by: Eric Blake <eblake@redhat.com>

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Still, why do we need quick group? make check uses "auto" group..
Some tests are considered important enough to run even not being quick.
Probably, everyone who don't want to run all tests, should run "auto" group,
not "quick"?
I, when want to check my changes, run all tests or limit them with
help of grep. I mostly run tests on tmpfs, so they all are quick enough.
Saving several minutes of cpu work doesn't worth missing a bug..


-- 
Best regards,
Vladimir

Re: [PATCH for-5.1?] iotests: Adjust which tests are quick

[Thomas Huth]

On 27/07/2020 22.07, Vladimir Sementsov-Ogievskiy wrote:
> 27.07.2020 22:51, Eric Blake wrote:
>> A quick run of './check -qcow2 -g migration' shows that test 169 is
>> NOT quick, but meanwhile several other tests ARE quick.  Let's adjust
>> the test designations accordingly.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
> 
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> 
> Still, why do we need quick group? make check uses "auto" group..
> Some tests are considered important enough to run even not being quick.
> Probably, everyone who don't want to run all tests, should run "auto"
> group, not "quick"?
> I, when want to check my changes, run all tests or limit them with
> help of grep. I mostly run tests on tmpfs, so they all are quick enough.
> Saving several minutes of cpu work doesn't worth missing a bug..

I think it still makes at least some sense:

- "quick" is for tests that run fast on the developers systems (i.e.
  only Linux, I guess) - can be used during development for a quick
  check in between when you don't want to wait too long

- "auto" group is for "make check", i.e. the test runs fine and
  reasonable fast on all systems (including macOS, *BSD and weird
  CI pipelines)

- The full check should be done before submitting block-layer related
  code, to make sure that no regression happened

Just my 0.02 €, of course.

 Thomas