Re: [PATCH] iotests: drop group file

[Kevin Wolf <kwolf@redhat.com>]

Am 24.03.2020 um 08:41 hat Vladimir Sementsov-Ogievskiy geschrieben:
> When sending iotests to upstream or do patch porting from one branch
> to another we very often have to resolve conflicts in group file, as
> many absolutely independent features are intersecting by this file.
> These conflicts are simple, but imagine how much time we all have
> already spent on resolving them? Let's finally get rid of group file.
> 
> This patch transposes group info: instead of collecting it in one file,
> let each test define its groups by itself.
> 
> Several steps are done to achive it:
> 
> 1. Define groups in test files automatically:
> 
>     grep '^[0-9]\{3\} ' group | while read line; do
>         file=$(awk '{print $1}' <<< "$line");
>         groups=$(sed -e 's/^... //' <<< "$line");
>         awk "NR==2{print \"# group: $groups\"}1" $file > tmp;
>         cat tmp > $file;
>     done
> 
> 2. Copy groups documentation into docs/devel/testing.rst, which already
>    has a section about iotests.
> 
> 3. Modify check script to work without group file.
> 
>    Here is a logic change: before, even if test do not belong to any
>    group (only iotest 142 currently) it should be defined in group
>    file. Now, test is not forced to define any group. Instead check
>    considers all files with names matching [0-9][0-9][0-9] as tests.

This has both a positive and a negative effect: Now you don't have to
modify another file when you add a new test, but it will be picked up
automatically. However, if you want to disable a test, you could
previously just remove it from groups (or comment it out), and now you
have to delete the test instead.

Downstream, I think we still disable a few tests because we're compiling
out features that are required for some tests to pass, and deleting the
test cases completely would probably move conflicts just to a different
place.

So I think we need a method to distuingish an enabled test that is in no
group from a disabled test.

>    check script is also refactored to make it simple to do next cool
>    thing about iotests: allow meaningful names for test-case files.

This one would also require us to be able to distinguish test case files
from arbitrary other files.

> -#
> -# test-group association ... one line per test
> -#
> -001 rw auto quick
> -002 rw auto quick
> -003 rw auto
> -004 rw auto quick
> -005 img auto quick
> -# 006 was removed, do not reuse

We lose these comments without a replacement. I wonder whether it's
important or if we can think of another way to make sure that numbers
aren't reused. (I'm not completely sure any more why we decided that we
don't want to reuse numbers. Maybe because of backports?)

Kevin