Re: [PATCH net-next 2/9] selftests: net: add skip all feature to ksft_run()

[Petr Machata <petrm@nvidia.com>]

Daniel Zahka <daniel.zahka@gmail.com> writes:

> When there is an entire test suite where each test case depends upon
> some feature, e.g., psp, it is easier to state the least common
> denominator of dpendencies up front, rather than doing:
>
> cfg.require_psp()
>
> at the start of each test.
>
> Signed-off-by: Daniel Zahka <daniel.zahka@gmail.com>
> ---
>  tools/testing/selftests/net/lib/py/ksft.py | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/net/lib/py/ksft.py b/tools/testing/selftests/net/lib/py/ksft.py
> index 8e35ed12ed9e..375020d3edf2 100644
> --- a/tools/testing/selftests/net/lib/py/ksft.py
> +++ b/tools/testing/selftests/net/lib/py/ksft.py
> @@ -210,7 +210,7 @@ def _ksft_intr(signum, frame):
>          ksft_pr(f"Ignoring SIGTERM (cnt: {term_cnt}), already exiting...")
>  
>  
> -def ksft_run(cases=None, globs=None, case_pfx=None, args=()):
> +def ksft_run(cases=None, globs=None, case_pfx=None, args=(), skip_all=None):
>      cases = cases or []
>  
>      if globs and case_pfx:
> @@ -241,6 +241,8 @@ def ksft_run(cases=None, globs=None, case_pfx=None, args=()):
>          cnt_key = ""
>  
>          try:
> +            if skip_all:
> +                raise KsftSkipEx()
>              case(*args)
>          except KsftSkipEx as e:
>              comment = "SKIP " + str(e)

Personally I'm not very fond of this. Calling a run helper just to have
it skip all tests... eh, wouldn't it make more sense to just not call
the function at all then? If all tests have PSP as prereq, just have the
test say so and bail out early?

Topic-specific tests are pretty common, so it's not nonsense to have a
facility that supports that. But this API just seems wrong to me. Run
these tests, except don't run them, just show they are all skipped.