Re: [PATCH nft v2 0/3] add "eval-exit-code" and skip tests based on kernel version

[Florian Westphal <fw@strlen.de>]

Thomas Haller <thaller@redhat.com> wrote:
> This is a follow-up and replaces the two patches:
> 
>   [PATCH nft 1/3] tests/shell: skip "table_onoff" test if kernel patch is missing
>   [PATCH nft 2/3] tests/shell: skip "vlan_8021ad_tag" test instead of failing
> 
> Instead, add a helper script "eval-exit-code" which makes it easy(?) to
> conditionally downgrade a test failure to a SKIP (exit 77) based on the
> kernel version.

I think we should leave things as-is.

If this proves to be an issue (esp. if we have crasher-tests...),
we could make a new subdir, e.g. tests/shell/testcases/kernel, place
thse tests there and then have the 'make check' export a new environment
variable, e.g. 'SKIP_KERNEL_TESTS" or whatever.

But for now I think things are fine, Fedora should catch up soon with
the vlan test case (its already in 6.5.y stable) so this will go away.

And for others this hints that they need to complain to their vendor,
or backport a the fix to resolve this bug.

I think SKIP should be reseved only for tests that fail a feature-probe
dependency.