Re: [PATCH net-next 0/4] selftests: kselftest_harness: support using xfail

[Jakub Sitnicki <jakub@cloudflare.com>]

On Tue, Feb 13, 2024 at 07:44 AM -08, Jakub Kicinski wrote:
> Hi!
>
> When running selftests for our subsystem in our CI we'd like all
> tests to pass. Currently some tests use SKIP for cases they
> expect to fail, because the kselftest_harness limits the return
> codes to pass/fail/skip.
>
> Clean up and support the use of the full range of ksft exit codes
> under kselftest_harness.
>
> To avoid conflicts and get the functionality into the networking
> tree ASAP I'd like to put the patches on shared branch so that
> both linux-kselftest and net-next can pull it in. Shuah, please
> LMK if that'd work for you, and if so which -rc should I base
> the branch on. Or is merging directly into net-next okay?
>
> Jakub Kicinski (4):
>   selftests: kselftest_harness: pass step via shared memory
>   selftests: kselftest_harness: use KSFT_* exit codes
>   selftests: kselftest_harness: support using xfail
>   selftests: ip_local_port_range: use XFAIL instead of SKIP
>
>  tools/testing/selftests/kselftest_harness.h   | 67 ++++++++++++++-----
>  .../selftests/net/ip_local_port_range.c       |  2 +-
>  2 files changed, 52 insertions(+), 17 deletions(-)

Nice!
We've been ignoring skipped tests in our internal CI.
But this is the wrong approach, as you point out.

For the series:

Tested-by: Jakub Sitnicki <jakub@cloudflare.com>