From: Jakub Sitnicki <jakub@cloudflare.com>
To: Jakub Kicinski <kuba@kernel.org>
Cc: shuah@kernel.org, keescook@chromium.org,
linux-kselftest@vger.kernel.org, netdev@vger.kernel.org
Subject: Re: [PATCH net-next 3/4] selftests: kselftest_harness: support using xfail
Date: Wed, 14 Feb 2024 22:46:46 +0100 [thread overview]
Message-ID: <87jzn6lnou.fsf@cloudflare.com> (raw)
In-Reply-To: <87o7ciltgh.fsf@cloudflare.com>
On Wed, Feb 14, 2024 at 08:40 PM +01, Jakub Sitnicki wrote:
[...]
> On second thought, if I can suggest a follow up change so this:
>
> ok 17 # XFAIL SCTP doesn't support IP_BIND_ADDRESS_NO_PORT
>
> ... becomes this
>
> ok 17 ip_local_port_range.ip4_stcp.late_bind # XFAIL SCTP doesn't support IP_BIND_ADDRESS_NO_PORT
>
> You see, we parse test results if they are in TAP format. Lack of test
> name for xfail'ed and skip'ed tests makes it difficult to report in CI
> which subtest was it. Happy to contribute it, once this series gets
> applied.
Should have said "harder", not "difficult". That was an overstatement.
Test name can be extracted from diagnostic lines preceeding the status.
# RUN ip_local_port_range.ip4_stcp.late_bind ...
# XFAIL SCTP doesn't support IP_BIND_ADDRESS_NO_PORT
# OK ip_local_port_range.ip4_stcp.late_bind
ok 17 ip_local_port_range.ip4_stcp.late_bind # XFAIL SCTP doesn't support IP_BIND_ADDRESS_NO_PORT
It just makes the TAP parser easier if the test name is included on the
status line. That would be the motivation here. Let me know what you
think.
[...]
next prev parent reply other threads:[~2024-02-14 21:51 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-13 15:44 [PATCH net-next 0/4] selftests: kselftest_harness: support using xfail Jakub Kicinski
2024-02-13 15:44 ` [PATCH net-next 1/4] selftests: kselftest_harness: pass step via shared memory Jakub Kicinski
2024-02-13 17:54 ` Kees Cook
2024-02-19 2:21 ` kernel test robot
2024-02-13 15:44 ` [PATCH net-next 2/4] selftests: kselftest_harness: use KSFT_* exit codes Jakub Kicinski
2024-02-13 17:55 ` Kees Cook
2024-02-13 15:44 ` [PATCH net-next 3/4] selftests: kselftest_harness: support using xfail Jakub Kicinski
2024-02-13 17:57 ` Kees Cook
2024-02-14 19:40 ` Jakub Sitnicki
2024-02-14 21:46 ` Jakub Sitnicki [this message]
2024-02-15 0:25 ` Jakub Kicinski
2024-02-15 0:40 ` Kees Cook
2024-02-15 22:06 ` Kees Cook
2024-02-15 22:42 ` Jakub Kicinski
2024-02-13 15:44 ` [PATCH net-next 4/4] selftests: ip_local_port_range: use XFAIL instead of SKIP Jakub Kicinski
2024-02-13 17:57 ` Kees Cook
2024-02-13 18:00 ` [PATCH net-next 0/4] selftests: kselftest_harness: support using xfail Kees Cook
2024-02-14 10:09 ` Jakub Sitnicki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87jzn6lnou.fsf@cloudflare.com \
--to=jakub@cloudflare.com \
--cc=keescook@chromium.org \
--cc=kuba@kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=shuah@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).