From: Jakub Kicinski <kuba@kernel.org>
To: davem@davemloft.net
Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com,
shuah@kernel.org, petrm@nvidia.com,
linux-kselftest@vger.kernel.org, willemdebruijn.kernel@gmail.com,
Jakub Kicinski <kuba@kernel.org>
Subject: [PATCH net-next v3 1/8] selftests: net: fix counting totals when some checks fail
Date: Wed, 17 Apr 2024 16:11:39 -0700 [thread overview]
Message-ID: <20240417231146.2435572-2-kuba@kernel.org> (raw)
In-Reply-To: <20240417231146.2435572-1-kuba@kernel.org>
Totals currently only pay attention to exceptions, if check fails
(say ksft_eq()) the test case will be counted as pass:
# At /ksft/drivers/net/./ping.py line 18:
# Check failed 1 != 2
not ok 1 ping.test_v4
ok 2 ping.test_v6
ok 3 ping.test_tcp
# Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0
^^^^^^^^^^^^^
Pay attention to the result.
Fixes: b86761ff6374 ("selftests: net: add scaffolding for Netlink tests in Python")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
tools/testing/selftests/net/lib/py/ksft.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/net/lib/py/ksft.py b/tools/testing/selftests/net/lib/py/ksft.py
index 3769b9197213..640dfbf47702 100644
--- a/tools/testing/selftests/net/lib/py/ksft.py
+++ b/tools/testing/selftests/net/lib/py/ksft.py
@@ -106,7 +106,10 @@ KSFT_RESULT = None
continue
ktap_result(KSFT_RESULT, cnt, case)
- totals['pass'] += 1
+ if KSFT_RESULT:
+ totals['pass'] += 1
+ else:
+ totals['fail'] += 1
print(
f"# Totals: pass:{totals['pass']} fail:{totals['fail']} xfail:{totals['xfail']} xpass:0 skip:{totals['skip']} error:0"
--
2.44.0
next prev parent reply other threads:[~2024-04-17 23:11 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-17 23:11 [PATCH net-next v3 0/8] selftests: drv-net: support testing with a remote system Jakub Kicinski
2024-04-17 23:11 ` Jakub Kicinski [this message]
2024-04-17 23:11 ` [PATCH net-next v3 2/8] selftests: net: set the exit code correctly in Python tests Jakub Kicinski
2024-04-17 23:11 ` [PATCH net-next v3 3/8] selftests: drv-net: define endpoint structures Jakub Kicinski
2024-04-17 23:11 ` [PATCH net-next v3 4/8] selftests: drv-net: factor out parsing of the env Jakub Kicinski
2024-04-17 23:11 ` [PATCH net-next v3 5/8] selftests: drv-net: construct environment for running tests which require an endpoint Jakub Kicinski
2024-04-17 23:11 ` [PATCH net-next v3 6/8] selftests: drv-net: add a trivial ping test Jakub Kicinski
2024-04-17 23:11 ` [PATCH net-next v3 7/8] selftests: net: support matching cases by name prefix Jakub Kicinski
2024-04-18 14:26 ` Willem de Bruijn
2024-04-18 19:06 ` Jakub Kicinski
2024-04-18 19:35 ` Willem de Bruijn
2024-04-17 23:11 ` [PATCH net-next v3 8/8] selftests: drv-net: add a TCP ping test case (and useful helpers) Jakub Kicinski
2024-04-18 14:44 ` Willem de Bruijn
2024-04-18 19:48 ` Jakub Kicinski
2024-04-18 23:20 ` [PATCH net-next v3 0/8] selftests: drv-net: support testing with a remote system patchwork-bot+netdevbpf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240417231146.2435572-2-kuba@kernel.org \
--to=kuba@kernel.org \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=linux-kselftest@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=petrm@nvidia.com \
--cc=shuah@kernel.org \
--cc=willemdebruijn.kernel@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).