netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jakub Kicinski <kuba@kernel.org>
To: davem@davemloft.net
Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com,
	shuah@kernel.org, petrm@nvidia.com,
	linux-kselftest@vger.kernel.org, willemdebruijn.kernel@gmail.com,
	Jakub Kicinski <kuba@kernel.org>
Subject: [PATCH net-next v3 1/8] selftests: net: fix counting totals when some checks fail
Date: Wed, 17 Apr 2024 16:11:39 -0700	[thread overview]
Message-ID: <20240417231146.2435572-2-kuba@kernel.org> (raw)
In-Reply-To: <20240417231146.2435572-1-kuba@kernel.org>

Totals currently only pay attention to exceptions, if check fails
(say ksft_eq()) the test case will be counted as pass:

  # At /ksft/drivers/net/./ping.py line 18:
  # Check failed 1 != 2
  not ok 1 ping.test_v4
  ok 2 ping.test_v6
  ok 3 ping.test_tcp
  # Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0
            ^^^^^^^^^^^^^

Pay attention to the result.

Fixes: b86761ff6374 ("selftests: net: add scaffolding for Netlink tests in Python")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
 tools/testing/selftests/net/lib/py/ksft.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/lib/py/ksft.py b/tools/testing/selftests/net/lib/py/ksft.py
index 3769b9197213..640dfbf47702 100644
--- a/tools/testing/selftests/net/lib/py/ksft.py
+++ b/tools/testing/selftests/net/lib/py/ksft.py
@@ -106,7 +106,10 @@ KSFT_RESULT = None
             continue
 
         ktap_result(KSFT_RESULT, cnt, case)
-        totals['pass'] += 1
+        if KSFT_RESULT:
+            totals['pass'] += 1
+        else:
+            totals['fail'] += 1
 
     print(
         f"# Totals: pass:{totals['pass']} fail:{totals['fail']} xfail:{totals['xfail']} xpass:0 skip:{totals['skip']} error:0"
-- 
2.44.0


  reply	other threads:[~2024-04-17 23:11 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-04-17 23:11 [PATCH net-next v3 0/8] selftests: drv-net: support testing with a remote system Jakub Kicinski
2024-04-17 23:11 ` Jakub Kicinski [this message]
2024-04-17 23:11 ` [PATCH net-next v3 2/8] selftests: net: set the exit code correctly in Python tests Jakub Kicinski
2024-04-17 23:11 ` [PATCH net-next v3 3/8] selftests: drv-net: define endpoint structures Jakub Kicinski
2024-04-17 23:11 ` [PATCH net-next v3 4/8] selftests: drv-net: factor out parsing of the env Jakub Kicinski
2024-04-17 23:11 ` [PATCH net-next v3 5/8] selftests: drv-net: construct environment for running tests which require an endpoint Jakub Kicinski
2024-04-17 23:11 ` [PATCH net-next v3 6/8] selftests: drv-net: add a trivial ping test Jakub Kicinski
2024-04-17 23:11 ` [PATCH net-next v3 7/8] selftests: net: support matching cases by name prefix Jakub Kicinski
2024-04-18 14:26   ` Willem de Bruijn
2024-04-18 19:06     ` Jakub Kicinski
2024-04-18 19:35       ` Willem de Bruijn
2024-04-17 23:11 ` [PATCH net-next v3 8/8] selftests: drv-net: add a TCP ping test case (and useful helpers) Jakub Kicinski
2024-04-18 14:44   ` Willem de Bruijn
2024-04-18 19:48     ` Jakub Kicinski
2024-04-18 23:20 ` [PATCH net-next v3 0/8] selftests: drv-net: support testing with a remote system patchwork-bot+netdevbpf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240417231146.2435572-2-kuba@kernel.org \
    --to=kuba@kernel.org \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=petrm@nvidia.com \
    --cc=shuah@kernel.org \
    --cc=willemdebruijn.kernel@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).