Linux Kernel Selftest development
 help / color / mirror / Atom feed
From: Rae Moar <rmoar@google.com>
To: frowand.list@gmail.com, davidgow@google.com,
	skhan@linuxfoundation.org, keescook@chromium.org,
	Tim.Bird@sony.com, brendanhiggins@google.com
Cc: corbet@lwn.net, guillaume.tucker@collabora.com,
	dlatypov@google.com, kernelci@lists.linux.dev,
	kunit-dev@googlegroups.com, linux-kselftest@vger.kernel.org,
	linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
	Rae Moar <rmoar@google.com>
Subject: [KTAP V2 PATCH] ktap_v2: allow prefix to KTAP lines
Date: Thu, 16 Mar 2023 22:59:26 +0000	[thread overview]
Message-ID: <20230316225926.494921-1-rmoar@google.com> (raw)

Change the KTAP v2 spec to allow variable prefixes to KTAP lines,
instead of fixed indentation of two spaces. However, the prefix must be
constant on the same level of testing (besides unknown lines).

This was proposed by Tim Bird in 2021 and then supported by Frank Rowand
in 2022 (see link below).

Link: https://lore.kernel.org/all/bc6e9ed7-d98b-c4da-2a59-ee0915c18f10@gmail.com/

As cited in the original proposal, it is useful in some Fuego tests to
include an identifier in the prefix. This is an example:

 KTAP version 1
 1..2
 [batch_id 4] KTAP version 1
 [batch_id 4] 1..2
 [batch_id 4] ok 1 cyclictest with 1000 cycles
 [batch_id 4] # problem setting CLOCK_REALTIME
 [batch_id 4] not ok 2 cyclictest with CLOCK_REALTIME
 not ok 1 check realtime
 [batch_id 4] KTAP version 1
 [batch_id 4] 1..1
 [batch_id 4] ok 1 IOZone read/write 4k blocks
 ok 2 check I/O performance

Here is a link to a version of the KUnit parser that is able to parse
variable length prefixes for KTAP version 2. Note that the prefix must
be constant at the same level of testing.

Link: https://kunit-review.googlesource.com/c/linux/+/5710

Signed-off-by: Rae Moar <rmoar@google.com>
---

This idea has already been proposed but I wanted to potentially
restart the discussion by demonstrating this change can by
implemented in the KUnit parser. Let me know what you think.

Note: this patch is based on Frank's ktap_spec_version_2 branch.

 Documentation/dev-tools/ktap.rst | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst
index ff77f4aaa6ef..ac61fdd97096 100644
--- a/Documentation/dev-tools/ktap.rst
+++ b/Documentation/dev-tools/ktap.rst
@@ -192,9 +192,11 @@ starting with another KTAP version line and test plan, and end with the overall
 result. If one of the subtests fail, for example, the parent test should also
 fail.
 
-Additionally, all lines in a subtest should be indented. One level of
-indentation is two spaces: "  ". The indentation should begin at the version
-line and should end before the parent test's result line.
+Additionally, all lines in a subtest should be indented. The standard for one
+level of indentation is two spaces: "  ". However, any prefix for indentation
+is allowed as long as the prefix is consistent throughout that level of
+testing. The indentation should begin at the version line and should end
+before the parent test's result line.
 
 "Unknown lines" are not considered to be lines in a subtest and thus are
 allowed to be either indented or not indented.
@@ -229,6 +231,19 @@ An example format with multiple levels of nested testing:
 	not ok 1 example_test_1
 	ok 2 example_test_2
 
+An example of a test with two nested subtests using prefixes:
+
+::
+
+	KTAP version 2
+	1..1
+	[prefix_1] KTAP version 2
+	[prefix_1] 1..2
+	[prefix_1] ok 1 test_1
+	[prefix_1] ok 2 test_2
+	# example passed
+	ok 1 example
+
 
 Major differences between TAP and KTAP
 --------------------------------------

base-commit: 906f02e42adfbd5ae70d328ee71656ecb602aaf5
-- 
2.40.0.rc1.284.g88254d51c5-goog


             reply	other threads:[~2023-03-16 22:59 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-16 22:59 Rae Moar [this message]
2023-03-27  2:12 ` [KTAP V2 PATCH] ktap_v2: allow prefix to KTAP lines Frank Rowand
2023-03-29 21:14   ` Rae Moar
2023-03-31 20:50     ` Frank Rowand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230316225926.494921-1-rmoar@google.com \
    --to=rmoar@google.com \
    --cc=Tim.Bird@sony.com \
    --cc=brendanhiggins@google.com \
    --cc=corbet@lwn.net \
    --cc=davidgow@google.com \
    --cc=dlatypov@google.com \
    --cc=frowand.list@gmail.com \
    --cc=guillaume.tucker@collabora.com \
    --cc=keescook@chromium.org \
    --cc=kernelci@lists.linux.dev \
    --cc=kunit-dev@googlegroups.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=skhan@linuxfoundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox