[PATCH v1] Documentation: dev-tools: Clarify requirements for result description

[PATCH v1] Documentation: dev-tools: Clarify requirements for result description

[Mark Brown]

Currently the KTAP specification says that a test result line is

  <result> <number> [<description>][ # [<directive>] [<diagnostic data>]]

and the description of a test can be "any sequence of words
(can't include #)" which specifies that there may be more than
one word but does not specify anything other than those words
which might be used to separate the words which probably isn't
what we want.  Given that practically we have tests using a range
of separators for words including combinations of spaces and
combinations of other symbols like underscores or punctuation
let's just clarify that the description can contain any character
other than # (marking the start of the directive/diagnostic) or
newline (marking the end of this test result).

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 Documentation/dev-tools/ktap.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst
index d0a9565b0f44..414c105b10a9 100644
--- a/Documentation/dev-tools/ktap.rst
+++ b/Documentation/dev-tools/ktap.rst
@@ -80,8 +80,8 @@ have the number 1 and the number then must increase by 1 for each additional
 subtest within the same test at the same nesting level.
 
 The description is a description of the test, generally the name of
-the test, and can be any string of words (can't include #). The
-description is optional, but recommended.
+the test, and can be any string of characters other than # or a
+newline.  The description is optional, but recommended.
 
 The directive and any diagnostic data is optional. If either are present, they
 must follow a hash sign, "#".

base-commit: 9abf2313adc1ca1b6180c508c25f22f9395cc780
-- 
2.30.2

RE: [PATCH v1] Documentation: dev-tools: Clarify requirements for result description

[Bird, Tim]

> -----Original Message-----
> From: Mark Brown <broonie@kernel.org>
> 
> Currently the KTAP specification says that a test result line is
> 
>   <result> <number> [<description>][ # [<directive>] [<diagnostic data>]]
> 
> and the description of a test can be "any sequence of words
> (can't include #)" which specifies that there may be more than
> one word but does not specify anything other than those words
> which might be used to separate the words which probably isn't
> what we want.  Given that practically we have tests using a range
> of separators for words including combinations of spaces and
> combinations of other symbols like underscores or punctuation
> let's just clarify that the description can contain any character
> other than # (marking the start of the directive/diagnostic) or
> newline (marking the end of this test result).
> 
> Signed-off-by: Mark Brown <broonie@kernel.org>
> ---
>  Documentation/dev-tools/ktap.rst | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst
> index d0a9565b0f44..414c105b10a9 100644
> --- a/Documentation/dev-tools/ktap.rst
> +++ b/Documentation/dev-tools/ktap.rst
> @@ -80,8 +80,8 @@ have the number 1 and the number then must increase by 1 for each additional
>  subtest within the same test at the same nesting level.
> 
>  The description is a description of the test, generally the name of
> -the test, and can be any string of words (can't include #). The
> -description is optional, but recommended.
> +the test, and can be any string of characters other than # or a
> +newline.  The description is optional, but recommended.
> 
>  The directive and any diagnostic data is optional. If either are present, they
>  must follow a hash sign, "#".
> 
> base-commit: 9abf2313adc1ca1b6180c508c25f22f9395cc780
> --
> 2.30.2

This change is OK by me.

Just a comment while we're on the subject of weird chars in the test description,
I'll describe what Fuego does with these.  In order to create a unique testcase
globally unique identifier (TGUID), that can be used in databases, json, filenames, etc.
Fuego converts spaces and other non-alphanumeric characters in the testcase description
to underscores.

In particular, it's important for Fuego to distinguish between testcase identifiers
and test suite identifiers (and Fuego uses '.' and '-') as delimiters and separators for
those.  The underscore conversion makes collisions possible, but unlikely when a
testcase identifier is qualified with a test suite identifier.

It would be nice if we could get some consistency between LTP and kselftest
(and the rest of the tests throughout the industry) on the delimiters used
(and allowed) for different test element identifiers.  Maybe the kernel
has enough weight in the OSS ecosystem to set a standard here that others can follow.
 -- Tim

Re: [PATCH v1] Documentation: dev-tools: Clarify requirements for result description

[Kees Cook]

On Thu, Nov 17, 2022 at 10:46:36AM +0000, Mark Brown wrote:
> Currently the KTAP specification says that a test result line is
> 
>   <result> <number> [<description>][ # [<directive>] [<diagnostic data>]]
> 
> and the description of a test can be "any sequence of words
> (can't include #)" which specifies that there may be more than
> one word but does not specify anything other than those words
> which might be used to separate the words which probably isn't
> what we want.  Given that practically we have tests using a range
> of separators for words including combinations of spaces and
> combinations of other symbols like underscores or punctuation
> let's just clarify that the description can contain any character
> other than # (marking the start of the directive/diagnostic) or
> newline (marking the end of this test result).
> 
> Signed-off-by: Mark Brown <broonie@kernel.org>

Reviewed-by: Kees Cook <keescook@chromium.org>

-- 
Kees Cook

Re: [PATCH v1] Documentation: dev-tools: Clarify requirements for result description

[David Gow]

[-- Attachment #1: Type: text/plain, Size: 1129 bytes --]

On Thu, Nov 17, 2022 at 6:49 PM Mark Brown <broonie@kernel.org> wrote:
>
> Currently the KTAP specification says that a test result line is
>
>   <result> <number> [<description>][ # [<directive>] [<diagnostic data>]]
>
> and the description of a test can be "any sequence of words
> (can't include #)" which specifies that there may be more than
> one word but does not specify anything other than those words
> which might be used to separate the words which probably isn't
> what we want.  Given that practically we have tests using a range
> of separators for words including combinations of spaces and
> combinations of other symbols like underscores or punctuation
> let's just clarify that the description can contain any character
> other than # (marking the start of the directive/diagnostic) or
> newline (marking the end of this test result).
>
> Signed-off-by: Mark Brown <broonie@kernel.org>
> ---

Thanks for clarifying this: I totally agree that "sequence of words"
wasn't a great description, and this is better.

+Rae in case she has any thoughts.

Reviewed-by: David Gow <davidgow@google.com>

Cheers,
-- David

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4003 bytes --]