Re: [PATCH] selftests/clone3: Report descriptive test names

[Shuah Khan <skhan@linuxfoundation.org>]

On 10/17/23 17:01, Mark Brown wrote:
> The clone3() selftests currently report test results in a format that does
> not mesh entirely well with automation. They log output for each test such
> as:
> 
>    # [1382411] Trying clone3() with flags 0 (size 0)
>    # I am the parent (1382411). My child's pid is 1382412
>    # I am the child, my PID is 1382412
>    # [1382411] clone3() with flags says: 0 expected 0
>    ok 1 [1382411] Result (0) matches expectation (0)
> 
> This is not ideal for automated parsers since the text after the "ok 1" is
> treated as the test name when comparing runs by a lot of automation (tests
> routinely get renumbered due to things like new tests being added based on
> logical groupings). The PID means that the test names will frequently vary
> and the rest of the name being a description of results means several tests
> have identical text there.
> 
> Address this by refactoring things so that we have a static descriptive
> name for each test which we use when logging passes, failures and skips
> and since we now have a stable name for the test to hand log that before
> starting the test to address the common issue reading logs where the test
> name is only printed after any diagnostics. The result is:
> 
>   # Running test 'simple clone3()'
>   # [1562777] Trying clone3() with flags 0 (size 0)
>   # I am the parent (1562777). My child's pid is 1562778
>   # I am the child, my PID is 1562778
>   # [1562777] clone3() with flags says: 0 expected 0
>   ok 1 simple clone3()
> 
> In order to handle skips a bit more neatly this is done in a moderately
> invasive fashion where we move from a sequence of function calls to having
> an array of test parameters. This hopefully also makes it a little easier
> to see what the tests are doing when looking at both the source and the
> logs.
> 


Good change. Thank you.

Applied to linux-kselftest next for Linux 6.7-rc1.
> Signed-off-by: Mark Brown <broonie@kernel.org>
> ---
>   tools/testing/selftests/clone3/clone3.c | 265 +++++++++++++++++++++++---------
>   1 file changed, 192 insertions(+), 73 deletions(-)
> 

thanks,
-- Shuah