[PATCH 0/2] tracing/selftests: Verbosity improvements to KTAP output

[PATCH 0/2] tracing/selftests: Verbosity improvements to KTAP output

[Mark Brown]

This series aims to improve the usability of the ftrace selftests when
running as part of the kselftest runner, mainly for use with automated
systems.  It fixes the output of verbose mode when run in KTAP output
mode and then enables verbose mode by default when invoked from the
kselftest runner so that the diagnostic information is there by default
when run in automated systems.

I've split this into two patches in case there is a concern with one
part but not the other, especially given the verbosity of the verbose
output when it triggers.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
Mark Brown (2):
      tracing/selftests: Support log output when generating KTAP output
      tracing/selftests: Default to verbose mode when running in kselftest

 tools/testing/selftests/ftrace/ftracetest      | 8 +++++++-
 tools/testing/selftests/ftrace/ftracetest-ktap | 2 +-
 2 files changed, 8 insertions(+), 2 deletions(-)
---
base-commit: 4cece764965020c22cff7665b18a012006359095
change-id: 20240319-kselftest-ftrace-ktap-verbose-72e37957e213

Best regards,
-- 
Mark Brown <broonie@kernel.org>

[PATCH 1/2] tracing/selftests: Support log output when generating KTAP output

[Mark Brown]

When -v is specified ftracetest will dump logs of test execution to the
console which if -K is also specified for KTAP output will result in
output that is not properly KTAP formatted. All that's required for KTAP
formatting is that anything we log have a '#' at the start of the line so
we can improve things by washing the output through a simple read loop.
This will help automated parsers when verbose mode is enabled.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 tools/testing/selftests/ftrace/ftracetest | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index 25d4e0fca385..cce72f8b03dc 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -255,7 +255,13 @@ prlog() { # messages
   [ "$LOG_FILE" ] && printf "$*$newline" | strip_esc >> $LOG_FILE
 }
 catlog() { #file
-  cat $1
+  if [ "${KTAP}" = "1" ]; then
+    cat $1 | while read line ; do
+      echo "# $line"
+    done
+  else
+    cat $1
+  fi
   [ "$LOG_FILE" ] && cat $1 | strip_esc >> $LOG_FILE
 }
 prlog "=== Ftrace unit tests ==="

-- 
2.30.2

[PATCH 2/2] tracing/selftests: Default to verbose mode when running in kselftest

[Mark Brown]

In order to facilitate debugging of issues from automated runs of the ftrace
selftests turn on verbose logging by default when run from the kselftest
runner. This is primarily used by automated systems where developers may
not have direct access to the system so defaulting to providing diagnostic
information which might help debug problems seems like a good idea.

When tests pass no extra output is generated, when they fail a full log of
the test run is provided. Since this really is rather verbose when there are
a large number of test failures or output is slow (eg, with a serial
console) this could substantially increase the run time for the tests which
might present problems with timeout detection for affected systems,
hopefully we keep the tests running well enough that this is not too much
of an issue.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 tools/testing/selftests/ftrace/ftracetest-ktap | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/ftrace/ftracetest-ktap b/tools/testing/selftests/ftrace/ftracetest-ktap
index b3284679ef3a..14e62ef3f3b9 100755
--- a/tools/testing/selftests/ftrace/ftracetest-ktap
+++ b/tools/testing/selftests/ftrace/ftracetest-ktap
@@ -5,4 +5,4 @@
 #
 # Copyright (C) Arm Ltd., 2023
 
-./ftracetest -K
+./ftracetest -K -v

-- 
2.30.2

Re: [PATCH 0/2] tracing/selftests: Verbosity improvements to KTAP output

[Steven Rostedt]

Shuah,

If Masami is OK with this, can you take this through your tree?

Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>

-- Steve


On Mon, 25 Mar 2024 16:15:49 +0000
Mark Brown <broonie@kernel.org> wrote:

> This series aims to improve the usability of the ftrace selftests when
> running as part of the kselftest runner, mainly for use with automated
> systems.  It fixes the output of verbose mode when run in KTAP output
> mode and then enables verbose mode by default when invoked from the
> kselftest runner so that the diagnostic information is there by default
> when run in automated systems.
> 
> I've split this into two patches in case there is a concern with one
> part but not the other, especially given the verbosity of the verbose
> output when it triggers.
> 
> Signed-off-by: Mark Brown <broonie@kernel.org>
> ---
> Mark Brown (2):
>       tracing/selftests: Support log output when generating KTAP output
>       tracing/selftests: Default to verbose mode when running in kselftest
> 
>  tools/testing/selftests/ftrace/ftracetest      | 8 +++++++-
>  tools/testing/selftests/ftrace/ftracetest-ktap | 2 +-
>  2 files changed, 8 insertions(+), 2 deletions(-)
> ---
> base-commit: 4cece764965020c22cff7665b18a012006359095
> change-id: 20240319-kselftest-ftrace-ktap-verbose-72e37957e213
> 
> Best regards,

Re: [PATCH 0/2] tracing/selftests: Verbosity improvements to KTAP output

[Shuah Khan]

On 3/26/24 10:08, Steven Rostedt wrote:
> 
> Shuah,
> 
> If Masami is OK with this, can you take this through your tree?
> 
> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
> 
> -- Steve
> 
> 

Yes. Will do once Masami gives me the okay.

> On Mon, 25 Mar 2024 16:15:49 +0000
> Mark Brown <broonie@kernel.org> wrote:
> 
>> This series aims to improve the usability of the ftrace selftests when
>> running as part of the kselftest runner, mainly for use with automated
>> systems.  It fixes the output of verbose mode when run in KTAP output
>> mode and then enables verbose mode by default when invoked from the
>> kselftest runner so that the diagnostic information is there by default
>> when run in automated systems.
>>
>> I've split this into two patches in case there is a concern with one
>> part but not the other, especially given the verbosity of the verbose
>> output when it triggers.
>>
>> Signed-off-by: Mark Brown <broonie@kernel.org>
>> ---
>> Mark Brown (2):
>>        tracing/selftests: Support log output when generating KTAP output
>>        tracing/selftests: Default to verbose mode when running in kselftest
>>
>>   tools/testing/selftests/ftrace/ftracetest      | 8 +++++++-
>>   tools/testing/selftests/ftrace/ftracetest-ktap | 2 +-
>>   2 files changed, 8 insertions(+), 2 deletions(-)
>> ---
>> base-commit: 4cece764965020c22cff7665b18a012006359095
>> change-id: 20240319-kselftest-ftrace-ktap-verbose-72e37957e213
>>
>> Best regards,
> 

thanks,
-- Shuah

Re: [PATCH 0/2] tracing/selftests: Verbosity improvements to KTAP output

[Shuah Khan]

On 3/26/24 13:11, Shuah Khan wrote:
> On 3/26/24 10:08, Steven Rostedt wrote:
>>
>> Shuah,
>>
>> If Masami is OK with this, can you take this through your tree?
>>
>> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
>>
>> -- Steve
>>
>>
> 
> Yes. Will do once Masami gives me the okay
Hi Masami,

Are you okay with this patch?

thanks,
-- Shuah

Re: [PATCH 0/2] tracing/selftests: Verbosity improvements to KTAP output

[Masami Hiramatsu]

On Thu, 4 Apr 2024 10:32:33 -0600
Shuah Khan <skhan@linuxfoundation.org> wrote:

> On 3/26/24 13:11, Shuah Khan wrote:
> > On 3/26/24 10:08, Steven Rostedt wrote:
> >>
> >> Shuah,
> >>
> >> If Masami is OK with this, can you take this through your tree?
> >>
> >> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org>
> >>
> >> -- Steve
> >>
> >>
> > 
> > Yes. Will do once Masami gives me the okay
> Hi Masami,
> 
> Are you okay with this patch?

Hi Shuah and Mark,

Sorry, I missed this series. This looks good to me.

Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>

Thank you!

> 
> thanks,
> -- Shuah
> 
> 


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

Re: [PATCH 1/2] tracing/selftests: Support log output when generating KTAP output

[Muhammad Usama Anjum]

On 3/25/24 9:15 PM, Mark Brown wrote:
> When -v is specified ftracetest will dump logs of test execution to the
> console which if -K is also specified for KTAP output will result in
> output that is not properly KTAP formatted. All that's required for KTAP
> formatting is that anything we log have a '#' at the start of the line so
> we can improve things by washing the output through a simple read loop.
> This will help automated parsers when verbose mode is enabled.
> 
> Signed-off-by: Mark Brown <broonie@kernel.org>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>

> ---
>  tools/testing/selftests/ftrace/ftracetest | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> index 25d4e0fca385..cce72f8b03dc 100755
> --- a/tools/testing/selftests/ftrace/ftracetest
> +++ b/tools/testing/selftests/ftrace/ftracetest
> @@ -255,7 +255,13 @@ prlog() { # messages
>    [ "$LOG_FILE" ] && printf "$*$newline" | strip_esc >> $LOG_FILE
>  }
>  catlog() { #file
> -  cat $1
> +  if [ "${KTAP}" = "1" ]; then
> +    cat $1 | while read line ; do
> +      echo "# $line"
> +    done
> +  else
> +    cat $1
> +  fi
>    [ "$LOG_FILE" ] && cat $1 | strip_esc >> $LOG_FILE
>  }
>  prlog "=== Ftrace unit tests ==="
> 

-- 
BR,
Muhammad Usama Anjum

Re: [PATCH 2/2] tracing/selftests: Default to verbose mode when running in kselftest

[Muhammad Usama Anjum]

On 3/25/24 9:15 PM, Mark Brown wrote:
> In order to facilitate debugging of issues from automated runs of the ftrace
> selftests turn on verbose logging by default when run from the kselftest
> runner. This is primarily used by automated systems where developers may
> not have direct access to the system so defaulting to providing diagnostic
> information which might help debug problems seems like a good idea.
> 
> When tests pass no extra output is generated, when they fail a full log of
> the test run is provided. Since this really is rather verbose when there are
> a large number of test failures or output is slow (eg, with a serial
> console) this could substantially increase the run time for the tests which
> might present problems with timeout detection for affected systems,
> hopefully we keep the tests running well enough that this is not too much
> of an issue.
> 
> Signed-off-by: Mark Brown <broonie@kernel.org>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>

> ---
>  tools/testing/selftests/ftrace/ftracetest-ktap | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/ftrace/ftracetest-ktap b/tools/testing/selftests/ftrace/ftracetest-ktap
> index b3284679ef3a..14e62ef3f3b9 100755
> --- a/tools/testing/selftests/ftrace/ftracetest-ktap
> +++ b/tools/testing/selftests/ftrace/ftracetest-ktap
> @@ -5,4 +5,4 @@
>  #
>  # Copyright (C) Arm Ltd., 2023
>  
> -./ftracetest -K
> +./ftracetest -K -v
> 

-- 
BR,
Muhammad Usama Anjum