* [PATCH] tools/perf/tests/shell/base_probe: Enhance print_overall_results to print summary information
@ 2024-10-14 15:40 Athira Rajeev
2024-10-16 5:13 ` Athira Rajeev
2024-10-17 10:22 ` Michael Petlan
0 siblings, 2 replies; 4+ messages in thread
From: Athira Rajeev @ 2024-10-14 15:40 UTC (permalink / raw)
To: acme, jolsa, adrian.hunter, irogers, namhyung
Cc: linux-perf-users, linuxppc-dev, akanksha, maddy, atrajeev, kjain,
disgoel, hbathini
Currently print_overall_results prints the number of
fails in the summary, example from base_probe tests in
testsuite_probe:
## [ FAIL ] ## perf_probe :: test_invalid_options SUMMARY ::
11 failures found
test_invalid_options contains multiple tests and out
of that 11 failed. Sometimes it could happen that it
is due to missing dependency in the build or environment
dependency.
Example, perf probe -L requires DWARF enabled. otherwise
it fails as below:
./perf probe -L
Error: switch `L' is not available because NO_DWARF=1
"-L" is tested as one of the option in :
for opt in '-a' '-d' '-L' '-V'; do
<<perf probe test>>
print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "missing argument
for $opt"
Here -a and -d doesn't require DWARF. Similarly there
are few other tests requiring DWARF. To hint the user that
missing dwarf could be one issue, update print_overall_results
to print a comment string along with summary hinting the possible
cause. Update test_invalid_options.sh and test_line_semantics.sh
to pass the info about dwarf requirement since these tests
failed when perf is built without DWARF.
With the change:
## [ FAIL ] ## perf_probe :: test_invalid_options SUMMARY ::
11 failures found :: Some of the tests need DWARF to run
Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
---
tools/perf/tests/shell/base_probe/test_invalid_options.sh | 2 +-
tools/perf/tests/shell/base_probe/test_line_semantics.sh | 2 +-
tools/perf/tests/shell/common/init.sh | 3 ++-
3 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/tools/perf/tests/shell/base_probe/test_invalid_options.sh b/tools/perf/tests/shell/base_probe/test_invalid_options.sh
index 1fedfd8b0d0d..e133bb0aa949 100755
--- a/tools/perf/tests/shell/base_probe/test_invalid_options.sh
+++ b/tools/perf/tests/shell/base_probe/test_invalid_options.sh
@@ -75,5 +75,5 @@ done
# print overall results
-print_overall_results "$TEST_RESULT"
+print_overall_results "$TEST_RESULT" "Some of the tests need DWARF to run"
exit $?
diff --git a/tools/perf/tests/shell/base_probe/test_line_semantics.sh b/tools/perf/tests/shell/base_probe/test_line_semantics.sh
index d8f4bde0f585..99f4f56a3292 100755
--- a/tools/perf/tests/shell/base_probe/test_line_semantics.sh
+++ b/tools/perf/tests/shell/base_probe/test_line_semantics.sh
@@ -51,5 +51,5 @@ done
# print overall results
-print_overall_results "$TEST_RESULT"
+print_overall_results "$TEST_RESULT" "Some of the tests need DWARF to run"
exit $?
diff --git a/tools/perf/tests/shell/common/init.sh b/tools/perf/tests/shell/common/init.sh
index 075f17623c8e..0862cbc1c6f7 100644
--- a/tools/perf/tests/shell/common/init.sh
+++ b/tools/perf/tests/shell/common/init.sh
@@ -46,10 +46,11 @@ print_results()
print_overall_results()
{
RETVAL="$1"; shift
+ TASK_COMMENT="$*"
if [ $RETVAL -eq 0 ]; then
_echo "$MALLPASS## [ PASS ] ##$MEND $TEST_NAME :: $THIS_TEST_NAME SUMMARY"
else
- _echo "$MALLFAIL## [ FAIL ] ##$MEND $TEST_NAME :: $THIS_TEST_NAME SUMMARY :: $RETVAL failures found"
+ _echo "$MALLFAIL## [ FAIL ] ##$MEND $TEST_NAME :: $THIS_TEST_NAME SUMMARY :: $RETVAL failures found :: $TASK_COMMENT"
fi
return $RETVAL
}
--
2.43.5
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] tools/perf/tests/shell/base_probe: Enhance print_overall_results to print summary information
2024-10-14 15:40 [PATCH] tools/perf/tests/shell/base_probe: Enhance print_overall_results to print summary information Athira Rajeev
@ 2024-10-16 5:13 ` Athira Rajeev
2024-10-17 10:22 ` Michael Petlan
1 sibling, 0 replies; 4+ messages in thread
From: Athira Rajeev @ 2024-10-16 5:13 UTC (permalink / raw)
To: vmolnaro, Arnaldo Carvalho de Melo, Namhyung Kim
Cc: jolsa, adrian.hunter, irogers, linux-perf-users, linuxppc-dev,
akanksha, maddy, kjain, disgoel, hbathini
> On 14 Oct 2024, at 9:10 PM, Athira Rajeev <atrajeev@linux.vnet.ibm.com> wrote:
>
> Currently print_overall_results prints the number of
> fails in the summary, example from base_probe tests in
> testsuite_probe:
>
> ## [ FAIL ] ## perf_probe :: test_invalid_options SUMMARY ::
> 11 failures found
>
> test_invalid_options contains multiple tests and out
> of that 11 failed. Sometimes it could happen that it
> is due to missing dependency in the build or environment
> dependency.
>
> Example, perf probe -L requires DWARF enabled. otherwise
> it fails as below:
> ./perf probe -L
> Error: switch `L' is not available because NO_DWARF=1
>
> "-L" is tested as one of the option in :
> for opt in '-a' '-d' '-L' '-V'; do
> <<perf probe test>>
> print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "missing argument
> for $opt"
>
> Here -a and -d doesn't require DWARF. Similarly there
> are few other tests requiring DWARF. To hint the user that
> missing dwarf could be one issue, update print_overall_results
> to print a comment string along with summary hinting the possible
> cause. Update test_invalid_options.sh and test_line_semantics.sh
> to pass the info about dwarf requirement since these tests
> failed when perf is built without DWARF.
>
> With the change:
> ## [ FAIL ] ## perf_probe :: test_invalid_options SUMMARY ::
> 11 failures found :: Some of the tests need DWARF to run
>
> Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
Adding Veronika in the mail. Sorry missed to add initially. Please share review comments on this fix.
Thanks
Athira
> ---
> tools/perf/tests/shell/base_probe/test_invalid_options.sh | 2 +-
> tools/perf/tests/shell/base_probe/test_line_semantics.sh | 2 +-
> tools/perf/tests/shell/common/init.sh | 3 ++-
> 3 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/tools/perf/tests/shell/base_probe/test_invalid_options.sh b/tools/perf/tests/shell/base_probe/test_invalid_options.sh
> index 1fedfd8b0d0d..e133bb0aa949 100755
> --- a/tools/perf/tests/shell/base_probe/test_invalid_options.sh
> +++ b/tools/perf/tests/shell/base_probe/test_invalid_options.sh
> @@ -75,5 +75,5 @@ done
>
>
> # print overall results
> -print_overall_results "$TEST_RESULT"
> +print_overall_results "$TEST_RESULT" "Some of the tests need DWARF to run"
> exit $?
> diff --git a/tools/perf/tests/shell/base_probe/test_line_semantics.sh b/tools/perf/tests/shell/base_probe/test_line_semantics.sh
> index d8f4bde0f585..99f4f56a3292 100755
> --- a/tools/perf/tests/shell/base_probe/test_line_semantics.sh
> +++ b/tools/perf/tests/shell/base_probe/test_line_semantics.sh
> @@ -51,5 +51,5 @@ done
>
>
> # print overall results
> -print_overall_results "$TEST_RESULT"
> +print_overall_results "$TEST_RESULT" "Some of the tests need DWARF to run"
> exit $?
> diff --git a/tools/perf/tests/shell/common/init.sh b/tools/perf/tests/shell/common/init.sh
> index 075f17623c8e..0862cbc1c6f7 100644
> --- a/tools/perf/tests/shell/common/init.sh
> +++ b/tools/perf/tests/shell/common/init.sh
> @@ -46,10 +46,11 @@ print_results()
> print_overall_results()
> {
> RETVAL="$1"; shift
> + TASK_COMMENT="$*"
> if [ $RETVAL -eq 0 ]; then
> _echo "$MALLPASS## [ PASS ] ##$MEND $TEST_NAME :: $THIS_TEST_NAME SUMMARY"
> else
> - _echo "$MALLFAIL## [ FAIL ] ##$MEND $TEST_NAME :: $THIS_TEST_NAME SUMMARY :: $RETVAL failures found"
> + _echo "$MALLFAIL## [ FAIL ] ##$MEND $TEST_NAME :: $THIS_TEST_NAME SUMMARY :: $RETVAL failures found :: $TASK_COMMENT"
> fi
> return $RETVAL
> }
> --
> 2.43.5
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] tools/perf/tests/shell/base_probe: Enhance print_overall_results to print summary information
2024-10-14 15:40 [PATCH] tools/perf/tests/shell/base_probe: Enhance print_overall_results to print summary information Athira Rajeev
2024-10-16 5:13 ` Athira Rajeev
@ 2024-10-17 10:22 ` Michael Petlan
2024-10-18 5:34 ` Athira Rajeev
1 sibling, 1 reply; 4+ messages in thread
From: Michael Petlan @ 2024-10-17 10:22 UTC (permalink / raw)
To: Athira Rajeev
Cc: acme, jolsa, adrian.hunter, irogers, namhyung, linux-perf-users,
linuxppc-dev, akanksha, maddy, kjain, disgoel, hbathini
On Mon, 14 Oct 2024, Athira Rajeev wrote:
> Currently print_overall_results prints the number of
> fails in the summary, example from base_probe tests in
> testsuite_probe:
>
> ## [ FAIL ] ## perf_probe :: test_invalid_options SUMMARY ::
> 11 failures found
>
> test_invalid_options contains multiple tests and out
> of that 11 failed. Sometimes it could happen that it
> is due to missing dependency in the build or environment
> dependency.
>
> Example, perf probe -L requires DWARF enabled. otherwise
> it fails as below:
> ./perf probe -L
> Error: switch `L' is not available because NO_DWARF=1
>
> "-L" is tested as one of the option in :
> for opt in '-a' '-d' '-L' '-V'; do
> <<perf probe test>>
> print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "missing argument
> for $opt"
>
> Here -a and -d doesn't require DWARF. Similarly there
> are few other tests requiring DWARF. To hint the user that
> missing dwarf could be one issue, update print_overall_results
> to print a comment string along with summary hinting the possible
> cause. Update test_invalid_options.sh and test_line_semantics.sh
> to pass the info about dwarf requirement since these tests
> failed when perf is built without DWARF.
>
> With the change:
> ## [ FAIL ] ## perf_probe :: test_invalid_options SUMMARY ::
> 11 failures found :: Some of the tests need DWARF to run
Hello Athira,
I admit that the idea of some hint why the test failed
might be useful, however a static hint that might or
might not be related to the particular failure seems to
be rather misleading to me. If the test fails for any
other reason, the user is still told about DWARF which
might not be true.
I think that such hints would need some test result post-
processing to determine the actual reason.
Michael
>
> Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
> ---
> tools/perf/tests/shell/base_probe/test_invalid_options.sh | 2 +-
> tools/perf/tests/shell/base_probe/test_line_semantics.sh | 2 +-
> tools/perf/tests/shell/common/init.sh | 3 ++-
> 3 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/tools/perf/tests/shell/base_probe/test_invalid_options.sh b/tools/perf/tests/shell/base_probe/test_invalid_options.sh
> index 1fedfd8b0d0d..e133bb0aa949 100755
> --- a/tools/perf/tests/shell/base_probe/test_invalid_options.sh
> +++ b/tools/perf/tests/shell/base_probe/test_invalid_options.sh
> @@ -75,5 +75,5 @@ done
>
>
> # print overall results
> -print_overall_results "$TEST_RESULT"
> +print_overall_results "$TEST_RESULT" "Some of the tests need DWARF to run"
> exit $?
> diff --git a/tools/perf/tests/shell/base_probe/test_line_semantics.sh b/tools/perf/tests/shell/base_probe/test_line_semantics.sh
> index d8f4bde0f585..99f4f56a3292 100755
> --- a/tools/perf/tests/shell/base_probe/test_line_semantics.sh
> +++ b/tools/perf/tests/shell/base_probe/test_line_semantics.sh
> @@ -51,5 +51,5 @@ done
>
>
> # print overall results
> -print_overall_results "$TEST_RESULT"
> +print_overall_results "$TEST_RESULT" "Some of the tests need DWARF to run"
> exit $?
> diff --git a/tools/perf/tests/shell/common/init.sh b/tools/perf/tests/shell/common/init.sh
> index 075f17623c8e..0862cbc1c6f7 100644
> --- a/tools/perf/tests/shell/common/init.sh
> +++ b/tools/perf/tests/shell/common/init.sh
> @@ -46,10 +46,11 @@ print_results()
> print_overall_results()
> {
> RETVAL="$1"; shift
> + TASK_COMMENT="$*"
> if [ $RETVAL -eq 0 ]; then
> _echo "$MALLPASS## [ PASS ] ##$MEND $TEST_NAME :: $THIS_TEST_NAME SUMMARY"
> else
> - _echo "$MALLFAIL## [ FAIL ] ##$MEND $TEST_NAME :: $THIS_TEST_NAME SUMMARY :: $RETVAL failures found"
> + _echo "$MALLFAIL## [ FAIL ] ##$MEND $TEST_NAME :: $THIS_TEST_NAME SUMMARY :: $RETVAL failures found :: $TASK_COMMENT"
> fi
> return $RETVAL
> }
> --
> 2.43.5
>
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] tools/perf/tests/shell/base_probe: Enhance print_overall_results to print summary information
2024-10-17 10:22 ` Michael Petlan
@ 2024-10-18 5:34 ` Athira Rajeev
0 siblings, 0 replies; 4+ messages in thread
From: Athira Rajeev @ 2024-10-18 5:34 UTC (permalink / raw)
To: Michael Petlan
Cc: acme, jolsa, adrian.hunter, irogers, namhyung, linux-perf-users,
linuxppc-dev, akanksha, maddy, kjain, disgoel, hbathini
> On 17 Oct 2024, at 3:52 PM, Michael Petlan <mpetlan@redhat.com> wrote:
>
> On Mon, 14 Oct 2024, Athira Rajeev wrote:
>> Currently print_overall_results prints the number of
>> fails in the summary, example from base_probe tests in
>> testsuite_probe:
>>
>> ## [ FAIL ] ## perf_probe :: test_invalid_options SUMMARY ::
>> 11 failures found
>>
>> test_invalid_options contains multiple tests and out
>> of that 11 failed. Sometimes it could happen that it
>> is due to missing dependency in the build or environment
>> dependency.
>>
>> Example, perf probe -L requires DWARF enabled. otherwise
>> it fails as below:
>> ./perf probe -L
>> Error: switch `L' is not available because NO_DWARF=1
>>
>> "-L" is tested as one of the option in :
>> for opt in '-a' '-d' '-L' '-V'; do
>> <<perf probe test>>
>> print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "missing argument
>> for $opt"
>>
>> Here -a and -d doesn't require DWARF. Similarly there
>> are few other tests requiring DWARF. To hint the user that
>> missing dwarf could be one issue, update print_overall_results
>> to print a comment string along with summary hinting the possible
>> cause. Update test_invalid_options.sh and test_line_semantics.sh
>> to pass the info about dwarf requirement since these tests
>> failed when perf is built without DWARF.
>>
>> With the change:
>> ## [ FAIL ] ## perf_probe :: test_invalid_options SUMMARY ::
>> 11 failures found :: Some of the tests need DWARF to run
>
> Hello Athira,
>
> I admit that the idea of some hint why the test failed
> might be useful, however a static hint that might or
> might not be related to the particular failure seems to
> be rather misleading to me. If the test fails for any
> other reason, the user is still told about DWARF which
> might not be true.
>
> I think that such hints would need some test result post-
> processing to determine the actual reason.
Sure, agree to your point. I will add post-processing check for dwarf and accordingly add the summary information
Will post a V2
Thanks
Athira
>
> Michael
>
>>
>> Signed-off-by: Athira Rajeev <atrajeev@linux.vnet.ibm.com>
>> ---
>> tools/perf/tests/shell/base_probe/test_invalid_options.sh | 2 +-
>> tools/perf/tests/shell/base_probe/test_line_semantics.sh | 2 +-
>> tools/perf/tests/shell/common/init.sh | 3 ++-
>> 3 files changed, 4 insertions(+), 3 deletions(-)
>>
>> diff --git a/tools/perf/tests/shell/base_probe/test_invalid_options.sh b/tools/perf/tests/shell/base_probe/test_invalid_options.sh
>> index 1fedfd8b0d0d..e133bb0aa949 100755
>> --- a/tools/perf/tests/shell/base_probe/test_invalid_options.sh
>> +++ b/tools/perf/tests/shell/base_probe/test_invalid_options.sh
>> @@ -75,5 +75,5 @@ done
>>
>>
>> # print overall results
>> -print_overall_results "$TEST_RESULT"
>> +print_overall_results "$TEST_RESULT" "Some of the tests need DWARF to run"
>> exit $?
>> diff --git a/tools/perf/tests/shell/base_probe/test_line_semantics.sh b/tools/perf/tests/shell/base_probe/test_line_semantics.sh
>> index d8f4bde0f585..99f4f56a3292 100755
>> --- a/tools/perf/tests/shell/base_probe/test_line_semantics.sh
>> +++ b/tools/perf/tests/shell/base_probe/test_line_semantics.sh
>> @@ -51,5 +51,5 @@ done
>>
>>
>> # print overall results
>> -print_overall_results "$TEST_RESULT"
>> +print_overall_results "$TEST_RESULT" "Some of the tests need DWARF to run"
>> exit $?
>> diff --git a/tools/perf/tests/shell/common/init.sh b/tools/perf/tests/shell/common/init.sh
>> index 075f17623c8e..0862cbc1c6f7 100644
>> --- a/tools/perf/tests/shell/common/init.sh
>> +++ b/tools/perf/tests/shell/common/init.sh
>> @@ -46,10 +46,11 @@ print_results()
>> print_overall_results()
>> {
>> RETVAL="$1"; shift
>> + TASK_COMMENT="$*"
>> if [ $RETVAL -eq 0 ]; then
>> _echo "$MALLPASS## [ PASS ] ##$MEND $TEST_NAME :: $THIS_TEST_NAME SUMMARY"
>> else
>> - _echo "$MALLFAIL## [ FAIL ] ##$MEND $TEST_NAME :: $THIS_TEST_NAME SUMMARY :: $RETVAL failures found"
>> + _echo "$MALLFAIL## [ FAIL ] ##$MEND $TEST_NAME :: $THIS_TEST_NAME SUMMARY :: $RETVAL failures found :: $TASK_COMMENT"
>> fi
>> return $RETVAL
>> }
>> --
>> 2.43.5
>>
>>
>>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2024-10-18 5:35 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-10-14 15:40 [PATCH] tools/perf/tests/shell/base_probe: Enhance print_overall_results to print summary information Athira Rajeev
2024-10-16 5:13 ` Athira Rajeev
2024-10-17 10:22 ` Michael Petlan
2024-10-18 5:34 ` Athira Rajeev
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).