[PATCH 00/11] perftool-testsuite 2nd batch

linux-perf-users.vger.kernel.org archive mirror
 help / color / mirror / Atom feed

* [PATCH 00/11] perftool-testsuite 2nd batch
@ 2024-06-24 13:07 Michael Petlan
  2024-06-24 13:07 ` [PATCH 01/11] perf tests shell: Skip base_* dirs in test script search Michael Petlan
                   ` (11 more replies)
  0 siblings, 12 replies; 43+ messages in thread
From: Michael Petlan @ 2024-06-24 13:07 UTC (permalink / raw)
  To: linux-perf-users, vmolnaro, acme, namhyung
  Cc: acme, irogers, atrajeev, masami.hiramatsu.pt

Hello,

posting a second batch of tests from perftool-testsuite [1], as a part
of the testsuite upstreaming effort.

This batch brings several perf-probe tests ideated by Masami Hiramatsu 
and a perf-report test.

We have also refactored settings files and fixed shellcheck warnings.

Finally, I have noticed that the scripts are run twice, once run by the
driver (e.g. perftool-testsuite_*.sh) and once by perf-test's testsuite
construction feature (that adds all executable shell scripts in all
sub directories automatically). This is undesirable, so I have fixed
it for now by skipping "base_*" dirs in the search. I see that this
approach is not ideal, because the aim of the search seems to be to get
rid of various test drivers, unify the way the tests are detected and
run, while allowing them to be in subdirectories, like the Carsten
Haitzler's coresight tests. However, I still think this is better
solution that just having a pile of scripts without hierarchy that
are grabbed and ordered into one-level queue of tests. Apart from
allowing the tests to share some setup procedure, etc., is helps to
keep the structure more clear.

A better design for future might be to have shell tests ordered in
directories, such as

  # cd tests/shell/
  # find . -name \*.sh
  coresight/asm_pure_loop.sh
  coresight/memcpy_thread_16k_10.sh
  coresight/thread_loop_check_tid_10.sh
  coresight/thread_loop_check_tid_2.sh
  coresight/unroll_loop_thread_10.sh
  base_probe/test_adding_kernel.sh
  base_probe/test_adding_blacklisted.sh
  base_probe/test_basic.sh
  base_probe/test_invalid_options.sh
  base_probe/test_line_semantics.sh
  base_report/setup.sh
  base_report/test_basic.sh
  [...]  

The directories might also contain some "header" file that would
bear the name of the parent testcase for all the tests inside it
(e.g. "perf probe shell tests").

Then, no separate drivers would be needed, perf-test testsuite-
creation mechanism would search for the shell scripts as it does
now, but it would create multilevel structure with subtests,
such as:

  97: perf probe shell tests
  97:1: perf probe basic functionality
  97:2: perf probe tests with arguments
  97:3: perf probe invalid options handling
  98: CoreSight tests
  98:1: CoreSight / ASM Pure loop
  98:2: CoreSight / Thread Loop ...
  [...]

... similar to how C-tests can be organized. In case there is
a setup.sh file in such a subdirectory, it would be run too.
With properly designed names, also tag-like (substring matching)
approach could be used to select desired tests (i.e. to override
the tree structure and pick tests related to some topic, no matter
if they are in base_stat or base_record subdirectory).

This plan also counts with processing the current test scripts and
possibly putting them to best matching place in the structure.

There is already a bunch of shell tests and I thinkg it might be
good to organize them into a structure.

Thoughts and ideas are welcome.

Thanks and regards,

Michael

[1] https://github.com/rfmvh/perftool-testsuite

Michael Petlan (2):
  perf tests shell: Skip base_* dirs in test script search
  perf testsuite: Install perf-report tests

Veronika Molnarova (9):
  perf testsuite: Merge settings files for shell tests
  perf testsuite: Fix shellcheck warnings
  perf testsuite probe: Add test for blacklisted kprobes handling
  perf testsuite probe: Add test for basic perf-probe options
  perf testsuite probe: Add test for invalid options
  perf testsuite probe: Add test for line semantics
  perf testsuite: Add common output checking helper
  perf testsuite report: Add test for perf-report basic functionality
  perf testsuite report: Add test case for perf report

 tools/perf/Makefile.perf                      |   2 +
 tools/perf/tests/shell/base_probe/settings.sh |  48 -----
 .../base_probe/test_adding_blacklisted.sh     |  65 ++++++
 .../shell/base_probe/test_adding_kernel.sh    |   3 -
 .../perf/tests/shell/base_probe/test_basic.sh |  78 ++++++++
 .../shell/base_probe/test_invalid_options.sh  |  77 +++++++
 .../shell/base_probe/test_line_semantics.sh   |  53 +++++
 tools/perf/tests/shell/base_report/setup.sh   |  30 +++
 .../shell/base_report/stderr-whitelist.txt    |   5 +
 .../tests/shell/base_report/test_basic.sh     | 188 ++++++++++++++++++
 .../shell/common/check_errors_whitelisted.pl  |  50 +++++
 tools/perf/tests/shell/common/init.sh         |  31 ++-
 tools/perf/tests/shell/common/settings.sh     |  28 ++-
 .../tests/shell/perftool-testsuite_report.sh  |  23 +++
 tools/perf/tests/tests-scripts.c              |   2 +
 15 files changed, 627 insertions(+), 56 deletions(-)
 delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh
 create mode 100755 tools/perf/tests/shell/base_report/setup.sh
 create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
 create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh
 create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl
 create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh

-- 
2.43.0

^ permalink raw reply	[flat|nested] 43+ messages in thread

* [PATCH 01/11] perf tests shell: Skip base_* dirs in test script search
  2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
@ 2024-06-24 13:07 ` Michael Petlan
  2024-06-24 13:08 ` [PATCH 02/11] perf testsuite: Merge settings files for shell tests Michael Petlan
                   ` (10 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: Michael Petlan @ 2024-06-24 13:07 UTC (permalink / raw)
  To: linux-perf-users, vmolnaro, acme, namhyung
  Cc: acme, irogers, atrajeev, masami.hiramatsu.pt

The test scripts in base_* directories currently have their own drivers
that run them. Before this patch, the shell test-suite generator causes
them to run twice. Fix that by skipping them in the generator.

A cleaner solution (for future) will be to use the directory structure
idea (introduced by Carsten Haitzler in 7391db645938 ("perf test:
Refactor shell tests allowing subdirs")) to generate test entries with
subtests, like:

$ perf test list
[...]
97: perf probe shell tests
97:1: perf probe basic functionality
97:2: perf probe tests with arguments
97:3: perf probe invalid options handling
[...]

There is already a lot of shell test scripts and many are about to come,
so there is a need for some hierarchy.

Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 tools/perf/tests/tests-scripts.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/perf/tests/tests-scripts.c b/tools/perf/tests/tests-scripts.c
index e2042b368269..01971e997425 100644
--- a/tools/perf/tests/tests-scripts.c
+++ b/tools/perf/tests/tests-scripts.c
@@ -222,6 +222,8 @@ static void append_scripts_in_dir(int dir_fd,
 			if (!S_ISDIR(st.st_mode))
 				continue;
 		}
+		if (strncmp(ent->d_name, "base_", 5) == 0)
+			continue; /* Skip scripts that have a separate driver. */
 		fd = openat(dir_fd, ent->d_name, O_PATH);
 		append_scripts_in_dir(fd, result, result_sz);
 	}
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 02/11] perf testsuite: Merge settings files for shell tests
  2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
  2024-06-24 13:07 ` [PATCH 01/11] perf tests shell: Skip base_* dirs in test script search Michael Petlan
@ 2024-06-24 13:08 ` Michael Petlan
  2024-06-24 13:08 ` [PATCH 03/11] perf testsuite: Fix shellcheck warnings Michael Petlan
                   ` (9 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: Michael Petlan @ 2024-06-24 13:08 UTC (permalink / raw)
  To: linux-perf-users, vmolnaro, acme, namhyung
  Cc: acme, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Merge perf testsuite setting files into common settings to reduce
duplicates and prevent errors.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 tools/perf/tests/shell/base_probe/settings.sh | 48 -------------------
 .../shell/base_probe/test_adding_kernel.sh    |  3 --
 tools/perf/tests/shell/common/init.sh         | 23 +++++++++
 tools/perf/tests/shell/common/settings.sh     | 23 +++++++++
 4 files changed, 46 insertions(+), 51 deletions(-)
 delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh

diff --git a/tools/perf/tests/shell/base_probe/settings.sh b/tools/perf/tests/shell/base_probe/settings.sh
deleted file mode 100644
index 123621c7f95e..000000000000
--- a/tools/perf/tests/shell/base_probe/settings.sh
+++ /dev/null
@@ -1,48 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-#
-#	settings.sh of perf_probe test
-#	Author: Michael Petlan <mpetlan@redhat.com>
-#	Author: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
-#
-
-export TEST_NAME="perf_probe"
-
-export MY_ARCH=`arch`
-
-if [ -n "$PERFSUITE_RUN_DIR" ]; then
-	# when $PERFSUITE_RUN_DIR is set to something, all the logs and temp files will be placed there
-	# --> the $PERFSUITE_RUN_DIR/perf_something/examples and $PERFSUITE_RUN_DIR/perf_something/logs
-	#     dirs will be used for that
-	export PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`
-	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
-	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
-	test -d "$MAKE_TARGET_DIR" || mkdir -p "$MAKE_TARGET_DIR"
-	export LOGS_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME/logs"
-	test -d "$LOGS_DIR" || mkdir -p "$LOGS_DIR"
-else
-	# when $PERFSUITE_RUN_DIR is not set, logs will be placed here
-	export CURRENT_TEST_DIR="."
-	export LOGS_DIR="."
-fi
-
-check_kprobes_available()
-{
-	test -e /sys/kernel/debug/tracing/kprobe_events
-}
-
-check_uprobes_available()
-{
-	test -e /sys/kernel/debug/tracing/uprobe_events
-}
-
-clear_all_probes()
-{
-	echo 0 > /sys/kernel/debug/tracing/events/enable
-	check_kprobes_available && echo > /sys/kernel/debug/tracing/kprobe_events
-	check_uprobes_available && echo > /sys/kernel/debug/tracing/uprobe_events
-}
-
-check_sdt_support()
-{
-	$CMD_PERF list sdt | grep sdt > /dev/null 2> /dev/null
-}
diff --git a/tools/perf/tests/shell/base_probe/test_adding_kernel.sh b/tools/perf/tests/shell/base_probe/test_adding_kernel.sh
index 63bb8974b38e..3363f8c57e46 100755
--- a/tools/perf/tests/shell/base_probe/test_adding_kernel.sh
+++ b/tools/perf/tests/shell/base_probe/test_adding_kernel.sh
@@ -15,10 +15,7 @@
 
 # include working environment
 . ../common/init.sh
-. ./settings.sh
 
-# shellcheck disable=SC2034 # the variable is later used after the working environment is included
-THIS_TEST_NAME=`basename $0 .sh`
 TEST_RESULT=0
 
 TEST_PROBE=${TEST_PROBE:-"inode_permission"}
diff --git a/tools/perf/tests/shell/common/init.sh b/tools/perf/tests/shell/common/init.sh
index aadeaf782e03..b99fba50ba94 100644
--- a/tools/perf/tests/shell/common/init.sh
+++ b/tools/perf/tests/shell/common/init.sh
@@ -115,3 +115,26 @@ detect_amd()
 	# 1 = is not AMD or unknown
 	grep "vendor_id" < /proc/cpuinfo | grep -q "AMD"
 }
+
+# base probe utility
+check_kprobes_available()
+{
+	test -e /sys/kernel/debug/tracing/kprobe_events
+}
+
+check_uprobes_available()
+{
+	test -e /sys/kernel/debug/tracing/uprobe_events
+}
+
+clear_all_probes()
+{
+	echo 0 > /sys/kernel/debug/tracing/events/enable
+	check_kprobes_available && echo > /sys/kernel/debug/tracing/kprobe_events
+	check_uprobes_available && echo > /sys/kernel/debug/tracing/uprobe_events
+}
+
+check_sdt_support()
+{
+	$CMD_PERF list sdt | grep sdt > /dev/null 2> /dev/null
+}
diff --git a/tools/perf/tests/shell/common/settings.sh b/tools/perf/tests/shell/common/settings.sh
index 361641dbaaad..dcdb7d49ac00 100644
--- a/tools/perf/tests/shell/common/settings.sh
+++ b/tools/perf/tests/shell/common/settings.sh
@@ -65,6 +65,29 @@ else
 	export MEND=""
 fi
 
+### general info
+DIR_PATH=`dirname "$(readlink -e "$0")"`
+
+export TEST_NAME=`basename $DIR_PATH | sed 's/base/perf/'`
+export MY_ARCH=`arch`
+
+# storing logs and temporary files variables
+if [ -n "$PERFSUITE_RUN_DIR" ]; then
+	# when $PERFSUITE_RUN_DIR is set to something, all the logs and temp files will be placed there
+	# --> the $PERFSUITE_RUN_DIR/perf_something/examples and $PERFSUITE_RUN_DIR/perf_something/logs
+	#     dirs will be used for that
+	export PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`
+	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
+	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
+	export LOGS_DIR="$CURRENT_TEST_DIR/logs"
+	test -d "$CURRENT_TEST_DIR" || mkdir -p "$CURRENT_TEST_DIR"
+	test -d "$LOGS_DIR" || mkdir -p "$LOGS_DIR"
+else
+	# when $PERFSUITE_RUN_DIR is not set, logs will be placed here
+	export CURRENT_TEST_DIR="."
+	export LOGS_DIR="."
+fi
+
 
 #### test parametrization
 if [ ! -d ./common ]; then
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 03/11] perf testsuite: Fix shellcheck warnings
  2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
  2024-06-24 13:07 ` [PATCH 01/11] perf tests shell: Skip base_* dirs in test script search Michael Petlan
  2024-06-24 13:08 ` [PATCH 02/11] perf testsuite: Merge settings files for shell tests Michael Petlan
@ 2024-06-24 13:08 ` Michael Petlan
  2024-06-24 13:08 ` [PATCH 04/11] perf testsuite probe: Add test for blacklisted kprobes handling Michael Petlan
                   ` (8 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: Michael Petlan @ 2024-06-24 13:08 UTC (permalink / raw)
  To: linux-perf-users, vmolnaro, acme, namhyung
  Cc: acme, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Shellcheck is becoming a standard when building perf to prevent
any unnecessary mistakes. Fix shellcheck warnings in perf testsuite.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 tools/perf/tests/shell/common/init.sh     | 8 ++++----
 tools/perf/tests/shell/common/settings.sh | 9 +++++----
 2 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/tools/perf/tests/shell/common/init.sh b/tools/perf/tests/shell/common/init.sh
index b99fba50ba94..075f17623c8e 100644
--- a/tools/perf/tests/shell/common/init.sh
+++ b/tools/perf/tests/shell/common/init.sh
@@ -26,8 +26,8 @@ print_results()
 	PERF_RETVAL="$1"; shift
 	CHECK_RETVAL="$1"; shift
 	FAILURE_REASON=""
-	TASK_COMMENT="$@"
-	if [ $PERF_RETVAL -eq 0 -a $CHECK_RETVAL -eq 0 ]; then
+	TASK_COMMENT="$*"
+	if [ $PERF_RETVAL -eq 0 ] && [ $CHECK_RETVAL -eq 0 ]; then
 		_echo "$MPASS-- [ PASS ] --$MEND $TEST_NAME :: $THIS_TEST_NAME :: $TASK_COMMENT"
 		return 0
 	else
@@ -56,7 +56,7 @@ print_overall_results()
 
 print_testcase_skipped()
 {
-	TASK_COMMENT="$@"
+	TASK_COMMENT="$*"
 	_echo "$MSKIP-- [ SKIP ] --$MEND $TEST_NAME :: $THIS_TEST_NAME :: $TASK_COMMENT :: testcase skipped"
 	return 0
 }
@@ -69,7 +69,7 @@ print_overall_skipped()
 
 print_warning()
 {
-	WARN_COMMENT="$@"
+	WARN_COMMENT="$*"
 	_echo "$MWARN-- [ WARN ] --$MEND $TEST_NAME :: $THIS_TEST_NAME :: $WARN_COMMENT"
 	return 0
 }
diff --git a/tools/perf/tests/shell/common/settings.sh b/tools/perf/tests/shell/common/settings.sh
index dcdb7d49ac00..2e50dd0bff9b 100644
--- a/tools/perf/tests/shell/common/settings.sh
+++ b/tools/perf/tests/shell/common/settings.sh
@@ -45,7 +45,7 @@ export TEST_IGNORE_MISSING_PMU=${TEST_IGNORE_MISSING_PMU:-n}
 export LC_ALL=C
 
 #### colors
-if [ -t 1 -o "$TESTLOG_FORCE_COLOR" = "yes" ]; then
+if [ -t 1 ] || [ "$TESTLOG_FORCE_COLOR" = "yes" ]; then
 	export MPASS="\e[32m"
 	export MALLPASS="\e[1;32m"
 	export MFAIL="\e[31m"
@@ -68,15 +68,15 @@ fi
 ### general info
 DIR_PATH=`dirname "$(readlink -e "$0")"`
 
-export TEST_NAME=`basename $DIR_PATH | sed 's/base/perf/'`
-export MY_ARCH=`arch`
+TEST_NAME=`basename $DIR_PATH | sed 's/base/perf/'`; export TEST_NAME
+MY_ARCH=`arch`; export MY_ARCH
 
 # storing logs and temporary files variables
 if [ -n "$PERFSUITE_RUN_DIR" ]; then
 	# when $PERFSUITE_RUN_DIR is set to something, all the logs and temp files will be placed there
 	# --> the $PERFSUITE_RUN_DIR/perf_something/examples and $PERFSUITE_RUN_DIR/perf_something/logs
 	#     dirs will be used for that
-	export PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`
+	PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`; export PERFSUITE_RUN_DIR
 	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
 	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
 	export LOGS_DIR="$CURRENT_TEST_DIR/logs"
@@ -93,6 +93,7 @@ fi
 if [ ! -d ./common ]; then
 	# set parameters based on runmode
 	if [ -f ../common/parametrization.$PERFTOOL_TESTSUITE_RUNMODE.sh ]; then
+		# shellcheck source=/dev/null
 		. ../common/parametrization.$PERFTOOL_TESTSUITE_RUNMODE.sh
 	fi
 	# if some parameters haven't been set until now, set them to default
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 04/11] perf testsuite probe: Add test for blacklisted kprobes handling
  2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
                   ` (2 preceding siblings ...)
  2024-06-24 13:08 ` [PATCH 03/11] perf testsuite: Fix shellcheck warnings Michael Petlan
@ 2024-06-24 13:08 ` Michael Petlan
  2024-06-24 13:08 ` [PATCH 05/11] perf testsuite probe: Add test for basic perf-probe options Michael Petlan
                   ` (7 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: Michael Petlan @ 2024-06-24 13:08 UTC (permalink / raw)
  To: linux-perf-users, vmolnaro, acme, namhyung
  Cc: acme, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Test perf probe interface. Blacklisted functions should be rejected
when there is an attempt to set a kprobe to them.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 .../base_probe/test_adding_blacklisted.sh     | 67 +++++++++++++++++++
 1 file changed, 67 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh

diff --git a/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
new file mode 100755
index 000000000000..b5dc10b2a738
--- /dev/null
+++ b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
@@ -0,0 +1,67 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_adding_blacklisted of perf_probe test
+#	Author: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
+#	Author: Michael Petlan <mpetlan@redhat.com>
+#
+#	Description:
+#
+#		Blacklisted functions should not be added successfully as probes,
+#	they must be skipped.
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+# skip if not supported
+BLACKFUNC=`head -n 1 /sys/kernel/debug/kprobes/blacklist 2> /dev/null | cut -f2`
+if [ -z "$BLACKFUNC" ]; then
+	print_overall_skipped
+	exit 0
+fi
+
+# remove all previously added probes
+clear_all_probes
+
+
+### adding blacklisted function
+
+# functions from blacklist should be skipped by perf probe
+! $CMD_PERF probe $BLACKFUNC > $LOGS_DIR/adding_blacklisted.log 2> $LOGS_DIR/adding_blacklisted.err
+PERF_EXIT_CODE=$?
+
+REGEX_SCOPE_FAIL="Failed to find scope of probe point"
+REGEX_SKIP_MESSAGE=" is blacklisted function, skip it\."
+REGEX_NOT_FOUND_MESSAGE="Probe point \'$BLACKFUNC\' not found."
+REGEX_ERROR_MESSAGE="Error: Failed to add events."
+REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
+REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
+REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
+../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"
+(( TEST_RESULT += $? ))
+
+
+### listing not-added probe
+
+# blacklisted probes should NOT appear in perf-list output
+$CMD_PERF list probe:\* > $LOGS_DIR/adding_blacklisted_list.log
+PERF_EXIT_CODE=$?
+
+../common/check_all_lines_matched.pl "$RE_LINE_EMPTY" "List of pre-defined events" "Metric Groups:" < $LOGS_DIR/adding_blacklisted_list.log
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "listing blacklisted probe (should NOT be listed)"
+(( TEST_RESULT += $? ))
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 05/11] perf testsuite probe: Add test for basic perf-probe options
  2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
                   ` (3 preceding siblings ...)
  2024-06-24 13:08 ` [PATCH 04/11] perf testsuite probe: Add test for blacklisted kprobes handling Michael Petlan
@ 2024-06-24 13:08 ` Michael Petlan
  2024-06-24 13:08 ` [PATCH 06/11] perf testsuite probe: Add test for invalid options Michael Petlan
                   ` (6 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: Michael Petlan @ 2024-06-24 13:08 UTC (permalink / raw)
  To: linux-perf-users, vmolnaro, acme, namhyung
  Cc: acme, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Test basic behavior of perf-probe subcommand. It is run as a part of
perftool-testsuite_probe test case.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 .../perf/tests/shell/base_probe/test_basic.sh | 80 +++++++++++++++++++
 1 file changed, 80 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh

diff --git a/tools/perf/tests/shell/base_probe/test_basic.sh b/tools/perf/tests/shell/base_probe/test_basic.sh
new file mode 100755
index 000000000000..09669ec479f2
--- /dev/null
+++ b/tools/perf/tests/shell/base_probe/test_basic.sh
@@ -0,0 +1,80 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_basic of perf_probe test
+#	Author: Michael Petlan <mpetlan@redhat.com>
+#	Author: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
+#
+#	Description:
+#
+#		This test tests basic functionality of perf probe command.
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+if ! check_kprobes_available; then
+	print_overall_skipped
+	exit 0
+fi
+
+
+### help message
+
+if [ "$PARAM_GENERAL_HELP_TEXT_CHECK" = "y" ]; then
+	# test that a help message is shown and looks reasonable
+	$CMD_PERF probe --help > $LOGS_DIR/basic_helpmsg.log 2> $LOGS_DIR/basic_helpmsg.err
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "PERF-PROBE" "NAME" "SYNOPSIS" "DESCRIPTION" "OPTIONS" "PROBE\s+SYNTAX" "PROBE\s+ARGUMENT" "LINE\s+SYNTAX" < $LOGS_DIR/basic_helpmsg.log
+	CHECK_EXIT_CODE=$?
+	../common/check_all_patterns_found.pl "LAZY\s+MATCHING" "FILTER\s+PATTERN" "EXAMPLES" "SEE\s+ALSO" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "vmlinux" "module=" "source=" "verbose" "quiet" "add=" "del=" "list.*EVENT" "line=" "vars=" "externs" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "no-inlines" "funcs.*FILTER" "filter=FILTER" "force" "dry-run" "max-probes" "exec=" "demangle-kernel" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_no_patterns_found.pl "No manual entry for" < $LOGS_DIR/basic_helpmsg.err
+	(( CHECK_EXIT_CODE += $? ))
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "help message"
+	(( TEST_RESULT += $? ))
+else
+	print_testcase_skipped "help message"
+fi
+
+
+### usage message
+
+# without any args perf-probe should print usage
+$CMD_PERF probe 2> $LOGS_DIR/basic_usage.log > /dev/null
+
+../common/check_all_patterns_found.pl "[Uu]sage" "perf probe" "verbose" "quiet" "add" "del" "force" "line" "vars" "externs" "range" < $LOGS_DIR/basic_usage.log
+CHECK_EXIT_CODE=$?
+
+print_results 0 $CHECK_EXIT_CODE "usage message"
+(( TEST_RESULT += $? ))
+
+
+### quiet switch
+
+# '--quiet' should mute all output
+$CMD_PERF probe --quiet --add vfs_read > $LOGS_DIR/basic_quiet01.log 2> $LOGS_DIR/basic_quiet01.err
+PERF_EXIT_CODE=$?
+$CMD_PERF probe --quiet --del vfs_read > $LOGS_DIR/basic_quiet03.log 2> $LOGS_DIR/basic_quiet02.err
+(( PERF_EXIT_CODE += $? ))
+
+test "`cat $LOGS_DIR/basic_quiet*log $LOGS_DIR/basic_quiet*err | wc -l`" -eq 0
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "quiet switch"
+(( TEST_RESULT += $? ))
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 06/11] perf testsuite probe: Add test for invalid options
  2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
                   ` (4 preceding siblings ...)
  2024-06-24 13:08 ` [PATCH 05/11] perf testsuite probe: Add test for basic perf-probe options Michael Petlan
@ 2024-06-24 13:08 ` Michael Petlan
  2024-06-24 13:08 ` [PATCH 07/11] perf testsuite probe: Add test for line semantics Michael Petlan
                   ` (5 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: Michael Petlan @ 2024-06-24 13:08 UTC (permalink / raw)
  To: linux-perf-users, vmolnaro, acme, namhyung
  Cc: acme, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Test if various incompatible options are correctly handled-rejected.
It is run as a part of perftool-testsuite_probe test case.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 .../shell/base_probe/test_invalid_options.sh  | 79 +++++++++++++++++++
 1 file changed, 79 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh

diff --git a/tools/perf/tests/shell/base_probe/test_invalid_options.sh b/tools/perf/tests/shell/base_probe/test_invalid_options.sh
new file mode 100755
index 000000000000..1fedfd8b0d0d
--- /dev/null
+++ b/tools/perf/tests/shell/base_probe/test_invalid_options.sh
@@ -0,0 +1,79 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_invalid_options of perf_probe test
+#	Author: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
+#	Author: Michael Petlan <mpetlan@redhat.com>
+#
+#	Description:
+#
+#		This test checks whether the invalid and incompatible options are reported
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+if ! check_kprobes_available; then
+	print_overall_skipped
+	exit 0
+fi
+
+
+### missing argument
+
+# some options require an argument
+for opt in '-a' '-d' '-L' '-V'; do
+	! $CMD_PERF probe $opt 2> $LOGS_DIR/invalid_options_missing_argument$opt.err
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "Error: switch .* requires a value" < $LOGS_DIR/invalid_options_missing_argument$opt.err
+	CHECK_EXIT_CODE=$?
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "missing argument for $opt"
+	(( TEST_RESULT += $? ))
+done
+
+
+### unnecessary argument
+
+# some options may omit the argument
+for opt in '-F' '-l'; do
+	$CMD_PERF probe -F > /dev/null 2> $LOGS_DIR/invalid_options_unnecessary_argument$opt.err
+	PERF_EXIT_CODE=$?
+
+	test ! -s $LOGS_DIR/invalid_options_unnecessary_argument$opt.err
+	CHECK_EXIT_CODE=$?
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "unnecessary argument for $opt"
+	(( TEST_RESULT += $? ))
+done
+
+
+### mutually exclusive options
+
+# some options are mutually exclusive
+test -e $LOGS_DIR/invalid_options_mutually_exclusive.log && rm -f $LOGS_DIR/invalid_options_mutually_exclusive.log
+for opt in '-a xxx -d xxx' '-a xxx -L foo' '-a xxx -V foo' '-a xxx -l' '-a xxx -F' \
+		'-d xxx -L foo' '-d xxx -V foo' '-d xxx -l' '-d xxx -F' \
+		'-L foo -V bar' '-L foo -l' '-L foo -F' '-V foo -l' '-V foo -F' '-l -F'; do
+	! $CMD_PERF probe $opt > /dev/null 2> $LOGS_DIR/aux.log
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "Error: switch .+ cannot be used with switch .+" < $LOGS_DIR/aux.log
+	CHECK_EXIT_CODE=$?
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "mutually exclusive options :: $opt"
+	(( TEST_RESULT += $? ))
+
+	# gather the logs
+	cat $LOGS_DIR/aux.log | grep "Error" >> $LOGS_DIR/invalid_options_mutually_exclusive.log
+done
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 07/11] perf testsuite probe: Add test for line semantics
  2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
                   ` (5 preceding siblings ...)
  2024-06-24 13:08 ` [PATCH 06/11] perf testsuite probe: Add test for invalid options Michael Petlan
@ 2024-06-24 13:08 ` Michael Petlan
  2024-06-24 13:08 ` [PATCH 08/11] perf testsuite: Add common output checking helper Michael Petlan
                   ` (4 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: Michael Petlan @ 2024-06-24 13:08 UTC (permalink / raw)
  To: linux-perf-users, vmolnaro, acme, namhyung
  Cc: acme, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

The perf-probe command uses a specific semantics to describe probes.
Test some patterns that are known to be both valid and invalid if
they are handled appropriately.

This test is run as a part of perftool-testsuite_probe test case.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 .../shell/base_probe/test_line_semantics.sh   | 55 +++++++++++++++++++
 1 file changed, 55 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh

diff --git a/tools/perf/tests/shell/base_probe/test_line_semantics.sh b/tools/perf/tests/shell/base_probe/test_line_semantics.sh
new file mode 100755
index 000000000000..d8f4bde0f585
--- /dev/null
+++ b/tools/perf/tests/shell/base_probe/test_line_semantics.sh
@@ -0,0 +1,55 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_line_semantics of perf_probe test
+#	Author: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
+#	Author: Michael Petlan <mpetlan@redhat.com>
+#
+#	Description:
+#
+#		This test checks whether the semantic errors of line option's
+#		arguments are properly reported.
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+if ! check_kprobes_available; then
+	print_overall_skipped
+	exit 0
+fi
+
+
+### acceptable --line descriptions
+
+# testing acceptance of valid patterns for the '--line' option
+VALID_PATTERNS="func func:10 func:0-10 func:2+10 func@source.c func@source.c:1 source.c:1 source.c:1+1 source.c:1-10"
+for desc in $VALID_PATTERNS; do
+	! ( $CMD_PERF probe --line $desc 2>&1 | grep -q "Semantic error" )
+	CHECK_EXIT_CODE=$?
+
+	print_results 0 $CHECK_EXIT_CODE "acceptable descriptions :: $desc"
+	(( TEST_RESULT += $? ))
+done
+
+
+### unacceptable --line descriptions
+
+# testing handling of invalid patterns for the '--line' option
+INVALID_PATTERNS="func:foo func:1-foo func:1+foo func;lazy\*pattern"
+for desc in $INVALID_PATTERNS; do
+	$CMD_PERF probe --line $desc 2>&1 | grep -q "Semantic error"
+	CHECK_EXIT_CODE=$?
+
+	print_results 0 $CHECK_EXIT_CODE "unacceptable descriptions :: $desc"
+	(( TEST_RESULT += $? ))
+done
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 08/11] perf testsuite: Add common output checking helper
  2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
                   ` (6 preceding siblings ...)
  2024-06-24 13:08 ` [PATCH 07/11] perf testsuite probe: Add test for line semantics Michael Petlan
@ 2024-06-24 13:08 ` Michael Petlan
  2024-06-24 13:08 ` [PATCH 09/11] perf testsuite report: Add test for perf-report basic functionality Michael Petlan
                   ` (3 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: Michael Petlan @ 2024-06-24 13:08 UTC (permalink / raw)
  To: linux-perf-users, vmolnaro, acme, namhyung
  Cc: acme, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

As a form of validation, it is a common practice to check the outputs
of commands whether they contain expected patterns or match a certain
regular expression.

This output checking helper is designed to allow checking stderr output
of perf commands for unexpected messages, while ignoring messages that
are known to be harmless, e.g.:
  "Lowering default frequency rate to \d+\."
  "\d+ out of order events recorded."
  etc.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 .../shell/common/check_errors_whitelisted.pl  | 51 +++++++++++++++++++
 1 file changed, 51 insertions(+)
 create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl

diff --git a/tools/perf/tests/shell/common/check_errors_whitelisted.pl b/tools/perf/tests/shell/common/check_errors_whitelisted.pl
new file mode 100755
index 000000000000..c57d355dd76e
--- /dev/null
+++ b/tools/perf/tests/shell/common/check_errors_whitelisted.pl
@@ -0,0 +1,51 @@
+#!/usr/bin/perl
+# SPDX-License-Identifier: GPL-2.0
+
+$whitelist_file = shift;
+
+if (defined $whitelist_file)
+{
+	open (INFILE, $whitelist_file) or die "Checker error: Unable to open the whitelist file: $whitelist_file\n";
+	@regexps = <INFILE>;
+	close INFILE or die "Checker error: Unable to close the whitelist file: $whitelist_file\n";
+}
+else
+{
+	@regexps = ();
+}
+
+$max_printed_lines = 20;
+$max_printed_lines = $ENV{TESTLOG_ERR_MSG_MAX_LINES} if (defined $ENV{TESTLOG_ERR_MSG_MAX_LINES});
+
+$quiet = 1;
+$quiet = 0 if (defined $ENV{TESTLOG_VERBOSITY} && $ENV{TESTLOG_VERBOSITY} ge 2);
+
+$passed = 1;
+$lines_printed = 0;
+
+while (<STDIN>)
+{
+	s/\n//;
+
+	$line_matched = 0;
+	for $r (@regexps)
+	{
+		chomp $r;
+		if (/$r/)
+		{
+			$line_matched = 1;
+			last;
+		}
+	}
+
+	unless ($line_matched)
+	{
+		if ($lines_printed++ < $max_printed_lines)
+		{
+			print "Line did not match any pattern: \"$_\"\n" unless $quiet;
+		}
+		$passed = 0;
+	}
+}
+
+exit ($passed == 0);
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 09/11] perf testsuite report: Add test for perf-report basic functionality
  2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
                   ` (7 preceding siblings ...)
  2024-06-24 13:08 ` [PATCH 08/11] perf testsuite: Add common output checking helper Michael Petlan
@ 2024-06-24 13:08 ` Michael Petlan
  2024-06-24 13:08 ` [PATCH 10/11] perf testsuite report: Add test case for perf report Michael Petlan
                   ` (2 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: Michael Petlan @ 2024-06-24 13:08 UTC (permalink / raw)
  To: linux-perf-users, vmolnaro, acme, namhyung
  Cc: acme, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Test basic execution and some options of perf-report subcommand, like
show-nr-samples, header, showcpuutilization, pid and symbol filtering.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 tools/perf/tests/shell/base_report/setup.sh   |  32 +++
 .../shell/base_report/stderr-whitelist.txt    |   5 +
 .../tests/shell/base_report/test_basic.sh     | 190 ++++++++++++++++++
 tools/perf/tests/shell/common/settings.sh     |   2 +
 4 files changed, 229 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_report/setup.sh
 create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
 create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh

diff --git a/tools/perf/tests/shell/base_report/setup.sh b/tools/perf/tests/shell/base_report/setup.sh
new file mode 100755
index 000000000000..4caa496660c6
--- /dev/null
+++ b/tools/perf/tests/shell/base_report/setup.sh
@@ -0,0 +1,32 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	setup.sh of perf report test
+#	Author: Michael Petlan <mpetlan@redhat.com>
+#
+#	Description:
+#
+#		We need some sample data for perf-report testing
+#
+#
+
+# include working environment
+. ../common/init.sh
+
+test -d "$HEADER_TAR_DIR" || mkdir -p "$HEADER_TAR_DIR"
+
+SW_EVENT="cpu-clock"
+
+$CMD_PERF record -asdg -e $SW_EVENT -o $CURRENT_TEST_DIR/perf.data -- $CMD_LONGER_SLEEP 2> $LOGS_DIR/setup.log
+PERF_EXIT_CODE=$?
+
+../common/check_all_patterns_found.pl "$RE_LINE_RECORD1" "$RE_LINE_RECORD2" < $LOGS_DIR/setup.log
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "prepare the perf.data file"
+TEST_RESULT=$?
+
+print_overall_results $TEST_RESULT
+exit $?
diff --git a/tools/perf/tests/shell/base_report/stderr-whitelist.txt b/tools/perf/tests/shell/base_report/stderr-whitelist.txt
new file mode 100644
index 000000000000..e3341401b47c
--- /dev/null
+++ b/tools/perf/tests/shell/base_report/stderr-whitelist.txt
@@ -0,0 +1,5 @@
+no symbols found in .*, maybe install a debug package
+was updated .*is prelink enabled.+ Restart the long running apps that use it
+Warning:
+\d+ out of order events recorded.
+detected invalid bpf_prog_info
diff --git a/tools/perf/tests/shell/base_report/test_basic.sh b/tools/perf/tests/shell/base_report/test_basic.sh
new file mode 100755
index 000000000000..47677cbd4df3
--- /dev/null
+++ b/tools/perf/tests/shell/base_report/test_basic.sh
@@ -0,0 +1,190 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_basic of perf_report test
+#	Author: Michael Petlan <mpetlan@redhat.com>
+#
+#	Description:
+#
+#		This test tests basic functionality of perf report command.
+#
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+
+### help message
+
+if [ "$PARAM_GENERAL_HELP_TEXT_CHECK" = "y" ]; then
+	# test that a help message is shown and looks reasonable
+	$CMD_PERF report --help > $LOGS_DIR/basic_helpmsg.log 2> $LOGS_DIR/basic_helpmsg.err
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "PERF-REPORT" "NAME" "SYNOPSIS" "DESCRIPTION" "OPTIONS" "OVERHEAD\s+CALCULATION" "SEE ALSO" < $LOGS_DIR/basic_helpmsg.log
+	CHECK_EXIT_CODE=$?
+	../common/check_all_patterns_found.pl "input" "verbose" "show-nr-samples" "show-cpu-utilization" "threads" "comms" "pid" "tid" "dsos" "symbols" "symbol-filter" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "hide-unresolved" "sort" "fields" "parent" "exclude-other" "column-widths" "field-separator" "dump-raw-trace" "children" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "call-graph" "max-stack" "inverted" "ignore-callees" "pretty" "stdio" "tui" "gtk" "vmlinux" "kallsyms" "modules" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "force" "symfs" "cpu" "disassembler-style" "source" "asm-raw" "show-total-period" "show-info" "branch-stack" "group" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "branch-history" "objdump" "demangle" "percent-limit" "percentage" "header" "itrace" "full-source-path" "show-ref-call-graph" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_no_patterns_found.pl "No manual entry for" < $LOGS_DIR/basic_helpmsg.err
+	(( CHECK_EXIT_CODE += $? ))
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "help message"
+	(( TEST_RESULT += $? ))
+else
+	print_testcase_skipped "help message"
+fi
+
+
+### basic execution
+
+# test that perf report is even working
+$CMD_PERF report -i $CURRENT_TEST_DIR/perf.data --stdio > $LOGS_DIR/basic_basic.log 2> $LOGS_DIR/basic_basic.err
+PERF_EXIT_CODE=$?
+
+REGEX_LOST_SAMPLES_INFO="#\s*Total Lost Samples:\s+$RE_NUMBER"
+REGEX_SAMPLES_INFO="#\s*Samples:\s+(?:$RE_NUMBER)\w?\s+of\s+event\s+'$RE_EVENT_ANY'"
+REGEX_LINES_HEADER="#\s*Children\s+Self\s+Command\s+Shared Object\s+Symbol"
+REGEX_LINES="\s*$RE_NUMBER%\s+$RE_NUMBER%\s+\S+\s+\[kernel\.(?:vmlinux)|(?:kallsyms)\]\s+\[[k\.]\]\s+\w+"
+../common/check_all_patterns_found.pl "$REGEX_LOST_SAMPLES_INFO" "$REGEX_SAMPLES_INFO" "$REGEX_LINES_HEADER" "$REGEX_LINES" < $LOGS_DIR/basic_basic.log
+CHECK_EXIT_CODE=$?
+../common/check_errors_whitelisted.pl "stderr-whitelist.txt" < $LOGS_DIR/basic_basic.err
+(( CHECK_EXIT_CODE += $? ))
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "basic execution"
+(( TEST_RESULT += $? ))
+
+
+### number of samples
+
+# '--show-nr-samples' should show number of samples for each symbol
+$CMD_PERF report --stdio -i $CURRENT_TEST_DIR/perf.data --show-nr-samples > $LOGS_DIR/basic_nrsamples.log 2> $LOGS_DIR/basic_nrsamples.err
+PERF_EXIT_CODE=$?
+
+REGEX_LINES_HEADER="#\s*Children\s+Self\s+Samples\s+Command\s+Shared Object\s+Symbol"
+REGEX_LINES="\s*$RE_NUMBER%\s+$RE_NUMBER%\s+$RE_NUMBER\s+\S+\s+\[kernel\.(?:vmlinux)|(?:kallsyms)\]\s+\[[k\.]\]\s+\w+"
+../common/check_all_patterns_found.pl "$REGEX_LINES_HEADER" "$REGEX_LINES" < $LOGS_DIR/basic_nrsamples.log
+CHECK_EXIT_CODE=$?
+../common/check_errors_whitelisted.pl "stderr-whitelist.txt" < $LOGS_DIR/basic_nrsamples.err
+(( CHECK_EXIT_CODE += $? ))
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "number of samples"
+(( TEST_RESULT += $? ))
+
+
+### header
+
+# '--header' and '--header-only' should show perf report header
+$CMD_PERF report -i $CURRENT_TEST_DIR/perf.data --stdio --header-only > $LOGS_DIR/basic_header.log
+PERF_EXIT_CODE=$?
+
+REGEX_LINE_TIMESTAMP="#\s+captured on\s*:\s*$RE_DATE_TIME"
+REGEX_LINE_HOSTNAME="#\s+hostname\s*:\s*$MY_HOSTNAME"
+REGEX_LINE_KERNEL="#\s+os release\s*:\s*${MY_KERNEL_VERSION//+/\\+}"
+REGEX_LINE_PERF="#\s+perf version\s*:\s*"
+REGEX_LINE_ARCH="#\s+arch\s*:\s*$MY_ARCH"
+REGEX_LINE_CPUS_ONLINE="#\s+nrcpus online\s*:\s*$MY_CPUS_ONLINE"
+REGEX_LINE_CPUS_AVAIL="#\s+nrcpus avail\s*:\s*$MY_CPUS_AVAILABLE"
+# disable precise check for "nrcpus avail" in BASIC runmode
+test $PERFTOOL_TESTSUITE_RUNMODE -lt $RUNMODE_STANDARD && REGEX_LINE_CPUS_AVAIL="#\s+nrcpus avail\s*:\s*$RE_NUMBER"
+../common/check_all_patterns_found.pl "$REGEX_LINE_TIMESTAMP" "$REGEX_LINE_HOSTNAME" "$REGEX_LINE_KERNEL" "$REGEX_LINE_PERF" "$REGEX_LINE_ARCH" "$REGEX_LINE_CPUS_ONLINE" "$REGEX_LINE_CPUS_AVAIL" < $LOGS_DIR/basic_header.log
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "header"
+(( TEST_RESULT += $? ))
+
+# '--header' and '--header-only' should use creation time
+OLD_TIMESTAMP=`$CMD_PERF report --stdio --header-only -i $CURRENT_TEST_DIR/perf.data | grep "captured on"`
+PERF_EXIT_CODE=$?
+
+( tar -C $CURRENT_TEST_DIR -c perf.data | xz > $CURRENT_TEST_DIR/perf.data.tar.xz ; xzcat $CURRENT_TEST_DIR/perf.data.tar.xz | tar x -C $HEADER_TAR_DIR )
+(( PERF_EXIT_CODE += $? ))
+
+NEW_TIMESTAMP=`$CMD_PERF report --stdio --header-only -i $HEADER_TAR_DIR/perf.data | grep "captured on"`
+(( PERF_EXIT_CODE += $? ))
+
+test "$OLD_TIMESTAMP" = "$NEW_TIMESTAMP"
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "header timestamp"
+(( TEST_RESULT += $? ))
+
+
+### show CPU utilization
+
+# '--showcpuutilization' should show percentage for both system and userspace mode
+$CMD_PERF report -i $CURRENT_TEST_DIR/perf.data --stdio --showcpuutilization > $LOGS_DIR/basic_cpuut.log 2> $LOGS_DIR/basic_cpuut.err
+PERF_EXIT_CODE=$?
+
+REGEX_LINES_HEADER="#\s*Children\s+Self\s+sys\s+usr\s+Command\s+Shared Object\s+Symbol"
+REGEX_LINES="\s*$RE_NUMBER%\s+$RE_NUMBER%\s+$RE_NUMBER%\s+$RE_NUMBER%\s+\S+\s+\[kernel\.(?:vmlinux)|(?:kallsyms)\]\s+\[[k\.]\]\s+\w+"
+../common/check_all_patterns_found.pl "$REGEX_LINES_HEADER" "$REGEX_LINES" < $LOGS_DIR/basic_cpuut.log
+CHECK_EXIT_CODE=$?
+../common/check_errors_whitelisted.pl "stderr-whitelist.txt" < $LOGS_DIR/basic_cpuut.err
+(( CHECK_EXIT_CODE += $? ))
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "show CPU utilization"
+(( TEST_RESULT += $? ))
+
+
+### pid
+
+# '--pid=' should limit the output for a process with the given pid only
+$CMD_PERF report --stdio -i $CURRENT_TEST_DIR/perf.data --pid=1 > $LOGS_DIR/basic_pid.log 2> $LOGS_DIR/basic_pid.err
+PERF_EXIT_CODE=$?
+
+grep -P -v '^#' $LOGS_DIR/basic_pid.log | grep -P '\s+[\d\.]+%' | ../common/check_all_lines_matched.pl "systemd|init"
+CHECK_EXIT_CODE=$?
+../common/check_errors_whitelisted.pl "stderr-whitelist.txt" < $LOGS_DIR/basic_pid.err
+(( CHECK_EXIT_CODE += $? ))
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "pid"
+(( TEST_RESULT += $? ))
+
+
+### non-existing symbol
+
+# '--symbols' should show only the given symbols
+$CMD_PERF report --stdio -i $CURRENT_TEST_DIR/perf.data --symbols=dummynonexistingsymbol > $LOGS_DIR/basic_symbols.log 2> $LOGS_DIR/basic_symbols.err
+PERF_EXIT_CODE=$?
+
+../common/check_all_lines_matched.pl "$RE_LINE_EMPTY" "$RE_LINE_COMMENT" < $LOGS_DIR/basic_symbols.log
+CHECK_EXIT_CODE=$?
+../common/check_errors_whitelisted.pl "stderr-whitelist.txt" < $LOGS_DIR/basic_symbols.err
+(( CHECK_EXIT_CODE += $? ))
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "non-existing symbol"
+(( TEST_RESULT += $? ))
+
+
+### symbol filter
+
+# '--symbol-filter' should filter symbols based on substrings
+$CMD_PERF report --stdio -i $CURRENT_TEST_DIR/perf.data --symbol-filter=map > $LOGS_DIR/basic_symbolfilter.log 2> $LOGS_DIR/basic_symbolfilter.err
+PERF_EXIT_CODE=$?
+
+grep -P -v '^#' $LOGS_DIR/basic_symbolfilter.log | grep -P '\s+[\d\.]+%' | ../common/check_all_lines_matched.pl "\[[k\.]\]\s+.*map"
+CHECK_EXIT_CODE=$?
+../common/check_errors_whitelisted.pl "stderr-whitelist.txt" < $LOGS_DIR/basic_symbolfilter.err
+(( CHECK_EXIT_CODE += $? ))
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "symbol filter"
+(( TEST_RESULT += $? ))
+
+
+# TODO: $CMD_PERF report -n --showcpuutilization -TUxDg 2> 01.log
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
diff --git a/tools/perf/tests/shell/common/settings.sh b/tools/perf/tests/shell/common/settings.sh
index 2e50dd0bff9b..cba1b338f96f 100644
--- a/tools/perf/tests/shell/common/settings.sh
+++ b/tools/perf/tests/shell/common/settings.sh
@@ -80,12 +80,14 @@ if [ -n "$PERFSUITE_RUN_DIR" ]; then
 	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
 	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
 	export LOGS_DIR="$CURRENT_TEST_DIR/logs"
+	export HEADER_TAR_DIR="$CURRENT_TEST_DIR/header_tar"
 	test -d "$CURRENT_TEST_DIR" || mkdir -p "$CURRENT_TEST_DIR"
 	test -d "$LOGS_DIR" || mkdir -p "$LOGS_DIR"
 else
 	# when $PERFSUITE_RUN_DIR is not set, logs will be placed here
 	export CURRENT_TEST_DIR="."
 	export LOGS_DIR="."
+	export HEADER_TAR_DIR="./header_tar"
 fi
 
 
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 10/11] perf testsuite report: Add test case for perf report
  2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
                   ` (8 preceding siblings ...)
  2024-06-24 13:08 ` [PATCH 09/11] perf testsuite report: Add test for perf-report basic functionality Michael Petlan
@ 2024-06-24 13:08 ` Michael Petlan
  2024-06-24 13:08 ` [PATCH 11/11] perf testsuite: Install perf-report tests Michael Petlan
  2024-06-27 22:29 ` [PATCH 00/11] perftool-testsuite 2nd batch Namhyung Kim
  11 siblings, 0 replies; 43+ messages in thread
From: Michael Petlan @ 2024-06-24 13:08 UTC (permalink / raw)
  To: linux-perf-users, vmolnaro, acme, namhyung
  Cc: acme, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Add new perf report test case that acts as an entry element in perf test
list. Runs multiple subtests from directory "base_report", which can be
expanded without further editing.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 .../tests/shell/perftool-testsuite_report.sh  | 23 +++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh

diff --git a/tools/perf/tests/shell/perftool-testsuite_report.sh b/tools/perf/tests/shell/perftool-testsuite_report.sh
new file mode 100755
index 000000000000..973012ce92a7
--- /dev/null
+++ b/tools/perf/tests/shell/perftool-testsuite_report.sh
@@ -0,0 +1,23 @@
+#!/bin/bash
+# perftool-testsuite_report
+# SPDX-License-Identifier: GPL-2.0
+
+test -d "$(dirname "$0")/base_report" || exit 2
+cd "$(dirname "$0")/base_report" || exit 2
+status=0
+
+PERFSUITE_RUN_DIR=$(mktemp -d /tmp/"$(basename "$0" .sh)".XXX)
+export PERFSUITE_RUN_DIR
+
+for testcase in setup.sh test_*; do                  # skip setup.sh if not present or not executable
+     test -x "$testcase" || continue
+     ./"$testcase"
+     (( status += $? ))
+done
+
+if ! [ "$PERFTEST_KEEP_LOGS" = "y" ]; then
+	rm -rf "$PERFSUITE_RUN_DIR"
+fi
+
+test $status -ne 0 && exit 1
+exit 0
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH 11/11] perf testsuite: Install perf-report tests
  2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
                   ` (9 preceding siblings ...)
  2024-06-24 13:08 ` [PATCH 10/11] perf testsuite report: Add test case for perf report Michael Petlan
@ 2024-06-24 13:08 ` Michael Petlan
  2024-06-27 22:29 ` [PATCH 00/11] perftool-testsuite 2nd batch Namhyung Kim
  11 siblings, 0 replies; 43+ messages in thread
From: Michael Petlan @ 2024-06-24 13:08 UTC (permalink / raw)
  To: linux-perf-users, vmolnaro, acme, namhyung
  Cc: acme, irogers, atrajeev, masami.hiramatsu.pt

Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 tools/perf/Makefile.perf | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 5c35c0d89306..a7eb1fa97b1c 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -1080,6 +1080,8 @@ install-tests: all install-gtk
 		$(INSTALL) tests/shell/common/*.pl '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/common'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_probe'; \
 		$(INSTALL) tests/shell/base_probe/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_probe'; \
+		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_report'; \
+		$(INSTALL) tests/shell/base_probe/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_report'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight' ; \
 		$(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
 	$(Q)$(MAKE) -C tests/shell/coresight install-tests
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* Re: [PATCH 00/11] perftool-testsuite 2nd batch
  2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
                   ` (10 preceding siblings ...)
  2024-06-24 13:08 ` [PATCH 11/11] perf testsuite: Install perf-report tests Michael Petlan
@ 2024-06-27 22:29 ` Namhyung Kim
  2024-07-02 11:08   ` [PATCH v2 " vmolnaro
  11 siblings, 1 reply; 43+ messages in thread
From: Namhyung Kim @ 2024-06-27 22:29 UTC (permalink / raw)
  To: Michael Petlan
  Cc: linux-perf-users, vmolnaro, acme, acme, irogers, atrajeev,
	masami.hiramatsu.pt

Hello,

On Mon, Jun 24, 2024 at 03:07:58PM +0200, Michael Petlan wrote:
> Hello,
> 
> posting a second batch of tests from perftool-testsuite [1], as a part
> of the testsuite upstreaming effort.
> 
> This batch brings several perf-probe tests ideated by Masami Hiramatsu 
> and a perf-report test.
> 
> We have also refactored settings files and fixed shellcheck warnings.
> 
> Finally, I have noticed that the scripts are run twice, once run by the
> driver (e.g. perftool-testsuite_*.sh) and once by perf-test's testsuite
> construction feature (that adds all executable shell scripts in all
> sub directories automatically). This is undesirable, so I have fixed
> it for now by skipping "base_*" dirs in the search. I see that this
> approach is not ideal, because the aim of the search seems to be to get
> rid of various test drivers, unify the way the tests are detected and
> run, while allowing them to be in subdirectories, like the Carsten
> Haitzler's coresight tests. However, I still think this is better
> solution that just having a pile of scripts without hierarchy that
> are grabbed and ordered into one-level queue of tests. Apart from
> allowing the tests to share some setup procedure, etc., is helps to
> keep the structure more clear.
> 
> A better design for future might be to have shell tests ordered in
> directories, such as
> 
>   # cd tests/shell/
>   # find . -name \*.sh
>   coresight/asm_pure_loop.sh
>   coresight/memcpy_thread_16k_10.sh
>   coresight/thread_loop_check_tid_10.sh
>   coresight/thread_loop_check_tid_2.sh
>   coresight/unroll_loop_thread_10.sh
>   base_probe/test_adding_kernel.sh
>   base_probe/test_adding_blacklisted.sh
>   base_probe/test_basic.sh
>   base_probe/test_invalid_options.sh
>   base_probe/test_line_semantics.sh
>   base_report/setup.sh
>   base_report/test_basic.sh
>   [...]  
> 
> The directories might also contain some "header" file that would
> bear the name of the parent testcase for all the tests inside it
> (e.g. "perf probe shell tests").
> 
> Then, no separate drivers would be needed, perf-test testsuite-
> creation mechanism would search for the shell scripts as it does
> now, but it would create multilevel structure with subtests,
> such as:
> 
>   97: perf probe shell tests
>   97:1: perf probe basic functionality
>   97:2: perf probe tests with arguments
>   97:3: perf probe invalid options handling
>   98: CoreSight tests
>   98:1: CoreSight / ASM Pure loop
>   98:2: CoreSight / Thread Loop ...
>   [...]
> 
> ... similar to how C-tests can be organized. In case there is
> a setup.sh file in such a subdirectory, it would be run too.
> With properly designed names, also tag-like (substring matching)
> approach could be used to select desired tests (i.e. to override
> the tree structure and pick tests related to some topic, no matter
> if they are in base_stat or base_record subdirectory).
> 
> This plan also counts with processing the current test scripts and
> possibly putting them to best matching place in the structure.
> 
> There is already a bunch of shell tests and I thinkg it might be
> good to organize them into a structure.
> 
> Thoughts and ideas are welcome.
> 
> Thanks and regards,
> 
> Michael

Thanks for the work, but it doesn't apply to the current
perf-tools-next.  Can you please rebase?

Thanks,
Namhyung

> 
> 
> [1] https://github.com/rfmvh/perftool-testsuite
> 
> 
> Michael Petlan (2):
>   perf tests shell: Skip base_* dirs in test script search
>   perf testsuite: Install perf-report tests
> 
> Veronika Molnarova (9):
>   perf testsuite: Merge settings files for shell tests
>   perf testsuite: Fix shellcheck warnings
>   perf testsuite probe: Add test for blacklisted kprobes handling
>   perf testsuite probe: Add test for basic perf-probe options
>   perf testsuite probe: Add test for invalid options
>   perf testsuite probe: Add test for line semantics
>   perf testsuite: Add common output checking helper
>   perf testsuite report: Add test for perf-report basic functionality
>   perf testsuite report: Add test case for perf report
> 
>  tools/perf/Makefile.perf                      |   2 +
>  tools/perf/tests/shell/base_probe/settings.sh |  48 -----
>  .../base_probe/test_adding_blacklisted.sh     |  65 ++++++
>  .../shell/base_probe/test_adding_kernel.sh    |   3 -
>  .../perf/tests/shell/base_probe/test_basic.sh |  78 ++++++++
>  .../shell/base_probe/test_invalid_options.sh  |  77 +++++++
>  .../shell/base_probe/test_line_semantics.sh   |  53 +++++
>  tools/perf/tests/shell/base_report/setup.sh   |  30 +++
>  .../shell/base_report/stderr-whitelist.txt    |   5 +
>  .../tests/shell/base_report/test_basic.sh     | 188 ++++++++++++++++++
>  .../shell/common/check_errors_whitelisted.pl  |  50 +++++
>  tools/perf/tests/shell/common/init.sh         |  31 ++-
>  tools/perf/tests/shell/common/settings.sh     |  28 ++-
>  .../tests/shell/perftool-testsuite_report.sh  |  23 +++
>  tools/perf/tests/tests-scripts.c              |   2 +
>  15 files changed, 627 insertions(+), 56 deletions(-)
>  delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh
>  create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
>  create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh
>  create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh
>  create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh
>  create mode 100755 tools/perf/tests/shell/base_report/setup.sh
>  create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
>  create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh
>  create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl
>  create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh
> 
> -- 
> 2.43.0
> 

^ permalink raw reply	[flat|nested] 43+ messages in thread

* [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-06-27 22:29 ` [PATCH 00/11] perftool-testsuite 2nd batch Namhyung Kim
@ 2024-07-02 11:08   ` vmolnaro
  2024-07-02 11:08     ` [PATCH v2 01/11] perf tests shell: Skip base_* dirs in test script search vmolnaro
                       ` (11 more replies)
  0 siblings, 12 replies; 43+ messages in thread
From: vmolnaro @ 2024-07-02 11:08 UTC (permalink / raw)
  To: linux-perf-users, acme, namhyung
  Cc: acme, mpetlan, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Hello,

rebased the patches for the current perf-tools-next branch in v2 as it
was not applying.

Thanks for the notice,
Veronika

Michael Petlan (2):
  perf tests shell: Skip base_* dirs in test script search
  perf testsuite: Install perf-report tests

Veronika Molnarova (9):
  perf testsuite: Merge settings files for shell tests
  perf testsuite: Fix shellcheck warnings
  perf testsuite probe: Add test for blacklisted kprobes handling
  perf testsuite probe: Add test for basic perf-probe options
  perf testsuite probe: Add test for invalid options
  perf testsuite probe: Add test for line semantics
  perf testsuite: Add common output checking helper
  perf testsuite report: Add test for perf-report basic functionality
  perf testsuite report: Add test case for perf report

 tools/perf/Makefile.perf                      |   2 +
 tools/perf/tests/shell/base_probe/settings.sh |  48 -----
 .../base_probe/test_adding_blacklisted.sh     |  67 ++++++
 .../shell/base_probe/test_adding_kernel.sh    |   3 -
 .../perf/tests/shell/base_probe/test_basic.sh |  80 ++++++++
 .../shell/base_probe/test_invalid_options.sh  |  79 ++++++++
 .../shell/base_probe/test_line_semantics.sh   |  55 +++++
 tools/perf/tests/shell/base_report/setup.sh   |  32 +++
 .../shell/base_report/stderr-whitelist.txt    |   5 +
 .../tests/shell/base_report/test_basic.sh     | 190 ++++++++++++++++++
 .../shell/common/check_errors_whitelisted.pl  |  51 +++++
 tools/perf/tests/shell/common/init.sh         |  31 ++-
 tools/perf/tests/shell/common/settings.sh     |  28 ++-
 .../tests/shell/perftool-testsuite_report.sh  |  23 +++
 tools/perf/tests/tests-scripts.c              |   2 +
 15 files changed, 640 insertions(+), 56 deletions(-)
 delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh
 create mode 100755 tools/perf/tests/shell/base_report/setup.sh
 create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
 create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh
 create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl
 create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh

-- 
2.43.0


^ permalink raw reply	[flat|nested] 43+ messages in thread

* [PATCH v2 01/11] perf tests shell: Skip base_* dirs in test script search
  2024-07-02 11:08   ` [PATCH v2 " vmolnaro
@ 2024-07-02 11:08     ` vmolnaro
  2024-07-02 11:08     ` [PATCH v2 02/11] perf testsuite: Merge settings files for shell tests vmolnaro
                       ` (10 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: vmolnaro @ 2024-07-02 11:08 UTC (permalink / raw)
  To: linux-perf-users, acme, namhyung
  Cc: acme, mpetlan, irogers, atrajeev, masami.hiramatsu.pt

From: Michael Petlan <mpetlan@redhat.com>

The test scripts in base_* directories currently have their own drivers
that run them. Before this patch, the shell test-suite generator causes
them to run twice. Fix that by skipping them in the generator.

A cleaner solution (for future) will be to use the directory structure
idea (introduced by Carsten Haitzler in 7391db645938 ("perf test:
Refactor shell tests allowing subdirs")) to generate test entries with
subtests, like:

$ perf test list
[...]
97: perf probe shell tests
97:1: perf probe basic functionality
97:2: perf probe tests with arguments
97:3: perf probe invalid options handling
[...]

There is already a lot of shell test scripts and many are about to come,
so there is a need for some hierarchy.

Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 tools/perf/tests/tests-scripts.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/perf/tests/tests-scripts.c b/tools/perf/tests/tests-scripts.c
index e2042b3682..01971e9974 100644
--- a/tools/perf/tests/tests-scripts.c
+++ b/tools/perf/tests/tests-scripts.c
@@ -222,6 +222,8 @@ static void append_scripts_in_dir(int dir_fd,
 			if (!S_ISDIR(st.st_mode))
 				continue;
 		}
+		if (strncmp(ent->d_name, "base_", 5) == 0)
+			continue; /* Skip scripts that have a separate driver. */
 		fd = openat(dir_fd, ent->d_name, O_PATH);
 		append_scripts_in_dir(fd, result, result_sz);
 	}
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH v2 02/11] perf testsuite: Merge settings files for shell tests
  2024-07-02 11:08   ` [PATCH v2 " vmolnaro
  2024-07-02 11:08     ` [PATCH v2 01/11] perf tests shell: Skip base_* dirs in test script search vmolnaro
@ 2024-07-02 11:08     ` vmolnaro
  2024-07-02 11:08     ` [PATCH v2 03/11] perf testsuite: Fix shellcheck warnings vmolnaro
                       ` (9 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: vmolnaro @ 2024-07-02 11:08 UTC (permalink / raw)
  To: linux-perf-users, acme, namhyung
  Cc: acme, mpetlan, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Merge perf testsuite setting files into common settings to reduce
duplicates and prevent errors.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 tools/perf/tests/shell/base_probe/settings.sh | 48 -------------------
 .../shell/base_probe/test_adding_kernel.sh    |  3 --
 tools/perf/tests/shell/common/init.sh         | 23 +++++++++
 tools/perf/tests/shell/common/settings.sh     | 23 +++++++++
 4 files changed, 46 insertions(+), 51 deletions(-)
 delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh

diff --git a/tools/perf/tests/shell/base_probe/settings.sh b/tools/perf/tests/shell/base_probe/settings.sh
deleted file mode 100644
index 123621c7f9..0000000000
--- a/tools/perf/tests/shell/base_probe/settings.sh
+++ /dev/null
@@ -1,48 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-#
-#	settings.sh of perf_probe test
-#	Author: Michael Petlan <mpetlan@redhat.com>
-#	Author: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
-#
-
-export TEST_NAME="perf_probe"
-
-export MY_ARCH=`arch`
-
-if [ -n "$PERFSUITE_RUN_DIR" ]; then
-	# when $PERFSUITE_RUN_DIR is set to something, all the logs and temp files will be placed there
-	# --> the $PERFSUITE_RUN_DIR/perf_something/examples and $PERFSUITE_RUN_DIR/perf_something/logs
-	#     dirs will be used for that
-	export PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`
-	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
-	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
-	test -d "$MAKE_TARGET_DIR" || mkdir -p "$MAKE_TARGET_DIR"
-	export LOGS_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME/logs"
-	test -d "$LOGS_DIR" || mkdir -p "$LOGS_DIR"
-else
-	# when $PERFSUITE_RUN_DIR is not set, logs will be placed here
-	export CURRENT_TEST_DIR="."
-	export LOGS_DIR="."
-fi
-
-check_kprobes_available()
-{
-	test -e /sys/kernel/debug/tracing/kprobe_events
-}
-
-check_uprobes_available()
-{
-	test -e /sys/kernel/debug/tracing/uprobe_events
-}
-
-clear_all_probes()
-{
-	echo 0 > /sys/kernel/debug/tracing/events/enable
-	check_kprobes_available && echo > /sys/kernel/debug/tracing/kprobe_events
-	check_uprobes_available && echo > /sys/kernel/debug/tracing/uprobe_events
-}
-
-check_sdt_support()
-{
-	$CMD_PERF list sdt | grep sdt > /dev/null 2> /dev/null
-}
diff --git a/tools/perf/tests/shell/base_probe/test_adding_kernel.sh b/tools/perf/tests/shell/base_probe/test_adding_kernel.sh
index 187dc8d4b1..d541ffd44a 100755
--- a/tools/perf/tests/shell/base_probe/test_adding_kernel.sh
+++ b/tools/perf/tests/shell/base_probe/test_adding_kernel.sh
@@ -15,10 +15,7 @@
 
 # include working environment
 . ../common/init.sh
-. ./settings.sh
 
-# shellcheck disable=SC2034 # the variable is later used after the working environment is included
-THIS_TEST_NAME=`basename $0 .sh`
 TEST_RESULT=0
 
 # shellcheck source=lib/probe_vfs_getname.sh
diff --git a/tools/perf/tests/shell/common/init.sh b/tools/perf/tests/shell/common/init.sh
index aadeaf782e..b99fba50ba 100644
--- a/tools/perf/tests/shell/common/init.sh
+++ b/tools/perf/tests/shell/common/init.sh
@@ -115,3 +115,26 @@ detect_amd()
 	# 1 = is not AMD or unknown
 	grep "vendor_id" < /proc/cpuinfo | grep -q "AMD"
 }
+
+# base probe utility
+check_kprobes_available()
+{
+	test -e /sys/kernel/debug/tracing/kprobe_events
+}
+
+check_uprobes_available()
+{
+	test -e /sys/kernel/debug/tracing/uprobe_events
+}
+
+clear_all_probes()
+{
+	echo 0 > /sys/kernel/debug/tracing/events/enable
+	check_kprobes_available && echo > /sys/kernel/debug/tracing/kprobe_events
+	check_uprobes_available && echo > /sys/kernel/debug/tracing/uprobe_events
+}
+
+check_sdt_support()
+{
+	$CMD_PERF list sdt | grep sdt > /dev/null 2> /dev/null
+}
diff --git a/tools/perf/tests/shell/common/settings.sh b/tools/perf/tests/shell/common/settings.sh
index 361641dbaa..dcdb7d49ac 100644
--- a/tools/perf/tests/shell/common/settings.sh
+++ b/tools/perf/tests/shell/common/settings.sh
@@ -65,6 +65,29 @@ else
 	export MEND=""
 fi
 
+### general info
+DIR_PATH=`dirname "$(readlink -e "$0")"`
+
+export TEST_NAME=`basename $DIR_PATH | sed 's/base/perf/'`
+export MY_ARCH=`arch`
+
+# storing logs and temporary files variables
+if [ -n "$PERFSUITE_RUN_DIR" ]; then
+	# when $PERFSUITE_RUN_DIR is set to something, all the logs and temp files will be placed there
+	# --> the $PERFSUITE_RUN_DIR/perf_something/examples and $PERFSUITE_RUN_DIR/perf_something/logs
+	#     dirs will be used for that
+	export PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`
+	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
+	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
+	export LOGS_DIR="$CURRENT_TEST_DIR/logs"
+	test -d "$CURRENT_TEST_DIR" || mkdir -p "$CURRENT_TEST_DIR"
+	test -d "$LOGS_DIR" || mkdir -p "$LOGS_DIR"
+else
+	# when $PERFSUITE_RUN_DIR is not set, logs will be placed here
+	export CURRENT_TEST_DIR="."
+	export LOGS_DIR="."
+fi
+
 
 #### test parametrization
 if [ ! -d ./common ]; then
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH v2 03/11] perf testsuite: Fix shellcheck warnings
  2024-07-02 11:08   ` [PATCH v2 " vmolnaro
  2024-07-02 11:08     ` [PATCH v2 01/11] perf tests shell: Skip base_* dirs in test script search vmolnaro
  2024-07-02 11:08     ` [PATCH v2 02/11] perf testsuite: Merge settings files for shell tests vmolnaro
@ 2024-07-02 11:08     ` vmolnaro
  2024-07-02 11:08     ` [PATCH v2 04/11] perf testsuite probe: Add test for blacklisted kprobes handling vmolnaro
                       ` (8 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: vmolnaro @ 2024-07-02 11:08 UTC (permalink / raw)
  To: linux-perf-users, acme, namhyung
  Cc: acme, mpetlan, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Shellcheck is becoming a standard when building perf to prevent
any unnecessary mistakes. Fix shellcheck warnings in perf testsuite.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 tools/perf/tests/shell/common/init.sh     | 8 ++++----
 tools/perf/tests/shell/common/settings.sh | 9 +++++----
 2 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/tools/perf/tests/shell/common/init.sh b/tools/perf/tests/shell/common/init.sh
index b99fba50ba..075f17623c 100644
--- a/tools/perf/tests/shell/common/init.sh
+++ b/tools/perf/tests/shell/common/init.sh
@@ -26,8 +26,8 @@ print_results()
 	PERF_RETVAL="$1"; shift
 	CHECK_RETVAL="$1"; shift
 	FAILURE_REASON=""
-	TASK_COMMENT="$@"
-	if [ $PERF_RETVAL -eq 0 -a $CHECK_RETVAL -eq 0 ]; then
+	TASK_COMMENT="$*"
+	if [ $PERF_RETVAL -eq 0 ] && [ $CHECK_RETVAL -eq 0 ]; then
 		_echo "$MPASS-- [ PASS ] --$MEND $TEST_NAME :: $THIS_TEST_NAME :: $TASK_COMMENT"
 		return 0
 	else
@@ -56,7 +56,7 @@ print_overall_results()
 
 print_testcase_skipped()
 {
-	TASK_COMMENT="$@"
+	TASK_COMMENT="$*"
 	_echo "$MSKIP-- [ SKIP ] --$MEND $TEST_NAME :: $THIS_TEST_NAME :: $TASK_COMMENT :: testcase skipped"
 	return 0
 }
@@ -69,7 +69,7 @@ print_overall_skipped()
 
 print_warning()
 {
-	WARN_COMMENT="$@"
+	WARN_COMMENT="$*"
 	_echo "$MWARN-- [ WARN ] --$MEND $TEST_NAME :: $THIS_TEST_NAME :: $WARN_COMMENT"
 	return 0
 }
diff --git a/tools/perf/tests/shell/common/settings.sh b/tools/perf/tests/shell/common/settings.sh
index dcdb7d49ac..2e50dd0bff 100644
--- a/tools/perf/tests/shell/common/settings.sh
+++ b/tools/perf/tests/shell/common/settings.sh
@@ -45,7 +45,7 @@ export TEST_IGNORE_MISSING_PMU=${TEST_IGNORE_MISSING_PMU:-n}
 export LC_ALL=C
 
 #### colors
-if [ -t 1 -o "$TESTLOG_FORCE_COLOR" = "yes" ]; then
+if [ -t 1 ] || [ "$TESTLOG_FORCE_COLOR" = "yes" ]; then
 	export MPASS="\e[32m"
 	export MALLPASS="\e[1;32m"
 	export MFAIL="\e[31m"
@@ -68,15 +68,15 @@ fi
 ### general info
 DIR_PATH=`dirname "$(readlink -e "$0")"`
 
-export TEST_NAME=`basename $DIR_PATH | sed 's/base/perf/'`
-export MY_ARCH=`arch`
+TEST_NAME=`basename $DIR_PATH | sed 's/base/perf/'`; export TEST_NAME
+MY_ARCH=`arch`; export MY_ARCH
 
 # storing logs and temporary files variables
 if [ -n "$PERFSUITE_RUN_DIR" ]; then
 	# when $PERFSUITE_RUN_DIR is set to something, all the logs and temp files will be placed there
 	# --> the $PERFSUITE_RUN_DIR/perf_something/examples and $PERFSUITE_RUN_DIR/perf_something/logs
 	#     dirs will be used for that
-	export PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`
+	PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`; export PERFSUITE_RUN_DIR
 	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
 	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
 	export LOGS_DIR="$CURRENT_TEST_DIR/logs"
@@ -93,6 +93,7 @@ fi
 if [ ! -d ./common ]; then
 	# set parameters based on runmode
 	if [ -f ../common/parametrization.$PERFTOOL_TESTSUITE_RUNMODE.sh ]; then
+		# shellcheck source=/dev/null
 		. ../common/parametrization.$PERFTOOL_TESTSUITE_RUNMODE.sh
 	fi
 	# if some parameters haven't been set until now, set them to default
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH v2 04/11] perf testsuite probe: Add test for blacklisted kprobes handling
  2024-07-02 11:08   ` [PATCH v2 " vmolnaro
                       ` (2 preceding siblings ...)
  2024-07-02 11:08     ` [PATCH v2 03/11] perf testsuite: Fix shellcheck warnings vmolnaro
@ 2024-07-02 11:08     ` vmolnaro
  2024-07-02 11:08     ` [PATCH v2 05/11] perf testsuite probe: Add test for basic perf-probe options vmolnaro
                       ` (7 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: vmolnaro @ 2024-07-02 11:08 UTC (permalink / raw)
  To: linux-perf-users, acme, namhyung
  Cc: acme, mpetlan, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Test perf probe interface. Blacklisted functions should be rejected
when there is an attempt to set a kprobe to them.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 .../base_probe/test_adding_blacklisted.sh     | 67 +++++++++++++++++++
 1 file changed, 67 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh

diff --git a/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
new file mode 100755
index 0000000000..b5dc10b2a7
--- /dev/null
+++ b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
@@ -0,0 +1,67 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_adding_blacklisted of perf_probe test
+#	Author: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
+#	Author: Michael Petlan <mpetlan@redhat.com>
+#
+#	Description:
+#
+#		Blacklisted functions should not be added successfully as probes,
+#	they must be skipped.
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+# skip if not supported
+BLACKFUNC=`head -n 1 /sys/kernel/debug/kprobes/blacklist 2> /dev/null | cut -f2`
+if [ -z "$BLACKFUNC" ]; then
+	print_overall_skipped
+	exit 0
+fi
+
+# remove all previously added probes
+clear_all_probes
+
+
+### adding blacklisted function
+
+# functions from blacklist should be skipped by perf probe
+! $CMD_PERF probe $BLACKFUNC > $LOGS_DIR/adding_blacklisted.log 2> $LOGS_DIR/adding_blacklisted.err
+PERF_EXIT_CODE=$?
+
+REGEX_SCOPE_FAIL="Failed to find scope of probe point"
+REGEX_SKIP_MESSAGE=" is blacklisted function, skip it\."
+REGEX_NOT_FOUND_MESSAGE="Probe point \'$BLACKFUNC\' not found."
+REGEX_ERROR_MESSAGE="Error: Failed to add events."
+REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
+REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
+REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
+../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"
+(( TEST_RESULT += $? ))
+
+
+### listing not-added probe
+
+# blacklisted probes should NOT appear in perf-list output
+$CMD_PERF list probe:\* > $LOGS_DIR/adding_blacklisted_list.log
+PERF_EXIT_CODE=$?
+
+../common/check_all_lines_matched.pl "$RE_LINE_EMPTY" "List of pre-defined events" "Metric Groups:" < $LOGS_DIR/adding_blacklisted_list.log
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "listing blacklisted probe (should NOT be listed)"
+(( TEST_RESULT += $? ))
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH v2 05/11] perf testsuite probe: Add test for basic perf-probe options
  2024-07-02 11:08   ` [PATCH v2 " vmolnaro
                       ` (3 preceding siblings ...)
  2024-07-02 11:08     ` [PATCH v2 04/11] perf testsuite probe: Add test for blacklisted kprobes handling vmolnaro
@ 2024-07-02 11:08     ` vmolnaro
  2024-07-02 11:08     ` [PATCH v2 06/11] perf testsuite probe: Add test for invalid options vmolnaro
                       ` (6 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: vmolnaro @ 2024-07-02 11:08 UTC (permalink / raw)
  To: linux-perf-users, acme, namhyung
  Cc: acme, mpetlan, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Test basic behavior of perf-probe subcommand. It is run as a part of
perftool-testsuite_probe test case.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 .../perf/tests/shell/base_probe/test_basic.sh | 80 +++++++++++++++++++
 1 file changed, 80 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh

diff --git a/tools/perf/tests/shell/base_probe/test_basic.sh b/tools/perf/tests/shell/base_probe/test_basic.sh
new file mode 100755
index 0000000000..09669ec479
--- /dev/null
+++ b/tools/perf/tests/shell/base_probe/test_basic.sh
@@ -0,0 +1,80 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_basic of perf_probe test
+#	Author: Michael Petlan <mpetlan@redhat.com>
+#	Author: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
+#
+#	Description:
+#
+#		This test tests basic functionality of perf probe command.
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+if ! check_kprobes_available; then
+	print_overall_skipped
+	exit 0
+fi
+
+
+### help message
+
+if [ "$PARAM_GENERAL_HELP_TEXT_CHECK" = "y" ]; then
+	# test that a help message is shown and looks reasonable
+	$CMD_PERF probe --help > $LOGS_DIR/basic_helpmsg.log 2> $LOGS_DIR/basic_helpmsg.err
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "PERF-PROBE" "NAME" "SYNOPSIS" "DESCRIPTION" "OPTIONS" "PROBE\s+SYNTAX" "PROBE\s+ARGUMENT" "LINE\s+SYNTAX" < $LOGS_DIR/basic_helpmsg.log
+	CHECK_EXIT_CODE=$?
+	../common/check_all_patterns_found.pl "LAZY\s+MATCHING" "FILTER\s+PATTERN" "EXAMPLES" "SEE\s+ALSO" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "vmlinux" "module=" "source=" "verbose" "quiet" "add=" "del=" "list.*EVENT" "line=" "vars=" "externs" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "no-inlines" "funcs.*FILTER" "filter=FILTER" "force" "dry-run" "max-probes" "exec=" "demangle-kernel" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_no_patterns_found.pl "No manual entry for" < $LOGS_DIR/basic_helpmsg.err
+	(( CHECK_EXIT_CODE += $? ))
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "help message"
+	(( TEST_RESULT += $? ))
+else
+	print_testcase_skipped "help message"
+fi
+
+
+### usage message
+
+# without any args perf-probe should print usage
+$CMD_PERF probe 2> $LOGS_DIR/basic_usage.log > /dev/null
+
+../common/check_all_patterns_found.pl "[Uu]sage" "perf probe" "verbose" "quiet" "add" "del" "force" "line" "vars" "externs" "range" < $LOGS_DIR/basic_usage.log
+CHECK_EXIT_CODE=$?
+
+print_results 0 $CHECK_EXIT_CODE "usage message"
+(( TEST_RESULT += $? ))
+
+
+### quiet switch
+
+# '--quiet' should mute all output
+$CMD_PERF probe --quiet --add vfs_read > $LOGS_DIR/basic_quiet01.log 2> $LOGS_DIR/basic_quiet01.err
+PERF_EXIT_CODE=$?
+$CMD_PERF probe --quiet --del vfs_read > $LOGS_DIR/basic_quiet03.log 2> $LOGS_DIR/basic_quiet02.err
+(( PERF_EXIT_CODE += $? ))
+
+test "`cat $LOGS_DIR/basic_quiet*log $LOGS_DIR/basic_quiet*err | wc -l`" -eq 0
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "quiet switch"
+(( TEST_RESULT += $? ))
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH v2 06/11] perf testsuite probe: Add test for invalid options
  2024-07-02 11:08   ` [PATCH v2 " vmolnaro
                       ` (4 preceding siblings ...)
  2024-07-02 11:08     ` [PATCH v2 05/11] perf testsuite probe: Add test for basic perf-probe options vmolnaro
@ 2024-07-02 11:08     ` vmolnaro
  2024-07-02 11:08     ` [PATCH v2 07/11] perf testsuite probe: Add test for line semantics vmolnaro
                       ` (5 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: vmolnaro @ 2024-07-02 11:08 UTC (permalink / raw)
  To: linux-perf-users, acme, namhyung
  Cc: acme, mpetlan, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Test if various incompatible options are correctly handled-rejected.
It is run as a part of perftool-testsuite_probe test case.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 .../shell/base_probe/test_invalid_options.sh  | 79 +++++++++++++++++++
 1 file changed, 79 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh

diff --git a/tools/perf/tests/shell/base_probe/test_invalid_options.sh b/tools/perf/tests/shell/base_probe/test_invalid_options.sh
new file mode 100755
index 0000000000..1fedfd8b0d
--- /dev/null
+++ b/tools/perf/tests/shell/base_probe/test_invalid_options.sh
@@ -0,0 +1,79 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_invalid_options of perf_probe test
+#	Author: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
+#	Author: Michael Petlan <mpetlan@redhat.com>
+#
+#	Description:
+#
+#		This test checks whether the invalid and incompatible options are reported
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+if ! check_kprobes_available; then
+	print_overall_skipped
+	exit 0
+fi
+
+
+### missing argument
+
+# some options require an argument
+for opt in '-a' '-d' '-L' '-V'; do
+	! $CMD_PERF probe $opt 2> $LOGS_DIR/invalid_options_missing_argument$opt.err
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "Error: switch .* requires a value" < $LOGS_DIR/invalid_options_missing_argument$opt.err
+	CHECK_EXIT_CODE=$?
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "missing argument for $opt"
+	(( TEST_RESULT += $? ))
+done
+
+
+### unnecessary argument
+
+# some options may omit the argument
+for opt in '-F' '-l'; do
+	$CMD_PERF probe -F > /dev/null 2> $LOGS_DIR/invalid_options_unnecessary_argument$opt.err
+	PERF_EXIT_CODE=$?
+
+	test ! -s $LOGS_DIR/invalid_options_unnecessary_argument$opt.err
+	CHECK_EXIT_CODE=$?
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "unnecessary argument for $opt"
+	(( TEST_RESULT += $? ))
+done
+
+
+### mutually exclusive options
+
+# some options are mutually exclusive
+test -e $LOGS_DIR/invalid_options_mutually_exclusive.log && rm -f $LOGS_DIR/invalid_options_mutually_exclusive.log
+for opt in '-a xxx -d xxx' '-a xxx -L foo' '-a xxx -V foo' '-a xxx -l' '-a xxx -F' \
+		'-d xxx -L foo' '-d xxx -V foo' '-d xxx -l' '-d xxx -F' \
+		'-L foo -V bar' '-L foo -l' '-L foo -F' '-V foo -l' '-V foo -F' '-l -F'; do
+	! $CMD_PERF probe $opt > /dev/null 2> $LOGS_DIR/aux.log
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "Error: switch .+ cannot be used with switch .+" < $LOGS_DIR/aux.log
+	CHECK_EXIT_CODE=$?
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "mutually exclusive options :: $opt"
+	(( TEST_RESULT += $? ))
+
+	# gather the logs
+	cat $LOGS_DIR/aux.log | grep "Error" >> $LOGS_DIR/invalid_options_mutually_exclusive.log
+done
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH v2 07/11] perf testsuite probe: Add test for line semantics
  2024-07-02 11:08   ` [PATCH v2 " vmolnaro
                       ` (5 preceding siblings ...)
  2024-07-02 11:08     ` [PATCH v2 06/11] perf testsuite probe: Add test for invalid options vmolnaro
@ 2024-07-02 11:08     ` vmolnaro
  2024-07-02 11:08     ` [PATCH v2 08/11] perf testsuite: Add common output checking helper vmolnaro
                       ` (4 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: vmolnaro @ 2024-07-02 11:08 UTC (permalink / raw)
  To: linux-perf-users, acme, namhyung
  Cc: acme, mpetlan, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

The perf-probe command uses a specific semantics to describe probes.
Test some patterns that are known to be both valid and invalid if
they are handled appropriately.

This test is run as a part of perftool-testsuite_probe test case.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 .../shell/base_probe/test_line_semantics.sh   | 55 +++++++++++++++++++
 1 file changed, 55 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh

diff --git a/tools/perf/tests/shell/base_probe/test_line_semantics.sh b/tools/perf/tests/shell/base_probe/test_line_semantics.sh
new file mode 100755
index 0000000000..d8f4bde0f5
--- /dev/null
+++ b/tools/perf/tests/shell/base_probe/test_line_semantics.sh
@@ -0,0 +1,55 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_line_semantics of perf_probe test
+#	Author: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
+#	Author: Michael Petlan <mpetlan@redhat.com>
+#
+#	Description:
+#
+#		This test checks whether the semantic errors of line option's
+#		arguments are properly reported.
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+if ! check_kprobes_available; then
+	print_overall_skipped
+	exit 0
+fi
+
+
+### acceptable --line descriptions
+
+# testing acceptance of valid patterns for the '--line' option
+VALID_PATTERNS="func func:10 func:0-10 func:2+10 func@source.c func@source.c:1 source.c:1 source.c:1+1 source.c:1-10"
+for desc in $VALID_PATTERNS; do
+	! ( $CMD_PERF probe --line $desc 2>&1 | grep -q "Semantic error" )
+	CHECK_EXIT_CODE=$?
+
+	print_results 0 $CHECK_EXIT_CODE "acceptable descriptions :: $desc"
+	(( TEST_RESULT += $? ))
+done
+
+
+### unacceptable --line descriptions
+
+# testing handling of invalid patterns for the '--line' option
+INVALID_PATTERNS="func:foo func:1-foo func:1+foo func;lazy\*pattern"
+for desc in $INVALID_PATTERNS; do
+	$CMD_PERF probe --line $desc 2>&1 | grep -q "Semantic error"
+	CHECK_EXIT_CODE=$?
+
+	print_results 0 $CHECK_EXIT_CODE "unacceptable descriptions :: $desc"
+	(( TEST_RESULT += $? ))
+done
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH v2 08/11] perf testsuite: Add common output checking helper
  2024-07-02 11:08   ` [PATCH v2 " vmolnaro
                       ` (6 preceding siblings ...)
  2024-07-02 11:08     ` [PATCH v2 07/11] perf testsuite probe: Add test for line semantics vmolnaro
@ 2024-07-02 11:08     ` vmolnaro
  2024-07-02 11:08     ` [PATCH v2 09/11] perf testsuite report: Add test for perf-report basic functionality vmolnaro
                       ` (3 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: vmolnaro @ 2024-07-02 11:08 UTC (permalink / raw)
  To: linux-perf-users, acme, namhyung
  Cc: acme, mpetlan, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

As a form of validation, it is a common practice to check the outputs
of commands whether they contain expected patterns or match a certain
regular expression.

This output checking helper is designed to allow checking stderr output
of perf commands for unexpected messages, while ignoring messages that
are known to be harmless, e.g.:
  "Lowering default frequency rate to \d+\."
  "\d+ out of order events recorded."
  etc.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 .../shell/common/check_errors_whitelisted.pl  | 51 +++++++++++++++++++
 1 file changed, 51 insertions(+)
 create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl

diff --git a/tools/perf/tests/shell/common/check_errors_whitelisted.pl b/tools/perf/tests/shell/common/check_errors_whitelisted.pl
new file mode 100755
index 0000000000..c57d355dd7
--- /dev/null
+++ b/tools/perf/tests/shell/common/check_errors_whitelisted.pl
@@ -0,0 +1,51 @@
+#!/usr/bin/perl
+# SPDX-License-Identifier: GPL-2.0
+
+$whitelist_file = shift;
+
+if (defined $whitelist_file)
+{
+	open (INFILE, $whitelist_file) or die "Checker error: Unable to open the whitelist file: $whitelist_file\n";
+	@regexps = <INFILE>;
+	close INFILE or die "Checker error: Unable to close the whitelist file: $whitelist_file\n";
+}
+else
+{
+	@regexps = ();
+}
+
+$max_printed_lines = 20;
+$max_printed_lines = $ENV{TESTLOG_ERR_MSG_MAX_LINES} if (defined $ENV{TESTLOG_ERR_MSG_MAX_LINES});
+
+$quiet = 1;
+$quiet = 0 if (defined $ENV{TESTLOG_VERBOSITY} && $ENV{TESTLOG_VERBOSITY} ge 2);
+
+$passed = 1;
+$lines_printed = 0;
+
+while (<STDIN>)
+{
+	s/\n//;
+
+	$line_matched = 0;
+	for $r (@regexps)
+	{
+		chomp $r;
+		if (/$r/)
+		{
+			$line_matched = 1;
+			last;
+		}
+	}
+
+	unless ($line_matched)
+	{
+		if ($lines_printed++ < $max_printed_lines)
+		{
+			print "Line did not match any pattern: \"$_\"\n" unless $quiet;
+		}
+		$passed = 0;
+	}
+}
+
+exit ($passed == 0);
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH v2 09/11] perf testsuite report: Add test for perf-report basic functionality
  2024-07-02 11:08   ` [PATCH v2 " vmolnaro
                       ` (7 preceding siblings ...)
  2024-07-02 11:08     ` [PATCH v2 08/11] perf testsuite: Add common output checking helper vmolnaro
@ 2024-07-02 11:08     ` vmolnaro
  2024-07-02 11:08     ` [PATCH v2 10/11] perf testsuite report: Add test case for perf report vmolnaro
                       ` (2 subsequent siblings)
  11 siblings, 0 replies; 43+ messages in thread
From: vmolnaro @ 2024-07-02 11:08 UTC (permalink / raw)
  To: linux-perf-users, acme, namhyung
  Cc: acme, mpetlan, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Test basic execution and some options of perf-report subcommand, like
show-nr-samples, header, showcpuutilization, pid and symbol filtering.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 tools/perf/tests/shell/base_report/setup.sh   |  32 +++
 .../shell/base_report/stderr-whitelist.txt    |   5 +
 .../tests/shell/base_report/test_basic.sh     | 190 ++++++++++++++++++
 tools/perf/tests/shell/common/settings.sh     |   2 +
 4 files changed, 229 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_report/setup.sh
 create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
 create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh

diff --git a/tools/perf/tests/shell/base_report/setup.sh b/tools/perf/tests/shell/base_report/setup.sh
new file mode 100755
index 0000000000..4caa496660
--- /dev/null
+++ b/tools/perf/tests/shell/base_report/setup.sh
@@ -0,0 +1,32 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	setup.sh of perf report test
+#	Author: Michael Petlan <mpetlan@redhat.com>
+#
+#	Description:
+#
+#		We need some sample data for perf-report testing
+#
+#
+
+# include working environment
+. ../common/init.sh
+
+test -d "$HEADER_TAR_DIR" || mkdir -p "$HEADER_TAR_DIR"
+
+SW_EVENT="cpu-clock"
+
+$CMD_PERF record -asdg -e $SW_EVENT -o $CURRENT_TEST_DIR/perf.data -- $CMD_LONGER_SLEEP 2> $LOGS_DIR/setup.log
+PERF_EXIT_CODE=$?
+
+../common/check_all_patterns_found.pl "$RE_LINE_RECORD1" "$RE_LINE_RECORD2" < $LOGS_DIR/setup.log
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "prepare the perf.data file"
+TEST_RESULT=$?
+
+print_overall_results $TEST_RESULT
+exit $?
diff --git a/tools/perf/tests/shell/base_report/stderr-whitelist.txt b/tools/perf/tests/shell/base_report/stderr-whitelist.txt
new file mode 100644
index 0000000000..e3341401b4
--- /dev/null
+++ b/tools/perf/tests/shell/base_report/stderr-whitelist.txt
@@ -0,0 +1,5 @@
+no symbols found in .*, maybe install a debug package
+was updated .*is prelink enabled.+ Restart the long running apps that use it
+Warning:
+\d+ out of order events recorded.
+detected invalid bpf_prog_info
diff --git a/tools/perf/tests/shell/base_report/test_basic.sh b/tools/perf/tests/shell/base_report/test_basic.sh
new file mode 100755
index 0000000000..47677cbd4d
--- /dev/null
+++ b/tools/perf/tests/shell/base_report/test_basic.sh
@@ -0,0 +1,190 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_basic of perf_report test
+#	Author: Michael Petlan <mpetlan@redhat.com>
+#
+#	Description:
+#
+#		This test tests basic functionality of perf report command.
+#
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+
+### help message
+
+if [ "$PARAM_GENERAL_HELP_TEXT_CHECK" = "y" ]; then
+	# test that a help message is shown and looks reasonable
+	$CMD_PERF report --help > $LOGS_DIR/basic_helpmsg.log 2> $LOGS_DIR/basic_helpmsg.err
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "PERF-REPORT" "NAME" "SYNOPSIS" "DESCRIPTION" "OPTIONS" "OVERHEAD\s+CALCULATION" "SEE ALSO" < $LOGS_DIR/basic_helpmsg.log
+	CHECK_EXIT_CODE=$?
+	../common/check_all_patterns_found.pl "input" "verbose" "show-nr-samples" "show-cpu-utilization" "threads" "comms" "pid" "tid" "dsos" "symbols" "symbol-filter" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "hide-unresolved" "sort" "fields" "parent" "exclude-other" "column-widths" "field-separator" "dump-raw-trace" "children" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "call-graph" "max-stack" "inverted" "ignore-callees" "pretty" "stdio" "tui" "gtk" "vmlinux" "kallsyms" "modules" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "force" "symfs" "cpu" "disassembler-style" "source" "asm-raw" "show-total-period" "show-info" "branch-stack" "group" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "branch-history" "objdump" "demangle" "percent-limit" "percentage" "header" "itrace" "full-source-path" "show-ref-call-graph" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_no_patterns_found.pl "No manual entry for" < $LOGS_DIR/basic_helpmsg.err
+	(( CHECK_EXIT_CODE += $? ))
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "help message"
+	(( TEST_RESULT += $? ))
+else
+	print_testcase_skipped "help message"
+fi
+
+
+### basic execution
+
+# test that perf report is even working
+$CMD_PERF report -i $CURRENT_TEST_DIR/perf.data --stdio > $LOGS_DIR/basic_basic.log 2> $LOGS_DIR/basic_basic.err
+PERF_EXIT_CODE=$?
+
+REGEX_LOST_SAMPLES_INFO="#\s*Total Lost Samples:\s+$RE_NUMBER"
+REGEX_SAMPLES_INFO="#\s*Samples:\s+(?:$RE_NUMBER)\w?\s+of\s+event\s+'$RE_EVENT_ANY'"
+REGEX_LINES_HEADER="#\s*Children\s+Self\s+Command\s+Shared Object\s+Symbol"
+REGEX_LINES="\s*$RE_NUMBER%\s+$RE_NUMBER%\s+\S+\s+\[kernel\.(?:vmlinux)|(?:kallsyms)\]\s+\[[k\.]\]\s+\w+"
+../common/check_all_patterns_found.pl "$REGEX_LOST_SAMPLES_INFO" "$REGEX_SAMPLES_INFO" "$REGEX_LINES_HEADER" "$REGEX_LINES" < $LOGS_DIR/basic_basic.log
+CHECK_EXIT_CODE=$?
+../common/check_errors_whitelisted.pl "stderr-whitelist.txt" < $LOGS_DIR/basic_basic.err
+(( CHECK_EXIT_CODE += $? ))
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "basic execution"
+(( TEST_RESULT += $? ))
+
+
+### number of samples
+
+# '--show-nr-samples' should show number of samples for each symbol
+$CMD_PERF report --stdio -i $CURRENT_TEST_DIR/perf.data --show-nr-samples > $LOGS_DIR/basic_nrsamples.log 2> $LOGS_DIR/basic_nrsamples.err
+PERF_EXIT_CODE=$?
+
+REGEX_LINES_HEADER="#\s*Children\s+Self\s+Samples\s+Command\s+Shared Object\s+Symbol"
+REGEX_LINES="\s*$RE_NUMBER%\s+$RE_NUMBER%\s+$RE_NUMBER\s+\S+\s+\[kernel\.(?:vmlinux)|(?:kallsyms)\]\s+\[[k\.]\]\s+\w+"
+../common/check_all_patterns_found.pl "$REGEX_LINES_HEADER" "$REGEX_LINES" < $LOGS_DIR/basic_nrsamples.log
+CHECK_EXIT_CODE=$?
+../common/check_errors_whitelisted.pl "stderr-whitelist.txt" < $LOGS_DIR/basic_nrsamples.err
+(( CHECK_EXIT_CODE += $? ))
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "number of samples"
+(( TEST_RESULT += $? ))
+
+
+### header
+
+# '--header' and '--header-only' should show perf report header
+$CMD_PERF report -i $CURRENT_TEST_DIR/perf.data --stdio --header-only > $LOGS_DIR/basic_header.log
+PERF_EXIT_CODE=$?
+
+REGEX_LINE_TIMESTAMP="#\s+captured on\s*:\s*$RE_DATE_TIME"
+REGEX_LINE_HOSTNAME="#\s+hostname\s*:\s*$MY_HOSTNAME"
+REGEX_LINE_KERNEL="#\s+os release\s*:\s*${MY_KERNEL_VERSION//+/\\+}"
+REGEX_LINE_PERF="#\s+perf version\s*:\s*"
+REGEX_LINE_ARCH="#\s+arch\s*:\s*$MY_ARCH"
+REGEX_LINE_CPUS_ONLINE="#\s+nrcpus online\s*:\s*$MY_CPUS_ONLINE"
+REGEX_LINE_CPUS_AVAIL="#\s+nrcpus avail\s*:\s*$MY_CPUS_AVAILABLE"
+# disable precise check for "nrcpus avail" in BASIC runmode
+test $PERFTOOL_TESTSUITE_RUNMODE -lt $RUNMODE_STANDARD && REGEX_LINE_CPUS_AVAIL="#\s+nrcpus avail\s*:\s*$RE_NUMBER"
+../common/check_all_patterns_found.pl "$REGEX_LINE_TIMESTAMP" "$REGEX_LINE_HOSTNAME" "$REGEX_LINE_KERNEL" "$REGEX_LINE_PERF" "$REGEX_LINE_ARCH" "$REGEX_LINE_CPUS_ONLINE" "$REGEX_LINE_CPUS_AVAIL" < $LOGS_DIR/basic_header.log
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "header"
+(( TEST_RESULT += $? ))
+
+# '--header' and '--header-only' should use creation time
+OLD_TIMESTAMP=`$CMD_PERF report --stdio --header-only -i $CURRENT_TEST_DIR/perf.data | grep "captured on"`
+PERF_EXIT_CODE=$?
+
+( tar -C $CURRENT_TEST_DIR -c perf.data | xz > $CURRENT_TEST_DIR/perf.data.tar.xz ; xzcat $CURRENT_TEST_DIR/perf.data.tar.xz | tar x -C $HEADER_TAR_DIR )
+(( PERF_EXIT_CODE += $? ))
+
+NEW_TIMESTAMP=`$CMD_PERF report --stdio --header-only -i $HEADER_TAR_DIR/perf.data | grep "captured on"`
+(( PERF_EXIT_CODE += $? ))
+
+test "$OLD_TIMESTAMP" = "$NEW_TIMESTAMP"
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "header timestamp"
+(( TEST_RESULT += $? ))
+
+
+### show CPU utilization
+
+# '--showcpuutilization' should show percentage for both system and userspace mode
+$CMD_PERF report -i $CURRENT_TEST_DIR/perf.data --stdio --showcpuutilization > $LOGS_DIR/basic_cpuut.log 2> $LOGS_DIR/basic_cpuut.err
+PERF_EXIT_CODE=$?
+
+REGEX_LINES_HEADER="#\s*Children\s+Self\s+sys\s+usr\s+Command\s+Shared Object\s+Symbol"
+REGEX_LINES="\s*$RE_NUMBER%\s+$RE_NUMBER%\s+$RE_NUMBER%\s+$RE_NUMBER%\s+\S+\s+\[kernel\.(?:vmlinux)|(?:kallsyms)\]\s+\[[k\.]\]\s+\w+"
+../common/check_all_patterns_found.pl "$REGEX_LINES_HEADER" "$REGEX_LINES" < $LOGS_DIR/basic_cpuut.log
+CHECK_EXIT_CODE=$?
+../common/check_errors_whitelisted.pl "stderr-whitelist.txt" < $LOGS_DIR/basic_cpuut.err
+(( CHECK_EXIT_CODE += $? ))
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "show CPU utilization"
+(( TEST_RESULT += $? ))
+
+
+### pid
+
+# '--pid=' should limit the output for a process with the given pid only
+$CMD_PERF report --stdio -i $CURRENT_TEST_DIR/perf.data --pid=1 > $LOGS_DIR/basic_pid.log 2> $LOGS_DIR/basic_pid.err
+PERF_EXIT_CODE=$?
+
+grep -P -v '^#' $LOGS_DIR/basic_pid.log | grep -P '\s+[\d\.]+%' | ../common/check_all_lines_matched.pl "systemd|init"
+CHECK_EXIT_CODE=$?
+../common/check_errors_whitelisted.pl "stderr-whitelist.txt" < $LOGS_DIR/basic_pid.err
+(( CHECK_EXIT_CODE += $? ))
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "pid"
+(( TEST_RESULT += $? ))
+
+
+### non-existing symbol
+
+# '--symbols' should show only the given symbols
+$CMD_PERF report --stdio -i $CURRENT_TEST_DIR/perf.data --symbols=dummynonexistingsymbol > $LOGS_DIR/basic_symbols.log 2> $LOGS_DIR/basic_symbols.err
+PERF_EXIT_CODE=$?
+
+../common/check_all_lines_matched.pl "$RE_LINE_EMPTY" "$RE_LINE_COMMENT" < $LOGS_DIR/basic_symbols.log
+CHECK_EXIT_CODE=$?
+../common/check_errors_whitelisted.pl "stderr-whitelist.txt" < $LOGS_DIR/basic_symbols.err
+(( CHECK_EXIT_CODE += $? ))
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "non-existing symbol"
+(( TEST_RESULT += $? ))
+
+
+### symbol filter
+
+# '--symbol-filter' should filter symbols based on substrings
+$CMD_PERF report --stdio -i $CURRENT_TEST_DIR/perf.data --symbol-filter=map > $LOGS_DIR/basic_symbolfilter.log 2> $LOGS_DIR/basic_symbolfilter.err
+PERF_EXIT_CODE=$?
+
+grep -P -v '^#' $LOGS_DIR/basic_symbolfilter.log | grep -P '\s+[\d\.]+%' | ../common/check_all_lines_matched.pl "\[[k\.]\]\s+.*map"
+CHECK_EXIT_CODE=$?
+../common/check_errors_whitelisted.pl "stderr-whitelist.txt" < $LOGS_DIR/basic_symbolfilter.err
+(( CHECK_EXIT_CODE += $? ))
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "symbol filter"
+(( TEST_RESULT += $? ))
+
+
+# TODO: $CMD_PERF report -n --showcpuutilization -TUxDg 2> 01.log
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
diff --git a/tools/perf/tests/shell/common/settings.sh b/tools/perf/tests/shell/common/settings.sh
index 2e50dd0bff..cba1b338f9 100644
--- a/tools/perf/tests/shell/common/settings.sh
+++ b/tools/perf/tests/shell/common/settings.sh
@@ -80,12 +80,14 @@ if [ -n "$PERFSUITE_RUN_DIR" ]; then
 	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
 	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
 	export LOGS_DIR="$CURRENT_TEST_DIR/logs"
+	export HEADER_TAR_DIR="$CURRENT_TEST_DIR/header_tar"
 	test -d "$CURRENT_TEST_DIR" || mkdir -p "$CURRENT_TEST_DIR"
 	test -d "$LOGS_DIR" || mkdir -p "$LOGS_DIR"
 else
 	# when $PERFSUITE_RUN_DIR is not set, logs will be placed here
 	export CURRENT_TEST_DIR="."
 	export LOGS_DIR="."
+	export HEADER_TAR_DIR="./header_tar"
 fi
 
 
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH v2 10/11] perf testsuite report: Add test case for perf report
  2024-07-02 11:08   ` [PATCH v2 " vmolnaro
                       ` (8 preceding siblings ...)
  2024-07-02 11:08     ` [PATCH v2 09/11] perf testsuite report: Add test for perf-report basic functionality vmolnaro
@ 2024-07-02 11:08     ` vmolnaro
  2024-07-02 11:08     ` [PATCH v2 11/11] perf testsuite: Install perf-report tests vmolnaro
  2024-08-28 14:10     ` [PATCH v2 00/11] perftool-testsuite 2nd batch Veronika Molnarova
  11 siblings, 0 replies; 43+ messages in thread
From: vmolnaro @ 2024-07-02 11:08 UTC (permalink / raw)
  To: linux-perf-users, acme, namhyung
  Cc: acme, mpetlan, irogers, atrajeev, masami.hiramatsu.pt

From: Veronika Molnarova <vmolnaro@redhat.com>

Add new perf report test case that acts as an entry element in perf test
list. Runs multiple subtests from directory "base_report", which can be
expanded without further editing.

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 .../tests/shell/perftool-testsuite_report.sh  | 23 +++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh

diff --git a/tools/perf/tests/shell/perftool-testsuite_report.sh b/tools/perf/tests/shell/perftool-testsuite_report.sh
new file mode 100755
index 0000000000..973012ce92
--- /dev/null
+++ b/tools/perf/tests/shell/perftool-testsuite_report.sh
@@ -0,0 +1,23 @@
+#!/bin/bash
+# perftool-testsuite_report
+# SPDX-License-Identifier: GPL-2.0
+
+test -d "$(dirname "$0")/base_report" || exit 2
+cd "$(dirname "$0")/base_report" || exit 2
+status=0
+
+PERFSUITE_RUN_DIR=$(mktemp -d /tmp/"$(basename "$0" .sh)".XXX)
+export PERFSUITE_RUN_DIR
+
+for testcase in setup.sh test_*; do                  # skip setup.sh if not present or not executable
+     test -x "$testcase" || continue
+     ./"$testcase"
+     (( status += $? ))
+done
+
+if ! [ "$PERFTEST_KEEP_LOGS" = "y" ]; then
+	rm -rf "$PERFSUITE_RUN_DIR"
+fi
+
+test $status -ne 0 && exit 1
+exit 0
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH v2 11/11] perf testsuite: Install perf-report tests
  2024-07-02 11:08   ` [PATCH v2 " vmolnaro
                       ` (9 preceding siblings ...)
  2024-07-02 11:08     ` [PATCH v2 10/11] perf testsuite report: Add test case for perf report vmolnaro
@ 2024-07-02 11:08     ` vmolnaro
  2024-08-28 14:10     ` [PATCH v2 00/11] perftool-testsuite 2nd batch Veronika Molnarova
  11 siblings, 0 replies; 43+ messages in thread
From: vmolnaro @ 2024-07-02 11:08 UTC (permalink / raw)
  To: linux-perf-users, acme, namhyung
  Cc: acme, mpetlan, irogers, atrajeev, masami.hiramatsu.pt

From: Michael Petlan <mpetlan@redhat.com>

Signed-off-by: Michael Petlan <mpetlan@redhat.com>
---
 tools/perf/Makefile.perf | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index e6d56b5553..b921b19c39 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -1081,6 +1081,8 @@ install-tests: all install-gtk
 		$(INSTALL) tests/shell/common/*.pl '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/common'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_probe'; \
 		$(INSTALL) tests/shell/base_probe/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_probe'; \
+		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_report'; \
+		$(INSTALL) tests/shell/base_probe/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_report'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight' ; \
 		$(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
 	$(Q)$(MAKE) -C tests/shell/coresight install-tests
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-07-02 11:08   ` [PATCH v2 " vmolnaro
                       ` (10 preceding siblings ...)
  2024-07-02 11:08     ` [PATCH v2 11/11] perf testsuite: Install perf-report tests vmolnaro
@ 2024-08-28 14:10     ` Veronika Molnarova
  2024-08-28 20:39       ` Arnaldo Carvalho de Melo
  11 siblings, 1 reply; 43+ messages in thread
From: Veronika Molnarova @ 2024-08-28 14:10 UTC (permalink / raw)
  To: linux-perf-users, acme, namhyung
  Cc: acme, mpetlan, irogers, atrajeev, masami.hiramatsu.pt

Ping

On 7/2/24 13:08, vmolnaro@redhat.com wrote:
> From: Veronika Molnarova <vmolnaro@redhat.com>
> 
> Hello,
> 
> rebased the patches for the current perf-tools-next branch in v2 as it
> was not applying.
> 
> Thanks for the notice,
> Veronika
> 
> Michael Petlan (2):
>   perf tests shell: Skip base_* dirs in test script search
>   perf testsuite: Install perf-report tests
> 
> Veronika Molnarova (9):
>   perf testsuite: Merge settings files for shell tests
>   perf testsuite: Fix shellcheck warnings
>   perf testsuite probe: Add test for blacklisted kprobes handling
>   perf testsuite probe: Add test for basic perf-probe options
>   perf testsuite probe: Add test for invalid options
>   perf testsuite probe: Add test for line semantics
>   perf testsuite: Add common output checking helper
>   perf testsuite report: Add test for perf-report basic functionality
>   perf testsuite report: Add test case for perf report
> 
>  tools/perf/Makefile.perf                      |   2 +
>  tools/perf/tests/shell/base_probe/settings.sh |  48 -----
>  .../base_probe/test_adding_blacklisted.sh     |  67 ++++++
>  .../shell/base_probe/test_adding_kernel.sh    |   3 -
>  .../perf/tests/shell/base_probe/test_basic.sh |  80 ++++++++
>  .../shell/base_probe/test_invalid_options.sh  |  79 ++++++++
>  .../shell/base_probe/test_line_semantics.sh   |  55 +++++
>  tools/perf/tests/shell/base_report/setup.sh   |  32 +++
>  .../shell/base_report/stderr-whitelist.txt    |   5 +
>  .../tests/shell/base_report/test_basic.sh     | 190 ++++++++++++++++++
>  .../shell/common/check_errors_whitelisted.pl  |  51 +++++
>  tools/perf/tests/shell/common/init.sh         |  31 ++-
>  tools/perf/tests/shell/common/settings.sh     |  28 ++-
>  .../tests/shell/perftool-testsuite_report.sh  |  23 +++
>  tools/perf/tests/tests-scripts.c              |   2 +
>  15 files changed, 640 insertions(+), 56 deletions(-)
>  delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh
>  create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
>  create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh
>  create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh
>  create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh
>  create mode 100755 tools/perf/tests/shell/base_report/setup.sh
>  create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
>  create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh
>  create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl
>  create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh
> 


^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-08-28 14:10     ` [PATCH v2 00/11] perftool-testsuite 2nd batch Veronika Molnarova
@ 2024-08-28 20:39       ` Arnaldo Carvalho de Melo
  2024-08-29 11:29         ` [PATCH v2 01/11] perf tests shell: Skip base_* dirs in test script search vmolnaro
                           ` (2 more replies)
  0 siblings, 3 replies; 43+ messages in thread
From: Arnaldo Carvalho de Melo @ 2024-08-28 20:39 UTC (permalink / raw)
  To: Veronika Molnarova
  Cc: linux-perf-users, acme, namhyung, mpetlan, irogers, atrajeev,
	masami.hiramatsu.pt

On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:
> Ping

Since this v2 series came from you, we need to have your Signed-off-by
in Michael's patch, please provide them in response to this message.

I'm adding them tentatively to my local branch, but I need your public
confirmation to follow the rules.

- Arnaldo
 
> On 7/2/24 13:08, vmolnaro@redhat.com wrote:
> > From: Veronika Molnarova <vmolnaro@redhat.com>
> > 
> > Hello,
> > 
> > rebased the patches for the current perf-tools-next branch in v2 as it
> > was not applying.
> > 
> > Thanks for the notice,
> > Veronika
> > 
> > Michael Petlan (2):
> >   perf tests shell: Skip base_* dirs in test script search
> >   perf testsuite: Install perf-report tests
> > 
> > Veronika Molnarova (9):
> >   perf testsuite: Merge settings files for shell tests
> >   perf testsuite: Fix shellcheck warnings
> >   perf testsuite probe: Add test for blacklisted kprobes handling
> >   perf testsuite probe: Add test for basic perf-probe options
> >   perf testsuite probe: Add test for invalid options
> >   perf testsuite probe: Add test for line semantics
> >   perf testsuite: Add common output checking helper
> >   perf testsuite report: Add test for perf-report basic functionality
> >   perf testsuite report: Add test case for perf report
> > 
> >  tools/perf/Makefile.perf                      |   2 +
> >  tools/perf/tests/shell/base_probe/settings.sh |  48 -----
> >  .../base_probe/test_adding_blacklisted.sh     |  67 ++++++
> >  .../shell/base_probe/test_adding_kernel.sh    |   3 -
> >  .../perf/tests/shell/base_probe/test_basic.sh |  80 ++++++++
> >  .../shell/base_probe/test_invalid_options.sh  |  79 ++++++++
> >  .../shell/base_probe/test_line_semantics.sh   |  55 +++++
> >  tools/perf/tests/shell/base_report/setup.sh   |  32 +++
> >  .../shell/base_report/stderr-whitelist.txt    |   5 +
> >  .../tests/shell/base_report/test_basic.sh     | 190 ++++++++++++++++++
> >  .../shell/common/check_errors_whitelisted.pl  |  51 +++++
> >  tools/perf/tests/shell/common/init.sh         |  31 ++-
> >  tools/perf/tests/shell/common/settings.sh     |  28 ++-
> >  .../tests/shell/perftool-testsuite_report.sh  |  23 +++
> >  tools/perf/tests/tests-scripts.c              |   2 +
> >  15 files changed, 640 insertions(+), 56 deletions(-)
> >  delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh
> >  create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
> >  create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh
> >  create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh
> >  create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh
> >  create mode 100755 tools/perf/tests/shell/base_report/setup.sh
> >  create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
> >  create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh
> >  create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl
> >  create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh
> > 

^ permalink raw reply	[flat|nested] 43+ messages in thread

* [PATCH v2 01/11] perf tests shell: Skip base_* dirs in test script search
  2024-08-28 20:39       ` Arnaldo Carvalho de Melo
@ 2024-08-29 11:29         ` vmolnaro
  2024-08-29 11:29         ` [PATCH v2 11/11] perf testsuite: Install perf-report tests vmolnaro
  2024-08-30 15:27         ` [PATCH v2 00/11] perftool-testsuite 2nd batch Arnaldo Carvalho de Melo
  2 siblings, 0 replies; 43+ messages in thread
From: vmolnaro @ 2024-08-29 11:29 UTC (permalink / raw)
  To: linux-perf-users, acme, acme, namhyung; +Cc: mpetlan, irogers, atrajeev

From: Michael Petlan <mpetlan@redhat.com>

The test scripts in base_* directories currently have their own drivers
that run them. Before this patch, the shell test-suite generator causes
them to run twice. Fix that by skipping them in the generator.

A cleaner solution (for future) will be to use the directory structure
idea (introduced by Carsten Haitzler in 7391db645938 ("perf test:
Refactor shell tests allowing subdirs")) to generate test entries with
subtests, like:

$ perf test list
[...]
97: perf probe shell tests
97:1: perf probe basic functionality
97:2: perf probe tests with arguments
97:3: perf probe invalid options handling
[...]

There is already a lot of shell test scripts and many are about to come,
so there is a need for some hierarchy.

Signed-off-by: Michael Petlan <mpetlan@redhat.com>
Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
---
 tools/perf/tests/tests-scripts.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/perf/tests/tests-scripts.c b/tools/perf/tests/tests-scripts.c
index e2042b3682..01971e9974 100644
--- a/tools/perf/tests/tests-scripts.c
+++ b/tools/perf/tests/tests-scripts.c
@@ -222,6 +222,8 @@ static void append_scripts_in_dir(int dir_fd,
 			if (!S_ISDIR(st.st_mode))
 				continue;
 		}
+		if (strncmp(ent->d_name, "base_", 5) == 0)
+			continue; /* Skip scripts that have a separate driver. */
 		fd = openat(dir_fd, ent->d_name, O_PATH);
 		append_scripts_in_dir(fd, result, result_sz);
 	}
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* [PATCH v2 11/11] perf testsuite: Install perf-report tests
  2024-08-28 20:39       ` Arnaldo Carvalho de Melo
  2024-08-29 11:29         ` [PATCH v2 01/11] perf tests shell: Skip base_* dirs in test script search vmolnaro
@ 2024-08-29 11:29         ` vmolnaro
  2024-08-30 15:27         ` [PATCH v2 00/11] perftool-testsuite 2nd batch Arnaldo Carvalho de Melo
  2 siblings, 0 replies; 43+ messages in thread
From: vmolnaro @ 2024-08-29 11:29 UTC (permalink / raw)
  To: linux-perf-users, acme, acme, namhyung; +Cc: mpetlan, irogers, atrajeev

From: Michael Petlan <mpetlan@redhat.com>

Signed-off-by: Michael Petlan <mpetlan@redhat.com>
Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
---
 tools/perf/Makefile.perf | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index e6d56b5553..b921b19c39 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -1081,6 +1081,8 @@ install-tests: all install-gtk
 		$(INSTALL) tests/shell/common/*.pl '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/common'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_probe'; \
 		$(INSTALL) tests/shell/base_probe/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_probe'; \
+		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_report'; \
+		$(INSTALL) tests/shell/base_probe/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_report'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight' ; \
 		$(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
 	$(Q)$(MAKE) -C tests/shell/coresight install-tests
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-08-28 20:39       ` Arnaldo Carvalho de Melo
  2024-08-29 11:29         ` [PATCH v2 01/11] perf tests shell: Skip base_* dirs in test script search vmolnaro
  2024-08-29 11:29         ` [PATCH v2 11/11] perf testsuite: Install perf-report tests vmolnaro
@ 2024-08-30 15:27         ` Arnaldo Carvalho de Melo
  2024-09-02 14:05           ` Veronika Molnarova
  2 siblings, 1 reply; 43+ messages in thread
From: Arnaldo Carvalho de Melo @ 2024-08-30 15:27 UTC (permalink / raw)
  To: Veronika Molnarova
  Cc: linux-perf-users, acme, namhyung, mpetlan, irogers, atrajeev,
	masami.hiramatsu.pt

On Wed, Aug 28, 2024 at 05:39:48PM -0300, Arnaldo Carvalho de Melo wrote:
> On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:
> > Ping
> 
So, is there some other knob to get further output from that FAIL case?
Like the command that is failing? I'll try to take a look at the
sources, but are you seeing this problem with what is in
perf-tools-next/perf-tools-next?

- Arnaldo

root@x1:~# perf test -vvvvv 88
capget syscall failed (No such file or directory - 2) fall back on root check
 88: perftool-testsuite_probe:
--- start ---
test child forked, pid 2780578
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
-- [ FAIL ] -- perf_probe :: test_adding_blacklisted :: adding blacklisted function warn_thunk_thunk (output regexp parsing)
-- [ PASS ] -- perf_probe :: test_adding_blacklisted :: listing blacklisted probe (should NOT be listed)
## [ FAIL ] ## perf_probe :: test_adding_blacklisted SUMMARY :: 1 failures found
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: 
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: -a
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: --add
-- [ PASS ] -- perf_probe :: test_adding_kernel :: listing added probe :: perf list
-- [ PASS ] -- perf_probe :: test_adding_kernel :: listing added probe :: perf probe -l
-- [ PASS ] -- perf_probe :: test_adding_kernel :: using added probe
-- [ PASS ] -- perf_probe :: test_adding_kernel :: deleting added probe
-- [ PASS ] -- perf_probe :: test_adding_kernel :: listing removed probe (should NOT be listed)
-- [ PASS ] -- perf_probe :: test_adding_kernel :: dry run :: adding probe
-- [ PASS ] -- perf_probe :: test_adding_kernel :: force-adding probes :: first probe adding
-- [ PASS ] -- perf_probe :: test_adding_kernel :: force-adding probes :: second probe adding (without force)
-- [ PASS ] -- perf_probe :: test_adding_kernel :: force-adding probes :: second probe adding (with force)
-- [ PASS ] -- perf_probe :: test_adding_kernel :: using doubled probe
-- [ PASS ] -- perf_probe :: test_adding_kernel :: removing multiple probes
-- [ PASS ] -- perf_probe :: test_adding_kernel :: wildcard adding support
-- [ PASS ] -- perf_probe :: test_adding_kernel :: non-existing variable
-- [ PASS ] -- perf_probe :: test_adding_kernel :: function with retval :: add
-- [ PASS ] -- perf_probe :: test_adding_kernel :: function with retval :: record
-- [ PASS ] -- perf_probe :: test_adding_kernel :: function argument probing :: script
## [ PASS ] ## perf_probe :: test_adding_kernel SUMMARY
-- [ SKIP ] -- perf_probe :: test_basic :: help message :: testcase skipped
-- [ PASS ] -- perf_probe :: test_basic :: usage message
-- [ PASS ] -- perf_probe :: test_basic :: quiet switch
## [ PASS ] ## perf_probe :: test_basic SUMMARY
-- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -a
-- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -d
-- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -L
-- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -V
-- [ PASS ] -- perf_probe :: test_invalid_options :: unnecessary argument for -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: unnecessary argument for -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -d xxx
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -L foo
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -V foo
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -L foo
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -V foo
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -L foo -V bar
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -L foo -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -L foo -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -V foo -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -V foo -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -l -F
## [ PASS ] ## perf_probe :: test_invalid_options SUMMARY
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func:10
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func:0-10
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func:2+10
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func@source.c
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func@source.c:1
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: source.c:1
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: source.c:1+1
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: source.c:1-10
-- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func:foo
-- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func:1-foo
-- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func:1+foo
-- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func;lazy\*pattern
## [ PASS ] ## perf_probe :: test_line_semantics SUMMARY
---- end(-1) ----
 88: perftool-testsuite_probe                                        : FAILED!
root@x1:~# 



^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-08-30 15:27         ` [PATCH v2 00/11] perftool-testsuite 2nd batch Arnaldo Carvalho de Melo
@ 2024-09-02 14:05           ` Veronika Molnarova
  2024-09-02 14:31             ` Arnaldo Carvalho de Melo
  0 siblings, 1 reply; 43+ messages in thread
From: Veronika Molnarova @ 2024-09-02 14:05 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo
  Cc: linux-perf-users, acme, namhyung, mpetlan, irogers, atrajeev,
	masami.hiramatsu.pt



On 8/30/24 17:27, Arnaldo Carvalho de Melo wrote:
> On Wed, Aug 28, 2024 at 05:39:48PM -0300, Arnaldo Carvalho de Melo wrote:
>> On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:
>>> Ping
>>
> So, is there some other knob to get further output from that FAIL case?
> Like the command that is failing? I'll try to take a look at the
> sources, but are you seeing this problem with what is in
> perf-tools-next/perf-tools-next?
> 
> - Arnaldo

Couldn't reproduce the issue on multiple systems. There is no further way
of getting more logs from the test case. The output of the failure is from
the regex checking when a line wasn't matched to any possible regex.
Looking at the test case "adding blacklisted function warn_thunk_thunk",
the failure is caused by command 'perf probe warn_thunk_thunk', which is
a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".

If the variable "PERFTEST_KEEP_LOGS=y" is set to keep debug logs located in
"/tmp/perftool-testsuite_probe.XXX", those could provide some more info.

Would it be possible to get system specification for the possible issue
reproduction?

Thanks,
Veronika
> 
> root@x1:~# perf test -vvvvv 88
> capget syscall failed (No such file or directory - 2) fall back on root check
>  88: perftool-testsuite_probe:
> --- start ---
> test child forked, pid 2780578
> Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
> Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
> -- [ FAIL ] -- perf_probe :: test_adding_blacklisted :: adding blacklisted function warn_thunk_thunk (output regexp parsing)
> -- [ PASS ] -- perf_probe :: test_adding_blacklisted :: listing blacklisted probe (should NOT be listed)
> ## [ FAIL ] ## perf_probe :: test_adding_blacklisted SUMMARY :: 1 failures found
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: 
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: -a
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: --add
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: listing added probe :: perf list
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: listing added probe :: perf probe -l
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: using added probe
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: deleting added probe
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: listing removed probe (should NOT be listed)
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: dry run :: adding probe
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: force-adding probes :: first probe adding
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: force-adding probes :: second probe adding (without force)
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: force-adding probes :: second probe adding (with force)
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: using doubled probe
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: removing multiple probes
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: wildcard adding support
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: non-existing variable
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: function with retval :: add
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: function with retval :: record
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: function argument probing :: script
> ## [ PASS ] ## perf_probe :: test_adding_kernel SUMMARY
> -- [ SKIP ] -- perf_probe :: test_basic :: help message :: testcase skipped
> -- [ PASS ] -- perf_probe :: test_basic :: usage message
> -- [ PASS ] -- perf_probe :: test_basic :: quiet switch
> ## [ PASS ] ## perf_probe :: test_basic SUMMARY
> -- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -a
> -- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -d
> -- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -L
> -- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -V
> -- [ PASS ] -- perf_probe :: test_invalid_options :: unnecessary argument for -F
> -- [ PASS ] -- perf_probe :: test_invalid_options :: unnecessary argument for -l
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -d xxx
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -L foo
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -V foo
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -l
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -F
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -L foo
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -V foo
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -l
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -F
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -L foo -V bar
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -L foo -l
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -L foo -F
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -V foo -l
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -V foo -F
> -- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -l -F
> ## [ PASS ] ## perf_probe :: test_invalid_options SUMMARY
> -- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func
> -- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func:10
> -- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func:0-10
> -- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func:2+10
> -- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func@source.c
> -- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func@source.c:1
> -- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: source.c:1
> -- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: source.c:1+1
> -- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: source.c:1-10
> -- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func:foo
> -- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func:1-foo
> -- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func:1+foo
> -- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func;lazy\*pattern
> ## [ PASS ] ## perf_probe :: test_line_semantics SUMMARY
> ---- end(-1) ----
>  88: perftool-testsuite_probe                                        : FAILED!
> root@x1:~# 
> 
> 
> 


^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-09-02 14:05           ` Veronika Molnarova
@ 2024-09-02 14:31             ` Arnaldo Carvalho de Melo
  2024-09-02 14:41               ` Arnaldo Carvalho de Melo
  0 siblings, 1 reply; 43+ messages in thread
From: Arnaldo Carvalho de Melo @ 2024-09-02 14:31 UTC (permalink / raw)
  To: Veronika Molnarova
  Cc: linux-perf-users, acme, namhyung, mpetlan, irogers, atrajeev,
	masami.hiramatsu.pt

On Mon, Sep 02, 2024 at 04:05:04PM +0200, Veronika Molnarova wrote:
> On 8/30/24 17:27, Arnaldo Carvalho de Melo wrote:
> > On Wed, Aug 28, 2024 at 05:39:48PM -0300, Arnaldo Carvalho de Melo wrote:
> >> On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:
> >>> Ping

> > So, is there some other knob to get further output from that FAIL case?
> > Like the command that is failing? I'll try to take a look at the
> > sources, but are you seeing this problem with what is in
> > perf-tools-next/perf-tools-next?
> 
> Couldn't reproduce the issue on multiple systems. There is no further way
> of getting more logs from the test case. The output of the failure is from
> the regex checking when a line wasn't matched to any possible regex.
> Looking at the test case "adding blacklisted function warn_thunk_thunk",
> the failure is caused by command 'perf probe warn_thunk_thunk', which is
> a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".
> 
> If the variable "PERFTEST_KEEP_LOGS=y" is set to keep debug logs located in
> "/tmp/perftool-testsuite_probe.XXX", those could provide some more info.
> 
> Would it be possible to get system specification for the possible issue
> reproduction?

root@x1:~# head /etc/os-release 
NAME="Fedora Linux"
VERSION="40 (Workstation Edition)"
ID=fedora
VERSION_ID=40
VERSION_CODENAME=""
PLATFORM_ID="platform:f40"
PRETTY_NAME="Fedora Linux 40 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:40"
root@x1:~# uname -a
Linux x1 6.10.4-200.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Sun Aug 11 15:32:50 UTC 2024 x86_64 GNU/Linux
root@x1:~#

root@x1:~# perf probe warn_thunk_thunk
A function DIE doesn't have decl_line. Maybe broken DWARF?
A function DIE doesn't have decl_line. Maybe broken DWARF?
Probe point 'warn_thunk_thunk' not found.
  Error: Failed to add events.
root@x1:~# perf probe -vvv -L warn_thunk_thunk
capget syscall failed (No such file or directory - 2) fall back on root check
Looking at the vmlinux_path (8 entries long)
Using /usr/lib/debug/lib/modules/6.10.4-200.fc40.x86_64/vmlinux for symbols
Open Debuginfo file: /usr/lib/debug/.build-id/32/6993e6ef36099bc55f5d9c9d3c6cefe2c9a5fb.debug
fname: (null), lineno:0
New line range: 0 to 2147483647
Failed to get the declared file name of warn_thunk_thunk
path: (null)
Debuginfo analysis failed.
  Error: Failed to show lines. Reason: Invalid argument (Code: -22)
root@x1:~#
root@x1:~# perf probe -L warn_thunk_thunk
Debuginfo analysis failed.
  Error: Failed to show lines.
root@x1:~# perf probe icmp_rcv
Added new event:
  probe:icmp_rcv       (on icmp_rcv)

You can now use it in all perf tools, such as:

	perf record -e probe:icmp_rcv -aR sleep 1

root@x1:~# perf probe -L icmp_rcv | head
<icmp_rcv@/usr/src/debug/kernel-6.10.4/linux-6.10.4-200.fc40.x86_64/net/ipv4/icmp.c:0>
      0  int icmp_rcv(struct sk_buff *skb)
         {
      2  	enum skb_drop_reason reason = SKB_DROP_REASON_NOT_SPECIFIED;
         	struct rtable *rt = skb_rtable(skb);
         	struct net *net = dev_net(rt->dst.dev);
         	struct icmphdr *icmph;
         
         	if (!xfrm4_policy_check(NULL, XFRM_POLICY_IN, skb)) {
      8  		struct sec_path *sp = skb_sec_path(skb);
root@x1:~#

root@x1:~# grep warn_thunk_thunk /proc/kallsyms 
ffffffff97004ae0 T __pfx_warn_thunk_thunk
ffffffff97004af0 T warn_thunk_thunk
root@x1:~#

root@x1:~# rm -f /tmp/perftool-testsuite_probe.*
rm: cannot remove '/tmp/perftool-testsuite_probe.C62': Is a directory
rm: cannot remove '/tmp/perftool-testsuite_probe.cyR': Is a directory
rm: cannot remove '/tmp/perftool-testsuite_probe.XOQ': Is a directory
root@x1:~# rm -rf /tmp/perftool-testsuite_probe.*
root@x1:~# export PERFTEST_KEEP_LOGS=y
root@x1:~# perf test 88
 88: perftool-testsuite_probe                                        : FAILED!
root@x1:~#
root@x1:~# ls -la /tmp/perftool-testsuite_probe.RJh/
total 0
drwx------.  3 root root   60 Sep  2 11:24 .
drwxrwxrwt. 30 root root 1900 Sep  2 11:24 ..
drwxr-xr-x.  3 root root   80 Sep  2 11:24 perf_probe
root@x1:~# ls -la /tmp/perftool-testsuite_probe.RJh/perf_probe/
total 80
drwxr-xr-x. 3 root root    80 Sep  2 11:24 .
drwx------. 3 root root    60 Sep  2 11:24 ..
drwxr-xr-x. 2 root root   740 Sep  2 11:24 logs
-rw-------. 1 root root 80170 Sep  2 11:24 perf.data
root@x1:~# ls -la /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/
total 108
drwxr-xr-x. 2 root root  740 Sep  2 11:24 .
drwxr-xr-x. 3 root root   80 Sep  2 11:24 ..
-rw-r--r--. 1 root root  191 Sep  2 11:24 adding_blacklisted.err
-rw-r--r--. 1 root root    0 Sep  2 11:24 adding_blacklisted_list.log
-rw-r--r--. 1 root root    0 Sep  2 11:24 adding_blacklisted.log
-rw-r--r--. 1 root root  165 Sep  2 11:24 adding_kernel_add--add.err
-rw-r--r--. 1 root root  165 Sep  2 11:24 adding_kernel_add-a.err
-rw-r--r--. 1 root root  165 Sep  2 11:24 adding_kernel_add.err
-rw-r--r--. 1 root root 7280 Sep  2 11:24 adding_kernel_adding_wildcard.err
-rw-r--r--. 1 root root  165 Sep  2 11:24 adding_kernel_dryrun.err
-rw-r--r--. 1 root root  165 Sep  2 11:24 adding_kernel_forceadd_01.err
-rw-r--r--. 1 root root  214 Sep  2 11:24 adding_kernel_forceadd_02.err
-rw-r--r--. 1 root root  169 Sep  2 11:24 adding_kernel_forceadd_03.err
-rw-r--r--. 1 root root  201 Sep  2 11:24 adding_kernel_func_retval_add.err
-rw-r--r--. 1 root root  161 Sep  2 11:24 adding_kernel_func_retval_record.err
-rw-r--r--. 1 root root 1651 Sep  2 11:24 adding_kernel_func_retval_script.log
-rw-r--r--. 1 root root   58 Sep  2 11:24 adding_kernel_list-l.log
-rw-r--r--. 1 root root   72 Sep  2 11:24 adding_kernel_list.log
-rw-r--r--. 1 root root    0 Sep  2 11:24 adding_kernel_list_removed.log
-rw-r--r--. 1 root root  328 Sep  2 11:24 adding_kernel_nonexisting.err
-rw-r--r--. 1 root root   38 Sep  2 11:24 adding_kernel_removing.err
-rw-r--r--. 1 root root   78 Sep  2 11:24 adding_kernel_removing_wildcard.err
-rw-r--r--. 1 root root  294 Sep  2 11:24 adding_kernel_using_probe.log
-rw-r--r--. 1 root root  131 Sep  2 11:24 adding_kernel_using_two.log
-rw-r--r--. 1 root root  589 Sep  2 11:24 aux.log
-rw-r--r--. 1 root root    0 Sep  2 11:24 basic_quiet01.err
-rw-r--r--. 1 root root    0 Sep  2 11:24 basic_quiet01.log
-rw-r--r--. 1 root root    0 Sep  2 11:24 basic_quiet02.err
-rw-r--r--. 1 root root    0 Sep  2 11:24 basic_quiet03.log
-rw-r--r--. 1 root root 3064 Sep  2 11:24 basic_usage.log
-rw-r--r--. 1 root root  938 Sep  2 11:24 invalid_options_missing_argument-a.err
-rw-r--r--. 1 root root  485 Sep  2 11:24 invalid_options_missing_argument-d.err
-rw-r--r--. 1 root root  517 Sep  2 11:24 invalid_options_missing_argument-L.err
-rw-r--r--. 1 root root  535 Sep  2 11:24 invalid_options_missing_argument-V.err
-rw-r--r--. 1 root root  750 Sep  2 11:24 invalid_options_mutually_exclusive.log
-rw-r--r--. 1 root root    0 Sep  2 11:24 invalid_options_unnecessary_argument-F.err
-rw-r--r--. 1 root root    0 Sep  2 11:24 invalid_options_unnecessary_argument-l.err
root@x1:~#

root@x1:~# grep "decl_line" /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/*
/tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?
/tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
A function DIE doesn't have decl_line. Maybe broken DWARF?
A function DIE doesn't have decl_line. Maybe broken DWARF?
Probe point 'warn_thunk_thunk' not found.
  Error: Failed to add events.
root@x1:~#
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
root@x1:~#

So is it just a matter of adding this output to the expected outputs?
In test_adding_blacklisted.sh?

- Arnaldo

^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-09-02 14:31             ` Arnaldo Carvalho de Melo
@ 2024-09-02 14:41               ` Arnaldo Carvalho de Melo
  2024-09-02 14:42                 ` Arnaldo Carvalho de Melo
  2024-09-02 15:10                 ` Veronika Molnarova
  0 siblings, 2 replies; 43+ messages in thread
From: Arnaldo Carvalho de Melo @ 2024-09-02 14:41 UTC (permalink / raw)
  To: Veronika Molnarova
  Cc: linux-perf-users, acme, namhyung, mpetlan, irogers, atrajeev,
	masami.hiramatsu.pt

On Mon, Sep 02, 2024 at 11:31:36AM -0300, Arnaldo Carvalho de Melo wrote:
> On Mon, Sep 02, 2024 at 04:05:04PM +0200, Veronika Molnarova wrote:
> > On 8/30/24 17:27, Arnaldo Carvalho de Melo wrote:
> > > On Wed, Aug 28, 2024 at 05:39:48PM -0300, Arnaldo Carvalho de Melo wrote:
> > >> On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:
> > > So, is there some other knob to get further output from that FAIL case?
> > > Like the command that is failing? I'll try to take a look at the
> > > sources, but are you seeing this problem with what is in
> > > perf-tools-next/perf-tools-next?

> > Couldn't reproduce the issue on multiple systems. There is no further way
> > of getting more logs from the test case. The output of the failure is from
> > the regex checking when a line wasn't matched to any possible regex.
> > Looking at the test case "adding blacklisted function warn_thunk_thunk",
> > the failure is caused by command 'perf probe warn_thunk_thunk', which is
> > a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".

root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
root@x1:~#

<SNIP>

> /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?
> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
> A function DIE doesn't have decl_line. Maybe broken DWARF?
> A function DIE doesn't have decl_line. Maybe broken DWARF?
> Probe point 'warn_thunk_thunk' not found.
>   Error: Failed to add events.
> root@x1:~#
> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
> root@x1:~#
> 
> So is it just a matter of adding this output to the expected outputs?
> In test_adding_blacklisted.sh?

Did the patch below, now it passes, Ack?

root@x1:~# export PERFTEST_KEEP_LOGS=y
root@x1:~# perf test 88
 88: perftool-testsuite_probe                                        : Ok
root@x1:~#

Or am I missing the point? :-)

- Arnaldo

diff --git a/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
index b5dc10b2a73810b3..14290d08a4a96eaf 100755
--- a/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
+++ b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
@@ -42,7 +42,8 @@ REGEX_ERROR_MESSAGE="Error: Failed to add events."
 REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
 REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
 REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
-../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
+REGEX_MISSING_DECL_LINE="A function DIE doesn't have decl_line. Maybe broken DWARF?"
+../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" "$REGEX_MISSING_DECL_LINE" < $LOGS_DIR/adding_blacklisted.err
 CHECK_EXIT_CODE=$?
 
 print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"

^ permalink raw reply related	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-09-02 14:41               ` Arnaldo Carvalho de Melo
@ 2024-09-02 14:42                 ` Arnaldo Carvalho de Melo
  2024-09-02 15:10                 ` Veronika Molnarova
  1 sibling, 0 replies; 43+ messages in thread
From: Arnaldo Carvalho de Melo @ 2024-09-02 14:42 UTC (permalink / raw)
  To: Veronika Molnarova
  Cc: linux-perf-users, acme, namhyung, mpetlan, irogers, atrajeev,
	masami.hiramatsu.pt

On Mon, Sep 02, 2024 at 11:41:52AM -0300, Arnaldo Carvalho de Melo wrote:
> On Mon, Sep 02, 2024 at 11:31:36AM -0300, Arnaldo Carvalho de Melo wrote:
> > On Mon, Sep 02, 2024 at 04:05:04PM +0200, Veronika Molnarova wrote:
> > > On 8/30/24 17:27, Arnaldo Carvalho de Melo wrote:
> > > > On Wed, Aug 28, 2024 at 05:39:48PM -0300, Arnaldo Carvalho de Melo wrote:
> > > >> On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:
> > > > So, is there some other knob to get further output from that FAIL case?
> > > > Like the command that is failing? I'll try to take a look at the
> > > > sources, but are you seeing this problem with what is in
> > > > perf-tools-next/perf-tools-next?
> 
> > > Couldn't reproduce the issue on multiple systems. There is no further way
> > > of getting more logs from the test case. The output of the failure is from
> > > the regex checking when a line wasn't matched to any possible regex.
> > > Looking at the test case "adding blacklisted function warn_thunk_thunk",
> > > the failure is caused by command 'perf probe warn_thunk_thunk', which is
> > > a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".
> 
> root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
> 0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
> root@x1:~#
> 
> <SNIP>
> 
> > /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?
> > root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
> > A function DIE doesn't have decl_line. Maybe broken DWARF?
> > A function DIE doesn't have decl_line. Maybe broken DWARF?
> > Probe point 'warn_thunk_thunk' not found.
> >   Error: Failed to add events.
> > root@x1:~#
> > root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
> > root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
> > root@x1:~#
> > 
> > So is it just a matter of adding this output to the expected outputs?
> > In test_adding_blacklisted.sh?
> 
> Did the patch below, now it passes, Ack?
> 
> root@x1:~# export PERFTEST_KEEP_LOGS=y
> root@x1:~# perf test 88
>  88: perftool-testsuite_probe                                        : Ok
> root@x1:~#
> 
> Or am I missing the point? :-)

The logs continue to point out that problem:

root@x1:~# cat /tmp/perftool-testsuite_probe.Fu4/perf_probe/logs/adding_blacklisted.err 
A function DIE doesn't have decl_line. Maybe broken DWARF?
A function DIE doesn't have decl_line. Maybe broken DWARF?
Probe point 'warn_thunk_thunk' not found.
  Error: Failed to add events.
root@x1:~#
 
> - Arnaldo
> 
> diff --git a/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
> index b5dc10b2a73810b3..14290d08a4a96eaf 100755
> --- a/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
> +++ b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
> @@ -42,7 +42,8 @@ REGEX_ERROR_MESSAGE="Error: Failed to add events."
>  REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
>  REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
>  REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
> -../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
> +REGEX_MISSING_DECL_LINE="A function DIE doesn't have decl_line. Maybe broken DWARF?"
> +../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" "$REGEX_MISSING_DECL_LINE" < $LOGS_DIR/adding_blacklisted.err
>  CHECK_EXIT_CODE=$?
>  
>  print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"

^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-09-02 14:41               ` Arnaldo Carvalho de Melo
  2024-09-02 14:42                 ` Arnaldo Carvalho de Melo
@ 2024-09-02 15:10                 ` Veronika Molnarova
  2024-09-02 18:46                   ` Arnaldo Carvalho de Melo
  1 sibling, 1 reply; 43+ messages in thread
From: Veronika Molnarova @ 2024-09-02 15:10 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo
  Cc: linux-perf-users, acme, namhyung, mpetlan, irogers, atrajeev,
	masami.hiramatsu.pt



On 9/2/24 16:41, Arnaldo Carvalho de Melo wrote:
> On Mon, Sep 02, 2024 at 11:31:36AM -0300, Arnaldo Carvalho de Melo wrote:
>> On Mon, Sep 02, 2024 at 04:05:04PM +0200, Veronika Molnarova wrote:
>>> On 8/30/24 17:27, Arnaldo Carvalho de Melo wrote:
>>>> On Wed, Aug 28, 2024 at 05:39:48PM -0300, Arnaldo Carvalho de Melo wrote:
>>>>> On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:
>>>> So, is there some other knob to get further output from that FAIL case?
>>>> Like the command that is failing? I'll try to take a look at the
>>>> sources, but are you seeing this problem with what is in
>>>> perf-tools-next/perf-tools-next?
> 
>>> Couldn't reproduce the issue on multiple systems. There is no further way
>>> of getting more logs from the test case. The output of the failure is from
>>> the regex checking when a line wasn't matched to any possible regex.
>>> Looking at the test case "adding blacklisted function warn_thunk_thunk",
>>> the failure is caused by command 'perf probe warn_thunk_thunk', which is
>>> a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".
> 
> root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
> 0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
> root@x1:~#
> 
> <SNIP>
> 
>> /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?
>> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
>> A function DIE doesn't have decl_line. Maybe broken DWARF?
>> A function DIE doesn't have decl_line. Maybe broken DWARF?
>> Probe point 'warn_thunk_thunk' not found.
>>   Error: Failed to add events.
>> root@x1:~#
>> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
>> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
>> root@x1:~#
>>
>> So is it just a matter of adding this output to the expected outputs?
>> In test_adding_blacklisted.sh?
> 
> Did the patch below, now it passes, Ack?
> 
> root@x1:~# export PERFTEST_KEEP_LOGS=y
> root@x1:~# perf test 88
>  88: perftool-testsuite_probe                                        : Ok
> root@x1:~#
> 
> Or am I missing the point? :-)
> 
> - Arnaldo

Yeah, that would fix the test case, the question is whether we should
accept it as a possible output. Found some old patch that should have
fixed the mentioned issue: 
https://lore.kernel.org/all/20230628082337.1857302-3-georgmueller@gmx.net/

> 
> diff --git a/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
> index b5dc10b2a73810b3..14290d08a4a96eaf 100755
> --- a/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
> +++ b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
> @@ -42,7 +42,8 @@ REGEX_ERROR_MESSAGE="Error: Failed to add events."
>  REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
>  REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
>  REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
> -../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
> +REGEX_MISSING_DECL_LINE="A function DIE doesn't have decl_line. Maybe broken DWARF?"
> +../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" "$REGEX_MISSING_DECL_LINE" < $LOGS_DIR/adding_blacklisted.err
>  CHECK_EXIT_CODE=$?
>  
>  print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"
> 


^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-09-02 15:10                 ` Veronika Molnarova
@ 2024-09-02 18:46                   ` Arnaldo Carvalho de Melo
  2024-09-02 21:00                     ` Arnaldo Carvalho de Melo
  2024-09-05 15:11                     ` Masami Hiramatsu
  0 siblings, 2 replies; 43+ messages in thread
From: Arnaldo Carvalho de Melo @ 2024-09-02 18:46 UTC (permalink / raw)
  To: Veronika Molnarova
  Cc: linux-perf-users, acme, namhyung, mpetlan, irogers, atrajeev,
	Masami Hiramatsu

On Mon, Sep 02, 2024 at 05:10:48PM +0200, Veronika Molnarova wrote:
> 
> 
> On 9/2/24 16:41, Arnaldo Carvalho de Melo wrote:
> > On Mon, Sep 02, 2024 at 11:31:36AM -0300, Arnaldo Carvalho de Melo wrote:
> >> On Mon, Sep 02, 2024 at 04:05:04PM +0200, Veronika Molnarova wrote:
> >>> On 8/30/24 17:27, Arnaldo Carvalho de Melo wrote:
> >>>> On Wed, Aug 28, 2024 at 05:39:48PM -0300, Arnaldo Carvalho de Melo wrote:
> >>>>> On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:
> >>>> So, is there some other knob to get further output from that FAIL case?
> >>>> Like the command that is failing? I'll try to take a look at the
> >>>> sources, but are you seeing this problem with what is in
> >>>> perf-tools-next/perf-tools-next?
> > 
> >>> Couldn't reproduce the issue on multiple systems. There is no further way
> >>> of getting more logs from the test case. The output of the failure is from
> >>> the regex checking when a line wasn't matched to any possible regex.
> >>> Looking at the test case "adding blacklisted function warn_thunk_thunk",
> >>> the failure is caused by command 'perf probe warn_thunk_thunk', which is
> >>> a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".
> > 
> > root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
> > 0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
> > root@x1:~#
> > 
> > <SNIP>
> > 
> >> /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?
> >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
> >> A function DIE doesn't have decl_line. Maybe broken DWARF?
> >> A function DIE doesn't have decl_line. Maybe broken DWARF?
> >> Probe point 'warn_thunk_thunk' not found.
> >>   Error: Failed to add events.
> >> root@x1:~#
> >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
> >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
> >> root@x1:~#
> >>
> >> So is it just a matter of adding this output to the expected outputs?
> >> In test_adding_blacklisted.sh?
> > 
> > Did the patch below, now it passes, Ack?
> > 
> > root@x1:~# export PERFTEST_KEEP_LOGS=y
> > root@x1:~# perf test 88
> >  88: perftool-testsuite_probe                                        : Ok
> > root@x1:~#
> > 
> > Or am I missing the point? :-)
> > 
> > - Arnaldo
> 
> Yeah, that would fix the test case, the question is whether we should

I was lazy, but yeah, that thought crossed my mind, good thing you did
the due diligence and found that patch! Looking at it now.

> accept it as a possible output. Found some old patch that should have
> fixed the mentioned issue: 
> https://lore.kernel.org/all/20230628082337.1857302-3-georgmueller@gmx.net/

I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I
have your opinion on the patch pointed out by Veronika?

Thanks in advance!

- Arnaldo
 
> > 
> > diff --git a/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
> > index b5dc10b2a73810b3..14290d08a4a96eaf 100755
> > --- a/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
> > +++ b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
> > @@ -42,7 +42,8 @@ REGEX_ERROR_MESSAGE="Error: Failed to add events."
> >  REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
> >  REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
> >  REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
> > -../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
> > +REGEX_MISSING_DECL_LINE="A function DIE doesn't have decl_line. Maybe broken DWARF?"
> > +../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" "$REGEX_MISSING_DECL_LINE" < $LOGS_DIR/adding_blacklisted.err
> >  CHECK_EXIT_CODE=$?
> >  
> >  print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"
> > 

^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-09-02 18:46                   ` Arnaldo Carvalho de Melo
@ 2024-09-02 21:00                     ` Arnaldo Carvalho de Melo
  2024-09-05 15:11                     ` Masami Hiramatsu
  1 sibling, 0 replies; 43+ messages in thread
From: Arnaldo Carvalho de Melo @ 2024-09-02 21:00 UTC (permalink / raw)
  To: Veronika Molnarova
  Cc: linux-perf-users, acme, namhyung, mpetlan, irogers, atrajeev,
	Masami Hiramatsu

On Mon, Sep 02, 2024 at 03:46:46PM -0300, Arnaldo Carvalho de Melo wrote:
> On Mon, Sep 02, 2024 at 05:10:48PM +0200, Veronika Molnarova wrote:
> > On 9/2/24 16:41, Arnaldo Carvalho de Melo wrote:
> > > On Mon, Sep 02, 2024 at 11:31:36AM -0300, Arnaldo Carvalho de Melo wrote:
> > >> On Mon, Sep 02, 2024 at 04:05:04PM +0200, Veronika Molnarova wrote:
> > >>> On 8/30/24 17:27, Arnaldo Carvalho de Melo wrote:
> > >> So is it just a matter of adding this output to the expected outputs?
> > >> In test_adding_blacklisted.sh?

> > > Did the patch below, now it passes, Ack?

> > > root@x1:~# export PERFTEST_KEEP_LOGS=y
> > > root@x1:~# perf test 88
> > >  88: perftool-testsuite_probe                                        : Ok
> > > root@x1:~#

> > > Or am I missing the point? :-)

> > Yeah, that would fix the test case, the question is whether we should
 
> I was lazy, but yeah, that thought crossed my mind, good thing you did
> the due diligence and found that patch! Looking at it now.
 
> > accept it as a possible output. Found some old patch that should have
> > fixed the mentioned issue: 
> > https://lore.kernel.org/all/20230628082337.1857302-3-georgmueller@gmx.net/
 
> I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I
> have your opinion on the patch pointed out by Veronika?

And b4 doesn't like it, I'm trying to find it manually

⬢[acme@toolbox perf-tools-next]$ b4 am -ctsl --cc-trailers 20230628082337.1857302-3-georgmueller@gmx.net
Grabbing thread from lore.kernel.org/all/20230628082337.1857302-3-georgmueller@gmx.net/t.mbox.gz
Checking for newer revisions
Grabbing search results from lore.kernel.org
Analyzing 21 messages in the thread
WARNING: duplicate messages found at index 1
   Subject 1: tools/perf: Fix to avoid crashing if DW_AT_decl_file is NULL
   Subject 2: perf probe: add test for regression introduced by switch to die_get_decl_file
  2 is not a reply... assume additional patch
WARNING: duplicate messages found at index 2
   Subject 1: tools/perf: Fix to use dwarf_attr_integrate for generic attr accessor
   Subject 2: perf probe: add test for regression introduced by switch to die_get_decl_file
  2 is not a reply... assume additional patch
^CTraceback (most recent call last):
  File "/home/acme/git/b4/src/b4/command.py", line 428, in <module>
    cmd()
  File "/home/acme/git/b4/src/b4/command.py", line 410, in cmd

And then I noticed that his _was_ already merged, see below, so we're
back to the previous square, can you try to continue this analysis, I'm
rather busy with preparing for the upcoming conferences, so any help I
can get is appreciated :-)

- Arnaldo

commit c66e1c68c13b872505f25ab641c44b77313ee7fe
Author: Georg Müller <georgmueller@gmx.net>
Date:   Wed Jun 28 10:45:51 2023 +0200

    perf probe: Read DWARF files from the correct CU
    
    After switching from dwarf_decl_file() to die_get_decl_file(), it is not
    possible to add probes for certain functions:
    
      $ perf probe -x /usr/lib/systemd/systemd-logind match_unit_removed
      A function DIE doesn't have decl_line. Maybe broken DWARF?
      A function DIE doesn't have decl_line. Maybe broken DWARF?
      Probe point 'match_unit_removed' not found.
         Error: Failed to add events.
    
    The problem is that die_get_decl_file() uses the wrong CU to search for
    the file. elfutils commit e1db5cdc9f has some good explanation for this:
    
        dwarf_decl_file uses dwarf_attr_integrate to get the DW_AT_decl_file
        attribute. This means the attribute might come from a different DIE
        in a different CU. If so, we need to use the CU associated with the
        attribute, not the original DIE, to resolve the file name.
    
    This patch uses the same source of information as elfutils: use attribute
    DW_AT_decl_file and use this CU to search for the file.
    
    Fixes: dc9a5d2ccd5c823c ("perf probe: Fix to get declared file name from clang DWARF5")
    Signed-off-by: Georg Müller <georgmueller@gmx.net>
    Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
    Cc: Adrian Hunter <adrian.hunter@intel.com>
    Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
    Cc: Ian Rogers <irogers@google.com>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Jiri Olsa <jolsa@kernel.org>
    Cc: Mark Rutland <mark.rutland@arm.com>
    Cc: Namhyung Kim <namhyung@kernel.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: regressions@lists.linux.dev
    Cc: stable@vger.kernel.org
    Link: https://lore.kernel.org/r/20230628084551.1860532-6-georgmueller@gmx.net
    Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>


^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-09-02 18:46                   ` Arnaldo Carvalho de Melo
  2024-09-02 21:00                     ` Arnaldo Carvalho de Melo
@ 2024-09-05 15:11                     ` Masami Hiramatsu
  2024-09-05 19:04                       ` Arnaldo Carvalho de Melo
  1 sibling, 1 reply; 43+ messages in thread
From: Masami Hiramatsu @ 2024-09-05 15:11 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo
  Cc: Veronika Molnarova, linux-perf-users, acme, namhyung, mpetlan,
	irogers, atrajeev, Masami Hiramatsu

On Mon, 2 Sep 2024 15:46:42 -0300
Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> On Mon, Sep 02, 2024 at 05:10:48PM +0200, Veronika Molnarova wrote:
> > 
> > 
> > On 9/2/24 16:41, Arnaldo Carvalho de Melo wrote:
> > > On Mon, Sep 02, 2024 at 11:31:36AM -0300, Arnaldo Carvalho de Melo wrote:
> > >> On Mon, Sep 02, 2024 at 04:05:04PM +0200, Veronika Molnarova wrote:
> > >>> On 8/30/24 17:27, Arnaldo Carvalho de Melo wrote:
> > >>>> On Wed, Aug 28, 2024 at 05:39:48PM -0300, Arnaldo Carvalho de Melo wrote:
> > >>>>> On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:
> > >>>> So, is there some other knob to get further output from that FAIL case?
> > >>>> Like the command that is failing? I'll try to take a look at the
> > >>>> sources, but are you seeing this problem with what is in
> > >>>> perf-tools-next/perf-tools-next?
> > > 
> > >>> Couldn't reproduce the issue on multiple systems. There is no further way
> > >>> of getting more logs from the test case. The output of the failure is from
> > >>> the regex checking when a line wasn't matched to any possible regex.
> > >>> Looking at the test case "adding blacklisted function warn_thunk_thunk",
> > >>> the failure is caused by command 'perf probe warn_thunk_thunk', which is
> > >>> a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".
> > > 
> > > root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
> > > 0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
> > > root@x1:~#
> > > 

right, thunk functions should not be probed.

> > > <SNIP>
> > > 
> > >> /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?
> > >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
> > >> A function DIE doesn't have decl_line. Maybe broken DWARF?
> > >> A function DIE doesn't have decl_line. Maybe broken DWARF?
> > >> Probe point 'warn_thunk_thunk' not found.

Yeah, thunk functions are defined by asm code. No DWARF is available.

> > >>   Error: Failed to add events.
> > >> root@x1:~#
> > >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
> > >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
> > >> root@x1:~#
> > >>
> > >> So is it just a matter of adding this output to the expected outputs?
> > >> In test_adding_blacklisted.sh?
> > > 
> > > Did the patch below, now it passes, Ack?
> > > 
> > > root@x1:~# export PERFTEST_KEEP_LOGS=y
> > > root@x1:~# perf test 88
> > >  88: perftool-testsuite_probe                                        : Ok
> > > root@x1:~#
> > > 
> > > Or am I missing the point? :-)
> > > 
> > > - Arnaldo
> > 
> > Yeah, that would fix the test case, the question is whether we should
> 
> I was lazy, but yeah, that thought crossed my mind, good thing you did
> the due diligence and found that patch! Looking at it now.
> 
> > accept it as a possible output. Found some old patch that should have
> > fixed the mentioned issue: 
> > https://lore.kernel.org/all/20230628082337.1857302-3-georgmueller@gmx.net/
> 
> I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I
> have your opinion on the patch pointed out by Veronika?

Thanks for update my email. Yeah, it looks good to me.

Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>

Thanks,

> 
> Thanks in advance!
> 
> - Arnaldo
>  
> > > 
> > > diff --git a/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
> > > index b5dc10b2a73810b3..14290d08a4a96eaf 100755
> > > --- a/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
> > > +++ b/tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
> > > @@ -42,7 +42,8 @@ REGEX_ERROR_MESSAGE="Error: Failed to add events."
> > >  REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
> > >  REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
> > >  REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
> > > -../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
> > > +REGEX_MISSING_DECL_LINE="A function DIE doesn't have decl_line. Maybe broken DWARF?"
> > > +../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" "$REGEX_MISSING_DECL_LINE" < $LOGS_DIR/adding_blacklisted.err
> > >  CHECK_EXIT_CODE=$?
> > >  
> > >  print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"
> > > 


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-09-05 15:11                     ` Masami Hiramatsu
@ 2024-09-05 19:04                       ` Arnaldo Carvalho de Melo
  2024-09-05 19:15                         ` Arnaldo Carvalho de Melo
  0 siblings, 1 reply; 43+ messages in thread
From: Arnaldo Carvalho de Melo @ 2024-09-05 19:04 UTC (permalink / raw)
  To: Masami Hiramatsu
  Cc: Veronika Molnarova, linux-perf-users, acme, namhyung, mpetlan,
	irogers, atrajeev

On Fri, Sep 06, 2024 at 12:11:04AM +0900, Masami Hiramatsu wrote:
> On Mon, 2 Sep 2024 15:46:42 -0300
> Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > On Mon, Sep 02, 2024 at 05:10:48PM +0200, Veronika Molnarova wrote:
> > > On 9/2/24 16:41, Arnaldo Carvalho de Melo wrote:
> > > > On Mon, Sep 02, 2024 at 11:31:36AM -0300, Arnaldo Carvalho de Melo wrote:
> > > >> On Mon, Sep 02, 2024 at 04:05:04PM +0200, Veronika Molnarova wrote:
> > > >>> On 8/30/24 17:27, Arnaldo Carvalho de Melo wrote:
> > > >>>> On Wed, Aug 28, 2024 at 05:39:48PM -0300, Arnaldo Carvalho de Melo wrote:
> > > >>>>> On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:
> > > >>>> So, is there some other knob to get further output from that FAIL case?
> > > >>>> Like the command that is failing? I'll try to take a look at the
> > > >>>> sources, but are you seeing this problem with what is in
> > > >>>> perf-tools-next/perf-tools-next?

> > > >>> Couldn't reproduce the issue on multiple systems. There is no further way
> > > >>> of getting more logs from the test case. The output of the failure is from
> > > >>> the regex checking when a line wasn't matched to any possible regex.
> > > >>> Looking at the test case "adding blacklisted function warn_thunk_thunk",
> > > >>> the failure is caused by command 'perf probe warn_thunk_thunk', which is
> > > >>> a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".
> > > > 
> > > > root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
> > > > 0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
> > > > root@x1:~#
> 
> right, thunk functions should not be probed.
> 
> > > > <SNIP>
> > > > 
> > > >> /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?
> > > >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
> > > >> A function DIE doesn't have decl_line. Maybe broken DWARF?
> > > >> A function DIE doesn't have decl_line. Maybe broken DWARF?
> > > >> Probe point 'warn_thunk_thunk' not found.
 
> Yeah, thunk functions are defined by asm code. No DWARF is available.

Sure, but then, how would that be detected by the test?

To recap, the
./tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh test
does:

# skip if not supported
BLACKFUNC=`head -n 1 /sys/kernel/debug/kprobes/blacklist 2> /dev/null | cut -f2`
if [ -z "$BLACKFUNC" ]; then
        print_overall_skipped
        exit 0
fi

# remove all previously added probes
clear_all_probes


### adding blacklisted function

# functions from blacklist should be skipped by perf probe
! $CMD_PERF probe $BLACKFUNC > $LOGS_DIR/adding_blacklisted.log 2> $LOGS_DIR/adding_blacklisted.err
PERF_EXIT_CODE=$?

REGEX_SCOPE_FAIL="Failed to find scope of probe point"
REGEX_SKIP_MESSAGE=" is blacklisted function, skip it\."
REGEX_NOT_FOUND_MESSAGE="Probe point \'$BLACKFUNC\' not found."
REGEX_ERROR_MESSAGE="Error: Failed to add events."
REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
CHECK_EXIT_CODE=$?


So it _tries_ to probe blacklisted functions and check what is that the
tooling does when asked to probe things it shouldn't probe.

So, I proposed that if we get that error message (A function DIE doesn't
have decl_line. Maybe broken DWARF?) we should skip it, but Veronika did
the right thing and asked if that wasn't really papering over some
problem and pointed to a patch that addressed a problem that lead to
that exact same message being produced, but that didn't help with our
current case, with another blacklisted function, warn_thunk_thunk.

You mentioned that this is an asm function, so no DWARF, let alone
correct DWARF, so the message is kinda ok, but we can't use that message
as it could paper over real problems like the one fixed by the patch
that Veronika pointed out.

How to figure out if a function has valid DWARF? Maybe for the moment we
should have _another_ blacklist, one for functions in the blacklist that
have no valid DWARF? A blacklist's blacklist?

Turtles all the way down? :-)

- Arnaldo
 
> > > >>   Error: Failed to add events.
> > > >> root@x1:~#
> > > >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
> > > >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
> > > >> root@x1:~#
> > > >>
> > > >> So is it just a matter of adding this output to the expected outputs?
> > > >> In test_adding_blacklisted.sh?
> > > > 
> > > > Did the patch below, now it passes, Ack?
> > > > 
> > > > root@x1:~# export PERFTEST_KEEP_LOGS=y
> > > > root@x1:~# perf test 88
> > > >  88: perftool-testsuite_probe                                        : Ok
> > > > root@x1:~#
> > > > 
> > > > Or am I missing the point? :-)

> > > Yeah, that would fix the test case, the question is whether we should

> > I was lazy, but yeah, that thought crossed my mind, good thing you did
> > the due diligence and found that patch! Looking at it now.

> > > accept it as a possible output. Found some old patch that should have
> > > fixed the mentioned issue: 
> > > https://lore.kernel.org/all/20230628082337.1857302-3-georgmueller@gmx.net/

> > I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I
> > have your opinion on the patch pointed out by Veronika?
 
> Thanks for update my email. Yeah, it looks good to me.
 
> Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>

Unfortunately that cset is already there:

commit c66e1c68c13b872505f25ab641c44b77313ee7fe
Author: Georg Müller <georgmueller@gmx.net>
Date:   Wed Jun 28 10:45:51 2023 +0200

    perf probe: Read DWARF files from the correct CU
    
    After switching from dwarf_decl_file() to die_get_decl_file(), it is not
    possible to add probes for certain functions:
    
      $ perf probe -x /usr/lib/systemd/systemd-logind match_unit_removed
      A function DIE doesn't have decl_line. Maybe broken DWARF?
      A function DIE doesn't have decl_line. Maybe broken DWARF?
      Probe point 'match_unit_removed' not found.
         Error: Failed to add events.


⬢[acme@toolbox perf-tools-next]$ git tag --contains c66e1c68c13b872505f25ab641c44b77313ee7fe | grep ^v6.1
v6.10
v6.10-rc1
v6.10-rc2
v6.10-rc3
v6.10-rc4
v6.10-rc5
v6.10-rc6
v6.10-rc7
v6.11-rc1
v6.11-rc2
v6.11-rc3
v6.11-rc4
v6.11-rc5
v6.11-rc6
⬢[acme@toolbox perf-tools-next]$

But..

root@number:~# perf test -vvvvv testsuite_probe |& head
capget syscall failed (No such file or directory - 2) fall back on root check
 88: perftool-testsuite_probe:
--- start ---
test child forked, pid 511749
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
-- [ FAIL ] -- perf_probe :: test_adding_blacklisted :: adding blacklisted function warn_thunk_thunk (output regexp parsing)
-- [ PASS ] -- perf_probe :: test_adding_blacklisted :: listing blacklisted probe (should NOT be listed)
## [ FAIL ] ## perf_probe :: test_adding_blacklisted SUMMARY :: 1 failures found
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: 
root@number:~# perf -v
perf version 6.11.rc6.g69e90b02e611
root@number:~#

^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-09-05 19:04                       ` Arnaldo Carvalho de Melo
@ 2024-09-05 19:15                         ` Arnaldo Carvalho de Melo
  2024-09-12 13:07                           ` Arnaldo Carvalho de Melo
  0 siblings, 1 reply; 43+ messages in thread
From: Arnaldo Carvalho de Melo @ 2024-09-05 19:15 UTC (permalink / raw)
  To: Masami Hiramatsu
  Cc: Veronika Molnarova, linux-perf-users, acme, namhyung, mpetlan,
	irogers, atrajeev

On Thu, Sep 05, 2024 at 04:04:29PM -0300, Arnaldo Carvalho de Melo wrote:
> On Fri, Sep 06, 2024 at 12:11:04AM +0900, Masami Hiramatsu wrote:
> > On Mon, 2 Sep 2024 15:46:42 -0300
> > Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > > On Mon, Sep 02, 2024 at 05:10:48PM +0200, Veronika Molnarova wrote:
> > > > On 9/2/24 16:41, Arnaldo Carvalho de Melo wrote:
> > > > > On Mon, Sep 02, 2024 at 11:31:36AM -0300, Arnaldo Carvalho de Melo wrote:
> > > > >> On Mon, Sep 02, 2024 at 04:05:04PM +0200, Veronika Molnarova wrote:
> > > > >>> On 8/30/24 17:27, Arnaldo Carvalho de Melo wrote:
> > > > >>>> On Wed, Aug 28, 2024 at 05:39:48PM -0300, Arnaldo Carvalho de Melo wrote:
> > > > >>>>> On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:
> > > > >>>> So, is there some other knob to get further output from that FAIL case?
> > > > >>>> Like the command that is failing? I'll try to take a look at the
> > > > >>>> sources, but are you seeing this problem with what is in
> > > > >>>> perf-tools-next/perf-tools-next?
> 
> > > > >>> Couldn't reproduce the issue on multiple systems. There is no further way
> > > > >>> of getting more logs from the test case. The output of the failure is from
> > > > >>> the regex checking when a line wasn't matched to any possible regex.
> > > > >>> Looking at the test case "adding blacklisted function warn_thunk_thunk",
> > > > >>> the failure is caused by command 'perf probe warn_thunk_thunk', which is
> > > > >>> a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".
> > > > > 
> > > > > root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
> > > > > 0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
> > > > > root@x1:~#
> > 
> > right, thunk functions should not be probed.
> > 
> > > > > <SNIP>
> > > > > 
> > > > >> /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?
> > > > >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
> > > > >> A function DIE doesn't have decl_line. Maybe broken DWARF?
> > > > >> A function DIE doesn't have decl_line. Maybe broken DWARF?
> > > > >> Probe point 'warn_thunk_thunk' not found.
>  
> > Yeah, thunk functions are defined by asm code. No DWARF is available.
> 
> Sure, but then, how would that be detected by the test?
> 
> To recap, the
> ./tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh test
> does:
> 
> # skip if not supported
> BLACKFUNC=`head -n 1 /sys/kernel/debug/kprobes/blacklist 2> /dev/null | cut -f2`
> if [ -z "$BLACKFUNC" ]; then
>         print_overall_skipped
>         exit 0
> fi
> 
> # remove all previously added probes
> clear_all_probes
> 
> 
> ### adding blacklisted function
> 
> # functions from blacklist should be skipped by perf probe
> ! $CMD_PERF probe $BLACKFUNC > $LOGS_DIR/adding_blacklisted.log 2> $LOGS_DIR/adding_blacklisted.err
> PERF_EXIT_CODE=$?
> 
> REGEX_SCOPE_FAIL="Failed to find scope of probe point"
> REGEX_SKIP_MESSAGE=" is blacklisted function, skip it\."
> REGEX_NOT_FOUND_MESSAGE="Probe point \'$BLACKFUNC\' not found."
> REGEX_ERROR_MESSAGE="Error: Failed to add events."
> REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
> REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
> REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
> ../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
> CHECK_EXIT_CODE=$?
> 
> 
> So it _tries_ to probe blacklisted functions and check what is that the
> tooling does when asked to probe things it shouldn't probe.
> 
> So, I proposed that if we get that error message (A function DIE doesn't
> have decl_line. Maybe broken DWARF?) we should skip it, but Veronika did
> the right thing and asked if that wasn't really papering over some
> problem and pointed to a patch that addressed a problem that lead to
> that exact same message being produced, but that didn't help with our
> current case, with another blacklisted function, warn_thunk_thunk.
> 
> You mentioned that this is an asm function, so no DWARF, let alone
> correct DWARF, so the message is kinda ok, but we can't use that message
> as it could paper over real problems like the one fixed by the patch
> that Veronika pointed out.
> 
> How to figure out if a function has valid DWARF? Maybe for the moment we
> should have _another_ blacklist, one for functions in the blacklist that
> have no valid DWARF? A blacklist's blacklist?
> 
> Turtles all the way down? :-)

We have the needed info to provide a more informative warning:

 <1><da258>: Abbrev Number: 0
  Compilation Unit @ offset 0xda259:
   Length:        0x3f (32-bit)
   Version:       5
   Unit Type:     DW_UT_compile (1)
   Abbrev Offset: 0x74c8
   Pointer Size:  8
 <0><da265>: Abbrev Number: 1 (DW_TAG_compile_unit)
    <da266>   DW_AT_stmt_list   : 0x12cbd
    <da26a>   DW_AT_ranges      : 0x2d6e
    <da26e>   DW_AT_name        : (indirect string, offset: 0x1ec2f): /home/acme/git/linux/arch/x86/entry/entry.S
    <da272>   DW_AT_comp_dir    : (indirect string, offset: 0x1be93): /home/acme/git/build/v6.11-rc5+
    <da276>   DW_AT_producer    : (indirect string, offset: 0x1beb3): GNU AS 2.41
    <da27a>   DW_AT_language    : 32769 (MIPS assembler)
 <1><da27c>: Abbrev Number: 2 (DW_TAG_subprogram)
    <da27d>   DW_AT_name        : (indirect string, offset: 0x1ec5b): entry_ibpb
    <da281>   DW_AT_external    : 1
    <da281>   DW_AT_type        : <0xda29a>
    <da282>   DW_AT_low_pc      : 0xffffffff82062790
    <da28a>   DW_AT_high_pc     : 23
 <1><da28b>: Abbrev Number: 2 (DW_TAG_subprogram)
    <da28c>   DW_AT_name        : (indirect string, offset: 0x1ec66): warn_thunk_thunk
    <da290>   DW_AT_external    : 1
    <da290>   DW_AT_type        : <0xda29a>
    <da291>   DW_AT_low_pc      : 0xffffffff81008060
    <da299>   DW_AT_high_pc     : 45
 <1><da29a>: Abbrev Number: 3 (DW_TAG_unspecified_type)

root@number:~# readelf -wi `pahole --running_kernel_vmlinux` | grep warn_thunk_thunk
    <da28c>   DW_AT_name        : (indirect string, offset: 0x1ec66): warn_thunk_thunk
root@number:~# pahole --running_kernel_vmlinux
/lib/modules/6.11.0-rc5+/build/vmlinux
root@number:~# perf buildid-list -k
4dd7f9c4507b82a5bf90671d524e5bb308104ffa
root@number:~# file /lib/modules/6.11.0-rc5+/build/vmlinux
/lib/modules/6.11.0-rc5+/build/vmlinux: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=4dd7f9c4507b82a5bf90671d524e5bb308104ffa, with debug_info, not stripped
root@number:~# perf buildid-list -i /lib/modules/6.11.0-rc5+/build/vmlinux
4dd7f9c4507b82a5bf90671d524e5bb308104ffa
root@number:~#

- Arnaldo

> 
> - Arnaldo
>  
> > > > >>   Error: Failed to add events.
> > > > >> root@x1:~#
> > > > >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
> > > > >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
> > > > >> root@x1:~#
> > > > >>
> > > > >> So is it just a matter of adding this output to the expected outputs?
> > > > >> In test_adding_blacklisted.sh?
> > > > > 
> > > > > Did the patch below, now it passes, Ack?
> > > > > 
> > > > > root@x1:~# export PERFTEST_KEEP_LOGS=y
> > > > > root@x1:~# perf test 88
> > > > >  88: perftool-testsuite_probe                                        : Ok
> > > > > root@x1:~#
> > > > > 
> > > > > Or am I missing the point? :-)
> 
> > > > Yeah, that would fix the test case, the question is whether we should
> 
> > > I was lazy, but yeah, that thought crossed my mind, good thing you did
> > > the due diligence and found that patch! Looking at it now.
> 
> > > > accept it as a possible output. Found some old patch that should have
> > > > fixed the mentioned issue: 
> > > > https://lore.kernel.org/all/20230628082337.1857302-3-georgmueller@gmx.net/
> 
> > > I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I
> > > have your opinion on the patch pointed out by Veronika?
>  
> > Thanks for update my email. Yeah, it looks good to me.
>  
> > Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
> 
> Unfortunately that cset is already there:
> 
> commit c66e1c68c13b872505f25ab641c44b77313ee7fe
> Author: Georg Müller <georgmueller@gmx.net>
> Date:   Wed Jun 28 10:45:51 2023 +0200
> 
>     perf probe: Read DWARF files from the correct CU
>     
>     After switching from dwarf_decl_file() to die_get_decl_file(), it is not
>     possible to add probes for certain functions:
>     
>       $ perf probe -x /usr/lib/systemd/systemd-logind match_unit_removed
>       A function DIE doesn't have decl_line. Maybe broken DWARF?
>       A function DIE doesn't have decl_line. Maybe broken DWARF?
>       Probe point 'match_unit_removed' not found.
>          Error: Failed to add events.

 
> 
> ⬢[acme@toolbox perf-tools-next]$ git tag --contains c66e1c68c13b872505f25ab641c44b77313ee7fe | grep ^v6.1
> v6.10
> v6.10-rc1
> v6.10-rc2
> v6.10-rc3
> v6.10-rc4
> v6.10-rc5
> v6.10-rc6
> v6.10-rc7
> v6.11-rc1
> v6.11-rc2
> v6.11-rc3
> v6.11-rc4
> v6.11-rc5
> v6.11-rc6
> ⬢[acme@toolbox perf-tools-next]$
> 
> But..
> 
> root@number:~# perf test -vvvvv testsuite_probe |& head
> capget syscall failed (No such file or directory - 2) fall back on root check
>  88: perftool-testsuite_probe:
> --- start ---
> test child forked, pid 511749
> Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
> Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
> -- [ FAIL ] -- perf_probe :: test_adding_blacklisted :: adding blacklisted function warn_thunk_thunk (output regexp parsing)
> -- [ PASS ] -- perf_probe :: test_adding_blacklisted :: listing blacklisted probe (should NOT be listed)
> ## [ FAIL ] ## perf_probe :: test_adding_blacklisted SUMMARY :: 1 failures found
> -- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: 
> root@number:~# perf -v
> perf version 6.11.rc6.g69e90b02e611
> root@number:~#

^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-09-05 19:15                         ` Arnaldo Carvalho de Melo
@ 2024-09-12 13:07                           ` Arnaldo Carvalho de Melo
  2024-09-19 13:14                             ` Veronika Molnarova
  0 siblings, 1 reply; 43+ messages in thread
From: Arnaldo Carvalho de Melo @ 2024-09-12 13:07 UTC (permalink / raw)
  To: Veronika Molnarova
  Cc: Masami Hiramatsu, linux-perf-users, acme, namhyung, mpetlan,
	irogers, atrajeev

On Thu, Sep 05, 2024 at 04:15:28PM -0300, Arnaldo Carvalho de Melo wrote:
> On Thu, Sep 05, 2024 at 04:04:29PM -0300, Arnaldo Carvalho de Melo wrote:
> > So, I proposed that if we get that error message (A function DIE doesn't
> > have decl_line. Maybe broken DWARF?) we should skip it, but Veronika did
> > the right thing and asked if that wasn't really papering over some
> > problem and pointed to a patch that addressed a problem that lead to
> > that exact same message being produced, but that didn't help with our
> > current case, with another blacklisted function, warn_thunk_thunk.

> > You mentioned that this is an asm function, so no DWARF, let alone
> > correct DWARF, so the message is kinda ok, but we can't use that message
> > as it could paper over real problems like the one fixed by the patch
> > that Veronika pointed out.

> > How to figure out if a function has valid DWARF? Maybe for the moment we
> > should have _another_ blacklist, one for functions in the blacklist that
> > have no valid DWARF? A blacklist's blacklist?
 
> We have the needed info to provide a more informative warning:

So, perhaps we should, in the test, when we get that "broken dwarf"
output, look at the debuginfo file for the kernel to see if the function
is one in a DW_UT_compile (CU) that has 

     <da27a>   DW_AT_language    : 32769 (MIPS assembler)

And then just have a -vv output stating that that function is being
skipped because it is DWARF from assembler?

Veronika, would you be willing to try to do that?

Thanks,

- Arnaldo
 
>  <1><da258>: Abbrev Number: 0
>   Compilation Unit @ offset 0xda259:
>    Length:        0x3f (32-bit)
>    Version:       5
>    Unit Type:     DW_UT_compile (1)
>    Abbrev Offset: 0x74c8
>    Pointer Size:  8
>  <0><da265>: Abbrev Number: 1 (DW_TAG_compile_unit)
>     <da266>   DW_AT_stmt_list   : 0x12cbd
>     <da26a>   DW_AT_ranges      : 0x2d6e
>     <da26e>   DW_AT_name        : (indirect string, offset: 0x1ec2f): /home/acme/git/linux/arch/x86/entry/entry.S
>     <da272>   DW_AT_comp_dir    : (indirect string, offset: 0x1be93): /home/acme/git/build/v6.11-rc5+
>     <da276>   DW_AT_producer    : (indirect string, offset: 0x1beb3): GNU AS 2.41
>     <da27a>   DW_AT_language    : 32769 (MIPS assembler)
>  <1><da27c>: Abbrev Number: 2 (DW_TAG_subprogram)
>     <da27d>   DW_AT_name        : (indirect string, offset: 0x1ec5b): entry_ibpb
>     <da281>   DW_AT_external    : 1
>     <da281>   DW_AT_type        : <0xda29a>
>     <da282>   DW_AT_low_pc      : 0xffffffff82062790
>     <da28a>   DW_AT_high_pc     : 23
>  <1><da28b>: Abbrev Number: 2 (DW_TAG_subprogram)
>     <da28c>   DW_AT_name        : (indirect string, offset: 0x1ec66): warn_thunk_thunk
>     <da290>   DW_AT_external    : 1
>     <da290>   DW_AT_type        : <0xda29a>
>     <da291>   DW_AT_low_pc      : 0xffffffff81008060
>     <da299>   DW_AT_high_pc     : 45
>  <1><da29a>: Abbrev Number: 3 (DW_TAG_unspecified_type)
> 
> root@number:~# readelf -wi `pahole --running_kernel_vmlinux` | grep warn_thunk_thunk
>     <da28c>   DW_AT_name        : (indirect string, offset: 0x1ec66): warn_thunk_thunk
> root@number:~# pahole --running_kernel_vmlinux
> /lib/modules/6.11.0-rc5+/build/vmlinux
> root@number:~# perf buildid-list -k
> 4dd7f9c4507b82a5bf90671d524e5bb308104ffa
> root@number:~# file /lib/modules/6.11.0-rc5+/build/vmlinux
> /lib/modules/6.11.0-rc5+/build/vmlinux: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=4dd7f9c4507b82a5bf90671d524e5bb308104ffa, with debug_info, not stripped
> root@number:~# perf buildid-list -i /lib/modules/6.11.0-rc5+/build/vmlinux
> 4dd7f9c4507b82a5bf90671d524e5bb308104ffa
> root@number:~#
> 
> - Arnaldo
> 
> > 
> > - Arnaldo
> >  
> > > > > >>   Error: Failed to add events.
> > > > > >> root@x1:~#
> > > > > >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
> > > > > >> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
> > > > > >> root@x1:~#
> > > > > >>
> > > > > >> So is it just a matter of adding this output to the expected outputs?
> > > > > >> In test_adding_blacklisted.sh?
> > > > > > 
> > > > > > Did the patch below, now it passes, Ack?
> > > > > > 
> > > > > > root@x1:~# export PERFTEST_KEEP_LOGS=y
> > > > > > root@x1:~# perf test 88
> > > > > >  88: perftool-testsuite_probe                                        : Ok
> > > > > > root@x1:~#
> > > > > > 
> > > > > > Or am I missing the point? :-)
> > 
> > > > > Yeah, that would fix the test case, the question is whether we should
> > 
> > > > I was lazy, but yeah, that thought crossed my mind, good thing you did
> > > > the due diligence and found that patch! Looking at it now.
> > 
> > > > > accept it as a possible output. Found some old patch that should have
> > > > > fixed the mentioned issue: 
> > > > > https://lore.kernel.org/all/20230628082337.1857302-3-georgmueller@gmx.net/
> > 
> > > > I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I
> > > > have your opinion on the patch pointed out by Veronika?
> >  
> > > Thanks for update my email. Yeah, it looks good to me.
> >  
> > > Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
> > 
> > Unfortunately that cset is already there:
> > 
> > commit c66e1c68c13b872505f25ab641c44b77313ee7fe
> > Author: Georg Müller <georgmueller@gmx.net>
> > Date:   Wed Jun 28 10:45:51 2023 +0200
> > 
> >     perf probe: Read DWARF files from the correct CU
> >     
> >     After switching from dwarf_decl_file() to die_get_decl_file(), it is not
> >     possible to add probes for certain functions:
> >     
> >       $ perf probe -x /usr/lib/systemd/systemd-logind match_unit_removed
> >       A function DIE doesn't have decl_line. Maybe broken DWARF?
> >       A function DIE doesn't have decl_line. Maybe broken DWARF?
> >       Probe point 'match_unit_removed' not found.
> >          Error: Failed to add events.
> 
>  
> > 
> > ⬢[acme@toolbox perf-tools-next]$ git tag --contains c66e1c68c13b872505f25ab641c44b77313ee7fe | grep ^v6.1
> > v6.10
> > v6.10-rc1
> > v6.10-rc2
> > v6.10-rc3
> > v6.10-rc4
> > v6.10-rc5
> > v6.10-rc6
> > v6.10-rc7
> > v6.11-rc1
> > v6.11-rc2
> > v6.11-rc3
> > v6.11-rc4
> > v6.11-rc5
> > v6.11-rc6
> > ⬢[acme@toolbox perf-tools-next]$
> > 
> > But..
> > 
> > root@number:~# perf test -vvvvv testsuite_probe |& head
> > capget syscall failed (No such file or directory - 2) fall back on root check
> >  88: perftool-testsuite_probe:
> > --- start ---
> > test child forked, pid 511749
> > Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
> > Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
> > -- [ FAIL ] -- perf_probe :: test_adding_blacklisted :: adding blacklisted function warn_thunk_thunk (output regexp parsing)
> > -- [ PASS ] -- perf_probe :: test_adding_blacklisted :: listing blacklisted probe (should NOT be listed)
> > ## [ FAIL ] ## perf_probe :: test_adding_blacklisted SUMMARY :: 1 failures found
> > -- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: 
> > root@number:~# perf -v
> > perf version 6.11.rc6.g69e90b02e611
> > root@number:~#

^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-09-12 13:07                           ` Arnaldo Carvalho de Melo
@ 2024-09-19 13:14                             ` Veronika Molnarova
  2024-09-26 22:33                               ` Namhyung Kim
  0 siblings, 1 reply; 43+ messages in thread
From: Veronika Molnarova @ 2024-09-19 13:14 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo
  Cc: Masami Hiramatsu, linux-perf-users, acme, namhyung, mpetlan,
	irogers, atrajeev



On 9/12/24 15:07, Arnaldo Carvalho de Melo wrote:
> On Thu, Sep 05, 2024 at 04:15:28PM -0300, Arnaldo Carvalho de Melo wrote:
>> On Thu, Sep 05, 2024 at 04:04:29PM -0300, Arnaldo Carvalho de Melo wrote:
>>> So, I proposed that if we get that error message (A function DIE doesn't
>>> have decl_line. Maybe broken DWARF?) we should skip it, but Veronika did
>>> the right thing and asked if that wasn't really papering over some
>>> problem and pointed to a patch that addressed a problem that lead to
>>> that exact same message being produced, but that didn't help with our
>>> current case, with another blacklisted function, warn_thunk_thunk.
> 
>>> You mentioned that this is an asm function, so no DWARF, let alone
>>> correct DWARF, so the message is kinda ok, but we can't use that message
>>> as it could paper over real problems like the one fixed by the patch
>>> that Veronika pointed out.
> 
>>> How to figure out if a function has valid DWARF? Maybe for the moment we
>>> should have _another_ blacklist, one for functions in the blacklist that
>>> have no valid DWARF? A blacklist's blacklist?
>  
>> We have the needed info to provide a more informative warning:
> 
> So, perhaps we should, in the test, when we get that "broken dwarf"
> output, look at the debuginfo file for the kernel to see if the function
> is one in a DW_UT_compile (CU) that has 
> 
>      <da27a>   DW_AT_language    : 32769 (MIPS assembler)
> 
> And then just have a -vv output stating that that function is being
> skipped because it is DWARF from assembler?
> 
> Veronika, would you be willing to try to do that?

Okay, I can try to parse the debuginfo to check the DWARF.

Thanks,
Veronika
> 
> Thanks,
> 
> - Arnaldo
>  
>>  <1><da258>: Abbrev Number: 0
>>   Compilation Unit @ offset 0xda259:
>>    Length:        0x3f (32-bit)
>>    Version:       5
>>    Unit Type:     DW_UT_compile (1)
>>    Abbrev Offset: 0x74c8
>>    Pointer Size:  8
>>  <0><da265>: Abbrev Number: 1 (DW_TAG_compile_unit)
>>     <da266>   DW_AT_stmt_list   : 0x12cbd
>>     <da26a>   DW_AT_ranges      : 0x2d6e
>>     <da26e>   DW_AT_name        : (indirect string, offset: 0x1ec2f): /home/acme/git/linux/arch/x86/entry/entry.S
>>     <da272>   DW_AT_comp_dir    : (indirect string, offset: 0x1be93): /home/acme/git/build/v6.11-rc5+
>>     <da276>   DW_AT_producer    : (indirect string, offset: 0x1beb3): GNU AS 2.41
>>     <da27a>   DW_AT_language    : 32769 (MIPS assembler)
>>  <1><da27c>: Abbrev Number: 2 (DW_TAG_subprogram)
>>     <da27d>   DW_AT_name        : (indirect string, offset: 0x1ec5b): entry_ibpb
>>     <da281>   DW_AT_external    : 1
>>     <da281>   DW_AT_type        : <0xda29a>
>>     <da282>   DW_AT_low_pc      : 0xffffffff82062790
>>     <da28a>   DW_AT_high_pc     : 23
>>  <1><da28b>: Abbrev Number: 2 (DW_TAG_subprogram)
>>     <da28c>   DW_AT_name        : (indirect string, offset: 0x1ec66): warn_thunk_thunk
>>     <da290>   DW_AT_external    : 1
>>     <da290>   DW_AT_type        : <0xda29a>
>>     <da291>   DW_AT_low_pc      : 0xffffffff81008060
>>     <da299>   DW_AT_high_pc     : 45
>>  <1><da29a>: Abbrev Number: 3 (DW_TAG_unspecified_type)
>>
>> root@number:~# readelf -wi `pahole --running_kernel_vmlinux` | grep warn_thunk_thunk
>>     <da28c>   DW_AT_name        : (indirect string, offset: 0x1ec66): warn_thunk_thunk
>> root@number:~# pahole --running_kernel_vmlinux
>> /lib/modules/6.11.0-rc5+/build/vmlinux
>> root@number:~# perf buildid-list -k
>> 4dd7f9c4507b82a5bf90671d524e5bb308104ffa
>> root@number:~# file /lib/modules/6.11.0-rc5+/build/vmlinux
>> /lib/modules/6.11.0-rc5+/build/vmlinux: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=4dd7f9c4507b82a5bf90671d524e5bb308104ffa, with debug_info, not stripped
>> root@number:~# perf buildid-list -i /lib/modules/6.11.0-rc5+/build/vmlinux
>> 4dd7f9c4507b82a5bf90671d524e5bb308104ffa
>> root@number:~#
>>
>> - Arnaldo
>>
>>>
>>> - Arnaldo
>>>  
>>>>>>>>   Error: Failed to add events.
>>>>>>>> root@x1:~#
>>>>>>>> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
>>>>>>>> root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
>>>>>>>> root@x1:~#
>>>>>>>>
>>>>>>>> So is it just a matter of adding this output to the expected outputs?
>>>>>>>> In test_adding_blacklisted.sh?
>>>>>>>
>>>>>>> Did the patch below, now it passes, Ack?
>>>>>>>
>>>>>>> root@x1:~# export PERFTEST_KEEP_LOGS=y
>>>>>>> root@x1:~# perf test 88
>>>>>>>  88: perftool-testsuite_probe                                        : Ok
>>>>>>> root@x1:~#
>>>>>>>
>>>>>>> Or am I missing the point? :-)
>>>
>>>>>> Yeah, that would fix the test case, the question is whether we should
>>>
>>>>> I was lazy, but yeah, that thought crossed my mind, good thing you did
>>>>> the due diligence and found that patch! Looking at it now.
>>>
>>>>>> accept it as a possible output. Found some old patch that should have
>>>>>> fixed the mentioned issue: 
>>>>>> https://lore.kernel.org/all/20230628082337.1857302-3-georgmueller@gmx.net/
>>>
>>>>> I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I
>>>>> have your opinion on the patch pointed out by Veronika?
>>>  
>>>> Thanks for update my email. Yeah, it looks good to me.
>>>  
>>>> Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
>>>
>>> Unfortunately that cset is already there:
>>>
>>> commit c66e1c68c13b872505f25ab641c44b77313ee7fe
>>> Author: Georg Müller <georgmueller@gmx.net>
>>> Date:   Wed Jun 28 10:45:51 2023 +0200
>>>
>>>     perf probe: Read DWARF files from the correct CU
>>>     
>>>     After switching from dwarf_decl_file() to die_get_decl_file(), it is not
>>>     possible to add probes for certain functions:
>>>     
>>>       $ perf probe -x /usr/lib/systemd/systemd-logind match_unit_removed
>>>       A function DIE doesn't have decl_line. Maybe broken DWARF?
>>>       A function DIE doesn't have decl_line. Maybe broken DWARF?
>>>       Probe point 'match_unit_removed' not found.
>>>          Error: Failed to add events.
>>
>>  
>>>
>>> ⬢[acme@toolbox perf-tools-next]$ git tag --contains c66e1c68c13b872505f25ab641c44b77313ee7fe | grep ^v6.1
>>> v6.10
>>> v6.10-rc1
>>> v6.10-rc2
>>> v6.10-rc3
>>> v6.10-rc4
>>> v6.10-rc5
>>> v6.10-rc6
>>> v6.10-rc7
>>> v6.11-rc1
>>> v6.11-rc2
>>> v6.11-rc3
>>> v6.11-rc4
>>> v6.11-rc5
>>> v6.11-rc6
>>> ⬢[acme@toolbox perf-tools-next]$
>>>
>>> But..
>>>
>>> root@number:~# perf test -vvvvv testsuite_probe |& head
>>> capget syscall failed (No such file or directory - 2) fall back on root check
>>>  88: perftool-testsuite_probe:
>>> --- start ---
>>> test child forked, pid 511749
>>> Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
>>> Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
>>> -- [ FAIL ] -- perf_probe :: test_adding_blacklisted :: adding blacklisted function warn_thunk_thunk (output regexp parsing)
>>> -- [ PASS ] -- perf_probe :: test_adding_blacklisted :: listing blacklisted probe (should NOT be listed)
>>> ## [ FAIL ] ## perf_probe :: test_adding_blacklisted SUMMARY :: 1 failures found
>>> -- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: 
>>> root@number:~# perf -v
>>> perf version 6.11.rc6.g69e90b02e611
>>> root@number:~#
> 


^ permalink raw reply	[flat|nested] 43+ messages in thread

* Re: [PATCH v2 00/11] perftool-testsuite 2nd batch
  2024-09-19 13:14                             ` Veronika Molnarova
@ 2024-09-26 22:33                               ` Namhyung Kim
  0 siblings, 0 replies; 43+ messages in thread
From: Namhyung Kim @ 2024-09-26 22:33 UTC (permalink / raw)
  To: Veronika Molnarova
  Cc: Arnaldo Carvalho de Melo, Masami Hiramatsu, linux-perf-users,
	acme, mpetlan, irogers, atrajeev

On Thu, Sep 19, 2024 at 03:14:40PM +0200, Veronika Molnarova wrote:
> 
> 
> On 9/12/24 15:07, Arnaldo Carvalho de Melo wrote:
> > On Thu, Sep 05, 2024 at 04:15:28PM -0300, Arnaldo Carvalho de Melo wrote:
> >> On Thu, Sep 05, 2024 at 04:04:29PM -0300, Arnaldo Carvalho de Melo wrote:
> >>> So, I proposed that if we get that error message (A function DIE doesn't
> >>> have decl_line. Maybe broken DWARF?) we should skip it, but Veronika did
> >>> the right thing and asked if that wasn't really papering over some
> >>> problem and pointed to a patch that addressed a problem that lead to
> >>> that exact same message being produced, but that didn't help with our
> >>> current case, with another blacklisted function, warn_thunk_thunk.
> > 
> >>> You mentioned that this is an asm function, so no DWARF, let alone
> >>> correct DWARF, so the message is kinda ok, but we can't use that message
> >>> as it could paper over real problems like the one fixed by the patch
> >>> that Veronika pointed out.
> > 
> >>> How to figure out if a function has valid DWARF? Maybe for the moment we
> >>> should have _another_ blacklist, one for functions in the blacklist that
> >>> have no valid DWARF? A blacklist's blacklist?
> >  
> >> We have the needed info to provide a more informative warning:
> > 
> > So, perhaps we should, in the test, when we get that "broken dwarf"
> > output, look at the debuginfo file for the kernel to see if the function
> > is one in a DW_UT_compile (CU) that has 
> > 
> >      <da27a>   DW_AT_language    : 32769 (MIPS assembler)
> > 
> > And then just have a -vv output stating that that function is being
> > skipped because it is DWARF from assembler?
> > 
> > Veronika, would you be willing to try to do that?
> 
> Okay, I can try to parse the debuginfo to check the DWARF.
 
I'll wait for v3 then.

Thanks,
Namhyung


^ permalink raw reply	[flat|nested] 43+ messages in thread

end of thread, other threads:[~2024-09-26 22:33 UTC | newest]

Thread overview: 43+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-06-24 13:07 [PATCH 00/11] perftool-testsuite 2nd batch Michael Petlan
2024-06-24 13:07 ` [PATCH 01/11] perf tests shell: Skip base_* dirs in test script search Michael Petlan
2024-06-24 13:08 ` [PATCH 02/11] perf testsuite: Merge settings files for shell tests Michael Petlan
2024-06-24 13:08 ` [PATCH 03/11] perf testsuite: Fix shellcheck warnings Michael Petlan
2024-06-24 13:08 ` [PATCH 04/11] perf testsuite probe: Add test for blacklisted kprobes handling Michael Petlan
2024-06-24 13:08 ` [PATCH 05/11] perf testsuite probe: Add test for basic perf-probe options Michael Petlan
2024-06-24 13:08 ` [PATCH 06/11] perf testsuite probe: Add test for invalid options Michael Petlan
2024-06-24 13:08 ` [PATCH 07/11] perf testsuite probe: Add test for line semantics Michael Petlan
2024-06-24 13:08 ` [PATCH 08/11] perf testsuite: Add common output checking helper Michael Petlan
2024-06-24 13:08 ` [PATCH 09/11] perf testsuite report: Add test for perf-report basic functionality Michael Petlan
2024-06-24 13:08 ` [PATCH 10/11] perf testsuite report: Add test case for perf report Michael Petlan
2024-06-24 13:08 ` [PATCH 11/11] perf testsuite: Install perf-report tests Michael Petlan
2024-06-27 22:29 ` [PATCH 00/11] perftool-testsuite 2nd batch Namhyung Kim
2024-07-02 11:08   ` [PATCH v2 " vmolnaro
2024-07-02 11:08     ` [PATCH v2 01/11] perf tests shell: Skip base_* dirs in test script search vmolnaro
2024-07-02 11:08     ` [PATCH v2 02/11] perf testsuite: Merge settings files for shell tests vmolnaro
2024-07-02 11:08     ` [PATCH v2 03/11] perf testsuite: Fix shellcheck warnings vmolnaro
2024-07-02 11:08     ` [PATCH v2 04/11] perf testsuite probe: Add test for blacklisted kprobes handling vmolnaro
2024-07-02 11:08     ` [PATCH v2 05/11] perf testsuite probe: Add test for basic perf-probe options vmolnaro
2024-07-02 11:08     ` [PATCH v2 06/11] perf testsuite probe: Add test for invalid options vmolnaro
2024-07-02 11:08     ` [PATCH v2 07/11] perf testsuite probe: Add test for line semantics vmolnaro
2024-07-02 11:08     ` [PATCH v2 08/11] perf testsuite: Add common output checking helper vmolnaro
2024-07-02 11:08     ` [PATCH v2 09/11] perf testsuite report: Add test for perf-report basic functionality vmolnaro
2024-07-02 11:08     ` [PATCH v2 10/11] perf testsuite report: Add test case for perf report vmolnaro
2024-07-02 11:08     ` [PATCH v2 11/11] perf testsuite: Install perf-report tests vmolnaro
2024-08-28 14:10     ` [PATCH v2 00/11] perftool-testsuite 2nd batch Veronika Molnarova
2024-08-28 20:39       ` Arnaldo Carvalho de Melo
2024-08-29 11:29         ` [PATCH v2 01/11] perf tests shell: Skip base_* dirs in test script search vmolnaro
2024-08-29 11:29         ` [PATCH v2 11/11] perf testsuite: Install perf-report tests vmolnaro
2024-08-30 15:27         ` [PATCH v2 00/11] perftool-testsuite 2nd batch Arnaldo Carvalho de Melo
2024-09-02 14:05           ` Veronika Molnarova
2024-09-02 14:31             ` Arnaldo Carvalho de Melo
2024-09-02 14:41               ` Arnaldo Carvalho de Melo
2024-09-02 14:42                 ` Arnaldo Carvalho de Melo
2024-09-02 15:10                 ` Veronika Molnarova
2024-09-02 18:46                   ` Arnaldo Carvalho de Melo
2024-09-02 21:00                     ` Arnaldo Carvalho de Melo
2024-09-05 15:11                     ` Masami Hiramatsu
2024-09-05 19:04                       ` Arnaldo Carvalho de Melo
2024-09-05 19:15                         ` Arnaldo Carvalho de Melo
2024-09-12 13:07                           ` Arnaldo Carvalho de Melo
2024-09-19 13:14                             ` Veronika Molnarova
2024-09-26 22:33                               ` Namhyung Kim

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).