[PATCH 1/2] perf test: Add debug line to diagnose broken metrics

[PATCH 1/2] perf test: Add debug line to diagnose broken metrics

[Ian Rogers]

Printing out the metric name and architecture makes finding the source
of a failure easier.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/tests/pmu-events.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/perf/tests/pmu-events.c b/tools/perf/tests/pmu-events.c
index f13368569d8b..478b33825790 100644
--- a/tools/perf/tests/pmu-events.c
+++ b/tools/perf/tests/pmu-events.c
@@ -1115,6 +1115,7 @@ static int test__parsing_fake(struct test_suite *test __maybe_unused,
 				break;
 			if (!pe->metric_expr)
 				continue;
+			pr_debug("Found metric '%s' for '%s'\n", pe->metric_name, map->cpuid);
 			err = metric_parse_fake(pe->metric_expr);
 			if (err)
 				return err;
-- 
2.37.0.rc0.161.g10f37bed90-goog

[PATCH 2/2] perf test: Make all metrics test more tolerant

[Ian Rogers]

Metric names are truncated so don't try to match all of one.
Allow AMX metrics to skip as floating point ones do.
Metrics for optane memory can also skip rather than fail.
Add a system wide check for uncore metrics.
Restructure code to avoid extensive nesting.

Some impetus for this in:
https://lore.kernel.org/lkml/d32376b5-5538-ff00-6620-e74ad4b4abf2@huawei.com/
Suggested-by: John Garry <john.garry@huawei.com>

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/tests/shell/stat_all_metrics.sh | 47 ++++++++++++++--------
 1 file changed, 31 insertions(+), 16 deletions(-)

diff --git a/tools/perf/tests/shell/stat_all_metrics.sh b/tools/perf/tests/shell/stat_all_metrics.sh
index e7c59e5a7a98..6e79349e42be 100755
--- a/tools/perf/tests/shell/stat_all_metrics.sh
+++ b/tools/perf/tests/shell/stat_all_metrics.sh
@@ -1,26 +1,41 @@
-#!/bin/sh
+#!/bin/bash
 # perf all metrics test
 # SPDX-License-Identifier: GPL-2.0
 
-set -e
-
 err=0
 for m in $(perf list --raw-dump metrics); do
   echo "Testing $m"
   result=$(perf stat -M "$m" true 2>&1)
-  if [[ ! "$result" =~ "$m" ]] && [[ ! "$result" =~ "<not supported>" ]]; then
-    # We failed to see the metric and the events are support. Possibly the
-    # workload was too small so retry with something longer.
-    result=$(perf stat -M "$m" perf bench internals synthesize 2>&1)
-    if [[ ! "$result" =~ "$m" ]]; then
-      echo "Metric '$m' not printed in:"
-      echo "$result"
-      if [[ "$result" =~ "FP_ARITH" && "$err" != "1" ]]; then
-        echo "Skip, not fail, for FP issues"
-        err=2
-      else
-        err=1
-      fi
+  if [[ "$result" =~ "${m:0:50}" ]] || [[ "$result" =~ "<not supported>" ]]
+  then
+    continue
+  fi
+  # Failed so try system wide.
+  result=$(perf stat -M "$m" -a true 2>&1)
+  if [[ "$result" =~ "${m:0:50}" ]]
+  then
+    continue
+  fi
+  # Failed again, possibly the workload was too small so retry with something
+  # longer.
+  result=$(perf stat -M "$m" perf bench internals synthesize 2>&1)
+  if [[ "$result" =~ "${m:0:50}" ]]
+  then
+    continue
+  fi
+  echo "Metric '$m' not printed in:"
+  echo "$result"
+  if [[ "$err" != "1" ]]
+  then
+    err=2
+    if [[ "$result" =~ "FP_ARITH" || "$result" =~ "AMX" ]]
+    then
+      echo "Skip, not fail, for FP issues"
+    elif [[ "$result" =~ "PMM" ]]
+    then
+      echo "Skip, not fail, for Optane memory issues"
+    else
+      err=1
     fi
   fi
 done
-- 
2.37.0.rc0.161.g10f37bed90-goog

Re: [PATCH 1/2] perf test: Add debug line to diagnose broken metrics

[Arnaldo Carvalho de Melo]

Em Thu, Jul 07, 2022 at 08:34:48AM -0700, Ian Rogers escreveu:
> Printing out the metric name and architecture makes finding the source
> of a failure easier.

Thanks, applied both patches.

- Arnaldo
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/tests/pmu-events.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/tools/perf/tests/pmu-events.c b/tools/perf/tests/pmu-events.c
> index f13368569d8b..478b33825790 100644
> --- a/tools/perf/tests/pmu-events.c
> +++ b/tools/perf/tests/pmu-events.c
> @@ -1115,6 +1115,7 @@ static int test__parsing_fake(struct test_suite *test __maybe_unused,
>  				break;
>  			if (!pe->metric_expr)
>  				continue;
> +			pr_debug("Found metric '%s' for '%s'\n", pe->metric_name, map->cpuid);
>  			err = metric_parse_fake(pe->metric_expr);
>  			if (err)
>  				return err;
> -- 
> 2.37.0.rc0.161.g10f37bed90-goog

-- 

- Arnaldo